From e54462b78844427829cb25850fefe8e5e145ae05 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Fri, 6 Nov 2026 09:44:36 -0500 Subject: [PATCH 01/18] wai-quantum: the reader holds its nesting (Q006), and the job service reads each request once, within one budget, off its request threads A POST of about 1.3 KB with some 600 nested parentheses in `qasm`, or in an intent's reference, ran the reader's recursion out of a tokio worker's 2 MiB stack and aborted the whole service. And a job's references were read three or four times each before any count, size or rate check, so a few small requests with heavy references held every worker for minutes, uncounted. - The reader's nesting. The parser refuses past 64 levels in a program's text (qasm::parse::MAX_NESTING: statements, parentheses, operators, brackets, calls, casts and types together, each expression tree as deep as its statement leaves room for, so a chain built in a loop is bounded too), the elaborator past 256 in all through the gates and subroutines it calls; both with the new code Q006. Binary operators are read by precedence climbing and each statement, expression and elaboration form by a function of its own, so a level stacks a few small frames. - Every other reader of a client's text: instantiate's integer expressions, loops and conditions (an else-if chain now followed in a loop), the pulse reader through the parser, the error-correction texts' REPEAT blocks (at most 32 deep; every pass counts toward the unroll bound; unrun blocks checked once on one copy), the editor's index (passed over past the limit), the answer layer's statement chains (32), and the dynamic expectation walk (a list, not a recursion, in the same order). - A budget of reading: qasm::within_steps (as branch #648 introduced it, read once per program, one budget of steps), extended to count gates and a generator's loop passes, and qasm::steps_left to tell its refusal. - The service admits by cost: a key's resend; the count (every later refusal counts); what reads nothing (at most 16 intents, 100 set items, exact numbers); then every program read once within 20 000 000 steps for the whole request (422, J031, past it); then the queue. Checks are counted before their bounds read anything. Admission, the checks and the MCP tools' verdicts run on threads of their own with 64 MiB stacks, four at a time, a panic failing only its request. - Tests: every construct nested thousands deep on a 2 MiB thread, refused; inside the limit, read; a seeded property test; the service binary started in a test and sent such programs through every door, answering /v1/health after each; heavy references refused in bounded time and counted; the wasm engine, the CLI and the Python package refusing cleanly. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/wai-quantum-lang.yml | 2 +- .github/workflows/wai-quantum-pulse.yml | 5 +- .github/workflows/wai-quantum-qec.yml | 3 + .github/workflows/wai-quantum-verdict.yml | 1 + .github/workflows/wai-web-wasm.yml | 9 +- wai/changes/2026-11-06-quantum-parse-depth.md | 43 + wai/wai-quantum-cli/docs/llms-full.txt | 4 + wai/wai-quantum-cli/tests/front_door.rs | 31 + wai/wai-quantum-py/tests/test_engine.py | 17 + wai/wai-quantum-service/README.md | 14 +- wai/wai-quantum-service/src/analysis.rs | 74 +- wai/wai-quantum-service/src/discovery.rs | 2 +- wai/wai-quantum-service/src/http.rs | 283 ++++- wai/wai-quantum-service/src/jobs.rs | 29 +- wai/wai-quantum-service/src/main.rs | 1 + wai/wai-quantum-service/src/mcp.rs | 56 +- wai/wai-quantum-service/src/offload.rs | 100 ++ wai/wai-quantum-service/tests/depth.rs | 224 ++++ wai/wai-quantum/docs/qasm-diagnostics.md | 1 + wai/wai-quantum/docs/qasm-feature-matrix.md | 3 + wai/wai-quantum/docs/qec.md | 10 +- wai/wai-quantum/src/engine/answer/eval.rs | 23 +- wai/wai-quantum/src/engine/answer/tests.rs | 28 + wai/wai-quantum/src/engine/dynamic.rs | 140 ++- wai/wai-quantum/src/engine/editor/index.rs | 72 +- wai/wai-quantum/src/qasm.rs | 82 +- wai/wai-quantum/src/qasm/depth_tests.rs | 259 +++++ wai/wai-quantum/src/qasm/diag.rs | 1 + wai/wai-quantum/src/qasm/elab.rs | 263 +++-- wai/wai-quantum/src/qasm/parse.rs | 1029 ++++++++++------- wai/wai-quantum/src/qasm/pulse/tests.rs | 18 + wai/wai-quantum/src/qasm/unroll.rs | 88 +- wai/wai-quantum/src/quantum_frame.rs | 72 +- wai/wai-quantum/src/quantum_qec_text.rs | 81 +- wai/wai-web/src/quantum_engine.rs | 29 + 35 files changed, 2380 insertions(+), 717 deletions(-) create mode 100644 wai/changes/2026-11-06-quantum-parse-depth.md create mode 100644 wai/wai-quantum-service/src/offload.rs create mode 100644 wai/wai-quantum-service/tests/depth.rs create mode 100644 wai/wai-quantum/src/qasm/depth_tests.rs diff --git a/.github/workflows/wai-quantum-lang.yml b/.github/workflows/wai-quantum-lang.yml index e098dda17..1bf0d2fb9 100644 --- a/.github/workflows/wai-quantum-lang.yml +++ b/.github/workflows/wai-quantum-lang.yml @@ -79,7 +79,7 @@ jobs: set -euo pipefail cargo test --locked --release --features full --lib qasm 2>&1 | tee "$RUNNER_TEMP/qasm.log" grep -qE '^test result: ok\. [1-9][0-9]+ passed; 0 failed' "$RUNNER_TEMP/qasm.log" || { echo "::error::the reader's tests did not pass"; exit 1; } - for m in qasm::diag qasm::lex qasm::parse qasm::value qasm::lang_tests qasm::tests quantum_toolchain::tests::qasm_round_trip; do + for m in qasm::diag qasm::lex qasm::parse qasm::value qasm::lang_tests qasm::depth_tests qasm::tests quantum_toolchain::tests::qasm_round_trip; do grep -q "^test $m.* \.\.\. ok$" "$RUNNER_TEMP/qasm.log" || { echo "::error::no test of $m ran"; exit 1; } done - name: The conformance corpus diff --git a/.github/workflows/wai-quantum-pulse.yml b/.github/workflows/wai-quantum-pulse.yml index 7885a947e..42cf6bcf6 100644 --- a/.github/workflows/wai-quantum-pulse.yml +++ b/.github/workflows/wai-quantum-pulse.yml @@ -74,7 +74,7 @@ jobs: run: | set -euo pipefail cargo test --locked --release --features full --lib -- qasm::pulse:: quantum_pulse:: engine::pulse:: 2>&1 | tee "$RUNNER_TEMP/pulse.log" - grep -qE '^test result: ok\. 26 passed; 0 failed' "$RUNNER_TEMP/pulse.log" || { echo "::error::the pulse tests did not all pass (26 expected)"; exit 1; } + grep -qE '^test result: ok\. 27 passed; 0 failed' "$RUNNER_TEMP/pulse.log" || { echo "::error::the pulse tests did not all pass (27 expected)"; exit 1; } for t in qasm::pulse::tests::a_program_reads_to_its_schedule_on_frames \ qasm::pulse::tests::barriers_align_frames_and_a_gate_starts_its_frames_together \ qasm::pulse::tests::phase_and_frequency_follow_the_frame_and_the_carrier_runs_on \ @@ -84,6 +84,7 @@ jobs: qasm::pulse::tests::ports_map_by_name_and_by_the_options \ qasm::pulse::tests::what_the_pulse_reader_does_not_take_is_refused_with_its_code_and_place \ qasm::pulse::tests::the_schedule_writes_as_json_with_every_instruction \ + qasm::pulse::tests::a_calibration_s_expression_nested_past_the_limit_is_refused \ quantum_pulse::tests::a_resonant_drive_turns_the_qubit_as_rabi_says \ quantum_pulse::tests::a_quarter_turn_of_phase_turns_the_axis_to_minus_y \ quantum_pulse::tests::a_detuned_drive_turns_less_and_faster \ @@ -206,4 +207,4 @@ jobs: run: | set -euo pipefail cargo test --locked --target wasm32-wasip2 --features full --lib -- qasm::pulse:: quantum_pulse:: engine::pulse:: 2>&1 | tee "$RUNNER_TEMP/wasip2.log" - grep -qE '^test result: ok\. 26 passed; 0 failed' "$RUNNER_TEMP/wasip2.log" || { echo "::error::the pulse tests did not all pass under wasip2 (26 expected)"; exit 1; } + grep -qE '^test result: ok\. 27 passed; 0 failed' "$RUNNER_TEMP/wasip2.log" || { echo "::error::the pulse tests did not all pass under wasip2 (27 expected)"; exit 1; } diff --git a/.github/workflows/wai-quantum-qec.yml b/.github/workflows/wai-quantum-qec.yml index 817525814..dff71fb97 100644 --- a/.github/workflows/wai-quantum-qec.yml +++ b/.github/workflows/wai-quantum-qec.yml @@ -101,6 +101,8 @@ jobs: quantum_qec_text::tests::any_layout_reads_to_its_rewritten_form \ quantum_qec_text::tests::what_does_not_read_is_refused_with_its_line \ quantum_qec_text::tests::a_model_unrolls_with_its_offsets_tags_and_parts \ + quantum_qec_text::tests::blocks_nested_past_the_limit_are_refused_on_their_line \ + quantum_qec_text::tests::an_empty_block_repeated_without_end_is_refused_not_run \ quantum_window::tests::narrow_windows_correct_every_edge_set_below_half_the_distance \ quantum_window::tests::a_window_over_everything_is_the_batch_decoder \ quantum_window::tests::narrow_windows_track_the_batch_decoder \ @@ -113,6 +115,7 @@ jobs: quantum_frame::tests::pauli_targets_agree_between_sampler_and_model \ quantum_frame::tests::tags_reach_the_model_and_its_text \ quantum_frame::tests::the_text_round_trips \ + quantum_frame::tests::blocks_that_write_nothing_end_and_unrun_ones_are_checked_once \ engine::mitigate::tests::pauli_channels_round_trip_through_rates_and_fidelities \ engine::mitigate::tests::the_twirled_circuit_is_the_job_where_the_noise_is_pauli \ engine::mitigate::tests::the_exact_inverse_undoes_the_noise \ diff --git a/.github/workflows/wai-quantum-verdict.yml b/.github/workflows/wai-quantum-verdict.yml index 9c3025626..106f76b12 100644 --- a/.github/workflows/wai-quantum-verdict.yml +++ b/.github/workflows/wai-quantum-verdict.yml @@ -110,6 +110,7 @@ jobs: for t in engine::answer::tests::streaming_at_every_split_ends_as_the_whole_text_reads \ engine::answer::tests::every_problem_has_its_code_and_says_what_becomes_of_the_statement \ engine::answer::tests::an_answer_s_calls_run_on_the_engine_and_its_seal_verifies_when_run_again \ + engine::answer::tests::a_chain_of_statements_past_the_limit_is_refused_not_overflowed \ engine::telemetry::tests::the_failures_become_a_corpus_in_the_conformance_format; do grep -q "^test $t \.\.\. ok$" "$RUNNER_TEMP/answer.log" || { echo "::error::$t did not run"; exit 1; } done diff --git a/.github/workflows/wai-web-wasm.yml b/.github/workflows/wai-web-wasm.yml index 8c9fe61c5..f0c2500a0 100644 --- a/.github/workflows/wai-web-wasm.yml +++ b/.github/workflows/wai-web-wasm.yml @@ -227,9 +227,10 @@ jobs: # course check whose bytes match the native build's pinned digest, a pulse # run whose final density matrix is the native build's bits (its pinned # digest), a program converted between formats to the QIR bytes the - # conformance set pins, and the variational library's gradient to the - # engine's pinned bytes beside a program built and checked. The count is the - # four tests above and these seventeen. + # conformance set pins, the variational library's gradient to the + # engine's pinned bytes beside a program built and checked, and a program + # nested thousands deep refused at every door (no trap). The count is the + # four tests above and these eighteen. - name: The job engine in a page (wasm32 tests under Node) shell: bash working-directory: wai/wai-web @@ -237,7 +238,7 @@ jobs: set -euo pipefail CARGO_TARGET_WASM32_UNKNOWN_UNKNOWN_RUNNER=wasm-bindgen-test-runner \ cargo test --locked --target wasm32-unknown-unknown --features quantum_engine 2>&1 | tee "$RUNNER_TEMP/wai-web-engine-wasm.log" - grep -q "test result: ok. 21 passed; 0 failed" "$RUNNER_TEMP/wai-web-engine-wasm.log" + grep -q "test result: ok. 22 passed; 0 failed" "$RUNNER_TEMP/wai-web-engine-wasm.log" # Its native half: the service's form and receipt, any angle on the state # vector, what does not run saying why, plan, lint, draw and methods as JSON, # critique, equivalence and variants as JSON, the classical shortcuts, the diff --git a/wai/changes/2026-11-06-quantum-parse-depth.md b/wai/changes/2026-11-06-quantum-parse-depth.md new file mode 100644 index 000000000..6f5ed580c --- /dev/null +++ b/wai/changes/2026-11-06-quantum-parse-depth.md @@ -0,0 +1,43 @@ +### WAI Quantum: the reader holds its nesting, and the job service reads each request once, within one budget, off its request threads + +- **The reader's nesting** (`wai_quantum::qasm`). A program nested past what the reader takes is refused with + the new code `Q006` rather than read by a recursion that ran a thread's stack out: 64 levels in its text + (`qasm::parse::MAX_NESTING`: statements in blocks and bodies, parentheses, operators, brackets, calls, casts + and types together, and every expression's tree as deep as its statement leaves room for, so a sum or an + index chain a hundred thousand long is refused too), and 256 in all through the gates and subroutines it + calls (the elaborator's statements, expressions, conditions and calls). The parser reads binary operators + by precedence climbing and each statement and expression form in a function of its own, so a level of + nesting stacks a few small frames; read in a release build on a 1 MiB stack, every construct nests several + times past the limit before it would run out. A modifier chain (`inv @ inv @ …`) is a list and reads at any + length. `docs/qasm-feature-matrix.md` and `docs/qasm-diagnostics.md` state the limits. +- **Every other reader of a client's text**: the generator's writer (`qasm::instantiate`: its integer + expressions, loops and conditions, an `else if` chain now followed in a loop), the pulse reader's + calibrations (through the parser), the error-correction circuit and model texts (`REPEAT` blocks at most + `quantum_qec_text::MAX_BLOCK_DEPTH`, 32, deep, refused on their line; a block's every pass counts toward the + bound on what a text unrolls to, so one repeated without end that writes nothing ends, refused; blocks + repeated no times are checked once on one copy), the editor's index (a statement nested past the reader's + limit is passed over without recursion), and the answer layer's statements (at most + `engine::answer::eval::CHAIN_DEPTH`, 32, each naming the next). The exact expectation values of a program + that measures in the middle walk its branches with a list rather than a recursion, in the same order, so a + program that branches thousands of times is refused at `MAX_LEAVES`, not run out of stack. +- **A budget of reading** (`qasm::within_steps`, `qasm::steps_left`): every program read on a thread within it + is read once (a program read again with no inputs is answered from its first reading) and every reading + draws on one budget of steps: each statement elaborated, each gate written, each pass of a loop a generator + is written out with. Past it, a reading is refused (`Q312`), and `steps_left` is `Some(0)`. +- **The job service: admission by cost.** A submission (`POST /v1/jobs`, `POST /v1/run`, a dry run, and the + MCP run tools and tasks) is answered from its key where it is one sent again, then counted against the + client's window, so every request refused after that counts; then held to what reads nothing (at most 16 + intents, each of a set's items too; at most 100 items in a set; exact whole numbers); and only then are its + programs read, each once, within one budget of 20 000 000 steps for the whole request (`read_steps` in + `GET /v1`'s limits): a request whose references or items take more is refused, `422` and `J031`. A check + (`/v1/critique` and the others) is counted before its bounds read anything, and its bounds read within the + same budget. +- **The job service: off the request threads.** Admission, every check's bounds and answer, and the verdicts + the MCP tools state beside an answer run on threads of their own, each with a 64 MiB stack, at most four + requests at once (a request that waits ten seconds for a turn is refused, `503` and `J032`); a panic there + fails its request with `500` and the service runs on. +- **Tests**: every nesting construct at thousands of levels on a + 2 MiB thread, refused; a seeded property test over mixed nesting; the service binary started in a test and + sent programs nested thousands deep through each submission, the MCP run tool and task, the checks and an + error-correction decode, answering `/v1/health` after each; and a request with sixteen heavy references, + refused within one budget in bounded time and counted toward its client's window. diff --git a/wai/wai-quantum-cli/docs/llms-full.txt b/wai/wai-quantum-cli/docs/llms-full.txt index 4ab3f937f..e5bc4ba01 100644 --- a/wai/wai-quantum-cli/docs/llms-full.txt +++ b/wai/wai-quantum-cli/docs/llms-full.txt @@ -988,6 +988,8 @@ Each construct below is supported (read as the language defines it), lowered (re | Comments `//` and `/* */` | supported; an unclosed block comment is refused | Q003 | | Identifiers, Unicode letters included (`θ`, `π`) | supported | | | Statements and blocks `{ }`, scopes | supported; qubits, gates, subroutines and externs are declared at the top level | Q105 | +| Nesting: statements in blocks and bodies, parentheses, operators, brackets, calls, casts and types | supported to 64 levels together in the text (a statement, each enclosing statement, and each level of its expressions' trees count one; a long `else if` chain is one level a branch, and reads as a `switch` at any length); deeper is refused, never read by a recursion that could run out of stack | Q006 | +| Nesting in all, through the gates and subroutines a program calls | supported to 256 levels (each statement, expression and call the reader runs counts one), and calls 32 deep; deeper is refused | Q006, Q209 | | `pragma …`, `#pragma …`, annotations `@keyword …` | warned: kept, not applied; `@wai.max_iterations N` bounds the `while` after it | Q013 | #### Literals @@ -1072,6 +1074,7 @@ Each construct below is supported (read as the language defines it), lowered (re | `pow(k) @`, `k` a whole number | lowered exactly: angles scaled or the gate repeated; negative `k` repeats the inverse | Q203 | | `pow(r) @`, `r` fractional | lowered where an exact principal root exists (`pow(1/2) @ x` is `sx`; rotations by less than 2π; phases; controlled forms); refused otherwise (`u`, `cu`, user gates) | Q204, Q205 | | A modifier on a subroutine, or nested as deep as 32 | refused | Q211, Q209 | +| A chain of modifiers (`inv @ inv @ …`) | read at any length: a list, not a nesting | | #### Measurement and reset @@ -1192,6 +1195,7 @@ Every code: | `Q003` | refused | A string, a block comment, a bracket or a calibration block that is not closed. | | `Q004` | refused | Something else was expected here: a `;`, a name, an expression or a statement (the message says which). | | `Q005` | refused | A statement in a place the language does not allow it: `else` with no `if`, `case` outside `switch`, a declaration inside a gate. | +| `Q006` | refused | A program nested deeper than the reader takes: statements, parentheses, operators, brackets and types more than 64 levels deep in its text, or more than 256 levels in all through the gates and subroutines it calls; refused, never read by a recursion that could run out of stack. | | `Q010` | warned | An `OPENQASM` version other than 2.x or 3.x: the program is read as OpenQASM 3. | | `Q011` | warned | An `include` of a file other than the standard libraries (`stdgates.inc`, `qelib1.inc`): its definitions are not read, so a gate it defines is unknown. | | `Q012` | lowered | `include "stdgates.inc"` or `"qelib1.inc"`: the standard gates, `U`, `gphase` and OpenQASM 2's names are built in. | diff --git a/wai/wai-quantum-cli/tests/front_door.rs b/wai/wai-quantum-cli/tests/front_door.rs index 1f76757fa..12bd3d8f8 100644 --- a/wai/wai-quantum-cli/tests/front_door.rs +++ b/wai/wai-quantum-cli/tests/front_door.rs @@ -124,3 +124,34 @@ fn the_docs_and_the_stdio_server_s_resources_agree() { assert_eq!(reply["result"]["contents"][0]["text"].as_str(), Some(language.as_str())); let _ = std::fs::remove_dir_all(&home); } + +/// A program nested thousands deep, and an error-correction circuit whose +/// blocks are, is refused by each command that reads one, with the +/// reader's code, and the command ends as a refusal does: by its own exit +/// code, never by a signal. +#[test] +fn a_program_nested_past_the_reader_s_limit_is_refused_by_each_command() { + let home = scratch("deep"); + let deep = home.join("deep.qasm"); + std::fs::write(&deep, format!("OPENQASM 3.0;\nqubit q;\nrz({}0.5{}) q;\n", "(".repeat(5_000), ")".repeat(5_000))).unwrap(); + let generator = home.join("generator.qasm"); + std::fs::write(&generator, format!("OPENQASM 3.0;\ninput int n;\nconst int a = {}n{};\nqubit q;\nx q;\n", "(".repeat(5_000), ")".repeat(5_000))).unwrap(); + let circuit = home.join("deep.stim"); + std::fs::write(&circuit, format!("{}H 0\n{}", "REPEAT 1 {\n".repeat(100_000), "}\n".repeat(100_000))).unwrap(); + let (deep, generator, circuit) = (deep.to_str().unwrap(), generator.to_str().unwrap(), circuit.to_str().unwrap()); + for (args, says) in [ + (vec!["lint", deep], "Q006"), + (vec!["run", deep], "Q006"), + (vec!["plan", deep], "Q006"), + (vec!["critique", deep], "Q006"), + (vec!["equivalent", deep, deep], "Q006"), + (vec!["instance", generator, "--input", "n=1"], "Q006"), + (vec!["qec", "decode", "--circuit", circuit], "nested deeper"), + ] { + let o = run(&home, &args); + assert!(o.status.code().is_some(), "{args:?} ended by a signal: {}", text(&o)); + assert!(text(&o).contains(says), "{args:?}: {}", text(&o)); + } + let _ = std::fs::remove_dir_all(&home); +} + diff --git a/wai/wai-quantum-py/tests/test_engine.py b/wai/wai-quantum-py/tests/test_engine.py index c0faf433a..85f42a166 100644 --- a/wai/wai-quantum-py/tests/test_engine.py +++ b/wai/wai-quantum-py/tests/test_engine.py @@ -123,6 +123,23 @@ def test_a_refusal_raises_the_engines_message_with_its_line(): wq.run(BELL, noise={"depolarising": 0.1}, key=DEMO) +def test_a_program_nested_past_the_readers_limit_raises_and_the_engine_goes_on(): + # Nested thousands deep: refused with the reader's code, in this process. + deep = "OPENQASM 3.0;\nqubit q;\nrz(" + "(" * 5000 + "0.5" + ")" * 5000 + ") q;\n" + with pytest.raises(wq.EngineError) as e: + wq.run(deep, key=DEMO) + assert e.value.code == "Q006" + assert wq.lint(deep)["ok"] is False + with pytest.raises(wq.EngineError): + wq.critique(deep) + blocks = "OPENQASM 3.0;\nqubit q;\n" + "{ " * 5000 + "x q;" + " }" * 5000 + "\n" + with pytest.raises(wq.EngineError) as e: + wq.run(blocks, key=DEMO) + assert e.value.code == "Q006" + # The interpreter goes on: the next program runs. + assert wq.run(BELL, shots=10, seed=1, key=DEMO).status == "done" + + def test_values_json_cannot_carry_are_refused_not_guessed(): with pytest.raises(ValueError, match="NaN"): wq.run(BELL, options={"x": float("nan")}, key=DEMO) diff --git a/wai/wai-quantum-service/README.md b/wai/wai-quantum-service/README.md index 09b3746af..c9d1d7069 100644 --- a/wai/wai-quantum-service/README.md +++ b/wai/wai-quantum-service/README.md @@ -183,6 +183,18 @@ A job is `queued`, `running`, `done`, `failed`, `cancelled`, or `interrupted`: t Every answer carries the client's submission window in the rate-limit fields of the IETF HTTP API working group: `RateLimit-Policy: "submissions";q=30;w=600` (at most 30 submissions in 600 s) and `RateLimit: "submissions";r=12;t=34` (12 left, one more in 34 s). A refusal past the limits carries `Retry-After`. +A submission is admitted in the order of what each step costs, and every step after the count counts, refused or not: + +1. A job sent again under its `Idempotency-Key` is answered with the key's job, and counts as nothing. +2. The submission is counted against the client's window. +3. What reads nothing: every whole number exact, at most 16 intents (each of a set's items too), at most 100 items in a set. +4. Its programs are read: the job's, a set's items and a sweep's points, and every intent's reference, each once however often admission reads it, and all of them within one budget of 20,000,000 steps (statements the reader runs, gates it writes, a generator's loop passes): as many as one program may take. A request whose programs take more is refused, `422` and `J031`, with the command that runs it on your own machine. +5. The job is kept and queued, within the client's and the queue's limits. + +A job given in another format (`"source"`) is counted before it is read from that format, since that reading is the costly part; one that turns out to be sent again under its key is given back its count. + +Reading happens off the service's request threads, at most four requests at a time, each on a thread of its own with a 64 MiB stack: a request that waits ten seconds for its turn is refused (`503`, `J032`), and one whose reading fails inside the service is answered `500` while the service runs on. The reader refuses programs nested past its limits (`Q006`: 64 levels in a program's text, 256 in all through the gates and subroutines it calls), so no program reaches that stack. + ## Sending a job again A connection can fail after the service took a job and before its answer came back. To send the job again safely, give `POST /v1/jobs` or `POST /v1/run` an `Idempotency-Key` (the same working group's field): 1 to 255 visible ASCII characters, with no quote or backslash, as a structured field's string or bare. Make it unguessable, a UUID or 128 random bits, since it is kept per client and a client is an address. @@ -205,7 +217,7 @@ Every job is in the service's store before its submission is answered, and again Finished jobs are kept for `WAI_QSVC_RETAIN_SECS` (a day here), then dropped. A record names its client by a keyed hash of the address, never the address. [docs/self-host.md](docs/self-host.md#the-job-store) describes the store. -The six checks answer at once, in the service's own process. Each counts against the client's submissions, as a job does. At most two run at a time, and a check that waits ten seconds for its turn is refused (`503`). Some checks are refused (`422`) with the command that runs them on your own machine: +The six checks answer at once, in the service's own process. Each counts against the client's submissions, as a job does, before it reads anything; then its bounds read its programs within one budget, as a job's admission does, off the request threads. At most two run at a time, and a check that waits ten seconds for its turn is refused (`503`). Some checks are refused (`422`) with the command that runs them on your own machine: - an equivalence check that would apply more than 4·10⁹ gate amplitudes; - an improvement whose candidates' checks would apply more than that; diff --git a/wai/wai-quantum-service/src/analysis.rs b/wai/wai-quantum-service/src/analysis.rs index fff9cb89c..394bb0f99 100644 --- a/wai/wai-quantum-service/src/analysis.rs +++ b/wai/wai-quantum-service/src/analysis.rs @@ -33,12 +33,19 @@ //! Every refusal ends with its code: J031 for what is larger than the //! service takes, J032 when every check is running. //! +//! **In the order of what each step costs.** A check is counted against the +//! client's submissions first, so every check refused after that counts; +//! then, off the runtime on a thread of its own ([`crate::offload`]), its +//! bounds read its programs, each once, within one budget of reading +//! ([`crate::http::READ_STEPS`]), and only then is it answered. +//! //! The critique and the quick shortcuts are bounded by the program's size. //! The tableau, the cut-rank planner's search and the light cones are linear //! in the gates or close to it. use crate::http::App; use serde_json::Value; +use std::sync::{Arc, LazyLock}; use std::time::Duration; use tokio::sync::Semaphore; use wai_quantum::engine::synth::lower; @@ -64,7 +71,12 @@ pub const ESTIMATE_MAX_ANGLES: usize = 256; /// The smallest share of the synthesis budget a rotation may get here. pub const ESTIMATE_MIN_EACH: f64 = 1e-12; -static TURNS: Semaphore = Semaphore::const_new(AT_ONCE); +static TURNS: LazyLock> = LazyLock::new(|| Arc::new(Semaphore::new(AT_ONCE))); + +/// The most intents a job states (a set's items each): an intent's +/// reference is a program to read and to check against the job's, so their +/// number is held before any is read. +pub const MAX_INTENTS: usize = 16; #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub enum Check { @@ -312,19 +324,31 @@ pub fn variational_bounds(op: &str, body: &Value) -> Result<(), (u16, String)> { Ok(()) } -/// `work` for `who` within the bounds above (counted, one of [`AT_ONCE`], -/// off the request's thread): its answer, or 422 with its refusal. -pub async fn bounded(app: &App, who: &str, work: F) -> Result +/// `work` for `who` within the bounds above: counted first, then, one of +/// [`AT_ONCE`] on a thread of its own ([`crate::offload`]), held to `check` +/// (which reads within one budget, [`crate::http::READ_STEPS`]) and run: its +/// answer, or 422 with its refusal. +pub async fn bounded(app: &App, who: &str, check: C, work: F) -> Result where + C: FnOnce() -> Result<(), (u16, String)> + Send + 'static, F: FnOnce() -> Result + Send + 'static, { app.service.count(who).map_err(|crate::jobs::Refused(code, why)| (code, why))?; - let Ok(Ok(turn)) = tokio::time::timeout(WAIT, TURNS.acquire()).await else { - return Err((503, format!("{AT_ONCE} checks are running; try again shortly"))); - }; - let v = tokio::task::spawn_blocking(work).await.map_err(|e| (500, format!("the check failed: {e}")))?; - drop(turn); - v.map_err(|e| (422, e)) + let turn = turn().await?; + crate::offload::on_thread(move || { + let _turn = turn; + crate::http::within_budget(check)?; + work().map_err(|e| (422, e)) + }) + .await? +} + +/// A turn among the [`AT_ONCE`] checks, held by the thread that runs one. +async fn turn() -> Result { + match tokio::time::timeout(WAIT, Arc::clone(&TURNS).acquire_owned()).await { + Ok(Ok(t)) => Ok(t), + _ => Err((503, tag(refusal::BUSY, format!("{AT_ONCE} checks are running; try again shortly")))), + } } /// Whether the service takes `check` on `body`: the reason it does not, with @@ -392,10 +416,22 @@ pub fn convert_bounds(body: &Value) -> Result<(), (u16, String)> { Ok(()) } +/// Whether a job states at most [`MAX_INTENTS`] intents, read without +/// reading any of them: the reason it does not (400, `J020`), or nothing. +pub fn intent_count(request: &Value) -> Result<(), (u16, String)> { + match request.get("intent") { + Some(Value::Array(list)) if list.len() > MAX_INTENTS => Err((400, tag(refusal::BAD_INTENT, format!("{} intents; a job here states at most {MAX_INTENTS}", list.len())))), + _ => Ok(()), + } +} + /// Whether the service checks a job's intents beside its run: each /// reference within the equivalence check's bound, or the reason it does -/// not (422, with the command that runs it locally). +/// not (422, with the command that runs it locally). It reads the program +/// and each reference: within a budget of reading ([`crate::http::plan_job`]), +/// each is read once however often its intents name it. pub fn intent_bounds(request: &Value) -> Result<(), (u16, String)> { + intent_count(request)?; let Some(i) = request.get("intent").filter(|i| !i.is_null()) else { return Ok(()) }; let intents = wai_quantum::engine::verdict::intents(i).map_err(|e| (400, e))?; let qasm = request.get("qasm").and_then(Value::as_str).unwrap_or(""); @@ -439,15 +475,17 @@ pub fn answer(check: Check, body: &Value) -> Value { /// `check` on `body` for `who`, within the bounds above: the answer, or the /// status and reason it was refused. +/// It is counted first, then held to its bounds and answered off the +/// runtime ([`bounded`]). pub async fn analyse(app: &App, who: &str, check: Check, body: Value) -> Result { - bounds(check, &body)?; app.service.count(who).map_err(|crate::jobs::Refused(code, why)| (code, why))?; - let Ok(Ok(turn)) = tokio::time::timeout(WAIT, TURNS.acquire()).await else { - return Err((503, tag(refusal::BUSY, format!("{AT_ONCE} checks are running; try again shortly")))); - }; - let v = tokio::task::spawn_blocking(move || answer(check, &body)).await.map_err(|e| (500, format!("the check failed: {e}")))?; - drop(turn); - Ok(v) + let turn = turn().await?; + crate::offload::on_thread(move || { + let _turn = turn; + crate::http::within_budget(|| bounds(check, &body))?; + Ok(answer(check, &body)) + }) + .await? } #[cfg(test)] diff --git a/wai/wai-quantum-service/src/discovery.rs b/wai/wai-quantum-service/src/discovery.rs index 637bbf5ca..a04b8d425 100644 --- a/wai/wai-quantum-service/src/discovery.rs +++ b/wai/wai-quantum-service/src/discovery.rs @@ -161,7 +161,7 @@ pub fn openapi(server: &str) -> Value { "time": { "type": "object", "description": "{\"seconds\", \"basis\"}: an estimate from this service's own timings of the method, or the planner's nominal rate; or {\"seconds\": null, \"why\", \"at_most\"} where the work does not predict the time." }, "energy": { "type": "object", "description": "{\"joules_micro_estimate\", \"energy_provenance\": \"Estimator\", \"basis\"} from the power of past metered jobs of the method here, or {\"unestimated\": why}. Never a reading." }, "within_time_limit": { "type": ["boolean", "null"] }, - "bounds": { "type": "object", "description": "The limits that apply: job_memory_bytes, job_seconds, active_jobs_per_client, memory_per_client_bytes, submissions_per_window, window_seconds, queue, set_items, body_bytes." }, + "bounds": { "type": "object", "description": "The limits that apply: job_memory_bytes, job_seconds, active_jobs_per_client, memory_per_client_bytes, submissions_per_window, window_seconds, queue, set_items, intents (the most a job states), read_steps (the steps all of a request's programs may take to read together), body_bytes." }, "client": { "type": "object", "description": "Where the client stands: active_jobs, memory_held_bytes, submissions_left, window_resets_in_seconds." }, "items": { "type": "array", "description": "A set's items, each with its index, point, method, memory, work, time and energy." } } diff --git a/wai/wai-quantum-service/src/http.rs b/wai/wai-quantum-service/src/http.rs index b3c85fa54..708188e4e 100644 --- a/wai/wai-quantum-service/src/http.rs +++ b/wai/wai-quantum-service/src/http.rs @@ -213,6 +213,8 @@ async fn describe(State(app): State) -> Response { "queue": c.max_queue, "result_retention_seconds": c.retain.as_secs(), "set_items": sets::MAX_ITEMS, + "intents": crate::analysis::MAX_INTENTS, + "read_steps": READ_STEPS, }, "load": { "queued": queued, "running": running, "memory_held_bytes": held << 20 }, "uptime_seconds": app.service.started.elapsed().map(|d| d.as_secs()).unwrap_or(0), @@ -236,8 +238,7 @@ async fn describe(State(app): State) -> Response { } /// The bounds that apply to a job here, for a dry run and the description. -pub(crate) fn bounds(app: &App) -> Value { - let c = &app.service.config; +fn bounds(c: &crate::jobs::Config) -> Value { json!({ "job_memory_bytes": c.job_mib << 20, "job_seconds": c.job_timeout.as_secs(), @@ -247,6 +248,8 @@ pub(crate) fn bounds(app: &App) -> Value { "window_seconds": c.per_client_window.as_secs(), "queue": c.max_queue, "set_items": sets::MAX_ITEMS, + "intents": crate::analysis::MAX_INTENTS, + "read_steps": READ_STEPS, "body_bytes": MAX_BODY, }) } @@ -259,47 +262,144 @@ async fn health() -> Response { reply(200, json!({ "ok": true })) } -/// The checks one job's request passes before it is planned: its numbers, -/// its compile and its intents within the service's bounds. +/// The checks one job's request passes before it is planned: its compile and +/// its intents within the service's bounds. They read its programs, so they +/// run within the request's budget of reading ([`plan_job`]). fn job_bounds(request: &Value) -> Result<(), (u16, String)> { - if let Some(at) = crate::receipt::inexact_number(request) { - return Err((400, format!("`{at}`: a whole number here must be within ±9007199254740991 (2^53 − 1), so that every JSON reader reads it alike and the receipt binds it"))); - } if let Some(c) = request.get("compile").filter(|c| !c.is_null()) { crate::analysis::compile_bounds(request.get("qasm").and_then(Value::as_str).unwrap_or(""), c)?; } crate::analysis::intent_bounds(request) } -/// `request` planned: one job's plan, or a set's plan with its items' (each -/// held to a job's bounds), or the refusal's status and reason. -pub(crate) fn plan_job(app: &App, request: &Value) -> Result<(catalog::Plan, Work), (u16, String)> { - let cap = app.service.config.job_mib << 20; - if catalog::is_set(request) { - let items = sets::expand(request).map_err(|e| (400, e))?; - for (i, it) in items.iter().enumerate() { - job_bounds(&it.request).map_err(|(c, e)| (c, format!("item {i}: {e}")))?; - } - let plans = sets::plan(&items, cap).map_err(|e| (400, e))?; - return Ok((sets::summary(&plans), Work::Set { items, plans })); +/// The checks that read nothing, made before anything is: every whole +/// number exact, at most [`crate::analysis::MAX_INTENTS`] intents (a set's +/// items each too), at most [`sets::MAX_ITEMS`] items in a set. +pub(crate) fn cheap_bounds(request: &Value) -> Result<(), (u16, String)> { + if let Some(at) = crate::receipt::inexact_number(request) { + return Err((400, format!("`{at}`: a whole number here must be within ±9007199254740991 (2^53 − 1), so that every JSON reader reads it alike and the receipt binds it"))); } - job_bounds(request)?; - Ok((catalog::plan(request, cap).map_err(|e| (400, e))?, Work::One)) + crate::analysis::intent_count(request)?; + if let Some(list) = request.get("programs").and_then(Value::as_array) { + if list.len() > sets::MAX_ITEMS { + return Err((400, tag(refusal::BAD_SET, format!("{} programs; a set holds at most {}", list.len(), sets::MAX_ITEMS)))); + } + for (i, it) in list.iter().enumerate() { + crate::analysis::intent_count(it).map_err(|(c, e)| (c, format!("item {i}: {e}")))?; + } + } + if let Some(grid) = request.get("sweep").and_then(Value::as_object) { + let points = grid.values().try_fold(1usize, |n, v| n.checked_mul(v.as_array().map_or(1, Vec::len)).filter(|&n| n <= sets::MAX_ITEMS)); + if points.is_none() { + return Err((400, tag(refusal::BAD_SET, format!("the sweep has more points than a set holds ({})", sets::MAX_ITEMS)))); + } + } + Ok(()) } -/// Plans and admits `request` for `who`: the job's id, or the refusal's -/// status and reason. A request sent again with its key is the key's job, -/// whatever planning it now would give. -pub(crate) fn admit(app: &App, who: &str, request: Value, key: Option) -> Result { - let request = with_source(request)?; +/// The steps one request's programs may take to read, all together: each +/// statement the reader elaborates, each gate it writes and each pass of a +/// loop a sweep is written out with (`wai_quantum::qasm::within_steps`). As +/// many as one program may take on reading, so a job whose own program reads +/// within the reader's bound reads here, and a request whose references or +/// items would read for longer is refused within that bound's time. Each +/// program is read once, however often its admission reads it. +pub const READ_STEPS: u64 = 20_000_000; + +/// The refusal of a request whose programs read past [`READ_STEPS`]. +fn read_too_long() -> (u16, String) { + ( + 422, + tag( + refusal::TOO_LARGE_HERE, + format!("reading this request's programs takes more than {READ_STEPS} steps together (statements, gates and loop passes, each program read once), more than the service reads for one request; run it on your machine (`wai-quantum run program.qasm`, with `--intent reference.qasm` for an intent)"), + ), + ) +} + +/// `read` within one budget of [`READ_STEPS`], each program it reads read +/// once; a refusal for the budget is [`read_too_long`]. +pub(crate) fn within_budget(read: impl FnOnce() -> Result) -> Result { + let (r, spent) = wai_quantum::qasm::within_steps(READ_STEPS, || { + let r = read(); + (r, wai_quantum::qasm::steps_left() == Some(0)) + }); + match r { + Err(_) if spent => Err(read_too_long()), + r => r, + } +} + +/// `request` planned: one job's plan, or a set's plan with its items' (each +/// held to a job's bounds), or the refusal's status and reason. The checks +/// that read nothing come first; then every program the request gives is +/// read, once, within one budget ([`READ_STEPS`]). +pub(crate) fn plan_job(service: &Service, request: &Value) -> Result<(catalog::Plan, Work), (u16, String)> { + cheap_bounds(request)?; + let cap = service.config.job_mib << 20; + within_budget(|| { + if catalog::is_set(request) { + let items = sets::expand(request).map_err(|e| (400, e))?; + for (i, it) in items.iter().enumerate() { + job_bounds(&it.request).map_err(|(c, e)| (c, format!("item {i}: {e}")))?; + } + let plans = sets::plan(&items, cap).map_err(|e| (400, e))?; + return Ok((sets::summary(&plans), Work::Set { items, plans })); + } + job_bounds(request)?; + Ok((catalog::plan(request, cap).map_err(|e| (400, e))?, Work::One)) + }) +} + +/// Plans and admits `request` for `who`, off the runtime +/// ([`crate::offload`]): the job's id, or the refusal's status and reason. +pub(crate) async fn admit(app: &App, who: &str, request: Value, key: Option) -> Result { + let service = Arc::clone(&app.service); + let who = who.to_string(); + crate::offload::run(move || admit_now(&service, &who, request, key.as_deref().map(Key))).await? +} + +/// [`admit`], on this thread, in the order of what each step costs: +/// +/// 1. a request sent again with its key is the key's job, whatever planning +/// it now would give, and counts as nothing; +/// 2. the submission is counted against the client's window, so every +/// request refused after this counts; +/// 3. the checks that read nothing ([`cheap_bounds`]); +/// 4. its programs read, each once, within one budget ([`plan_job`]); +/// 5. the job kept and queued, within the client's and the queue's limits. +/// +/// A job given in another format (`"source"`) is counted before it is read +/// from that format, since that reading is the costly part; one that turns +/// out to be sent again under its key is then given back its count. +pub(crate) fn admit_now(service: &Service, who: &str, request: Value, key: Option) -> Result { let refused = |Refused(code, why)| (code, why); - if let Some(k) = key - && let Some(id) = app.service.named(who, k, &request).map_err(refused)? + let sourced = request.get("source").is_some_and(|s| !s.is_null()); + if !sourced + && let Some(k) = key + && let Some(id) = service.named(who, k, &request).map_err(refused)? { return Ok(id); } - let (plan, work) = plan_job(app, &request)?; - app.service.submit_work(who, request, plan, work, key).map_err(refused) + service.count(who).map_err(refused)?; + // One budget for the whole request: its source's reading and its plan's. + within_budget(|| { + let request = if sourced { with_source(request)? } else { request }; + if sourced + && let Some(k) = key + && let Some(id) = service.named(who, k, &request).map_err(refused)? + { + service.refund(who); + return Ok(id); + } + let (plan, work) = plan_job(service, &request)?; + let (id, new) = service.submit_counted(who, request, plan, work, key).map_err(refused)?; + if !new { + // Sent at once with another under its key, which made the job. + service.refund(who); + } + Ok(id) + }) } /// The `Idempotency-Key` a request gives, if it gives one: a structured @@ -330,20 +430,32 @@ pub(crate) fn is_dry_run(request: &Value) -> bool { /// The plan of `request` for `who`, running nothing: its method, memory, /// estimated time and energy (each with its basis), the bounds that apply, /// where the client stands, and whether the service would take it now, or -/// the refusal it would give. It counts as one submission, as a check does. -pub(crate) fn dry_run(app: &App, who: &str, request: &Value) -> Result { - app.service.count(who).map_err(|Refused(code, why)| (code, why))?; +/// the refusal it would give. It counts as one submission, as a check does, +/// before anything is read; it is read as a job is ([`admit_now`]), off the +/// runtime. +pub(crate) async fn dry_run(app: &App, who: &str, request: &Value) -> Result { + let service = Arc::clone(&app.service); + let (who, request) = (who.to_string(), request.clone()); + crate::offload::run(move || { + service.count(&who).map_err(|Refused(code, why)| (code, why))?; + within_budget(|| dry_run_now(&service, &who, &request)) + }) + .await? +} + +/// [`dry_run`], here, its submission counted. +fn dry_run_now(service: &Service, who: &str, request: &Value) -> Result { let expanded = with_source(request.clone())?; let request = &expanded; - let window = app.service.window(who); - let (active, held) = app.service.holding(who); + let window = service.window(who); + let (active, held) = service.holding(who); let mut out = json!({ "dry_run": true, "ran": false, - "bounds": bounds(app), + "bounds": bounds(&service.config), "client": { "active_jobs": active, "memory_held_bytes": held << 20, "submissions_left": window.remaining, "window_resets_in_seconds": window.reset_secs }, }); - let (plan, work) = match plan_job(app, request) { + let (plan, work) = match plan_job(service, request) { Ok(p) => p, Err((status, why)) => { out["accepted"] = json!(false); @@ -352,7 +464,7 @@ pub(crate) fn dry_run(app: &App, who: &str, request: &Value) -> Result { let (t, e) = estimate(&plan); @@ -364,7 +476,7 @@ pub(crate) fn dry_run(app: &App, who: &str, request: &Value) -> Result = each.iter().map(|(_, e)| e["joules_micro_estimate"].as_u64()).sum(); let time = match secs { Some(s) => json!({ "seconds": s, "basis": "the sum of the items' estimates, each with its own basis", "note": "an estimate: the set is bounded by one job's time limit, not by this" }), - None => json!({ "seconds": null, "why": "an item's time is not estimated (each item's own says why)", "at_most": app.service.config.job_timeout.as_secs() }), + None => json!({ "seconds": null, "why": "an item's time is not estimated (each item's own says why)", "at_most": service.config.job_timeout.as_secs() }), }; let energy = match uj { Some(u) => json!({ "joules_micro_estimate": u, "energy_provenance": "Estimator", "basis": "the sum of the items' estimates", "note": "an estimate from past readings, never a reading of this job" }), @@ -386,14 +498,14 @@ pub(crate) fn dry_run(app: &App, who: &str, request: &Value) -> Result json!(s <= app.service.config.job_timeout.as_secs_f64()), + Some(s) => json!(s <= service.config.job_timeout.as_secs_f64()), None => Value::Null, }; out["time"] = time; @@ -422,7 +534,7 @@ async fn submit(State(app): State, ConnectInfo(peer): ConnectInfo reply(200, plan), Err((code, why)) => error(code, &why), }; @@ -431,7 +543,7 @@ async fn submit(State(app): State, ConnectInfo(peer): ConnectInfo k, Err((code, why)) => return error(code, &why), }; - match admit(&app, &who, request, key.as_deref().map(Key)) { + match admit(&app, &who, request, key).await { Ok(id) => reply(202, app.service.get(&id).unwrap_or(json!({ "id": id }))), Err((code, why)) => error(code, &why), } @@ -444,7 +556,7 @@ async fn run(State(app): State, ConnectInfo(peer): ConnectInfo reply(200, plan), Err((code, why)) => error(code, &why), }; @@ -453,7 +565,7 @@ async fn run(State(app): State, ConnectInfo(peer): ConnectInfo k, Err((code, why)) => return error(code, &why), }; - let id = match admit(&app, &who, request, key.as_deref().map(Key)) { + let id = match admit(&app, &who, request, key).await { Ok(id) => id, Err((code, why)) => return error(code, &why), }; @@ -566,10 +678,8 @@ async fn verify(State(app): State, ConnectInfo(peer): ConnectInfo reply(200, checked), Err((code, why)) => error(code, &why), }; @@ -623,9 +733,9 @@ pub(crate) async fn estimate(app: &App, who: &str, request: Value) -> Result Router { #[cfg(test)] mod tests { - use super::{identity, idempotency_key}; + use super::{admit_now, identity, idempotency_key}; + use crate::jobs::{Config, Key, Service}; use axum::http::{HeaderMap, HeaderValue}; + use serde_json::json; use std::net::IpAddr; + use std::time::Duration; + + fn service(burst: usize) -> std::sync::Arc { + Service::new(Config { + budget_mib: 1024, + job_mib: 512, + workers: 1, + min_free_pct: 0, + job_timeout: Duration::from_secs(5), + retain: Duration::from_secs(60), + max_queue: 16, + per_client_active: 16, + per_client_mib: 1 << 20, + sandbox: true, + per_client_burst: burst, + per_client_window: Duration::from_secs(600), + }) + } + + /// Admission in the order of what each step costs: a refusal after the + /// count counts, whether the program is too deep, an intent too many, + /// or its references too long to read; a job sent again under its key + /// counts as nothing; and each reference is read once, however often + /// the request names it and however often its admission reads it. + #[test] + fn a_refused_job_counts_and_each_reference_is_read_once_within_one_budget() { + let s = service(100); + let bell = "OPENQASM 3.0;\nqubit[1] q;\nbit[1] c;\nh q[0];\nc = measure q;\n"; + let deep = format!("OPENQASM 3.0;\nqubit[1] q;\nrz({}0.5{}) q[0];\n", "(".repeat(5_000), ")".repeat(5_000)); + let left = |s: &Service| s.window("a").remaining; + let (code, why) = admit_now(&s, "a", json!({ "qasm": deep }), None).unwrap_err(); + assert!(code == 400 && why.ends_with("[Q006]"), "{code} {why}"); + assert_eq!(left(&s), 99); + let many: Vec<_> = (0..17).map(|_| json!({ "equivalent": bell })).collect(); + let (code, why) = admit_now(&s, "a", json!({ "qasm": bell, "intent": many }), None).unwrap_err(); + assert!(code == 400 && why.ends_with("[J020]"), "{code} {why}"); + assert_eq!(left(&s), 98); + // Sixteen intents naming one reference that loops four million + // times: read once, it is within one budget; read for each intent + // and each step of admission, it was far past it. + let heavy = "OPENQASM 3.0;\nqubit[1] q;\nbit[1] c;\nint a = 0;\nfor int i in [0:1999999] { a += 1; }\nh q[0];\nc = measure q;\n"; + let same: Vec<_> = (0..16).map(|_| json!({ "equivalent": heavy })).collect(); + let job = json!({ "qasm": bell, "shots": 10, "intent": same }); + let id = admit_now(&s, "a", job.clone(), Some(Key("k"))).unwrap(); + assert_eq!(left(&s), 97); + // Sent again under its key: the same job, counted as nothing. + assert_eq!(admit_now(&s, "a", job, Some(Key("k"))).unwrap(), id); + assert_eq!(left(&s), 97); + // Sixteen different ones are not: refused, as too large here, and counted. + let each: Vec<_> = (0..16).map(|k| json!({ "equivalent": format!("{heavy}// {k}\n") })).collect(); + let (code, why) = admit_now(&s, "a", json!({ "qasm": bell, "shots": 10, "intent": each }), None).unwrap_err(); + assert!(code == 422 && why.ends_with("[J031]") && why.contains("steps"), "{code} {why}"); + assert_eq!(left(&s), 96); + } #[test] fn an_idempotency_key_reads_as_a_structured_string_or_bare() { diff --git a/wai/wai-quantum-service/src/jobs.rs b/wai/wai-quantum-service/src/jobs.rs index 7b93f48ff..784549965 100644 --- a/wai/wai-quantum-service/src/jobs.rs +++ b/wai/wai-quantum-service/src/jobs.rs @@ -639,6 +639,15 @@ impl Service { Ok(()) } + /// Takes back `client`'s latest submission: one counted before it was + /// known to be a job sent again under its key, which counts as nothing. + pub fn refund(&self, client: &str) { + let name = self.name(client); + if let Some(c) = self.clients.lock().unwrap().get_mut(&name) { + c.submissions.pop_back(); + } + } + /// Where `client` stands in its submission window. pub fn window(&self, client: &str) -> Window { let now = Instant::now(); @@ -714,7 +723,19 @@ impl Service { /// [`Service::submit`] for one program or a set's items, sent with a key /// or without one: a request sent again with its key is answered with /// the key's job, before any limit, and counts as nothing. + #[cfg(test)] pub fn submit_work(&self, client: &str, request: Value, plan: Plan, work: Work, key: Option) -> Result { + self.keep_work(client, request, plan, work, key, true).map(|(id, _)| id) + } + + /// [`Service::submit_work`] for a submission already counted against + /// its client's window (`http::admit` counts before it reads anything): + /// the job's id, and whether it is a new job (not the key's). + pub fn submit_counted(&self, client: &str, request: Value, plan: Plan, work: Work, key: Option) -> Result<(String, bool), Refused> { + self.keep_work(client, request, plan, work, key, false) + } + + fn keep_work(&self, client: &str, request: Value, plan: Plan, work: Work, key: Option, count: bool) -> Result<(String, bool), Refused> { let name = self.name(client); let canonical = canon::canonical(&request); let request_digest = store::digest(&canonical); @@ -724,7 +745,7 @@ impl Service { if let (Some(keys), Some(k)) = (&keys, &key) && let Some(id) = self.named_by(keys, k, &request_digest)? { - return Ok(id); + return Ok((id, false)); } // A full queue is checked again below, as the job is queued. if let Some(r) = self.would_refuse(client, &plan) @@ -732,7 +753,9 @@ impl Service { { return Err(r); } - self.count(client)?; + if count { + self.count(client)?; + } let unkept = |e: String| Refused(503, tag(refusal::BUSY, format!("the job store could not keep the job ({e}); try again later"))); let programs: Vec<&str> = match &work { Work::One => request.get("qasm").and_then(Value::as_str).into_iter().collect(), @@ -776,7 +799,7 @@ impl Service { drop(keys); self.changed(&id); self.wake.notify_one(); - Ok(id) + Ok((id, true)) } /// The job's report, if it is known. diff --git a/wai/wai-quantum-service/src/main.rs b/wai/wai-quantum-service/src/main.rs index ce3b2ff7b..16fd307b8 100644 --- a/wai/wai-quantum-service/src/main.rs +++ b/wai/wai-quantum-service/src/main.rs @@ -30,6 +30,7 @@ mod energy; mod http; mod jobs; mod mcp; +mod offload; use wai_quantum::engine::{catalog, run, sets}; mod receipt; mod store; diff --git a/wai/wai-quantum-service/src/mcp.rs b/wai/wai-quantum-service/src/mcp.rs index 3aa1cbf77..396fcaeee 100644 --- a/wai/wai-quantum-service/src/mcp.rs +++ b/wai/wai-quantum-service/src/mcp.rs @@ -520,12 +520,12 @@ fn no_task(id: &Value, app: &App) -> Value { } /// A run's job admitted as a task: its id, or the refusal as data. -fn admit_task(app: &App, who: &str, name: &str, args: Value) -> Result { +async fn admit_task(app: &App, who: &str, name: &str, args: Value) -> Result { let (args, key) = take_key(args)?; if let Some(r) = run_refusal(name, &args) { return Err(r["structuredContent"]["error"].as_str().unwrap_or("refused").to_string()); } - admit_run(app, who, args, key.as_deref()).map_err(|r| r["structuredContent"]["error"].as_str().unwrap_or("refused").to_string()) + admit_run(app, who, args, key).await.map_err(|r| r["structuredContent"]["error"].as_str().unwrap_or("refused").to_string()) } /// What `tasks/result` answers for a task that has ended: the answer @@ -551,29 +551,38 @@ fn run_refusal(name: &str, args: &Value) -> Option { } /// A run's job admitted as the run tools admit it, its verdict asked: its -/// id, or the tool's refusal. -fn admit_run(app: &App, who: &str, args: Value, key: Option<&str>) -> Result { +/// id, or the tool's refusal. It is admitted as `POST /v1/jobs` admits one +/// (`http::admit`), off the runtime: counted before anything is read. +async fn admit_run(app: &App, who: &str, mut args: Value, key: Option) -> Result { let refused = |why: String| tool_result(format!("refused: {why}"), json!({ "error": why }), true); // Every run an agent asks for states its verdict. - let mut args = crate::http::with_source(args).map_err(|(_, why)| refused(why))?; if !verdict::asked(&args) { args["verdict"] = json!(true); } - crate::http::admit(app, who, args, key.map(crate::jobs::Key)).map_err(|(_, why)| refused(why)) + crate::http::admit(app, who, args, key).await.map_err(|(_, why)| refused(why)) } /// A tool's answer as `tools/call` gives it: the call's, the verdict of /// what it read beside it, and what a view draws, for a host that draws it. +/// The verdicts and the view read the programs again, so they are made off +/// the runtime ([`crate::offload`]), within one budget of reading; where no +/// turn comes, the answer goes without them. async fn tool_answer(app: &App, who: &str, name: &str, args: Value, base: &str, views: bool) -> Value { - let mut answer = call(app, who, name, args.clone(), base).await; + let answer = call(app, who, name, args.clone(), base).await; + let (name, plain) = (name.to_string(), answer.clone()); + crate::offload::run(move || crate::http::within_budget(|| Ok(finish(&name, &args, answer, views)))).await.and_then(|r| r).unwrap_or(plain) +} + +/// [`tool_answer`]'s verdicts and view added to the call's `answer`. +fn finish(name: &str, args: &Value, mut answer: Value, views: bool) -> Value { // A set's answer, and a dry run's, state the verdict of what they read. let sc = &answer["structuredContent"]; let extra = match name { - "run_quantum_circuit" | "run_on_target" | "run_quantum_set" if crate::http::is_dry_run(&args) && sc.get("dry_run").is_some() => dry_run_verdict(sc, &args), + "run_quantum_circuit" | "run_on_target" | "run_quantum_set" if crate::http::is_dry_run(args) && sc.get("dry_run").is_some() => dry_run_verdict(sc, args), "run_quantum_set" if sc.is_object() && sc.get("error").is_none() => set_verdict(sc), - "run_quantum_set" => sc["error"].as_str().map(|e| verdict::of_refusal(&args, e)), + "run_quantum_set" => sc["error"].as_str().map(|e| verdict::of_refusal(args, e)), "get_quantum_job" if sc["method"] == "set" => set_verdict(sc), - "dry_run_quantum_job" => dry_run_verdict(sc, &args), + "dry_run_quantum_job" => dry_run_verdict(sc, args), _ => None, }; if let Some(v) = extra.filter(|v| !v.is_null()) { @@ -582,7 +591,7 @@ async fn tool_answer(app: &App, who: &str, name: &str, args: Value, base: &str, // What the view draws, to a host that draws it. if views && answer["structuredContent"].is_object() - && let Some(d) = views::data(name, &args, &answer["structuredContent"]) + && let Some(d) = views::data(name, args, &answer["structuredContent"]) { answer["_meta"] = Value::Object([(views::META.to_string(), d)].into_iter().collect()); } @@ -594,12 +603,12 @@ async fn tool_answer(app: &App, who: &str, name: &str, args: Value, base: &str, Some(e) if answer["isError"] == true => { let mut r = wai_quantum::engine::refusal::refusal_json(e); r["error"] = json!(e); - r["verdict"] = verdict::of_refusal(&args, e); + r["verdict"] = verdict::of_refusal(args, e); answer["structuredContent"] = r; None } _ if name == "get_quantum_job" && sc["method"] == "set" => None, - _ => verdict::for_tool(name, &args, sc), + _ => verdict::for_tool(name, args, sc), }; if let Some(v) = v { answer["structuredContent"][verdict::verdict_key(name)] = v; @@ -681,7 +690,7 @@ pub async fn handle(app: &App, who: &str, msg: &Value, base: &str, views: bool) if crate::http::is_dry_run(&args) { return Some(fail(&id, -32602, "a dry run runs nothing and answers at once: call it without a task")); } - match admit_task(app, who, name, args.clone()) { + match admit_task(app, who, name, args.clone()).await { Ok(job) => ok(&id, json!({ "task": task_of(&app.service.get(&job).unwrap_or(json!({ "id": job })), app.service.config.retain.as_millis() as u64) })), Err(why) => { let mut data = wai_quantum::engine::refusal::refusal_json(&why); @@ -771,10 +780,10 @@ async fn call(app: &App, who: &str, name: &str, args: Value, base: &str) -> Valu Err(why) => tool_result(format!("refused: {why}"), json!({ "error": why }), true), } } - "dry_run_quantum_job" => dry_run_answer(app, who, &args), - "run_quantum_circuit" | "run_on_target" | "run_quantum_set" if crate::http::is_dry_run(&args) => dry_run_answer(app, who, &args), + "dry_run_quantum_job" => dry_run_answer(app, who, &args).await, + "run_quantum_circuit" | "run_on_target" | "run_quantum_set" if crate::http::is_dry_run(&args) => dry_run_answer(app, who, &args).await, "run_quantum_circuit" | "run_on_target" | "run_quantum_set" => { - let id = match admit_run(app, who, args, key.as_deref()) { + let id = match admit_run(app, who, args, key).await { Ok(id) => id, Err(refused) => return refused, }; @@ -803,11 +812,8 @@ async fn call(app: &App, who: &str, name: &str, args: Value, base: &str) -> Valu Err((_, why)) => tool_result(format!("refused: {why}"), json!({ "error": why }), true), }, "verify_quantum_receipt" if args["result"]["format"] == wai_quantum::engine::estimate::FORMAT => { - if let Err((_, why)) = crate::analysis::estimate_bounds(&args["request"]) { - return tool_result(format!("refused: {why}"), json!({ "error": why }), true); - } - let body = args.clone(); - match crate::analysis::bounded(app, who, move || wai_quantum::engine::estimate::verify(&body)).await { + let (body, request) = (args.clone(), args["request"].clone()); + match crate::analysis::bounded(app, who, move || crate::analysis::estimate_bounds(&request), move || wai_quantum::engine::estimate::verify(&body)).await { Ok(v) => tool_result(if v["valid"] == true { "valid: the seal, every estimate's receipt, and the request re-run here all hold".into() } else { format!("not valid: signed {}, receipts {}, reproduced {}", v["signed"], v["receipts"], v["reproduced"]) }, v, false), Err((_, why)) => tool_result(format!("refused: {why}"), json!({ "error": why }), true), } @@ -977,8 +983,8 @@ async fn call(app: &App, who: &str, name: &str, args: Value, base: &str) -> Valu } /// A dry run's answer, as a tool gives it. -fn dry_run_answer(app: &App, who: &str, args: &Value) -> Value { - match crate::http::dry_run(app, who, args) { +async fn dry_run_answer(app: &App, who: &str, args: &Value) -> Value { + match crate::http::dry_run(app, who, args).await { Ok(v) => { let secs = v["time"]["seconds"].as_f64().map(|t| format!("about {t:.3} s ({})", v["time"]["basis"].as_str().unwrap_or(""))).unwrap_or_else(|| format!("time not estimated ({})", v["time"]["why"].as_str().unwrap_or(""))); let energy = v["energy"]["joules_micro_estimate"].as_u64().map(|u| format!("about {:.3} J, an estimate", u as f64 / 1e6)).unwrap_or_else(|| format!("energy not estimated ({})", v["energy"]["unestimated"].as_str().unwrap_or(""))); @@ -1457,7 +1463,7 @@ mod tests { let bell = "OPENQASM 3.0; qubit[2] q; bit[2] c; h q[0]; cx q[0], q[1]; c = measure q;"; // The job a first call made, as the call admits it (with its verdict // asked), cancelled so that no call waits for it here. - let id = crate::http::admit(&app, "test", json!({ "qasm": bell, "shots": 20, "verdict": true }), Some(crate::jobs::Key("agent-1"))).unwrap(); + let id = crate::http::admit_now(&app.service, "test", json!({ "qasm": bell, "shots": 20, "verdict": true }), Some(crate::jobs::Key("agent-1"))).unwrap(); assert_eq!(app.service.cancel(&id), Some(true)); let again = run(json!({ "qasm": bell, "shots": 20, "idempotency_key": "agent-1" })); assert_eq!(again["isError"], false, "{again}"); diff --git a/wai/wai-quantum-service/src/offload.rs b/wai/wai-quantum-service/src/offload.rs new file mode 100644 index 000000000..f07ec1a0c --- /dev/null +++ b/wai/wai-quantum-service/src/offload.rs @@ -0,0 +1,100 @@ +//! Work that reads a client's programs, off the async runtime. +//! +//! Reading a program (the reader's elaboration of its loops, a set's +//! sweep written out, a reference program's equivalence bound, a plan) is +//! bounded work, but it is work: done on the runtime's few threads it would +//! hold them, and every other request, while it ran. So every request's +//! admission (a job, a set, a dry run), every check's bounds and answer, and +//! the verdicts the MCP tools state beside an answer, run here instead: +//! +//! - **On a thread of its own**, with a stack of [`STACK`]: far more than the +//! reader's own nesting limits need (`wai_quantum::qasm::parse::MAX_NESTING`, +//! and the elaborator's in all), so a program nested to them reads, and +//! anything nested past them is refused by the reader long before this +//! stack could be reached. +//! - **One of [`AT_ONCE`]**: a request waits up to [`WAIT`] for a turn, and is +//! then refused (503, `J032`). A turn is held by the thread, so a client +//! that hangs up does not free one while its reading still runs. +//! - **A panic fails the request, not the service**: it is caught on the +//! thread, and the request is answered 500 with what it said. + +use std::panic::{AssertUnwindSafe, catch_unwind}; +use std::sync::{Arc, LazyLock}; +use std::time::Duration; +use tokio::sync::Semaphore; +use wai_quantum::engine::refusal::{self, tag}; + +/// The stack each thread that reads a request runs on. +pub const STACK: usize = 64 << 20; +/// Requests read at once; the rest wait their turn. +pub const AT_ONCE: usize = 4; +/// How long a request waits for a turn before it is refused. +pub const WAIT: Duration = Duration::from_secs(10); + +static TURNS: LazyLock> = LazyLock::new(|| Arc::new(Semaphore::new(AT_ONCE))); + +/// `work`, one of [`AT_ONCE`], on a thread of its own with a stack of +/// [`STACK`]: its value, or the refusal (503 when no turn comes within +/// [`WAIT`], 500 when it panicked). +pub async fn run(work: impl FnOnce() -> T + Send + 'static) -> Result { + let Ok(Ok(turn)) = tokio::time::timeout(WAIT, Arc::clone(&TURNS).acquire_owned()).await else { + return Err((503, tag(refusal::BUSY, format!("{AT_ONCE} requests are being read; try again shortly")))); + }; + on_thread(move || { + let _turn = turn; + work() + }) + .await +} + +/// `work` on a thread of its own with a stack of [`STACK`], without a turn +/// (the caller holds one of its own): its value, or the refusal. +pub async fn on_thread(work: impl FnOnce() -> T + Send + 'static) -> Result { + let (tx, rx) = tokio::sync::oneshot::channel(); + std::thread::Builder::new() + .name("wai-qsvc-read".into()) + .stack_size(STACK) + .spawn(move || { + let _ = tx.send(catch_unwind(AssertUnwindSafe(work))); + }) + .map_err(|e| (503, tag(refusal::BUSY, format!("the service could not start a thread to read the request ({e}); try again shortly"))))?; + match rx.await { + Ok(Ok(v)) => Ok(v), + Ok(Err(p)) => { + let what = p.downcast_ref::<&str>().map(|s| s.to_string()).or_else(|| p.downcast_ref::().cloned()).unwrap_or_else(|| "no message".into()); + Err((500, format!("reading this request failed inside the service ({what}); nothing was admitted, and the service runs on"))) + } + Err(_) => Err((500, "reading this request ended without an answer; nothing was admitted".into())), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// A panic in the work is the request's 500, and the next request is + /// read as usual. + #[tokio::test] + async fn a_panic_fails_its_request_and_the_next_one_runs() { + let (code, why) = run(|| -> u32 { panic!("a test's panic") }).await.unwrap_err(); + assert_eq!(code, 500); + assert!(why.contains("a test's panic"), "{why}"); + assert_eq!(run(|| 7).await.unwrap(), 7); + } + + /// The work's thread has the stack this module states: a recursion that + /// would overflow a runtime thread's 2 MiB returns here. + #[tokio::test] + async fn the_work_runs_on_a_stack_of_its_own_size() { + // At least 1 KiB a frame, kept: some 8 MiB of stack in all. + fn deep(n: u64) -> u64 { + let pad = std::hint::black_box([n; 128]); + if n == 0 { + return 1; + } + deep(n - 1).wrapping_mul(3).wrapping_add(pad[(n % 128) as usize]) + } + let want = (1..=8000u64).fold(1u64, |v, k| v.wrapping_mul(3).wrapping_add(k)); + assert_eq!(run(|| deep(8000)).await.unwrap(), want); + } +} diff --git a/wai/wai-quantum-service/tests/depth.rs b/wai/wai-quantum-service/tests/depth.rs new file mode 100644 index 000000000..ace012a7f --- /dev/null +++ b/wai/wai-quantum-service/tests/depth.rs @@ -0,0 +1,224 @@ +//! The service itself, started here, sent programs nested thousands deep in +//! every place a client can put one: the job's own program and an intent's +//! reference (`POST /v1/jobs`, `POST /v1/run`, a dry run), the MCP run tool, +//! the checks, and an error-correction circuit. A stack overflow in any of +//! them would take the whole process down, on a POST of a few kilobytes. Each +//! is refused with the reader's code, and the service still answers +//! `/v1/health` after each. +//! +//! And a request whose references would take long to read is refused +//! within one budget, in bounded time, and counts toward its client's +//! window as every refused request does. + +use serde_json::{Value, json}; +use std::io::{Read, Write}; +use std::net::{TcpListener, TcpStream}; +use std::path::Path; +use std::process::{Child, Command, Stdio}; +use std::time::{Duration, Instant}; + +fn free_port() -> u16 { + TcpListener::bind("127.0.0.1:0").unwrap().local_addr().unwrap().port() +} + +/// One HTTP/1.1 exchange: the status, the headers and the body. +fn http(port: u16, method: &str, path: &str, body: Option<&Value>) -> (u16, String, String) { + let mut s = TcpStream::connect(("127.0.0.1", port)).unwrap(); + s.set_read_timeout(Some(Duration::from_secs(120))).unwrap(); + let text = body.map(Value::to_string).unwrap_or_default(); + write!(s, "{method} {path} HTTP/1.1\r\nHost: 127.0.0.1\r\nContent-Type: application/json\r\nAccept: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{text}", text.len()).unwrap(); + let mut out = String::new(); + let _ = s.read_to_string(&mut out); + let status = out.split_whitespace().nth(1).and_then(|c| c.parse().ok()).unwrap_or(0); + let (head, body) = out.split_once("\r\n\r\n").map(|(h, b)| (h.to_string(), b.to_string())).unwrap_or_default(); + (status, head, body) +} + +fn json_of(port: u16, method: &str, path: &str, body: Option<&Value>) -> (u16, Value) { + let (code, _, text) = http(port, method, path, body); + (code, serde_json::from_str(&text).unwrap_or_else(|e| panic!("{method} {path}: {code}: {e}: {text}"))) +} + +/// The service, its store at `dir`, `burst` submissions a window, once it +/// answers. +fn start(dir: &Path, port: u16, burst: u32) -> Child { + let child = Command::new(env!("CARGO_BIN_EXE_wai-quantum-service")) + .arg("serve") + .env("WAI_QSVC_ADDR", format!("127.0.0.1:{port}")) + .env("WAI_QSVC_STORE", dir.join("store")) + .env("WAI_QSVC_KEY", dir.join("signing.key")) + .env("WAI_QSVC_JOB_GB", "2") + .env("WAI_QSVC_BUDGET_GB", "4") + .env("WAI_QSVC_WORKERS", "1") + .env("WAI_QSVC_ENERGY", "0") + .env("WAI_QSVC_CLIENT_BURST", burst.to_string()) + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .spawn() + .expect("the service starts"); + let t = Instant::now(); + while TcpStream::connect(("127.0.0.1", port)).is_err() || http(port, "GET", "/v1/health", None).0 != 200 { + assert!(t.elapsed() < Duration::from_secs(60), "the service did not answer"); + std::thread::sleep(Duration::from_millis(50)); + } + child +} + +/// The service is up and answering, and its process has not ended. +fn alive(port: u16, svc: &mut Child, after: &str) { + assert!(svc.try_wait().unwrap().is_none(), "the service ended after {after}"); + let (code, h) = json_of(port, "GET", "/v1/health", None); + assert_eq!((code, h["ok"].as_bool()), (200, Some(true)), "after {after}: {h}"); +} + +fn dir(name: &str) -> std::path::PathBuf { + let d = std::env::temp_dir().join(format!("wai-qsvc-{name}-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&d); + std::fs::create_dir_all(&d).unwrap(); + d +} + +const HEAD: &str = "OPENQASM 3.0;\nqubit[1] q;\nbit[1] c;\n"; + +/// `rz(((…0.5…))) q[0];`, `n` parentheses deep: about 2n bytes. +fn parens(n: usize) -> String { + format!("{HEAD}rz({}0.5{}) q[0];\nc = measure q;\n", "(".repeat(n), ")".repeat(n)) +} + +#[test] +fn programs_nested_thousands_deep_are_refused_and_the_service_runs_on() { + let d = dir("depth"); + let port = free_port(); + let mut svc = start(&d, port, 1000); + let deep = parens(5_000); + let blocks = format!("{HEAD}{}x q[0];{}\nc = measure q;\n", "{ ".repeat(5_000), " }".repeat(5_000)); + let sum = format!("{HEAD}rz(0.5{}) q[0];\nc = measure q;\n", " + 0".repeat(50_000)); + let refused = |code: u16, body: &Value, what: &str| { + assert!((400..500).contains(&code), "{what}: {code} {body}"); + let text = body.to_string(); + assert!(text.contains("Q006"), "{what}: {body}"); + }; + + // The job's own program, through both submissions and a dry run. + for (path, qasm, what) in [("/v1/jobs", &deep, "parentheses"), ("/v1/run", &blocks, "blocks"), ("/v1/jobs", &sum, "a sum")] { + let (code, b) = json_of(port, "POST", path, Some(&json!({ "qasm": qasm, "shots": 10 }))); + refused(code, &b, what); + alive(port, &mut svc, what); + } + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&json!({ "qasm": deep, "shots": 10, "dry_run": true }))); + assert_eq!(code, 200, "{b}"); + assert_eq!(b["accepted"], false, "{b}"); + refused(400, &b["refusal"], "a dry run"); + alive(port, &mut svc, "a dry run"); + + // An intent's reference: `rz(((((…0…))))) q[0];`. + let reference = format!("{HEAD}rz({}0{}) q[0];\n", "(".repeat(5_000), ")".repeat(5_000)); + let bell = format!("{HEAD}h q[0];\nc = measure q;\n"); + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&json!({ "qasm": bell, "shots": 10, "intent": { "equivalent": reference } }))); + refused(code, &b, "a reference"); + alive(port, &mut svc, "a reference"); + // A set's items, and a sweep's generator. + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&json!({ "programs": [bell, deep], "shots": 10 }))); + refused(code, &b, "a set"); + alive(port, &mut svc, "a set"); + let generator = format!("OPENQASM 3.0;\ninput int n;\nconst int a = {}n{};\nqubit[1] q;\nx q[0];\n", "(".repeat(5_000), ")".repeat(5_000)); + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&json!({ "qasm": generator, "sweep": { "n": [1, 2] }, "shots": 10 }))); + refused(code, &b, "a sweep"); + alive(port, &mut svc, "a sweep"); + + // The MCP run tool, and a run as a task. + let call = |args: Value, task: bool| { + let mut params = json!({ "name": "run_quantum_circuit", "arguments": args }); + if task { + params["task"] = json!({ "ttl": 60000 }); + } + json_of(port, "POST", "/mcp", Some(&json!({ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": params }))) + }; + let (code, b) = call(json!({ "qasm": deep, "shots": 10 }), false); + assert_eq!((code, b["result"]["isError"].as_bool()), (200, Some(true)), "{b}"); + assert!(b.to_string().contains("Q006"), "{b}"); + alive(port, &mut svc, "the MCP run tool"); + let (code, b) = call(json!({ "qasm": bell, "shots": 10, "intent": { "equivalent": reference } }), true); + assert_eq!(code, 200, "{b}"); + assert!(b["error"].is_object() && b.to_string().contains("Q006"), "{b}"); + alive(port, &mut svc, "an MCP task"); + + // The checks: each reads its programs, past its count, off the runtime. + for (path, body) in [ + ("/v1/critique", json!({ "qasm": deep })), + ("/v1/equivalent", json!({ "a": deep, "b": bell })), + ("/v1/compile", json!({ "qasm": deep })), + ("/v1/estimate", json!({ "qasm": deep })), + ("/v1/shortcuts", json!({ "qasm": blocks })), + ] { + let (code, b) = json_of(port, "POST", path, Some(&body)); + assert!(code < 500, "{path}: {code} {b}"); + assert!(b.to_string().contains("Q006"), "{path}: {b}"); + alive(port, &mut svc, path); + } + + // An error-correction circuit's blocks, nested fifty thousand deep (as + // deep as a body of a megabyte holds). + let circuit = format!("{}H 0\n{}", "REPEAT 1 {\n".repeat(50_000), "}\n".repeat(50_000)); + let (code, b) = json_of(port, "POST", "/v1/qec/decode", Some(&json!({ "circuit": circuit, "shots": 10 }))); + assert!((400..500).contains(&code) && b.to_string().contains("nested deeper"), "{code} {b}"); + alive(port, &mut svc, "a circuit's blocks"); + + // Nested to what people write, a job still runs. + let (code, b) = json_of(port, "POST", "/v1/run", Some(&json!({ "qasm": parens(30), "shots": 10, "seed": 1 }))); + assert_eq!((code, b["status"].as_str()), (200, Some("done")), "{b}"); + + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// References that each take long to read, many of them: refused within one +/// budget for the whole request, each read once, in bounded time, as too +/// large for the service, and counted toward the client's window, so a +/// client cannot send such requests without end. +#[test] +fn a_request_with_heavy_references_is_refused_in_bounded_time_and_counted() { + let d = dir("budget"); + let port = free_port(); + let mut svc = start(&d, port, 4); + // Each reference loops five million times on reading, and each differs, + // so none is read from another's reading. + let heavy = |k: usize| format!("OPENQASM 3.0;\nqubit[1] q;\nint a = 0;\nfor int i in [0:4999999] {{ a += 1; }}\n// {k}\nx q[0];\n"); + let job = json!({ + "qasm": format!("{HEAD}x q[0];\n"), + "shots": 10, + "intent": (0..16).map(|k| json!({ "equivalent": heavy(k) })).collect::>(), + }); + let t = Instant::now(); + let (code, _, body) = http(port, "POST", "/v1/jobs", Some(&job)); + let took = t.elapsed(); + assert_eq!(code, 422, "{body}"); + assert!(body.contains("J031") && body.contains("steps"), "{body}"); + // One budget is about as long as one program the reader takes at its + // bound; sixteen references read three or four times each took minutes. + assert!(took < Duration::from_secs(60), "{took:?}"); + alive(port, &mut svc, "heavy references"); + + // Too many intents are refused before any is read. + let many = json!({ "qasm": format!("{HEAD}x q[0];\n"), "intent": (0..17).map(|k| json!({ "equivalent": heavy(k) })).collect::>() }); + let t = Instant::now(); + let (code, _, body) = http(port, "POST", "/v1/jobs", Some(&many)); + assert!(code == 400 && body.contains("J020") && body.contains("at most 16"), "{code} {body}"); + assert!(t.elapsed() < Duration::from_secs(5), "{:?}", t.elapsed()); + + // Two refused requests and two deep ones count: the fourth is the + // window's last, and the fifth is refused for the window. + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&json!({ "qasm": parens(5_000), "shots": 10 }))); + assert!(code == 400 && b.to_string().contains("Q006"), "{code} {b}"); + let (code, head, _) = http(port, "POST", "/v1/jobs", Some(&json!({ "qasm": parens(5_000), "shots": 10 }))); + assert_eq!(code, 400, "{head}"); + assert!(head.to_ascii_lowercase().contains("ratelimit: \"submissions\";r=0"), "{head}"); + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&json!({ "qasm": format!("{HEAD}x q[0];\n"), "shots": 10 }))); + assert!(code == 429 && b.to_string().contains("J030"), "{code} {b}"); + alive(port, &mut svc, "the window"); + + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} diff --git a/wai/wai-quantum/docs/qasm-diagnostics.md b/wai/wai-quantum/docs/qasm-diagnostics.md index 06322fd70..eef66bda3 100644 --- a/wai/wai-quantum/docs/qasm-diagnostics.md +++ b/wai/wai-quantum/docs/qasm-diagnostics.md @@ -45,6 +45,7 @@ engine. A test holds this page to the code's table. | `Q003` | refused | A string, a block comment, a bracket or a calibration block that is not closed. | | `Q004` | refused | Something else was expected here: a `;`, a name, an expression or a statement (the message says which). | | `Q005` | refused | A statement in a place the language does not allow it: `else` with no `if`, `case` outside `switch`, a declaration inside a gate. | +| `Q006` | refused | A program nested deeper than the reader takes: statements, parentheses, operators, brackets and types more than 64 levels deep in its text, or more than 256 levels in all through the gates and subroutines it calls; refused, never read by a recursion that could run out of stack. | | `Q010` | warned | An `OPENQASM` version other than 2.x or 3.x: the program is read as OpenQASM 3. | | `Q011` | warned | An `include` of a file other than the standard libraries (`stdgates.inc`, `qelib1.inc`): its definitions are not read, so a gate it defines is unknown. | | `Q012` | lowered | `include "stdgates.inc"` or `"qelib1.inc"`: the standard gates, `U`, `gphase` and OpenQASM 2's names are built in. | diff --git a/wai/wai-quantum/docs/qasm-feature-matrix.md b/wai/wai-quantum/docs/qasm-feature-matrix.md index 6213d823b..2e494a7a6 100644 --- a/wai/wai-quantum/docs/qasm-feature-matrix.md +++ b/wai/wai-quantum/docs/qasm-feature-matrix.md @@ -30,6 +30,8 @@ and in the browser build. | Comments `//` and `/* */` | supported; an unclosed block comment is refused | Q003 | | Identifiers, Unicode letters included (`θ`, `π`) | supported | | | Statements and blocks `{ }`, scopes | supported; qubits, gates, subroutines and externs are declared at the top level | Q105 | +| Nesting: statements in blocks and bodies, parentheses, operators, brackets, calls, casts and types | supported to 64 levels together in the text (a statement, each enclosing statement, and each level of its expressions' trees count one; a long `else if` chain is one level a branch, and reads as a `switch` at any length); deeper is refused, never read by a recursion that could run out of stack | Q006 | +| Nesting in all, through the gates and subroutines a program calls | supported to 256 levels (each statement, expression and call the reader runs counts one), and calls 32 deep; deeper is refused | Q006, Q209 | | `pragma …`, `#pragma …`, annotations `@keyword …` | warned: kept, not applied; `@wai.max_iterations N` bounds the `while` after it | Q013 | ## Literals @@ -114,6 +116,7 @@ and in the browser build. | `pow(k) @`, `k` a whole number | lowered exactly: angles scaled or the gate repeated; negative `k` repeats the inverse | Q203 | | `pow(r) @`, `r` fractional | lowered where an exact principal root exists (`pow(1/2) @ x` is `sx`; rotations by less than 2π; phases; controlled forms); refused otherwise (`u`, `cu`, user gates) | Q204, Q205 | | A modifier on a subroutine, or nested as deep as 32 | refused | Q211, Q209 | +| A chain of modifiers (`inv @ inv @ …`) | read at any length: a list, not a nesting | | ## Measurement and reset diff --git a/wai/wai-quantum/docs/qec.md b/wai/wai-quantum/docs/qec.md index 7ce59abce..2b39e3c29 100644 --- a/wai/wai-quantum/docs/qec.md +++ b/wai/wai-quantum/docs/qec.md @@ -123,9 +123,13 @@ instructions (`DEPOLARIZE1[leak](0.01) 0`), Pauli-target observables `SHIFT_COORDS`, and in models `^`-separated decompositions, `detector` and `logical_observable` declarations, `repeat` and `shift_detectors`. A circuit's model written back keeps its tags and coordinates. What the reader -does not take it refuses with the line, never skipping it. The -[conformance vectors](../../quantum-conformance/qec-text/) hold each form, -the model each circuit gives, and each refusal. +does not take it refuses with the line, never skipping it. Blocks nest at +most 32 deep (`MAX_BLOCK_DEPTH`): a deeper one is refused on the line that +opens it, so neither reading, unrolling nor writing a text recurses past +that. Every pass through a block counts toward the bound on what a text +unrolls to, so a block repeated without end that writes nothing still ends, +refused. The [conformance vectors](../../quantum-conformance/qec-text/) hold +each form, the model each circuit gives, and each refusal. ## Mitigation diff --git a/wai/wai-quantum/src/engine/answer/eval.rs b/wai/wai-quantum/src/engine/answer/eval.rs index 2ce04e744..0edb0201f 100644 --- a/wai/wai-quantum/src/engine/answer/eval.rs +++ b/wai/wai-quantum/src/engine/answer/eval.rs @@ -34,14 +34,14 @@ pub struct Host<'a> { /// The value of the statement `name`: what its view shows or its call /// gives, or why the engine refused it. pub fn value(a: &Answer, name: &str, host: &Host) -> Result { - Ev { a, host, memo: BTreeMap::new() }.stmt(name) + Ev { a, host, memo: BTreeMap::new(), depth: 0 }.stmt(name) } /// Every statement that is ready, carried out: `{"format", "digest", /// "results": {name: {"ok": true, "value"} | {"ok": false, "error", "code", /// …}}}`, in the order of their lines. pub fn evaluate(a: &Answer, host: &Host) -> Value { - let mut ev = Ev { a, host, memo: BTreeMap::new() }; + let mut ev = Ev { a, host, memo: BTreeMap::new(), depth: 0 }; let mut results = serde_json::Map::new(); for c in &a.stmts { if c.status != Status::Ready || matches!(c.stmt.body, Body::Program { .. }) { @@ -111,7 +111,7 @@ pub fn verify(text: &str, job_bytes: u64) -> Result { return Err(format!("the receipt names {name}, which is not a seal the answer can run ({})", c.status.as_str())); } let host = Host { job_bytes, key: None }; - let mut ev = Ev { a: &a, host: &host, memo: BTreeMap::new() }; + let mut ev = Ev { a: &a, host: &host, memo: BTreeMap::new(), depth: 0 }; let (request, result) = ev.sealed(seal_list(&c.stmt.body).ok_or("the seal lists nothing")?)?; let key = m["receipt"]["key"].clone(); let signer = key.as_str().and_then(job::did_of); @@ -134,10 +134,17 @@ fn seal_list(b: &Body) -> Option<&[Expr]> { } } +/// How many statements deep one statement's value may reach through the +/// statements it names (`a = @Seal([b])`, `b = @Seal([c])`, …): each is run +/// under the one that names it. +pub const CHAIN_DEPTH: usize = 32; + struct Ev<'a> { a: &'a Answer, host: &'a Host<'a>, memo: BTreeMap>, + /// How many statements are being run, each under the one that names it. + depth: usize, } impl Ev<'_> { @@ -145,6 +152,16 @@ impl Ev<'_> { if let Some(r) = self.memo.get(name) { return r.clone(); } + if self.depth >= CHAIN_DEPTH { + return Err(format!("{name} is reached through more than {CHAIN_DEPTH} statements, each naming the next: name fewer in a chain")); + } + self.depth += 1; + let r = self.stmt_here(name); + self.depth -= 1; + r + } + + fn stmt_here(&mut self, name: &str) -> Result { let c = self.a.get(name).ok_or_else(|| format!("no statement {name}"))?; let r = if c.status != Status::Ready { Err(format!("{name} is {}: nothing to run", c.status.as_str())) diff --git a/wai/wai-quantum/src/engine/answer/tests.rs b/wai/wai-quantum/src/engine/answer/tests.rs index 572f6b2e5..6af4cf788 100644 --- a/wai/wai-quantum/src/engine/answer/tests.rs +++ b/wai/wai-quantum/src/engine/answer/tests.rs @@ -295,3 +295,31 @@ fn the_prompt_names_every_view_and_call() { assert!(a.diags.is_empty(), "{:?}", a.diags); assert!(p.ends_with(EXAMPLE)); } + +/// A statement whose value runs through others, each sealing the next, is +/// refused past [`super::eval::CHAIN_DEPTH`] rather than run by a recursion +/// as deep as the answer is long; within it, it runs. +#[test] +fn a_chain_of_statements_past_the_limit_is_refused_not_overflowed() { + let chain = |n: usize| { + let mut t = String::from("```qasm p\nOPENQASM 3.0;\nqubit[1] q;\nbit[1] c;\nh q[0];\nc = measure q;\n```\n"); + for k in 0..n { + t += &format!("s{k} = @Seal([s{}])\n", k + 1); + } + t + &format!("s{n} = @Sample(p, 10)\n") + }; + let value = |text: String| { + std::thread::Builder::new() + .stack_size(2 << 20) + .spawn(move || { + let k = key(); + super::eval::value(&parse(&text), "s0", &super::eval::Host { job_bytes: 1 << 28, key: Some(&k) }) + }) + .unwrap() + .join() + .unwrap() + }; + let e = value(chain(2_000)).unwrap_err(); + assert!(e.contains("more than 32 statements"), "{e}"); + assert!(value(chain(4)).is_ok()); +} diff --git a/wai/wai-quantum/src/engine/dynamic.rs b/wai/wai-quantum/src/engine/dynamic.rs index 1b208714d..e948e8e03 100644 --- a/wai/wai-quantum/src/engine/dynamic.rs +++ b/wai/wai-quantum/src/engine/dynamic.rs @@ -214,58 +214,73 @@ impl Walk<'_> { /// The exact mixture's expectation values from step `pc`, the branch /// weighted by `weight`, added into `sums`; `leaves` counts the branches. + /// + /// The branches still to finish wait on a list, the latest first: at a + /// measurement with two outcomes, outcome 0 runs at once in a copy and + /// outcome 1 waits in this state. The leaves are reached in the order a + /// depth-first walk reaches them (so the sums are the same bits), and no + /// recursion follows the program's measurements, however many it makes. #[allow(clippy::too_many_arguments)] - fn expect(&mut self, mut pc: usize, mut state: E, mut bits: Vec, weight: f64, observables: &[Vec<(u32, char)>], sums: &mut [f64], leaves: &mut u64) -> Result<(), String> { - while pc < self.steps.len() { - match &self.steps[pc] { - Instr::Gate(i) => state.gate(*i)?, - Instr::Measure { .. } if self.skip.get(pc) == Some(&true) => {} - Instr::Measure { qubit, .. } | Instr::Reset(qubit) => { - let q = *qubit; - let (bit, reset) = step_target(&self.steps[pc]); - let p1 = state.prob_one(q)?; - let branches: Vec<(bool, f64)> = [(false, 1.0 - p1), (true, p1)].into_iter().filter(|&(_, p)| p > NEGLIGIBLE).collect(); - if let [(o, p), _] = branches[..] { - // Outcome 0 in a copy; outcome 1 in this state. - let mut child = self.copy(&state)?; - settle(&mut child, q, o, reset)?; - let mut child_bits = bits.clone(); - if let Some(b) = bit { - child_bits[b] = o; + fn expect(&mut self, pc: usize, state: E, bits: Vec, weight: f64, observables: &[Vec<(u32, char)>], sums: &mut [f64], leaves: &mut u64) -> Result<(), String> { + let mut todo = vec![(pc, state, bits, weight)]; + while let Some((mut pc, mut state, mut bits, mut weight)) = todo.pop() { + while pc < self.steps.len() { + match &self.steps[pc] { + Instr::Gate(i) => state.gate(*i)?, + Instr::Measure { .. } if self.skip.get(pc) == Some(&true) => {} + Instr::Measure { qubit, .. } | Instr::Reset(qubit) => { + let q = *qubit; + let (bit, reset) = step_target(&self.steps[pc]); + let p1 = state.prob_one(q)?; + let branches: Vec<(bool, f64)> = [(false, 1.0 - p1), (true, p1)].into_iter().filter(|&(_, p)| p > NEGLIGIBLE).collect(); + let (o, p) = *branches.last().ok_or("a measurement with no possible outcome")?; + if let [(o0, p0), _] = branches[..] { + // Outcome 0 in a copy, run now; outcome 1 in this state, after it. + let mut child = self.copy(&state)?; + settle(&mut child, q, o0, reset)?; + let mut child_bits = bits.clone(); + if let Some(b) = bit { + child_bits[b] = o0; + } + settle(&mut state, q, o, reset)?; + if let Some(b) = bit { + bits[b] = o; + } + todo.push((pc + 1, std::mem::replace(&mut state, child), std::mem::replace(&mut bits, child_bits), weight * p)); + weight *= p0; + } else { + settle(&mut state, q, o, reset)?; + if let Some(b) = bit { + bits[b] = o; + } } - self.expect(pc + 1, child, child_bits, weight * p, observables, sums, leaves)?; - self.live -= 1; } - let (o, p) = *branches.last().ok_or("a measurement with no possible outcome")?; - settle(&mut state, q, o, reset)?; - if let Some(b) = bit { - bits[b] = o; + Instr::JumpUnless(cond, to) => { + if !cond.holds(&bits) { + pc = *to; + continue; + } } - if branches.len() == 2 { - return self.expect(pc + 1, state, bits, weight * p, observables, sums, leaves); - } - } - Instr::JumpUnless(cond, to) => { - if !cond.holds(&bits) { + Instr::Jump(to) => { pc = *to; continue; } } - Instr::Jump(to) => { - pc = *to; - continue; - } + pc += 1; + } + *leaves += 1; + if *leaves > MAX_LEAVES { + return Err(format!( + "this program's measurements branch {MAX_LEAVES}+ ways, more than the exact expectation values follow; ask for counts, whose shots split only where they fall" + )); + } + for (s, o) in sums.iter_mut().zip(observables) { + *s += weight * state.expectation(o)?; + } + // This branch's state is done with; the next waiting one is alive. + if !todo.is_empty() { + self.live -= 1; } - pc += 1; - } - *leaves += 1; - if *leaves > MAX_LEAVES { - return Err(format!( - "this program's measurements branch {MAX_LEAVES}+ ways, more than the exact expectation values follow; ask for counts, whose shots split only where they fall" - )); - } - for (s, o) in sums.iter_mut().zip(observables) { - *s += weight * state.expectation(o)?; } Ok(()) } @@ -350,3 +365,42 @@ fn settle(state: &mut E, q: u32, outcome: bool, reset: bool) -> Resul pub fn measurements(steps: &[Instr]) -> usize { steps.iter().filter(|s| matches!(s, Instr::Measure { .. } | Instr::Reset(_))).count() } + +#[cfg(test)] +mod tests { + use serde_json::json; + + /// A program whose measurements branch thousands of times, each under + /// the last one's outcome, is walked without recursion: past + /// [`super::MAX_LEAVES`] the walk is refused, on a 2 MiB stack, rather + /// than recursing as deep as the program measures. + #[test] + fn a_program_that_branches_thousands_deep_is_refused_not_recursed() { + let src = format!("OPENQASM 3.0;\nqubit q;\nbit c;\nx q;\nc = measure q;\n{}", "if (c) { h q; c = measure q; }\n".repeat(5_000)); + let r = std::thread::Builder::new() + .stack_size(2 << 20) + .spawn(move || { + let req = json!({ "qasm": src, "observables": ["Z0"], "method": "statevector" }); + let plan = crate::engine::catalog::plan(&req, 1 << 30)?; + crate::engine::run::run(&req, &plan) + }) + .unwrap() + .join() + .unwrap(); + let e = r.unwrap_err(); + assert!(e.contains("4096+"), "{e}"); + } + + /// Walked with a list rather than a recursion, the branches are summed + /// into the exact mixture: q0 turned by 1 reads 1 with probability + /// sin²(½), and q1 copies it, so ⟨Z0⟩ = ⟨Z1⟩ = cos 1 and ⟨X1⟩ = 0. + #[test] + fn the_branches_sum_to_the_exact_mixture() { + let req = json!({ "qasm": "OPENQASM 3.0;\nqubit[2] q;\nbit c;\nry(1.0) q[0];\nc = measure q[0];\nif (c) x q[1];\n", "observables": ["Z0", "Z1", "X1"], "method": "statevector" }); + let plan = crate::engine::catalog::plan(&req, 1 << 30).unwrap(); + let r = crate::engine::run::run(&req, &plan).unwrap(); + let v: Vec = r["expectations"].as_array().unwrap().iter().map(|x| x["value"].as_f64().unwrap()).collect(); + assert!((v[0] - 1f64.cos()).abs() < 1e-12 && (v[1] - 1f64.cos()).abs() < 1e-12 && v[2].abs() < 1e-12, "{v:?}"); + } +} + diff --git a/wai/wai-quantum/src/engine/editor/index.rs b/wai/wai-quantum/src/engine/editor/index.rs index 6333448ae..89304fc63 100644 --- a/wai/wai-quantum/src/engine/editor/index.rs +++ b/wai/wai-quantum/src/engine/editor/index.rs @@ -180,7 +180,7 @@ impl Index { /// The index of a lexed program. pub fn build(src: &str, lexed: &Lexed) -> Index { let toks: Vec = lexed.tokens.iter().copied().filter(|t| !t.is_comment()).collect(); - let mut b = Builder { src, t: &toks, pos: 0, ix: Index::default(), scope: 0 }; + let mut b = Builder { src, t: &toks, pos: 0, ix: Index::default(), scope: 0, depth: 0 }; b.ix.scopes.push(Scope { kind: ScopeKind::Global, parent: None, span: (0, src.len()) }); while b.pos < toks.len() { let before = b.pos; @@ -294,8 +294,16 @@ struct Builder<'a> { pos: usize, ix: Index, scope: usize, + /// How many statements the builder is inside ([`MAX_DEPTH`]). + depth: usize, } +/// How deep statements nest for the index, as the reader takes them +/// ([`crate::qasm::parse::MAX_NESTING`]): a statement deeper is passed over +/// whole, its names not indexed, so a text nested past it (which the reader +/// refuses) never runs the builder out of stack. +const MAX_DEPTH: usize = crate::qasm::parse::MAX_NESTING; + impl Builder<'_> { fn tok(&self, i: usize) -> Option { self.t.get(i).copied() @@ -474,6 +482,42 @@ impl Builder<'_> { } fn statement(&mut self) { + if self.depth >= MAX_DEPTH { + return self.pass_over(); + } + self.depth += 1; + self.statement_here(); + self.depth -= 1; + } + + /// The statement at the cursor passed over without recursion: to its + /// `;` (read) or its block's `}` (read), or up to the `}` that closes + /// the block it stands in (not read). + fn pass_over(&mut self) { + let mut depth = 0usize; + while self.pos < self.t.len() { + match self.text(self.pos) { + "(" | "[" | "{" => depth += 1, + ")" | "]" => depth = depth.saturating_sub(1), + "}" if depth == 0 => return, + "}" => { + depth -= 1; + if depth == 0 { + self.pos += 1; + return; + } + } + ";" if depth == 0 => { + self.pos += 1; + return; + } + _ => {} + } + self.pos += 1; + } + } + + fn statement_here(&mut self) { let Some(tok) = self.tok(self.pos) else { return }; let stmt = tok.start; match tok.kind { @@ -1038,4 +1082,30 @@ mod tests { assert!(!ix.visible(0).iter().any(|&i| ix.decls[i].name == "i")); assert_eq!(ix.callable_of(ix.scope_at(inside_gate)).map(|d| ix.decls[d].name.as_str()), Some("g")); } + + /// Statements nested past the reader's limit (which the reader refuses) + /// are passed over, not recursed into: braces a megabyte deep, unclosed, + /// or `if` after `if`, index on a 2 MiB stack, and what stands beside + /// them is still found. + #[test] + fn nesting_past_the_reader_s_limit_is_passed_over_not_recursed_into() { + let on_stack = |src: String| std::thread::Builder::new().stack_size(2 << 20).spawn(move || index(&src)).unwrap().join().unwrap(); + for src in [ + format!("qubit[2] q;\n{}h q[0];\n", "{ ".repeat(200_000)), + format!("qubit[2] q;\n{}h q[0];{}\nbit c;\n", "{ ".repeat(100_000), " }".repeat(100_000)), + format!("qubit[2] q;\n{}h q[0];\nbit c;\n", "if (true) ".repeat(100_000)), + format!("qubit[2] q;\ngate g a {{ {}h a;{} }}\nbit c;\n", "{ ".repeat(100_000), " }".repeat(100_000)), + ] { + let closed = src.contains("bit c"); + let ix = on_stack(src); + assert_eq!(decl(&ix, "q").kind, DeclKind::QubitRegister); + if closed { + assert_eq!(decl(&ix, "c").kind, DeclKind::BitRegister); + } + } + // Nested within the limit, every use is still found. + let src = format!("qubit[2] q;\n{}h q[0];{}\n", "{ ".repeat(40), " }".repeat(40)); + let ix = on_stack(src); + assert!(ix.refs.iter().any(|r| r.name == "q" && r.decl.is_some())); + } } diff --git a/wai/wai-quantum/src/qasm.rs b/wai/wai-quantum/src/qasm.rs index a789a9eb0..80b93019c 100644 --- a/wai/wai-quantum/src/qasm.rs +++ b/wai/wai-quantum/src/qasm.rs @@ -368,10 +368,81 @@ pub fn read(src: &str) -> Result { /// [`read`] for a generator, at `inputs`. pub fn read_with(src: &str, inputs: &Inputs) -> Result { - let toks = lex::lex(src)?; - let (stmts, version) = parse::parse(src, &toks)?; - let (program, lines, diagnostics) = elab::elaborate(src, &toks, &stmts, version, inputs)?; - Ok(Reading { program, lines, diagnostics }) + // Within a budget of reading, a program read before (with no inputs) is not read again. + if inputs.is_empty() + && let Some(seen) = READS.with_borrow(|r| r.as_ref().and_then(|r| r.read.get(src).cloned())) + { + return seen; + } + let reading = (|| { + let toks = lex::lex(src)?; + let (stmts, version) = parse::parse(src, &toks)?; + let (program, lines, diagnostics) = elab::elaborate(src, &toks, &stmts, version, inputs)?; + Ok(Reading { program, lines, diagnostics }) + })(); + if inputs.is_empty() { + READS.with_borrow_mut(|r| { + if let Some(r) = r { + r.read.insert(src.to_string(), reading.clone()); + } + }); + } + reading +} + +/// A budget of reading on one thread ([`within_steps`]): the steps left +/// (statements elaborated, gates written, a generator's loops written out), +/// and each program read under it. +struct Reads { + left: u64, + read: std::collections::HashMap>, +} + +thread_local! { + static READS: std::cell::RefCell> = const { std::cell::RefCell::new(None) }; +} + +/// One step of reading (a statement elaborated, a gate written, a pass of a +/// generator's loop written out), spent from this thread's budget of reading +/// when one is set: whether the budget was already spent. +pub(crate) fn spend_step() -> bool { + READS.with_borrow_mut(|r| match r { + Some(r) if r.left == 0 => true, + Some(r) => { + r.left -= 1; + false + } + None => false, + }) +} + +/// Runs `f` with every program it reads on this thread read once (a program +/// read again with no inputs is answered from its first reading) and every +/// reading within one budget of `steps` together: each statement the reader +/// elaborates, each gate it writes, and each pass of a loop a generator is +/// written out with ([`instantiate`]) spending one; past it, a reading is +/// refused ([`diag::UNROLL_LIMIT`]), and [`steps_left`] is `Some(0)`. What a +/// service holds the programs of one request to, so its reading is bounded +/// however many programs the request carries and however often it reads +/// each. A budget set inside another runs within the outer one. +pub fn within_steps(steps: u64, f: impl FnOnce() -> T) -> T { + struct Scope(bool); + impl Drop for Scope { + fn drop(&mut self) { + if self.0 { + READS.with_borrow_mut(|r| *r = None); + } + } + } + let _scope = Scope(READS.with_borrow_mut(|r| r.is_none().then(|| *r = Some(Reads { left: steps, read: Default::default() })).is_some())); + f() +} + +/// The steps left of this thread's budget of reading ([`within_steps`]), or +/// `None` outside one: `Some(0)` once a reading was refused for it, so a +/// caller tells that refusal from the program's own. +pub fn steps_left() -> Option { + READS.with_borrow(|r| r.as_ref().map(|r| r.left)) } /// The integer `input`s a program declares (`input uint n;`, `input int k;`), @@ -1461,6 +1532,9 @@ impl Program { #[cfg(all(test, feature = "quantum_qasm"))] mod lang_tests; +#[cfg(all(test, feature = "quantum_qasm"))] +mod depth_tests; + #[cfg(all(test, feature = "quantum_qasm"))] mod tests { use super::*; diff --git a/wai/wai-quantum/src/qasm/depth_tests.rs b/wai/wai-quantum/src/qasm/depth_tests.rs new file mode 100644 index 000000000..06399f163 --- /dev/null +++ b/wai/wai-quantum/src/qasm/depth_tests.rs @@ -0,0 +1,259 @@ +//! Nesting: every recursive construct the reader takes, nested thousands +//! deep, refused with `Q006`; nested to just inside the limits, read. Each +//! runs on a thread of 2 MiB, a runtime worker's stack (and the default for a +//! test's), in whatever profile the tests are built in. + +use super::*; +use crate::qasm::parse::MAX_NESTING; + +/// A runtime worker's stack, and a test thread's. +const STACK: usize = 2 << 20; + +/// `f` on a thread with a stack of `bytes`. A stack overflow aborts the whole +/// test binary, which is what these tests guard against. +fn on_stack(bytes: usize, f: impl FnOnce() -> T + Send + 'static) -> T { + std::thread::Builder::new().stack_size(bytes).spawn(f).expect("a thread").join().expect("no panic") +} + +/// The code `src` is read with: `Ok` and its gates, or the refusal's code. +fn read_on_stack(src: String) -> Result { + on_stack(STACK, move || read(&src).map(|r| r.program.ops.len()).map_err(|d| d.code)) +} + +const HEAD: &str = "OPENQASM 3.0;\ninclude \"stdgates.inc\";\nqubit q;\nbit c;\n"; + +/// A program whose one statement nests `n` deep in the construct `kind`. +fn nested(kind: &str, n: usize) -> String { + let r = |s: &str| s.repeat(n); + let body = match kind { + "paren" => format!("rz({}0.5{}) q;\n", r("("), r(")")), + "minus" => format!("rz({}0.5) q;\n", r("- ")), + "not" => format!("bool b = {}true;\nx q;\n", r("!")), + "tilde" => format!("const uint k = {}1;\nx q;\n", r("~")), + "sum" => format!("rz(0.5{}) q;\n", r(" + 0")), + "power" => format!("rz(1{}) q;\n", r(" ** 1")), + "call" => format!("rz({}0.5{}) q;\n", r("sin("), r(")")), + "cast" => format!("rz({}0.5{}) q;\n", r("float("), r(")")), + "set" => format!("for int i in {}1{} x q;\n", r("{"), r("}")), + "index" => format!("if (c{} == 1) x q;\n", r("[0]")), + "concat" => format!("let a = q{};\nx q;\n", r(" ++ q")), + "block" => format!("{}x q;{}\n", r("{ "), r(" }")), + "if" => format!("{}x q;\n", r("if (true) ")), + "if_block" => format!("{}x q;{}\n", r("if (true) { "), r(" }")), + "else_if" => format!("{}x q;\n", r("if (false) x q; else ")), + "for" => format!("{}x q;{}\n", r("for int i in [0:0] { "), r(" }")), + "while" => format!("int k = 0;\n{}x q;\n", r("while (k > 0) ")), + "switch" => format!("int k = 0;\n{}x q;{}\n", r("switch (k) { case 0 { "), r(" } }")), + "box" => format!("{}x q;{}\n", r("box { "), r(" }")), + "gate_body" => format!("gate g a {{ {}x a;{} }}\ng q;\n", r("{ "), r(" }")), + "def_body" => format!("def f(qubit a) {{ {}x a;{} }}\nf(q);\n", r("{ "), r(" }")), + "complex" => format!("input {}float{} z;\nx q;\n", r("complex["), r("]")), + "array" => format!("def f(readonly {}int{} a) {{ }}\nx q;\n", r("array["), r(", 1]")), + "durationof" => format!("rz({}0.5{}) q;\n", r("durationof({ "), r(" })")), + "measure" => format!("c = {}measure q{};\n", r("("), r(")")), + other => panic!("no construct {other}"), + }; + format!("{HEAD}{body}") +} + +/// Every construct that nests, each read by a recursion of the parser, the +/// elaborator, or both. +const NESTING: [&str; 25] = [ + "paren", "minus", "not", "tilde", "sum", "power", "call", "cast", "set", "index", "concat", "block", "if", "if_block", "else_if", "for", "while", "switch", "box", "gate_body", "def_body", "complex", "array", + "durationof", "measure", +]; + +/// Nested thousands deep, as a request of a few kilobytes can be, each is +/// refused, `Q006`, on a 2 MiB stack: an overflow would end the process. +#[test] +fn every_construct_nested_past_the_limit_is_refused_not_overflowed() { + for kind in NESTING { + for n in [MAX_NESTING + 1, 5_000] { + assert_eq!(read_on_stack(nested(kind, n)), Err(diag::NESTING), "{kind} nested {n} deep"); + } + } + // Chains read in a loop build their tree as deep as they are long: a + // sum, a power, an index or a `++` of a hundred thousand is refused too. + for kind in ["sum", "power", "index", "concat", "minus", "not"] { + assert_eq!(read_on_stack(nested(kind, 100_000)), Err(diag::NESTING), "{kind} 100000 long"); + } +} + +/// What people write nests a few levels; to just inside the limit, each +/// construct still reads, and the reader's answer is the one it gave before. +#[test] +fn every_construct_nested_inside_the_limit_reads() { + // The statement and the call's arguments take two levels; the rest is + // the construct's. + let inside = MAX_NESTING - 8; + for kind in NESTING { + // `durationof({ … })` nests a block and a statement in each level. + let n = if kind == "durationof" { inside / 3 } else { inside }; + let r = read_on_stack(nested(kind, n)); + match kind { + // Read, or refused for what they are (a set of sets, a bit's + // bit), never for their depth. + "set" | "index" | "concat" | "tilde" | "complex" | "array" | "durationof" | "measure" => assert!(r.is_ok() || r.is_err_and(|c| c != diag::NESTING), "{kind}: {r:?}"), + _ => assert!(r.is_ok(), "{kind} nested {n} deep: {r:?}"), + } + } + // The angle comes through every level: the statement, the call's + // arguments and the number take three levels, the parentheses the rest. + let p = on_stack(STACK, || parse(&nested("paren", MAX_NESTING - 3)).unwrap()); + assert_eq!(p.ops[0].params, vec![0.5]); + let p = on_stack(STACK, || parse(&nested("minus", MAX_NESTING - 3)).unwrap()); + assert_eq!(p.ops[0].params, vec![if (MAX_NESTING - 3) % 2 == 0 { 0.5 } else { -0.5 }]); +} + +/// A modifier chain is a list, and an `include` a statement of its own: +/// neither nests, so either reads at any length (the gates the chain makes +/// are bounded by the reader's gate limit). +#[test] +fn modifier_chains_and_includes_read_at_any_length() { + let inv = format!("{HEAD}{}x q;\n", "inv @ ".repeat(100_001)); + assert_eq!(read_on_stack(inv), Ok(1)); + let includes = format!("{HEAD}{}x q;\n", "include \"other.inc\";\n".repeat(2_000)); + let r = on_stack(STACK, move || read(&includes).map(|r| (r.program.ops.len(), r.diagnostics.iter().all(|d| d.code == diag::UNKNOWN_INCLUDE)))); + assert_eq!(r, Ok((1, true))); +} + +/// Calls stack their bodies: within one body the text's limit holds, and +/// past what calls stack up to, in all, the elaborator refuses (`Q006`) +/// before the stack runs out. Each body here nests within the text's limit, +/// and thirty of them, each called from the one before, nest past it in all. +#[test] +fn calls_that_stack_past_the_limit_in_all_are_refused() { + let chain = |levels: usize, calls: usize| { + let mut s = format!("{HEAD}def f0(float t, qubit a) {{ rz(t) a; }}\n"); + for i in 1..=calls { + s += &format!("def f{i}(float t, qubit a) {{ {}f{}(t, a); }}\n", "if (true) ".repeat(levels), i - 1); + } + s + &format!("f{calls}(0.5, q);\n") + }; + assert_eq!(read_on_stack(chain(60, 31)), Err(diag::NESTING)); + // Shallower bodies read, through as many calls. + assert_eq!(read_on_stack(chain(4, 31)), Ok(1)); + // Calls deeper than the call limit keep their own code. + assert_eq!(read_on_stack(chain(0, 40)), Err(diag::TOO_DEEP)); +} + +/// A generator written out (`instantiate`) folds its integers and unrolls +/// its loops by a reader of its own: nested past the limit, it refuses too. +#[test] +fn a_generator_nested_past_the_limit_is_refused_when_written_out() { + let inst = |src: String| on_stack(STACK, move || instantiate(&src, &Inputs::new()).map(|_| ()).map_err(|e| e[e.len() - 5..e.len() - 1].to_string())); + for (what, src) in [ + ("parentheses", format!("{HEAD}const int a = {}1{};\nx q;\n", "(".repeat(5_000), ")".repeat(5_000))), + ("minus", format!("{HEAD}const int a = {}1;\nx q;\n", "- ".repeat(10_000))), + ("power", format!("{HEAD}const int a = 1{};\nx q;\n", " ** 1".repeat(10_000))), + ("loops", format!("{HEAD}{}x q;{}\n", "for int i in [0:0] { ".repeat(5_000), " }".repeat(5_000))), + ("conditions", format!("{HEAD}const int k = 1;\n{}x q;\n", "if (k == 1) ".repeat(5_000))), + ("else if", format!("{HEAD}const int k = 1;\n{}x q;\n", "if (k == 0) x q; else ".repeat(5_000))), + ] { + assert_eq!(inst(src), Err("Q006".into()), "{what}"); + } + // Inside the limit, written out as before. + let ok = format!("{HEAD}const int a = {}1{};\nx q;\n", "(".repeat(40), ")".repeat(40)); + assert!(on_stack(STACK, move || instantiate(&ok, &Inputs::new())).is_ok()); +} + +/// A budget of reading ([`within_steps`]) holds every program read under it +/// together, statements, gates and a generator's loop passes alike; each +/// program read once however often it is read; and says when it ran out. +#[test] +fn a_budget_holds_every_reading_under_it_together_and_each_program_is_read_once() { + let heavy = |k: usize| format!("OPENQASM 3.0;\nqubit q;\nint a = 0;\nfor int i in [0:99999] {{ a += 1; }}\n// {k}\nx q;\n"); + // One heavy program reads within a budget its size; read again it costs nothing. + let (first, again, left) = within_steps(500_000, || { + let first = read(&heavy(0)).is_ok(); + let spent = steps_left(); + let again = read(&heavy(0)).is_ok(); + (first, again, (spent, steps_left())) + }); + assert!(first && again); + assert!(left.0 == left.1 && left.0.is_some_and(|l| l < 400_000), "{left:?}"); + // Ten different ones do not: the budget refuses the reading that runs it out. + let (refused, left) = within_steps(500_000, || { + let r: Vec> = (0..10).map(|k| read(&heavy(k))).collect(); + (r.iter().position(Result::is_err), steps_left()) + }); + assert!(refused.is_some_and(|k| (1..10).contains(&k)), "{refused:?}"); + assert_eq!(left, Some(0)); + // Gates count: a power that writes a million gates is past a budget of a thousand. + let gates = "OPENQASM 3.0;\ngate g a { x a; h a; }\nqubit q;\npow(5000) @ g q;\n".to_string(); + let r = within_steps(1_000, || read(&gates).map(|_| ()).map_err(|d| d.code)); + assert_eq!(r, Err(diag::UNROLL_LIMIT)); + // Outside a budget, nothing is counted. + assert_eq!(steps_left(), None); + assert!(read(&heavy(0)).is_ok()); +} + +/// A small property test, with a seeded generator rather than a crate: any +/// mix of nesting constructs, at any depth, reads or is refused, and never +/// runs a 2 MiB stack out; inside the limit it reads, past it the refusal is +/// `Q006`. What the limit holds is the parser's recursion and each tree's +/// height together: a construct that nests by recursion (parentheses), one +/// that nests by height (a prefix `-`), or both (a call, a cast, a product +/// in parentheses). Either alone is at least the nesting; their sum, at most. +#[test] +fn any_mix_of_nesting_reads_or_is_refused_never_overflows() { + on_stack(STACK, || { + let mut x: u64 = 0x9e37_79b9_7f4a_7c15; + let mut next = move |n: u64| { + x ^= x << 13; + x ^= x >> 7; + x ^= x << 17; + x % n + }; + for _ in 0..300 { + let depth = next(400) as usize; + let statements = next(depth as u64 + 1) as usize; + let (mut open, mut close) = (String::new(), String::new()); + for _ in 0..statements { + match next(3) { + 0 => { + open += "{ "; + close = format!(" }}{close}"); + } + // With its own braces: `if (true) { … }` is one level, + // its body the braces' statements. + 1 => { + open += "if (true) { "; + close = format!(" }}{close}"); + } + _ => { + open += "for int i in [0:0] { "; + close = format!(" }}{close}"); + } + } + } + let (mut pre, mut post, mut recursion, mut height) = (String::new(), String::new(), 0, 0); + for _ in statements..depth { + let (p, q, r, h) = [("(", ")", 1, 0), ("- ", "", 0, 1), ("sin(", ")", 1, 1), ("float(", ")", 1, 1), ("(", " * 1)", 1, 1)][next(5) as usize]; + pre += p; + post = format!("{q}{post}"); + recursion += r; + height += h; + } + let src = format!("{HEAD}{open}rz({pre}0.5{post}) q;{close}\n"); + let r = read(&src).map(|_| ()).map_err(|d| d.code); + // The statements, the call's arguments and the number take their + // levels; a product's right operand, one more at most. + let at_least = statements + 2 + recursion.max(height) + 1; + let at_most = statements + 2 + recursion + height + 2; + if at_most <= MAX_NESTING { + assert_eq!(r, Ok(()), "{depth} deep:\n{src}"); + } else if at_least > MAX_NESTING { + assert_eq!(r, Err(diag::NESTING), "{depth} deep:\n{src}"); + } + } + }); +} + +/// The table's meaning for `Q006` states the limits the reader holds. +#[test] +fn the_code_states_the_limits() { + let m = diag::lookup(diag::NESTING).expect("Q006 is in the table").meaning; + assert!(m.contains(&format!("more than {MAX_NESTING} levels")), "{m}"); + assert!(m.contains(&format!("more than {} levels in all", super::elab::MAX_NEST)), "{m}"); +} diff --git a/wai/wai-quantum/src/qasm/diag.rs b/wai/wai-quantum/src/qasm/diag.rs index 60d9a1589..c9869c551 100644 --- a/wai/wai-quantum/src/qasm/diag.rs +++ b/wai/wai-quantum/src/qasm/diag.rs @@ -163,6 +163,7 @@ table! { UNCLOSED = "Q003", Refused, "A string, a block comment, a bracket or a calibration block that is not closed."; EXPECTED = "Q004", Refused, "Something else was expected here: a `;`, a name, an expression or a statement (the message says which)."; MISPLACED = "Q005", Refused, "A statement in a place the language does not allow it: `else` with no `if`, `case` outside `switch`, a declaration inside a gate."; + NESTING = "Q006", Refused, "A program nested deeper than the reader takes: statements, parentheses, operators, brackets and types more than 64 levels deep in its text, or more than 256 levels in all through the gates and subroutines it calls; refused, never read by a recursion that could run out of stack."; // Versions, includes, pragmas and annotations. UNKNOWN_VERSION = "Q010", Warned, "An `OPENQASM` version other than 2.x or 3.x: the program is read as OpenQASM 3."; diff --git a/wai/wai-quantum/src/qasm/elab.rs b/wai/wai-quantum/src/qasm/elab.rs index 45b0330f1..13a9ba882 100644 --- a/wai/wai-quantum/src/qasm/elab.rs +++ b/wai/wai-quantum/src/qasm/elab.rs @@ -26,6 +26,12 @@ use super::{ /// How deep gate definitions and subroutine calls may nest. pub(crate) const MAX_DEPTH: usize = 32; +/// How deep the reader's own recursion may go in all: statements, +/// expressions, conditions, gate definitions and subroutine calls, each a +/// level. Within one body the text's nesting ([`super::parse::MAX_NESTING`]) +/// bounds it; this bounds what calls stack on top of each other, so that +/// reading never runs out of stack. +pub(crate) const MAX_NEST: usize = 256; /// The most controls modifiers may add: a phase on m qubits is written as /// 2^m − 1 `cx` and as many `p`. pub(crate) const MAX_CONTROLS: u32 = 16; @@ -232,6 +238,8 @@ pub(crate) struct Elab<'a> { diags: Vec, count: u64, depth: usize, + /// How deep the reader's recursion stands ([`MAX_NEST`]). + nest: usize, runtime: usize, loops: Vec, frames: Vec, @@ -289,6 +297,7 @@ pub(crate) fn elaborate_with(src: &str, toks: &[Token], stmts: &[Stmt], version: diags: Vec::new(), count: 0, depth: 0, + nest: 0, runtime: 0, loops: Vec::new(), frames: Vec::new(), @@ -331,11 +340,30 @@ impl<'a> Elab<'a> { } } + /// `f` one level deeper in the reader's recursion, or the refusal past + /// [`MAX_NEST`]. + fn deeper(&mut self, at: Loc, f: impl FnOnce(&mut Self) -> R) -> R { + if self.nest >= MAX_NEST { + return self.fail( + diag::NESTING, + at, + format!("the program nests deeper than {MAX_NEST} levels in all through the gates and subroutines it calls (statements, expressions and calls together); call fewer levels deep, or nest less in each"), + ); + } + self.nest += 1; + let r = f(self); + self.nest -= 1; + r + } + fn tick(&mut self, at: Loc) -> R<()> { self.count += 1; if self.count > MAX_STEPS { return self.fail(diag::UNROLL_LIMIT, at, format!("the program unrolls past {MAX_STEPS} statements")); } + if super::spend_step() { + return self.fail(diag::UNROLL_LIMIT, at, "the programs read for one request elaborate past its budget of statements together (wai_quantum::qasm::within_steps)"); + } Ok(()) } @@ -452,6 +480,9 @@ impl<'a> Elab<'a> { if self.ops.len() >= MAX_OPS { return self.fail(diag::TOO_MANY_GATES, at, format!("more than {MAX_OPS} gates")); } + if super::spend_step() { + return self.fail(diag::UNROLL_LIMIT, at, "the programs read for one request write past its budget of statements and gates together (wai_quantum::qasm::within_steps)"); + } for &q in &qubits { self.touched[q as usize] = true; } @@ -946,15 +977,53 @@ impl<'a> Elab<'a> { } fn exec(&mut self, s: &Stmt) -> R { + self.deeper(s.at, |e| e.exec_here(s)) + } + + /// A statement run. The forms that hold other statements are run here; + /// the rest by [`Elab::exec_leaf`], so a nested statement stacks small + /// frames. + fn exec_here(&mut self, s: &Stmt) -> R { self.tick(s.at)?; if self.frames.last().is_none_or(|f| !f.gate) { self.top = s.at.line; } + if !s.anns.is_empty() { + self.annotations(s); + } + match &s.kind { + S::If { cond, then, other } => self.if_stmt(cond, then, other.as_deref()), + S::For { ty, var, over, body } => self.for_stmt(ty.as_ref(), var, over, body, s.at), + S::While { cond, body } => self.while_stmt(cond, body, s), + S::Switch { on, cases, default } => self.switch(on, cases, default.as_deref(), s.at), + S::Box { duration, body } => self.box_stmt(duration.as_ref(), body), + S::Block(b) => self.scoped(b), + _ => self.exec_leaf(s), + } + } + + #[inline(never)] + fn annotations(&mut self, s: &Stmt) { for (k, _, at) in &s.anns { if !(k == "wai.max_iterations" && matches!(s.kind, S::While { .. })) { self.note(diag::UNUSED_PRAGMA, *at, format!("the annotation `@{k}` is kept, not applied")); } } + } + + #[inline(never)] + fn box_stmt(&mut self, duration: Option<&Expr>, body: &[Stmt]) -> R { + let d = duration.map(|e| self.duration(e)).transpose()?; + self.extra.timing.push(Timing { at: self.ops.len(), kind: TimingKind::BoxStart, qubits: Vec::new(), duration: d }); + let f = self.scoped(body)?; + self.extra.timing.push(Timing { at: self.ops.len(), kind: TimingKind::BoxEnd, qubits: Vec::new(), duration: None }); + Ok(f) + } + + /// A statement that holds no other: a declaration, a gate's call, a + /// measurement, a jump. (A subroutine it calls runs its body under it.) + #[inline(never)] + fn exec_leaf(&mut self, s: &Stmt) -> R { let at = s.at; match &s.kind { S::Version(v) => { @@ -1084,13 +1153,6 @@ impl<'a> Elab<'a> { let qs = self.qubit_list(operands)?; self.extra.timing.push(Timing { at: self.ops.len(), kind: TimingKind::Delay, qubits: qs, duration: Some(d) }); } - S::Box { duration, body } => { - let d = duration.as_ref().map(|e| self.duration(e)).transpose()?; - self.extra.timing.push(Timing { at: self.ops.len(), kind: TimingKind::BoxStart, qubits: Vec::new(), duration: d }); - let f = self.scoped(body)?; - self.extra.timing.push(Timing { at: self.ops.len(), kind: TimingKind::BoxEnd, qubits: Vec::new(), duration: None }); - return Ok(f); - } S::Assign { target, op, value } => self.assign(target, op, value, at)?, S::Expr(e) => { if let E::Call(name, args) = &e.kind @@ -1101,10 +1163,6 @@ impl<'a> Elab<'a> { let _ = self.ev(e)?; } } - S::If { cond, then, other } => return self.if_stmt(cond, then, other.as_deref()), - S::For { ty, var, over, body } => return self.for_stmt(ty.as_ref(), var, over, body, at), - S::While { cond, body } => return self.while_stmt(cond, body, s), - S::Switch { on, cases, default } => return self.switch(on, cases, default.as_deref(), at), S::Break | S::Continue => { let brk = matches!(s.kind, S::Break); let word = if brk { "break" } else { "continue" }; @@ -1137,7 +1195,7 @@ impl<'a> Elab<'a> { } return Ok(Flow::End); } - S::Block(b) => return self.scoped(b), + S::If { .. } | S::For { .. } | S::While { .. } | S::Switch { .. } | S::Box { .. } | S::Block(_) => unreachable!("run by exec_here"), } Ok(Flow::Normal) } @@ -1733,6 +1791,10 @@ impl<'a> Elab<'a> { /// quotients, a constant power, the built-in functions and a cast to a /// real type. Anything else that depends on a parameter is refused. fn tangent(&mut self, e: &Expr) -> R> { + self.deeper(e.at, |x| x.tangent_here(e)) + } + + fn tangent_here(&mut self, e: &Expr) -> R> { let zero = self.tan.as_ref().map(Tan::zero).unwrap_or_default(); if !self.depends(e) { return Ok(zero); @@ -1888,6 +1950,10 @@ impl<'a> Elab<'a> { /// A gate by name, without modifiers, as primitives. fn expand(&mut self, name: &Name, params: &[f64], qubits: &[u32], at: Loc, depth: usize) -> R> { + self.deeper(at, |e| e.expand_here(name, params, qubits, at, depth)) + } + + fn expand_here(&mut self, name: &Name, params: &[f64], qubits: &[u32], at: Loc, depth: usize) -> R> { // The derivatives of these parameters, when a gradient is asked. let pending = match self.tan.as_mut() { Some(t) => Some(t.pending.take().unwrap_or_else(|| vec![t.zero(); params.len()])), @@ -1966,6 +2032,10 @@ impl<'a> Elab<'a> { /// A statement of a gate's body, its gates added to `prims`. fn gate_stmt(&mut self, s: &Stmt, prims: &mut Vec, depth: usize) -> R<()> { + self.deeper(s.at, |e| e.gate_stmt_here(s, prims, depth)) + } + + fn gate_stmt_here(&mut self, s: &Stmt, prims: &mut Vec, depth: usize) -> R<()> { self.tick(s.at)?; match &s.kind { S::Call { mods, name, params, operands, .. } => { @@ -2093,6 +2163,10 @@ impl<'a> Elab<'a> { /// A subroutine called: its body inlined. `dest` takes a returned bit; /// the result is the returned value, if any. fn call_def(&mut self, name: &Name, args: &[Expr], dest: Option>>, at: Loc) -> R> { + self.deeper(at, |e| e.call_def_here(name, args, dest, at)) + } + + fn call_def_here(&mut self, name: &Name, args: &[Expr], dest: Option>>, at: Loc) -> R> { let def = self.defs.get(&name.text).cloned().expect("checked by the caller"); if self.depth >= MAX_DEPTH { return self.fail(diag::TOO_DEEP, at, format!("subroutine calls nest deeper than {MAX_DEPTH}")); @@ -2592,6 +2666,10 @@ impl<'a> Elab<'a> { // ----------------------------------------------------------------------- fn cond(&mut self, e: &Expr) -> R { + self.deeper(e.at, |x| x.cond_here(e)) + } + + fn cond_here(&mut self, e: &Expr) -> R { Ok(match &e.kind { E::Unary("!", x) => match self.cond(x)? { Ct::Const(b) => Ct::Const(!b), @@ -2739,6 +2817,31 @@ impl<'a> Elab<'a> { // ----------------------------------------------------------------------- fn ev(&mut self, e: &Expr) -> R { + self.deeper(e.at, |x| x.ev_here(e)) + } + + /// An expression's value. The forms that hold other expressions are each + /// read by a function of their own, so a nested expression stacks small + /// frames. + fn ev_here(&mut self, e: &Expr) -> R { + let at = e.at; + match &e.kind { + E::Name(n) => self.name_ev(n, at), + E::Index(base, items) => self.index_ev(base, items, at), + E::Unary(op, x) => self.ev_unary(op, x, at), + E::Binary(op @ ("&&" | "||"), a, b) => self.ev_logic(op, a, b), + E::Binary(op, a, b) => self.ev_binary(op, a, b, at), + E::Call(name, args) => self.call_ev(name, args, at), + E::Cast(ty, x) => self.ev_cast(ty, x, at), + E::Set(items) => self.ev_set(items), + E::Concat(..) => self.reference(e), + _ => self.ev_leaf(e), + } + } + + /// A literal, or a form refused where a value is read. + #[inline(never)] + fn ev_leaf(&mut self, e: &Expr) -> R { let at = e.at; Ok(match &e.kind { E::Int(v) => { @@ -2765,75 +2868,85 @@ impl<'a> Elab<'a> { } Ev::Val(Val::Bits(digits.iter().rev().map(|&c| c == '1').collect())) } - E::Name(n) => self.name_ev(n, at)?, - E::Index(base, items) => self.index_ev(base, items, at)?, - E::Unary(op, x) => match self.evs(x)? { - Ev::Val(v) => Ev::Val(self.unary(op, v, at)?), - Ev::Bits(_, n) => return self.runtime_value(&n, at), - Ev::Qubits(_, n) => return self.fail(diag::WRONG_TYPE, at, format!("`{n}` is qubits, not a value")), - u => u, - }, - E::Binary(op @ ("&&" | "||"), a, b) => { - let x = self.ev(a)?; - match x { - Ev::Val(v) => { - let Some(t) = v.truthy() else { return self.fail(diag::WRONG_TYPE, a.at, format!("`{op}` reads true or false, not {}", v.kind())) }; - if (*op == "&&" && !t) || (*op == "||" && t) { - Ev::Val(Val::Bool(t)) - } else { - match self.ev(b)? { - Ev::Val(w) => match w.truthy() { - Some(u) => Ev::Val(Val::Bool(u)), - None => return self.fail(diag::WRONG_TYPE, b.at, format!("`{op}` reads true or false, not {}", w.kind())), - }, - Ev::Bits(_, n) => return self.runtime_value(&n, b.at), - o => o, - } - } - } - Ev::Bits(_, n) => return self.runtime_value(&n, a.at), - o => o, - } - } - E::Binary(op, a, b) => { - let x = self.evs(a)?; - let y = self.evs(b)?; - match (x, y) { - (Ev::Val(x), Ev::Val(y)) => Ev::Val(self.binop(op, x, y, at)?), - (Ev::Bits(_, n), _) | (_, Ev::Bits(_, n)) => return self.runtime_value(&n, at), - (Ev::Qubits(_, n), _) | (_, Ev::Qubits(_, n)) => return self.fail(diag::WRONG_TYPE, at, format!("`{n}` is qubits, not a value")), - (Ev::Unknown(w), _) | (_, Ev::Unknown(w)) => Ev::Unknown(w), - } - } - E::Call(name, args) => self.call_ev(name, args, at)?, - E::Cast(ty, x) => { - let t = self.ty(ty)?; - match self.evs(x)? { - Ev::Val(v) => Ev::Val(self.cast(v, &t, at)?), - Ev::Bits(b, n) if matches!(t, Ty::Int(_) | Ty::Uint(_) | Ty::Bool | Ty::Bit(_)) => Ev::Bits(b, n), - Ev::Bits(_, n) => return self.runtime_value(&n, at), - o => o, - } - } E::Measure(_) => return self.fail(diag::MEASURE_PLACE, at, "a measurement gives bits: assign it to a bit register (`c = measure q;`)"), - E::Set(items) => { - let mut out = Vec::with_capacity(items.len()); - for i in items { - match self.evs(i)? { - Ev::Val(v) => out.push(v), - Ev::Unknown(w) => return Ok(Ev::Unknown(w)), - Ev::Bits(_, n) => return self.runtime_value(&n, i.at), - Ev::Qubits(_, n) => return self.fail(diag::WRONG_TYPE, i.at, format!("`{n}` is qubits, not a value")), - } - } - Ev::Val(Val::Array(out)) - } E::Range(..) => return self.fail(diag::WRONG_TYPE, at, "a range is read only in an index or a `for` loop"), E::DurationOf(_) => return self.fail(diag::DURATION_OF, at, "`durationof` needs the duration of each gate, which the program does not give the reader"), - E::Concat(..) => self.reference(e)?, + E::Name(_) | E::Index(..) | E::Unary(..) | E::Binary(..) | E::Call(..) | E::Cast(..) | E::Set(_) | E::Concat(..) => unreachable!("read by ev_here"), }) } + #[inline(never)] + fn ev_unary(&mut self, op: &str, x: &Expr, at: Loc) -> R { + Ok(match self.evs(x)? { + Ev::Val(v) => Ev::Val(self.unary(op, v, at)?), + Ev::Bits(_, n) => return self.runtime_value(&n, at), + Ev::Qubits(_, n) => return self.fail(diag::WRONG_TYPE, at, format!("`{n}` is qubits, not a value")), + u => u, + }) + } + + /// `a && b`, `a || b`: `b` read only where `a` does not decide it. + #[inline(never)] + fn ev_logic(&mut self, op: &str, a: &Expr, b: &Expr) -> R { + let x = self.ev(a)?; + Ok(match x { + Ev::Val(v) => { + let Some(t) = v.truthy() else { return self.fail(diag::WRONG_TYPE, a.at, format!("`{op}` reads true or false, not {}", v.kind())) }; + if (op == "&&" && !t) || (op == "||" && t) { + Ev::Val(Val::Bool(t)) + } else { + match self.ev(b)? { + Ev::Val(w) => match w.truthy() { + Some(u) => Ev::Val(Val::Bool(u)), + None => return self.fail(diag::WRONG_TYPE, b.at, format!("`{op}` reads true or false, not {}", w.kind())), + }, + Ev::Bits(_, n) => return self.runtime_value(&n, b.at), + o => o, + } + } + } + Ev::Bits(_, n) => return self.runtime_value(&n, a.at), + o => o, + }) + } + + #[inline(never)] + fn ev_binary(&mut self, op: &str, a: &Expr, b: &Expr, at: Loc) -> R { + let x = self.evs(a)?; + let y = self.evs(b)?; + Ok(match (x, y) { + (Ev::Val(x), Ev::Val(y)) => Ev::Val(self.binop(op, x, y, at)?), + (Ev::Bits(_, n), _) | (_, Ev::Bits(_, n)) => return self.runtime_value(&n, at), + (Ev::Qubits(_, n), _) | (_, Ev::Qubits(_, n)) => return self.fail(diag::WRONG_TYPE, at, format!("`{n}` is qubits, not a value")), + (Ev::Unknown(w), _) | (_, Ev::Unknown(w)) => Ev::Unknown(w), + }) + } + + #[inline(never)] + fn ev_cast(&mut self, ty: &Type, x: &Expr, at: Loc) -> R { + let t = self.ty(ty)?; + Ok(match self.evs(x)? { + Ev::Val(v) => Ev::Val(self.cast(v, &t, at)?), + Ev::Bits(b, n) if matches!(t, Ty::Int(_) | Ty::Uint(_) | Ty::Bool | Ty::Bit(_)) => Ev::Bits(b, n), + Ev::Bits(_, n) => return self.runtime_value(&n, at), + o => o, + }) + } + + #[inline(never)] + fn ev_set(&mut self, items: &[Expr]) -> R { + let mut out = Vec::with_capacity(items.len()); + for i in items { + match self.evs(i)? { + Ev::Val(v) => out.push(v), + Ev::Unknown(w) => return Ok(Ev::Unknown(w)), + Ev::Bits(_, n) => return self.runtime_value(&n, i.at), + Ev::Qubits(_, n) => return self.fail(diag::WRONG_TYPE, i.at, format!("`{n}` is qubits, not a value")), + } + } + Ok(Ev::Val(Val::Array(out))) + } + fn name_ev(&mut self, n: &str, at: Loc) -> R { match n { "pi" | "π" => return Ok(Ev::Val(Val::Float(PI))), diff --git a/wai/wai-quantum/src/qasm/parse.rs b/wai/wai-quantum/src/qasm/parse.rs index bd144a8e9..c46fcce0e 100644 --- a/wai/wai-quantum/src/qasm/parse.rs +++ b/wai/wai-quantum/src/qasm/parse.rs @@ -21,16 +21,54 @@ pub struct Parser<'a> { toks: &'a [Token], pos: usize, pub version: Version, + /// How many statements, parentheses, operators, brackets and types the + /// parser is inside ([`MAX_NESTING`]). + depth: usize, } type R = Result; +/// How deep a program's text may nest: statements inside blocks and bodies, +/// parentheses, operators, brackets, calls, casts and types together, and +/// every expression's tree as deep as its statement leaves room for. Past it +/// the program is refused ([`diag::NESTING`], `Q006`) rather than read by a +/// recursion that could run out of stack: on a thread of 1 MiB (a wasm +/// module's) the reader and everything after it stay well inside it. +/// Programs people write nest a few levels; a long `else if` chain is one +/// level a branch, and reads as a `switch` at any length. +pub const MAX_NESTING: usize = 64; + /// Words that start a declaration of a classical type. const TYPES: [&str; 10] = ["bit", "int", "uint", "float", "angle", "bool", "complex", "duration", "stretch", "array"]; impl<'a> Parser<'a> { pub fn new(src: &'a str, toks: &'a [Token]) -> Self { - Parser { src, toks, pos: 0, version: Version::Unstated } + Parser { src, toks, pos: 0, version: Version::Unstated, depth: 0 } + } + + /// The refusal of what nests past [`MAX_NESTING`], at the current token. + fn too_deep(&self) -> R { + self.err( + diag::NESTING, + format!("this nests deeper than {MAX_NESTING} levels (statements, parentheses, operators, brackets and types together); write it in steps, with a `const` or a gate for a part, or a long `else if` chain as a `switch`"), + ) + } + + /// `f` one level deeper, or the refusal past [`MAX_NESTING`]. + fn nested(&mut self, f: impl FnOnce(&mut Self) -> R) -> R { + if self.depth >= MAX_NESTING { + return self.too_deep(); + } + self.depth += 1; + let r = f(self); + self.depth -= 1; + r + } + + /// Whether an expression of height `h` (its longest path to a leaf) + /// fits at this depth. + fn fits(&self, h: usize) -> R<()> { + if self.depth + h > MAX_NESTING { self.too_deep() } else { Ok(()) } } /// How many of its tokens have been read: a reader of another grammar @@ -148,6 +186,10 @@ impl<'a> Parser<'a> { } pub fn statement(&mut self) -> R { + self.nested(Self::statement_here) + } + + fn statement_here(&mut self) -> R { let mut anns = Vec::new(); while let Some(Tok::Annotation(k, r)) = self.peek() { anns.push((k.clone(), r.clone(), self.here())); @@ -158,6 +200,8 @@ impl<'a> Parser<'a> { Ok(Stmt { kind, at: at.to(self.last()), anns }) } + /// One statement's form. A dispatcher: each form is read by a function of + /// its own, so the frames a nested statement stacks are each small. fn statement_kind(&mut self) -> R { let Some(tok) = self.peek() else { return self.err(diag::EXPECTED, "expected a statement") }; match tok { @@ -169,7 +213,7 @@ impl<'a> Parser<'a> { self.pos += 1; return Ok(S::Verbatim(*v, t.clone())); } - Tok::Sym("{") => return Ok(S::Block(self.block()?)), + Tok::Sym("{") => return self.block().map(S::Block), Tok::Ident(_) => {} Tok::Annotation(..) => return self.err(diag::EXPECTED, "expected a statement after the annotation"), _ => return self.err(diag::EXPECTED, "expected a statement"), @@ -181,261 +225,30 @@ impl<'a> Parser<'a> { return self.declaration(How::Plain); } match word { - "OPENQASM" => { + "OPENQASM" => self.version_line(), + "include" => self.include(), + "qubit" => self.qubit(), + "qreg" | "creg" => self.old_register(word == "creg"), + "const" | "input" | "output" => { self.pos += 1; - let mut v = String::new(); - while let Some(t) = self.peek() { - if *t == Tok::Sym(";") { - break; - } - v.push_str(&match &self.toks[self.pos].tok { - Tok::Float(_) | Tok::Int(_) => self.src[self.toks[self.pos].start..self.toks[self.pos].end].to_string(), - t => text(t), - }); - self.pos += 1; - } - self.end()?; - Ok(S::Version(v)) - } - "include" => { - self.pos += 1; - let file = match self.peek() { - Some(Tok::Str(s)) => s.clone(), - _ => return self.err(diag::EXPECTED, "expected a file name in quotes"), - }; - self.pos += 1; - self.end()?; - Ok(S::Include(file)) - } - "qubit" => { - self.pos += 1; - let size = if self.eat("[") { - let e = self.expr()?; - self.expect("]")?; - Some(e) - } else { - None - }; - let name = self.name()?; - self.end()?; - Ok(S::Qubit { name, size }) - } - "qreg" | "creg" => { - let old = word == "creg"; - self.pos += 1; - let name = self.name()?; - let size = if self.eat("[") { - let e = self.expr()?; - self.expect("]")?; - Some(e) - } else { - None - }; - if old { - let init = if self.eat("=") { Some(self.expr()?) } else { None }; - self.end()?; - return Ok(S::Decl { ty: Type::Bit(Some(Box::new(size.unwrap_or(Expr { kind: E::Int(1), at: name.at })))), name, init, how: How::Plain }); - } - self.end()?; - Ok(S::Qubit { name, size }) - } - "const" => { - self.pos += 1; - self.declaration(How::Const) - } - "input" => { - self.pos += 1; - self.declaration(How::Input) - } - "output" => { - self.pos += 1; - self.declaration(How::Output) - } - "let" => { - self.pos += 1; - let name = self.name()?; - self.expect("=")?; - let value = self.concat()?; - self.end()?; - Ok(S::Alias { name, value }) - } - "gate" | "opaque" => { - let opaque = word == "opaque"; - self.pos += 1; - let name = self.name()?; - let mut params = Vec::new(); - if self.eat("(") && !self.eat(")") { - loop { - params.push(self.name()?); - if self.eat(")") { - break; - } - self.expect(",")?; - } - } - let mut qubits = vec![self.name()?]; - while self.eat(",") { - qubits.push(self.name()?); - } - if opaque { - self.end()?; - return Ok(S::Opaque { name, params, qubits }); - } - let body = self.block()?; - Ok(S::Gate { name, params, qubits, body }) - } - "def" => { - self.pos += 1; - let name = self.name()?; - self.expect("(")?; - let mut args = Vec::new(); - if !self.eat(")") { - loop { - args.push(self.arg()?); - if self.eat(")") { - break; - } - self.expect(",")?; - } - } - let ret = if self.eat("->") { Some(self.ty()?) } else { None }; - let body = self.block()?; - Ok(S::Def { name, args, ret, body }) - } - "extern" => { - self.pos += 1; - let name = self.name()?; - self.expect("(")?; - let mut depth = 1usize; - while depth > 0 { - match self.peek() { - Some(Tok::Sym("(")) => depth += 1, - Some(Tok::Sym(")")) => depth -= 1, - None => return self.err(diag::UNCLOSED, "expected `)`"), - _ => {} - } - self.pos += 1; - } - if self.eat("->") { - self.ty()?; - } - self.end()?; - Ok(S::Extern { name }) - } - "measure" => { - self.pos += 1; - let src = self.operand()?; - let target = if self.eat("->") { Some(self.operand()?) } else { None }; - self.end()?; - Ok(S::Measure { src, target }) - } - "reset" => { - self.pos += 1; - let q = self.operand()?; - self.end()?; - Ok(S::Reset(q)) - } - "barrier" => { - self.pos += 1; - let operands = self.operands_opt()?; - self.end()?; - Ok(S::Barrier(operands)) - } - "delay" => { - self.pos += 1; - self.expect("[")?; - let duration = self.expr()?; - self.expect("]")?; - let operands = self.operands_opt()?; - self.end()?; - Ok(S::Delay { duration, operands }) - } - "box" => { - self.pos += 1; - let duration = if self.eat("[") { - let e = self.expr()?; - self.expect("]")?; - Some(e) - } else { - None - }; - let body = self.block()?; - Ok(S::Box { duration, body }) - } - "if" => { - self.pos += 1; - self.expect("(")?; - let cond = self.expr()?; - self.expect(")")?; - let then = self.body()?; - let other = if self.eat_word("else") { Some(self.body()?) } else { None }; - Ok(S::If { cond, then, other }) + self.declaration(match word { + "const" => How::Const, + "input" => How::Input, + _ => How::Output, + }) } + "let" => self.alias(), + "gate" | "opaque" => self.gate(word == "opaque"), + "def" => self.def(), + "extern" => self.extern_decl(), + "measure" | "reset" | "barrier" | "delay" => self.quantum(word), + "box" => self.box_stmt(), + "if" => self.if_stmt(), "else" => self.err(diag::MISPLACED, "`else` with no `if` before it"), "case" | "default" => self.err(diag::MISPLACED, format!("`{word}` outside a `switch`")), - "for" => { - self.pos += 1; - let ty = if matches!(self.peek(), Some(Tok::Ident(w)) if TYPES.contains(&w.as_str())) && !matches!(self.peek_at(1), Some(Tok::Ident(w)) if w == "in") { - Some(self.ty()?) - } else { - None - }; - let var = self.name()?; - if !self.eat_word("in") { - return self.err(diag::EXPECTED, format!("expected `in` after `for {}`", var.text)); - } - let over = if self.is_sym("[") { - let open = self.here(); - self.pos += 1; - let r = self.index_item()?; - self.expect("]")?; - if !matches!(r.kind, E::Range(..)) { - return Err(Diagnostic::new(diag::BAD_RANGE, Some(open.span(self.src)), "a range is `[start:end]` or `[start:step:end]`")); - } - r - } else { - self.expr()? - }; - let body = self.body()?; - Ok(S::For { ty, var, over, body }) - } - "while" => { - self.pos += 1; - self.expect("(")?; - let cond = self.expr()?; - self.expect(")")?; - let body = self.body()?; - Ok(S::While { cond, body }) - } - "switch" => { - self.pos += 1; - self.expect("(")?; - let on = self.expr()?; - self.expect(")")?; - self.expect("{")?; - let (mut cases, mut default) = (Vec::new(), None); - while !self.eat("}") { - let at = self.here(); - if self.eat_word("case") { - let mut values = vec![self.expr()?]; - while self.eat(",") { - values.push(self.expr()?); - } - let body = self.block()?; - cases.push(Case { values, body, at }); - } else if self.eat_word("default") { - if default.is_some() { - return Err(Diagnostic::new(diag::MISPLACED, Some(at.span(self.src)), "a `switch` with two `default`s")); - } - default = Some(self.block()?); - } else if self.pos >= self.toks.len() { - return self.err(diag::UNCLOSED, "expected `}`"); - } else { - return self.err(diag::EXPECTED, "expected `case` or `default`"); - } - } - Ok(S::Switch { on, cases, default }) - } + "for" => self.for_stmt(), + "while" => self.while_stmt(), + "switch" => self.switch(), "break" | "continue" | "end" => { self.pos += 1; self.end()?; @@ -445,50 +258,314 @@ impl<'a> Parser<'a> { _ => S::End, }) } - "return" => { - self.pos += 1; - let value = if self.is_sym(";") { None } else { Some(self.expr()?) }; - self.end()?; - Ok(S::Return(value)) - } + "return" => self.return_stmt(), "ctrl" | "negctrl" | "inv" | "pow" if matches!(self.peek_at(1), Some(Tok::Sym("@" | "("))) => { let mods = self.modifiers()?; let name = self.name()?; self.call(mods, name) } + _ => self.named(), + } + } + + #[inline(never)] + fn version_line(&mut self) -> R { + self.pos += 1; + let mut v = String::new(); + while let Some(t) = self.peek() { + if *t == Tok::Sym(";") { + break; + } + v.push_str(&match &self.toks[self.pos].tok { + Tok::Float(_) | Tok::Int(_) => self.src[self.toks[self.pos].start..self.toks[self.pos].end].to_string(), + t => text(t), + }); + self.pos += 1; + } + self.end()?; + Ok(S::Version(v)) + } + + #[inline(never)] + fn include(&mut self) -> R { + self.pos += 1; + let file = match self.peek() { + Some(Tok::Str(s)) => s.clone(), + _ => return self.err(diag::EXPECTED, "expected a file name in quotes"), + }; + self.pos += 1; + self.end()?; + Ok(S::Include(file)) + } + + /// `[e]`, if one comes next: a register's or a type's size. + fn size(&mut self) -> R> { + if self.eat("[") { + let e = self.expr()?; + self.expect("]")?; + Ok(Some(e)) + } else { + Ok(None) + } + } + + #[inline(never)] + fn qubit(&mut self) -> R { + self.pos += 1; + let size = self.size()?; + let name = self.name()?; + self.end()?; + Ok(S::Qubit { name, size }) + } + + #[inline(never)] + fn old_register(&mut self, creg: bool) -> R { + self.pos += 1; + let name = self.name()?; + let size = self.size()?; + if creg { + let init = if self.eat("=") { Some(self.expr()?) } else { None }; + self.end()?; + return Ok(S::Decl { ty: Type::Bit(Some(Box::new(size.unwrap_or(Expr { kind: E::Int(1), at: name.at })))), name, init, how: How::Plain }); + } + self.end()?; + Ok(S::Qubit { name, size }) + } + + #[inline(never)] + fn alias(&mut self) -> R { + self.pos += 1; + let name = self.name()?; + self.expect("=")?; + let value = self.concat()?; + self.end()?; + Ok(S::Alias { name, value }) + } + + #[inline(never)] + fn gate(&mut self, opaque: bool) -> R { + self.pos += 1; + let name = self.name()?; + let mut params = Vec::new(); + if self.eat("(") && !self.eat(")") { + loop { + params.push(self.name()?); + if self.eat(")") { + break; + } + self.expect(",")?; + } + } + let mut qubits = vec![self.name()?]; + while self.eat(",") { + qubits.push(self.name()?); + } + if opaque { + self.end()?; + return Ok(S::Opaque { name, params, qubits }); + } + let body = self.block()?; + Ok(S::Gate { name, params, qubits, body }) + } + + #[inline(never)] + fn def(&mut self) -> R { + self.pos += 1; + let name = self.name()?; + self.expect("(")?; + let mut args = Vec::new(); + if !self.eat(")") { + loop { + args.push(self.arg()?); + if self.eat(")") { + break; + } + self.expect(",")?; + } + } + let ret = if self.eat("->") { Some(self.ty()?) } else { None }; + let body = self.block()?; + Ok(S::Def { name, args, ret, body }) + } + + #[inline(never)] + fn extern_decl(&mut self) -> R { + self.pos += 1; + let name = self.name()?; + self.expect("(")?; + let mut depth = 1usize; + while depth > 0 { + match self.peek() { + Some(Tok::Sym("(")) => depth += 1, + Some(Tok::Sym(")")) => depth -= 1, + None => return self.err(diag::UNCLOSED, "expected `)`"), + _ => {} + } + self.pos += 1; + } + if self.eat("->") { + self.ty()?; + } + self.end()?; + Ok(S::Extern { name }) + } + + /// `measure`, `reset`, `barrier` or `delay`, the word read next. + #[inline(never)] + fn quantum(&mut self, word: &str) -> R { + self.pos += 1; + let s = match word { + "measure" => { + let src = self.operand()?; + let target = if self.eat("->") { Some(self.operand()?) } else { None }; + S::Measure { src, target } + } + "reset" => S::Reset(self.operand()?), + "barrier" => S::Barrier(self.operands_opt()?), _ => { - let name = self.name()?; - match self.peek() { - Some(Tok::Sym("=" | "+=" | "-=" | "*=" | "/=" | "%=" | "**=" | "&=" | "|=" | "^=" | "~=" | "<<=" | ">>=")) => { - let target = Expr { kind: E::Name(name.text), at: name.at }; - self.assignment(target) - } - Some(Tok::Sym("[")) => { - // `c[0] = …` assigns; `x[20ns] q;` is a gate with a duration. - let open = self.pos; - let target = self.postfix(Expr { kind: E::Name(name.text.clone()), at: name.at })?; - if matches!(self.peek(), Some(Tok::Sym(s)) if s.ends_with('=') && !matches!(*s, "==" | "!=" | "<=" | ">=")) { - self.assignment(target) - } else { - self.pos = open; - self.call(Vec::new(), name) - } - } - Some(Tok::Sym("(")) if name.text != "gphase" => { - // `f(a, b);` calls a subroutine; `rz(t) q;` is a gate. - let open = self.pos; - let args = self.args()?; - if self.is_sym(";") { - self.pos += 1; - let at = name.at.to(self.last()); - return Ok(S::Expr(Expr { kind: E::Call(name, args), at })); - } - self.pos = open; - self.call(Vec::new(), name) - } - _ => self.call(Vec::new(), name), + self.expect("[")?; + let duration = self.expr()?; + self.expect("]")?; + let operands = self.operands_opt()?; + S::Delay { duration, operands } + } + }; + self.end()?; + Ok(s) + } + + #[inline(never)] + fn box_stmt(&mut self) -> R { + self.pos += 1; + let duration = self.size()?; + let body = self.block()?; + Ok(S::Box { duration, body }) + } + + /// `(e)`, a condition. + fn condition(&mut self) -> R { + self.expect("(")?; + let cond = self.expr()?; + self.expect(")")?; + Ok(cond) + } + + #[inline(never)] + fn if_stmt(&mut self) -> R { + self.pos += 1; + let cond = self.condition()?; + let then = self.body()?; + let other = if self.eat_word("else") { Some(self.body()?) } else { None }; + Ok(S::If { cond, then, other }) + } + + #[inline(never)] + fn for_stmt(&mut self) -> R { + self.pos += 1; + let ty = if matches!(self.peek(), Some(Tok::Ident(w)) if TYPES.contains(&w.as_str())) && !matches!(self.peek_at(1), Some(Tok::Ident(w)) if w == "in") { + Some(self.ty()?) + } else { + None + }; + let var = self.name()?; + if !self.eat_word("in") { + return self.err(diag::EXPECTED, format!("expected `in` after `for {}`", var.text)); + } + let over = if self.is_sym("[") { + let open = self.here(); + self.pos += 1; + let (r, _) = self.index_item()?; + self.expect("]")?; + if !matches!(r.kind, E::Range(..)) { + return Err(Diagnostic::new(diag::BAD_RANGE, Some(open.span(self.src)), "a range is `[start:end]` or `[start:step:end]`")); + } + r + } else { + self.expr()? + }; + let body = self.body()?; + Ok(S::For { ty, var, over, body }) + } + + #[inline(never)] + fn while_stmt(&mut self) -> R { + self.pos += 1; + let cond = self.condition()?; + let body = self.body()?; + Ok(S::While { cond, body }) + } + + #[inline(never)] + fn switch(&mut self) -> R { + self.pos += 1; + let on = self.condition()?; + self.expect("{")?; + let (mut cases, mut default) = (Vec::new(), None); + while !self.eat("}") { + let at = self.here(); + if self.eat_word("case") { + let mut values = vec![self.expr()?]; + while self.eat(",") { + values.push(self.expr()?); + } + let body = self.block()?; + cases.push(Case { values, body, at }); + } else if self.eat_word("default") { + if default.is_some() { + return Err(Diagnostic::new(diag::MISPLACED, Some(at.span(self.src)), "a `switch` with two `default`s")); + } + default = Some(self.block()?); + } else if self.pos >= self.toks.len() { + return self.err(diag::UNCLOSED, "expected `}`"); + } else { + return self.err(diag::EXPECTED, "expected `case` or `default`"); + } + } + Ok(S::Switch { on, cases, default }) + } + + #[inline(never)] + fn return_stmt(&mut self) -> R { + self.pos += 1; + let value = if self.is_sym(";") { None } else { Some(self.expr()?) }; + self.end()?; + Ok(S::Return(value)) + } + + /// A statement that starts with a name: an assignment, a subroutine's + /// call or a gate's. + #[inline(never)] + fn named(&mut self) -> R { + let name = self.name()?; + match self.peek() { + Some(Tok::Sym("=" | "+=" | "-=" | "*=" | "/=" | "%=" | "**=" | "&=" | "|=" | "^=" | "~=" | "<<=" | ">>=")) => { + let target = Expr { kind: E::Name(name.text), at: name.at }; + self.assignment(target) + } + Some(Tok::Sym("[")) => { + // `c[0] = …` assigns; `x[20ns] q;` is a gate with a duration. + let open = self.pos; + let (target, _) = self.postfix(Expr { kind: E::Name(name.text.clone()), at: name.at }, 1)?; + if matches!(self.peek(), Some(Tok::Sym(s)) if s.ends_with('=') && !matches!(*s, "==" | "!=" | "<=" | ">=")) { + self.assignment(target) + } else { + self.pos = open; + self.call(Vec::new(), name) } } + Some(Tok::Sym("(")) if name.text != "gphase" => { + // `f(a, b);` calls a subroutine; `rz(t) q;` is a gate. + let open = self.pos; + let args = self.args()?; + if self.is_sym(";") { + self.pos += 1; + let at = name.at.to(self.last()); + return Ok(S::Expr(Expr { kind: E::Call(name, args), at })); + } + self.pos = open; + self.call(Vec::new(), name) + } + _ => self.call(Vec::new(), name), } } @@ -543,7 +620,7 @@ impl<'a> Parser<'a> { "stretch" => Type::Stretch, "complex" => { if self.eat("[") { - let t = self.ty()?; + let t = self.nested(Self::ty)?; self.expect("]")?; Type::Complex(Some(Box::new(t))) } else { @@ -552,7 +629,7 @@ impl<'a> Parser<'a> { } "array" => { self.expect("[")?; - let t = self.ty()?; + let t = self.nested(Self::ty)?; let mut dims = Vec::new(); while self.eat(",") { if self.is_sym("#") { @@ -664,15 +741,28 @@ impl<'a> Parser<'a> { /// `(e, e, …)`, its parenthesis not yet read. fn args(&mut self) -> R> { + self.args_h().map(|(a, _)| a) + } + + /// [`Parser::args`], with the height of the tallest. + fn args_h(&mut self) -> R<(Vec, usize)> { self.expect("(")?; - let mut out = Vec::new(); - if self.eat(")") { - return Ok(out); + self.nested(|p| p.list(")")) + } + + /// `e, e, …` up to and including `close`, its opening already read: the + /// items, and the height of the tallest. + fn list(&mut self, close: &str) -> R<(Vec, usize)> { + let (mut out, mut h) = (Vec::new(), 0); + if self.eat(close) { + return Ok((out, h)); } loop { - out.push(self.expr()?); - if self.eat(")") { - return Ok(out); + let (e, he) = self.expr_h()?; + out.push(e); + h = h.max(he); + if self.eat(close) { + return Ok((out, h)); } self.expect(",")?; } @@ -692,15 +782,21 @@ impl<'a> Parser<'a> { /// A qubit or bit reference: a name, possibly indexed. fn operand(&mut self) -> R { + self.operand_h().map(|(e, _)| e) + } + + fn operand_h(&mut self) -> R<(Expr, usize)> { let n = self.name()?; - self.postfix(Expr { kind: E::Name(n.text), at: n.at }) + self.postfix(Expr { kind: E::Name(n.text), at: n.at }, 1) } /// `a ++ b ++ …` (an alias's value). fn concat(&mut self) -> R { - let mut a = self.expr()?; + let (mut a, mut ha) = self.expr_h()?; while self.eat("++") { - let b = self.expr()?; + let (b, hb) = self.expr_h()?; + ha = 1 + ha.max(hb); + self.fits(ha)?; let at = a.at.to(b.at); a = Expr { kind: E::Concat(Box::new(a), Box::new(b)), at }; } @@ -708,198 +804,253 @@ impl<'a> Parser<'a> { } pub fn expr(&mut self) -> R { + self.expr_h().map(|(e, _)| e) + } + + /// An expression and its height: the longest path from it to a leaf, + /// which every reader of its tree recurses along, held with the depth + /// it stands at to [`MAX_NESTING`]. + fn expr_h(&mut self) -> R<(Expr, usize)> { self.binary(0) } - /// Binary operators by precedence, loosest first. OpenQASM 3 reads `^` + /// Binary operators by precedence, loosest first: the operator at + /// `LEVELS[i]` binds operands of the levels after it. OpenQASM 3 reads `^` /// as exclusive or; a program that does not say `OPENQASM 3` reads it as /// OpenQASM 2 did, a power, binding as tightly as `**`. - fn binary(&mut self, level: usize) -> R { + /// + /// Read by precedence climbing: the operators at `min` or tighter, each + /// left to right, its right operand read at the next level; the tree is + /// the one reading each level in turn makes, with one frame a level of + /// nesting rather than one a precedence level. + fn binary(&mut self, min: usize) -> R<(Expr, usize)> { const LEVELS: [&[&str]; 10] = [&["||"], &["&&"], &["|"], &["^"], &["&"], &["==", "!="], &["<", "<=", ">", ">="], &["<<", ">>"], &["+", "-"], &["*", "/", "%"]]; - if level == LEVELS.len() { - return self.unary(); - } - let mut a = self.binary(level + 1)?; + let (mut a, mut ha) = self.unary()?; loop { - let op = match self.peek() { - Some(Tok::Sym(s)) if LEVELS[level].contains(s) && !(*s == "^" && self.version != Version::Three) => *s, - _ => return Ok(a), + let found = match self.peek() { + Some(Tok::Sym(s)) if !(*s == "^" && self.version != Version::Three) => LEVELS.iter().position(|l| l.contains(s)).map(|level| (*s, level)), + _ => None, }; + let Some((op, level)) = found.filter(|&(_, level)| level >= min) else { return Ok((a, ha)) }; self.pos += 1; - let b = self.binary(level + 1)?; + let (b, hb) = self.nested(|p| p.binary(level + 1))?; + ha = 1 + ha.max(hb); + self.fits(ha)?; let at = a.at.to(b.at); a = Expr { kind: E::Binary(op, Box::new(a), Box::new(b)), at }; } } - fn unary(&mut self) -> R { - let at = self.here(); - for op in ["-", "!", "~", "+"] { - if self.eat(op) { - let e = self.unary()?; - if op == "+" { - return Ok(e); - } - let at = at.to(e.at); - return Ok(Expr { kind: E::Unary(op, Box::new(e)), at }); - } + /// Prefix operators, then a power: `-x`, `!b`, `~k`, `+x`, `a ** b` + /// (`a ^ b` outside OpenQASM 3). A run of prefix operators is read in one + /// frame, the innermost applied first. + fn unary(&mut self) -> R<(Expr, usize)> { + let mut ops = Vec::new(); + while let Some(op) = ["-", "!", "~", "+"].into_iter().find(|op| self.is_sym(op)) { + ops.push((op, self.here())); + self.pos += 1; } - let base = self.postfix_atom()?; + let (mut e, mut h) = self.power()?; + for (op, at) in ops.into_iter().rev() { + if op == "+" { + continue; + } + h += 1; + self.fits(h)?; + let at = at.to(e.at); + e = Expr { kind: E::Unary(op, Box::new(e)), at }; + } + Ok((e, h)) + } + + /// `a`, or `a ** b` with `b` read as a prefix expression: `**` (and `^` + /// outside OpenQASM 3) groups to the right. + fn power(&mut self) -> R<(Expr, usize)> { + let (base, hb) = self.postfix_atom()?; let caret = self.version != Version::Three && self.is_sym("^"); if self.is_sym("**") || caret { let op = if caret { "^" } else { "**" }; self.pos += 1; - let e = self.unary()?; + let (e, he) = self.nested(Self::unary)?; + let h = 1 + hb.max(he); + self.fits(h)?; let at = base.at.to(e.at); - return Ok(Expr { kind: E::Binary(op, Box::new(base), Box::new(e)), at }); + return Ok((Expr { kind: E::Binary(op, Box::new(base), Box::new(e)), at }, h)); } - Ok(base) + Ok((base, hb)) } - fn postfix_atom(&mut self) -> R { - let a = self.atom()?; - self.postfix(a) + fn postfix_atom(&mut self) -> R<(Expr, usize)> { + let (a, h) = self.atom()?; + self.postfix(a, h) } - fn postfix(&mut self, mut a: Expr) -> R { + fn postfix(&mut self, mut a: Expr, mut h: usize) -> R<(Expr, usize)> { while self.is_sym("[") { self.pos += 1; - let mut items = Vec::new(); - if self.is_sym("{") { - let open = self.here(); - self.pos += 1; - let mut set = Vec::new(); - if !self.eat("}") { - loop { - set.push(self.expr()?); - if self.eat("}") { - break; - } - self.expect(",")?; - } + let (items, hi) = self.nested(|p| { + if p.is_sym("{") { + let open = p.here(); + p.pos += 1; + let (set, hs) = p.nested(|p| p.list("}"))?; + return Ok((vec![Expr { kind: E::Set(set), at: open.to(p.last()) }], hs + 1)); } - items.push(Expr { kind: E::Set(set), at: open.to(self.last()) }); - } else { + let (mut items, mut hi) = (Vec::new(), 0); loop { - items.push(self.index_item()?); - if !self.eat(",") { - break; + let (e, he) = p.index_item()?; + items.push(e); + hi = hi.max(he); + if !p.eat(",") { + return Ok((items, hi)); } } - } + })?; self.expect("]")?; + h = 1 + h.max(hi); + self.fits(h)?; let at = a.at.to(self.last()); a = Expr { kind: E::Index(Box::new(a), items), at }; } - Ok(a) + Ok((a, h)) } /// One item inside `[ ]`: an expression, or a range `a:b`, `a:s:b`, `:`. - fn index_item(&mut self) -> R { + fn index_item(&mut self) -> R<(Expr, usize)> { let at = self.here(); - let first = if self.is_sym(":") { None } else { Some(Box::new(self.expr()?)) }; + let mut h = 0; + let mut part = |p: &mut Self| -> R>> { + let (e, he) = p.expr_h()?; + h = h.max(he); + Ok(Some(Box::new(e))) + }; + let first = if self.is_sym(":") { None } else { part(self)? }; if !self.eat(":") { - return Ok(*first.expect("an expression was read")); + return Ok((*first.expect("an expression was read"), h)); } - let second = if self.is_sym(":") || self.is_sym("]") || self.is_sym(",") { None } else { Some(Box::new(self.expr()?)) }; + let second = if self.is_sym(":") || self.is_sym("]") || self.is_sym(",") { None } else { part(self)? }; let kind = if self.eat(":") { - let third = if self.is_sym("]") || self.is_sym(",") { None } else { Some(Box::new(self.expr()?)) }; + let third = if self.is_sym("]") || self.is_sym(",") { None } else { part(self)? }; E::Range(first, second, third) } else { E::Range(first, None, second) }; - Ok(Expr { kind, at: at.to(self.last()) }) + let h = h + 1; + self.fits(h)?; + Ok((Expr { kind, at: at.to(self.last()) }, h)) } - fn atom(&mut self) -> R { + /// A number, a name, a call, a cast, a set, a measurement, or an + /// expression in parentheses. A dispatcher: the forms that nest are each + /// read by a function of their own, so each level stacks small frames. + fn atom(&mut self) -> R<(Expr, usize)> { + match self.peek() { + Some(Tok::Sym("(")) => self.paren(), + Some(Tok::Sym("{")) => self.set(), + Some(Tok::Ident(w)) if matches!(w.as_str(), "true" | "false" | "measure" | "durationof") => self.word(), + Some(Tok::Ident(w)) if TYPES.contains(&w.as_str()) && w != "array" && matches!(self.peek_at(1), Some(Tok::Sym("(" | "["))) => self.cast(), + Some(Tok::Ident(_)) => self.name_or_call(), + _ => self.literal(), + } + } + + /// `( e )`: the expression, its place widened to the parentheses. + #[inline(never)] + fn paren(&mut self) -> R<(Expr, usize)> { + let at = self.here(); + self.pos += 1; + let (e, h) = self.nested(Self::expr_h)?; + self.expect(")")?; + Ok((Expr { kind: e.kind, at: at.to(self.last()) }, h)) + } + + /// `{a, b, c}`: an array's value, a loop's set. + #[inline(never)] + fn set(&mut self) -> R<(Expr, usize)> { + let at = self.here(); + self.pos += 1; + let (set, h) = self.nested(|p| p.list("}"))?; + self.node(E::Set(set), at, h + 1) + } + + /// A name, or a call `f(a, …)`. + #[inline(never)] + fn name_or_call(&mut self) -> R<(Expr, usize)> { + let at = self.here(); + let name = self.name()?; + if self.is_sym("(") { + let (args, h) = self.args_h()?; + return self.node(E::Call(name, args), at, h + 1); + } + self.node(E::Name(name.text), at, 1) + } + + /// A cast: `float(x)`, `int[8](x)`, `angle[20](t)`. + #[inline(never)] + fn cast(&mut self) -> R<(Expr, usize)> { + let at = self.here(); + let save = self.pos; + let (ty, (e, h)) = self.nested(|p| { + let ty = p.ty()?; + if !p.is_sym("(") { + p.pos = save; + let t = match p.peek() { + Some(Tok::Ident(w)) => w.clone(), + _ => String::new(), + }; + return p.err(diag::EXPECTED, format!("expected `(` after the type `{t}` to cast")); + } + p.pos += 1; + let e = p.expr_h()?; + p.expect(")")?; + Ok((ty, e)) + })?; + self.node(E::Cast(ty, Box::new(e)), at, h + 1) + } + + /// `true`, `false`, `measure q`, `durationof({ … })`. + #[inline(never)] + fn word(&mut self) -> R<(Expr, usize)> { + let at = self.here(); + let Some(Tok::Ident(w)) = self.peek() else { unreachable!("a word") }; + let w = w.clone(); + self.pos += 1; + match w.as_str() { + "true" | "false" => self.node(E::Bool(w == "true"), at, 1), + "measure" => { + let (q, h) = self.nested(Self::operand_h)?; + self.node(E::Measure(Box::new(q)), at, h + 1) + } + _ => { + self.expect("(")?; + let body = self.nested(Self::block)?; + self.expect(")")?; + self.node(E::DurationOf(body), at, 1) + } + } + } + + /// A number, a duration or a string. + #[inline(never)] + fn literal(&mut self) -> R<(Expr, usize)> { let at = self.here(); let Some(tok) = self.peek() else { return self.err(diag::EXPECTED, "expected a number, a name or `(`") }; let kind = match tok { - Tok::Int(v) => { - self.pos += 1; - E::Int(*v) - } - Tok::Float(v) => { - self.pos += 1; - E::Float(*v) - } - Tok::Imag(v) => { - self.pos += 1; - E::Imag(*v) - } - Tok::Duration(v, u) => { - self.pos += 1; - E::Duration(*v, *u) - } - Tok::Str(s) => { - self.pos += 1; - E::Str(s.clone()) - } - Tok::Sym("(") => { - self.pos += 1; - let e = self.expr()?; - self.expect(")")?; - return Ok(Expr { kind: e.kind, at: at.to(self.last()) }); - } - Tok::Sym("{") => { - self.pos += 1; - let mut set = Vec::new(); - if !self.eat("}") { - loop { - set.push(self.expr()?); - if self.eat("}") { - break; - } - self.expect(",")?; - } - } - E::Set(set) - } - Tok::Ident(w) => { - let w = w.clone(); - match w.as_str() { - "true" | "false" => { - self.pos += 1; - E::Bool(w == "true") - } - "measure" => { - self.pos += 1; - E::Measure(Box::new(self.operand()?)) - } - "durationof" => { - self.pos += 1; - self.expect("(")?; - let body = self.block()?; - self.expect(")")?; - E::DurationOf(body) - } - t if TYPES.contains(&t) && t != "array" && matches!(self.peek_at(1), Some(Tok::Sym("(" | "["))) => { - // A cast: `float(x)`, `int[8](x)`, `angle[20](t)`. - let save = self.pos; - let ty = self.ty()?; - if !self.is_sym("(") { - self.pos = save; - return self.err(diag::EXPECTED, format!("expected `(` after the type `{t}` to cast")); - } - self.pos += 1; - let e = self.expr()?; - self.expect(")")?; - E::Cast(ty, Box::new(e)) - } - _ => { - let name = self.name()?; - if self.is_sym("(") { - let args = self.args()?; - E::Call(name, args) - } else { - E::Name(name.text) - } - } - } - } + Tok::Int(v) => E::Int(*v), + Tok::Float(v) => E::Float(*v), + Tok::Imag(v) => E::Imag(*v), + Tok::Duration(v, u) => E::Duration(*v, *u), + Tok::Str(s) => E::Str(s.clone()), t => return self.err(diag::EXPECTED, format!("expected a number, a name or `(`, not `{}`", text(t))), }; - Ok(Expr { kind, at: at.to(self.last()) }) + self.pos += 1; + self.node(kind, at, 1) + } + + /// The expression `kind` from `at` to the last token read, of height `h`, + /// or the refusal past [`MAX_NESTING`]. + fn node(&self, kind: E, at: Loc, h: usize) -> R<(Expr, usize)> { + self.fits(h)?; + Ok((Expr { kind, at: at.to(self.last()) }, h)) } } diff --git a/wai/wai-quantum/src/qasm/pulse/tests.rs b/wai/wai-quantum/src/qasm/pulse/tests.rs index b3b532347..8553a240a 100644 --- a/wai/wai-quantum/src/qasm/pulse/tests.rs +++ b/wai/wai-quantum/src/qasm/pulse/tests.rs @@ -211,3 +211,21 @@ fn the_schedule_writes_as_json_with_every_instruction() { assert_eq!(j["instructions"][2]["bit"], "c"); assert_eq!(j["calls"][0]["name"], "measure"); } + +/// An expression in a calibration nested past the reader's limit (the pulse +/// reader reads a calibration's expressions with the program's parser) is +/// refused, `Q006`, on a runtime worker's 2 MiB stack, as deep as a request +/// of a few kilobytes nests it. +#[test] +fn a_calibration_s_expression_nested_past_the_limit_is_refused() { + let deep = format!("{CAL}defcal x $0 {{ play(q0, gaussian({}0.2{}, 160dt, 40dt)); }}\nx $0;\n", "(".repeat(5_000), ")".repeat(5_000)); + #[cfg(not(target_family = "wasm"))] + let d = std::thread::Builder::new().stack_size(2 << 20).spawn(move || refused(&deep)).unwrap().join().unwrap(); + // No threads under WASI: the module's own stack. + #[cfg(target_family = "wasm")] + let d = refused(&deep); + assert_eq!(d.code, crate::qasm::diag::NESTING, "{d}"); + let inside = format!("{CAL}defcal x $0 {{ play(q0, gaussian({}0.2{}, 160dt, 40dt)); }}\nx $0;\n", "(".repeat(40), ")".repeat(40)); + assert_eq!(plays(&sched(&inside)).len(), 1); +} + diff --git a/wai/wai-quantum/src/qasm/unroll.rs b/wai/wai-quantum/src/qasm/unroll.rs index 08cf7c7bd..bc85ac5ce 100644 --- a/wai/wai-quantum/src/qasm/unroll.rs +++ b/wai/wai-quantum/src/qasm/unroll.rs @@ -9,12 +9,14 @@ //! as it was, its integer names still replaced by their values; the reader //! reads that too. +use std::cell::Cell; use std::collections::{BTreeSet, HashMap}; use super::Inputs; use super::diag::{self, Diagnostic}; use super::elab::{MAX_STEPS, RESERVED}; use super::lex::{Tok, Token, lex, text}; +use super::parse::MAX_NESTING; type R = Result; @@ -210,7 +212,7 @@ fn unroll(src: &str, toks: Vec, inputs: &Inputs) -> R> { } return Ok(toks); } - let mut u = Unroller { src, toks: &toks, inputs, env: HashMap::new(), shadow: BTreeSet::new(), taken: taken_names(&toks), out: Vec::with_capacity(toks.len()), used: Vec::new(), iterations: 0 }; + let mut u = Unroller { src, toks: &toks, inputs, env: HashMap::new(), shadow: BTreeSet::new(), taken: taken_names(&toks), out: Vec::with_capacity(toks.len()), used: Vec::new(), iterations: 0, depth: Cell::new(0) }; u.seq(0, toks.len())?; if let Some(unused) = inputs.keys().find(|k| !u.used.contains(k)) { return err(diag::INPUT_VALUE, None, src, format!("the program declares no `input {unused}`")); @@ -266,9 +268,36 @@ struct Unroller<'a> { out: Vec, used: Vec<&'a String>, iterations: u64, + /// How deep the unroller's recursion stands: statements in statements, + /// held to the reader's [`MAX_NESTING`]. + depth: Cell, +} + +/// The refusal of what nests past [`MAX_NESTING`], at `t`. +fn too_deep(t: Option<&Token>, src: &str) -> R { + err( + diag::NESTING, + t, + src, + format!("this nests deeper than {MAX_NESTING} levels (statements, parentheses and operators together); write it in steps, with a `const` for a part, or a long `else if` chain as a `switch`"), + ) } impl<'a> Unroller<'a> { + /// One level deeper, at the token `i`, or the refusal past [`MAX_NESTING`]. + fn enter(&self, i: usize) -> R<()> { + let d = self.depth.get(); + if d >= MAX_NESTING { + return too_deep(self.tok(i.min(self.toks.len().saturating_sub(1))), self.src); + } + self.depth.set(d + 1); + Ok(()) + } + + fn leave(&self) { + self.depth.set(self.depth.get() - 1); + } + fn tok(&self, i: usize) -> Option<&'a Token> { self.toks.get(i).or(self.toks.last()) } @@ -347,6 +376,13 @@ impl<'a> Unroller<'a> { /// The index just past the statement at `i` (a loop or `if` included). fn statement_end(&self, i: usize) -> R { + self.enter(i)?; + let r = self.statement_end_here(i); + self.leave(); + r + } + + fn statement_end_here(&self, i: usize) -> R { match self.ident(i) { Some("if") => self.if_extent(i), Some("for") => { @@ -372,17 +408,23 @@ impl<'a> Unroller<'a> { } } - /// The index just past the `if` statement at `i`, its `else` included. - fn if_extent(&self, i: usize) -> R { - if !self.sym(i + 1, "(") { - return self.fail(diag::EXPECTED, i, "expected `(` after `if`"); + /// The index just past the `if` statement at `i`, its `else` included: + /// an `else if` chain followed in a loop, at any length. + fn if_extent(&self, mut i: usize) -> R { + loop { + if !self.sym(i + 1, "(") { + return self.fail(diag::EXPECTED, i, "expected `(` after `if`"); + } + let close = self.close(i + 1, ("(", ")"))?; + let (_, next) = self.body(close)?; + if self.ident(next) != Some("else") { + return Ok(next); + } + if self.ident(next + 1) != Some("if") { + return Ok(self.body(next + 1)?.1); + } + i = next + 1; } - let close = self.close(i + 1, ("(", ")"))?; - let (_, mut next) = self.body(close)?; - if self.ident(next) == Some("else") { - next = if self.ident(next + 1) == Some("if") { self.if_extent(next + 1)? } else { self.body(next + 1)?.1 }; - } - Ok(next) } /// Whether the next token written starts a statement. @@ -438,7 +480,7 @@ impl<'a> Unroller<'a> { /// The integer the tokens `lo..hi` evaluate to. fn value(&self, lo: usize, hi: usize) -> R { - let mut e = IntExpr { src: self.src, toks: &self.toks[lo..hi], pos: 0, env: &self.env, at: self.tok(lo) }; + let mut e = IntExpr { src: self.src, toks: &self.toks[lo..hi], pos: 0, env: &self.env, at: self.tok(lo), depth: 0 }; let v = e.or()?; if e.pos != e.toks.len() { return err(diag::EXPECTED, e.toks.get(e.pos), self.src, format!("unexpected `{}` in an integer expression", text(&e.toks[e.pos].tok))); @@ -505,6 +547,13 @@ impl<'a> Unroller<'a> { } fn seq(&mut self, lo: usize, hi: usize) -> R<()> { + self.enter(lo)?; + let r = self.seq_here(lo, hi); + self.leave(); + r + } + + fn seq_here(&mut self, lo: usize, hi: usize) -> R<()> { let mut i = lo; while i < hi { let statement = self.at_statement_start(); @@ -704,6 +753,9 @@ impl<'a> Unroller<'a> { if self.iterations > MAX_STEPS { return self.fail(diag::UNROLL_LIMIT, at, format!("the program unrolls past {MAX_STEPS} statements")); } + if super::spend_step() { + return self.fail(diag::UNROLL_LIMIT, at, "the programs read for one request unroll past its budget of steps together (wai_quantum::qasm::within_steps)"); + } let old = self.bind(name, v, at)?; self.seq(blo, bhi)?; self.restore(name, old); @@ -721,6 +773,8 @@ struct IntExpr<'a> { pos: usize, env: &'a HashMap, at: Option<&'a Token>, + /// How deep the reading stands: every level passes through `unary`. + depth: usize, } impl IntExpr<'_> { @@ -812,6 +866,16 @@ impl IntExpr<'_> { } fn unary(&mut self) -> R { + if self.depth >= MAX_NESTING { + return too_deep(self.toks.get(self.pos).or(self.at), self.src); + } + self.depth += 1; + let r = self.unary_here(); + self.depth -= 1; + r + } + + fn unary_here(&mut self) -> R { if self.eat("-") { let v = self.unary()?; return self.checked(v.checked_neg()); diff --git a/wai/wai-quantum/src/quantum_frame.rs b/wai/wai-quantum/src/quantum_frame.rs index 48aa2a028..708dfdc80 100644 --- a/wai/wai-quantum/src/quantum_frame.rs +++ b/wai/wai-quantum/src/quantum_frame.rs @@ -169,27 +169,52 @@ pub fn parse(text: &str) -> Result { pub fn lower(prog: &crate::quantum_qec_text::CircuitText) -> Result { let mut c = Circuit::default(); let mut shift: Vec = Vec::new(); - lower_items(&prog.items, &mut c, &mut shift)?; + lower_items(&prog.items, &mut c, &mut shift, &mut Unrolling { depth: 0, passes: 0, checking: false })?; Ok(c) } -fn lower_items(items: &[crate::quantum_qec_text::Item], c: &mut Circuit, shift: &mut Vec) -> Result<(), FrameError> { - use crate::quantum_qec_text::Item; +/// Where unrolling stands: how deep in blocks, how many passes through a +/// block's body it has made (each counted with the operations against +/// [`MAX_OPS`], so a block that adds nothing still ends), and whether it is +/// checking the body of a block repeated no times. +struct Unrolling { + depth: usize, + passes: u64, + checking: bool, +} + +fn lower_items(items: &[crate::quantum_qec_text::Item], c: &mut Circuit, shift: &mut Vec, u: &mut Unrolling) -> Result<(), FrameError> { + use crate::quantum_qec_text::{Item, MAX_BLOCK_DEPTH}; for it in items { match it { Item::Op(i) => lower_instruction(i, c, shift)?, Item::Repeat { count, body, line, .. } => { + let past = |m: String| FrameError::Parse { line: *line, message: m }; + if u.depth >= MAX_BLOCK_DEPTH { + return Err(past(format!("blocks nested deeper than {MAX_BLOCK_DEPTH}"))); + } + u.depth += 1; // A block repeated no times is still read, so what it holds - // is refused if it would be. - if *count == 0 { - lower_items(body, &mut c.clone(), &mut shift.clone())?; - } - for _ in 0..*count { - lower_items(body, c, shift)?; - if c.ops.len() > MAX_OPS { - return Err(FrameError::Parse { line: *line, message: format!("the circuit unrolls to more than {MAX_OPS} operations") }); - } - } + // is refused if it would be: once, on a copy, and a block + // inside one being checked is read once on that copy too. + let r = if *count == 0 && !u.checking { + u.checking = true; + let r = lower_items(body, &mut c.clone(), &mut shift.clone(), u); + u.checking = false; + r + } else { + let times = if u.checking { 1 } else { *count }; + (0..times).try_for_each(|_| { + u.passes += 1; + lower_items(body, c, shift, u)?; + if c.ops.len() as u64 + u.passes > MAX_OPS as u64 { + return Err(past(format!("the circuit unrolls to more than {MAX_OPS} operations and passes through its blocks"))); + } + Ok(()) + }) + }; + u.depth -= 1; + r?; } } } @@ -1412,4 +1437,25 @@ mod tests { assert_eq!(m.hyperedges(), 0, "d = {d}"); } } + + /// A block repeated without end that writes no operation (`TICK`, + /// coordinates, nothing) counts its passes with the operations, so it + /// ends at the bound, refused; blocks repeated no times, nested, are + /// each checked once on one copy, not on a copy each. + #[test] + fn blocks_that_write_nothing_end_and_unrun_ones_are_checked_once() { + let t = std::time::Instant::now(); + for body in ["TICK\n", "SHIFT_COORDS(1)\n", ""] { + let text = format!("H 0\nREPEAT 18446744073709551615 {{\n{body}}}\n"); + let e = parse(&text).unwrap_err().to_string(); + assert!(e.contains("unrolls to more than"), "{e}"); + } + assert!(t.elapsed() < std::time::Duration::from_secs(120), "{:?}", t.elapsed()); + let n = crate::quantum_qec_text::MAX_BLOCK_DEPTH; + let unrun = format!("{}H 0\n{}{}M 0\n", "REPEAT 0 {\n".repeat(n), "}\n".repeat(n), "H 0\n".repeat(1000)); + let c = parse(&unrun).unwrap(); + assert_eq!(c.ops.len(), 1001); + // What a block repeated no times holds is still refused. + assert!(parse("REPEAT 0 {\nREPEAT 0 {\nNOT_AN_INSTRUCTION 0\n}\n}\n").is_err()); + } } diff --git a/wai/wai-quantum/src/quantum_qec_text.rs b/wai/wai-quantum/src/quantum_qec_text.rs index 57c24215c..1238b5679 100644 --- a/wai/wai-quantum/src/quantum_qec_text.rs +++ b/wai/wai-quantum/src/quantum_qec_text.rs @@ -32,6 +32,10 @@ //! What a circuit means is [`crate::quantum_frame`]'s question: an //! instruction this module reads but that module does not implement is //! refused there, with its line, not skipped. +//! +//! Blocks nest at most [`MAX_BLOCK_DEPTH`] deep; a deeper one is refused on +//! its line, so reading, unrolling and writing a text never runs out of +//! stack. use std::fmt::Write as _; @@ -244,15 +248,27 @@ fn lines(text: &str) -> Vec<(usize, &str)> { text.lines().enumerate().map(|(i, l)| (i + 1, l.split('#').next().unwrap_or("").trim())).filter(|(_, l)| !l.is_empty()).collect() } +/// How deep `REPEAT` and `repeat` blocks may nest: far past any circuit's +/// or model's own (one or two levels), and shallow enough that each reader +/// of the blocks, which recurses into them, stays well inside its stack. +pub const MAX_BLOCK_DEPTH: usize = 32; + +fn too_deep(line: usize) -> TextError { + err(line, format!("blocks nested deeper than {MAX_BLOCK_DEPTH}: write the inner ones out, or fold them into one with the product of their counts")) +} + /// Read a circuit's text. pub fn parse_circuit(text: &str) -> Result { let ls = lines(text); let mut pos = 0; - let items = circuit_block(&ls, &mut pos, None)?; + let items = circuit_block(&ls, &mut pos, None, 0)?; Ok(CircuitText { items }) } -fn circuit_block(ls: &[(usize, &str)], pos: &mut usize, open: Option) -> Result, TextError> { +fn circuit_block(ls: &[(usize, &str)], pos: &mut usize, open: Option, depth: usize) -> Result, TextError> { + if let (Some(line), true) = (open, depth > MAX_BLOCK_DEPTH) { + return Err(too_deep(line)); + } let mut items = Vec::new(); while *pos < ls.len() { let (line, text) = ls[*pos]; @@ -269,7 +285,7 @@ fn circuit_block(ls: &[(usize, &str)], pos: &mut usize, open: Option) -> if !h.args.is_empty() { return Err(err(line, "REPEAT takes no arguments")); } - let body = circuit_block(ls, pos, Some(line))?; + let body = circuit_block(ls, pos, Some(line), depth + 1)?; items.push(Item::Repeat { count, tag: h.tag, body, line }); continue; } @@ -353,11 +369,14 @@ fn write_items(out: &mut String, items: &[Item], depth: usize) { pub fn parse_dem(text: &str) -> Result { let ls = lines(text); let mut pos = 0; - let items = dem_block(&ls, &mut pos, None)?; + let items = dem_block(&ls, &mut pos, None, 0)?; Ok(DemText { items }) } -fn dem_block(ls: &[(usize, &str)], pos: &mut usize, open: Option) -> Result, TextError> { +fn dem_block(ls: &[(usize, &str)], pos: &mut usize, open: Option, depth: usize) -> Result, TextError> { + if let (Some(line), true) = (open, depth > MAX_BLOCK_DEPTH) { + return Err(too_deep(line)); + } let mut items = Vec::new(); while *pos < ls.len() { let (line, text) = ls[*pos]; @@ -372,7 +391,7 @@ fn dem_block(ls: &[(usize, &str)], pos: &mut usize, open: Option) -> Resu match h.name { "repeat" => { let count = h.rest.strip_suffix('{').map(str::trim).and_then(|n| n.parse::().ok()).ok_or_else(|| err(line, "expected 'repeat {'"))?; - let body = dem_block(ls, pos, Some(line))?; + let body = dem_block(ls, pos, Some(line), depth + 1)?; items.push(DemItem::Repeat { count, tag: h.tag, body, line }); } "error" | "detector" | "logical_observable" | "shift_detectors" => { @@ -415,7 +434,7 @@ impl DemText { /// mechanisms with one tag are combined as independent events. pub fn to_model(&self) -> Result { let mut walk = DemWalk::default(); - walk.items(&self.items)?; + walk.items(&self.items, 0)?; Ok(walk.finish()) } } @@ -468,12 +487,18 @@ struct DemWalk { } impl DemWalk { - fn items(&mut self, items: &[DemItem]) -> Result<(), TextError> { + /// The items unrolled, `depth` blocks deep. A block's every pass counts + /// as work, so one that holds nothing still ends within the bound. + fn items(&mut self, items: &[DemItem], depth: usize) -> Result<(), TextError> { for it in items { match it { DemItem::Repeat { count, body, line, .. } => { + if depth >= MAX_BLOCK_DEPTH { + return Err(too_deep(*line)); + } for _ in 0..*count { - self.items(body)?; + self.work += 1; + self.items(body, depth + 1)?; if self.work > MAX_UNROLLED { return Err(err(*line, format!("the blocks unroll to more than {MAX_UNROLLED} instructions"))); } @@ -701,4 +726,42 @@ logical_observable L1 assert!(parse_dem("error(0.1) D0 ^ ^ D1\n").unwrap().to_model().is_err()); assert!(parse_dem("error(1.5) D0\n").unwrap().to_model().is_err()); } + + /// `f` on a thread of 2 MiB, a runtime worker's stack. + fn on_stack(f: impl FnOnce() -> T + Send + 'static) -> T { + std::thread::Builder::new().stack_size(2 << 20).spawn(f).unwrap().join().unwrap() + } + + /// Blocks nested past [`MAX_BLOCK_DEPTH`], as deep as a megabyte of + /// `REPEAT 1 {` lines makes them, are refused on the line that opens one + /// too deep; nested to the limit they read, unroll and write back. + #[test] + fn blocks_nested_past_the_limit_are_refused_on_their_line() { + let circuit = |n: usize| format!("{}H 0\n{}", "REPEAT 1 {\n".repeat(n), "}\n".repeat(n)); + let model = |n: usize| format!("{}error(0.1) D0\n{}", "repeat 1 {\n".repeat(n), "}\n".repeat(n)); + for n in [MAX_BLOCK_DEPTH + 1, 100_000] { + let e = on_stack(move || parse_circuit(&circuit(n)).unwrap_err()); + assert_eq!(e.line, MAX_BLOCK_DEPTH + 1, "{}", e.message); + assert!(e.message.contains("nested deeper"), "{}", e.message); + let e = on_stack(move || parse_dem(&model(n)).unwrap_err()); + assert_eq!(e.line, MAX_BLOCK_DEPTH + 1, "{}", e.message); + } + let c = on_stack(move || parse_circuit(&circuit(MAX_BLOCK_DEPTH)).unwrap()); + assert_eq!(parse_circuit(&c.to_text()).unwrap(), c); + assert_eq!(on_stack(move || crate::quantum_frame::lower(&c).map(|c| c.ops.len())).unwrap(), 1); + let m = on_stack(move || parse_dem(&model(MAX_BLOCK_DEPTH)).unwrap().to_model().map(|m| m.mechanisms.len())); + assert_eq!(m.unwrap(), 1); + } + + /// A block that holds no mechanism still counts each pass toward the + /// bound, so one repeated 2^64 − 1 times ends at once, refused. + #[test] + fn an_empty_block_repeated_without_end_is_refused_not_run() { + let t = std::time::Instant::now(); + let e = parse_dem("repeat 18446744073709551615 {\n}\nerror(0.1) D0\n").unwrap().to_model().unwrap_err(); + assert!(e.message.contains("unroll"), "{}", e.message); + let e = parse_dem("repeat 18446744073709551615 {\nrepeat 18446744073709551615 {\n}\n}\n").unwrap().to_model().unwrap_err(); + assert!(e.message.contains("unroll"), "{}", e.message); + assert!(t.elapsed() < std::time::Duration::from_secs(60), "{:?}", t.elapsed()); + } } diff --git a/wai/wai-web/src/quantum_engine.rs b/wai/wai-web/src/quantum_engine.rs index d210905dc..d3cb729a2 100644 --- a/wai/wai-web/src/quantum_engine.rs +++ b/wai/wai-web/src/quantum_engine.rs @@ -1622,4 +1622,33 @@ mod wasm_tests { assert_eq!(body["status"], "done", "{req}: {body}"); } } + + /// A program nested thousands deep is refused with the reader's code, + /// never a trap, through every door that reads one: a run, the lint, the + /// critique, a generator written out; an error-correction circuit's + /// blocks are refused on their line, and an answer's chain of seals past + /// its depth; and the engine answers the next request as before. + #[wasm_bindgen_test] + fn a_program_nested_past_the_reader_s_limit_is_refused_not_trapped() { + let e = QuantumEngine::new(&[9u8; 32], 512).unwrap(); + let deep = format!("OPENQASM 3.0;\nqubit q;\nrz({}0.5{}) q;\n", "(".repeat(5_000), ")".repeat(5_000)); + let blocks = format!("OPENQASM 3.0;\nqubit q;\n{}x q;{}\n", "{ ".repeat(5_000), " }".repeat(5_000)); + for src in [&deep, &blocks] { + assert!(run(&e, json!({ "qasm": src, "shots": 10 })).to_string().contains("Q006")); + assert!(engine_lint(src).contains("Q006")); + assert!(engine_critique(src).contains("Q006")); + } + let generator = format!("OPENQASM 3.0;\ninput int n;\nconst int a = {}n{};\nqubit q;\nx q;\n", "(".repeat(5_000), ")".repeat(5_000)); + assert!(engine_instantiate(&generator, "{\"n\": 1}").contains("Q006")); + let circuit = format!("{}H 0\n{}", "REPEAT 1 {\n".repeat(50_000), "}\n".repeat(50_000)); + assert!(e.qec("decode", &json!({ "circuit": circuit, "shots": 10 }).to_string()).contains("nested deeper")); + let mut chain = String::from("```qasm p\nOPENQASM 3.0;\nqubit[1] q;\nbit[1] c;\nh q[0];\nc = measure q;\n```\n"); + for k in 0..2_000 { + chain += &format!("s{k} = @Seal([s{}])\n", k + 1); + } + chain += "s2000 = @Sample(p, 10)\n"; + assert!(e.answer_value(&chain, "s0", true).contains("more than 32 statements")); + let body = run(&e, json!({ "qasm": "OPENQASM 3.0; qubit[2] q; h q[0]; cx q[0], q[1];", "shots": 20 })); + assert_eq!(body["status"], "done", "{body}"); + } } -- 2.54.0 (Apple Git-157) From 2e00ac8ffb37983a8a87a23e46fa7ef2b3492067 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Fri, 6 Nov 2026 23:40:03 -0500 Subject: [PATCH 02/18] wai-quantum: what a program would hold is refused before it is made, and reading goes to a child that can be stopped Co-Authored-By: Claude Opus 5.5 --- wai/changes/2026-11-06-quantum-parse-depth.md | 31 ++- wai/wai-quantum-cli/docs/llms-full.txt | 2 + wai/wai-quantum-service/README.md | 21 +- wai/wai-quantum-service/src/analysis.rs | 119 ++++++--- wai/wai-quantum-service/src/http.rs | 200 +++++++++----- wai/wai-quantum-service/src/jobs.rs | 62 ++++- wai/wai-quantum-service/src/main.rs | 6 + wai/wai-quantum-service/src/mcp.rs | 19 +- wai/wai-quantum-service/src/offload.rs | 234 +++++++++++++--- wai/wai-quantum-service/tests/depth.rs | 137 ++++++++++ wai/wai-quantum/docs/qasm-diagnostics.md | 1 + wai/wai-quantum/docs/qasm-feature-matrix.md | 1 + wai/wai-quantum/src/qasm.rs | 17 ++ wai/wai-quantum/src/qasm/depth_tests.rs | 85 ++++++ wai/wai-quantum/src/qasm/diag.rs | 1 + wai/wai-quantum/src/qasm/elab.rs | 250 ++++++++++++++++-- wai/wai-quantum/src/qasm/unroll.rs | 19 +- wai/wai-quantum/src/qasm/value.rs | 8 + wai/wai-web/src/quantum_engine.rs | 9 +- 19 files changed, 1040 insertions(+), 182 deletions(-) diff --git a/wai/changes/2026-11-06-quantum-parse-depth.md b/wai/changes/2026-11-06-quantum-parse-depth.md index 6f5ed580c..af1d8f309 100644 --- a/wai/changes/2026-11-06-quantum-parse-depth.md +++ b/wai/changes/2026-11-06-quantum-parse-depth.md @@ -1,4 +1,4 @@ -### WAI Quantum: the reader holds its nesting, and the job service reads each request once, within one budget, off its request threads +### WAI Quantum: the reader holds its nesting and its memory, and the job service reads each request once, within one budget, in a capped child - **The reader's nesting** (`wai_quantum::qasm`). A program nested past what the reader takes is refused with the new code `Q006` rather than read by a recursion that ran a thread's stack out: 64 levels in its text @@ -20,6 +20,17 @@ `engine::answer::eval::CHAIN_DEPTH`, 32, each naming the next). The exact expectation values of a program that measures in the middle walk its branches with a list rather than a recursion, in the same order, so a program that branches thousands of times is refused at `MAX_LEAVES`, not run out of stack. +- **The reader's memory** (`Q130`, new). A program whose reading would hold more than the reader takes is + refused before it is made: one classical value (an array or a bit register, each element counted at its width, + so `array[bit[1048576], 1000]` is a gigabyte) of more than 64 MiB, or more than 512 MiB in all of its values and + of its gates, steps, measurements and timing as they grow (a `reset` of a wide register in a loop grew them + without end). A declaration of a hundred bytes, `array[int[8], 4294967295] a;`, asked for 137 GB, and an + allocation that fails ends the process. Making or copying a value counts toward the reader's steps, at one + for every 128 bytes; an element read from an array copies the element alone; a loop over a range makes each + value as it reaches it, not all of them first. A statement on a register of tens of thousands of qubits is + checked in its own length, where each qubit was compared with every one before it (a loop of resets on such a + register ran for minutes within a few hundred steps). A bit string wider than an integer, given to one, is + refused (`Q112`) where the reader panicked. - **A budget of reading** (`qasm::within_steps`, `qasm::steps_left`): every program read on a thread within it is read once (a program read again with no inputs is answered from its first reading) and every reading draws on one budget of steps: each statement elaborated, each gate written, each pass of a loop a generator @@ -32,10 +43,20 @@ `GET /v1`'s limits): a request whose references or items take more is refused, `422` and `J031`. A check (`/v1/critique` and the others) is counted before its bounds read anything, and its bounds read within the same budget. -- **The job service: off the request threads.** Admission, every check's bounds and answer, and the verdicts - the MCP tools state beside an answer run on threads of their own, each with a 64 MiB stack, at most four - requests at once (a request that waits ten seconds for a turn is refused, `503` and `J032`); a panic there - fails its request with `500` and the service runs on. +- **The job service: in a child.** Admission, a dry run, every check's bounds and answer, the + error-correction, pulse and variational calls and the estimates, and the verdicts the MCP tools state beside + an answer are read by a child, the service's own binary run as `wai-quantum-service read`, sandboxed as a + worker is, at most four at once (a request that waits ten seconds for a turn is refused, `503` and `J032`). A + child past `WAI_QSVC_READ_GB` of memory (4) or `WAI_QSVC_READ_SECS` (300) is stopped and its request refused, + `422` and `J031`; one that panics, overflows or aborts is answered `500`; the service runs on. An + error-correction decode's circuit text, which no bound held, is read within those caps. A run's MCP arguments + that are not an object are refused, where they dropped the connection. +- **The job service: a retry costs nothing.** What refuses whatever is sent (the client's jobs at once, the + memory they hold, a full queue) is checked before the submission is counted, and a refusal after the count + that tells the client to send it again (`429` for its memory, `503`) is given back its count, so obeying + `Retry-After` never spends a client's window. A job given in another format under a key that names one of the + client's jobs is read from that format before it is counted, so a send after a lost answer gets its job even + past the window. - **Tests**: every nesting construct at thousands of levels on a 2 MiB thread, refused; a seeded property test over mixed nesting; the service binary started in a test and sent programs nested thousands deep through each submission, the MCP run tool and task, the checks and an diff --git a/wai/wai-quantum-cli/docs/llms-full.txt b/wai/wai-quantum-cli/docs/llms-full.txt index e5bc4ba01..9764e54aa 100644 --- a/wai/wai-quantum-cli/docs/llms-full.txt +++ b/wai/wai-quantum-cli/docs/llms-full.txt @@ -990,6 +990,7 @@ Each construct below is supported (read as the language defines it), lowered (re | Statements and blocks `{ }`, scopes | supported; qubits, gates, subroutines and externs are declared at the top level | Q105 | | Nesting: statements in blocks and bodies, parentheses, operators, brackets, calls, casts and types | supported to 64 levels together in the text (a statement, each enclosing statement, and each level of its expressions' trees count one; a long `else if` chain is one level a branch, and reads as a `switch` at any length); deeper is refused, never read by a recursion that could run out of stack | Q006 | | Nesting in all, through the gates and subroutines a program calls | supported to 256 levels (each statement, expression and call the reader runs counts one), and calls 32 deep; deeper is refused | Q006, Q209 | +| What a program holds on reading: its classical values, gates, steps, measurements and timing | supported to 64 MiB for one value (an array or a bit register, each element counted at its width) and 512 MiB in all; more is refused before it is made. Making or copying a value counts toward the reader's 20 000 000 steps at one step for every 128 bytes, and an element read from an array copies the element alone | Q130, Q312 | | `pragma …`, `#pragma …`, annotations `@keyword …` | warned: kept, not applied; `@wai.max_iterations N` bounds the `while` after it | Q013 | #### Literals @@ -1230,6 +1231,7 @@ Every code: | `Q127` | refused | An assignment to a constant, an input or a loop variable. | | `Q128` | refused | A measurement where a value is needed that is not a bit (measurement gives bits, into a bit register). | | `Q129` | refused | A parameter (`input float`, `input angle`) a gradient cannot follow to the gates: under a modifier, in a classical variable, a condition, an index, a size, a loop's range, a subroutine or `gphase`, or one whose type rounds it (`angle[n]`, `float[32]`); a gradient follows a parameter through arithmetic, the built-in functions and gate definitions into a gate's arguments. | +| `Q130` | refused | A program whose reading would hold more memory than the reader takes: a classical value (an array, a bit register) of more than 64 MiB, each element counted at its width, or more than 512 MiB in all of its values, gates, steps, measurements and timing; refused before it is made. | | `Q200` | lowered | A `gate` the program defines, inlined at each call with its parameters bound. | | `Q201` | lowered | `ctrl @` and `negctrl @` on any gate, lowered exactly to standard gates (a negative control is a control between two `x`). | | `Q202` | lowered | `inv @` on any gate, user gates included: each gate's exact inverse, in reverse order. | diff --git a/wai/wai-quantum-service/README.md b/wai/wai-quantum-service/README.md index c9d1d7069..d3a7ca9d9 100644 --- a/wai/wai-quantum-service/README.md +++ b/wai/wai-quantum-service/README.md @@ -183,17 +183,16 @@ A job is `queued`, `running`, `done`, `failed`, `cancelled`, or `interrupted`: t Every answer carries the client's submission window in the rate-limit fields of the IETF HTTP API working group: `RateLimit-Policy: "submissions";q=30;w=600` (at most 30 submissions in 600 s) and `RateLimit: "submissions";r=12;t=34` (12 left, one more in 34 s). A refusal past the limits carries `Retry-After`. -A submission is admitted in the order of what each step costs, and every step after the count counts, refused or not: +A submission is admitted in the order of what each step costs: -1. A job sent again under its `Idempotency-Key` is answered with the key's job, and counts as nothing. -2. The submission is counted against the client's window. -3. What reads nothing: every whole number exact, at most 16 intents (each of a set's items too), at most 100 items in a set. -4. Its programs are read: the job's, a set's items and a sweep's points, and every intent's reference, each once however often admission reads it, and all of them within one budget of 20,000,000 steps (statements the reader runs, gates it writes, a generator's loop passes): as many as one program may take. A request whose programs take more is refused, `422` and `J031`, with the command that runs it on your own machine. -5. The job is kept and queued, within the client's and the queue's limits. +1. A job sent again under its `Idempotency-Key` is answered with the key's job, and counts as nothing. A job given in another format (`"source"`) under a key that names one of your jobs is read from that format first, without counting, so a send after a lost answer gets its job even past the window; another request under the key counts. +2. What reads nothing and refuses whatever is sent: your jobs at once (`429`), the memory they hold (`429`), a full queue (`503`). These are not counted, so obeying `Retry-After` spends nothing. +3. The submission is counted against your window. Every refusal after this counts, except one that tells you to send it again (`429` for your memory, `503`), which is given back its count. +4. What reads nothing: every whole number exact, at most 16 intents (each of a set's items too), at most 100 items in a set. +5. Its programs are read: the job's, a set's items and a sweep's points, and every intent's reference, each once however often admission reads it, and all of them within one budget of 20,000,000 steps (statements the reader runs, gates it writes, a generator's loop passes, and every 128 bytes of a value it makes or copies): as many as one program may take. A request whose programs take more is refused, `422` and `J031`, with the command that runs it on your own machine. +6. The job is kept and queued, within your and the queue's limits. -A job given in another format (`"source"`) is counted before it is read from that format, since that reading is the costly part; one that turns out to be sent again under its key is given back its count. - -Reading happens off the service's request threads, at most four requests at a time, each on a thread of its own with a 64 MiB stack: a request that waits ten seconds for its turn is refused (`503`, `J032`), and one whose reading fails inside the service is answered `500` while the service runs on. The reader refuses programs nested past its limits (`Q006`: 64 levels in a program's text, 256 in all through the gates and subroutines it calls), so no program reaches that stack. +Reading happens outside the service's own process, as each job's run does: each request is read by a child, this binary run as `wai-quantum-service read`, sandboxed as a worker is, at most four at a time (a request that waits ten seconds for its turn is refused, `503`, `J032`). A child that holds more than `WAI_QSVC_READ_GB` of memory, or reads for longer than `WAI_QSVC_READ_SECS`, is stopped and the request refused, `422`, `J031`; one that fails is answered `500`. Either way the service runs on. The checks, the error-correction, pulse and variational calls, the estimates, and the verdicts the MCP tools state are read the same way. The reader itself refuses a program nested past its limits (`Q006`: 64 levels in a program's text, 256 in all through the gates and subroutines it calls) and one that would hold more than it takes (`Q130`: a value of more than 64 MiB, or 512 MiB in all), before it is made. ## Sending a job again @@ -217,7 +216,7 @@ Every job is in the service's store before its submission is answered, and again Finished jobs are kept for `WAI_QSVC_RETAIN_SECS` (a day here), then dropped. A record names its client by a keyed hash of the address, never the address. [docs/self-host.md](docs/self-host.md#the-job-store) describes the store. -The six checks answer at once, in the service's own process. Each counts against the client's submissions, as a job does, before it reads anything; then its bounds read its programs within one budget, as a job's admission does, off the request threads. At most two run at a time, and a check that waits ten seconds for its turn is refused (`503`). Some checks are refused (`422`) with the command that runs them on your own machine: +The six checks answer at once, each read and answered in a reading child as a job's admission is. Each counts against the client's submissions, as a job does, before it reads anything (a check refused because every check is running, `503`, is given back its count); then its bounds read its programs within one budget. At most two run at a time, and a check that waits ten seconds for its turn is refused (`503`). Some checks are refused (`422`) with the command that runs them on your own machine: - an equivalence check that would apply more than 4·10⁹ gate amplitudes; - an improvement whose candidates' checks would apply more than that; @@ -329,6 +328,8 @@ These variables set the limits: | `WAI_QSVC_CLIENT_GB` | the job size | memory one client's queued and running jobs may hold together | | `WAI_QSVC_CLIENT_BURST` | 30 | submissions one client may make per window | | `WAI_QSVC_CLIENT_WINDOW_SECS` | 600 | the window | +| `WAI_QSVC_READ_GB` | 4 | the memory one request's reading may hold, in its child | +| `WAI_QSVC_READ_SECS` | 300 | the time one request's reading may take | | `WAI_QSVC_MAX_QUEUE` | 256 | jobs waiting at once | | `WAI_QSVC_RETAIN_SECS` | 86400 | how long finished jobs are kept, across restarts | | `WAI_QSVC_STORE` | `~/.local/share/wai-quantum-service/store` | the job store's directory, or `memory` | diff --git a/wai/wai-quantum-service/src/analysis.rs b/wai/wai-quantum-service/src/analysis.rs index 394bb0f99..4fa8d0010 100644 --- a/wai/wai-quantum-service/src/analysis.rs +++ b/wai/wai-quantum-service/src/analysis.rs @@ -1,8 +1,9 @@ //! Design checks the engine answers at once, without a job: a program's //! critique, two programs' equivalence, the improvements its fixes make, its //! classical shortcuts and its compile (`wai_quantum::engine::{critique, -//! equiv, shortcuts, compile}`). They run in this process, off the request's -//! thread, within bounds: +//! equiv, shortcuts, compile}`). Each is read and answered by a reading +//! child ([`crate::offload`]), whose memory and time are capped, within +//! bounds: //! //! - each counts against the client's submissions, as a job does; //! - at most [`AT_ONCE`] run at a time, and a request that waits [`WAIT`] for @@ -34,10 +35,10 @@ //! service takes, J032 when every check is running. //! //! **In the order of what each step costs.** A check is counted against the -//! client's submissions first, so every check refused after that counts; -//! then, off the runtime on a thread of its own ([`crate::offload`]), its -//! bounds read its programs, each once, within one budget of reading -//! ([`crate::http::READ_STEPS`]), and only then is it answered. +//! client's submissions first, so every check refused after that counts +//! (but for one told to try again, `503`, given back its count); then, in its +//! reading child, its bounds read its programs, each once, within one budget +//! of reading ([`crate::http::READ_STEPS`]), and only then is it answered. //! //! The critique and the quick shortcuts are bounded by the program's size. //! The tableau, the cut-rank planner's search and the light cones are linear @@ -89,6 +90,27 @@ pub enum Check { Convert, } +impl Check { + const ALL: [(Check, &'static str); 7] = [ + (Check::Critique, "critique"), + (Check::Equivalent, "equivalent"), + (Check::Improve, "improve"), + (Check::Shortcuts, "shortcuts"), + (Check::Compile, "compile"), + (Check::Repair, "repair"), + (Check::Convert, "convert"), + ]; + + /// The check's name, as a reading child is given it. + pub fn name(self) -> &'static str { + Self::ALL.iter().find(|(c, _)| *c == self).map(|(_, n)| *n).expect("every check is named") + } + + pub fn named(name: &str) -> Option { + Self::ALL.iter().find(|(_, n)| *n == name).map(|(c, _)| *c) + } +} + fn program<'a>(body: &'a Value, key: &str) -> Result<&'a str, (u16, String)> { body.get(key).and_then(Value::as_str).ok_or_else(|| (400, format!("give \"{key}\": an OpenQASM program"))) } @@ -324,26 +346,62 @@ pub fn variational_bounds(op: &str, body: &Value) -> Result<(), (u16, String)> { Ok(()) } -/// `work` for `who` within the bounds above: counted first, then, one of -/// [`AT_ONCE`] on a thread of its own ([`crate::offload`]), held to `check` -/// (which reads within one budget, [`crate::http::READ_STEPS`]) and run: its -/// answer, or 422 with its refusal. -pub async fn bounded(app: &App, who: &str, check: C, work: F) -> Result -where - C: FnOnce() -> Result<(), (u16, String)> + Send + 'static, - F: FnOnce() -> Result + Send + 'static, -{ - app.service.count(who).map_err(|crate::jobs::Refused(code, why)| (code, why))?; - let turn = turn().await?; - crate::offload::on_thread(move || { - let _turn = turn; - crate::http::within_budget(check)?; - work().map_err(|e| (422, e)) - }) - .await? +/// The engine call `kind` (`estimate`, `estimate_verify`, `qec`, `pulse`, +/// `variational`) on `body`, held to its bounds first, within one budget of +/// reading: its answer, or the refusal. What a reading child runs; `key` is +/// the receipt key's secret in hex, for an estimate, which seals as it goes. +pub fn call(kind: &str, op: &str, body: &Value, key: Option<&str>) -> Result { + let engine = |r: Result| r.map_err(|e| (422, e)); + match kind { + "estimate" => { + crate::http::within_budget(|| estimate_bounds(body))?; + let hex = key.ok_or((500, "an estimate is sealed, and no key was given".to_string()))?; + let bytes: Vec = (0..hex.len()).step_by(2).filter_map(|i| hex.get(i..i + 2).and_then(|b| u8::from_str_radix(b, 16).ok())).collect(); + let secret: [u8; 32] = bytes.try_into().map_err(|_| (500, "the receipt key did not read".to_string()))?; + engine(wai_quantum::engine::estimate::sealed(body, &wai_quantum::ed25519_dalek::SigningKey::from_bytes(&secret))) + } + "estimate_verify" => { + crate::http::within_budget(|| estimate_bounds(&body["request"]))?; + engine(wai_quantum::engine::estimate::verify(body)) + } + "qec" => { + let op = ["code", "decode", "threshold"].into_iter().find(|o| *o == op).ok_or((400, format!("no error-correction call {op}")))?; + qec_bounds(op, body)?; + engine(wai_quantum::engine::qec::call(op, body)) + } + "pulse" => { + let op = ["schedule", "check"].into_iter().find(|o| *o == op).ok_or((400, format!("no pulse call {op}")))?; + crate::http::within_budget(|| pulse_bounds(op, body))?; + engine(wai_quantum::engine::pulse::call(op, body)) + } + "variational" => { + let op = ["gradient", "optimize", "vqe", "qaoa", "build"].into_iter().find(|o| *o == op).ok_or((400, format!("no variational call {op}")))?; + crate::http::within_budget(|| variational_bounds(op, body))?; + engine(wai_quantum::engine::variational::call(op, body)) + } + other => Err((400, format!("no engine call {other}"))), + } } -/// A turn among the [`AT_ONCE`] checks, held by the thread that runs one. +/// The engine call `read` for `who` within the bounds above: counted first, +/// then, one of [`AT_ONCE`], held to its bounds and run in a reading child +/// whose memory and time are capped ([`crate::offload`]): its answer, or the +/// refusal. A refusal that tells the client to try again (`503`) is given +/// back its count. +pub async fn bounded(app: &App, who: &str, read: crate::offload::Read) -> Result { + app.service.count(who).map_err(|crate::jobs::Refused(code, why)| (code, why))?; + let r = async { + let _turn = turn().await?; + crate::offload::read(&app.service.config, read).await + } + .await; + if matches!(r, Err((503, _))) { + app.service.refund(who); + } + r +} + +/// A turn among the [`AT_ONCE`] checks, held while one runs. async fn turn() -> Result { match tokio::time::timeout(WAIT, Arc::clone(&TURNS).acquire_owned()).await { Ok(Ok(t)) => Ok(t), @@ -475,17 +533,10 @@ pub fn answer(check: Check, body: &Value) -> Value { /// `check` on `body` for `who`, within the bounds above: the answer, or the /// status and reason it was refused. -/// It is counted first, then held to its bounds and answered off the -/// runtime ([`bounded`]). +/// It is counted first, then held to its bounds and answered in a reading +/// child ([`bounded`]). pub async fn analyse(app: &App, who: &str, check: Check, body: Value) -> Result { - app.service.count(who).map_err(|crate::jobs::Refused(code, why)| (code, why))?; - let turn = turn().await?; - crate::offload::on_thread(move || { - let _turn = turn; - crate::http::within_budget(|| bounds(check, &body))?; - Ok(answer(check, &body)) - }) - .await? + bounded(app, who, crate::offload::Read::Check { check: check.name().into(), body }).await } #[cfg(test)] diff --git a/wai/wai-quantum-service/src/http.rs b/wai/wai-quantum-service/src/http.rs index 708188e4e..ca380fe12 100644 --- a/wai/wai-quantum-service/src/http.rs +++ b/wai/wai-quantum-service/src/http.rs @@ -334,9 +334,8 @@ pub(crate) fn within_budget(read: impl FnOnce() -> Result) /// held to a job's bounds), or the refusal's status and reason. The checks /// that read nothing come first; then every program the request gives is /// read, once, within one budget ([`READ_STEPS`]). -pub(crate) fn plan_job(service: &Service, request: &Value) -> Result<(catalog::Plan, Work), (u16, String)> { +pub(crate) fn plan_job(cap: u64, request: &Value) -> Result<(catalog::Plan, Work), (u16, String)> { cheap_bounds(request)?; - let cap = service.config.job_mib << 20; within_budget(|| { if catalog::is_set(request) { let items = sets::expand(request).map_err(|e| (400, e))?; @@ -351,40 +350,80 @@ pub(crate) fn plan_job(service: &Service, request: &Value) -> Result<(catalog::P }) } -/// Plans and admits `request` for `who`, off the runtime -/// ([`crate::offload`]): the job's id, or the refusal's status and reason. +/// Plans and admits `request` for `who`: the job's id, or the refusal's +/// status and reason ([`admit_to`]). pub(crate) async fn admit(app: &App, who: &str, request: Value, key: Option) -> Result { - let service = Arc::clone(&app.service); - let who = who.to_string(); - crate::offload::run(move || admit_now(&service, &who, request, key.as_deref().map(Key))).await? + admit_to(&app.service, who, request, key.as_deref()).await } -/// [`admit`], on this thread, in the order of what each step costs: +/// A job's reading as its child gives it back ([`crate::offload::Read::Admit`]): +/// the request as read (from its source, where it has one), and its plan and +/// work, or the refusal its planning gave. +type Admitted = (Value, Result<(catalog::Plan, Work), (u16, String)>); + +fn admitted(v: Value) -> Result { + let bad = |e: serde_json::Error| (500, format!("the reading's answer did not read: {e}")); + let request = v["request"].clone(); + if let Some(r) = v["refused"].as_array() { + return Ok((request, Err((r[0].as_u64().unwrap_or(400) as u16, r[1].as_str().unwrap_or("refused").to_string())))); + } + let plan: catalog::Plan = serde_json::from_value(v["plan"].clone()).map_err(bad)?; + let work = match v["items"].as_array() { + None => Work::One, + Some(items) => Work::Set { + items: items.iter().map(|i| sets::Item { request: i["request"].clone(), point: Some(i["point"].clone()).filter(|p| !p.is_null()) }).collect(), + plans: serde_json::from_value(v["plans"].clone()).map_err(bad)?, + }, + }; + Ok((request, Ok((plan, work)))) +} + +/// [`admit`], in the order of what each step costs: /// /// 1. a request sent again with its key is the key's job, whatever planning -/// it now would give, and counts as nothing; -/// 2. the submission is counted against the client's window, so every -/// request refused after this counts; -/// 3. the checks that read nothing ([`cheap_bounds`]); -/// 4. its programs read, each once, within one budget ([`plan_job`]); -/// 5. the job kept and queued, within the client's and the queue's limits. -/// -/// A job given in another format (`"source"`) is counted before it is read -/// from that format, since that reading is the costly part; one that turns -/// out to be sent again under its key is then given back its count. -pub(crate) fn admit_now(service: &Service, who: &str, request: Value, key: Option) -> Result { +/// it now would give, and counts as nothing; one given in another format +/// (`"source"`) under a key that names a job of the client's is read from +/// that format first, uncounted, and another request under the key counts; +/// 2. what reads nothing and refuses whatever is sent (the client's jobs at +/// once, a full queue) refuses uncounted, so a client told to try again +/// shortly spends nothing doing so; +/// 3. the submission is counted against the client's window, so every +/// request refused after this counts, but for one it is told to send again +/// (`429` for its memory, `503`), which is given back its count; +/// 4. the checks that read nothing ([`cheap_bounds`]); +/// 5. its programs read, each once, within one budget ([`plan_job`]), in a +/// child whose memory and time are capped ([`crate::offload`]); +/// 6. the job kept and queued, within the client's and the queue's limits. +pub(crate) async fn admit_to(service: &Service, who: &str, request: Value, key: Option<&str>) -> Result { let refused = |Refused(code, why)| (code, why); + let key = key.map(Key); + let job_bytes = service.config.job_mib << 20; let sourced = request.get("source").is_some_and(|s| !s.is_null()); - if !sourced - && let Some(k) = key - && let Some(id) = service.named(who, k, &request).map_err(refused)? - { - return Ok(id); + if let Some(k) = key { + if !sourced { + if let Some(id) = service.named(who, k, &request).map_err(refused)? { + return Ok(id); + } + } else if service.key_live(who, k) { + let read = crate::offload::read(&service.config, crate::offload::Read::Admit { request: request.clone(), job_bytes, plan: false }).await?; + match service.named(who, k, &read["request"]) { + Ok(Some(id)) => return Ok(id), + Ok(None) => {} + Err(r) => { + let _ = service.count(who); + return Err(refused(r)); + } + } + } + } + if let Some(r) = service.refused_before_reading(who) { + return Err(refused(r)); } service.count(who).map_err(refused)?; - // One budget for the whole request: its source's reading and its plan's. - within_budget(|| { - let request = if sourced { with_source(request)? } else { request }; + let r = async { + cheap_bounds(&request)?; + let read = crate::offload::read(&service.config, crate::offload::Read::Admit { request, job_bytes, plan: true }).await?; + let (request, planned) = admitted(read)?; if sourced && let Some(k) = key && let Some(id) = service.named(who, k, &request).map_err(refused)? @@ -392,14 +431,19 @@ pub(crate) fn admit_now(service: &Service, who: &str, request: Value, key: Optio service.refund(who); return Ok(id); } - let (plan, work) = plan_job(service, &request)?; + let (plan, work) = planned?; let (id, new) = service.submit_counted(who, request, plan, work, key).map_err(refused)?; if !new { // Sent at once with another under its key, which made the job. service.refund(who); } Ok(id) - }) + } + .await; + if let Err((429 | 503, _)) = r { + service.refund(who); + } + r } /// The `Idempotency-Key` a request gives, if it gives one: a structured @@ -431,22 +475,27 @@ pub(crate) fn is_dry_run(request: &Value) -> bool { /// estimated time and energy (each with its basis), the bounds that apply, /// where the client stands, and whether the service would take it now, or /// the refusal it would give. It counts as one submission, as a check does, -/// before anything is read; it is read as a job is ([`admit_now`]), off the -/// runtime. +/// before anything is read; it is read as a job is ([`admit_to`]), in a child. pub(crate) async fn dry_run(app: &App, who: &str, request: &Value) -> Result { - let service = Arc::clone(&app.service); - let (who, request) = (who.to_string(), request.clone()); - crate::offload::run(move || { - service.count(&who).map_err(|Refused(code, why)| (code, why))?; - within_budget(|| dry_run_now(&service, &who, &request)) - }) - .await? + let service = &app.service; + service.count(who).map_err(|Refused(code, why)| (code, why))?; + let read = crate::offload::read(&service.config, crate::offload::Read::Admit { request: request.clone(), job_bytes: service.config.job_mib << 20, plan: true }).await; + let read = match read.and_then(admitted) { + Ok(r) => r, + Err(e) => { + if matches!(e.0, 429 | 503) { + service.refund(who); + } + return Err(e); + } + }; + dry_run_now(service, who, read) } -/// [`dry_run`], here, its submission counted. -fn dry_run_now(service: &Service, who: &str, request: &Value) -> Result { - let expanded = with_source(request.clone())?; +/// [`dry_run`]'s answer, its submission counted and its request read. +fn dry_run_now(service: &Service, who: &str, (expanded, planned): Admitted) -> Result { let request = &expanded; + let _ = request; let window = service.window(who); let (active, held) = service.holding(who); let mut out = json!({ @@ -455,7 +504,7 @@ fn dry_run_now(service: &Service, who: &str, request: &Value) -> Result p, Err((status, why)) => { out["accepted"] = json!(false); @@ -678,8 +727,7 @@ async fn verify(State(app): State, ConnectInfo(peer): ConnectInfo reply(200, checked), Err((code, why)) => error(code, &why), }; @@ -733,9 +781,8 @@ pub(crate) async fn estimate(app: &App, who: &str, request: Value) -> Result Router { #[cfg(test)] mod tests { - use super::{admit_now, identity, idempotency_key}; - use crate::jobs::{Config, Key, Service}; + use super::{admit_to, identity, idempotency_key}; + use crate::jobs::{Config, Service}; use axum::http::{HeaderMap, HeaderValue}; use serde_json::json; use std::net::IpAddr; @@ -1026,6 +1070,9 @@ mod tests { sandbox: true, per_client_burst: burst, per_client_window: Duration::from_secs(600), + read_mib: 4096, + read_timeout: Duration::from_secs(300), + read_in_child: false, }) } @@ -1034,37 +1081,64 @@ mod tests { /// or its references too long to read; a job sent again under its key /// counts as nothing; and each reference is read once, however often /// the request names it and however often its admission reads it. - #[test] - fn a_refused_job_counts_and_each_reference_is_read_once_within_one_budget() { + #[tokio::test] + async fn a_refused_job_counts_and_each_reference_is_read_once_within_one_budget() { let s = service(100); let bell = "OPENQASM 3.0;\nqubit[1] q;\nbit[1] c;\nh q[0];\nc = measure q;\n"; let deep = format!("OPENQASM 3.0;\nqubit[1] q;\nrz({}0.5{}) q[0];\n", "(".repeat(5_000), ")".repeat(5_000)); let left = |s: &Service| s.window("a").remaining; - let (code, why) = admit_now(&s, "a", json!({ "qasm": deep }), None).unwrap_err(); + let (code, why) = admit_to(&s, "a", json!({ "qasm": deep }), None).await.unwrap_err(); assert!(code == 400 && why.ends_with("[Q006]"), "{code} {why}"); assert_eq!(left(&s), 99); let many: Vec<_> = (0..17).map(|_| json!({ "equivalent": bell })).collect(); - let (code, why) = admit_now(&s, "a", json!({ "qasm": bell, "intent": many }), None).unwrap_err(); + let (code, why) = admit_to(&s, "a", json!({ "qasm": bell, "intent": many }), None).await.unwrap_err(); assert!(code == 400 && why.ends_with("[J020]"), "{code} {why}"); assert_eq!(left(&s), 98); - // Sixteen intents naming one reference that loops four million + // Sixteen intents naming one reference that loops two million // times: read once, it is within one budget; read for each intent // and each step of admission, it was far past it. let heavy = "OPENQASM 3.0;\nqubit[1] q;\nbit[1] c;\nint a = 0;\nfor int i in [0:1999999] { a += 1; }\nh q[0];\nc = measure q;\n"; let same: Vec<_> = (0..16).map(|_| json!({ "equivalent": heavy })).collect(); let job = json!({ "qasm": bell, "shots": 10, "intent": same }); - let id = admit_now(&s, "a", job.clone(), Some(Key("k"))).unwrap(); + let id = admit_to(&s, "a", job.clone(), Some("k")).await.unwrap(); assert_eq!(left(&s), 97); // Sent again under its key: the same job, counted as nothing. - assert_eq!(admit_now(&s, "a", job, Some(Key("k"))).unwrap(), id); + assert_eq!(admit_to(&s, "a", job, Some("k")).await.unwrap(), id); assert_eq!(left(&s), 97); // Sixteen different ones are not: refused, as too large here, and counted. let each: Vec<_> = (0..16).map(|k| json!({ "equivalent": format!("{heavy}// {k}\n") })).collect(); - let (code, why) = admit_now(&s, "a", json!({ "qasm": bell, "shots": 10, "intent": each }), None).unwrap_err(); + let (code, why) = admit_to(&s, "a", json!({ "qasm": bell, "shots": 10, "intent": each }), None).await.unwrap_err(); assert!(code == 422 && why.ends_with("[J031]") && why.contains("steps"), "{code} {why}"); assert_eq!(left(&s), 96); } + /// A client told to try again shortly spends nothing doing so: refused + /// for its jobs at once before anything is counted or read, its window + /// stays whole; and a job sent again under its key, once its window is + /// spent, is still its job, as an honest retry after a lost answer is. + #[tokio::test] + async fn a_client_told_to_retry_spends_nothing_and_a_key_s_job_comes_back_past_the_window() { + let s = Service::new(Config { per_client_active: 1, ..service(3).config.clone() }); + let bell = |k: u32| json!({ "qasm": format!("OPENQASM 3.0;\nqubit[1] q;\nbit[1] c;\nh q[0];\nc = measure q;\n// {k}\n"), "shots": 10 }); + let id = admit_to(&s, "a", bell(0), Some("lost")).await.unwrap(); + assert_eq!(s.window("a").remaining, 2); + for k in 1..20 { + let (code, why) = admit_to(&s, "a", bell(k), None).await.unwrap_err(); + assert!(code == 429 && why.contains("wait for one to finish"), "{code} {why}"); + } + assert_eq!(s.window("a").remaining, 2, "a refusal it was told to retry spent the window"); + // Its window spent by refusals that do count (programs that do not read): + let broken = json!({ "qasm": "OPENQASM 3.0;\nqubit q;\nfoo q;\n" }); + assert_eq!(s.cancel(&id), Some(true)); + for _ in 0..2 { + assert_eq!(admit_to(&s, "a", broken.clone(), None).await.unwrap_err().0, 400); + } + assert_eq!(s.window("a").remaining, 0); + assert_eq!(admit_to(&s, "a", bell(99), None).await.unwrap_err().0, 429); + // and the job its key named still comes back for it. + assert_eq!(admit_to(&s, "a", bell(0), Some("lost")).await.unwrap(), id); + } + #[test] fn an_idempotency_key_reads_as_a_structured_string_or_bare() { let read = |v: &str| { diff --git a/wai/wai-quantum-service/src/jobs.rs b/wai/wai-quantum-service/src/jobs.rs index 784549965..d62d8ee4d 100644 --- a/wai/wai-quantum-service/src/jobs.rs +++ b/wai/wai-quantum-service/src/jobs.rs @@ -95,6 +95,13 @@ pub struct Config { /// Submissions one client may make in a window. pub per_client_burst: usize, pub per_client_window: Duration, + /// The memory, MiB, and the time one request's reading may take + /// ([`crate::offload`]): the cap on its reading child. + pub read_mib: u64, + pub read_timeout: Duration, + /// Whether requests are read in a child (the service), or on a thread of + /// the process (the service's own tests, whose binary is not the service). + pub read_in_child: bool, } fn env_num(key: &str, default: T) -> T { @@ -107,7 +114,8 @@ impl Config { /// `WAI_QSVC_RETAIN_SECS` (86400), `WAI_QSVC_MAX_QUEUE` (256), /// `WAI_QSVC_CLIENT_ACTIVE` (3), `WAI_QSVC_CLIENT_GB` (the job size), /// `WAI_QSVC_CLIENT_BURST` (30 in `WAI_QSVC_CLIENT_WINDOW_SECS`, 600), - /// `WAI_QSVC_SANDBOX` (1; 0 runs workers unsandboxed). + /// `WAI_QSVC_SANDBOX` (1; 0 runs workers unsandboxed), `WAI_QSVC_READ_GB` + /// (4) and `WAI_QSVC_READ_SECS` (300), what one request's reading may take. pub fn from_env() -> Config { let job_gb = env_num("WAI_QSVC_JOB_GB", 72u64); Config { @@ -123,6 +131,9 @@ impl Config { sandbox: env_num("WAI_QSVC_SANDBOX", 1u8) != 0, per_client_burst: env_num("WAI_QSVC_CLIENT_BURST", 30usize), per_client_window: Duration::from_secs(env_num("WAI_QSVC_CLIENT_WINDOW_SECS", 600u64)), + read_mib: (env_num("WAI_QSVC_READ_GB", 4f64).max(0.0) * 1024.0) as u64, + read_timeout: Duration::from_secs(env_num("WAI_QSVC_READ_SECS", 300u64)), + read_in_child: !cfg!(test), } } } @@ -665,6 +676,32 @@ impl Service { (mine.len(), mine.iter().map(|j| j.plan.memory_bytes.div_ceil(1 << 20)).sum()) } + /// Whether `client` is refused whatever it sends, by what reads nothing: + /// its jobs at once, or a full queue. Checked before a submission is + /// counted, so a client told to try again shortly spends nothing doing so. + pub fn refused_before_reading(&self, client: &str) -> Option { + let (active, held) = self.held(&self.name(client)); + if active >= self.config.per_client_active { + return Some(Refused(429, tag(refusal::RATE, format!("{active} of your jobs are queued or running; wait for one to finish (at most {} at once)", self.config.per_client_active)))); + } + if held >= self.config.per_client_mib { + return Some(Refused(429, tag(refusal::RATE, format!("your queued and running jobs already hold about {held} MiB, all one client may hold at once; wait for one to finish")))); + } + let queued = self.queue.lock().unwrap().len(); + if queued >= self.config.max_queue { + return Some(Refused(503, tag(refusal::BUSY, format!("the queue is full ({} jobs); try again later", self.config.max_queue)))); + } + None + } + + /// Whether `key` names a job of `client`'s that still answers for it: + /// a send under it may be the same job sent again. + pub fn key_live(&self, client: &str, key: Key) -> bool { + let digest = Self::key_digest(&self.name(client), key); + let Some(id) = self.keys.lock().unwrap().get(&digest).cloned() else { return false }; + self.jobs.lock().unwrap().get(&id).is_some_and(|j| j.status.holds_key()) + } + /// Whether `client` may submit a job planned as `plan` now, by every /// limit but its submission window: the refusal it would get, or nothing. pub fn would_refuse(&self, client: &str, plan: &Plan) -> Option { @@ -1285,6 +1322,22 @@ struct Ran { } /// The worker command at a lower priority than the host's other work. +/// This binary run in `mode` (`read`, a request's reading), in the sandbox a +/// worker runs in where `sandbox` says so, at the service's own priority: +/// its client waits for it. +pub(crate) fn sandboxed(exe: &std::path::Path, sandbox: bool, mode: &str) -> tokio::process::Command { + let mut cmd = if cfg!(target_os = "macos") && sandbox { + let exe = std::fs::canonicalize(exe).unwrap_or_else(|_| exe.to_path_buf()); + let mut c = tokio::process::Command::new("/usr/bin/sandbox-exec"); + c.arg("-p").arg(worker_profile(&exe)).arg(exe); + c + } else { + tokio::process::Command::new(exe) + }; + cmd.arg(mode); + cmd +} + fn low_priority(exe: &std::path::Path, sandbox: bool) -> tokio::process::Command { let mut cmd = if cfg!(target_os = "macos") { let mut c = tokio::process::Command::new("/usr/sbin/taskpolicy"); @@ -1321,7 +1374,7 @@ pub fn worker_profile(exe: &std::path::Path) -> String { ) } -fn kill(pid: u32) { +pub(crate) fn kill(pid: u32) { if pid > 0 { let _ = std::process::Command::new("kill").args(["-9", &pid.to_string()]).status(); } @@ -1329,7 +1382,7 @@ fn kill(pid: u32) { /// A process's resident memory, bytes, the worker's child included where the /// platform folds it in. -fn rss_bytes(pid: u32) -> Option { +pub(crate) fn rss_bytes(pid: u32) -> Option { if pid == 0 { return None; } @@ -1402,6 +1455,9 @@ mod tests { sandbox: true, per_client_burst: 3, per_client_window: Duration::from_secs(60), + read_mib: 4096, + read_timeout: Duration::from_secs(300), + read_in_child: false, } } diff --git a/wai/wai-quantum-service/src/main.rs b/wai/wai-quantum-service/src/main.rs index 16fd307b8..20e770384 100644 --- a/wai/wai-quantum-service/src/main.rs +++ b/wai/wai-quantum-service/src/main.rs @@ -11,6 +11,8 @@ //! wai-quantum-service serve the HTTP API (WAI_QSVC_ADDR, 127.0.0.1:8241) //! wai-quantum-service worker one job: {"request", "plan"} (or a set's //! {"set": [...]}) on stdin, the result on stdout +//! wai-quantum-service read one request's reading (crate::offload::Read) +//! on stdin, its answer on stdout //! wai-quantum-service run FILE one job or set from a JSON file, here, printed //! wai-quantum-service rotate-key retire the receipt key and make a new one //! wai-quantum-service api-key NAME a new API key for a private deployment @@ -43,6 +45,10 @@ fn main() { let args: Vec = std::env::args().collect(); match args.get(1).map(String::as_str) { Some("worker") => worker(), + Some("read") => { + watch_parent(); + offload::child() + } Some("run") => match args.get(2) { Some(path) => run_file(path), None => usage(), diff --git a/wai/wai-quantum-service/src/mcp.rs b/wai/wai-quantum-service/src/mcp.rs index 396fcaeee..668f0a966 100644 --- a/wai/wai-quantum-service/src/mcp.rs +++ b/wai/wai-quantum-service/src/mcp.rs @@ -552,9 +552,12 @@ fn run_refusal(name: &str, args: &Value) -> Option { /// A run's job admitted as the run tools admit it, its verdict asked: its /// id, or the tool's refusal. It is admitted as `POST /v1/jobs` admits one -/// (`http::admit`), off the runtime: counted before anything is read. +/// (`http::admit`): counted before anything is read, and read in a child. async fn admit_run(app: &App, who: &str, mut args: Value, key: Option) -> Result { let refused = |why: String| tool_result(format!("refused: {why}"), json!({ "error": why }), true); + if !args.is_object() { + return Err(refused(wai_quantum::engine::refusal::tag(wai_quantum::engine::refusal::BAD_FIELD, "a run's arguments are an object: {\"qasm\": …, \"shots\": …}"))); + } // Every run an agent asks for states its verdict. if !verdict::asked(&args) { args["verdict"] = json!(true); @@ -569,12 +572,12 @@ async fn admit_run(app: &App, who: &str, mut args: Value, key: Option) - /// turn comes, the answer goes without them. async fn tool_answer(app: &App, who: &str, name: &str, args: Value, base: &str, views: bool) -> Value { let answer = call(app, who, name, args.clone(), base).await; - let (name, plain) = (name.to_string(), answer.clone()); - crate::offload::run(move || crate::http::within_budget(|| Ok(finish(&name, &args, answer, views)))).await.and_then(|r| r).unwrap_or(plain) + let plain = answer.clone(); + crate::offload::read(&app.service.config, crate::offload::Read::Finish { name: name.to_string(), args, answer, views }).await.unwrap_or(plain) } /// [`tool_answer`]'s verdicts and view added to the call's `answer`. -fn finish(name: &str, args: &Value, mut answer: Value, views: bool) -> Value { +pub(crate) fn finish(name: &str, args: &Value, mut answer: Value, views: bool) -> Value { // A set's answer, and a dry run's, state the verdict of what they read. let sc = &answer["structuredContent"]; let extra = match name { @@ -694,7 +697,8 @@ pub async fn handle(app: &App, who: &str, msg: &Value, base: &str, views: bool) Ok(job) => ok(&id, json!({ "task": task_of(&app.service.get(&job).unwrap_or(json!({ "id": job })), app.service.config.retain.as_millis() as u64) })), Err(why) => { let mut data = wai_quantum::engine::refusal::refusal_json(&why); - data["verdict"] = verdict::of_refusal(&args, &why); + // The verdict reads the program again: in a reading child. + data["verdict"] = crate::offload::read(&app.service.config, crate::offload::Read::Refusal { args: args.clone(), why: why.clone() }).await.unwrap_or(Value::Null); json!({ "jsonrpc": "2.0", "id": id, "error": { "code": -32602, "message": format!("no task was made: {why}"), "data": data } }) } } @@ -812,8 +816,7 @@ async fn call(app: &App, who: &str, name: &str, args: Value, base: &str) -> Valu Err((_, why)) => tool_result(format!("refused: {why}"), json!({ "error": why }), true), }, "verify_quantum_receipt" if args["result"]["format"] == wai_quantum::engine::estimate::FORMAT => { - let (body, request) = (args.clone(), args["request"].clone()); - match crate::analysis::bounded(app, who, move || crate::analysis::estimate_bounds(&request), move || wai_quantum::engine::estimate::verify(&body)).await { + match crate::analysis::bounded(app, who, crate::offload::Read::Call { kind: "estimate_verify".into(), op: String::new(), body: args.clone(), key: None }).await { Ok(v) => tool_result(if v["valid"] == true { "valid: the seal, every estimate's receipt, and the request re-run here all hold".into() } else { format!("not valid: signed {}, receipts {}, reproduced {}", v["signed"], v["receipts"], v["reproduced"]) }, v, false), Err((_, why)) => tool_result(format!("refused: {why}"), json!({ "error": why }), true), } @@ -1463,7 +1466,7 @@ mod tests { let bell = "OPENQASM 3.0; qubit[2] q; bit[2] c; h q[0]; cx q[0], q[1]; c = measure q;"; // The job a first call made, as the call admits it (with its verdict // asked), cancelled so that no call waits for it here. - let id = crate::http::admit_now(&app.service, "test", json!({ "qasm": bell, "shots": 20, "verdict": true }), Some(crate::jobs::Key("agent-1"))).unwrap(); + let id = rt.block_on(crate::http::admit_to(&app.service, "test", json!({ "qasm": bell, "shots": 20, "verdict": true }), Some("agent-1"))).unwrap(); assert_eq!(app.service.cancel(&id), Some(true)); let again = run(json!({ "qasm": bell, "shots": 20, "idempotency_key": "agent-1" })); assert_eq!(again["isError"], false, "{again}"); diff --git a/wai/wai-quantum-service/src/offload.rs b/wai/wai-quantum-service/src/offload.rs index f07ec1a0c..a8d214559 100644 --- a/wai/wai-quantum-service/src/offload.rs +++ b/wai/wai-quantum-service/src/offload.rs @@ -1,30 +1,42 @@ -//! Work that reads a client's programs, off the async runtime. +//! Work that reads a client's programs, out of the service's own process. //! -//! Reading a program (the reader's elaboration of its loops, a set's -//! sweep written out, a reference program's equivalence bound, a plan) is -//! bounded work, but it is work: done on the runtime's few threads it would -//! hold them, and every other request, while it ran. So every request's -//! admission (a job, a set, a dry run), every check's bounds and answer, and -//! the verdicts the MCP tools state beside an answer, run here instead: +//! Reading a program (the reader's elaboration of its loops, a set's sweep +//! written out, a reference program's equivalence bound, a plan, a check's +//! answer) is bounded work: the reader holds its nesting, its steps and its +//! memory to limits of its own. But a limit the service merely trusts is one +//! a mistake can lift, and what is at stake is every job the service holds: +//! a stack overflow or an allocation that fails ends a Rust process, and +//! neither can be caught (an allocator may not unwind, so exhaustion aborts). +//! So each request's reading runs in a child, the service's own binary run +//! as `wai-quantum-service read`, the same boundary each job runs behind: //! -//! - **On a thread of its own**, with a stack of [`STACK`]: far more than the -//! reader's own nesting limits need (`wai_quantum::qasm::parse::MAX_NESTING`, -//! and the elaborator's in all), so a program nested to them reads, and -//! anything nested past them is refused by the reader long before this -//! stack could be reached. -//! - **One of [`AT_ONCE`]**: a request waits up to [`WAIT`] for a turn, and is -//! then refused (503, `J032`). A turn is held by the thread, so a client -//! that hangs up does not free one while its reading still runs. -//! - **A panic fails the request, not the service**: it is caught on the -//! thread, and the request is answered 500 with what it said. +//! - **Its memory is capped** ([`crate::jobs::Config::read_mib`]): polled as a +//! job's is, and killed past it; the request is refused, `422` and `J031`, +//! with the command that runs it on the client's own machine. **Its time is +//! capped** the same way ([`crate::jobs::Config::read_timeout`]). +//! - **It ends alone**: a child that panics, overflows its stack or aborts is +//! answered `500`, and the service runs on. +//! - **It is sandboxed** where jobs are (macOS: no network, no file writes, +//! no further processes), and is given only what it reads ([`Read`]). +//! - **One of [`AT_ONCE`]** at a time: a request waits up to [`WAIT`] for a +//! turn, and is then refused (`503`, `J032`). A turn is held until its child +//! has ended, so a client that hangs up frees none while its reading runs. +//! +//! The service's own tests read on a thread of their own ([`STACK`]) instead, +//! as a test binary is not the service's (`Config::read_in_child`). +use crate::jobs::Config; +use serde::{Deserialize, Serialize}; +use serde_json::{Value, json}; use std::panic::{AssertUnwindSafe, catch_unwind}; +use std::process::Stdio; use std::sync::{Arc, LazyLock}; -use std::time::Duration; +use std::time::{Duration, Instant}; +use tokio::io::{AsyncReadExt, AsyncWriteExt}; use tokio::sync::Semaphore; use wai_quantum::engine::refusal::{self, tag}; -/// The stack each thread that reads a request runs on. +/// The stack a reading runs on where it runs on a thread. pub const STACK: usize = 64 << 20; /// Requests read at once; the rest wait their turn. pub const AT_ONCE: usize = 4; @@ -33,22 +45,162 @@ pub const WAIT: Duration = Duration::from_secs(10); static TURNS: LazyLock> = LazyLock::new(|| Arc::new(Semaphore::new(AT_ONCE))); -/// `work`, one of [`AT_ONCE`], on a thread of its own with a stack of -/// [`STACK`]: its value, or the refusal (503 when no turn comes within -/// [`WAIT`], 500 when it panicked). -pub async fn run(work: impl FnOnce() -> T + Send + 'static) -> Result { - let Ok(Ok(turn)) = tokio::time::timeout(WAIT, Arc::clone(&TURNS).acquire_owned()).await else { - return Err((503, tag(refusal::BUSY, format!("{AT_ONCE} requests are being read; try again shortly")))); - }; - on_thread(move || { - let _turn = turn; - work() - }) - .await +/// One request's reading, as its child takes it on standard input. +#[derive(Clone, Debug, Serialize, Deserialize)] +#[serde(tag = "read", rename_all = "snake_case")] +pub enum Read { + /// A job or a set: its source read where it has one, and with `plan`, + /// held to the checks that read and planned within `job_bytes` + /// (`http::plan_job`). The answer is `{"request", "plan", "items", + /// "plans"}`, `plan` null without planning, or `{"request", "refused": + /// [status, why]}` where the planning refused it. + Admit { request: Value, job_bytes: u64, plan: bool }, + /// A design check (`analysis::Check`, by name): its bounds, then its answer. + Check { check: String, body: Value }, + /// An engine call held to its bounds: `kind` is `estimate`, + /// `estimate_verify`, `qec`, `pulse` or `variational`; `key`, the receipt + /// key's secret in hex, for an estimate, which seals as it goes. + Call { kind: String, op: String, body: Value, key: Option }, + /// The verdicts and the view an MCP tool's answer states (`mcp::finish`). + Finish { name: String, args: Value, answer: Value, views: bool }, + /// The verdict a refused MCP task states. + Refusal { args: Value, why: String }, } -/// `work` on a thread of its own with a stack of [`STACK`], without a turn -/// (the caller holds one of its own): its value, or the refusal. +/// `read`, one of [`AT_ONCE`], in a child whose memory and time are capped +/// by `config` (or on a thread of its own, where `config` says so): its +/// answer, or the refusal. +pub async fn read(config: &Config, r: Read) -> Result { + let turn = turn().await?; + if !config.read_in_child { + return on_thread(move || { + let _turn = turn; + run(r) + }) + .await?; + } + let out = in_child(config, &r).await; + drop(turn); + out +} + +async fn turn() -> Result { + match tokio::time::timeout(WAIT, Arc::clone(&TURNS).acquire_owned()).await { + Ok(Ok(t)) => Ok(t), + _ => Err((503, tag(refusal::BUSY, format!("{AT_ONCE} requests are being read; try again shortly")))), + } +} + +/// `read`, carried out here: what the child does. +pub fn run(r: Read) -> Result { + match r { + Read::Admit { request, job_bytes, plan } => crate::http::within_budget(|| { + let request = crate::http::with_source(request)?; + if !plan { + return Ok(json!({ "request": request, "plan": null })); + } + // A refusal of the plan is the answer's, beside the request as read. + let (p, work) = match crate::http::plan_job(job_bytes, &request) { + Ok(planned) => planned, + Err((code, why)) => return Ok(json!({ "request": request, "refused": [code, why] })), + }; + let (items, plans) = match work { + crate::jobs::Work::One => (Value::Null, Value::Null), + crate::jobs::Work::Set { items, plans } => ( + items.iter().map(|i| json!({ "request": i.request, "point": i.point })).collect(), + serde_json::to_value(plans).expect("plans are JSON"), + ), + }; + Ok(json!({ "request": request, "plan": p, "items": items, "plans": plans })) + }), + Read::Check { check, body } => { + let check = crate::analysis::Check::named(&check).ok_or((400, format!("no check {check}")))?; + crate::http::within_budget(|| crate::analysis::bounds(check, &body))?; + Ok(crate::analysis::answer(check, &body)) + } + Read::Call { kind, op, body, key } => crate::analysis::call(&kind, &op, &body, key.as_deref()), + Read::Finish { name, args, answer, views } => crate::http::within_budget(|| Ok(crate::mcp::finish(&name, &args, answer, views))), + Read::Refusal { args, why } => crate::http::within_budget(|| Ok(wai_quantum::engine::verdict::of_refusal(&args, &why))), + } +} + +/// `wai-quantum-service read`: one [`Read`] on standard input, its answer +/// on standard output as `{"ok": …}` or `{"refused": [status, why]}`. +pub fn child() { + let mut input = String::new(); + let out = match std::io::Read::read_to_string(&mut std::io::stdin(), &mut input).map_err(|e| e.to_string()).and_then(|_| serde_json::from_str::(&input).map_err(|e| e.to_string())) { + Ok(r) => match run(r) { + Ok(v) => json!({ "ok": v }), + Err((code, why)) => json!({ "refused": [code, why] }), + }, + Err(e) => json!({ "refused": [500, format!("the reading did not arrive: {e}")] }), + }; + println!("{out}"); +} + +/// The local command a refusal for size names. +const LOCAL: &str = "run it on your own machine (`wai-quantum run`, `wai-quantum critique` and the rest read without these limits)"; + +async fn in_child(config: &Config, r: &Read) -> Result { + let exe = std::env::current_exe().map_err(|e| (500, format!("the service could not find itself: {e}")))?; + let mut cmd = crate::jobs::sandboxed(&exe, config.sandbox, "read"); + cmd.stdin(Stdio::piped()).stdout(Stdio::piped()).stderr(Stdio::piped()).kill_on_drop(true); + let mut child = cmd.spawn().map_err(|e| (503, tag(refusal::BUSY, format!("the service could not start a reader ({e}); try again shortly"))))?; + let pid = child.id().unwrap_or(0); + let text = serde_json::to_vec(r).expect("a reading is JSON"); + let mut stdin = child.stdin.take().expect("piped"); + let writer = tokio::spawn(async move { + let _ = stdin.write_all(&text).await; + }); + let mut stdout = child.stdout.take().expect("piped"); + let mut stderr = child.stderr.take().expect("piped"); + let reader = tokio::spawn(async move { + let mut out = Vec::new(); + let _ = stdout.read_to_end(&mut out).await; + out + }); + let errors = tokio::spawn(async move { + let mut out = Vec::new(); + let _ = stderr.read_to_end(&mut out).await; + out + }); + let limit = config.read_mib << 20; + let deadline = Instant::now() + config.read_timeout; + let ended = loop { + match tokio::time::timeout(Duration::from_millis(50), child.wait()).await { + Ok(Ok(status)) => break Ok(status), + Ok(Err(e)) => break Err((500, format!("the reader could not be watched: {e}"))), + Err(_) => {} + } + if crate::jobs::rss_bytes(pid).is_some_and(|rss| rss > limit) { + crate::jobs::kill(pid); + let _ = child.wait().await; + break Err((422, tag(refusal::TOO_LARGE_HERE, format!("reading this request took more than {} MiB, more than the service reads with; {LOCAL}", config.read_mib)))); + } + if Instant::now() > deadline { + crate::jobs::kill(pid); + let _ = child.wait().await; + break Err((422, tag(refusal::TOO_LARGE_HERE, format!("reading this request took more than {} s, longer than the service reads for; {LOCAL}", config.read_timeout.as_secs())))); + } + }; + let _ = writer.await; + let out = reader.await.unwrap_or_default(); + let err = errors.await.unwrap_or_default(); + let status = ended?; + let parsed = serde_json::from_slice::(&out).ok(); + match parsed { + Some(v) if status.success() && v.get("ok").is_some() => Ok(v["ok"].clone()), + Some(v) if status.success() && v["refused"][0].is_u64() => Err((v["refused"][0].as_u64().unwrap_or(500) as u16, v["refused"][1].as_str().unwrap_or("refused").to_string())), + _ => { + let detail = String::from_utf8_lossy(&err); + let last = detail.lines().rev().find(|l| !l.trim().is_empty()).unwrap_or("").trim(); + Err((500, format!("reading this request failed inside the service ({status}{}{last}); nothing was admitted, and the service runs on", if last.is_empty() { "" } else { ": " }))) + } + } +} + +/// `work` on a thread of its own with a stack of [`STACK`]: its value, or +/// the refusal (500 when it panicked). pub async fn on_thread(work: impl FnOnce() -> T + Send + 'static) -> Result { let (tx, rx) = tokio::sync::oneshot::channel(); std::thread::Builder::new() @@ -76,10 +228,10 @@ mod tests { /// read as usual. #[tokio::test] async fn a_panic_fails_its_request_and_the_next_one_runs() { - let (code, why) = run(|| -> u32 { panic!("a test's panic") }).await.unwrap_err(); + let (code, why) = on_thread(|| -> u32 { panic!("a test's panic") }).await.unwrap_err(); assert_eq!(code, 500); assert!(why.contains("a test's panic"), "{why}"); - assert_eq!(run(|| 7).await.unwrap(), 7); + assert_eq!(on_thread(|| 7).await.unwrap(), 7); } /// The work's thread has the stack this module states: a recursion that @@ -95,6 +247,16 @@ mod tests { deep(n - 1).wrapping_mul(3).wrapping_add(pad[(n % 128) as usize]) } let want = (1..=8000u64).fold(1u64, |v, k| v.wrapping_mul(3).wrapping_add(k)); - assert_eq!(run(|| deep(8000)).await.unwrap(), want); + assert_eq!(on_thread(|| deep(8000)).await.unwrap(), want); + } + + /// A reading goes to its child and back as JSON, each kind of it. + #[test] + fn a_reading_is_json_both_ways() { + let r = Read::Call { kind: "qec".into(), op: "decode".into(), body: json!({ "circuit": "H 0\n" }), key: None }; + let back: Read = serde_json::from_value(serde_json::to_value(&r).unwrap()).unwrap(); + assert!(matches!(back, Read::Call { ref kind, .. } if kind == "qec")); + let a = run(Read::Admit { request: json!({ "qasm": "OPENQASM 3.0; qubit q; h q;", "shots": 4 }), job_bytes: 1 << 30, plan: true }).unwrap(); + assert!(a["plan"]["method"].is_string() && a["items"].is_null(), "{a}"); } } diff --git a/wai/wai-quantum-service/tests/depth.rs b/wai/wai-quantum-service/tests/depth.rs index ace012a7f..d440f5b20 100644 --- a/wai/wai-quantum-service/tests/depth.rs +++ b/wai/wai-quantum-service/tests/depth.rs @@ -42,7 +42,13 @@ fn json_of(port: u16, method: &str, path: &str, body: Option<&Value>) -> (u16, V /// The service, its store at `dir`, `burst` submissions a window, once it /// answers. fn start(dir: &Path, port: u16, burst: u32) -> Child { + start_with(dir, port, burst, &[]) +} + +/// [`start`], with `env` set beside. +fn start_with(dir: &Path, port: u16, burst: u32, env: &[(&str, &str)]) -> Child { let child = Command::new(env!("CARGO_BIN_EXE_wai-quantum-service")) + .envs(env.iter().copied()) .arg("serve") .env("WAI_QSVC_ADDR", format!("127.0.0.1:{port}")) .env("WAI_QSVC_STORE", dir.join("store")) @@ -222,3 +228,134 @@ fn a_request_with_heavy_references_is_refused_in_bounded_time_and_counted() { svc.wait().unwrap(); let _ = std::fs::remove_dir_all(&d); } + +/// Memory, through every door that reads a program: a declaration of a +/// hundred bytes that asks for 137 GB, an array of 10^10 booleans, a bit +/// register of 4·10^9 bits, resets of a wide register in a loop. Each ended +/// the process (an allocation that fails aborts it) or grew it without end. +/// Now each is refused (`Q130`) and the service answers `/v1/health` after. +/// A bit string wider than an integer is a refusal, not a 500; a run whose +/// MCP arguments are not an object, a refusal, not a dropped connection. +#[test] +fn programs_that_would_hold_without_limit_are_refused_and_the_service_runs_on() { + let d = dir("memory"); + let port = free_port(); + let mut svc = start(&d, port, 1000); + let program = |body: &str| format!("OPENQASM 3.0;\nqubit[1] q;\n{body}x q[0];\n"); + for body in [ + "array[int[8], 4294967295] a;\n", + "array[bool, 100000, 100000] a;\n", + "bit[4000000000] c;\n", + "qubit[60000] r;\nfor int i in [0:999999] reset r;\n", + ] { + let qasm = program(body); + for (path, req) in [("/v1/jobs", json!({ "qasm": qasm, "dry_run": true })), ("/v1/jobs", json!({ "qasm": qasm, "shots": 10 })), ("/v1/critique", json!({ "qasm": qasm }))] { + let t = Instant::now(); + let (code, b) = json_of(port, "POST", path, Some(&req)); + assert!(code < 500 && b.to_string().contains("Q130"), "{path} {body}: {code} {b}"); + assert!(t.elapsed() < Duration::from_secs(60), "{path} {body}: {:?}", t.elapsed()); + alive(port, &mut svc, body); + } + } + let wide = program(&format!("int[32] x = \"{}\";\n", "1".repeat(200))); + for (path, req) in [("/v1/jobs", json!({ "qasm": wide, "dry_run": true })), ("/v1/jobs", json!({ "qasm": wide, "shots": 10 })), ("/v1/critique", json!({ "qasm": wide }))] { + let (code, b) = json_of(port, "POST", path, Some(&req)); + assert!(code < 500 && b.to_string().contains("Q112"), "{path}: {code} {b}"); + } + for args in [json!([1, 2]), json!(5), json!("x")] { + let (code, b) = json_of(port, "POST", "/mcp", Some(&json!({ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "run_quantum_circuit", "arguments": args } }))); + assert_eq!(code, 200, "{b}"); + assert!(b["result"]["isError"] == true || b["error"].is_object(), "{b}"); + alive(port, &mut svc, "non-object arguments"); + } + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// The reading child's own caps, what holds where a limit of the reader's +/// might not: given 256 MiB, a set whose programs each read within the +/// reader's limits but together hold more is refused for its memory, as is an error-correction circuit that unrolls past it; +/// the service runs on. +#[test] +fn a_reading_past_its_memory_is_stopped_and_refused_and_the_service_runs_on() { + let d = dir("readcap"); + let port = free_port(); + let mut svc = start_with(&d, port, 1000, &[("WAI_QSVC_READ_GB", "0.25"), ("WAI_QSVC_READ_SECS", "120")]); + let big = |k: u32| format!("OPENQASM 3.0;\nqubit[2] q;\nfor int i in [0:999998] {{ x q[0]; x q[1]; }}\n// {k}\n"); + let t = Instant::now(); + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&json!({ "programs": (0..2).map(big).collect::>(), "shots": 10 }))); + assert!(code == 422 && b.to_string().contains("J031") && b.to_string().contains("MiB"), "{code} {b}"); + assert!(t.elapsed() < Duration::from_secs(120), "{:?}", t.elapsed()); + alive(port, &mut svc, "a set past the reading's memory"); + let circuit = "H 0 1 2 3 4 5 6 7\nREPEAT 100000000 {\n X_ERROR(0.1) 0 1 2 3 4 5 6 7\n}\nM 0\nDETECTOR rec[-1]\n"; + let t = Instant::now(); + let (code, b) = json_of(port, "POST", "/v1/qec/decode", Some(&json!({ "circuit": circuit, "shots": 10 }))); + assert!(code == 422 && b.to_string().contains("J031"), "{code} {b}"); + assert!(t.elapsed() < Duration::from_secs(150), "{:?}", t.elapsed()); + alive(port, &mut svc, "a circuit past the reading's memory"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// A client told to try again shortly spends nothing by doing so: refused +/// for its jobs at once, before anything is counted or read, its window +/// stays whole however often it obeys `Retry-After`; and a job sent again +/// under its key, its window spent, is still its job, given as OpenQASM or +/// from another format. +#[test] +fn a_client_told_to_retry_spends_nothing_and_its_key_s_job_comes_back_past_the_window() { + let d = dir("retry"); + let port = free_port(); + let mut svc = start_with(&d, port, 4, &[("WAI_QSVC_CLIENT_ACTIVE", "1")]); + let left = |head: &str| head.to_ascii_lowercase().lines().find_map(|l| l.strip_prefix("ratelimit: \"submissions\";r=").map(|r| r.split(';').next().unwrap_or("").to_string())).unwrap_or_default(); + let post = |path: &str, body: &Value, key: Option<&str>| { + let mut s = TcpStream::connect(("127.0.0.1", port)).unwrap(); + s.set_read_timeout(Some(Duration::from_secs(120))).unwrap(); + let text = body.to_string(); + let k = key.map(|k| format!("Idempotency-Key: {k}\r\n")).unwrap_or_default(); + write!(s, "POST {path} HTTP/1.1\r\nHost: 127.0.0.1\r\nContent-Type: application/json\r\n{k}Content-Length: {}\r\nConnection: close\r\n\r\n{text}", text.len()).unwrap(); + let mut out = String::new(); + let _ = s.read_to_string(&mut out); + let code: u16 = out.split_whitespace().nth(1).and_then(|c| c.parse().ok()).unwrap_or(0); + let (head, body) = out.split_once("\r\n\r\n").map(|(h, b)| (h.to_string(), b.to_string())).unwrap_or_default(); + (code, head, body) + }; + // A job that runs long, its answer lost: sent under a key, given from a + // source in OpenQASM. + let mut long = String::from("OPENQASM 3.0;\nqubit[24] q;\nh q;\n"); + for k in 0..6000 { + long += &format!("rx(0.{}) q[{}];\ncx q[{}], q[{}];\n", k % 9 + 1, k % 24, k % 24, (k + 1) % 24); + } + let job = json!({ "source": { "format": "qasm", "program": long }, "observables": ["Z0"] }); + let (code, head, first) = post("/v1/jobs", &job, Some("lost-answer")); + assert_eq!(code, 202, "{first}"); + assert_eq!(left(&head), "3", "{head}"); + let id = serde_json::from_str::(&first).unwrap()["id"].as_str().unwrap().to_string(); + // Told twenty times to wait for it, every five seconds as Retry-After + // says: nothing spent. + let bell = json!({ "qasm": "OPENQASM 3.0; qubit q; h q;", "shots": 10 }); + for _ in 0..20 { + let (code, head, body) = post("/v1/jobs", &bell, None); + assert!(code == 429 && body.contains("wait for one to finish"), "{code} {body}"); + assert_eq!(left(&head), "3", "{head}"); + assert!(head.to_ascii_lowercase().contains("retry-after: 5"), "{head}"); + } + // Its window spent by refusals that do count, + for _ in 0..3 { + let (code, _, body) = post("/v1/critique", &json!({ "qasm": "OPENQASM 3.0; qubit q; foo q;" }), None); + assert_eq!(code, 200, "{body}"); + } + let (code, _, _) = post("/v1/critique", &json!({ "qasm": "OPENQASM 3.0; qubit q; h q;" }), None); + assert_eq!(code, 429); + // its job, sent again under its key, still comes back to it. + let (code, _, again) = post("/v1/jobs", &job, Some("lost-answer")); + assert_eq!(code, 202, "{again}"); + assert_eq!(serde_json::from_str::(&again).unwrap()["id"], id.as_str(), "{again}"); + let _ = http(port, "DELETE", &format!("/v1/jobs/{id}"), None); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + diff --git a/wai/wai-quantum/docs/qasm-diagnostics.md b/wai/wai-quantum/docs/qasm-diagnostics.md index eef66bda3..4dc89f335 100644 --- a/wai/wai-quantum/docs/qasm-diagnostics.md +++ b/wai/wai-quantum/docs/qasm-diagnostics.md @@ -80,6 +80,7 @@ engine. A test holds this page to the code's table. | `Q127` | refused | An assignment to a constant, an input or a loop variable. | | `Q128` | refused | A measurement where a value is needed that is not a bit (measurement gives bits, into a bit register). | | `Q129` | refused | A parameter (`input float`, `input angle`) a gradient cannot follow to the gates: under a modifier, in a classical variable, a condition, an index, a size, a loop's range, a subroutine or `gphase`, or one whose type rounds it (`angle[n]`, `float[32]`); a gradient follows a parameter through arithmetic, the built-in functions and gate definitions into a gate's arguments. | +| `Q130` | refused | A program whose reading would hold more memory than the reader takes: a classical value (an array, a bit register) of more than 64 MiB, each element counted at its width, or more than 512 MiB in all of its values, gates, steps, measurements and timing; refused before it is made. | | `Q200` | lowered | A `gate` the program defines, inlined at each call with its parameters bound. | | `Q201` | lowered | `ctrl @` and `negctrl @` on any gate, lowered exactly to standard gates (a negative control is a control between two `x`). | | `Q202` | lowered | `inv @` on any gate, user gates included: each gate's exact inverse, in reverse order. | diff --git a/wai/wai-quantum/docs/qasm-feature-matrix.md b/wai/wai-quantum/docs/qasm-feature-matrix.md index 2e494a7a6..b60739649 100644 --- a/wai/wai-quantum/docs/qasm-feature-matrix.md +++ b/wai/wai-quantum/docs/qasm-feature-matrix.md @@ -32,6 +32,7 @@ and in the browser build. | Statements and blocks `{ }`, scopes | supported; qubits, gates, subroutines and externs are declared at the top level | Q105 | | Nesting: statements in blocks and bodies, parentheses, operators, brackets, calls, casts and types | supported to 64 levels together in the text (a statement, each enclosing statement, and each level of its expressions' trees count one; a long `else if` chain is one level a branch, and reads as a `switch` at any length); deeper is refused, never read by a recursion that could run out of stack | Q006 | | Nesting in all, through the gates and subroutines a program calls | supported to 256 levels (each statement, expression and call the reader runs counts one), and calls 32 deep; deeper is refused | Q006, Q209 | +| What a program holds on reading: its classical values, gates, steps, measurements and timing | supported to 64 MiB for one value (an array or a bit register, each element counted at its width) and 512 MiB in all; more is refused before it is made. Making or copying a value counts toward the reader's 20 000 000 steps at one step for every 128 bytes, and an element read from an array copies the element alone | Q130, Q312 | | `pragma …`, `#pragma …`, annotations `@keyword …` | warned: kept, not applied; `@wai.max_iterations N` bounds the `while` after it | Q013 | ## Literals diff --git a/wai/wai-quantum/src/qasm.rs b/wai/wai-quantum/src/qasm.rs index 80b93019c..4c222e7d7 100644 --- a/wai/wai-quantum/src/qasm.rs +++ b/wai/wai-quantum/src/qasm.rs @@ -416,6 +416,23 @@ pub(crate) fn spend_step() -> bool { }) } +/// `n` steps of reading spent at once (values made or copied, counted by +/// their bytes), from this thread's budget when one is set: whether the +/// budget ran out. What is left is spent, and the budget then holds no more. +pub(crate) fn spend_steps(n: u64) -> bool { + READS.with_borrow_mut(|r| match r { + Some(r) if r.left < n => { + r.left = 0; + true + } + Some(r) => { + r.left -= n; + false + } + None => false, + }) +} + /// Runs `f` with every program it reads on this thread read once (a program /// read again with no inputs is answered from its first reading) and every /// reading within one budget of `steps` together: each statement the reader diff --git a/wai/wai-quantum/src/qasm/depth_tests.rs b/wai/wai-quantum/src/qasm/depth_tests.rs index 06399f163..f7ff34ba8 100644 --- a/wai/wai-quantum/src/qasm/depth_tests.rs +++ b/wai/wai-quantum/src/qasm/depth_tests.rs @@ -257,3 +257,88 @@ fn the_code_states_the_limits() { assert!(m.contains(&format!("more than {MAX_NESTING} levels")), "{m}"); assert!(m.contains(&format!("more than {} levels in all", super::elab::MAX_NEST)), "{m}"); } + +/// Memory: a value as large as its declaration asks for, a register's steps +/// grown in a loop, a bit string wider than an integer: each refused, on a +/// 2 MiB thread, before what it would hold is made. A declaration of a +/// hundred bytes asked for 137 GB here, which no refusal stopped: an +/// allocation that fails ends the process. +#[test] +fn what_a_program_would_hold_past_the_limit_is_refused_before_it_is_made() { + for (what, body) in [ + ("an array of 2^32 − 1 integers", "array[int[8], 4294967295] a;\n"), + ("an array of 10^10 booleans", "array[bool, 100000, 100000] a;\n"), + ("an array of wide bit strings", "array[bit[1048576], 1000] a;\n"), + ("a bit register of 4·10^9 bits", "bit[4000000000] c;\n"), + ("resets of a wide register, repeated", "qubit[60000] r;\nfor int i in [0:999999] reset r;\n"), + ("measurements of a wide register, repeated", "qubit[60000] r;\nbit[60000] m;\nfor int i in [0:999999] m = measure r;\n"), + ("barriers on a wide register, repeated", "qubit[60000] r;\nfor int i in [0:999999] barrier r;\n"), + ] { + let t = std::time::Instant::now(); + assert_eq!(read_on_stack(format!("{HEAD}{body}x q;\n")), Err(diag::TOO_LARGE), "{what}"); + assert!(t.elapsed() < std::time::Duration::from_secs(60), "{what}: {:?}", t.elapsed()); + eprintln!("{what}: refused in {:?}", t.elapsed()); + } + // Within the limits, as programs hold them, each reads. + for body in ["array[int[8], 1000] a;\n", "array[float, 64, 64] m;\n", "bit[1048576] wide;\n", "qubit[64] r;\nfor int i in [0:99] reset r;\n"] { + assert!(read_on_stack(format!("{HEAD}{body}x q;\n")).is_ok(), "{body}"); + } + // Reading an element of a large array copies the element, not the array; + // copying the whole array is counted as reading time. + let elements = format!("{HEAD}array[int[32], 100000] a;\nint s = 0;\nfor int i in [0:99999] {{ s += a[i]; }}\nx q;\n"); + assert!(read_on_stack(elements).is_ok()); + // Within a budget of a million steps, a loop that copies a 3.2 MB array + // runs out of it after some forty copies, not after a hundred thousand. + let copies = format!("{HEAD}array[int[32], 100000] a;\narray[int[32], 100000] b;\nfor int i in [0:99999] {{ b = a; }}\nx q;\n"); + let r = on_stack(STACK, move || within_steps(1_000_000, || read(&copies).map(|_| ()).map_err(|d| d.code))); + assert_eq!(r, Err(diag::UNROLL_LIMIT)); +} + +/// A statement on a register of tens of thousands of qubits costs its +/// length: each qubit is checked once, where each was compared with every one +/// before it (some 1.8·10⁹ comparisons a statement, so a loop of resets ran +/// for minutes within a few hundred steps). +#[test] +fn a_statement_on_a_wide_register_costs_its_length() { + let t = std::time::Instant::now(); + for body in ["for int i in [0:49] reset r;\n", "for int i in [0:49] barrier r;\n", "bit[60000] m;\nfor int i in [0:49] m = measure r;\n", "for int i in [0:9] x r;\n"] { + assert!(read_on_stack(format!("{HEAD}qubit[60000] r;\n{body}x q;\n")).is_ok(), "{body}"); + } + assert!(t.elapsed() < std::time::Duration::from_secs(30), "{:?}", t.elapsed()); + assert_eq!(read_on_stack(format!("{HEAD}qubit[4] r;\nlet a = r[0:3] ++ r[1:2];\nreset a;\n")), Err(diag::QUBIT_TWICE)); +} + +/// A bit string wider than an integer holds, given to an integer, is refused +/// with its code; the reader once panicked on it. +#[test] +fn a_bit_string_wider_than_an_integer_is_refused_not_a_panic() { + for body in [format!("int[32] x = \"{}\";\n", "1".repeat(200)), "bit[200] b;\nint[32] x = b;\n".to_string(), format!("uint y = \"{}\";\n", "0".repeat(121))] { + assert_eq!(read_on_stack(format!("{HEAD}{body}x q;\n")), Err(diag::INTEGER_RANGE), "{body}"); + } + assert!(read_on_stack(format!("{HEAD}int[32] x = \"{}\";\nx q;\n", "1".repeat(20))).is_ok()); +} + +/// A loop over a range makes each value as it reaches it: a range of twenty +/// million that breaks at once holds no list of twenty million values (608 +/// MB, before its first pass). Measured in a process of its own, by its peak +/// resident memory, where the system says it (Linux). +#[cfg(target_os = "linux")] +#[test] +fn a_range_is_not_made_all_at_once() { + const CHILD: &str = "WAI_QUANTUM_DEPTH_RANGE_CHILD"; + let peak = || std::fs::read_to_string("/proc/self/status").ok().and_then(|s| s.lines().find(|l| l.starts_with("VmHWM:")).and_then(|l| l.split_whitespace().nth(1)?.parse::().ok())).unwrap_or(0); + if std::env::var_os(CHILD).is_some() { + let src = format!("{HEAD}for int i in [0:19999999] {{ break; }}\nx q;\n"); + assert!(read(&src).is_ok()); + println!("peak-kib {}", peak()); + return; + } + let out = std::process::Command::new(std::env::current_exe().unwrap()) + .args(["--exact", "qasm::depth_tests::a_range_is_not_made_all_at_once", "--nocapture", "--test-threads", "1"]) + .env(CHILD, "1") + .output() + .unwrap(); + let text = String::from_utf8_lossy(&out.stdout); + let kib: u64 = text.split("peak-kib ").nth(1).and_then(|v| v.split_whitespace().next()?.parse().ok()).unwrap_or_else(|| panic!("no peak: {text}")); + assert!(out.status.success() && kib < 300 << 10, "peak {kib} KiB:\n{text}"); +} diff --git a/wai/wai-quantum/src/qasm/diag.rs b/wai/wai-quantum/src/qasm/diag.rs index c9869c551..020ec5e12 100644 --- a/wai/wai-quantum/src/qasm/diag.rs +++ b/wai/wai-quantum/src/qasm/diag.rs @@ -202,6 +202,7 @@ table! { ASSIGN_CONSTANT = "Q127", Refused, "An assignment to a constant, an input or a loop variable."; MEASURE_PLACE = "Q128", Refused, "A measurement where a value is needed that is not a bit (measurement gives bits, into a bit register)."; NOT_DIFFERENTIABLE = "Q129", Refused, "A parameter (`input float`, `input angle`) a gradient cannot follow to the gates: under a modifier, in a classical variable, a condition, an index, a size, a loop's range, a subroutine or `gphase`, or one whose type rounds it (`angle[n]`, `float[32]`); a gradient follows a parameter through arithmetic, the built-in functions and gate definitions into a gate's arguments."; + TOO_LARGE = "Q130", Refused, "A program whose reading would hold more memory than the reader takes: a classical value (an array, a bit register) of more than 64 MiB, each element counted at its width, or more than 512 MiB in all of its values, gates, steps, measurements and timing; refused before it is made."; // Gates and modifiers. GATE_INLINED = "Q200", Lowered, "A `gate` the program defines, inlined at each call with its parameters bound."; diff --git a/wai/wai-quantum/src/qasm/elab.rs b/wai/wai-quantum/src/qasm/elab.rs index 13a9ba882..3f1aba049 100644 --- a/wai/wai-quantum/src/qasm/elab.rs +++ b/wai/wai-quantum/src/qasm/elab.rs @@ -32,6 +32,21 @@ pub(crate) const MAX_DEPTH: usize = 32; /// bounds it; this bounds what calls stack on top of each other, so that /// reading never runs out of stack. pub(crate) const MAX_NEST: usize = 256; +/// The most memory one program's reading may hold, in bytes: its classical +/// values as each is made, and its gates, steps, measurements and timing as +/// they grow, counted together and never given back. A reading is refused +/// past it ([`diag::TOO_LARGE`]) before it allocates more, so what a program +/// asks for on reading is bounded by this, not by the host. +pub(crate) const MAX_HELD: u64 = 512 << 20; +/// The most one classical value (an array, a bit string) may take, in bytes. +pub(crate) const MAX_VALUE: u64 = 64 << 20; +/// Making or copying this many bytes counts as one step of reading, toward +/// [`MAX_STEPS`] and a budget of reading (`qasm::within_steps`): a program +/// that copies a large array in a loop costs what its copying costs. +const BYTES_A_STEP: u64 = 128; +/// What a value and an entry of the program's lists take, at least, in bytes. +const VAL_BYTES: u64 = 32; +const ENTRY_BYTES: u64 = 48; /// The most controls modifiers may add: a phase on m qubits is written as /// 2^m − 1 `cx` and as many `p`. pub(crate) const MAX_CONTROLS: u32 = 16; @@ -240,6 +255,10 @@ pub(crate) struct Elab<'a> { depth: usize, /// How deep the reader's recursion stands ([`MAX_NEST`]). nest: usize, + /// The bytes the reading holds ([`MAX_HELD`]), and how far into the + /// program's lists (its ops, steps, measurements, timing) they count. + held: u64, + seen: (usize, usize, usize, usize), runtime: usize, loops: Vec, frames: Vec, @@ -298,6 +317,8 @@ pub(crate) fn elaborate_with(src: &str, toks: &[Token], stmts: &[Stmt], version: count: 0, depth: 0, nest: 0, + held: 0, + seen: (0, 0, 0, 0), runtime: 0, loops: Vec::new(), frames: Vec::new(), @@ -316,6 +337,27 @@ pub(crate) fn elaborate_with(src: &str, toks: &[Token], stmts: &[Stmt], version: Ok((p, l, d, tangents)) } +/// A loop's values: a range, each value made as the loop reaches it, or a +/// list already made. +enum Values { + Range { from: i128, step: i128, count: i128, k: i128 }, + List(std::vec::IntoIter), +} + +impl Iterator for Values { + type Item = Val; + + fn next(&mut self) -> Option { + match self { + Values::Range { from, step, count, k } => (*k < *count).then(|| { + *k += 1; + Val::Int(*from + (*k - 1) * *step) + }), + Values::List(items) => items.next(), + } + } +} + fn is_physical(name: &str) -> Option { name.strip_prefix('$').filter(|d| !d.is_empty() && d.bytes().all(|b| b.is_ascii_digit())).and_then(|d| d.parse().ok()) } @@ -364,9 +406,72 @@ impl<'a> Elab<'a> { if super::spend_step() { return self.fail(diag::UNROLL_LIMIT, at, "the programs read for one request elaborate past its budget of statements together (wai_quantum::qasm::within_steps)"); } + self.grown(at) + } + + /// What the program's lists have grown by since they were last counted + /// (gates, steps, measurements and timing, each with what it carries), + /// held. Counted at every statement, so no statement's growth goes + /// uncounted past the next. + fn grown(&mut self, at: Loc) -> R<()> { + let (o, st, m, ti) = self.seen; + let mut bytes = 0u64; + for op in &self.ops[o..] { + bytes += ENTRY_BYTES + 4 * op.qubits.len() as u64 + 8 * op.params.len() as u64; + } + bytes += ENTRY_BYTES * (self.steps.len() - st) as u64 + 8 * (self.measured.len() - m) as u64; + for t in &self.extra.timing[ti..] { + bytes += ENTRY_BYTES + 4 * t.qubits.len() as u64; + } + self.seen = (self.ops.len(), self.steps.len(), self.measured.len(), self.extra.timing.len()); + self.hold(bytes, at) + } + + /// `bytes` more held by the reading: refused past [`MAX_HELD`], and + /// counted as steps of reading at one for every [`BYTES_A_STEP`]. + fn hold(&mut self, bytes: u64, at: Loc) -> R<()> { + self.held = self.held.saturating_add(bytes); + if self.held > MAX_HELD { + return self.fail( + diag::TOO_LARGE, + at, + format!("reading the program would hold more than {} MiB (its classical values, gates, steps and timing together); read it where it can, on your own machine, or make it smaller", MAX_HELD >> 20), + ); + } + self.charge(bytes, at) + } + + /// `bytes` made or copied, counted as steps of reading (toward + /// [`MAX_STEPS`] and a budget of reading) at one for every + /// [`BYTES_A_STEP`]. + fn charge(&mut self, bytes: u64, at: Loc) -> R<()> { + let steps = bytes / BYTES_A_STEP; + if steps == 0 { + return Ok(()); + } + self.count = self.count.saturating_add(steps); + if self.count > MAX_STEPS { + return self.fail(diag::UNROLL_LIMIT, at, format!("the program unrolls past {MAX_STEPS} statements (the values it makes and copies counted with them)")); + } + if super::spend_steps(steps) { + return self.fail(diag::UNROLL_LIMIT, at, "the programs read for one request elaborate past its budget of steps together (wai_quantum::qasm::within_steps)"); + } Ok(()) } + /// A value of `bytes` about to be made: refused past [`MAX_VALUE`], else + /// held. + fn make(&mut self, bytes: Option, at: Loc) -> R<()> { + match bytes { + Some(b) if b <= MAX_VALUE => self.hold(b, at), + _ => self.fail( + diag::TOO_LARGE, + at, + format!("this value would take more than {} MiB (each element counted at its width); the reader holds no classical value larger", MAX_VALUE >> 20), + ), + } + } + // ----------------------------------------------------------------------- // Before running: physical qubits, and the names registers and gates take // ----------------------------------------------------------------------- @@ -1282,6 +1387,27 @@ impl<'a> Elab<'a> { }) } + /// The bytes a value of type `t` takes, at least; `None` past `u64`. + fn value_size(t: &Ty) -> Option { + match t { + Ty::Bit(n) => Some(VAL_BYTES + u64::from(n.unwrap_or(1))), + Ty::Array(inner, dims) => dims.iter().try_fold(Self::value_size(inner)?, |b, &d| b.checked_mul(d as u64)?.checked_add(VAL_BYTES)), + _ => Some(VAL_BYTES), + } + } + + /// The bytes the value `v` takes, at least: what copying it copies. + fn size_of(v: &Val) -> u64 { + match v { + Val::Array(items) => match items.first() { + Some(Val::Array(_) | Val::Bits(_)) => VAL_BYTES + items.iter().map(Self::size_of).sum::(), + _ => VAL_BYTES * (1 + items.len() as u64), + }, + Val::Bits(b) => VAL_BYTES + b.len() as u64, + _ => VAL_BYTES, + } + } + /// The default value of a type, for a declaration with none. fn zero(&self, t: &Ty) -> Option { Some(match t { @@ -1311,7 +1437,10 @@ impl<'a> Elab<'a> { Ty::Int(w) | Ty::Uint(w) => { let unsigned = matches!(t, Ty::Uint(_)); let i = match &v { - Val::Int(_) | Val::Bool(_) | Val::Bits(_) => v.integer().expect("integral"), + Val::Int(_) | Val::Bool(_) | Val::Bits(_) => match v.integer() { + Some(i) => i, + None => return self.fail(diag::INTEGER_RANGE, at, format!("`{name}` is {}, and {} bits are more than an integer here holds (120): cast a slice of them", t.name(), v.bits_len())), + }, Val::Float(f) if self.version == Version::Two && f.fract() == 0.0 => *f as i128, _ => return bad(self, &v), }; @@ -1437,6 +1566,8 @@ impl<'a> Elab<'a> { Some(e) => self.size(e)?, None => 1, }; + // Each bit a number the steps name, and a place in each outcome. + self.make(Some(VAL_BYTES + 4 * u64::from(n)), at)?; let bits = if self.global() { if self.lookup(&name.text).is_some() { return self.fail(diag::DECLARED_TWICE, name.at, format!("{} is declared twice", name.text)); @@ -1496,7 +1627,10 @@ impl<'a> Elab<'a> { How::Plain | How::Output => { let v = match init { None if t == Ty::Stretch => Some(Val::Duration(Dur::Stretch(name.text.clone()))), - None => self.zero(&t), + None => { + self.make(Self::value_size(&t), at)?; + self.zero(&t) + } Some(e) => match self.evs(e)? { Ev::Val(v) => Some(self.coerce(v, &t, &name.text, e.at)?), Ev::Unknown(why) => { @@ -2418,7 +2552,7 @@ impl<'a> Elab<'a> { Ok(Flow::Normal) } - fn loop_values(&mut self, over: &Expr) -> R> { + fn loop_values(&mut self, over: &Expr) -> R { match &over.kind { E::Range(a, s, b) => { let (Some(a), Some(b)) = (a, b) else { return self.fail(diag::BAD_RANGE, over.at, "a loop's range gives its start and its end") }; @@ -2435,18 +2569,19 @@ impl<'a> Elab<'a> { if count as u128 > MAX_STEPS as u128 { return self.fail(diag::UNROLL_LIMIT, over.at, format!("the loop runs {count} times, so the program unrolls past {MAX_STEPS} statements")); } - Ok((0..count).map(|k| Val::Int(a + k * step)).collect()) + // Written out one value at a time, never all at once. + Ok(Values::Range { from: a, step, count, k: 0 }) } E::Set(items) => { let mut out = Vec::with_capacity(items.len()); for i in items { out.push(self.known(i)?); } - Ok(out) + Ok(Values::List(out.into_iter())) } _ => match self.ev(over)? { - Ev::Val(Val::Array(v)) => Ok(v), - Ev::Val(Val::Bits(b)) => Ok(b.into_iter().map(Val::Bool).collect()), + Ev::Val(Val::Array(v)) => Ok(Values::List(v.into_iter())), + Ev::Val(Val::Bits(b)) => Ok(Values::List(b.into_iter().map(Val::Bool).collect::>().into_iter())), Ev::Bits(_, n) => self.fail(diag::NOT_KNOWN, over.at, format!("`{n}` holds measured bits: a loop runs over values known on reading")), _ => self.fail(diag::BAD_RANGE, over.at, "a loop runs over a range `[a:b]`, a set `{a, b}` or an array"), }, @@ -2467,7 +2602,7 @@ impl<'a> Elab<'a> { }) } - fn for_body(&mut self, values: Vec, t: Option, var: &Name, body: &[Stmt], at: Loc) -> R { + fn for_body(&mut self, values: Values, t: Option, var: &Name, body: &[Stmt], at: Loc) -> R { for v in values { self.tick(at)?; self.scopes.push(Scope::new()); @@ -2963,6 +3098,10 @@ impl<'a> Elab<'a> { } return Ok(Ev::Qubits(Rc::new(vec![q]), n.to_string())); } + if let Some(Binding::Var { val: Some(v @ (Val::Array(_) | Val::Bits(_))), .. }) = self.lookup(n) { + let bytes = Self::size_of(v); + self.charge(bytes, at)?; + } Ok(match self.lookup(n) { Some(Binding::Var { val: Some(v), .. }) => Ev::Val(v.clone()), Some(Binding::Var { val: None, why, .. }) => Ev::Unknown(if why.is_empty() { format!("`{n}` is not known on reading") } else { why.clone() }), @@ -3033,6 +3172,9 @@ impl<'a> Elab<'a> { } fn index_ev(&mut self, base: &Expr, items: &[Expr], at: Loc) -> R { + if let Some(v) = self.element_in_place(base, items, at)? { + return Ok(Ev::Val(v)); + } let b = self.ev(base)?; Ok(match b { Ev::Qubits(q, n) => { @@ -3049,16 +3191,75 @@ impl<'a> Elab<'a> { } /// An element (or slice) of a known value. + /// `a[i]`, `a[i, j]`: one element of an array variable, found in place, + /// so reading an element copies the element and not the array. `None` + /// where the index is a range or a set, the name is not an array known on + /// reading, or a gradient follows the name: read the usual way. + fn element_in_place(&mut self, base: &Expr, items: &[Expr], at: Loc) -> R> { + let E::Name(n) = &base.kind else { return Ok(None) }; + if self.tan.is_some() || items.iter().any(|i| matches!(i.kind, E::Range(..) | E::Set(_))) { + return Ok(None); + } + let at_path = |s: &Self, path: &[usize]| -> Option { + let Some(Binding::Var { val: Some(v), .. }) = s.lookup(n) else { return None }; + let mut cur = v; + for &k in path { + let Val::Array(a) = cur else { return None }; + cur = a.get(k)?; + } + Some(match cur { + Val::Array(a) => Val::Array(Vec::with_capacity(a.len())), + other => other.clone(), + }) + }; + let len_at = |s: &Self, path: &[usize]| -> Option { + let Some(Binding::Var { val: Some(v), .. }) = s.lookup(n) else { return None }; + let mut cur = v; + for &k in path { + let Val::Array(a) = cur else { return None }; + cur = a.get(k)?; + } + match cur { + Val::Array(a) => Some(a.len()), + _ => None, + } + }; + let mut path = Vec::with_capacity(items.len()); + for item in items { + let Some(len) = len_at(self, &path) else { return Ok(None) }; + let sel = self.select("the array", len, std::slice::from_ref(item), at)?; + let [k] = sel[..] else { return Ok(None) }; + path.push(k); + } + // An element that is itself an array is copied whole, as it is read. + match at_path(self, &path) { + Some(Val::Array(_)) => { + let Some(Binding::Var { val: Some(v), .. }) = self.lookup(n) else { return Ok(None) }; + let mut cur = v; + for &k in &path { + let Val::Array(a) = cur else { return Ok(None) }; + cur = &a[k]; + } + let copy = cur.clone(); + self.charge(Self::size_of(©), at)?; + Ok(Some(copy)) + } + other => Ok(other), + } + } + fn element(&mut self, v: &Val, items: &[Expr], at: Loc) -> R { match v { - Val::Array(a) => { - let mut cur = Val::Array(a.clone()); + Val::Array(_) => { + // Walked by reference: only what is selected is copied. + let mut cur = std::borrow::Cow::Borrowed(v); for item in items { - let Val::Array(a) = &cur else { return self.fail(diag::WRONG_TYPE, at, "an index into something that is not an array") }; + let Val::Array(a) = cur.as_ref() else { return self.fail(diag::WRONG_TYPE, at, "an index into something that is not an array") }; let sel = self.select("the array", a.len(), std::slice::from_ref(item), at)?; - cur = if sel.len() == 1 && !matches!(item.kind, E::Range(..) | E::Set(_)) { a[sel[0]].clone() } else { Val::Array(sel.iter().map(|&i| a[i].clone()).collect()) }; + let next = if sel.len() == 1 && !matches!(item.kind, E::Range(..) | E::Set(_)) { a[sel[0]].clone() } else { Val::Array(sel.iter().map(|&i| a[i].clone()).collect()) }; + cur = std::borrow::Cow::Owned(next); } - Ok(cur) + Ok(cur.into_owned()) } Val::Bits(b) => { let sel = self.select("the bits", b.len(), items, at)?; @@ -3150,8 +3351,15 @@ impl<'a> Elab<'a> { Ev::Bits(_, n) => return self.fail(diag::WRONG_TYPE, e.at, format!("`{n}` is a bit register, not qubits")), _ => return self.fail(diag::WRONG_TYPE, e.at, "expected qubits"), }; + // One pass with a mark for each qubit, not each against those before + // it: a register of tens of thousands is checked in its own length. + let mut seen = if qs.len() > 32 { vec![false; self.n_qubits as usize] } else { Vec::new() }; for (i, q) in qs.iter().enumerate() { - if qs[..i].contains(q) { + let twice = match seen.get_mut(*q as usize) { + Some(s) => std::mem::replace(s, true), + None => qs[..i].contains(q), + }; + if twice { return self.fail(diag::QUBIT_TWICE, e.at, format!("qubit {q} named twice in one reference")); } } @@ -3185,10 +3393,20 @@ impl<'a> Elab<'a> { if ops.is_empty() { return Ok((0..self.n_qubits).collect()); } - let mut out = Vec::new(); + // One operand names each of its qubits once already ([`Elab::qubits_of`]). + if let [o] = ops { + return self.qubits_of(o); + } + // Each qubit once, in the order first named: a register of tens of + // thousands is listed in one pass, not compared against itself. + let (mut out, mut seen) = (Vec::new(), vec![false; self.n_qubits as usize]); for o in ops { for q in self.qubits_of(o)? { - if !out.contains(&q) { + let first = match seen.get_mut(q as usize) { + Some(s) => !std::mem::replace(s, true), + None => !out.contains(&q), + }; + if first { out.push(q); } } diff --git a/wai/wai-quantum/src/qasm/unroll.rs b/wai/wai-quantum/src/qasm/unroll.rs index bc85ac5ce..2044cd746 100644 --- a/wai/wai-quantum/src/qasm/unroll.rs +++ b/wai/wai-quantum/src/qasm/unroll.rs @@ -14,7 +14,7 @@ use std::collections::{BTreeSet, HashMap}; use super::Inputs; use super::diag::{self, Diagnostic}; -use super::elab::{MAX_STEPS, RESERVED}; +use super::elab::{MAX_HELD, MAX_STEPS, RESERVED}; use super::lex::{Tok, Token, lex, text}; use super::parse::MAX_NESTING; @@ -310,6 +310,14 @@ impl<'a> Unroller<'a> { if self.out.len() >= MAX_TOKENS { return err(diag::UNROLL_LIMIT, Some(&t), self.src, format!("the program unrolls past {MAX_TOKENS} tokens")); } + // The tokens written are held as the reader holds a program's lists + // (`elab::MAX_HELD`), and every sixteen count as a step of reading. + if (self.out.len() as u64 + 1) * std::mem::size_of::() as u64 > MAX_HELD { + return err(diag::TOO_LARGE, Some(&t), self.src, format!("the program written out would hold more than {} MiB of tokens; write it out on your own machine, or at smaller inputs", MAX_HELD >> 20)); + } + if self.out.len() % 16 == 15 && super::spend_step() { + return err(diag::UNROLL_LIMIT, Some(&t), self.src, "the programs read for one request unroll past its budget of steps together (wai_quantum::qasm::within_steps)"); + } self.out.push(t); Ok(()) } @@ -707,7 +715,8 @@ impl<'a> Unroller<'a> { return self.fail(diag::EXPECTED, at, format!("expected `in` after `for {name}`")); } j += 1; - let values: Option> = if self.sym(j, "[") { + // A range's values are made one at a time, as the loop reaches each. + let values: Option>> = if self.sym(j, "[") { let close = self.close(j, ("[", "]"))?; let parts = self.split(j + 1, close - 1, ":"); if !parts.iter().all(|p| self.foldable(p.0, p.1)) { @@ -727,15 +736,15 @@ impl<'a> Unroller<'a> { if count as u64 > MAX_STEPS { return self.fail(diag::UNROLL_LIMIT, at, format!("the loop runs {count} times, so the program unrolls past {MAX_STEPS} statements")); } - Some((0..count).map(|k| a + k * step).collect()) + Some(Box::new((0..count).map(move |k| a + k * step))) } } else if self.sym(j, "{") { let close = self.close(j, ("{", "}"))?; let parts = self.split(j + 1, close - 1, ","); if parts.iter().all(|p| self.foldable(p.0, p.1)) { - let vs = parts.into_iter().map(|(lo, hi)| self.value(lo, hi)).collect::>()?; + let vs: Vec = parts.into_iter().map(|(lo, hi)| self.value(lo, hi)).collect::>()?; j = close; - Some(vs) + Some(Box::new(vs.into_iter())) } else { None } diff --git a/wai/wai-quantum/src/qasm/value.rs b/wai/wai-quantum/src/qasm/value.rs index 61b15d75f..19f08675a 100644 --- a/wai/wai-quantum/src/qasm/value.rs +++ b/wai/wai-quantum/src/qasm/value.rs @@ -137,6 +137,14 @@ impl Val { /// The value as an integer, if it is one: an integer, a bool, known bits /// (unsigned, the lowest first), or a float with no fraction. + /// How many bits a bit string holds (0 for anything else). + pub fn bits_len(&self) -> usize { + match self { + Val::Bits(b) => b.len(), + _ => 0, + } + } + pub fn integer(&self) -> Option { match self { Val::Int(v) => Some(*v), diff --git a/wai/wai-web/src/quantum_engine.rs b/wai/wai-web/src/quantum_engine.rs index d3cb729a2..79990daad 100644 --- a/wai/wai-web/src/quantum_engine.rs +++ b/wai/wai-web/src/quantum_engine.rs @@ -1623,8 +1623,9 @@ mod wasm_tests { } } - /// A program nested thousands deep is refused with the reader's code, - /// never a trap, through every door that reads one: a run, the lint, the + /// A program nested thousands deep, or one that asks for more memory + /// than the reader holds, is refused with the reader's code, never a + /// trap, through every door that reads one: a run, the lint, the /// critique, a generator written out; an error-correction circuit's /// blocks are refused on their line, and an answer's chain of seals past /// its depth; and the engine answers the next request as before. @@ -1648,6 +1649,10 @@ mod wasm_tests { } chain += "s2000 = @Sample(p, 10)\n"; assert!(e.answer_value(&chain, "s0", true).contains("more than 32 statements")); + // A declaration that asks for more than the page holds is refused, not a trap. + let huge = "OPENQASM 3.0;\nqubit q;\narray[int[8], 4294967295] a;\nx q;\n"; + assert!(run(&e, json!({ "qasm": huge, "shots": 10 })).to_string().contains("Q130")); + assert!(engine_critique(huge).contains("Q130")); let body = run(&e, json!({ "qasm": "OPENQASM 3.0; qubit[2] q; h q[0]; cx q[0], q[1];", "shots": 20 })); assert_eq!(body["status"], "done", "{body}"); } -- 2.54.0 (Apple Git-157) From 6da10a041833dcad20fe8e327287cf4f58164f60 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Sat, 7 Nov 2026 02:35:17 -0500 Subject: [PATCH 03/18] wai-quantum: no anonymous request ends the job service or holds it without limit The 33 findings of #676's review, each with a test that failed first. - QEC texts: a detector or qubit index past 2^24 is refused on its line, offsets counted, where 79 bytes asked for 2.4 PB. - Reading: a flat run of operators, an else-if chain and a switch's case values are held flat and walked in loops, at any length, left to right, so they read on 1 MiB and the generator's writer agrees; an array has at most 7 dimensions. - Every read of a client's program is counted first and runs in a child capped in memory, time and answer: admission, dry runs, each check's bounds and answer in one budget, the engine calls, a set's receipt, and MCP verdicts. - Turns: a client's share, a bound on those waiting, and no refund. A resend under its key is matched as received, reading nothing, and comes back past the window and with no turn to be had. A budget spent is a refusal whatever a check made of it. Retry-After is the pace that would not spend the window. - MCP: a tool that says nothing of a program takes no turn; a poll reads nothing; a call not counted states why, so every answer keeps its verdict and its schema. - Connections are held to a time for headers and for a body. A restart reads no program: a job keeps its plan and a set its items. - Answers: the chain limit refuses the same statements in any order. A decode is held to 1,024 fired detectors. Two tests that could only hang now fail by name. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/wai-quantum-qec.yml | 1 + .github/workflows/wai-quantum-verdict.yml | 1 + wai/changes/2026-11-06-quantum-parse-depth.md | 131 +++--- wai/wai-quantum-cli/docs/llms-full.txt | 8 +- wai/wai-quantum-service/Cargo.lock | 2 + wai/wai-quantum-service/Cargo.toml | 3 + wai/wai-quantum-service/README.md | 41 +- wai/wai-quantum-service/docs/scheduler.md | 2 +- wai/wai-quantum-service/docs/self-host.md | 6 + wai/wai-quantum-service/src/analysis.rs | 108 +++-- wai/wai-quantum-service/src/discovery.rs | 30 +- wai/wai-quantum-service/src/http.rs | 253 +++++++---- wai/wai-quantum-service/src/jobs.rs | 255 ++++++++--- wai/wai-quantum-service/src/main.rs | 3 +- wai/wai-quantum-service/src/mcp.rs | 242 +++++++++-- wai/wai-quantum-service/src/offload.rs | 247 ++++++++--- wai/wai-quantum-service/tests/depth.rs | 292 ++++++++++++- wai/wai-quantum-service/tests/verify.rs | 3 +- wai/wai-quantum/docs/job-refusals.md | 6 +- wai/wai-quantum/docs/qasm-diagnostics.md | 4 +- wai/wai-quantum/docs/qasm-feature-matrix.md | 4 +- wai/wai-quantum/docs/qec.md | 5 +- wai/wai-quantum/src/engine/answer/eval.rs | 73 +++- wai/wai-quantum/src/engine/answer/tests.rs | 36 ++ wai/wai-quantum/src/engine/formats/emit.rs | 25 +- wai/wai-quantum/src/engine/refusal.rs | 6 +- wai/wai-quantum/src/engine/sets.rs | 16 +- wai/wai-quantum/src/engine/verdict.rs | 12 + wai/wai-quantum/src/engine/views.rs | 10 +- wai/wai-quantum/src/qasm.rs | 1 + wai/wai-quantum/src/qasm/ast.rs | 11 +- wai/wai-quantum/src/qasm/depth_tests.rs | 126 +++++- wai/wai-quantum/src/qasm/diag.rs | 4 +- wai/wai-quantum/src/qasm/elab.rs | 399 +++++++++++++----- wai/wai-quantum/src/qasm/parse.rs | 68 ++- wai/wai-quantum/src/qasm/pulse.rs | 10 + wai/wai-quantum/src/qasm/unroll.rs | 23 +- wai/wai-quantum/src/quantum_frame.rs | 14 +- wai/wai-quantum/src/quantum_qec_text.rs | 104 ++++- 39 files changed, 2006 insertions(+), 579 deletions(-) diff --git a/.github/workflows/wai-quantum-qec.yml b/.github/workflows/wai-quantum-qec.yml index 6c1847ab8..4215bff43 100644 --- a/.github/workflows/wai-quantum-qec.yml +++ b/.github/workflows/wai-quantum-qec.yml @@ -103,6 +103,7 @@ jobs: quantum_qec_text::tests::a_model_unrolls_with_its_offsets_tags_and_parts \ quantum_qec_text::tests::blocks_nested_past_the_limit_are_refused_on_their_line \ quantum_qec_text::tests::an_empty_block_repeated_without_end_is_refused_not_run \ + quantum_qec_text::tests::an_index_past_what_a_text_holds_is_refused_on_its_line \ quantum_window::tests::narrow_windows_correct_every_edge_set_below_half_the_distance \ quantum_window::tests::a_window_over_everything_is_the_batch_decoder \ quantum_window::tests::narrow_windows_track_the_batch_decoder \ diff --git a/.github/workflows/wai-quantum-verdict.yml b/.github/workflows/wai-quantum-verdict.yml index a5821c825..1890ba570 100644 --- a/.github/workflows/wai-quantum-verdict.yml +++ b/.github/workflows/wai-quantum-verdict.yml @@ -111,6 +111,7 @@ jobs: engine::answer::tests::every_problem_has_its_code_and_says_what_becomes_of_the_statement \ engine::answer::tests::an_answer_s_calls_run_on_the_engine_and_its_seal_verifies_when_run_again \ engine::answer::tests::a_chain_of_statements_past_the_limit_is_refused_not_overflowed \ + engine::answer::tests::the_chain_limit_refuses_the_same_statements_in_any_order \ engine::telemetry::tests::the_failures_become_a_corpus_in_the_conformance_format; do grep -q "^test $t \.\.\. ok$" "$RUNNER_TEMP/answer.log" || { echo "::error::$t did not run"; exit 1; } done diff --git a/wai/changes/2026-11-06-quantum-parse-depth.md b/wai/changes/2026-11-06-quantum-parse-depth.md index af1d8f309..00cd48004 100644 --- a/wai/changes/2026-11-06-quantum-parse-depth.md +++ b/wai/changes/2026-11-06-quantum-parse-depth.md @@ -1,64 +1,91 @@ -### WAI Quantum: the reader holds its nesting and its memory, and the job service reads each request once, within one budget, in a capped child +### WAI Quantum: the reader holds its nesting and its memory, and no request can end the job service or hold it without limit - **The reader's nesting** (`wai_quantum::qasm`). A program nested past what the reader takes is refused with the new code `Q006` rather than read by a recursion that ran a thread's stack out: 64 levels in its text (`qasm::parse::MAX_NESTING`: statements in blocks and bodies, parentheses, operators, brackets, calls, casts - and types together, and every expression's tree as deep as its statement leaves room for, so a sum or an - index chain a hundred thousand long is refused too), and 256 in all through the gates and subroutines it - calls (the elaborator's statements, expressions, conditions and calls). The parser reads binary operators - by precedence climbing and each statement and expression form in a function of its own, so a level of - nesting stacks a few small frames; read in a release build on a 1 MiB stack, every construct nests several - times past the limit before it would run out. A modifier chain (`inv @ inv @ …`) is a list and reads at any - length. `docs/qasm-feature-matrix.md` and `docs/qasm-diagnostics.md` state the limits. + and types together, and every expression's tree as deep as its statement leaves room for), and 256 in all + through the gates and subroutines it calls (the elaborator's statements, expressions, conditions and calls). + An array has at most 7 dimensions (`MAX_ARRAY_DIMS`): they are a comma apart in the text, and each is a level of + its value. What people write is not refused: a run of operators of one precedence (`a + b - c`, a condition of + a thousand `&&`s) and an `else if` chain are held flat and read, run and dropped in loops, at any length, on a + stack of 1 MiB, left to right, so a floating-point sum is the one it was; a switch's case list on measured bits + is flat too, where it crashed the reader at about ten thousand values. What newly refuses a program main read: + more than about 60 levels of real nesting (parentheses, blocks, a `**` chain, an index chain), a `++` chain of + that length, and a recursive subroutine whose body nests more than about seven levels, which reaches 256 before + its 32 calls (Q006). QIR and jeff conditions now stop at the 30 levels of parentheses the reader takes (Q705), + from 32, with the converter's own refusal. `docs/qasm-feature-matrix.md` and `docs/qasm-diagnostics.md` state + the limits. - **Every other reader of a client's text**: the generator's writer (`qasm::instantiate`: its integer - expressions, loops and conditions, an `else if` chain now followed in a loop), the pulse reader's - calibrations (through the parser), the error-correction circuit and model texts (`REPEAT` blocks at most - `quantum_qec_text::MAX_BLOCK_DEPTH`, 32, deep, refused on their line; a block's every pass counts toward the - bound on what a text unrolls to, so one repeated without end that writes nothing ends, refused; blocks - repeated no times are checked once on one copy), the editor's index (a statement nested past the reader's - limit is passed over without recursion), and the answer layer's statements (at most - `engine::answer::eval::CHAIN_DEPTH`, 32, each naming the next). The exact expectation values of a program - that measures in the middle walk its branches with a list rather than a recursion, in the same order, so a - program that branches thousands of times is refused at `MAX_LEAVES`, not run out of stack. + expressions, loops and conditions), the pulse reader's calibrations (through the parser), the error-correction + circuit and model texts (`REPEAT` blocks at most `quantum_qec_text::MAX_BLOCK_DEPTH`, 32, deep, refused on their + line; a block's every pass counts toward the bound on what a text unrolls to, so one repeated without end that + writes nothing ends, refused, and `REPEAT n { H 0 }` is refused from about 25 000 000 passes, not 50 000 000 + operations; blocks repeated no times are checked once on one copy), the editor's index (a statement nested past + the reader's limit is passed over without recursion), and the answer layer's statements (at most + `engine::answer::eval::CHAIN_DEPTH`, 32, each naming the next, the same statements refused in any order). The + exact expectation values of a program that measures in the middle walk its branches with a list rather than a + recursion, in the same order, so a program that branches thousands of times is refused at `MAX_LEAVES`, not run + out of stack. +- **An index in a few bytes of text** (`quantum_qec_text::MAX_QUBITS`, `MAX_DETECTORS`, each 2²⁴). What is made for + a detector or a qubit is as long as its index: `shift_detectors 100000000000000` and a `detector(1) D0` asked for + 2.4 PB in 79 bytes, and an allocation that fails ends the process. Each index is refused where it is read, on + its line, offsets counted, however it is named. - **The reader's memory** (`Q130`, new). A program whose reading would hold more than the reader takes is refused before it is made: one classical value (an array or a bit register, each element counted at its width, so `array[bit[1048576], 1000]` is a gigabyte) of more than 64 MiB, or more than 512 MiB in all of its values and of its gates, steps, measurements and timing as they grow (a `reset` of a wide register in a loop grew them - without end). A declaration of a hundred bytes, `array[int[8], 4294967295] a;`, asked for 137 GB, and an - allocation that fails ends the process. Making or copying a value counts toward the reader's steps, at one - for every 128 bytes; an element read from an array copies the element alone; a loop over a range makes each - value as it reaches it, not all of them first. A statement on a register of tens of thousands of qubits is - checked in its own length, where each qubit was compared with every one before it (a loop of resets on such a - register ran for minutes within a few hundred steps). A bit string wider than an integer, given to one, is - refused (`Q112`) where the reader panicked. + without end). A declaration of a hundred bytes, `array[int[8], 4294967295] a;`, asked for 137 GB. Making or + copying a value counts toward the reader's steps, at one for every 128 bytes; an element read from an array + copies the element alone; a loop over a range makes each value as it reaches it, not all of them first. A + statement on a register of tens of thousands of qubits is checked in its own length, where each qubit was + compared with every one before it. Each operand of a long run or a set counts as a statement. A bit string wider + than an integer, given to one, is refused (`Q112`) where the reader panicked. - **A budget of reading** (`qasm::within_steps`, `qasm::steps_left`): every program read on a thread within it is read once (a program read again with no inputs is answered from its first reading) and every reading - draws on one budget of steps: each statement elaborated, each gate written, each pass of a loop a generator - is written out with. Past it, a reading is refused (`Q312`), and `steps_left` is `Some(0)`. + draws on one budget of steps: each statement elaborated, each gate written, what is made or copied by its + size, each pass of a loop a generator is written out with and what it writes. Past it, a reading is refused + (`Q312`), and `steps_left` is `Some(0)`. - **The job service: admission by cost.** A submission (`POST /v1/jobs`, `POST /v1/run`, a dry run, and the - MCP run tools and tasks) is answered from its key where it is one sent again, then counted against the - client's window, so every request refused after that counts; then held to what reads nothing (at most 16 - intents, each of a set's items too; at most 100 items in a set; exact whole numbers); and only then are its - programs read, each once, within one budget of 20 000 000 steps for the whole request (`read_steps` in - `GET /v1`'s limits): a request whose references or items take more is refused, `422` and `J031`. A check - (`/v1/critique` and the others) is counted before its bounds read anything, and its bounds read within the - same budget. -- **The job service: in a child.** Admission, a dry run, every check's bounds and answer, the - error-correction, pulse and variational calls and the estimates, and the verdicts the MCP tools state beside - an answer are read by a child, the service's own binary run as `wai-quantum-service read`, sandboxed as a - worker is, at most four at once (a request that waits ten seconds for a turn is refused, `503` and `J032`). A - child past `WAI_QSVC_READ_GB` of memory (4) or `WAI_QSVC_READ_SECS` (300) is stopped and its request refused, - `422` and `J031`; one that panics, overflows or aborts is answered `500`; the service runs on. An - error-correction decode's circuit text, which no bound held, is read within those caps. A run's MCP arguments - that are not an object are refused, where they dropped the connection. -- **The job service: a retry costs nothing.** What refuses whatever is sent (the client's jobs at once, the - memory they hold, a full queue) is checked before the submission is counted, and a refusal after the count - that tells the client to send it again (`429` for its memory, `503`) is given back its count, so obeying - `Retry-After` never spends a client's window. A job given in another format under a key that names one of the - client's jobs is read from that format before it is counted, so a send after a lost answer gets its job even - past the window. -- **Tests**: every nesting construct at thousands of levels on a - 2 MiB thread, refused; a seeded property test over mixed nesting; the service binary started in a test and - sent programs nested thousands deep through each submission, the MCP run tool and task, the checks and an - error-correction decode, answering `/v1/health` after each; and a request with sixteen heavy references, - refused within one budget in bounded time and counted toward its client's window. + MCP run tools and tasks) sent again under its key is answered first, reading nothing: a job given in another + format (`"source"`) is matched by the digest of the request as received, kept with the job. Then come the refusals + that read nothing and are not counted (the client's jobs at once, the memory they hold, a full queue, a spent + window), then a turn, then the count, so every request refused after that counts and one told to try again has + read and spent nothing; there is no refund. Then what reads nothing (at most 16 intents, each of a set's items + too; at most 100 items in a set; exact whole numbers), and only then are its programs read, each once, within + one budget of 22 000 000 steps for the whole request (`read_steps` in `GET /v1`'s limits), the reader's bounds on + a job's statements and gates together, so a job's own program that reads within them reads here. A request past + it, by its own program, its references or its items, is refused, `422` and `J031`. `Retry-After` on a `429` is + now the pace at which a refusal that counts would not spend the window (20 s by default), not 5. +- **The job service: every check.** A check (`/v1/critique` and the others, and the estimate, error-correction, + pulse and variational calls) is counted once it has its turn and before its bounds read anything, so their + `422`s count. A check's answer reads within the same budget as its bounds, each program once, so a repair of forty + fixes to a slow program ends at the budget where it ran for minutes. A budget spent is a refusal whatever a bound + made of it: an equivalence bound read a refusal as no work and let the check run. `/v1/verify` and MCP + `verify_quantum_receipt` on a set are counted and read as a check is, as its sweep is written out again, and a + seal that does not hold is not. A decode is held to 1 024 fired detectors, as matching grows faster than their + square. +- **The job service: in a child.** Admission, a dry run, every check's bounds and answer, the error-correction, + pulse and variational calls and the estimates, a set's receipt, and the verdicts the MCP tools state beside an + answer are read by a child, the service's own binary run as `wai-quantum-service read`, sandboxed as a worker + is. Four requests are read at once and two checks beside them; a client has `WAI_QSVC_CLIENT_READS` (2) being read + or waiting; at most 32 wait, each for ten seconds; past any of these a request is refused, `503` and `J032`. A + child past `WAI_QSVC_READ_GB` of memory (4) or `WAI_QSVC_READ_SECS` (120) is stopped and its request refused, `422` + and `J031`; one that panics, overflows or aborts is answered `500`; the service runs on. A client that hangs up + ends its child. +- **The job service: MCP.** A tool that says nothing about a program takes no turn. A verdict reads the program only + for a call that was counted; a poll states what its job holds, and reads nothing, however often it is asked; a + call refused before it was counted (a bad key, a tool the service does not have, past the window) or told to try + again states a verdict that gives the reason. One that finds no turn for its verdict does the same, so every + answer about a program has one, and follows its schema. A refused task is the same, where it read the program + on a runtime worker with no budget. A run's MCP arguments that are not an object are refused, where they dropped + the connection. +- **The job service: connections and restarts.** A request's header section is held to `WAI_QSVC_HEADER_SECS` and + its body to `WAI_QSVC_BODY_SECS` (30 each; past the second, `408`). A restart reads no program: a job keeps the + plan it was admitted with and a set its items, where a queue of slow programs held the port unbound for the sum of + their reads. +- **Tests**: every nesting construct at thousands of levels on a 2 MiB thread, refused; runs, `else if` chains and + case lists of a hundred thousand on 1 MiB, read to what the tree gives; a seeded property test over mixed nesting; + the service binary started in a test and sent each of these through each door, answering `/v1/health` after each; + refused MCP tasks that hold no worker; heavy requests from eight clients beside a health probe; a client's share + of the turns; a reading stopped at its deadline; a hang-up that frees its turn; stuck connections let go; and a + restart that reads nothing. Two tests that could only end by the job's own limit now fail by name. diff --git a/wai/wai-quantum-cli/docs/llms-full.txt b/wai/wai-quantum-cli/docs/llms-full.txt index 375e4dff0..ec53d9785 100644 --- a/wai/wai-quantum-cli/docs/llms-full.txt +++ b/wai/wai-quantum-cli/docs/llms-full.txt @@ -989,8 +989,8 @@ Each construct below is supported (read as the language defines it), lowered (re | Comments `//` and `/* */` | supported; an unclosed block comment is refused | Q003 | | Identifiers, Unicode letters included (`θ`, `π`) | supported | | | Statements and blocks `{ }`, scopes | supported; qubits, gates, subroutines and externs are declared at the top level | Q105 | -| Nesting: statements in blocks and bodies, parentheses, operators, brackets, calls, casts and types | supported to 64 levels together in the text (a statement, each enclosing statement, and each level of its expressions' trees count one; a long `else if` chain is one level a branch, and reads as a `switch` at any length); deeper is refused, never read by a recursion that could run out of stack | Q006 | -| Nesting in all, through the gates and subroutines a program calls | supported to 256 levels (each statement, expression and call the reader runs counts one), and calls 32 deep; deeper is refused | Q006, Q209 | +| Nesting: statements in blocks and bodies, parentheses, operators, brackets, calls, casts and types | supported to 64 levels together in the text (a statement, each enclosing statement, and each level of its expressions' trees count one). A run of operators of one precedence (`a + b - c`, `c[0] && c[1] && …`) is one level, and an `else if` chain one level, at any length: both are read and run in loops, left to right. Deeper is refused, never read by a recursion that could run out of stack; an array has at most 7 dimensions | Q006 | +| Nesting in all, through the gates and subroutines a program calls | supported to 256 levels (each statement, expression and call the reader runs counts one), and calls 32 deep; deeper is refused. Both hold: a recursive subroutine whose body nests more than about seven levels reaches 256 before 32 calls | Q006, Q209 | | What a program holds on reading: its classical values, gates, steps, measurements and timing | supported to 64 MiB for one value (an array or a bit register, each element counted at its width) and 512 MiB in all; more is refused before it is made. Making or copying a value counts toward the reader's 20 000 000 steps at one step for every 128 bytes, and an element read from an array copies the element alone | Q130, Q312 | | `pragma …`, `#pragma …`, annotations `@keyword …` | warned: kept, not applied; `@wai.max_iterations N` bounds the `while` after it | Q013 | @@ -1197,7 +1197,7 @@ Every code: | `Q003` | refused | A string, a block comment, a bracket or a calibration block that is not closed. | | `Q004` | refused | Something else was expected here: a `;`, a name, an expression or a statement (the message says which). | | `Q005` | refused | A statement in a place the language does not allow it: `else` with no `if`, `case` outside `switch`, a declaration inside a gate. | -| `Q006` | refused | A program nested deeper than the reader takes: statements, parentheses, operators, brackets and types more than 64 levels deep in its text, or more than 256 levels in all through the gates and subroutines it calls; refused, never read by a recursion that could run out of stack. | +| `Q006` | refused | A program nested deeper than the reader takes: statements, parentheses, operators, brackets and types more than 64 levels deep in its text (a run of one precedence, `a + b + c`, and an `else if` chain each count one, however long), an array of more than 7 dimensions, or more than 256 levels in all through the gates and subroutines it calls; refused, never read by a recursion that could run out of stack. | | `Q010` | warned | An `OPENQASM` version other than 2.x or 3.x: the program is read as OpenQASM 3. | | `Q011` | warned | An `include` of a file other than the standard libraries (`stdgates.inc`, `qelib1.inc`): its definitions are not read, so a gate it defines is unknown. | | `Q012` | lowered | `include "stdgates.inc"` or `"qelib1.inc"`: the standard gates, `U`, `gphase` and OpenQASM 2's names are built in. | @@ -1258,7 +1258,7 @@ Every code: | `Q309` | lowered | `while` whose condition is known on reading: unrolled until it fails. | | `Q310` | refused | A `while` whose condition reads measured bits and that declares no bound: add `@wai.max_iterations N` before it. | | `Q311` | warned | A `while` on measured bits run at most its declared bound (`@wai.max_iterations N`): a run that still meets its condition after N iterations stops looping there. | -| `Q312` | refused | A program that does not finish unrolling within the reader's bound (20 000 000 statements and loop iterations). | +| `Q312` | refused | A program that does not finish unrolling within the reader's bound (20 000 000 statements and loop iterations), or that runs out a caller's budget of reading (`qasm::within_steps`: statements, gates and a generator's loop passes, shared by every program read under it). | | `Q313` | refused | A range with a zero step, or that does not read as `[start:end]` or `[start:step:end]`. | | `Q314` | refused | `break` or `continue` outside a loop, or `return` outside a subroutine. | | `Q315` | refused | A `return` with a value under a condition on measured bits. | diff --git a/wai/wai-quantum-service/Cargo.lock b/wai/wai-quantum-service/Cargo.lock index a00790795..f305ee5c4 100644 --- a/wai/wai-quantum-service/Cargo.lock +++ b/wai/wai-quantum-service/Cargo.lock @@ -876,6 +876,8 @@ dependencies = [ "blake3", "futures-util", "getrandom 0.4.3", + "hyper", + "hyper-util", "libc", "serde", "serde_json", diff --git a/wai/wai-quantum-service/Cargo.toml b/wai/wai-quantum-service/Cargo.toml index 9234b1c50..83d7275df 100644 --- a/wai/wai-quantum-service/Cargo.toml +++ b/wai/wai-quantum-service/Cargo.toml @@ -9,6 +9,9 @@ publish = false [dependencies] axum = "0.8" +# The connection loop (http::serve): axum's own takes no header timeout. +hyper = { version = "1", features = ["http1", "server"] } +hyper-util = { version = "0.1", features = ["tokio", "service"] } tokio = { version = "1", features = ["rt-multi-thread", "macros", "process", "time", "sync", "net", "io-util"] } serde = { version = "1", features = ["derive"] } # float_roundtrip: a number a client sends back reads as the double it was diff --git a/wai/wai-quantum-service/README.md b/wai/wai-quantum-service/README.md index 98ef015e6..8a483d51a 100644 --- a/wai/wai-quantum-service/README.md +++ b/wai/wai-quantum-service/README.md @@ -67,7 +67,7 @@ curl -s https://wai.transaction.science/quantum-jobs/v1/run -H 'content-type: ap | `memory_bytes`, `work` | the memory it would be admitted with, and the planner's work | | `time` | `seconds` with its `basis`: the median rate of the last jobs of the method this service timed, or, before it has timed three, the planner's nominal rate; or `seconds: null` with why, where the planner's work does not predict the time | | `energy` | `joules_micro_estimate`, class `Estimator`, from the median power of the last metered jobs of the method here; or `unestimated` with why. Never a reading | -| `bounds` | the limits that apply: memory and time per job, jobs and memory per client, submissions per window, the queue, the most items in a set | +| `bounds` | the limits that apply: memory and time per job, jobs and memory per client, submissions per window, the queue, the most items in a set, the most intents, the steps a request's programs may take to read, the largest body | | `client` | where you stand: active jobs, memory held, submissions left | | `accepted` | whether the service would take the job now, by every limit; if not, `status` and `refusal` (its message and code) say why | | `items` | for a set: each item's plan and estimates | @@ -145,7 +145,7 @@ Counts are always seeded samples. | `GET /v1/jobs/{id}` | the job: its status, the request as submitted, its result when it is done, and the digests it is kept under; `?wait=S` (at most 25) holds the answer until its status changes | | `GET /v1/jobs/{id}/events` | the job's status as server-sent events: a `status` event at each change, then one `job` event with the whole job when it ends | | `DELETE /v1/jobs/{id}` | cancel a queued or running job | -| `POST /v1/verify` | `{"request": ..., "result": ...}`: whether the result's receipt seals that request and that result and every rule of [the provider protocol's section 8.4](../extensions/quantum-providers.md#84-what-a-verifier-checks) holds, read from the body's own text: `valid`, `reasons`, `key`, `form`, and what a provider's result says of itself | +| `POST /v1/verify` | `{"request": ..., "result": ...}` (a set's is counted and read as a check is, since its sweep is written out again; one whose seal does not hold is not): whether the result's receipt seals that request and that result and every rule of [the provider protocol's section 8.4](../extensions/quantum-providers.md#84-what-a-verifier-checks) holds, read from the body's own text: `valid`, `reasons`, `key`, `form`, and what a provider's result says of itself | | `POST /v1/critique` | `{"qasm"}`: the program's measures (gates by kind, two-qubit gates, depth, T count, whether it is Clifford, a modelled fidelity) and its smells, each with its lines | | `POST /v1/equivalent` | `{"a", "b"}`: whether two programs are the same circuit up to a global phase, and how that was decided | | `POST /v1/convert` | `{"to", "program" \| "program_base64", "from"?, "profile"?, "check"?}`: the program carried between OpenQASM 3, QIR and jeff (as base64), with the program as OpenQASM 3, each note with its code, what was read and written, and the check (the output read back and held to the program by the equivalence ladder); a refusal is the reader's, with its code and place ([the formats page](../wai-quantum/docs/formats.md)) | @@ -175,7 +175,7 @@ The three error-correction calls and the five of the variational and algorithm l A job may give a `target`: a built-in target's name, or a target document (`wai.quantum.target/1`, its schema in `GET /v1/openapi.json`). The program is compiled to the target's gates, placed on the qubits and routes most likely to succeed, and simulated under the noise its calibration gives. The result's `target` names the layout, its expected success, the noise and the target's hash, inside the receipt. The service reads the document it is given and fetches nothing. -Every refusal says why and ends with its code, which the body also carries as `code` ([the job layer's codes](../wai-quantum/docs/job-refusals.md), beside the reader's): `400` for a program or job that does not read (a set that does not read: `J007`), `422` for one larger than a job may be (`J031`), `429` past the per-client limits (`J030`), `503` when the queue is full (`J032`), on a private deployment `401` without one of its API keys (`J033`), and an `Idempotency-Key` that does not read (`400`) or was sent before with another request (`422`): `J034`. `GET /v1` states the limits. +Every refusal says why and ends with its code, which the body also carries as `code` ([the job layer's codes](../wai-quantum/docs/job-refusals.md), beside the reader's): `400` for a program or job that does not read (a set that does not read: `J007`), `422` for one larger than a job may be (`J031`), `429` past the per-client limits (`J030`), `503` when the queue is full or no turn to read it came (`J032`), on a private deployment `401` without one of its API keys (`J033`), and an `Idempotency-Key` that does not read (`400`) or was sent before with another request (`422`): `J034`. `GET /v1` states the limits. ## Status and limits @@ -185,20 +185,23 @@ Every answer carries the client's submission window in the rate-limit fields of A submission is admitted in the order of what each step costs: -1. A job sent again under its `Idempotency-Key` is answered with the key's job, and counts as nothing. A job given in another format (`"source"`) under a key that names one of your jobs is read from that format first, without counting, so a send after a lost answer gets its job even past the window; another request under the key counts. -2. What reads nothing and refuses whatever is sent: your jobs at once (`429`), the memory they hold (`429`), a full queue (`503`). These are not counted, so obeying `Retry-After` spends nothing. -3. The submission is counted against your window. Every refusal after this counts, except one that tells you to send it again (`429` for your memory, `503`), which is given back its count. -4. What reads nothing: every whole number exact, at most 16 intents (each of a set's items too), at most 100 items in a set. -5. Its programs are read: the job's, a set's items and a sweep's points, and every intent's reference, each once however often admission reads it, and all of them within one budget of 20,000,000 steps (statements the reader runs, gates it writes, a generator's loop passes, and every 128 bytes of a value it makes or copies): as many as one program may take. A request whose programs take more is refused, `422` and `J031`, with the command that runs it on your own machine. -6. The job is kept and queued, within your and the queue's limits. +1. A job sent again under its `Idempotency-Key` is answered with the key's job, before any limit: it reads nothing, takes no turn and counts as nothing. A job given in another format (`"source"`) is matched as it was received, so it too comes back past the window. Sent again in some other form, it is read and counted to tell whether it is the same request. +2. What reads nothing and refuses whatever is sent: your jobs at once (`429`), the memory they hold (`429`), a full queue (`503`), a spent window (`429`). None of these is counted, so waiting as told spends nothing. +3. It waits for a turn to be read. If none comes, it is refused (`503`, `J032`): nothing was read and nothing is counted. +4. With its turn the submission is counted against your window. Every refusal after this counts. +5. What reads nothing: every whole number exact, at most 16 intents (each of a set's items too), at most 100 items in a set. +6. Its programs are read: the job's own, a set's items and a sweep's points, and every intent's reference, each once however often admission reads it, and all of them within one budget of 22,000,000 steps (statements the reader runs, gates it writes, a generator's loop passes and what it writes out, and every 128 bytes of a value it makes or copies). That is as many as one program may take, its statements and its gates each to the reader's bound, so a job whose own program reads within the reader's bounds reads here. A request whose programs take more is refused, `422` and `J031`, with the command that runs it on your machine. +7. The job is kept and queued, within your and the queue's limits. -Reading happens outside the service's own process, as each job's run does: each request is read by a child, this binary run as `wai-quantum-service read`, sandboxed as a worker is, at most four at a time (a request that waits ten seconds for its turn is refused, `503`, `J032`). A child that holds more than `WAI_QSVC_READ_GB` of memory, or reads for longer than `WAI_QSVC_READ_SECS`, is stopped and the request refused, `422`, `J031`; one that fails is answered `500`. Either way the service runs on. The checks, the error-correction, pulse and variational calls, the estimates, and the verdicts the MCP tools state are read the same way. The reader itself refuses a program nested past its limits (`Q006`: 64 levels in a program's text, 256 in all through the gates and subroutines it calls) and one that would hold more than it takes (`Q130`: a value of more than 64 MiB, or 512 MiB in all), before it is made. +`Retry-After` is the pace at which a refusal that counts would not spend your window (the window over its submissions, at least 5 s), or, past the window, when the next submission opens. A `503` carries it too. + +Reading happens outside the service's own process, as each job's run does: each request is read by a child, this binary run as `wai-quantum-service read`, sandboxed as a worker is. Four are read at a time, and two checks beside them. A client has `WAI_QSVC_CLIENT_READS` (2) requests being read or waiting, across both; at most 32 requests wait, each for ten seconds; past any of these the request is refused, `503` and `J032`. A child that holds more than `WAI_QSVC_READ_GB` (4) of memory, reads for longer than `WAI_QSVC_READ_SECS` (120), or answers with more than `WAI_QSVC_READ_ANSWER_MIB` (256), is stopped and the request refused, `422` and `J031`. One that panics, overflows its stack or aborts is answered `500`, and the service runs on. A client that hangs up ends its child. A request that takes longer than `WAI_QSVC_HEADER_SECS` to send its header section is closed, and one longer than `WAI_QSVC_BODY_SECS` to send its body is refused, `408` (30 each). ## Sending a job again A connection can fail after the service took a job and before its answer came back. To send the job again safely, give `POST /v1/jobs` or `POST /v1/run` an `Idempotency-Key` (the same working group's field): 1 to 255 visible ASCII characters, with no quote or backslash, as a structured field's string or bare. Make it unguessable, a UUID or 128 random bits, since it is kept per client and a client is an address. -- Sent again with the same key and the same request, the job is answered with the job the key names, before any limit, and counts as no submission. +- Sent again with the same key and the same request, the job is answered with the job the key names, before any limit, and counts as no submission. A send that overtakes its first, both still being read, is read and counts, and gets the one job. - The same key with another request is refused, `422` and `J034`. - A key names its job for as long as the job is kept, whatever its status, with one exception: a job the service interrupted. That job's key runs the request again. - The record keeps a digest of the key with the client's name, never the key. @@ -214,15 +217,17 @@ Every job is in the service's store before its submission is answered, and again - a job the service stops again while it runs once more comes back `interrupted`, with the reason, so a job that stops the service whenever it runs does not run for ever; - a queued one is queued again, in its order. +A restart reads no program: a job keeps the plan it was admitted with, and a set its items, so what is queued delays no restart. One too large for the service's limits as they now stand fails, saying why. + Finished jobs are kept for `WAI_QSVC_RETAIN_SECS` (a day here), then dropped. A record names its client by a keyed hash of the address, never the address. [docs/self-host.md](docs/self-host.md#the-job-store) describes the store. -The six checks answer at once, each read and answered in a reading child as a job's admission is. Each counts against the client's submissions, as a job does, before it reads anything (a check refused because every check is running, `503`, is given back its count); then its bounds read its programs within one budget. At most two run at a time, and a check that waits ten seconds for its turn is refused (`503`). Some checks are refused (`422`) with the command that runs them on your own machine: +The six checks answer at once, each read and answered in a reading child as a job's admission is. Each counts against the client's submissions, as a job does, once it has its turn and before it reads anything (a check that waits ten seconds and finds no turn is refused, `503`, having read and spent nothing). At most two run at a time. Its bounds and its answer then read its programs within one budget, each read once: a check that runs the budget out is refused, `422` and `J031`, whatever it had made of it, so a repair of many fixes to a slow program ends at the budget. Some checks are refused (`422`) with the command that runs them on your own machine: - an equivalence check that would apply more than 4·10⁹ gate amplitudes; - an improvement whose candidates' checks would apply more than that; - a thorough search for shortcuts; - a compile of more than 20,000 gates, or with more than 256 rotations to approximate, or a budget that leaves a rotation less than 10⁻¹² (`wai-quantum compile`). A job that asks to be compiled first is held to the same bounds; -- an error-correction call past its bounds here (`wai-quantum qec`): a decode of more than 200,000 shots, of a memory experiment past distance 7, or of more than 2 MiB of circuit or model text; a threshold of more than 16 points or 200,000 shots in all, or at circuit level past distance 7; the distance certificate of `bb:288`; +- an error-correction call past its bounds here (`wai-quantum qec`): a decode of more than 200,000 shots, of more than 1,024 fired detectors, of a memory experiment past distance 7, or of more than 2 MiB of circuit or model text; a threshold of more than 16 points or 200,000 shots in all, or at circuit level past distance 7; the distance certificate of `bb:288`; - a gradient, an optimisation, a VQE or a QAOA run past its bounds here (`wai-quantum gradient`, `optimize`, `vqe`, `qaoa`): more than 16 qubits, more than 500 steps, or about 4·10⁹ gate amplitudes in all; phase estimation on more than 12 counting qubits, or amplitude estimation on more than 8 (`wai-quantum build`); - an estimate of more than 200,000 gates, or with more than 256 distinct rotation angles to synthesise, or a budget that leaves a rotation less than 10⁻¹² (`wai-quantum estimate`). Verifying an estimate re-runs it, within the same bounds. - a pulse check whose run takes more than 2·10¹⁰ of the simulator's multiply-adds, or a pulse call on more than 1 MiB of program (`wai-quantum pulse check`). A pulse job itself is held to the limits every job is. @@ -234,11 +239,13 @@ The six checks answer at once, each read and answered in a reading child as a jo - resources: the language the engine reads (`wai-quantum://docs/language`), its diagnostics, the methods with their accuracy classes (their reach in a job's memory here), and worked examples, each with what it must do; - prompts: writing and verifying a circuit for an intent, explaining a result and its receipt, and repairing a program from its diagnostics. Each names this server's own tools. +Every answer about a program states a verdict. Reading the program again for it costs what reading it cost, so it is read, in a reading child, only for a call that was counted. A poll (`get_quantum_job`) states what its job holds: the result's verdict, or, for a job admitted and not yet ended, that it reads. A tool that says nothing about a program takes no turn. A call refused before it was counted, told to try again, or that finds no turn for its verdict states a verdict that gives the reason and nothing of the program. + A client whose `initialize` says its host draws interactive views (the extension `io.modelcontextprotocol/ui`, version 2026-01-26) gets the engine's answers as views, as from the command line's server: each tool that has one names its `ui://wai-quantum/*` resource, the six views are listed and read, and each answer carries what its view draws in its `_meta`. The service keeps no session, so the client's word is carried in the session id it is given at `initialize` (`wq-views-` and 32 random hex digits, kept nowhere), which a client sends back with every request (`Mcp-Session-Id`); a client that draws no views gets no session id and none of this. For a job still running when the run's wait ends, the result view follows it with `poll_quantum_job`, which the host keeps from the model (`visibility: ["app"]`); a model reads a job with `get_quantum_job`. A client that speaks MCP 2025-11-25 may call the three runs as tasks (`tools/call` with `task`): `initialize` declares `tasks` (`cancel`, and `requests.tools.call`), and each run's `execution.taskSupport` is `optional`. -- **The task is the job.** Its `taskId` is the job's id. `tools/call` answers at once with the task, `working`. A program that does not read makes no task: the answer is an error (`-32602`) whose `data` is the refusal, with its codes and verdict. +- **The task is the job.** Its `taskId` is the job's id. `tools/call` answers at once with the task, `working`. A program that does not read makes no task: the answer is an error (`-32602`) whose `data` is the refusal, with its codes and verdict (of the program where the request was counted, as above). - **Statuses.** `tasks/get` gives the job as a task: `working` while it is queued or running, `completed` once done, `cancelled`, or `failed` for a job that failed or that the service interrupted, with why. Its `ttl` is the service's retention, which a job outlasts from its creation, since it is kept that long after it ends. - **The result.** `tasks/result` gives what `get_quantum_job` gives for the job that ended, naming the task in its `_meta` (`io.modelcontextprotocol/related-task`), an error (`isError`) where the job failed. - Asked while the task still works, by a client that takes `text/event-stream`, the answer is an event stream: kept open, and the answer its one event when the job ends. @@ -329,7 +336,11 @@ These variables set the limits: | `WAI_QSVC_CLIENT_BURST` | 30 | submissions one client may make per window | | `WAI_QSVC_CLIENT_WINDOW_SECS` | 600 | the window | | `WAI_QSVC_READ_GB` | 4 | the memory one request's reading may hold, in its child | -| `WAI_QSVC_READ_SECS` | 300 | the time one request's reading may take | +| `WAI_QSVC_READ_SECS` | 120 | the time one request's reading may take | +| `WAI_QSVC_READ_ANSWER_MIB` | 256 | the most one request's reading may give back to the service | +| `WAI_QSVC_CLIENT_READS` | 2 | how many of one client's requests may be read or wait to be, at once | +| `WAI_QSVC_HEADER_SECS` | 30 | how long a client may take to send a request's header section | +| `WAI_QSVC_BODY_SECS` | 30 | how long a client may take to send a request's body | | `WAI_QSVC_MAX_QUEUE` | 256 | jobs waiting at once | | `WAI_QSVC_RETAIN_SECS` | 86400 | how long finished jobs are kept, across restarts | | `WAI_QSVC_STORE` | `~/.local/share/wai-quantum-service/store` | the job store's directory, or `memory` | diff --git a/wai/wai-quantum-service/docs/scheduler.md b/wai/wai-quantum-service/docs/scheduler.md index 21f147095..c388fdee2 100644 --- a/wai/wai-quantum-service/docs/scheduler.md +++ b/wai/wai-quantum-service/docs/scheduler.md @@ -51,4 +51,4 @@ Before committing an allocation to it, a job can be planned without running: `wa ## Limits and keys -The public service counts a client by its address. A batch system submits from a few login or compute nodes on behalf of many people, so on a shared deployment give the scheduler its own API key (`WAI_QUANTUM_API_KEY`, see [self-host.md](self-host.md#api-keys-for-a-private-deployment)). Its limits are then counted by the key's name, and `WAI_QSVC_CLIENT_ACTIVE`, `WAI_QSVC_CLIENT_GB` and `WAI_QSVC_CLIENT_BURST` size them. Every answer carries the client's submission window in `RateLimit-Policy` and `RateLimit`, and a refusal past it carries `Retry-After`. The adapter exits 75 on such a refusal, which a scheduler can treat as "requeue". +The public service counts a client by its address. A batch system submits from a few login or compute nodes on behalf of many people, so on a shared deployment give the scheduler its own API key (`WAI_QUANTUM_API_KEY`, see [self-host.md](self-host.md#api-keys-for-a-private-deployment)). Its limits are then counted by the key's name, and `WAI_QSVC_CLIENT_ACTIVE`, `WAI_QSVC_CLIENT_GB`, `WAI_QSVC_CLIENT_BURST` and `WAI_QSVC_CLIENT_READS` (how many of its submissions may be read at once) size them. Every answer carries the client's submission window in `RateLimit-Policy` and `RateLimit`, and a refusal past it carries `Retry-After`. The adapter exits 75 on such a refusal, which a scheduler can treat as "requeue". diff --git a/wai/wai-quantum-service/docs/self-host.md b/wai/wai-quantum-service/docs/self-host.md index 4a82dfc22..d58ab2fe8 100644 --- a/wai/wai-quantum-service/docs/self-host.md +++ b/wai/wai-quantum-service/docs/self-host.md @@ -50,6 +50,12 @@ Everything is set by environment variables. | `WAI_QSVC_CLIENT_ACTIVE` | 3 | queued or running jobs one client may hold | | `WAI_QSVC_CLIENT_GB` | the job size | the memory one client's jobs may hold together | | `WAI_QSVC_CLIENT_BURST` | 30 | submissions one client may make in a window | +| `WAI_QSVC_READ_GB` | 4 | the memory one request's reading may take | +| `WAI_QSVC_READ_SECS` | 120 | the time one request's reading may take | +| `WAI_QSVC_READ_ANSWER_MIB` | 256 | the most one request's reading may give back to the service | +| `WAI_QSVC_CLIENT_READS` | 2 | how many of one client's requests may be read or wait to be, at once | +| `WAI_QSVC_HEADER_SECS` | 30 | how long a client may take to send a request's header section | +| `WAI_QSVC_BODY_SECS` | 30 | how long a client may take to send a request's body | | `WAI_QSVC_CLIENT_WINDOW_SECS` | 600 | the window | | `WAI_QSVC_MAX_QUEUE` | 256 | jobs waiting at once | | `WAI_QSVC_SANDBOX` | 1 | 0 runs the macOS worker unsandboxed, for debugging | diff --git a/wai/wai-quantum-service/src/analysis.rs b/wai/wai-quantum-service/src/analysis.rs index 4fa8d0010..84811d119 100644 --- a/wai/wai-quantum-service/src/analysis.rs +++ b/wai/wai-quantum-service/src/analysis.rs @@ -6,8 +6,8 @@ //! bounds: //! //! - each counts against the client's submissions, as a job does; -//! - at most [`AT_ONCE`] run at a time, and a request that waits [`WAIT`] for -//! a turn is refused (503); +//! - at most [`crate::offload::CHECKS_AT_ONCE`] run at a time, and a request +//! that waits [`crate::offload::WAIT`] for a turn is refused (503); //! - an equivalence check that would apply more than [`EQUIV_MAX_WORK`] gate //! amplitudes is refused (422) with the command that runs it on the client's //! own machine. So is an improvement whose candidates' checks would, and so @@ -25,8 +25,9 @@ //! - a job's intents are held to the same bound as the equivalence check //! ([`intent_bounds`]), so a check that would take long is refused before //! the job is queued, with the command that runs it locally; -//! - a repair ([`Check::Repair`]) is bounded by the program's size: it reads -//! the program once a repair, at most `engine::repair::MAX_ROUNDS` times; +//! - a repair ([`Check::Repair`]) reads the program again for each fix, at +//! most `engine::repair::MAX_ROUNDS` times, within the check's one budget +//! of reading: a fix makes a new text, so each round costs what it reads; //! - a conversion between formats ([`Check::Convert`]) reads at most //! [`CONVERT_MAX_BYTES`], and its check is held to the equivalence check's //! bound. @@ -34,29 +35,24 @@ //! Every refusal ends with its code: J031 for what is larger than the //! service takes, J032 when every check is running. //! -//! **In the order of what each step costs.** A check is counted against the -//! client's submissions first, so every check refused after that counts -//! (but for one told to try again, `503`, given back its count); then, in its -//! reading child, its bounds read its programs, each once, within one budget -//! of reading ([`crate::http::READ_STEPS`]), and only then is it answered. +//! **In the order of what each step costs.** A check past the client's +//! window is refused first; then it waits for a turn; with its turn it is +//! counted, so every check refused after that counts, and one told to try +//! again (`503`) has read and spent nothing. Then, in its reading child, its +//! bounds read its programs and its answer is made, each program read once, +//! all within one budget of reading ([`crate::http::READ_STEPS`]). //! //! The critique and the quick shortcuts are bounded by the program's size. //! The tableau, the cut-rank planner's search and the light cones are linear -//! in the gates or close to it. +//! in the gates or close to it. The other engine calls are held to their own +//! work bounds above, and in their child to its memory and its time. use crate::http::App; use serde_json::Value; -use std::sync::{Arc, LazyLock}; -use std::time::Duration; -use tokio::sync::Semaphore; use wai_quantum::engine::synth::lower; use wai_quantum::engine::refusal::{self, tag}; use wai_quantum::engine::{compile, critique, equiv, repair, shortcuts}; -/// Checks that run at once; the rest wait their turn. -pub const AT_ONCE: usize = 2; -/// How long a check waits for a turn before it is refused. -pub const WAIT: Duration = Duration::from_secs(10); /// Gate applications times amplitudes one equivalence check may take here. pub const EQUIV_MAX_WORK: f64 = 4e9; /// The most gates a compile here takes. @@ -72,8 +68,6 @@ pub const ESTIMATE_MAX_ANGLES: usize = 256; /// The smallest share of the synthesis budget a rotation may get here. pub const ESTIMATE_MIN_EACH: f64 = 1e-12; -static TURNS: LazyLock> = LazyLock::new(|| Arc::new(Semaphore::new(AT_ONCE))); - /// The most intents a job states (a set's items each): an intent's /// reference is a program to read and to check against the job's, so their /// number is held before any is read. @@ -203,6 +197,10 @@ pub const QEC_MAX_POINTS: usize = 16; pub const QEC_MAX_CIRCUIT_DISTANCE: u64 = 7; /// The most bytes of circuit or model text a decode reads here. pub const QEC_MAX_TEXT: usize = 2 << 20; +/// The most detectors a decode here is given as fired. Matching grows faster +/// than the square of them, in time and in memory: a thousand take about a +/// second and a hundred MiB. +pub const QEC_MAX_FIRED: usize = 1024; /// Whether the service takes the error-correction call `op` on `body`: the /// reason it does not, with the status to answer, or nothing. The engine's @@ -228,6 +226,9 @@ pub fn qec_bounds(op: &str, body: &Value) -> Result<(), (u16, String)> { if body.get("shots").and_then(Value::as_u64).is_some_and(|s| s > QEC_MAX_SHOTS) { return far(format!("more than {QEC_MAX_SHOTS} shots; {}", local("decode"))); } + if body.get("detectors").and_then(Value::as_array).is_some_and(|d| d.len() > QEC_MAX_FIRED) { + return far(format!("more than {QEC_MAX_FIRED} fired detectors; {}", local("decode"))); + } if body.pointer("/memory/distance").and_then(Value::as_u64).is_some_and(|d| d > QEC_MAX_CIRCUIT_DISTANCE) { return far(format!("a memory experiment past distance {QEC_MAX_CIRCUIT_DISTANCE}; {}", local("decode"))); } @@ -383,30 +384,17 @@ pub fn call(kind: &str, op: &str, body: &Value, key: Option<&str>) -> Result Result { - app.service.count(who).map_err(|crate::jobs::Refused(code, why)| (code, why))?; - let r = async { - let _turn = turn().await?; - crate::offload::read(&app.service.config, read).await - } - .await; - if matches!(r, Err((503, _))) { - app.service.refund(who); - } - r -} - -/// A turn among the [`AT_ONCE`] checks, held while one runs. -async fn turn() -> Result { - match tokio::time::timeout(WAIT, Arc::clone(&TURNS).acquire_owned()).await { - Ok(Ok(t)) => Ok(t), - _ => Err((503, tag(refusal::BUSY, format!("{AT_ONCE} checks are running; try again shortly")))), - } + let refused = |crate::jobs::Refused(code, why)| (code, why); + app.service.window_open(who).map_err(refused)?; + let turn = app.service.gate.turn(crate::offload::Pool::Checks, who).await?; + app.service.count(who).map_err(refused)?; + crate::offload::read(&app.service.config, turn, read).await } /// Whether the service takes `check` on `body`: the reason it does not, with @@ -533,8 +521,8 @@ pub fn answer(check: Check, body: &Value) -> Value { /// `check` on `body` for `who`, within the bounds above: the answer, or the /// status and reason it was refused. -/// It is counted first, then held to its bounds and answered in a reading -/// child ([`bounded`]). +/// It is held to its bounds and answered in a reading child, within one +/// budget ([`bounded`]). pub async fn analyse(app: &App, who: &str, check: Check, body: Value) -> Result { bounded(app, who, crate::offload::Read::Check { check: check.name().into(), body }).await } @@ -544,6 +532,38 @@ mod tests { use super::*; use serde_json::json; + /// A check that has run its budget out has no answer, whatever its + /// reading made of that: a program it could not read is not one of no + /// work. Each bound that reads refuses, under a budget too small for it, + /// where each used to read the refusal as nothing to do. + #[test] + fn a_budget_spent_is_a_refusal_whatever_the_check_made_of_it() { + let wide = |k: u32| format!("OPENQASM 3.0;\nqubit[12] q;\nint a = 0;\nfor int i in [0:99999] {{ a += 1; }}\nrz(0.1) q[0];\n// {k}\n"); + for (what, read) in [ + ("equivalent", Box::new(|| bounds(Check::Equivalent, &serde_json::json!({ "a": wide(0), "b": wide(1) }))) as Box Result<(), (u16, String)>>), + ("intent", Box::new(|| intent_bounds(&serde_json::json!({ "qasm": wide(2), "intent": { "equivalent": wide(3) } })))), + ("improve", Box::new(|| bounds(Check::Improve, &serde_json::json!({ "qasm": wide(4) })))), + ("gradient", Box::new(|| variational_bounds("gradient", &serde_json::json!({ "qasm": wide(5) })))), + ("convert", Box::new(|| convert_bounds(&serde_json::json!({ "program": wide(6), "from": "qasm", "to": "qasm" })))), + ] { + let e = wai_quantum::qasm::within_steps(1_000, || crate::http::within_budget(read)).unwrap_err(); + assert!(e.0 == 422 && e.1.ends_with("[J031]") && e.1.contains("steps"), "{what}: {e:?}"); + } + // An answer made under it is held the same way. + let e = wai_quantum::qasm::within_steps(1_000, || crate::http::within_budget(|| Ok(answer(Check::Critique, &serde_json::json!({ "qasm": wide(7) }))))).unwrap_err(); + assert!(e.0 == 422 && e.1.ends_with("[J031]"), "{e:?}"); + } + + /// Matching grows faster than the square of the detectors fired: a decode + /// is held to how many it is given, and told so with the local command. + #[test] + fn a_decode_is_held_to_the_detectors_it_is_given() { + let body = |n: usize| serde_json::json!({ "model": "error(0.1) D0 D1\n", "detectors": vec![0; n] }); + assert!(qec_bounds("decode", &body(QEC_MAX_FIRED)).is_ok()); + let e = qec_bounds("decode", &body(QEC_MAX_FIRED + 1)).unwrap_err(); + assert!(e.0 == 422 && e.1.contains("wai-quantum qec decode") && e.1.ends_with("[J031]"), "{e:?}"); + } + #[test] fn the_bounds_refuse_what_would_take_long_and_say_where_it_runs() { let small = json!({ "qasm": "qubit[2] q; h q[0]; h q[0]; cx q[0], q[1];" }); diff --git a/wai/wai-quantum-service/src/discovery.rs b/wai/wai-quantum-service/src/discovery.rs index 3583f7ab5..a3aee04eb 100644 --- a/wai/wai-quantum-service/src/discovery.rs +++ b/wai/wai-quantum-service/src/discovery.rs @@ -248,10 +248,11 @@ pub fn openapi(server: &str) -> Value { "200": { "description": "The finished job, or with dry_run the plan (DryRun)", "headers": { "RateLimit": { "$ref": "#/components/headers/RateLimit" }, "RateLimit-Policy": { "$ref": "#/components/headers/RateLimit-Policy" } }, "content": { "application/json": { "schema": { "oneOf": [{ "$ref": "#/components/schemas/Job" }, { "$ref": "#/components/schemas/DryRun" }] } } } }, "202": { "description": "Still running: poll GET /v1/jobs/{id}", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Job" } } } }, "400": { "description": "A program, job or Idempotency-Key that does not read", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, - "422": { "description": "Larger than one job may be, or an Idempotency-Key sent before with another request (J034)" }, + "422": { "description": "Larger than one job may be (J031: its plan, or its reading past read_steps, the reading child's memory or its time), or an Idempotency-Key sent before with another request (J034)" }, "401": { "description": "A private deployment's key missing or not one it holds (J033)" }, "429": { "description": "Past the per-client limits (with Retry-After)" }, - "503": { "description": "The queue is full" } + "503": { "description": "The queue is full, or no turn to read it came (J032): nothing was read, and nothing counted" }, + "500": { "description": "Its reading failed inside the service: nothing was admitted, and the service runs on" } } } }, "/v1/jobs": { "post": { @@ -262,10 +263,11 @@ pub fn openapi(server: &str) -> Value { "202": { "description": "Accepted, and kept in the service's store before this answer; sent again with its Idempotency-Key, the key's job", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Job" } } } }, "200": { "description": "With dry_run: the plan, nothing run", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/DryRun" } } } }, "400": { "description": "A program, job or Idempotency-Key that does not read", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, - "422": { "description": "Larger than one job may be, or an Idempotency-Key sent before with another request (J034)" }, + "422": { "description": "Larger than one job may be (J031: its plan, or its reading past read_steps, the reading child's memory or its time), or an Idempotency-Key sent before with another request (J034)" }, "401": { "description": "A private deployment's key missing or not one it holds (J033)" }, "429": { "description": "Past the per-client limits (with Retry-After)" }, - "503": { "description": "The queue is full" } + "503": { "description": "The queue is full, or no turn to read it came (J032): nothing was read, and nothing counted" }, + "500": { "description": "Its reading failed inside the service: nothing was admitted, and the service runs on" } } } }, "/v1/jobs/{id}": { @@ -284,22 +286,22 @@ pub fn openapi(server: &str) -> Value { "/v1/verify": { "post": { "summary": "Whether a result's receipt seals that request and that result and every rule of its form holds ({\"valid\", \"reasons\", \"key\", \"form\"}); an estimate is also re-run, and each of its resource receipts checked ({\"valid\", \"signed\", \"receipts\", \"reproduced\"}); a set's receipt is checked over its head, every item as its request gives it, each item's own receipt, the root and each path ({\"valid\", \"signed\", \"derived\", \"items_sealed\", \"root\"})", "requestBody": { "required": true, "content": { "application/json": { "schema": { "type": "object", "required": ["request", "result"], "properties": { "request": { "$ref": "#/components/schemas/JobRequest" }, "result": { "type": "object" } } } } } }, - "responses": { "200": { "description": "{\"valid\": bool, \"reasons\": [string]}" }, "400": { "description": "The body is not JSON, not a job, or has no receipt" } } + "responses": { "200": { "description": "{\"valid\": bool, \"reasons\": [string]}" }, "400": { "description": "The body is not JSON, not a job, or has no receipt" }, "422": { "description": "A set whose sweep takes more to write out again than a request may read (J031)" }, "429": { "description": "A set, counted as a check is, past the per-client limits (with Retry-After)" }, "503": { "description": "A set, with every check running or your share of them (J032)" } } } }, "/v1/critique": { "post": { "summary": "A program's measures and smells, each with its lines", "requestBody": { "required": true, "content": { "application/json": { "schema": prog.clone() } } }, - "responses": { "200": { "description": "The critique, or {\"ok\": false, \"error\", \"line\"} for a program that does not read" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running" } } + "responses": { "200": { "description": "The critique, or {\"ok\": false, \"error\", \"line\"} for a program that does not read" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" } } } }, "/v1/equivalent": { "post": { "summary": "Whether two programs are the same circuit up to a global phase, and how that was decided", "requestBody": { "required": true, "content": { "application/json": { "schema": { "type": "object", "required": ["a", "b"], "properties": { "a": { "type": "string" }, "b": { "type": "string" } } } } } }, - "responses": { "200": { "description": "{\"verdict\": \"equivalent\" | \"different\" | \"unchecked\", \"how\", \"why\"}" }, "422": { "description": "A check longer than the service takes; the reason names the command that runs it locally" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running" } } + "responses": { "200": { "description": "{\"verdict\": \"equivalent\" | \"different\" | \"unchecked\", \"how\", \"why\"}" }, "422": { "description": "A check longer than the service takes; the reason names the command that runs it locally" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" } } } }, "/v1/convert": { "post": { "summary": "A program carried between formats (OpenQASM 3, QIR as LLVM text, jeff as base64), the output read back and held to it by the equivalence ladder", "requestBody": { "required": true, "content": { "application/json": { "schema": { "type": "object", "required": ["to"], "properties": { "program": { "type": "string", "description": "The program's text: OpenQASM or QIR." }, "program_base64": { "type": "string", "description": "The program's bytes as base64: jeff." }, "from": { "enum": ["qasm", "qir", "jeff"] }, "to": { "enum": ["qasm", "qir", "jeff"] }, "profile": { "enum": ["base", "adaptive"] }, "check": { "type": "boolean" } } } } } }, - "responses": { "200": { "description": "{\"ok\": true, \"format\", \"from\", \"to\", \"program\" | \"program_base64\", \"qasm\", \"notes\", \"read\", \"written\", \"check\"}, or the refusal {\"error\", \"code\", \"codes\", \"outcome\", \"meaning\"} with the reader's code (Q700 to Q721) and place" }, "422": { "description": "A program or a check larger than the service takes" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running" } } + "responses": { "200": { "description": "{\"ok\": true, \"format\", \"from\", \"to\", \"program\" | \"program_base64\", \"qasm\", \"notes\", \"read\", \"written\", \"check\"}, or the refusal {\"error\", \"code\", \"codes\", \"outcome\", \"meaning\"} with the reader's code (Q700 to Q721) and place" }, "422": { "description": "A program or a check larger than the service takes" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" } } } }, "/v1/improve": { "post": { "summary": "The program with its smells fixed, each candidate checked the same circuit", @@ -319,7 +321,7 @@ pub fn openapi(server: &str) -> Value { "400": { "description": "A program or options that do not read" }, "422": { "description": "A compile the service does not take (too many gates or rotations, or ε too fine for them: the reason names the local command), or one the engine refuses" }, "429": { "description": "Past the per-client limits" }, - "503": { "description": "Every check is running" } + "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" } } } }, "/v1/estimate": { "post": { @@ -330,13 +332,13 @@ pub fn openapi(server: &str) -> Value { "400": { "description": "A request, program or document that does not read" }, "422": { "description": "An estimate the service does not take (too many gates or distinct rotation angles, or a budget too fine for its rotations: the reason names the local command), or one the engine refuses (its code, QE02 to QE08)" }, "429": { "description": "Past the per-client limits" }, - "503": { "description": "Every check is running" } + "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" } } } }, "/v1/repair": { "post": { "summary": "The program repaired on the lines its diagnostics flag alone: the mechanical fixes, a proposed repair held to those lines, and what needs a person", "requestBody": { "required": true, "content": { "application/json": { "schema": { "type": "object", "required": ["qasm"], "properties": { "qasm": { "type": "string" }, "proposed": { "type": "string", "description": "A repaired version: taken only if it changes no line the diagnostics do not flag." }, "max_iterations": { "type": "integer", "minimum": 1 }, "dyadic_bits": { "type": "integer", "minimum": 1, "maximum": 32 }, "sizes": { "type": "object", "additionalProperties": { "type": "integer" } }, "diagnostics": { "type": "array" }, "rules": { "type": "boolean" } } } } } }, - "responses": { "200": { "description": "{\"ok\", \"changed\", \"qasm\", \"repairs\": [{\"code\", \"rule\", \"by\", \"flagged\", \"edits\"}], \"needs_person\", \"verdict\"}, or {\"ok\": false, \"error\", \"refusal\"} for a proposal rejected (J021)" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running" } } + "responses": { "200": { "description": "{\"ok\", \"changed\", \"qasm\", \"repairs\": [{\"code\", \"rule\", \"by\", \"flagged\", \"edits\"}], \"needs_person\", \"verdict\"}, or {\"ok\": false, \"error\", \"refusal\"} for a proposal rejected (J021)" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" } } } }, "/mcp": { "post": { "summary": "The Model Context Protocol endpoint (streamable HTTP, JSON responses): tools to run circuits and sets, plan a job without running it, read jobs, verify receipts, list examples, and critique, compare, improve, price, compile, estimate and repair programs", "responses": { "200": { "description": "A JSON-RPC response" } } } } } @@ -349,7 +351,7 @@ pub fn openapi(server: &str) -> Value { /// The pulse calls' paths. fn pulse_paths() -> Vec<(&'static str, Value)> { - let responses = |ok: &str| json!({ "200": { "description": ok }, "400": { "description": "A body that does not read" }, "422": { "description": "A program the pulse reader refuses (its Q code), a request the engine refuses (J040, J041), or a check larger than the service takes (J031, naming the local command)" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running" } }); + let responses = |ok: &str| json!({ "200": { "description": ok }, "400": { "description": "A body that does not read" }, "422": { "description": "A program the pulse reader refuses (its Q code), a request the engine refuses (J040, J041), or a check larger than the service takes (J031, naming the local command)" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" } }); vec![ ("/v1/pulse/schedule", json!({ "post": { "summary": "A pulse program's schedule on frames: every play, delay, phase and frequency change, capture and barrier at its sample, each waveform's samples, the gate calls, and the digests a pulse run's receipt binds; sealed with the service's key", @@ -367,7 +369,7 @@ fn pulse_paths() -> Vec<(&'static str, Value)> { /// The variational and algorithm library's paths, apart, so the document's /// macro stays within its depth. fn variational_paths() -> Vec<(&'static str, Value)> { - let responses = |ok: &str| json!({ "200": { "description": ok }, "400": { "description": "A body that does not read" }, "422": { "description": "A request the engine refuses (J050 params, J051 a gradient, J052 an optimisation, J053 a builder, Q129 a parameter it cannot follow) or larger than the service takes (J031, naming the local command)" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running" } }); + let responses = |ok: &str| json!({ "200": { "description": ok }, "400": { "description": "A body that does not read" }, "422": { "description": "A request the engine refuses (J050 params, J051 a gradient, J052 an optimisation, J053 a builder, Q129 a parameter it cannot follow) or larger than the service takes (J031, naming the local command)" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" } }); let body = || json!({ "required": true, "content": { "application/json": { "schema": { "type": "object" } } } }); vec![ ("/v1/gradient", json!({ "post": { @@ -401,7 +403,7 @@ fn variational_paths() -> Vec<(&'static str, Value)> { /// The error-correction calls' paths, apart, so the document's macro stays /// within its depth. fn qec_paths() -> Vec<(&'static str, Value)> { - let responses = |ok: &str| json!({ "200": { "description": ok }, "400": { "description": "A body that does not read" }, "422": { "description": "A request the engine refuses (J009) or larger than the service takes (J031, naming the local command)" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running" } }); + let responses = |ok: &str| json!({ "200": { "description": ok }, "400": { "description": "A body that does not read" }, "422": { "description": "A request the engine refuses (J009) or larger than the service takes (J031, naming the local command)" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" } }); vec![ ("/v1/qec/code", json!({ "post": { "summary": "An error-correcting code (surface:D, toric:L, steane, color:D, bb:72 to bb:288, gross): its checks, stabilizers, k, layout, the decoders it takes, and its distance certificate (a witness and every lighter support ruled out); with a certificate, its check; sealed with the service's key", diff --git a/wai/wai-quantum-service/src/http.rs b/wai/wai-quantum-service/src/http.rs index 07d597185..0fe8987d2 100644 --- a/wai/wai-quantum-service/src/http.rs +++ b/wai/wai-quantum-service/src/http.rs @@ -305,13 +305,15 @@ pub(crate) fn cheap_bounds(request: &Value) -> Result<(), (u16, String)> { } /// The steps one request's programs may take to read, all together: each -/// statement the reader elaborates, each gate it writes and each pass of a -/// loop a sweep is written out with (`wai_quantum::qasm::within_steps`). As -/// many as one program may take on reading, so a job whose own program reads -/// within the reader's bound reads here, and a request whose references or -/// items would read for longer is refused within that bound's time. Each -/// program is read once, however often its admission reads it. -pub const READ_STEPS: u64 = 20_000_000; +/// statement the reader elaborates, each gate it writes, what is made or +/// copied by its size, and each pass of a loop a sweep is written out with +/// (`wai_quantum::qasm::within_steps`). As many as one program may take on +/// reading, its statements and its gates each to the reader's bound, so a +/// job whose own program reads within the reader's bounds reads here, and a +/// request whose references or items would read for longer is refused within +/// about that bound's time. Each program is read once, however often its +/// admission reads it. +pub const READ_STEPS: u64 = wai_quantum::qasm::MAX_STEPS + wai_quantum::qasm::MAX_OPS as u64; /// The refusal of a request whose programs read past [`READ_STEPS`]. fn read_too_long() -> (u16, String) { @@ -325,16 +327,15 @@ fn read_too_long() -> (u16, String) { } /// `read` within one budget of [`READ_STEPS`], each program it reads read -/// once; a refusal for the budget is [`read_too_long`]. +/// once. A budget spent is [`read_too_long`], whatever `read` made of it: what +/// ran out reading tells nothing of the program, so it is no answer, and no +/// bound a check held the program to has been met. pub(crate) fn within_budget(read: impl FnOnce() -> Result) -> Result { let (r, spent) = wai_quantum::qasm::within_steps(READ_STEPS, || { let r = read(); (r, wai_quantum::qasm::steps_left() == Some(0)) }); - match r { - Err(_) if spent => Err(read_too_long()), - r => r, - } + if spent { Err(read_too_long()) } else { r } } /// `request` planned: one job's plan, or a set's plan with its items' (each @@ -377,80 +378,53 @@ fn admitted(v: Value) -> Result { let plan: catalog::Plan = serde_json::from_value(v["plan"].clone()).map_err(bad)?; let work = match v["items"].as_array() { None => Work::One, - Some(items) => Work::Set { - items: items.iter().map(|i| sets::Item { request: i["request"].clone(), point: Some(i["point"].clone()).filter(|p| !p.is_null()) }).collect(), - plans: serde_json::from_value(v["plans"].clone()).map_err(bad)?, - }, + Some(_) => Work::Set { items: crate::jobs::items_of(&v["items"]), plans: serde_json::from_value(v["plans"].clone()).map_err(bad)? }, }; Ok((request, Ok((plan, work)))) } /// [`admit`], in the order of what each step costs: /// -/// 1. a request sent again with its key is the key's job, whatever planning -/// it now would give, and counts as nothing; one given in another format -/// (`"source"`) under a key that names a job of the client's is read from -/// that format first, uncounted, and another request under the key counts; +/// 1. a request sent again with its key is the key's job, as it was received +/// or as it read, whatever planning it now would give: it reads nothing, +/// takes no turn and counts as nothing ([`Service::named`]); /// 2. what reads nothing and refuses whatever is sent (the client's jobs at -/// once, a full queue) refuses uncounted, so a client told to try again -/// shortly spends nothing doing so; -/// 3. the submission is counted against the client's window, so every -/// request refused after this counts, but for one it is told to send again -/// (`429` for its memory, `503`), which is given back its count; -/// 4. the checks that read nothing ([`cheap_bounds`]); -/// 5. its programs read, each once, within one budget ([`plan_job`]), in a +/// once, a full queue, a spent window) refuses uncounted, so a client told +/// to wait spends nothing waiting; +/// 3. it waits for a turn (`503` if none comes: nothing read, nothing spent); +/// 4. with its turn the submission is counted against the client's window, so +/// every request refused after this counts; +/// 5. the checks that read nothing ([`cheap_bounds`]); +/// 6. its programs read, each once, within one budget ([`plan_job`]), in a /// child whose memory and time are capped ([`crate::offload`]); -/// 6. the job kept and queued, within the client's and the queue's limits. +/// 7. the job kept and queued, within the client's and the queue's limits. pub(crate) async fn admit_to(service: &Service, who: &str, request: Value, key: Option<&str>) -> Result { let refused = |Refused(code, why)| (code, why); let key = key.map(Key); - let job_bytes = service.config.job_mib << 20; - let sourced = request.get("source").is_some_and(|s| !s.is_null()); - if let Some(k) = key { - if !sourced { - if let Some(id) = service.named(who, k, &request).map_err(refused)? { - return Ok(id); - } - } else if service.key_live(who, k) { - let read = crate::offload::read(&service.config, crate::offload::Read::Admit { request: request.clone(), job_bytes, plan: false }).await?; - match service.named(who, k, &read["request"]) { - Ok(Some(id)) => return Ok(id), - Ok(None) => {} - Err(r) => { - let _ = service.count(who); - return Err(refused(r)); - } - } - } + if let Some(k) = key + && let Some(id) = service.named(who, k, &request).map_err(refused)? + { + return Ok(id); } if let Some(r) = service.refused_before_reading(who) { return Err(refused(r)); } + service.window_open(who).map_err(refused)?; + let turn = service.gate.turn(crate::offload::Pool::Reading, who).await?; service.count(who).map_err(refused)?; - let r = async { - cheap_bounds(&request)?; - let read = crate::offload::read(&service.config, crate::offload::Read::Admit { request, job_bytes, plan: true }).await?; - let (request, planned) = admitted(read)?; - if sourced - && let Some(k) = key - && let Some(id) = service.named(who, k, &request).map_err(refused)? - { - service.refund(who); - return Ok(id); - } - let (plan, work) = planned?; - let (id, new) = service.submit_counted(who, request, plan, work, key).map_err(refused)?; - if !new { - // Sent at once with another under its key, which made the job. - service.refund(who); - } - Ok(id) + cheap_bounds(&request)?; + let received = crate::jobs::sourced(&request).then(|| crate::jobs::received_digest(&request)); + let read = crate::offload::read(&service.config, turn, crate::offload::Read::Admit { request, job_bytes: service.config.job_mib << 20 }).await?; + let (request, planned) = admitted(read)?; + // Sent in another form under a key: it may read as the key's job. + if received.is_some() + && let Some(k) = key + && let Some(id) = service.named(who, k, &request).map_err(refused)? + { + return Ok(id); } - .await; - if let Err((429 | 503, _)) = r { - service.refund(who); - } - r + let (plan, work) = planned?; + service.submit_counted(who, request, plan, work, key, received).map_err(refused) } /// The `Idempotency-Key` a request gives, if it gives one: a structured @@ -485,24 +459,16 @@ pub(crate) fn is_dry_run(request: &Value) -> bool { /// before anything is read; it is read as a job is ([`admit_to`]), in a child. pub(crate) async fn dry_run(app: &App, who: &str, request: &Value) -> Result { let service = &app.service; - service.count(who).map_err(|Refused(code, why)| (code, why))?; - let read = crate::offload::read(&service.config, crate::offload::Read::Admit { request: request.clone(), job_bytes: service.config.job_mib << 20, plan: true }).await; - let read = match read.and_then(admitted) { - Ok(r) => r, - Err(e) => { - if matches!(e.0, 429 | 503) { - service.refund(who); - } - return Err(e); - } - }; - dry_run_now(service, who, read) + let refused = |Refused(code, why)| (code, why); + service.window_open(who).map_err(refused)?; + let turn = service.gate.turn(crate::offload::Pool::Reading, who).await?; + service.count(who).map_err(refused)?; + let read = crate::offload::read(&service.config, turn, crate::offload::Read::Admit { request: request.clone(), job_bytes: service.config.job_mib << 20 }).await?; + dry_run_now(service, who, admitted(read)?) } /// [`dry_run`]'s answer, its submission counted and its request read. -fn dry_run_now(service: &Service, who: &str, (expanded, planned): Admitted) -> Result { - let request = &expanded; - let _ = request; +fn dry_run_now(service: &Service, who: &str, (_, planned): Admitted) -> Result { let window = service.window(who); let (active, held) = service.holding(who); let mut out = json!({ @@ -748,9 +714,15 @@ async fn verify(State(app): State, ConnectInfo(peer): ConnectInfo error(code, &why), }; } - // A set: its receipt over its head, every item, the root and the paths (§8.6). + // A set: its receipt over its head, every item, the root and the paths + // (§8.6). A sweep's programs are written out again to check its items, so + // it is counted and read as a check is. if v["result"]["format"] == sets::FORMAT { - return reply(200, wai_quantum::engine::job::verdict_of_text(text).unwrap_or_else(|e| json!({ "valid": false, "error": e }))); + let who = client(&app, peer, &headers); + return match crate::analysis::bounded(&app, &who, crate::offload::Read::Verify { text: text.to_string() }).await { + Ok(checked) => reply(200, checked), + Err((code, why)) => error(code, &why), + }; } let (Some(_), Some(result)) = (v.get("request"), v.get("result")) else { return error(400, "expected {\"request\": ..., \"result\": ...}, the result carrying its receipt"); @@ -1014,10 +986,53 @@ fn rate_headers(app: &App, who: &str, r: &mut Response) { if let Ok(v) = HeaderValue::from_str(&format!("\"submissions\";r={};t={}", w.remaining, w.reset_secs)) { h.insert("ratelimit", v); } + // Sent again at this pace, a refusal that counts never spends the window. + let pace = (window / w.limit.max(1) as u64).max(5); if r.status() == StatusCode::TOO_MANY_REQUESTS - && let Ok(v) = HeaderValue::from_str(&(if w.remaining == 0 { w.reset_secs.max(1) } else { 5 }).to_string()) + && let Ok(v) = HeaderValue::from_str(&(if w.remaining == 0 { w.reset_secs.max(1) } else { pace }).to_string()) { r.headers_mut().insert("retry-after", v); + } else if r.status() == StatusCode::SERVICE_UNAVAILABLE { + r.headers_mut().insert("retry-after", HeaderValue::from_static("5")); + } +} + +/// A request with its body read, within `wait`: a client that stops sending +/// holds its connection no longer than that (`408`). Only then is it handled. +async fn body_in_time(State(wait): State, request: axum::extract::Request, next: axum::middleware::Next) -> Response { + let (parts, body) = request.into_parts(); + match tokio::time::timeout(wait, axum::body::to_bytes(body, MAX_BODY + 1)).await { + Ok(Ok(bytes)) => next.run(axum::extract::Request::from_parts(parts, axum::body::Body::from(bytes))).await, + Ok(Err(_)) => error(413, &format!("the body is larger than the {MAX_BODY} bytes a request may be, or it broke off")), + Err(_) => error(408, &format!("the body did not arrive within {} s", wait.as_secs())), + } +} + +/// Serves `router` on `listener` until the service stops. Each connection is +/// held to `config.header_wait` for a request's header section, and the +/// router to `body_wait` for its body, so a client that stops sending holds a +/// connection for no longer than those (axum's own loop takes no such limit). +pub async fn serve(listener: tokio::net::TcpListener, router: Router, header_wait: Duration) { + let service = hyper_util::service::TowerToHyperService::new(router); + loop { + let (stream, peer) = match listener.accept().await { + Ok(accepted) => accepted, + Err(e) => { + // Out of descriptors, say: those held free themselves. + eprintln!("accepting a connection: {e}"); + tokio::time::sleep(Duration::from_secs(1)).await; + continue; + } + }; + let service = service.clone(); + tokio::spawn(async move { + let with_peer = hyper::service::service_fn(move |mut r: hyper::Request| { + r.extensions_mut().insert(ConnectInfo(peer)); + hyper::service::Service::call(&service, r) + }); + let io = hyper_util::rt::TokioIo::new(stream); + let _ = hyper::server::conn::http1::Builder::new().timer(hyper_util::rt::TokioTimer::new()).header_read_timeout(header_wait).serve_connection(io, with_peer).await; + }); } } @@ -1056,6 +1071,7 @@ pub fn router(app: Shared) -> Router { .route("/mcp", post(mcp).get(mcp_get)) .route("/", get(index)) .fallback(|| async { error(404, "no such endpoint; GET /v1 lists them") }) + .layer(axum::middleware::from_fn_with_state(app.service.config.body_wait, body_in_time)) .layer(axum::middleware::from_fn_with_state(Arc::clone(&app), guard)) .layer(DefaultBodyLimit::max(MAX_BODY)) .with_state(app) @@ -1086,7 +1102,11 @@ mod tests { per_client_window: Duration::from_secs(600), read_mib: 4096, read_timeout: Duration::from_secs(300), + read_answer_mib: 256, read_in_child: false, + per_client_reads: 2, + header_wait: Duration::from_secs(30), + body_wait: Duration::from_secs(30), }) } @@ -1126,6 +1146,65 @@ mod tests { assert_eq!(left(&s), 96); } + /// A job's own program that reads within the reader's bounds reads here: + /// the budget covers its statements and its gates, each to its bound. + #[test] + fn the_budget_is_as_large_as_one_program_may_be() { + assert!(super::READ_STEPS >= wai_quantum::qasm::MAX_STEPS + wai_quantum::qasm::MAX_OPS as u64); + } + + /// With no turn to be had, a request is told so within the wait, and has + /// spent nothing, by any route that reads. Past its window it is told so + /// at once, without waiting. + #[tokio::test] + async fn a_request_with_no_turn_is_refused_in_time_and_spends_nothing() { + let s = service(2); + let mut held = Vec::new(); + for i in 0..crate::offload::AT_ONCE { + held.push(s.gate.turn(crate::offload::Pool::Reading, &format!("other {i}")).await.ok().unwrap()); + } + let t = std::time::Instant::now(); + let (code, why) = admit_to(&s, "a", json!({ "qasm": "OPENQASM 3.0; qubit q; h q;", "shots": 1 }), None).await.unwrap_err(); + assert!(code == 503 && why.ends_with("[J032]"), "{code} {why}"); + assert!(t.elapsed() < crate::offload::WAIT + Duration::from_secs(5), "{:?}", t.elapsed()); + assert_eq!(s.window("a").remaining, 2); + s.count("a").unwrap(); + s.count("a").unwrap(); + let t = std::time::Instant::now(); + let (code, why) = admit_to(&s, "a", json!({ "qasm": "OPENQASM 3.0; qubit q; h q;", "shots": 1 }), None).await.unwrap_err(); + assert!(code == 429 && why.ends_with("[J030]") && t.elapsed() < Duration::from_secs(1), "{code} {why}"); + } + + /// A job in another format sent again under its key is matched as it was + /// received: no reading, no turn, no count, whatever the window says. A + /// resend that is another request under the key counts as one, and reads. + /// There is no reading to be had for free, so nothing is given back. + #[tokio::test] + async fn a_source_job_sent_again_under_its_key_reads_nothing_and_counts_nothing() { + let s = service(2); + let job = json!({ "source": { "format": "qasm", "program": "OPENQASM 3.0;\nqubit[1] q;\nbit[1] c;\nh q[0];\nc = measure q;\n" }, "shots": 10 }); + let id = admit_to(&s, "a", job.clone(), Some("k")).await.unwrap(); + assert_eq!(s.window("a").remaining, 1); + let mut held = Vec::new(); + for i in 0..crate::offload::AT_ONCE { + held.push(s.gate.turn(crate::offload::Pool::Reading, &format!("other {i}")).await.ok().unwrap()); + } + s.count("a").unwrap(); + let t = std::time::Instant::now(); + assert_eq!(admit_to(&s, "a", job.clone(), Some("k")).await.unwrap(), id); + assert!(t.elapsed() < Duration::from_secs(1), "{:?}", t.elapsed()); + drop(held); + // Another program in the same format under the key is read to tell, + // and counts: the key names one job. + let s = service(3); + admit_to(&s, "a", job.clone(), Some("k")).await.unwrap(); + assert_eq!(s.window("a").remaining, 2); + let other = json!({ "source": { "format": "qasm", "program": "OPENQASM 3.0;\nqubit[1] q;\nbit[1] c;\nx q[0];\nc = measure q;\n" }, "shots": 10 }); + let (code, why) = admit_to(&s, "a", other, Some("k")).await.unwrap_err(); + assert!(code == 422 && why.ends_with("[J034]"), "{code} {why}"); + assert_eq!(s.window("a").remaining, 1); + } + /// A client told to try again shortly spends nothing doing so: refused /// for its jobs at once before anything is counted or read, its window /// stays whole; and a job sent again under its key, once its window is diff --git a/wai/wai-quantum-service/src/jobs.rs b/wai/wai-quantum-service/src/jobs.rs index ee527071b..35a808d16 100644 --- a/wai/wai-quantum-service/src/jobs.rs +++ b/wai/wai-quantum-service/src/jobs.rs @@ -99,9 +99,19 @@ pub struct Config { /// ([`crate::offload`]): the cap on its reading child. pub read_mib: u64, pub read_timeout: Duration, + /// The most a reading may answer with, MiB: a sweep's items, written out, + /// come into the service's own memory. + pub read_answer_mib: u64, /// Whether requests are read in a child (the service), or on a thread of /// the process (the service's own tests, whose binary is not the service). pub read_in_child: bool, + /// How many of one client's requests may be read at once, waiting + /// included ([`crate::offload::Gate`]). + pub per_client_reads: usize, + /// How long a client may take over a request's header section, and over + /// its body: past them the connection is closed, or refused `408`. + pub header_wait: Duration, + pub body_wait: Duration, } fn env_num(key: &str, default: T) -> T { @@ -115,7 +125,11 @@ impl Config { /// `WAI_QSVC_CLIENT_ACTIVE` (3), `WAI_QSVC_CLIENT_GB` (the job size), /// `WAI_QSVC_CLIENT_BURST` (30 in `WAI_QSVC_CLIENT_WINDOW_SECS`, 600), /// `WAI_QSVC_SANDBOX` (1; 0 runs workers unsandboxed), `WAI_QSVC_READ_GB` - /// (4) and `WAI_QSVC_READ_SECS` (300), what one request's reading may take. + /// (4), `WAI_QSVC_READ_SECS` (120) and `WAI_QSVC_READ_ANSWER_MIB` (256), + /// what one request's reading may take and give, + /// `WAI_QSVC_CLIENT_READS` (2), how many a client may have read at once, + /// `WAI_QSVC_HEADER_SECS` and `WAI_QSVC_BODY_SECS` (30 each), how long a + /// client may take to send a request. pub fn from_env() -> Config { let job_gb = env_num("WAI_QSVC_JOB_GB", 72u64); Config { @@ -132,7 +146,11 @@ impl Config { per_client_burst: env_num("WAI_QSVC_CLIENT_BURST", 30usize), per_client_window: Duration::from_secs(env_num("WAI_QSVC_CLIENT_WINDOW_SECS", 600u64)), read_mib: (env_num("WAI_QSVC_READ_GB", 4f64).max(0.0) * 1024.0) as u64, - read_timeout: Duration::from_secs(env_num("WAI_QSVC_READ_SECS", 300u64)), + read_timeout: Duration::from_secs(env_num("WAI_QSVC_READ_SECS", 120u64)), + read_answer_mib: env_num("WAI_QSVC_READ_ANSWER_MIB", 256u64), + per_client_reads: env_num("WAI_QSVC_CLIENT_READS", 2usize).max(1), + header_wait: Duration::from_secs(env_num("WAI_QSVC_HEADER_SECS", 30u64).max(1)), + body_wait: Duration::from_secs(env_num("WAI_QSVC_BODY_SECS", 30u64).max(1)), read_in_child: !cfg!(test), } } @@ -206,6 +224,8 @@ pub enum Work { pub struct Digests { pub request: Option, pub programs: Vec, + /// A set's items as written out, so that a restart reads no program. + pub items: Option, pub result: Option, pub receipt: Option, } @@ -230,12 +250,53 @@ pub struct Job { /// The digest of the key it was sent with, and its client's name /// ([`Service::key_digest`]), if it was sent with one. pub key: Option, + /// The digest of the request as it was received, where that is not the + /// request as it reads (a `"source"`): what lets a send under its key be + /// matched before it is read. + pub received: Option, /// When the service stopped while the job ran, in milliseconds since /// 1970, each time it did. pub interruptions: Vec, child: Option, } +tokio::task_local! { + /// Whether the request being answered on this task was counted. + static COUNTED: std::cell::Cell; +} + +/// `work`'s answer, and whether it counted a submission: what says that a +/// client has paid for more reading in the same request. +pub async fn noting_count(work: F) -> (F::Output, bool) { + COUNTED + .scope(std::cell::Cell::new(false), async { + let out = work.await; + (out, COUNTED.with(std::cell::Cell::get)) + }) + .await +} + +/// Whether `request` carries its program in another format, so that what it +/// reads as is known only by reading it. +pub fn sourced(request: &Value) -> bool { + request.get("source").is_some_and(|s| !s.is_null()) +} + +/// The digest a request is matched by as it was received. +pub fn received_digest(request: &Value) -> String { + store::digest(&canon::canonical(request)) +} + +/// A set's items as JSON, each its request and its point. +pub fn items_json(items: &[sets::Item]) -> Value { + items.iter().map(|i| json!({ "request": i.request, "point": i.point })).collect() +} + +/// [`items_json`] read back. +pub fn items_of(v: &Value) -> Vec { + v.as_array().into_iter().flatten().map(|i| sets::Item { request: i["request"].clone(), point: Some(i["point"].clone()).filter(|p| !p.is_null()) }).collect() +} + fn now_ms() -> u64 { SystemTime::now().duration_since(UNIX_EPOCH).map(|d| d.as_millis() as u64).unwrap_or(0) } @@ -322,6 +383,12 @@ impl Job { if let Some(k) = &self.key { r["key"] = json!(k); } + if let Some(d) = &self.received { + r["received"] = json!(d); + } + if let Some(d) = &self.digests.items { + r["set_items"] = json!(d); + } if !self.interruptions.is_empty() { r["interruptions"] = json!(self.interruptions); } @@ -382,6 +449,8 @@ pub struct Service { clients: Mutex>, /// The job each key's digest names. Taken before `jobs` or `queue`. keys: Mutex>, + /// The turns requests are read in. + pub gate: crate::offload::Gate, wake: Notify, memory: Arc, workers: Arc, @@ -431,7 +500,9 @@ impl Service { fn build(config: Config, store: Box, restored: Restored) -> Service { let memory = Arc::new(Semaphore::new(usize::try_from(config.budget_mib).unwrap_or(usize::MAX).min(Semaphore::MAX_PERMITS))); let workers = Arc::new(Semaphore::new(config.workers)); + let gate = crate::offload::Gate::new(config.per_client_reads); Service { + gate, config, store, jobs: Mutex::new(HashMap::new()), @@ -453,8 +524,10 @@ impl Service { /// A service over `store`, its jobs reloaded: finished ones as they /// were; running ones queued again to run once more, or `interrupted` /// with the reason when they were run again already; queued ones queued - /// again in their order (planned again under `config`; one that no longer - /// plans fails, saying why); expired ones dropped. + /// again in their order (one too large for `config` fails, saying why); + /// expired ones dropped. No program is read: a job keeps its plan, and a + /// set its items, so a restart's time follows what the store holds, not + /// what its programs cost to read. pub fn open(config: Config, store: Box) -> Result, String> { let (records, unreadable) = store.records()?; for id in &unreadable { @@ -498,26 +571,19 @@ impl Service { } match job.status { Status::Queued => { - let cap = s.config.job_mib << 20; - let replanned = match &job.work { - Work::One => crate::catalog::plan(&job.request, cap).map(|p| (p, Work::One)), - Work::Set { items, .. } => sets::plan(items, cap).map(|plans| (sets::summary(&plans), Work::Set { items: items.clone(), plans })), - }; - match replanned.and_then(|(p, w)| if p.memory_bytes.div_ceil(1 << 20) > s.config.job_mib { Err(format!("it needs about {} MiB, more than one job may take here now", p.memory_bytes >> 20)) } else { Ok((p, w)) }) { - Ok((p, w)) => { - job.plan = p; - job.work = w; - queued.push((job.submitted_ms, job.id.clone())); - if rerun { - s.restored.rerun += 1; - } else { - s.restored.requeued += 1; - } - } - Err(e) => { - job.status = Status::Failed; - job.finished_ms = Some(now); - job.error = Some(format!("on the service's restart, the job no longer plans under its limits: {e}")); + // Its plan is the one it was admitted with: a restart reads + // no program, so what is queued delays no restart. + let mib = job.plan.memory_bytes.div_ceil(1 << 20); + if mib > s.config.job_mib { + job.status = Status::Failed; + job.finished_ms = Some(now); + job.error = Some(format!("on the service's restart, the job no longer plans under its limits: it needs about {mib} MiB, more than one job may take here now")); + } else { + queued.push((job.submitted_ms, job.id.clone())); + if rerun { + s.restored.rerun += 1; + } else { + s.restored.requeued += 1; } } s.store.put_record(&job.id, &job.record())?; @@ -559,8 +625,13 @@ impl Service { let work = match rec.get("set_plans") { Some(p) => { let plans: Vec = serde_json::from_value(p.clone()).map_err(|e| format!("its set's plans do not read: {e}"))?; - let items = sets::expand(&request)?; - if items.len() != plans.len() { + let items = match rec.get("set_items") { + Some(d) => items_of(&json_of(d)?), + // Kept before sets kept their items; what has ended needs none. + None if status.active() => wai_quantum::qasm::within_steps(crate::http::READ_STEPS, || sets::expand(&request))?, + None => Vec::new(), + }; + if status.active() && items.len() != plans.len() { return Err("its set's plans do not match its items".into()); } Work::Set { items, plans } @@ -592,10 +663,12 @@ impl Service { digests: Digests { request: s("request"), programs: rec["programs"].as_array().into_iter().flatten().filter_map(|d| d.as_str().map(str::to_string)).collect(), + items: s("set_items"), result: s("result"), receipt: s("receipt"), }, key: s("key"), + received: s("received"), interruptions: rec["interruptions"].as_array().into_iter().flatten().filter_map(Value::as_u64).collect(), child: None, }) @@ -636,6 +709,18 @@ impl Service { /// Counts a submission against `client`'s window, or says why it is one /// too many. A job and an analysis ([`crate::analysis`]) each count once. pub fn count(&self, client: &str) -> Result<(), Refused> { + self.in_window(client, true) + } + + /// Whether `client`'s window has room, spending none of it: what a + /// request is asked before it waits for a turn. It is counted once it has + /// its turn ([`crate::offload`]), so a request told to try again while + /// the service is busy has read nothing and spent nothing. + pub fn window_open(&self, client: &str) -> Result<(), Refused> { + self.in_window(client, false) + } + + fn in_window(&self, client: &str, spend: bool) -> Result<(), Refused> { let now = Instant::now(); let name = self.name(client); let mut clients = self.clients.lock().unwrap(); @@ -646,17 +731,11 @@ impl Service { if c.submissions.len() >= self.config.per_client_burst { return Err(Refused(429, tag(refusal::RATE, format!("at most {} submissions in {} s; try again shortly", self.config.per_client_burst, self.config.per_client_window.as_secs())))); } - c.submissions.push_back(now); - Ok(()) - } - - /// Takes back `client`'s latest submission: one counted before it was - /// known to be a job sent again under its key, which counts as nothing. - pub fn refund(&self, client: &str) { - let name = self.name(client); - if let Some(c) = self.clients.lock().unwrap().get_mut(&name) { - c.submissions.pop_back(); + if spend { + c.submissions.push_back(now); + let _ = COUNTED.try_with(|counted| counted.set(true)); } + Ok(()) } /// Where `client` stands in its submission window. @@ -694,14 +773,6 @@ impl Service { None } - /// Whether `key` names a job of `client`'s that still answers for it: - /// a send under it may be the same job sent again. - pub fn key_live(&self, client: &str, key: Key) -> bool { - let digest = Self::key_digest(&self.name(client), key); - let Some(id) = self.keys.lock().unwrap().get(&digest).cloned() else { return false }; - self.jobs.lock().unwrap().get(&id).is_some_and(|j| j.status.holds_key()) - } - /// Whether `client` may submit a job planned as `plan` now, by every /// limit but its submission window: the refusal it would get, or nothing. pub fn would_refuse(&self, client: &str, plan: &Plan) -> Option { @@ -730,24 +801,28 @@ impl Service { } /// The job `digest` names, if it still answers for it: its id when it - /// was sent with the request whose digest is `request`, or the refusal - /// of a key sent with another. - fn named_by(&self, keys: &HashMap, digest: &str, request: &str) -> Result, Refused> { + /// was sent with the request whose digest is `request`, as it was received + /// or as it read. Another request is the refusal of a key sent with + /// another, where `read` says that `request` is what it reads as; one not + /// yet read may read as the same request, so it is no answer yet. + fn named_by(&self, keys: &HashMap, digest: &str, request: &str, read: bool) -> Result, Refused> { let Some(id) = keys.get(digest) else { return Ok(None) }; let jobs = self.jobs.lock().unwrap(); match jobs.get(id) { - Some(j) if j.status.holds_key() && j.digests.request.as_deref() == Some(request) => Ok(Some(id.clone())), - Some(j) if j.status.holds_key() => Err(Refused(422, tag(refusal::IDEMPOTENCY_KEY, "this idempotency key was sent with another request; a key names one job, so send this request with a new key"))), + Some(j) if j.status.holds_key() && (j.digests.request.as_deref() == Some(request) || j.received.as_deref() == Some(request)) => Ok(Some(id.clone())), + Some(j) if j.status.holds_key() && read => Err(Refused(422, tag(refusal::IDEMPOTENCY_KEY, "this idempotency key was sent with another request; a key names one job, so send this request with a new key"))), _ => Ok(None), } } /// The job `key` names for `client` and `request`, if one does: what a - /// submission sent again is answered with, before it is planned. + /// submission sent again is answered with, before it is planned. It + /// reads nothing: a request in another format (`"source"`) is matched as + /// it was received, and sent any other way it is read, and counted, first. pub fn named(&self, client: &str, key: Key, request: &Value) -> Result, Refused> { let digest = Self::key_digest(&self.name(client), key); let keys = self.keys.lock().unwrap(); - self.named_by(&keys, &digest, &store::digest(&canon::canonical(request))) + self.named_by(&keys, &digest, &received_digest(request), !sourced(request)) } /// Admits a planned job for `client`, or says why not. Its request and @@ -762,17 +837,19 @@ impl Service { /// the key's job, before any limit, and counts as nothing. #[cfg(test)] pub fn submit_work(&self, client: &str, request: Value, plan: Plan, work: Work, key: Option) -> Result { - self.keep_work(client, request, plan, work, key, true).map(|(id, _)| id) + self.keep_work(client, request, plan, work, key, None, true) } /// [`Service::submit_work`] for a submission already counted against - /// its client's window (`http::admit` counts before it reads anything): - /// the job's id, and whether it is a new job (not the key's). - pub fn submit_counted(&self, client: &str, request: Value, plan: Plan, work: Work, key: Option) -> Result<(String, bool), Refused> { - self.keep_work(client, request, plan, work, key, false) + /// its client's window (`http::admit` counts before it reads anything), + /// `received` being the digest of the request as it came, where that is + /// not the request as it read. + pub fn submit_counted(&self, client: &str, request: Value, plan: Plan, work: Work, key: Option, received: Option) -> Result { + self.keep_work(client, request, plan, work, key, received, false) } - fn keep_work(&self, client: &str, request: Value, plan: Plan, work: Work, key: Option, count: bool) -> Result<(String, bool), Refused> { + #[allow(clippy::too_many_arguments)] + fn keep_work(&self, client: &str, request: Value, plan: Plan, work: Work, key: Option, received: Option, count: bool) -> Result { let name = self.name(client); let canonical = canon::canonical(&request); let request_digest = store::digest(&canonical); @@ -780,9 +857,9 @@ impl Service { // Held until the job is kept, so two sends of one key make one job. let mut keys = key.as_ref().map(|_| self.keys.lock().unwrap()); if let (Some(keys), Some(k)) = (&keys, &key) - && let Some(id) = self.named_by(keys, k, &request_digest)? + && let Some(id) = self.named_by(keys, k, &request_digest, true)? { - return Ok((id, false)); + return Ok(id); } // A full queue is checked again below, as the job is queued. if let Some(r) = self.would_refuse(client, &plan) @@ -802,6 +879,9 @@ impl Service { for p in programs { digests.programs.push(self.store.put(p.as_bytes()).map_err(unkept)?); } + if let Work::Set { items, .. } = &work { + digests.items = Some(self.store.put(items_json(items).to_string().as_bytes()).map_err(unkept)?); + } let id = job_id(); let job = Job { id: id.clone(), @@ -818,6 +898,7 @@ impl Service { peak_rss: 0, digests, key: key.clone(), + received, interruptions: Vec::new(), child: None, }; @@ -836,7 +917,7 @@ impl Service { drop(keys); self.changed(&id); self.wake.notify_one(); - Ok((id, true)) + Ok(id) } /// The job's report, if it is known. @@ -989,7 +1070,7 @@ impl Service { fn sweep(&self) { let keep: BTreeSet = { let jobs = self.jobs.lock().unwrap(); - jobs.values().flat_map(|j| j.digests.request.iter().chain(&j.digests.programs).chain(&j.digests.result).chain(&j.digests.receipt).cloned().collect::>()).collect() + jobs.values().flat_map(|j| j.digests.request.iter().chain(&j.digests.programs).chain(&j.digests.items).chain(&j.digests.result).chain(&j.digests.receipt).cloned().collect::>()).collect() }; if let Err(e) = self.store.sweep(&keep) { eprintln!("the job store: sweeping: {e}"); @@ -1392,9 +1473,22 @@ pub(crate) fn rss_bytes(pid: u32) -> Option { let kb: u64 = s.lines().find(|l| l.starts_with("VmRSS:"))?.split_whitespace().nth(1)?.parse().ok()?; return Some(kb * 1024); } - let o = std::process::Command::new("ps").args(["-o", "rss=", "-p", &pid.to_string()]).output().ok()?; - let kb: u64 = String::from_utf8_lossy(&o.stdout).trim().parse().ok()?; - Some(kb * 1024) + // Asked of the kernel: this runs on the runtime, for every child, many + // times a second, where a `ps` per reading held a worker for milliseconds. + #[cfg(target_os = "macos")] + { + let mut info: libc::proc_taskinfo = unsafe { std::mem::zeroed() }; + let size = std::mem::size_of::() as i32; + // SAFETY: `info` is a `proc_taskinfo`, and `size` is its length. + let got = unsafe { libc::proc_pidinfo(pid as i32, libc::PROC_PIDTASKINFO, 0, (&raw mut info).cast(), size) }; + (got == size).then_some(info.pti_resident_size) + } + #[cfg(not(target_os = "macos"))] + { + let o = std::process::Command::new("ps").args(["-o", "rss=", "-p", &pid.to_string()]).output().ok()?; + let kb: u64 = String::from_utf8_lossy(&o.stdout).trim().parse().ok()?; + Some(kb * 1024) + } } /// Whether the host's free memory covers `bytes` with `reserve_pct` of its @@ -1458,7 +1552,11 @@ mod tests { per_client_window: Duration::from_secs(60), read_mib: 4096, read_timeout: Duration::from_secs(300), + read_answer_mib: 256, read_in_child: false, + per_client_reads: 2, + header_wait: Duration::from_secs(30), + body_wait: Duration::from_secs(30), } } @@ -1537,6 +1635,29 @@ mod tests { /// A key outlives the service: reopened, the store's job still answers /// for it, but for the job that was running, now interrupted. + /// A restart reads no program: what is queued keeps its plan, and a set + /// its items, so a queue of slow programs delays no restart. These could + /// not be read at all, and come back queued, as they were. + #[test] + fn a_restart_reads_no_program() { + let dir = store_dir("noread"); + let cfg = || Config { per_client_active: 4, per_client_burst: 10, ..config() }; + let unreadable = "OPENQASM 3.0; qubit q; this does not read"; + let (one, set); + { + let s = Service::with_store(cfg(), Box::new(store::Files::open(&dir.join("store")).unwrap())); + one = s.submit_work("a", json!({ "qasm": unreadable, "shots": 5 }), plan(1 << 20), Work::One, None).unwrap(); + let items = vec![sets::Item { request: json!({ "qasm": unreadable, "shots": 5 }), point: Some(json!({ "n": 1 })) }]; + let request = json!({ "qasm": format!("input int n;\n{unreadable}"), "sweep": { "n": [1] } }); + set = s.submit_work("a", request, plan(1 << 20), Work::Set { items, plans: vec![plan(1 << 20)] }, None).unwrap(); + } + let s = Service::open(cfg(), Box::new(store::Files::open(&dir.join("store")).unwrap())).unwrap(); + assert_eq!(s.get(&one).unwrap()["status"], "queued", "{}", s.get(&one).unwrap()); + assert_eq!(s.get(&set).unwrap()["status"], "queued", "{}", s.get(&set).unwrap()); + let jobs = s.jobs.lock().unwrap(); + assert!(matches!(&jobs[&set].work, Work::Set { items, .. } if items.len() == 1 && items[0].point == Some(json!({ "n": 1 })))); + } + #[test] fn a_key_names_its_job_across_a_restart() { let dir = store_dir("keys"); @@ -1742,7 +1863,7 @@ mod tests { let key = wai_quantum::ed25519_dalek::SigningKey::from_bytes(&[19; 32]); let request = json!({ "qasm": "OPENQASM 3.0; qubit[2] q; h q[0]; cx q[0], q[1];", "shots": 20, "observables": ["Z0 Z1"] }); let (p, mut result) = wai_quantum::engine::job::plan_and_run(&request, 1 << 30).unwrap(); - let mut job = Job { id: job_id(), client: "test".into(), request: request.clone(), plan: p, work: Work::One, status: Status::Queued, submitted_ms: now_ms(), started_ms: None, finished_ms: None, result: None, error: None, peak_rss: 0, digests: Digests::default(), key: None, interruptions: Vec::new(), child: None }; + let mut job = Job { id: job_id(), client: "test".into(), request: request.clone(), plan: p, work: Work::One, status: Status::Queued, submitted_ms: now_ms(), started_ms: None, finished_ms: None, result: None, error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None }; let r = crate::mcp::schemas(); let check = |j: &Job| { let v = j.to_json(); @@ -1772,7 +1893,7 @@ mod tests { let items = wai_quantum::engine::sets::expand(&set_request).unwrap(); let plans = wai_quantum::engine::sets::plan(&items, 1 << 30).unwrap(); let body = wai_quantum::engine::sets::run(&set_request, 1 << 30, &key, json!({ "unmetered": "a test" })).unwrap(); - let set = Job { id: job_id(), client: "test".into(), request: set_request, plan: wai_quantum::engine::sets::summary(&plans), work: Work::Set { items, plans }, status: Status::Done, submitted_ms: now_ms(), started_ms: Some(now_ms()), finished_ms: Some(now_ms()), result: Some(body["result"].clone()), error: None, peak_rss: 1 << 20, digests: Digests::default(), key: None, interruptions: Vec::new(), child: None }; + let set = Job { id: job_id(), client: "test".into(), request: set_request, plan: wai_quantum::engine::sets::summary(&plans), work: Work::Set { items, plans }, status: Status::Done, submitted_ms: now_ms(), started_ms: Some(now_ms()), finished_ms: Some(now_ms()), result: Some(body["result"].clone()), error: None, peak_rss: 1 << 20, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None }; let done = check(&set); assert_eq!(wai_quantum::engine::sets::verify(&done)["valid"], true, "{done}"); } @@ -1886,7 +2007,7 @@ mod tests { assert!(t["seconds"].is_null() && t["at_most"] == 5, "{t}"); // Timed jobs here take over, and their metered power gives an energy estimate. for _ in 0..3 { - let job = Job { id: job_id(), client: "x".into(), request: json!({}), plan: Plan { work: 1_000_000_000, ..p.clone() }, work: Work::One, status: Status::Done, submitted_ms: 0, started_ms: None, finished_ms: Some(1), result: Some(json!({ "elapsed_ms": 500, "energy": { "joules_micro": 5_000_000, "energy_uncertainty": { "window_us": 500_000 } } })), error: None, peak_rss: 0, digests: Digests::default(), key: None, interruptions: Vec::new(), child: None }; + let job = Job { id: job_id(), client: "x".into(), request: json!({}), plan: Plan { work: 1_000_000_000, ..p.clone() }, work: Work::One, status: Status::Done, submitted_ms: 0, started_ms: None, finished_ms: Some(1), result: Some(json!({ "elapsed_ms": 500, "energy": { "joules_micro": 5_000_000, "energy_uncertainty": { "window_us": 500_000 } } })), error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None }; s.time(&job); } let (t, e) = s.estimate(&p); diff --git a/wai/wai-quantum-service/src/main.rs b/wai/wai-quantum-service/src/main.rs index 20e770384..6632f50ec 100644 --- a/wai/wai-quantum-service/src/main.rs +++ b/wai/wai-quantum-service/src/main.rs @@ -220,6 +220,7 @@ fn serve() { }); eprintln!("wai-quantum-service {} on {addr}, receipt key {}", env!("CARGO_PKG_VERSION"), app.signer.public_key()); let router = http::router(Arc::clone(&app)); - axum::serve(listener, router.into_make_service_with_connect_info::()).await.expect("the server"); + let header_wait = app.service.config.header_wait; + http::serve(listener, router, header_wait).await; }); } diff --git a/wai/wai-quantum-service/src/mcp.rs b/wai/wai-quantum-service/src/mcp.rs index d9c0719b0..14d8b67dc 100644 --- a/wai/wai-quantum-service/src/mcp.rs +++ b/wai/wai-quantum-service/src/mcp.rs @@ -82,13 +82,46 @@ fn output_of(tool: &str) -> Option<&'static str> { }) } +/// What an answer's verdict may read. Reading costs what reading a request +/// costs, so it is done in a reading child, for a request that was counted; +/// a poll, or a call that was refused before it was counted, reads nothing. +#[derive(Clone, Copy, Debug)] +pub enum Reading<'a> { + /// The programs, again: the verdict of what they say. + Read, + /// What the job holds: admitted, it read, and a poll reads it no more. + Stored, + /// Nothing: the verdict says why it is not of the program. + Unread(&'a str), +} + +impl Reading<'_> { + /// The verdict of `request` refused for `error`. + fn of_refusal(self, request: &Value, error: &str) -> Value { + match self { + Reading::Read => verdict::of_refusal(request, error), + Reading::Stored => verdict::of_admitted(Some(error)), + Reading::Unread(_) => verdict::of_refusal(&Value::Null, error), + } + } + + /// The verdict of the program `src`. + fn of_text(self, src: &str) -> Value { + match self { + Reading::Read => verdict::of_text(src), + Reading::Stored => verdict::of_admitted(None), + Reading::Unread(why) => verdict::of_refusal(&Value::Null, why), + } + } +} + /// The verdict a dry run states: of the refusal it would give, or of what /// it read (each item of a set). -fn dry_run_verdict(sc: &Value, args: &Value) -> Option { +fn dry_run_verdict(sc: &Value, args: &Value, reading: Reading) -> Option { Some(match sc["refusal"]["message"].as_str().or_else(|| sc["error"].as_str()) { - Some(e) => verdict::of_refusal(args, e), - None if wai_quantum::engine::catalog::is_set(args) => set_verdict(&json!({ "request": args })).unwrap_or(Value::Null), - None => verdict::of_text(args["qasm"].as_str().unwrap_or("")), + Some(e) => reading.of_refusal(args, e), + None if wai_quantum::engine::catalog::is_set(args) => set_verdict(&json!({ "request": args }), reading).unwrap_or(Value::Null), + None => reading.of_text(args["qasm"].as_str().unwrap_or("")), }) } @@ -419,18 +452,19 @@ pub use wai_quantum::engine::views::job_text as summary; /// A set's verdict: its weakest item's (an item not yet run, or whose run /// stated none, what the reader said of it), as the set's, the intents /// holding only if every item's do. -pub fn set_verdict(job: &Value) -> Option { +pub fn set_verdict(job: &Value, reading: Reading) -> Option { let each: Vec = match job.pointer("/result/items").and_then(Value::as_array) { Some(list) => list .iter() .map(|it| { it.pointer("/result/verdict").cloned().unwrap_or_else(|| match it["error"].as_str() { - Some(e) => verdict::of_refusal(&it["request"], e), - None => verdict::of_text(it["request"]["qasm"].as_str().unwrap_or("")), + Some(e) => reading.of_refusal(&it["request"], e), + None => reading.of_text(it["request"]["qasm"].as_str().unwrap_or("")), }) }) .collect(), - None => wai_quantum::engine::sets::expand(job.get("request").unwrap_or(job)).ok()?.iter().map(|it| verdict::of_text(it.request["qasm"].as_str().unwrap_or(""))).collect(), + None if matches!(reading, Reading::Read) => wai_quantum::engine::sets::expand(job.get("request").unwrap_or(job)).ok()?.iter().map(|it| verdict::of_text(it.request["qasm"].as_str().unwrap_or(""))).collect(), + None => vec![reading.of_text("")], }; let n = each.len(); let holds = if each.iter().any(|v| v["holds"] == false) { @@ -527,6 +561,24 @@ async fn admit_task(app: &App, who: &str, name: &str, args: Value) -> Result Value { + if !counted || told_to_wait(&why) { + return verdict::of_refusal(&Value::Null, &why); + } + let Ok(turn) = app.service.gate.turn(crate::offload::Pool::Reading, who).await else { return verdict::of_refusal(&Value::Null, &why) }; + let unread = verdict::of_refusal(&Value::Null, &why); + crate::offload::read(&app.service.config, turn, crate::offload::Read::Refusal { args, why }).await.unwrap_or(unread) +} + +/// Whether `why` tells the client to come back: nothing of its request was +/// read, or the service is too busy to read more of it. +fn told_to_wait(why: &str) -> bool { + [wai_quantum::engine::refusal::RATE, wai_quantum::engine::refusal::BUSY].iter().any(|code| why.ends_with(&format!("[{code}]"))) +} + /// What `tasks/result` answers for a task that has ended: the answer /// get_quantum_job gives for its job, an error where the job failed or the /// service interrupted it, the task named in its `_meta`. @@ -566,25 +618,49 @@ async fn admit_run(app: &App, who: &str, mut args: Value, key: Option) - /// A tool's answer as `tools/call` gives it: the call's, the verdict of /// what it read beside it, and what a view draws, for a host that draws it. -/// The verdicts and the view read the programs again, so they are made off -/// the runtime ([`crate::offload`]), within one budget of reading; where no -/// turn comes, the answer goes without them. +/// Reading the programs again costs what reading them cost, so only a call +/// that was counted does, in a reading child ([`crate::offload`]) within one +/// budget. A tool that states nothing about a program takes no turn. A poll +/// states what its job holds. A call refused before it was counted, told to +/// try again, or that finds no turn, states why its verdict is not of the +/// program: every answer about a program has a verdict. async fn tool_answer(app: &App, who: &str, name: &str, args: Value, base: &str, views: bool) -> Value { - let answer = call(app, who, name, args.clone(), base).await; - let plain = answer.clone(); - crate::offload::read(&app.service.config, crate::offload::Read::Finish { name: name.to_string(), args, answer, views }).await.unwrap_or(plain) + let (answer, counted) = crate::jobs::noting_count(call(app, who, name, args.clone(), base)).await; + if !states_verdict(name) && !(views && views::for_tool(name).is_some()) { + return answer; + } + let sc = &answer["structuredContent"]; + let error = sc["error"].as_str().filter(|_| answer["isError"] == true).map(str::to_string); + // A job's answer, polled or sent again, or with its verdict in hand. + let job = sc.get("id").is_some() && sc.get("status").is_some(); + if job && (!counted || (sc.pointer("/result/verdict").is_some() && !views) || matches!(name, "get_quantum_job" | "poll_quantum_job")) { + return finish(name, &args, answer, views, Reading::Stored); + } + if !counted || error.as_deref().is_some_and(told_to_wait) { + let why = error.unwrap_or_else(|| "this answer was not read again".to_string()); + return finish(name, &args, answer, views, Reading::Unread(&why)); + } + let finished = match app.service.gate.turn(crate::offload::Pool::Reading, who).await { + Ok(turn) => crate::offload::read(&app.service.config, turn, crate::offload::Read::Finish { name: name.to_string(), args: args.clone(), answer: answer.clone(), views }).await, + Err(e) => Err(e), + }; + match finished { + Ok(v) => v, + Err((_, why)) => finish(name, &args, answer, views, Reading::Unread(&why)), + } } -/// [`tool_answer`]'s verdicts and view added to the call's `answer`. -pub(crate) fn finish(name: &str, args: &Value, mut answer: Value, views: bool) -> Value { +/// [`tool_answer`]'s verdicts and view added to the call's `answer`, reading +/// what `reading` says it may. +pub(crate) fn finish(name: &str, args: &Value, mut answer: Value, views: bool, reading: Reading) -> Value { // A set's answer, and a dry run's, state the verdict of what they read. let sc = &answer["structuredContent"]; let extra = match name { - "run_quantum_circuit" | "run_on_target" | "run_quantum_set" if crate::http::is_dry_run(args) && sc.get("dry_run").is_some() => dry_run_verdict(sc, args), - "run_quantum_set" if sc.is_object() && sc.get("error").is_none() => set_verdict(sc), - "run_quantum_set" => sc["error"].as_str().map(|e| verdict::of_refusal(args, e)), - "get_quantum_job" if sc["method"] == "set" => set_verdict(sc), - "dry_run_quantum_job" => dry_run_verdict(sc, args), + "run_quantum_circuit" | "run_on_target" | "run_quantum_set" if crate::http::is_dry_run(args) && sc.get("dry_run").is_some() => dry_run_verdict(sc, args, reading), + "run_quantum_set" if sc.is_object() && sc.get("error").is_none() => set_verdict(sc, reading), + "run_quantum_set" => sc["error"].as_str().map(|e| reading.of_refusal(args, e)), + "get_quantum_job" if sc["method"] == "set" => set_verdict(sc, reading), + "dry_run_quantum_job" => dry_run_verdict(sc, args, reading), _ => None, }; if let Some(v) = extra.filter(|v| !v.is_null()) { @@ -593,7 +669,7 @@ pub(crate) fn finish(name: &str, args: &Value, mut answer: Value, views: bool) - // What the view draws, to a host that draws it. if views && answer["structuredContent"].is_object() - && let Some(d) = views::data(name, args, &answer["structuredContent"]) + && let Some(d) = views::data_of(name, args, &answer["structuredContent"], matches!(reading, Reading::Read)) { answer["_meta"] = Value::Object([(views::META.to_string(), d)].into_iter().collect()); } @@ -605,12 +681,16 @@ pub(crate) fn finish(name: &str, args: &Value, mut answer: Value, views: bool) - Some(e) if answer["isError"] == true => { let mut r = wai_quantum::engine::refusal::refusal_json(e); r["error"] = json!(e); - r["verdict"] = verdict::of_refusal(args, e); + r["verdict"] = reading.of_refusal(args, e); answer["structuredContent"] = r; None } _ if name == "get_quantum_job" && sc["method"] == "set" => None, - _ => verdict::for_tool(name, args, sc), + _ => match reading { + Reading::Read => verdict::for_tool(name, args, sc), + // What the answer holds, or what can be said without reading. + _ => sc.pointer("/result/verdict").or_else(|| sc.get("verdict").filter(|v| v.is_object())).cloned().or_else(|| Some(reading.of_text(""))), + }, }; if let Some(v) = v { answer["structuredContent"][verdict::verdict_key(name)] = v; @@ -692,12 +772,11 @@ pub async fn handle(app: &App, who: &str, msg: &Value, base: &str, views: bool) if crate::http::is_dry_run(&args) { return Some(fail(&id, -32602, "a dry run runs nothing and answers at once: call it without a task")); } - match admit_task(app, who, name, args.clone()).await { - Ok(job) => ok(&id, json!({ "task": task_of(&app.service.get(&job).unwrap_or(json!({ "id": job })), app.service.config.retain.as_millis() as u64) })), - Err(why) => { + match crate::jobs::noting_count(admit_task(app, who, name, args.clone())).await { + (Ok(job), _) => ok(&id, json!({ "task": task_of(&app.service.get(&job).unwrap_or(json!({ "id": job })), app.service.config.retain.as_millis() as u64) })), + (Err(why), counted) => { let mut data = wai_quantum::engine::refusal::refusal_json(&why); - // The verdict reads the program again: in a reading child. - data["verdict"] = crate::offload::read(&app.service.config, crate::offload::Read::Refusal { args: args.clone(), why: why.clone() }).await.unwrap_or(Value::Null); + data["verdict"] = task_verdict(app, who, args, why.clone(), counted).await; json!({ "jsonrpc": "2.0", "id": id, "error": { "code": -32602, "message": format!("no task was made: {why}"), "data": data } }) } } @@ -820,19 +899,31 @@ async fn call(app: &App, who: &str, name: &str, args: Value, base: &str) -> Valu Err((_, why)) => tool_result(format!("refused: {why}"), json!({ "error": why }), true), } } - "verify_quantum_receipt" if args["result"]["format"] == wai_quantum::engine::sets::FORMAT => { - let v = wai_quantum::engine::sets::verify(&args); - let text = if v["valid"] == true { - format!("valid: the set's receipt seals its request and head, and every one of its {} items is sealed by its own receipt and its path to the root", v["items"]) - } else { - format!("not valid: signed {}, items as the request gives them {}, each item sealed {}, root {}", v["signed"], v["derived"], v["items_sealed"], v["root"]) - }; - tool_result(text, v, false) - } + // A set's programs are written out again to check its items: it is + // counted and read as a check is. + "verify_quantum_receipt" if args["result"]["format"] == wai_quantum::engine::sets::FORMAT => match crate::analysis::bounded(app, who, crate::offload::Read::VerifySet { job: args.clone() }).await { + Ok(v) => { + let text = if v["valid"] == true { + format!("valid: the set's receipt seals its request and head, and every one of its {} items is sealed by its own receipt and its path to the root", v["items"]) + } else { + format!("not valid: signed {}, items as the request gives them {}, each item sealed {}, root {}", v["signed"], v["derived"], v["items_sealed"], v["root"]) + }; + tool_result(text, v, false) + } + Err((_, why)) => tool_result(format!("refused: {why}"), json!({ "error": why }), true), + }, "verify_quantum_receipt" if args["job"].is_string() => { // The job's text, read as the protocol's section 8.2 reads it. let text = args["job"].as_str().unwrap_or_default(); - let v = wai_quantum::engine::job::verdict_of_text(text).unwrap_or_else(|_| wai_quantum::engine::job::verdict_of(&wai_quantum::engine::job::check_text(text))); + let is_set = serde_json::from_str::(text).is_ok_and(|j| j["result"]["format"] == wai_quantum::engine::sets::FORMAT); + let v = if is_set { + match crate::analysis::bounded(app, who, crate::offload::Read::Verify { text: text.to_string() }).await { + Ok(v) => v, + Err((_, why)) => return tool_result(format!("refused: {why}"), json!({ "error": why }), true), + } + } else { + wai_quantum::engine::job::verdict_of_text(text).unwrap_or_else(|_| wai_quantum::engine::job::verdict_of(&wai_quantum::engine::job::check_text(text))) + }; let line = if v["valid"] == true { "valid: the receipt seals exactly this request and this result, and every rule of its form holds".to_string() } else { format!("not valid: {}", v["reasons"]) }; tool_result(line, v, false) } @@ -1103,6 +1194,79 @@ mod tests { rt.block_on(handle(&app, "test", &msg, "http://127.0.0.1:1", views)).unwrap() } + /// Reading a program again costs what reading it cost, so a call that + /// was not counted reads nothing: each states a verdict, as every answer + /// about a program does, of why it was refused, with no code of the + /// reader's, as reading this program gives. + #[test] + fn a_call_that_was_not_counted_reads_nothing_and_still_states_a_verdict() { + let bad = "OPENQASM 3.0;\nqubit q;\nno_such_gate q;\n"; + let unread = |v: &Value, what: &str| { + assert_eq!(v["level"], "refused", "{what}: {v}"); + assert!(v["parsed"].get("code").is_none() && v["parsed"]["ok"] == false, "{what}: {v}"); + }; + for (tool, args) in [ + ("lint_qasm", json!({ "qasm": bad })), + ("get_quantum_job", json!({ "id": "no such job", "qasm": bad })), + ("run_quantum_circuit", json!({ "qasm": bad, "idempotency_key": 5 })), + ("run_quantum_circuit", json!({ "qasm": bad, "idempotency_key": "" })), + ] { + let r = rpc("tools/call", json!({ "name": tool, "arguments": args })); + assert_eq!(r["result"]["isError"], true, "{tool}: {r}"); + unread(&r["result"]["structuredContent"]["verdict"], tool); + } + // A refused task is the same, on the error it answers with. + let r = rpc("tools/call", json!({ "name": "run_quantum_circuit", "arguments": { "qasm": bad, "idempotency_key": 5 }, "task": {} })); + unread(&r["error"]["data"]["verdict"], "task"); + // Counted, it is read: the same program, the reader's own code. + let r = rpc("tools/call", json!({ "name": "run_quantum_circuit", "arguments": { "qasm": bad } })); + assert!(r["result"]["structuredContent"]["verdict"]["parsed"]["code"].is_string(), "{r}"); + } + + /// With every turn held, a tool that reads nothing answers at once, and + /// one that needs a turn for its verdict still has one, saying why it is + /// not of the program; past its window a run reads nothing either. + #[tokio::test] + async fn the_tools_are_answered_with_no_turn_to_be_had() { + let dir = std::env::temp_dir().join(format!("wai-qsvc-mcp-turns-{}", std::process::id())); + let signer = std::sync::Arc::new(crate::receipt::Signer::load_or_create(&dir.join("key")).unwrap()); + let service = crate::jobs::Service::new(crate::jobs::Config { per_client_burst: 3, ..crate::jobs::Config::from_env() }); + let app = App::bare(service, signer, "http://127.0.0.1:1"); + let call = |name: &str, args: Value| json!({ "jsonrpc": "2.0", "id": 7, "method": "tools/call", "params": { "name": name, "arguments": args } }); + let mut held = Vec::new(); + for i in 0..crate::offload::AT_ONCE { + held.push(app.service.gate.turn(crate::offload::Pool::Reading, &format!("other {i}")).await.ok().unwrap()); + } + let t = std::time::Instant::now(); + let r = handle(&app, "test", &call("list_targets", json!({})), "http://127.0.0.1:1", false).await.unwrap(); + assert!(r["result"]["structuredContent"]["targets"].is_array() && t.elapsed() < std::time::Duration::from_secs(1), "{r}"); + let r = handle(&app, "test", &call("critique_qasm", json!({ "qasm": "OPENQASM 3.0; qubit q; h q;" })), "http://127.0.0.1:1", false).await.unwrap(); + let v = &r["result"]["structuredContent"]["verdict"]; + assert!(v["statement"].as_str().unwrap().contains("[J032]") && v["level"] == "refused", "{r}"); + let tool = tools().as_array().unwrap().iter().find(|t| t["name"] == "critique_qasm").cloned().unwrap(); + let bad = schemas().check_value(&tool["outputSchema"], &r["result"]["structuredContent"]); + assert!(bad.is_empty(), "{bad:?}"); + drop(held); + app.service.count("test").unwrap(); + app.service.count("test").unwrap(); + let r = handle(&app, "test", &call("run_quantum_circuit", json!({ "qasm": "OPENQASM 3.0;\nqubit q;\nno_such_gate q;\n" })), "http://127.0.0.1:1", false).await.unwrap(); + let sc = &r["result"]["structuredContent"]; + assert!(sc["error"].as_str().unwrap().ends_with("[J030]") && sc["verdict"]["parsed"].get("code").is_none(), "{r}"); + } + + /// A poll states what its job holds, read at its admission, and reads + /// nothing more, however often it is asked. + #[test] + fn a_poll_reads_nothing() { + let v = verdict::of_admitted(None); + assert!(v["level"] == "parsed" && v["parsed"]["ok"] == true, "{v}"); + let job = json!({ "id": "j", "status": "queued", "request": { "qasm": "this does not read" } }); + let answer = tool_result("queued".into(), job, false); + let done = finish("get_quantum_job", &json!({ "id": "j" }), answer, true, Reading::Stored); + assert_eq!(done["structuredContent"]["verdict"]["level"], "parsed", "{done}"); + assert!(done["_meta"][views::META].get("layout").is_none(), "{done}"); + } + /// A client whose host draws views gets them here as from the command /// line's server, and the view's own tool for a running job, kept from /// the model; any other client gets none of it. diff --git a/wai/wai-quantum-service/src/offload.rs b/wai/wai-quantum-service/src/offload.rs index a8d214559..40428ba53 100644 --- a/wai/wai-quantum-service/src/offload.rs +++ b/wai/wai-quantum-service/src/offload.rs @@ -2,25 +2,32 @@ //! //! Reading a program (the reader's elaboration of its loops, a set's sweep //! written out, a reference program's equivalence bound, a plan, a check's -//! answer) is bounded work: the reader holds its nesting, its steps and its -//! memory to limits of its own. But a limit the service merely trusts is one -//! a mistake can lift, and what is at stake is every job the service holds: -//! a stack overflow or an allocation that fails ends a Rust process, and -//! neither can be caught (an allocator may not unwind, so exhaustion aborts). -//! So each request's reading runs in a child, the service's own binary run -//! as `wai-quantum-service read`, the same boundary each job runs behind: +//! answer, a set's receipt checked) is bounded work: the reader holds its +//! nesting, its steps and its memory to limits of its own. But a limit the +//! service merely trusts is one a mistake can lift, and what is at stake is +//! every job the service holds: a stack overflow or an allocation that fails +//! ends a Rust process, and neither can be caught (an allocator may not +//! unwind, so exhaustion aborts). And steps bound work, not time: a statement +//! that copies what the program holds costs one. So each request's reading +//! runs in a child, the service's own binary run as `wai-quantum-service +//! read`, the same boundary each job runs behind: //! //! - **Its memory is capped** ([`crate::jobs::Config::read_mib`]): polled as a //! job's is, and killed past it; the request is refused, `422` and `J031`, //! with the command that runs it on the client's own machine. **Its time is -//! capped** the same way ([`crate::jobs::Config::read_timeout`]). +//! capped** the same way ([`crate::jobs::Config::read_timeout`]): nothing a +//! request reads holds a turn past it. **Its answer is capped** +//! ([`crate::jobs::Config::read_answer_mib`]): it comes into this process. //! - **It ends alone**: a child that panics, overflows its stack or aborts is //! answered `500`, and the service runs on. //! - **It is sandboxed** where jobs are (macOS: no network, no file writes, //! no further processes), and is given only what it reads ([`Read`]). -//! - **One of [`AT_ONCE`]** at a time: a request waits up to [`WAIT`] for a -//! turn, and is then refused (`503`, `J032`). A turn is held until its child -//! has ended, so a client that hangs up frees none while its reading runs. +//! - **It waits its turn** ([`Gate`]): [`AT_ONCE`] are read at once, and +//! [`CHECKS_AT_ONCE`] checks beside them. A client has +//! [`crate::jobs::Config::per_client_reads`] requests in all being read or +//! waiting, at most [`MAX_WAITING`] wait, each for at most [`WAIT`], and the +//! rest are refused (`503`, `J032`), having read and spent nothing. A +//! client that hangs up ends its child and frees its turn. //! //! The service's own tests read on a thread of their own ([`STACK`]) instead, //! as a test binary is not the service's (`Config::read_in_child`). @@ -30,32 +37,126 @@ use serde::{Deserialize, Serialize}; use serde_json::{Value, json}; use std::panic::{AssertUnwindSafe, catch_unwind}; use std::process::Stdio; -use std::sync::{Arc, LazyLock}; +use std::collections::HashMap; +use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; +use std::sync::{Arc, Mutex}; use std::time::{Duration, Instant}; use tokio::io::{AsyncReadExt, AsyncWriteExt}; -use tokio::sync::Semaphore; +use tokio::sync::{OwnedSemaphorePermit, Semaphore}; use wai_quantum::engine::refusal::{self, tag}; /// The stack a reading runs on where it runs on a thread. pub const STACK: usize = 64 << 20; /// Requests read at once; the rest wait their turn. pub const AT_ONCE: usize = 4; +/// Checks answered at once, beside them. +pub const CHECKS_AT_ONCE: usize = 2; /// How long a request waits for a turn before it is refused. pub const WAIT: Duration = Duration::from_secs(10); +/// How many requests may wait for a turn: each holds its body meanwhile. +pub const MAX_WAITING: usize = 32; -static TURNS: LazyLock> = LazyLock::new(|| Arc::new(Semaphore::new(AT_ONCE))); +/// What a request is read among. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Pool { + /// Jobs and dry runs, and the verdicts of MCP answers. + Reading, + /// The design checks and the engine calls ([`crate::analysis`]). + Checks, +} + +type Clients = Arc>>; + +/// The turns requests are read in: the service's two pools, each client's +/// share of them, and who is waiting. +pub struct Gate { + reading: Arc, + checks: Arc, + waiting: Arc, + clients: Clients, + per_client: usize, +} + +/// A turn, held until its request has been read; dropped, the next goes. +pub struct Turn { + _permit: OwnedSemaphorePermit, + _client: ClientHold, +} + +struct ClientHold { + clients: Clients, + who: String, +} + +impl Drop for ClientHold { + fn drop(&mut self) { + let mut clients = self.clients.lock().unwrap(); + if let Some(n) = clients.get_mut(&self.who) { + *n -= 1; + if *n == 0 { + clients.remove(&self.who); + } + } + } +} + +struct Waiting(Arc); + +impl Drop for Waiting { + fn drop(&mut self) { + self.0.fetch_sub(1, Ordering::SeqCst); + } +} + +impl Gate { + pub fn new(per_client: usize) -> Gate { + Gate { reading: Arc::new(Semaphore::new(AT_ONCE)), checks: Arc::new(Semaphore::new(CHECKS_AT_ONCE)), waiting: Arc::new(AtomicUsize::new(0)), clients: Arc::default(), per_client } + } + + /// A turn for `who` in `pool`, or the refusal (`503`, `J032`): at once for + /// a client with its share in hand or with too many waiting, after + /// [`WAIT`] for one that found no turn. + pub async fn turn(&self, pool: Pool, who: &str) -> Result { + let busy = |why: String| (503, tag(refusal::BUSY, why)); + let client = { + let mut clients = self.clients.lock().unwrap(); + let held = clients.entry(who.to_string()).or_insert(0); + if *held >= self.per_client { + return Err(busy(format!("{} of your requests are being read; send the next when one is answered", self.per_client))); + } + *held += 1; + ClientHold { clients: Arc::clone(&self.clients), who: who.to_string() } + }; + let (turns, full) = match pool { + Pool::Reading => (&self.reading, format!("{AT_ONCE} requests are being read; try again shortly")), + Pool::Checks => (&self.checks, format!("{CHECKS_AT_ONCE} checks are running; try again shortly")), + }; + if let Ok(permit) = Arc::clone(turns).try_acquire_owned() { + return Ok(Turn { _permit: permit, _client: client }); + } + if self.waiting.fetch_add(1, Ordering::SeqCst) >= MAX_WAITING { + self.waiting.fetch_sub(1, Ordering::SeqCst); + return Err(busy(full)); + } + let _waiting = Waiting(Arc::clone(&self.waiting)); + match tokio::time::timeout(WAIT, Arc::clone(turns).acquire_owned()).await { + Ok(Ok(permit)) => Ok(Turn { _permit: permit, _client: client }), + _ => Err(busy(full)), + } + } +} /// One request's reading, as its child takes it on standard input. #[derive(Clone, Debug, Serialize, Deserialize)] #[serde(tag = "read", rename_all = "snake_case")] pub enum Read { - /// A job or a set: its source read where it has one, and with `plan`, - /// held to the checks that read and planned within `job_bytes` - /// (`http::plan_job`). The answer is `{"request", "plan", "items", - /// "plans"}`, `plan` null without planning, or `{"request", "refused": + /// A job or a set: its source read where it has one, held to the checks + /// that read and planned within `job_bytes` (`http::plan_job`). The answer + /// is `{"request", "plan", "items", "plans"}`, or `{"request", "refused": /// [status, why]}` where the planning refused it. - Admit { request: Value, job_bytes: u64, plan: bool }, - /// A design check (`analysis::Check`, by name): its bounds, then its answer. + Admit { request: Value, job_bytes: u64 }, + /// A design check (`analysis::Check`, by name): its bounds and its + /// answer, within one budget. Check { check: String, body: Value }, /// An engine call held to its bounds: `kind` is `estimate`, /// `estimate_verify`, `qec`, `pulse` or `variational`; `key`, the receipt @@ -65,13 +166,16 @@ pub enum Read { Finish { name: String, args: Value, answer: Value, views: bool }, /// The verdict a refused MCP task states. Refusal { args: Value, why: String }, + /// A set's receipt checked (`POST /v1/verify`), from the job's text. + Verify { text: String }, + /// A set's receipt checked (the MCP tool), from the parsed job. + VerifySet { job: Value }, } -/// `read`, one of [`AT_ONCE`], in a child whose memory and time are capped -/// by `config` (or on a thread of its own, where `config` says so): its -/// answer, or the refusal. -pub async fn read(config: &Config, r: Read) -> Result { - let turn = turn().await?; +/// `read`, in its `turn`, in a child whose memory and time are capped by +/// `config` (or on a thread of its own, where `config` says so): its answer, +/// or the refusal. +pub async fn read(config: &Config, turn: Turn, r: Read) -> Result { if !config.read_in_child { return on_thread(move || { let _turn = turn; @@ -84,21 +188,11 @@ pub async fn read(config: &Config, r: Read) -> Result { out } -async fn turn() -> Result { - match tokio::time::timeout(WAIT, Arc::clone(&TURNS).acquire_owned()).await { - Ok(Ok(t)) => Ok(t), - _ => Err((503, tag(refusal::BUSY, format!("{AT_ONCE} requests are being read; try again shortly")))), - } -} - /// `read`, carried out here: what the child does. pub fn run(r: Read) -> Result { match r { - Read::Admit { request, job_bytes, plan } => crate::http::within_budget(|| { - let request = crate::http::with_source(request)?; - if !plan { - return Ok(json!({ "request": request, "plan": null })); - } + Read::Admit { request, job_bytes } => wai_quantum::qasm::within_steps(crate::http::READ_STEPS, || { + let request = crate::http::within_budget(|| crate::http::with_source(request))?; // A refusal of the plan is the answer's, beside the request as read. let (p, work) = match crate::http::plan_job(job_bytes, &request) { Ok(planned) => planned, @@ -106,21 +200,22 @@ pub fn run(r: Read) -> Result { }; let (items, plans) = match work { crate::jobs::Work::One => (Value::Null, Value::Null), - crate::jobs::Work::Set { items, plans } => ( - items.iter().map(|i| json!({ "request": i.request, "point": i.point })).collect(), - serde_json::to_value(plans).expect("plans are JSON"), - ), + crate::jobs::Work::Set { items, plans } => (crate::jobs::items_json(&items), serde_json::to_value(plans).expect("plans are JSON")), }; Ok(json!({ "request": request, "plan": p, "items": items, "plans": plans })) }), Read::Check { check, body } => { let check = crate::analysis::Check::named(&check).ok_or((400, format!("no check {check}")))?; - crate::http::within_budget(|| crate::analysis::bounds(check, &body))?; - Ok(crate::analysis::answer(check, &body)) + crate::http::within_budget(|| { + crate::analysis::bounds(check, &body)?; + Ok(crate::analysis::answer(check, &body)) + }) } Read::Call { kind, op, body, key } => crate::analysis::call(&kind, &op, &body, key.as_deref()), - Read::Finish { name, args, answer, views } => crate::http::within_budget(|| Ok(crate::mcp::finish(&name, &args, answer, views))), + Read::Finish { name, args, answer, views } => crate::http::within_budget(|| Ok(crate::mcp::finish(&name, &args, answer, views, crate::mcp::Reading::Read))), Read::Refusal { args, why } => crate::http::within_budget(|| Ok(wai_quantum::engine::verdict::of_refusal(&args, &why))), + Read::Verify { text } => crate::http::within_budget(|| Ok(wai_quantum::engine::job::verdict_of_text(&text).unwrap_or_else(|e| json!({ "valid": false, "error": e })))), + Read::VerifySet { job } => crate::http::within_budget(|| Ok(wai_quantum::engine::sets::verify(&job))), } } @@ -154,14 +249,20 @@ async fn in_child(config: &Config, r: &Read) -> Result { }); let mut stdout = child.stdout.take().expect("piped"); let mut stderr = child.stderr.take().expect("piped"); + // Its answer comes into this process, so it is held here as its memory + // is there: a sweep written out can be far larger than the request. + let answer_limit = config.read_answer_mib << 20; + let too_long = Arc::new(AtomicBool::new(false)); + let over = Arc::clone(&too_long); let reader = tokio::spawn(async move { let mut out = Vec::new(); - let _ = stdout.read_to_end(&mut out).await; + let _ = (&mut stdout).take(answer_limit + 1).read_to_end(&mut out).await; + over.store(out.len() as u64 > answer_limit, Ordering::SeqCst); out }); let errors = tokio::spawn(async move { let mut out = Vec::new(); - let _ = stderr.read_to_end(&mut out).await; + let _ = (&mut stderr).take(1 << 20).read_to_end(&mut out).await; out }); let limit = config.read_mib << 20; @@ -172,6 +273,11 @@ async fn in_child(config: &Config, r: &Read) -> Result { Ok(Err(e)) => break Err((500, format!("the reader could not be watched: {e}"))), Err(_) => {} } + if too_long.load(Ordering::SeqCst) { + crate::jobs::kill(pid); + let _ = child.wait().await; + break Err((422, tag(refusal::TOO_LARGE_HERE, format!("reading this request gave more than {} MiB, more than the service keeps of one; {LOCAL}", config.read_answer_mib)))); + } if crate::jobs::rss_bytes(pid).is_some_and(|rss| rss > limit) { crate::jobs::kill(pid); let _ = child.wait().await; @@ -250,13 +356,60 @@ mod tests { assert_eq!(on_thread(|| deep(8000)).await.unwrap(), want); } + /// A client has its share of the turns, waiting included, so one client + /// cannot take every turn: past it, it is told at once, and its share comes + /// back as its requests are answered. Other clients go on. + #[tokio::test] + async fn a_client_is_held_to_its_share_of_the_turns() { + let gate = Gate::new(2); + let (a, b) = (gate.turn(Pool::Reading, "a").await.unwrap(), gate.turn(Pool::Checks, "a").await.unwrap()); + let t = Instant::now(); + let (code, why) = gate.turn(Pool::Reading, "a").await.err().unwrap(); + assert_eq!(code, 503); + assert!(why.contains("[J032]") && t.elapsed() < Duration::from_secs(1), "{why}"); + let other = gate.turn(Pool::Reading, "b").await; + assert!(other.is_ok()); + drop((a, b)); + assert!(gate.turn(Pool::Reading, "a").await.is_ok()); + assert!(gate.clients.lock().unwrap().get("a").is_none(), "a client that holds nothing is not kept"); + } + + /// Past [`MAX_WAITING`] a request is told at once, and those that wait + /// are told within [`WAIT`]: nothing waits without limit. + #[tokio::test] + async fn a_wait_for_a_turn_is_bounded_in_time_and_in_number() { + let gate = Arc::new(Gate::new(1)); + let mut held = Vec::new(); + for i in 0..AT_ONCE { + held.push(gate.turn(Pool::Reading, &format!("holder {i}")).await.ok().unwrap()); + } + let waiting: Vec<_> = (0..MAX_WAITING) + .map(|i| { + let g = Arc::clone(&gate); + tokio::spawn(async move { g.turn(Pool::Reading, &format!("waiter {i}")).await.err().map(|e| e.0) }) + }) + .collect(); + while gate.waiting.load(Ordering::SeqCst) < MAX_WAITING { + tokio::time::sleep(Duration::from_millis(5)).await; + } + let t = Instant::now(); + let (code, why) = gate.turn(Pool::Reading, "one more").await.err().unwrap(); + assert_eq!(code, 503); + assert!(why.contains("[J032]") && t.elapsed() < Duration::from_secs(1), "{why}"); + for w in waiting { + assert_eq!(w.await.unwrap(), Some(503)); + } + assert_eq!(gate.waiting.load(Ordering::SeqCst), 0); + drop(held); + } + /// A reading goes to its child and back as JSON, each kind of it. #[test] fn a_reading_is_json_both_ways() { let r = Read::Call { kind: "qec".into(), op: "decode".into(), body: json!({ "circuit": "H 0\n" }), key: None }; let back: Read = serde_json::from_value(serde_json::to_value(&r).unwrap()).unwrap(); assert!(matches!(back, Read::Call { ref kind, .. } if kind == "qec")); - let a = run(Read::Admit { request: json!({ "qasm": "OPENQASM 3.0; qubit q; h q;", "shots": 4 }), job_bytes: 1 << 30, plan: true }).unwrap(); + let a = run(Read::Admit { request: json!({ "qasm": "OPENQASM 3.0; qubit q; h q;", "shots": 4 }), job_bytes: 1 << 30 }).unwrap(); assert!(a["plan"]["method"].is_string() && a["items"].is_null(), "{a}"); } } diff --git a/wai/wai-quantum-service/tests/depth.rs b/wai/wai-quantum-service/tests/depth.rs index d440f5b20..c37ea661f 100644 --- a/wai/wai-quantum-service/tests/depth.rs +++ b/wai/wai-quantum-service/tests/depth.rs @@ -98,7 +98,7 @@ fn programs_nested_thousands_deep_are_refused_and_the_service_runs_on() { let mut svc = start(&d, port, 1000); let deep = parens(5_000); let blocks = format!("{HEAD}{}x q[0];{}\nc = measure q;\n", "{ ".repeat(5_000), " }".repeat(5_000)); - let sum = format!("{HEAD}rz(0.5{}) q[0];\nc = measure q;\n", " + 0".repeat(50_000)); + let power = format!("{HEAD}rz(1{}) q[0];\nc = measure q;\n", " ** 1".repeat(50_000)); let refused = |code: u16, body: &Value, what: &str| { assert!((400..500).contains(&code), "{what}: {code} {body}"); let text = body.to_string(); @@ -106,7 +106,7 @@ fn programs_nested_thousands_deep_are_refused_and_the_service_runs_on() { }; // The job's own program, through both submissions and a dry run. - for (path, qasm, what) in [("/v1/jobs", &deep, "parentheses"), ("/v1/run", &blocks, "blocks"), ("/v1/jobs", &sum, "a sum")] { + for (path, qasm, what) in [("/v1/jobs", &deep, "parentheses"), ("/v1/run", &blocks, "blocks"), ("/v1/jobs", &power, "a power")] { let (code, b) = json_of(port, "POST", path, Some(&json!({ "qasm": qasm, "shots": 10 }))); refused(code, &b, what); alive(port, &mut svc, what); @@ -296,6 +296,17 @@ fn a_reading_past_its_memory_is_stopped_and_refused_and_the_service_runs_on() { alive(port, &mut svc, "a circuit past the reading's memory"); svc.kill().unwrap(); svc.wait().unwrap(); + // A sweep written out comes back into the service: held too, however + // small its request, as that memory is the one every job shares. + let port = free_port(); + let mut svc = start_with(&d, port, 1000, &[("WAI_QSVC_READ_ANSWER_MIB", "1")]); + let body = "x q;\n".repeat(500); + let sweep = json!({ "qasm": format!("OPENQASM 3.0;\ninput int n;\nqubit q;\nfor int i in [1:n] {{\n{body}}}\n"), "sweep": { "n": (0..100).map(|k| 100 + k).collect::>() }, "shots": 1 }); + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&sweep)); + assert!(code == 422 && b.to_string().contains("J031") && b.to_string().contains("gave more than"), "{code} {b}"); + alive(port, &mut svc, "a sweep past what a reading gives"); + svc.kill().unwrap(); + svc.wait().unwrap(); let _ = std::fs::remove_dir_all(&d); } @@ -333,14 +344,14 @@ fn a_client_told_to_retry_spends_nothing_and_its_key_s_job_comes_back_past_the_w assert_eq!(code, 202, "{first}"); assert_eq!(left(&head), "3", "{head}"); let id = serde_json::from_str::(&first).unwrap()["id"].as_str().unwrap().to_string(); - // Told twenty times to wait for it, every five seconds as Retry-After - // says: nothing spent. + // Told twenty times to wait for it: nothing spent. let bell = json!({ "qasm": "OPENQASM 3.0; qubit q; h q;", "shots": 10 }); for _ in 0..20 { let (code, head, body) = post("/v1/jobs", &bell, None); assert!(code == 429 && body.contains("wait for one to finish"), "{code} {body}"); assert_eq!(left(&head), "3", "{head}"); - assert!(head.to_ascii_lowercase().contains("retry-after: 5"), "{head}"); + // Sent again at that pace, what counts would never spend the window. + assert!(head.to_ascii_lowercase().contains("retry-after: 150"), "{head}"); } // Its window spent by refusals that do count, for _ in 0..3 { @@ -359,3 +370,274 @@ fn a_client_told_to_retry_spends_nothing_and_its_key_s_job_comes_back_past_the_w let _ = std::fs::remove_dir_all(&d); } +/// An index in a few bytes of text would make what is as long as the index: +/// a decoder's table for 10^14 detectors, in 79 bytes. Each way of naming one +/// is refused for what it is, and the service runs on. +#[test] +fn an_index_past_what_a_model_holds_is_refused_and_the_service_runs_on() { + let d = dir("indices"); + let port = free_port(); + let mut svc = start(&d, port, 1000); + for body in [ + json!({ "model": "shift_detectors 100000000000000\ndetector(1) D0\n", "detectors": [] }), + json!({ "model": "shift_detectors 100000000000000\ndetector D0\n", "detectors": [] }), + json!({ "model": "detector(1) D4294967294\n", "detectors": [] }), + json!({ "circuit": "M 4000000000\nDETECTOR rec[-1]\n", "shots": 10 }), + ] { + let (code, b) = json_of(port, "POST", "/v1/qec/decode", Some(&body)); + assert!((400..500).contains(&code) && b.to_string().contains("past the"), "{body}: {code} {b}"); + alive(port, &mut svc, "an index"); + } + let (code, b) = json_of(port, "POST", "/mcp", Some(&json!({ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "qec_decode", "arguments": { "model": "shift_detectors 100000000000000\ndetector(1) D0\n", "detectors": [] } } }))); + assert!(code == 200 && b.to_string().contains("past the"), "{code} {b}"); + alive(port, &mut svc, "an index through MCP"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// A program a reader takes seconds over, each differing, so none is read +/// from another's reading. +fn slow(k: usize) -> String { + format!("OPENQASM 3.0;\nqubit[1] q;\nint a = 0;\nfor int i in [0:4999999] {{ a += 1; }}\n// {k}\nx q[0];\n") +} + +/// `n` requests at once, each as `send` makes it: the seconds each took. +fn at_once(n: usize, send: impl Fn(usize) + Sync) -> Vec { + std::thread::scope(|t| { + let handles: Vec<_> = (0..n) + .map(|i| { + let send = &send; + t.spawn(move || { + let began = Instant::now(); + send(i); + began.elapsed().as_secs_f64() + }) + }) + .collect(); + handles.into_iter().map(|h| h.join().unwrap()).collect() + }) +} + +/// How long `/v1/health` takes while `during` runs beside it, at its worst. +fn health_during(port: u16, during: impl FnOnce() + Send) -> f64 { + let done = std::sync::atomic::AtomicBool::new(false); + std::thread::scope(|t| { + let probe = t.spawn(|| { + let mut worst = 0f64; + while !done.load(std::sync::atomic::Ordering::SeqCst) { + let began = Instant::now(); + assert_eq!(http(port, "GET", "/v1/health", None).0, 200); + worst = worst.max(began.elapsed().as_secs_f64()); + std::thread::sleep(Duration::from_millis(20)); + } + worst + }); + // Set on the way out however `during` ends, so a failure in it fails here. + struct Done<'a>(&'a std::sync::atomic::AtomicBool); + impl Drop for Done<'_> { + fn drop(&mut self) { + self.0.store(true, std::sync::atomic::Ordering::SeqCst); + } + } + { + let _done = Done(&done); + during(); + } + probe.join().unwrap() + }) +} + +/// A refused MCP task used to read its program on a runtime worker, with no +/// budget and no count: four of them stalled every route. A task refused +/// before anything was counted reads nothing, and spends nothing. +#[test] +fn a_refused_mcp_task_reads_nothing_and_holds_no_worker() { + let d = dir("tasks"); + let port = free_port(); + let mut svc = start(&d, port, 1000); + let worst = health_during(port, || { + let took = at_once(4, |k| { + let (code, b) = json_of(port, "POST", "/mcp", Some(&json!({ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "run_quantum_circuit", "task": {}, "arguments": { "qasm": slow(k), "idempotency_key": 5 } } }))); + assert!(code == 200 && b["error"]["data"]["verdict"]["level"] == "refused", "{b}"); + }); + assert!(took.iter().all(|&t| t < 1.0), "{took:?}"); + }); + assert!(worst < 1.0, "health took {worst} s"); + alive(port, &mut svc, "refused tasks"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// Every check counts, refused or not, before its bounds read anything; a +/// set's receipt is read as a check is, as its sweep is written out again. +#[test] +fn a_check_and_a_set_s_receipt_count_before_they_read() { + let d = dir("counted"); + let port = free_port(); + let mut svc = start(&d, port, 5); + let left = |head: &str| head.to_ascii_lowercase().lines().find_map(|l| l.strip_prefix("ratelimit: \"submissions\";r=").map(|r| r.split(';').next().unwrap_or("").to_string())).unwrap_or_default(); + let wide = format!("OPENQASM 3.0;\nqubit[22] q;\nh q;\n{}", "rz(0.3) q;\n".repeat(200)); + let (code, head, body) = http(port, "POST", "/v1/equivalent", Some(&json!({ "a": wide, "b": wide }))); + assert!(code == 422 && body.contains("J031"), "{code} {body}"); + assert_eq!(left(&head), "4", "{head}"); + let (code, head, body) = http(port, "POST", "/v1/critique", Some(&json!({ "qasm": format!("OPENQASM 3.0;\nqubit q;\nrz({}0.5{}) q;\n", "(".repeat(5_000), ")".repeat(5_000)) }))); + assert!(code == 200 && body.contains("Q006"), "{code} {body}"); + assert_eq!(left(&head), "3", "{head}"); + let set = json!({ "request": { "qasm": "input int n;\nqubit q;\nx q;\n", "sweep": { "n": [1, 2] } }, "result": { "format": "wai.quantum.set/1", "count": 2, "items": [], "receipt": {} } }); + let (code, head, body) = http(port, "POST", "/v1/verify", Some(&set)); + assert!(code == 200 && body.contains("\"valid\":false"), "{code} {body}"); + assert_eq!(left(&head), "2", "{head}"); + let (code, b) = json_of(port, "POST", "/mcp", Some(&json!({ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "verify_quantum_receipt", "arguments": set } }))); + assert!(code == 200 && b["result"]["structuredContent"]["valid"] == false, "{b}"); + let (_, head, _) = http(port, "GET", "/v1", None); + assert_eq!(left(&head), "1", "{head}"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// Reading runs off the request threads: eight clients' heavy requests at +/// once leave `/v1/health` answering, as a job's reading used to stall it. +/// A client has its share of the turns, told at once past it. +#[test] +fn reading_leaves_the_runtime_free_and_a_client_has_its_share() { + let d = dir("runtime"); + let port = free_port(); + let mut svc = start_with(&d, port, 1000, &[("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1")]); + let from = |who: &str, path: &str, body: &Value| { + let mut s = TcpStream::connect(("127.0.0.1", port)).unwrap(); + s.set_read_timeout(Some(Duration::from_secs(120))).unwrap(); + let text = body.to_string(); + write!(s, "POST {path} HTTP/1.1\r\nHost: 127.0.0.1\r\nContent-Type: application/json\r\nX-Forwarded-For: {who}\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{text}", text.len()).unwrap(); + let mut out = String::new(); + let _ = s.read_to_string(&mut out); + out + }; + let worst = health_during(port, || { + at_once(8, |k| { + let job = json!({ "qasm": format!("{HEAD}x q[0];\n"), "shots": 10, "intent": (0..16).map(|i| json!({ "equivalent": slow(100 * k + i) })).collect::>() }); + let out = from(&format!("10.0.0.{k}"), "/v1/jobs", &job); + assert!(out.contains("J031") || out.contains("J032"), "{out}"); + }); + }); + assert!(worst < 1.0, "health took {worst} s"); + // One client, three at once: the third is told at once, its share in hand. + let answers = std::sync::Mutex::new(Vec::new()); + at_once(3, |k| { + if k == 2 { + std::thread::sleep(Duration::from_millis(300)); + } + let began = Instant::now(); + let out = from("10.0.1.1", "/v1/critique", &json!({ "qasm": slow(1000 + k) })); + answers.lock().unwrap().push((out.contains("J032"), began.elapsed().as_secs_f64())); + }); + let answers = answers.into_inner().unwrap(); + assert!(answers.iter().any(|&(busy, took)| busy && took < 1.0), "{answers:?}"); + alive(port, &mut svc, "the turns"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// Steps bound work, not time: a reading past its deadline is stopped, in its +/// child, and refused, and the turn it held goes to the next. A client that +/// hangs up ends its reading, and its turn comes back at once. +#[test] +fn a_reading_past_its_time_is_stopped_and_a_hang_up_ends_it() { + let d = dir("deadline"); + let port = free_port(); + let mut svc = start_with(&d, port, 1000, &[("WAI_QSVC_READ_SECS", "1"), ("WAI_QSVC_CLIENT_READS", "1")]); + let long = "OPENQASM 3.0;\nqubit q;\nint a = 0;\nfor int i in [0:9899999] { a += 1; }\nx q;\n"; + let t = Instant::now(); + let (code, b) = json_of(port, "POST", "/v1/critique", Some(&json!({ "qasm": long }))); + assert!(code == 422 && b.to_string().contains("J031") && b.to_string().contains("1 s"), "{code} {b}"); + assert!(t.elapsed() < Duration::from_secs(10), "{:?}", t.elapsed()); + alive(port, &mut svc, "a reading past its time"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let port = free_port(); + let mut svc = start_with(&d, port, 1000, &[("WAI_QSVC_CLIENT_READS", "1")]); + let text = json!({ "qasm": long }).to_string(); + let mut s = TcpStream::connect(("127.0.0.1", port)).unwrap(); + write!(s, "POST /v1/critique HTTP/1.1\r\nHost: 127.0.0.1\r\nContent-Type: application/json\r\nContent-Length: {}\r\n\r\n{text}", text.len()).unwrap(); + std::thread::sleep(Duration::from_millis(300)); + drop(s); + let began = Instant::now(); + loop { + let (code, _, body) = http(port, "POST", "/v1/critique", Some(&json!({ "qasm": "OPENQASM 3.0; qubit q; h q;" }))); + if code == 200 { + break; + } + assert!(code == 503 && began.elapsed() < Duration::from_secs(2), "{code} {body} after {:?}", began.elapsed()); + std::thread::sleep(Duration::from_millis(50)); + } + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// A check's answer reads within the check's budget: a repair of forty fixes +/// of a program a read takes seconds over ran for minutes. Each pass of a +/// repair is a new text, so it ends at the budget, refused. +#[test] +fn a_check_s_answer_is_held_to_its_budget() { + let d = dir("answer"); + let port = free_port(); + let mut svc = start(&d, port, 1000); + let program = format!("OPENQASM 3.0;\nqubit[1] q;\nbit[1] c;\nint x = 0;\nfor int i in [0:9899999] {{ x = i; }}\n{}", "measure q[0];\n".repeat(40)); + let t = Instant::now(); + let (code, b) = json_of(port, "POST", "/v1/repair", Some(&json!({ "qasm": program }))); + assert!(code == 422 && b.to_string().contains("J031"), "{code} {b}"); + assert!(t.elapsed() < Duration::from_secs(60), "{:?}", t.elapsed()); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// A program within the reader's own bounds is taken here: its statements and +/// its gates each near their bound, past what one count of both would allow. +#[test] +fn a_job_within_the_reader_s_bounds_reads_here() { + let d = dir("own"); + let port = free_port(); + let mut svc = start(&d, port, 1000); + let program = "OPENQASM 3.0;\nqubit q;\nint a = 0;\nfor int i in [0:8499999] { a += 1; }\nfor int i in [0:1399999] { x q; }\n"; + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&json!({ "qasm": program, "shots": 1, "dry_run": true }))); + assert!(code == 200 && b["accepted"] == true, "{code} {b}"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// A client that stops sending holds its connection for no longer than the +/// limits say: a header section left open is closed, a body left open is +/// refused `408`, and the service answers beside them. +#[test] +fn a_stuck_connection_is_let_go() { + let d = dir("stuck"); + let port = free_port(); + let mut svc = start_with(&d, port, 1000, &[("WAI_QSVC_HEADER_SECS", "1"), ("WAI_QSVC_BODY_SECS", "1")]); + let open = |text: &str| { + let mut s = TcpStream::connect(("127.0.0.1", port)).unwrap(); + s.set_read_timeout(Some(Duration::from_secs(10))).unwrap(); + s.write_all(text.as_bytes()).unwrap(); + s + }; + let mut header = open("POST /v1/jobs HTTP/1.1\r\nHost: 127.0.0.1\r\nX-Half: "); + let mut body = open("POST /v1/jobs HTTP/1.1\r\nHost: 127.0.0.1\r\nContent-Type: application/json\r\nContent-Length: 100\r\n\r\n{\"qasm\""); + alive(port, &mut svc, "stuck connections"); + let t = Instant::now(); + let mut out = String::new(); + let _ = header.read_to_string(&mut out); + assert!(t.elapsed() < Duration::from_secs(5), "{:?}", t.elapsed()); + out.clear(); + let _ = body.read_to_string(&mut out); + assert!(out.starts_with("HTTP/1.1 408"), "{out}"); + assert!(t.elapsed() < Duration::from_secs(5), "{:?}", t.elapsed()); + alive(port, &mut svc, "connections let go"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} diff --git a/wai/wai-quantum-service/tests/verify.rs b/wai/wai-quantum-service/tests/verify.rs index 1d892a7a3..9a793465b 100644 --- a/wai/wai-quantum-service/tests/verify.rs +++ b/wai/wai-quantum-service/tests/verify.rs @@ -35,7 +35,8 @@ fn start(dir: &Path, port: u16) -> Child { .env("WAI_QSVC_BUDGET_GB", "4") .env("WAI_QSVC_WORKERS", "1") .env("WAI_QSVC_ENERGY", "0") - .env("WAI_QSVC_RATE", "100000") + // A set's receipt counts, as a check does: the corpus holds more than a window's. + .env("WAI_QSVC_CLIENT_BURST", "100000") .stdout(Stdio::null()) .stderr(Stdio::null()) .spawn() diff --git a/wai/wai-quantum/docs/job-refusals.md b/wai/wai-quantum/docs/job-refusals.md index 87384a3d3..88ee954a3 100644 --- a/wai/wai-quantum/docs/job-refusals.md +++ b/wai/wai-quantum/docs/job-refusals.md @@ -65,12 +65,12 @@ browser engine. A test holds this page to the code's table. | `J011` | refused | The method named cannot take this job: a gate it does not hold, more qubits, memory or work than it takes here, or an answer it does not give (noise it cannot take is `J061`); the message says which, and `auto` picks a method that can. | | `J012` | refused | No method takes this job: each method's reason is listed, so the job can be changed to suit one. | | `J013` | refused | A placement on the fabric (`fabric`) this host cannot honour: no member by that name, no certified accelerator member here, a state or a partition larger than a member holds, a partition count that is not a power of two, or a method or program the fabric's members do not run (the message says which, and what is here). | -| `J020` | refused | An intent that does not read: not one of `distribution`, `equivalent` or `prepares`, a distribution whose outcomes are not the program's bits or whose probabilities do not sum to 1, or a reference program that does not read. | +| `J020` | refused | An intent that does not read: not one of `distribution`, `equivalent` or `prepares`, a distribution whose outcomes are not the program's bits or whose probabilities do not sum to 1, or a reference program that does not read, or more intents than the host takes. | | `J021` | refused | A proposed repair that changes a line no diagnostic flags, or adds lines away from one: rejected, and the program left as it was. | | `J022` | refused | A repair's choices that do not read (`max_iterations`, `dyadic_bits`, `sizes`), or a proposal that is not a program's text. | | `J030` | retry | The host takes no more from this client for now (submissions in a window, or jobs at once): send it again shortly. | -| `J031` | refused | A job or a check larger than this host takes: run it on your own machine, with the command line or its MCP server. | -| `J032` | retry | The host is full (its queue, or the checks running at once): send it again later. | +| `J031` | refused | A job or a check larger than this host takes, its reading included: run it on your own machine, with the command line or its MCP server. | +| `J032` | retry | The host is full (its queue, the checks running at once, or the requests being read, yours or all of them): send it again later. | | `J033` | refused | A host that takes requests only with one of its API keys: give yours as `Authorization: Bearer `; a key it does not hold is refused the same way. | | `J034` | refused | An idempotency key (`Idempotency-Key`, or a tool's `idempotency_key`) that does not read, or that this client sent before with another request: a key names one job, so another request needs another key. A key is 1 to 255 visible ASCII characters, and should be unguessable (a UUID, or 128 random bits). | | `J040` | refused | A pulse run that does not read or cannot be done: a field it does not take, options or device parameters that do not read, a target without the qubit, or a schedule outside the simulator's model (two tones on one qubit at once, a play after its qubit's capture, more levels and qubits than it holds, a frame far from its qubit); the message says which. | diff --git a/wai/wai-quantum/docs/qasm-diagnostics.md b/wai/wai-quantum/docs/qasm-diagnostics.md index 4dc89f335..aa9fe2d95 100644 --- a/wai/wai-quantum/docs/qasm-diagnostics.md +++ b/wai/wai-quantum/docs/qasm-diagnostics.md @@ -45,7 +45,7 @@ engine. A test holds this page to the code's table. | `Q003` | refused | A string, a block comment, a bracket or a calibration block that is not closed. | | `Q004` | refused | Something else was expected here: a `;`, a name, an expression or a statement (the message says which). | | `Q005` | refused | A statement in a place the language does not allow it: `else` with no `if`, `case` outside `switch`, a declaration inside a gate. | -| `Q006` | refused | A program nested deeper than the reader takes: statements, parentheses, operators, brackets and types more than 64 levels deep in its text, or more than 256 levels in all through the gates and subroutines it calls; refused, never read by a recursion that could run out of stack. | +| `Q006` | refused | A program nested deeper than the reader takes: statements, parentheses, operators, brackets and types more than 64 levels deep in its text (a run of one precedence, `a + b + c`, and an `else if` chain each count one, however long), an array of more than 7 dimensions, or more than 256 levels in all through the gates and subroutines it calls; refused, never read by a recursion that could run out of stack. | | `Q010` | warned | An `OPENQASM` version other than 2.x or 3.x: the program is read as OpenQASM 3. | | `Q011` | warned | An `include` of a file other than the standard libraries (`stdgates.inc`, `qelib1.inc`): its definitions are not read, so a gate it defines is unknown. | | `Q012` | lowered | `include "stdgates.inc"` or `"qelib1.inc"`: the standard gates, `U`, `gphase` and OpenQASM 2's names are built in. | @@ -106,7 +106,7 @@ engine. A test holds this page to the code's table. | `Q309` | lowered | `while` whose condition is known on reading: unrolled until it fails. | | `Q310` | refused | A `while` whose condition reads measured bits and that declares no bound: add `@wai.max_iterations N` before it. | | `Q311` | warned | A `while` on measured bits run at most its declared bound (`@wai.max_iterations N`): a run that still meets its condition after N iterations stops looping there. | -| `Q312` | refused | A program that does not finish unrolling within the reader's bound (20 000 000 statements and loop iterations). | +| `Q312` | refused | A program that does not finish unrolling within the reader's bound (20 000 000 statements and loop iterations), or that runs out a caller's budget of reading (`qasm::within_steps`: statements, gates and a generator's loop passes, shared by every program read under it). | | `Q313` | refused | A range with a zero step, or that does not read as `[start:end]` or `[start:step:end]`. | | `Q314` | refused | `break` or `continue` outside a loop, or `return` outside a subroutine. | | `Q315` | refused | A `return` with a value under a condition on measured bits. | diff --git a/wai/wai-quantum/docs/qasm-feature-matrix.md b/wai/wai-quantum/docs/qasm-feature-matrix.md index b60739649..cbb1063b7 100644 --- a/wai/wai-quantum/docs/qasm-feature-matrix.md +++ b/wai/wai-quantum/docs/qasm-feature-matrix.md @@ -30,8 +30,8 @@ and in the browser build. | Comments `//` and `/* */` | supported; an unclosed block comment is refused | Q003 | | Identifiers, Unicode letters included (`θ`, `π`) | supported | | | Statements and blocks `{ }`, scopes | supported; qubits, gates, subroutines and externs are declared at the top level | Q105 | -| Nesting: statements in blocks and bodies, parentheses, operators, brackets, calls, casts and types | supported to 64 levels together in the text (a statement, each enclosing statement, and each level of its expressions' trees count one; a long `else if` chain is one level a branch, and reads as a `switch` at any length); deeper is refused, never read by a recursion that could run out of stack | Q006 | -| Nesting in all, through the gates and subroutines a program calls | supported to 256 levels (each statement, expression and call the reader runs counts one), and calls 32 deep; deeper is refused | Q006, Q209 | +| Nesting: statements in blocks and bodies, parentheses, operators, brackets, calls, casts and types | supported to 64 levels together in the text (a statement, each enclosing statement, and each level of its expressions' trees count one). A run of operators of one precedence (`a + b - c`, `c[0] && c[1] && …`) is one level, and an `else if` chain one level, at any length: both are read and run in loops, left to right. Deeper is refused, never read by a recursion that could run out of stack; an array has at most 7 dimensions | Q006 | +| Nesting in all, through the gates and subroutines a program calls | supported to 256 levels (each statement, expression and call the reader runs counts one), and calls 32 deep; deeper is refused. Both hold: a recursive subroutine whose body nests more than about seven levels reaches 256 before 32 calls | Q006, Q209 | | What a program holds on reading: its classical values, gates, steps, measurements and timing | supported to 64 MiB for one value (an array or a bit register, each element counted at its width) and 512 MiB in all; more is refused before it is made. Making or copying a value counts toward the reader's 20 000 000 steps at one step for every 128 bytes, and an element read from an array copies the element alone | Q130, Q312 | | `pragma …`, `#pragma …`, annotations `@keyword …` | warned: kept, not applied; `@wai.max_iterations N` bounds the `while` after it | Q013 | diff --git a/wai/wai-quantum/docs/qec.md b/wai/wai-quantum/docs/qec.md index 2b39e3c29..bf3e72c02 100644 --- a/wai/wai-quantum/docs/qec.md +++ b/wai/wai-quantum/docs/qec.md @@ -124,7 +124,10 @@ instructions (`DEPOLARIZE1[leak](0.01) 0`), Pauli-target observables `logical_observable` declarations, `repeat` and `shift_detectors`. A circuit's model written back keeps its tags and coordinates. What the reader does not take it refuses with the line, never skipping it. Blocks nest at -most 32 deep (`MAX_BLOCK_DEPTH`): a deeper one is refused on the line that +most 32 deep (`MAX_BLOCK_DEPTH`), a circuit names qubits below 2²⁴ +(`MAX_QUBITS`) and a model detectors below 2²⁴ (`MAX_DETECTORS`, offsets +counted): what is made for an index is as long as the index, so each is +refused on its line, as is a block nested deeper, on the line that opens it, so neither reading, unrolling nor writing a text recurses past that. Every pass through a block counts toward the bound on what a text unrolls to, so a block repeated without end that writes nothing still ends, diff --git a/wai/wai-quantum/src/engine/answer/eval.rs b/wai/wai-quantum/src/engine/answer/eval.rs index d37c87fee..12f7a938d 100644 --- a/wai/wai-quantum/src/engine/answer/eval.rs +++ b/wai/wai-quantum/src/engine/answer/eval.rs @@ -34,14 +34,14 @@ pub struct Host<'a> { /// The value of the statement `name`: what its view shows or its call /// gives, or why the engine refused it. pub fn value(a: &Answer, name: &str, host: &Host) -> Result { - Ev { a, host, memo: BTreeMap::new(), depth: 0 }.stmt(name) + Ev::new(a, host).stmt(name) } /// Every statement that is ready, carried out: `{"format", "digest", /// "results": {name: {"ok": true, "value"} | {"ok": false, "error", "code", /// …}}}`, in the order of their lines. pub fn evaluate(a: &Answer, host: &Host) -> Value { - let mut ev = Ev { a, host, memo: BTreeMap::new(), depth: 0 }; + let mut ev = Ev::new(a, host); let mut results = serde_json::Map::new(); for c in &a.stmts { if c.status != Status::Ready || matches!(c.stmt.body, Body::Program { .. }) { @@ -111,7 +111,7 @@ pub fn verify(text: &str, job_bytes: u64) -> Result { return Err(format!("the receipt names {name}, which is not a seal the answer can run ({})", c.status.as_str())); } let host = Host { job_bytes, key: None }; - let mut ev = Ev { a: &a, host: &host, memo: BTreeMap::new(), depth: 0 }; + let mut ev = Ev::new(&a, &host); let (request, result) = ev.sealed(seal_list(&c.stmt.body).ok_or("the seal lists nothing")?)?; let key = m["receipt"]["key"].clone(); let signer = key.as_str().and_then(job::did_of); @@ -142,37 +142,74 @@ pub const CHAIN_DEPTH: usize = 32; struct Ev<'a> { a: &'a Answer, host: &'a Host<'a>, - memo: BTreeMap>, + /// Each statement's answer, whatever reached it, with how many statements + /// long its own chain is: reached too deep for that, it is refused as it + /// would be run, so the order statements are asked in changes nothing. + memo: BTreeMap, usize)>, + /// What a statement gave where the chain's limit cut it short, with the + /// depth it was reached at: an answer of that depth, not the statement's, + /// so it stands only as deep or deeper, and is never run again there. + cut: BTreeMap, usize)>, /// How many statements are being run, each under the one that names it. depth: usize, + /// The deepest the statement being run has reached, and whether the + /// limit cut it short. + reached: usize, + was_cut: bool, } -impl Ev<'_> { +impl<'a> Ev<'a> { + fn new(a: &'a Answer, host: &'a Host<'a>) -> Self { + Ev { a, host, memo: BTreeMap::new(), cut: BTreeMap::new(), depth: 0, reached: 0, was_cut: false } + } + + fn too_deep(&mut self, name: &str) -> Result { + self.was_cut = true; + Err(format!("{name} is reached through more than {CHAIN_DEPTH} statements, each naming the next: name fewer in a chain")) + } + fn stmt(&mut self, name: &str) -> Result { - if let Some(r) = self.memo.get(name) { + if let Some((r, long)) = self.memo.get(name) { + if self.depth + long > CHAIN_DEPTH { + return self.too_deep(name); + } + self.reached = self.reached.max(self.depth + long); return r.clone(); } - if self.depth >= CHAIN_DEPTH { - return Err(format!("{name} is reached through more than {CHAIN_DEPTH} statements, each naming the next: name fewer in a chain")); + if let Some((r, at)) = self.cut.get(name) + && self.depth >= *at + { + let r = r.clone(); + self.was_cut = true; + return r; } + if self.depth >= CHAIN_DEPTH { + return self.too_deep(name); + } + let at = self.depth; self.depth += 1; + let (reached, was_cut) = (std::mem::replace(&mut self.reached, self.depth), std::mem::replace(&mut self.was_cut, false)); let r = self.stmt_here(name); self.depth -= 1; + if self.was_cut { + self.cut.insert(name.to_string(), (r.clone(), at)); + } else { + self.memo.insert(name.to_string(), (r.clone(), self.reached - at)); + } + self.reached = self.reached.max(reached); + self.was_cut |= was_cut; r } fn stmt_here(&mut self, name: &str) -> Result { let c = self.a.get(name).ok_or_else(|| format!("no statement {name}"))?; - let r = if c.status != Status::Ready { - Err(format!("{name} is {}: nothing to run", c.status.as_str())) - } else { - match &c.stmt.body { - Body::Program { qasm, .. } => Ok(json!({ "qasm": qasm, "digest": blake3::hash(qasm.as_bytes()).to_hex().to_string() })), - Body::Expr(e) => self.expr(e), - } - }; - self.memo.insert(name.to_string(), r.clone()); - r + if c.status != Status::Ready { + return Err(format!("{name} is {}: nothing to run", c.status.as_str())); + } + match &c.stmt.body { + Body::Program { qasm, .. } => Ok(json!({ "qasm": qasm, "digest": blake3::hash(qasm.as_bytes()).to_hex().to_string() })), + Body::Expr(e) => self.expr(e), + } } fn expr(&mut self, e: &Expr) -> Result { diff --git a/wai/wai-quantum/src/engine/answer/tests.rs b/wai/wai-quantum/src/engine/answer/tests.rs index 6af4cf788..e5b7f7ff7 100644 --- a/wai/wai-quantum/src/engine/answer/tests.rs +++ b/wai/wai-quantum/src/engine/answer/tests.rs @@ -323,3 +323,39 @@ fn a_chain_of_statements_past_the_limit_is_refused_not_overflowed() { assert!(e.contains("more than 32 statements"), "{e}"); assert!(value(chain(4)).is_ok()); } + +/// Whether a statement of a chain is refused is its own: the same statements +/// are refused, whatever order they are asked in, and they are the ones +/// whose chains are too long read alone. Statements that name each other +/// twice are not run again for each way down. +#[test] +fn the_chain_limit_refuses_the_same_statements_in_any_order() { + let head = "```qasm p\nOPENQASM 3.0;\nqubit[1] q;\nbit[1] c;\nh q[0];\nc = measure q;\n```\n"; + let lines = |n: usize, list: &str| -> Vec { + let mut v: Vec = (0..n).map(|k| format!("s{k} = @Seal([{}])\n", list.replace('#', &format!("s{}", k + 1)))).collect(); + v.push(format!("s{n} = @Sample(p, 10)\n")); + v + }; + let refused = |text: String| { + let k = key(); + let host = super::eval::Host { job_bytes: 1 << 28, key: Some(&k) }; + let a = parse(&text); + let out = super::eval::evaluate(&a, &host); + let mut bad: Vec = out["results"].as_object().unwrap().iter().filter(|(_, r)| r["ok"] != true).map(|(n, _)| n[1..].parse().unwrap()).collect(); + bad.sort_unstable(); + let alone: Vec = out["results"].as_object().unwrap().keys().filter(|n| super::eval::value(&a, n, &host).is_err()).map(|n| n[1..].parse().unwrap()).collect(); + (bad, alone) + }; + let forward = lines(40, "#"); + let (bad, alone) = refused(format!("{head}{}", forward.concat())); + let mut alone = alone; + alone.sort_unstable(); + assert_eq!(bad, (0..=8).collect::>()); + assert_eq!(alone, bad); + let reverse: Vec = forward.iter().rev().cloned().collect(); + assert_eq!(refused(format!("{head}{}", reverse.concat())).0, bad); + let t = std::time::Instant::now(); + let twice = lines(60, "#, #"); + assert!(!refused(format!("{head}{}", twice.concat())).0.is_empty()); + assert!(t.elapsed() < std::time::Duration::from_secs(120), "{:?}", t.elapsed()); +} diff --git a/wai/wai-quantum/src/engine/formats/emit.rs b/wai/wai-quantum/src/engine/formats/emit.rs index f891a9232..813759719 100644 --- a/wai/wai-quantum/src/engine/formats/emit.rs +++ b/wai/wai-quantum/src/engine/formats/emit.rs @@ -106,8 +106,11 @@ impl Cond { /// The most terms a condition may have. pub(crate) const MAX_COND: usize = 1024; -/// The deepest a condition's parentheses may nest once written. -pub(crate) const MAX_COND_DEPTH: usize = 32; +/// The deepest a condition's parentheses may nest once written: what the +/// reader takes of it (a level of parentheses costs it two of +/// [`crate::qasm::MAX_NESTING`], the parenthesis and the operand it is), so a +/// condition that fits this reads back. +pub(crate) const MAX_COND_DEPTH: usize = 30; /// One statement. #[derive(Clone, Debug, PartialEq)] @@ -222,6 +225,24 @@ pub(crate) fn cond_fits(c: &Cond) -> bool { mod tests { use super::*; + /// A condition within the converters' bounds reads back: a flat run as + /// long as they take, and parentheses as deep, of either operator. + #[test] + fn a_condition_that_fits_reads_back() { + let reads = |c: Cond| { + assert!(cond_fits(&c), "size {} depth {}", c.size(), c.depth()); + let n = 64; + let body = vec![Stmt::Measure { qubit: 0, bit: 0 }, Stmt::If { cond: c, then: vec![Stmt::gate("x", vec![], vec![0])], other: vec![] }]; + let text = Circuit { n_qubits: 1, n_bits: n, body }.to_qasm("read from a test"); + crate::qasm::read(&text).unwrap_or_else(|d| panic!("{d}: {text}")); + }; + let bit = |k: usize| Cond::Bit((k % 64) as u32); + let run = |and: bool, n: usize| (1..n).fold(bit(0), |a, k| if and { Cond::And(Box::new(a), Box::new(bit(k))) } else { Cond::Or(Box::new(a), Box::new(bit(k))) }); + reads(run(true, MAX_COND / 2)); + reads(run(false, MAX_COND / 2)); + reads((0..MAX_COND_DEPTH).fold(bit(0), |c, k| if k % 2 == 0 { Cond::And(Box::new(bit(k + 1)), Box::new(c)) } else { Cond::Or(Box::new(bit(k + 1)), Box::new(c)) })); + } + #[test] fn a_circuit_writes_as_openqasm_and_reads_back() { let c = Circuit { diff --git a/wai/wai-quantum/src/engine/refusal.rs b/wai/wai-quantum/src/engine/refusal.rs index 08b760a5a..1d3dcd08f 100644 --- a/wai/wai-quantum/src/engine/refusal.rs +++ b/wai/wai-quantum/src/engine/refusal.rs @@ -90,14 +90,14 @@ table! { FABRIC = "J013", Refused, "A placement on the fabric (`fabric`) this host cannot honour: no member by that name, no certified accelerator member here, a state or a partition larger than a member holds, a partition count that is not a power of two, or a method or program the fabric's members do not run (the message says which, and what is here)."; // Intents and repairs. - BAD_INTENT = "J020", Refused, "An intent that does not read: not one of `distribution`, `equivalent` or `prepares`, a distribution whose outcomes are not the program's bits or whose probabilities do not sum to 1, or a reference program that does not read."; + BAD_INTENT = "J020", Refused, "An intent that does not read: not one of `distribution`, `equivalent` or `prepares`, a distribution whose outcomes are not the program's bits or whose probabilities do not sum to 1, or a reference program that does not read, or more intents than the host takes."; REPAIR_REJECTED = "J021", Refused, "A proposed repair that changes a line no diagnostic flags, or adds lines away from one: rejected, and the program left as it was."; BAD_REPAIR = "J022", Refused, "A repair's choices that do not read (`max_iterations`, `dyadic_bits`, `sizes`), or a proposal that is not a program's text."; // The host. RATE = "J030", Retry, "The host takes no more from this client for now (submissions in a window, or jobs at once): send it again shortly."; - TOO_LARGE_HERE = "J031", Refused, "A job or a check larger than this host takes: run it on your own machine, with the command line or its MCP server."; - BUSY = "J032", Retry, "The host is full (its queue, or the checks running at once): send it again later."; + TOO_LARGE_HERE = "J031", Refused, "A job or a check larger than this host takes, its reading included: run it on your own machine, with the command line or its MCP server."; + BUSY = "J032", Retry, "The host is full (its queue, the checks running at once, or the requests being read, yours or all of them): send it again later."; KEY_REQUIRED = "J033", Refused, "A host that takes requests only with one of its API keys: give yours as `Authorization: Bearer `; a key it does not hold is refused the same way."; IDEMPOTENCY_KEY = "J034", Refused, "An idempotency key (`Idempotency-Key`, or a tool's `idempotency_key`) that does not read, or that this client sent before with another request: a key names one job, so another request needs another key. A key is 1 to 255 visible ASCII characters, and should be unguessable (a UUID, or 128 random bits)."; diff --git a/wai/wai-quantum/src/engine/sets.rs b/wai/wai-quantum/src/engine/sets.rs index 969a67a10..858406f5a 100644 --- a/wai/wai-quantum/src/engine/sets.rs +++ b/wai/wai-quantum/src/engine/sets.rs @@ -409,8 +409,9 @@ pub fn verify(job: &Value) -> Value { // The seal alone; the rules on its numbers, by its form, are §8.6's (`job::check`). let signed = receipt::verify(request, &head, set_receipt); let items = result["items"].as_array().cloned().unwrap_or_default(); - let derived = match expand(request) { - Ok(want) => { + // Writing a sweep out reads its programs: for a seal that holds, and no other. + let derived = match signed.then(|| expand(request).ok()).flatten() { + Some(want) => { want.len() == items.len() && want.iter().zip(&items).enumerate().all(|(i, (w, got))| { got["index"].as_u64() == Some(i as u64) @@ -418,7 +419,7 @@ pub fn verify(job: &Value) -> Value { && w.point.as_ref().is_none_or(|p| receipt::canonical(p) == receipt::canonical(&got["point"])) }) } - Err(_) => false, + None => false, }; let count_ok = result["count"].as_u64() == Some(items.len() as u64); let each = items.iter().all(|it| verify_item(request, &head, set_receipt, it)); @@ -454,6 +455,15 @@ mod tests { SigningKey::from_bytes(&[5; 32]) } + /// Writing a sweep out reads it: a seal that does not hold is told so + /// without that, so forging one buys no reading. + #[test] + fn a_seal_that_does_not_hold_is_not_written_out() { + let forged = json!({ "request": { "qasm": "input int n;\nqubit q;\nfor int i in [1:n] { x q; }\n", "sweep": { "n": [1000] } }, "result": { "format": FORMAT, "count": 0, "items": [], "receipt": {} } }); + let (v, left) = crate::qasm::within_steps(1_000_000, || (verify(&forged), crate::qasm::steps_left())); + assert_eq!((v["signed"].clone(), v["derived"].clone(), left), (json!(false), json!(false), Some(1_000_000))); + } + #[test] fn a_list_of_programs_is_one_job_each_the_set_s_fields_shared() { let set = json!({ "programs": [BELL, { "qasm": "OPENQASM 3.0; qubit q; rx(0.3) q;", "observables": ["Z0"], "shots": 0 }], "shots": 100, "seed": 7, "dry_run": false }); diff --git a/wai/wai-quantum/src/engine/verdict.rs b/wai/wai-quantum/src/engine/verdict.rs index 200635c0f..3bbdb5d36 100644 --- a/wai/wai-quantum/src/engine/verdict.rs +++ b/wai/wai-quantum/src/engine/verdict.rs @@ -720,6 +720,18 @@ pub fn of_refusal(request: &Value, error: &str) -> Value { v } +/// The verdict of a job a service admitted, given without reading it again +/// (a client's every poll would otherwise read it): admission read it, so it +/// reads; `error` is why it failed since, if it did. +pub fn of_admitted(error: Option<&str>) -> Value { + let mut v = base(json!({ "ok": true, "diagnostics": [] })); + if let Some(e) = error { + v["refusal"] = refusal::refusal_json(e); + } + conclude(&mut v); + v +} + /// The verdict of a pulse run ([`super::pulse`]): what the pulse reader /// said of the program (the result's diagnostics), and the run's method and /// accuracy. A pulse run checks no intent. diff --git a/wai/wai-quantum/src/engine/views.rs b/wai/wai-quantum/src/engine/views.rs index 56cc7e71a..8fbc650ef 100644 --- a/wai/wai-quantum/src/engine/views.rs +++ b/wai/wai-quantum/src/engine/views.rs @@ -217,13 +217,21 @@ fn program_of<'a>(tool: &str, args: &'a Value, answer: &'a Value) -> Option<&'a /// view, the tool, the program, its layout and description (or why it is not /// drawn), and for a target, its map. `None` for a tool with no view. pub fn data(tool: &str, args: &Value, answer: &Value) -> Option { + data_of(tool, args, answer, true) +} + +/// [`data`], without reading anything where `draw` is false: the view, the +/// tool and the program, for an answer that cannot pay to read it again (a +/// poll, which a view makes every few seconds). The view shows the answer, +/// and no drawing. +pub fn data_of(tool: &str, args: &Value, answer: &Value, draw: bool) -> Option { let view = for_tool(tool)?; let mut d = json!({ "view": view.name(), "tool": tool, "protocol": PROTOCOL }); match view { View::Result | View::Circuit | View::Critique => { let src = program_of(tool, args, answer); d["program"] = json!(src); - if let (Some(o), Some(x)) = (d.as_object_mut(), drawing(src).as_object()) { + if draw && let (Some(o), Some(x)) = (d.as_object_mut(), drawing(src).as_object()) { o.extend(x.clone()); } } diff --git a/wai/wai-quantum/src/qasm.rs b/wai/wai-quantum/src/qasm.rs index 4c222e7d7..b63f25f58 100644 --- a/wai/wai-quantum/src/qasm.rs +++ b/wai/wai-quantum/src/qasm.rs @@ -50,6 +50,7 @@ mod unroll; pub mod pulse; pub use diag::{Diagnostic, Outcome, Span}; +pub use elab::MAX_STEPS; #[cfg(feature = "quantum_qasm")] use crate::gates::{C, Gate as Matrix, complex}; diff --git a/wai/wai-quantum/src/qasm/ast.rs b/wai/wai-quantum/src/qasm/ast.rs index 56e5d78cb..9e2af2c41 100644 --- a/wai/wai-quantum/src/qasm/ast.rs +++ b/wai/wai-quantum/src/qasm/ast.rs @@ -54,6 +54,12 @@ pub enum E { Index(Box, Vec), Unary(&'static str, Box), Binary(&'static str, Box, Box), + /// A run of left-associative operators of one precedence, `a + b - c`: + /// its first operand and each operator with the operand after it, applied + /// left to right. Written flat, so that a run of any length holds no + /// recursion: reading it, evaluating it and dropping it are each a loop. + /// A single operator is a [`E::Binary`]. + Chain(Box, Vec<(&'static str, Expr)>), /// A built-in function, a subroutine or an `extern`. Call(Name, Vec), Cast(Type, Box), @@ -176,7 +182,10 @@ pub enum S { /// `x = e;`, `x += e;`, `c[0] = measure q[0];`. Assign { target: Expr, op: &'static str, value: Expr }, Expr(Expr), - If { cond: Expr, then: Vec, other: Option> }, + /// `if (c) … else if (d) … else …`: each condition with its branch, in + /// order, and the last `else`. An `else if` chain is flat, as a run of + /// operators is. + If { arms: Vec<(Expr, Vec)>, other: Option> }, For { ty: Option, var: Name, over: Expr, body: Vec }, While { cond: Expr, body: Vec }, Switch { on: Expr, cases: Vec, default: Option> }, diff --git a/wai/wai-quantum/src/qasm/depth_tests.rs b/wai/wai-quantum/src/qasm/depth_tests.rs index f7ff34ba8..61f8c531d 100644 --- a/wai/wai-quantum/src/qasm/depth_tests.rs +++ b/wai/wai-quantum/src/qasm/depth_tests.rs @@ -58,9 +58,8 @@ fn nested(kind: &str, n: usize) -> String { /// Every construct that nests, each read by a recursion of the parser, the /// elaborator, or both. -const NESTING: [&str; 25] = [ - "paren", "minus", "not", "tilde", "sum", "power", "call", "cast", "set", "index", "concat", "block", "if", "if_block", "else_if", "for", "while", "switch", "box", "gate_body", "def_body", "complex", "array", - "durationof", "measure", +const NESTING: [&str; 23] = [ + "paren", "minus", "not", "tilde", "power", "call", "cast", "set", "index", "concat", "block", "if", "if_block", "for", "while", "switch", "box", "gate_body", "def_body", "complex", "array", "durationof", "measure", ]; /// Nested thousands deep, as a request of a few kilobytes can be, each is @@ -72,9 +71,9 @@ fn every_construct_nested_past_the_limit_is_refused_not_overflowed() { assert_eq!(read_on_stack(nested(kind, n)), Err(diag::NESTING), "{kind} nested {n} deep"); } } - // Chains read in a loop build their tree as deep as they are long: a - // sum, a power, an index or a `++` of a hundred thousand is refused too. - for kind in ["sum", "power", "index", "concat", "minus", "not"] { + // Chains whose tree is as deep as they are long: a power, an index or a + // `++` of a hundred thousand is refused too. + for kind in ["power", "index", "concat", "minus", "not"] { assert_eq!(read_on_stack(nested(kind, 100_000)), Err(diag::NESTING), "{kind} 100000 long"); } } @@ -105,6 +104,120 @@ fn every_construct_nested_inside_the_limit_reads() { assert_eq!(p.ops[0].params, vec![if (MAX_NESTING - 3) % 2 == 0 { 0.5 } else { -0.5 }]); } +/// A run of operators of one precedence, and an `else if` chain, are flat: +/// read, run and dropped in loops, so they read at any length, on a stack of +/// 1 MiB, and only what nests counts. Each reads to what the same text in +/// parentheses gives, left to right. +#[test] +fn a_run_of_operators_and_an_else_if_chain_read_at_any_length() { + const SMALL: usize = 1 << 20; + let reads = |src: String| on_stack(SMALL, move || read(&src).map(|r| r.program.ops.len()).map_err(|d| d.code)); + for n in [MAX_NESTING + 1, 100_000] { + let run = |op: &str, term: &str| format!("{term}{}", format!(" {op} {term}").repeat(n)); + for (what, src) in [ + ("sum", format!("{HEAD}rz({}) q;\n", run("+", "0.5"))), + ("product", format!("{HEAD}rz({}) q;\n", run("*", "1.0"))), + ("declared", format!("{HEAD}float x = {};\nx q;\n", run("-", "0.5"))), + ("or", format!("{HEAD}bool b = {};\nx q;\n", run("||", "false"))), + ("and", format!("{HEAD}bool b = {};\nx q;\n", run("&&", "true"))), + ("known bits", format!("{HEAD}if ({}) x q;\n", run("&&", "true"))), + ("measured bits", format!("{HEAD}c = measure q;\nif ({}) x q;\n", run("&&", "c"))), + ("measured bits or", format!("{HEAD}c = measure q;\nif ({}) x q;\n", run("||", "c"))), + ("else if known", format!("{HEAD}{}x q;\n", "if (false) y q; else ".repeat(n))), + ("else if measured", format!("{HEAD}c = measure q;\n{}x q;\n", "if (c == 1) y q; else ".repeat(n))), + ] { + assert!(reads(src).is_ok_and(|ops| ops >= 1), "{what} of {n}"); + } + } + // Left to right, in the double's own order: not rebalanced. + let terms = [0.1, 0.7, 1e-17, 0.3, 1e17, -1e17, 0.2]; + let flat: String = terms.iter().map(|t| format!("{t:e}")).collect::>().join(" + "); + let want = terms[1..].iter().fold(terms[0], |a, t| a + t); + let p = parse(&format!("{HEAD}rz({flat}) q;\n")).unwrap(); + assert_eq!(p.ops[0].params, vec![want]); + for (run, tree) in [("7 - 2 + 3 - 4", "((7 - 2) + 3) - 4"), ("2 * 7 / 4 % 3", "((2 * 7) / 4) % 3"), ("1 << 3 >> 1 << 2", "((1 << 3) >> 1) << 2"), ("1 < 2 == true != false", "((1 < 2) == true) != false")] { + let v = |e: &str| parse(&format!("{HEAD}const int k = {e};\nqubit[32] r;\nx r[k];\n")).map(|p| p.ops[0].qubits.clone()); + if run.contains('<') && run.contains("==") { + let v = |e: &str| parse(&format!("{HEAD}if ({e}) x q;\n")).map(|p| p.ops.len()); + assert_eq!(v(run), v(tree), "{run}"); + } else { + assert_eq!(v(run), v(tree), "{run}"); + } + } + // A derivative runs along a run too, as it does along the tree. + let names = ["a".to_string()]; + for (run, tree) in [("a + 2.0 - a * 3.0 + 1.5", "((a + 2.0) - (a * 3.0)) + 1.5"), ("a * 2.0 * a / 3.0", "((a * 2.0) * a) / 3.0")] { + let derivative = |e: &str| tangents(&format!("{HEAD}const float a = 0.7;\nrz({e}) q;\n"), &names).map(|(_, t)| t).map_err(|d| d.code); + assert_eq!(derivative(run), derivative(tree), "{run}"); + } + // An `else if` chain's steps are the ones the same branches nested write, + // and what its branches assign differently is no longer known after it. + let chain = "if (c == 1) x q; else if (c == 0) y q; else if (true) z q; else h q;"; + let nest = "if (c == 1) x q; else { if (c == 0) y q; else { if (true) z q; else h q; } }"; + let steps = |body: &str| read(&format!("{HEAD}c = measure q;\n{body}\n")).unwrap().program; + assert_eq!(steps(chain), steps(nest)); + let after = |a: u32, b: u32, c: u32| read_on_stack(format!("{HEAD}qubit[3] r;\nc = measure q;\nint k = 0;\nif (c == 1) k = {a}; else if (c == 0) k = {b}; else k = {c};\nx r[k];\n")); + assert_eq!(after(1, 1, 1), Ok(1)); + assert_eq!(after(1, 2, 1), Err(diag::NOT_KNOWN)); + assert_eq!(after(1, 1, 2), Err(diag::NOT_KNOWN)); +} + +/// A switch's case list on measured bits makes one condition of every value +/// listed: held flat, however many there are, on a stack of 1 MiB. +#[test] +fn a_switch_on_measured_bits_lists_any_number_of_values() { + let src = format!("OPENQASM 3.0;\ninclude \"stdgates.inc\";\nqubit[2] q;\nbit[2] c;\nc = measure q;\nswitch (c) {{ case {}1 {{ x q[0]; }} }}\n", "1, ".repeat(600_000)); + let r = on_stack(1 << 20, move || read(&src).map(|r| r.program.ops.len()).map_err(|d| d.code)); + assert!(r.is_ok_and(|ops| ops >= 1), "{r:?}"); +} + +/// An array's dimensions cost the text no level, as they are a comma apart, +/// but each is a level of its value, which what copies, measures and writes +/// it recurses on: so they are held, where the type is read. +#[test] +fn an_array_past_its_dimensions_is_refused_where_its_type_is_read() { + let array = |n: usize| format!("{HEAD}array[int[8]{}] a;\nx q;\n", ", 1".repeat(n)); + assert_eq!(read_on_stack(array(crate::qasm::parse::MAX_ARRAY_DIMS)), Ok(1)); + for n in [crate::qasm::parse::MAX_ARRAY_DIMS + 1, 100_000] { + assert_eq!(read_on_stack(array(n)), Err(diag::NESTING), "{n} dimensions"); + } +} + +/// What a generator's loops cost is spent from the budget as they are written +/// out: a pass of a loop that writes nothing, and what a loop writes, by its +/// size, so that a sweep's points are held to the request's one budget. +#[test] +fn a_generator_s_passes_and_what_they_write_spend_the_budget() { + let (r, left) = within_steps(1_000, || (instantiate("input int n;\nqubit q;\nfor int i in [1:n] { }\nx q;\n", &[("n".to_string(), 999_999)].into_iter().collect()), steps_left())); + assert!(r.is_err_and(|e| e.contains("Q312"))); + assert_eq!(left, Some(0)); + let body = "x q;\n".repeat(1_000); + let fat = format!("input int n;\nqubit q;\nfor int i in [1:n] {{\n{body}}}\n"); + let (r, left) = within_steps(1_000, || (instantiate(&fat, &[("n".to_string(), 3_000)].into_iter().collect()), steps_left())); + assert!(r.is_err_and(|e| e.contains("Q312"))); + assert_eq!(left, Some(0)); +} + +/// A generator's integer `const`s fold in the writer, and both readers of a +/// generator agree on a flat sum, and an `else if` chain, of any length. +#[test] +fn both_readers_of_a_generator_take_a_flat_sum() { + let src = format!("input int nn;\nqubit[1] q;\nconst int s = nn{};\nfor int i in [0:s % 3] {{ x q[0]; }}\n", " + nn".repeat(999)); + let inputs: Inputs = [("nn".to_string(), 2)].into_iter().collect(); + let direct = read_with(&src, &inputs).unwrap(); + let written = read(&instantiate(&src, &inputs).unwrap()).unwrap(); + assert_eq!(direct.program.ops.len(), written.program.ops.len()); + // An `else if` chain as well, decided on writing or left to the reader. + for arm in ["if (nn == 0) x q[0]; else ", "if (true) x q[0]; else "] { + let src = format!("input int nn;\nqubit[1] q;\n{}x q[0];\n", arm.repeat(5_000)); + let (a, b) = on_stack(1 << 20, move || { + let inputs: Inputs = [("nn".to_string(), 2)].into_iter().collect(); + (read_with(&src, &inputs).map(|r| r.program.ops.len()), instantiate(&src, &inputs).and_then(|w| read(&w).map(|r| r.program.ops.len()).map_err(|d| d.to_string()))) + }); + assert_eq!(a.map_err(|d| d.to_string()), b, "{arm}"); + } +} + /// A modifier chain is a list, and an `include` a statement of its own: /// neither nests, so either reads at any length (the gates the chain makes /// are bounded by the reader's gate limit). @@ -148,7 +261,6 @@ fn a_generator_nested_past_the_limit_is_refused_when_written_out() { ("power", format!("{HEAD}const int a = 1{};\nx q;\n", " ** 1".repeat(10_000))), ("loops", format!("{HEAD}{}x q;{}\n", "for int i in [0:0] { ".repeat(5_000), " }".repeat(5_000))), ("conditions", format!("{HEAD}const int k = 1;\n{}x q;\n", "if (k == 1) ".repeat(5_000))), - ("else if", format!("{HEAD}const int k = 1;\n{}x q;\n", "if (k == 0) x q; else ".repeat(5_000))), ] { assert_eq!(inst(src), Err("Q006".into()), "{what}"); } diff --git a/wai/wai-quantum/src/qasm/diag.rs b/wai/wai-quantum/src/qasm/diag.rs index 020ec5e12..5524ea4b2 100644 --- a/wai/wai-quantum/src/qasm/diag.rs +++ b/wai/wai-quantum/src/qasm/diag.rs @@ -163,7 +163,7 @@ table! { UNCLOSED = "Q003", Refused, "A string, a block comment, a bracket or a calibration block that is not closed."; EXPECTED = "Q004", Refused, "Something else was expected here: a `;`, a name, an expression or a statement (the message says which)."; MISPLACED = "Q005", Refused, "A statement in a place the language does not allow it: `else` with no `if`, `case` outside `switch`, a declaration inside a gate."; - NESTING = "Q006", Refused, "A program nested deeper than the reader takes: statements, parentheses, operators, brackets and types more than 64 levels deep in its text, or more than 256 levels in all through the gates and subroutines it calls; refused, never read by a recursion that could run out of stack."; + NESTING = "Q006", Refused, "A program nested deeper than the reader takes: statements, parentheses, operators, brackets and types more than 64 levels deep in its text (a run of one precedence, `a + b + c`, and an `else if` chain each count one, however long), an array of more than 7 dimensions, or more than 256 levels in all through the gates and subroutines it calls; refused, never read by a recursion that could run out of stack."; // Versions, includes, pragmas and annotations. UNKNOWN_VERSION = "Q010", Warned, "An `OPENQASM` version other than 2.x or 3.x: the program is read as OpenQASM 3."; @@ -232,7 +232,7 @@ table! { WHILE_UNROLLED = "Q309", Lowered, "`while` whose condition is known on reading: unrolled until it fails."; UNBOUNDED_WHILE = "Q310", Refused, "A `while` whose condition reads measured bits and that declares no bound: add `@wai.max_iterations N` before it."; WHILE_CUT = "Q311", Warned, "A `while` on measured bits run at most its declared bound (`@wai.max_iterations N`): a run that still meets its condition after N iterations stops looping there."; - UNROLL_LIMIT = "Q312", Refused, "A program that does not finish unrolling within the reader's bound (20 000 000 statements and loop iterations)."; + UNROLL_LIMIT = "Q312", Refused, "A program that does not finish unrolling within the reader's bound (20 000 000 statements and loop iterations), or that runs out a caller's budget of reading (`qasm::within_steps`: statements, gates and a generator's loop passes, shared by every program read under it)."; BAD_RANGE = "Q313", Refused, "A range with a zero step, or that does not read as `[start:end]` or `[start:step:end]`."; OUTSIDE_LOOP = "Q314", Refused, "`break` or `continue` outside a loop, or `return` outside a subroutine."; RUNTIME_RETURN = "Q315", Refused, "A `return` with a value under a condition on measured bits."; diff --git a/wai/wai-quantum/src/qasm/elab.rs b/wai/wai-quantum/src/qasm/elab.rs index 3f1aba049..841c17a78 100644 --- a/wai/wai-quantum/src/qasm/elab.rs +++ b/wai/wai-quantum/src/qasm/elab.rs @@ -51,7 +51,7 @@ const ENTRY_BYTES: u64 = 48; /// 2^m − 1 `cx` and as many `p`. pub(crate) const MAX_CONTROLS: u32 = 16; /// The most statements and loop iterations a program may run on reading. -pub(crate) const MAX_STEPS: u64 = 20_000_000; +pub const MAX_STEPS: u64 = 20_000_000; /// The most times one `while` may run on reading. const MAX_WHILE: u64 = 1_000_000; /// Bits a block or subroutine declares for itself are numbered from here @@ -180,8 +180,25 @@ enum Ct { Const(bool), Atom(Cond), Not(Box), - And(Box, Box), - Or(Box, Box), + /// Every one holds, tried in order. A run is held flat, so a condition or + /// a case list of any length is walked by a loop. + All(Vec), + /// One of them holds, tried in order. + Any(Vec), +} + +/// An `if` chain's branches on measured bits, as they are read. +struct Branching { + /// Conditions on measured bits read so far. + runtime: usize, + /// The scopes the first of them was reached with. + start: Option>, + /// The scopes of the branches that go on, joined. + joined: Option>, + /// How the rest ended, the least of them. + flow: Option, + /// Jumps from a branch's end past the rest. + ends: Vec, } #[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)] @@ -946,16 +963,8 @@ impl<'a> Elab<'a> { if bits.is_empty() { Ct::Const(c.equal) } else { Ct::Atom(Cond { bits, value, equal: c.equal }) } } Ct::Not(x) => negate(self.fold(*x)), - Ct::And(a, b) => match (self.fold(*a), self.fold(*b)) { - (Ct::Const(false), _) | (_, Ct::Const(false)) => Ct::Const(false), - (Ct::Const(true), t) | (t, Ct::Const(true)) => t, - (a, b) => Ct::And(Box::new(a), Box::new(b)), - }, - Ct::Or(a, b) => match (self.fold(*a), self.fold(*b)) { - (Ct::Const(true), _) | (_, Ct::Const(true)) => Ct::Const(true), - (Ct::Const(false), t) | (t, Ct::Const(false)) => t, - (a, b) => Ct::Or(Box::new(a), Box::new(b)), - }, + Ct::All(list) => joined(list.into_iter().map(|t| self.fold(t)), true), + Ct::Any(list) => joined(list.into_iter().map(|t| self.fold(t)), false), t => t, } } @@ -998,14 +1007,13 @@ impl<'a> Elab<'a> { self.steps.push(Instr::JumpUnless(c.clone(), usize::MAX)); } Ct::Not(x) => self.jump_true(x, patches), - Ct::And(a, b) => { - self.jump_false(a, patches); - self.jump_false(b, patches); - } - Ct::Or(a, b) => { + Ct::All(list) => list.iter().for_each(|t| self.jump_false(t, patches)), + Ct::Any(list) => { let mut yes = Vec::new(); - self.jump_true(a, &mut yes); - self.jump_false(b, patches); + if let Some((last, first)) = list.split_last() { + first.iter().for_each(|t| self.jump_true(t, &mut yes)); + self.jump_false(last, patches); + } let h = self.here(); self.patch(&yes, h); } @@ -1022,14 +1030,13 @@ impl<'a> Elab<'a> { self.steps.push(Instr::JumpUnless(Cond { equal: !c.equal, ..c.clone() }, usize::MAX)); } Ct::Not(x) => self.jump_false(x, patches), - Ct::Or(a, b) => { - self.jump_true(a, patches); - self.jump_true(b, patches); - } - Ct::And(a, b) => { + Ct::Any(list) => list.iter().for_each(|t| self.jump_true(t, patches)), + Ct::All(list) => { let mut no = Vec::new(); - self.jump_false(a, &mut no); - self.jump_true(b, patches); + if let Some((last, first)) = list.split_last() { + first.iter().for_each(|t| self.jump_false(t, &mut no)); + self.jump_true(last, patches); + } let h = self.here(); self.patch(&no, h); } @@ -1097,7 +1104,7 @@ impl<'a> Elab<'a> { self.annotations(s); } match &s.kind { - S::If { cond, then, other } => self.if_stmt(cond, then, other.as_deref()), + S::If { arms, other } => self.if_stmt(arms, other.as_deref()), S::For { ty, var, over, body } => self.for_stmt(ty.as_ref(), var, over, body, s.at), S::While { cond, body } => self.while_stmt(cond, body, s), S::Switch { on, cases, default } => self.switch(on, cases, default.as_deref(), s.at), @@ -1327,7 +1334,7 @@ impl<'a> Elab<'a> { match &s.kind { S::Call { operands, .. } | S::Barrier(operands) => operands.iter().try_for_each(check), S::For { body, .. } | S::Block(body) | S::Box { body, .. } => body.iter().try_for_each(|st| self.check_gate_operands(st, gate, qubits)), - S::If { then, other, .. } => then.iter().chain(other.iter().flatten()).try_for_each(|st| self.check_gate_operands(st, gate, qubits)), + S::If { arms, other } => arms.iter().flat_map(|(_, then)| then).chain(other.iter().flatten()).try_for_each(|st| self.check_gate_operands(st, gate, qubits)), _ => Ok(()), } } @@ -1914,6 +1921,10 @@ impl<'a> Elab<'a> { Self::walk(a, f); Self::walk(b, f); } + E::Chain(first, rest) => { + Self::walk(first, f); + rest.iter().for_each(|(_, x)| Self::walk(x, f)); + } E::Call(_, args) | E::Set(args) => args.iter().for_each(|a| Self::walk(a, f)), E::Range(a, b, c) => [a, b, c].into_iter().flatten().for_each(|x| Self::walk(x, f)), _ => {} @@ -1928,6 +1939,26 @@ impl<'a> Elab<'a> { self.deeper(e.at, |x| x.tangent_here(e)) } + /// The derivative of a run of `*` and `/`, `t` being its first operand's: + /// the product and quotient rules applied left to right, on the value of + /// the run so far. The caller holds [`Elab::trunc`] for it. + fn tangent_products(&mut self, first: &Expr, rest: &[(&'static str, Expr)], mut t: Vec) -> R> { + let mut v = self.known(first)?; + let mut whole = first.at; + for (op, x) in rest { + self.tick(x.at)?; + let (va, vb) = (self.real_of(v.clone(), self.trunc, whole)?, self.real(x)?); + let tx = self.tangent(x)?; + // (a·b)' = a'·b + a·b'; (a/b)' = a'/b − a·b'/b² + let (p, q) = if *op == "*" { (vb, va) } else { (1.0 / vb, -va / (vb * vb)) }; + t = t.iter().zip(&tx).map(|(a, b)| p * a + q * b).collect(); + whole = whole.to(x.at); + let y = self.known(x)?; + v = self.binop(op, v, y, whole)?; + } + Ok(t) + } + fn tangent_here(&mut self, e: &Expr) -> R> { let zero = self.tan.as_ref().map(Tan::zero).unwrap_or_default(); if !self.depends(e) { @@ -1965,6 +1996,27 @@ impl<'a> Elab<'a> { ta.iter().map(|t| k * t).collect() } E::Binary(op, _, _) => return refuse(self, &format!("`{op}`")), + E::Chain(first, rest) => { + let sums = rest.iter().all(|(op, _)| matches!(*op, "+" | "-")); + if !sums && let Some((op, _)) = rest.iter().rev().find(|(op, _)| !matches!(*op, "*" | "/")) { + return refuse(self, &format!("`{op}`")); + } + let t = self.tangent(first)?; + if sums { + let mut t = t; + for (op, x) in rest { + self.tick(x.at)?; + let tx = self.tangent(x)?; + t = axpy(1.0, &t, if *op == "+" { 1.0 } else { -1.0 }, &tx); + } + t + } else { + let saved = std::mem::replace(&mut self.trunc, false); + let r = self.tangent_products(first, rest, t); + self.trunc = saved; + r? + } + } E::Call(f, args) if args.len() == 1 => { let x = self.real(&args[0])?; let t = self.tangent(&args[0])?; @@ -2205,10 +2257,17 @@ impl<'a> Elab<'a> { self.scopes.pop(); r?; } - S::If { cond, then, other } => { - let c = self.known(cond)?; - let Some(b) = c.truthy() else { return self.fail(diag::WRONG_TYPE, cond.at, "a condition is true or false") }; - let body = if b { Some(then.as_slice()) } else { other.as_deref() }; + S::If { arms, other } => { + let mut body = other.as_deref(); + for (cond, then) in arms { + self.tick(cond.at)?; + let c = self.known(cond)?; + let Some(b) = c.truthy() else { return self.fail(diag::WRONG_TYPE, cond.at, "a condition is true or false") }; + if b { + body = Some(then.as_slice()); + break; + } + } for st in body.unwrap_or(&[]) { self.gate_stmt(st, prims, depth)?; } @@ -2490,68 +2549,96 @@ impl<'a> Elab<'a> { // Control flow // ----------------------------------------------------------------------- - fn if_stmt(&mut self, cond: &Expr, then: &[Stmt], other: Option<&[Stmt]>) -> R { - let t = self.cond(cond)?; - if let Ct::Const(b) = t { - // Decided on reading: the branch taken is read in place. - return match (b, other) { - (true, _) => self.block(then), - (false, Some(o)) => self.block(o), - (false, None) => Ok(Flow::Normal), - }; - } - self.dynamic = true; - let mut no = Vec::new(); - self.jump_false(&t, &mut no); - let before = self.scopes.clone(); - self.runtime += 1; - let ft = self.scoped(then); - let ft = match ft { - Ok(f) => f, - Err(e) => { - self.runtime -= 1; - return Err(e); + /// An `if` and its `else if`s, read in one loop however many there are. + /// A condition decided on reading leaves its branch, or the rest, to be + /// read in place. One on measured bits jumps past its branch; the rest + /// then run at run time, from the scopes it was reached with, and where + /// the branches join, what they assigned differently is no longer known. + fn if_stmt(&mut self, arms: &[(Expr, Vec)], other: Option<&[Stmt]>) -> R { + let mut branching = Branching { runtime: 0, start: None, joined: None, flow: None, ends: Vec::new() }; + let r = self.if_arms(arms, other, &mut branching); + self.runtime -= branching.runtime; + let f = r?; + let Some(start) = branching.start else { return Ok(f) }; + let h = self.here(); + self.patch(&branching.ends, h); + match branching.joined { + Some(scopes) => { + self.scopes = scopes; + Ok(Flow::Normal) + } + None => { + self.scopes = start; + Ok(branching.flow.unwrap_or(Flow::Normal)) } - }; - let mut outs = Vec::new(); - if ft == Flow::Normal { - outs.push(self.scopes.clone()); } - let fe = if let Some(o) = other { - let mut j = Vec::new(); - self.jump(&mut j); - let h = self.here(); - self.patch(&no, h); - self.scopes = before.clone(); - let fe = self.scoped(o); - let fe = match fe { - Ok(f) => f, - Err(e) => { - self.runtime -= 1; - return Err(e); + } + + /// [`Elab::if_stmt`]'s loop: the flow of the branch taken, where every + /// condition was decided on reading. + fn if_arms(&mut self, arms: &[(Expr, Vec)], other: Option<&[Stmt]>, b: &mut Branching) -> R { + let mut last = other; + for (i, (cond, then)) in arms.iter().enumerate() { + if i > 0 { + self.tick(cond.at)?; + } + let t = self.cond(cond)?; + if let Ct::Const(holds) = t { + if holds { + last = Some(then); + break; } - }; - if fe == Flow::Normal { - outs.push(self.scopes.clone()); + continue; } - let h = self.here(); - self.patch(&j, h); - fe - } else { + self.dynamic = true; + let mut no = Vec::new(); + self.jump_false(&t, &mut no); + let before = self.scopes.clone(); + b.start.get_or_insert_with(|| before.clone()); + self.runtime += 1; + b.runtime += 1; + let f = self.scoped(then)?; + self.leave_branch(f, b); + if i + 1 == arms.len() && other.is_none() { + let h = self.here(); + self.patch(&no, h); + self.scopes = before; + self.leave_branch(Flow::Normal, b); + return Ok(Flow::Normal); + } + self.jump(&mut b.ends); let h = self.here(); self.patch(&no, h); - outs.push(before.clone()); - Flow::Normal - }; - self.runtime -= 1; - if outs.is_empty() { self.scopes = before; - return Ok(ft.min(fe)); } - self.scopes = self.merge(outs); + if b.start.is_none() { + // Every condition was decided on reading: read in place. + return match last { + Some(stmts) => self.block(stmts), + None => Ok(Flow::Normal), + }; + } + let f = match last { + Some(stmts) => self.scoped(stmts)?, + None => Flow::Normal, + }; + self.leave_branch(f, b); Ok(Flow::Normal) } + /// A branch left with `f`: one that goes on joins the others. + fn leave_branch(&mut self, f: Flow, b: &mut Branching) { + if f != Flow::Normal { + b.flow = Some(b.flow.map_or(f, |g| g.min(f))); + return; + } + let here = self.scopes.clone(); + b.joined = Some(match b.joined.take() { + Some(so_far) => self.merge(vec![so_far, here]), + None => here, + }); + } + fn loop_values(&mut self, over: &Expr) -> R { match &over.kind { E::Range(a, s, b) => { @@ -2759,7 +2846,7 @@ impl<'a> Elab<'a> { let a = self.compare_bits(&bits, &name, "==", y, e.at)?; t = Some(match t { None => a, - Some(p) => Ct::Or(Box::new(p), Box::new(a)), + Some(p) => either(p, a), }); } let t = t.unwrap_or(Ct::Const(false)); @@ -2804,30 +2891,39 @@ impl<'a> Elab<'a> { self.deeper(e.at, |x| x.cond_here(e)) } + /// `left` `op` `right` (`&&` or `||`): `right` is read only where `left` + /// leaves it to decide. + fn cond_then(&mut self, op: &str, left: Ct, right: &Expr) -> R { + let (stops, join): (bool, fn(Ct, Ct) -> Ct) = if op == "&&" { (false, both) } else { (true, either) }; + Ok(match left { + Ct::Const(b) if b == stops => Ct::Const(b), + Ct::Const(_) => self.cond(right)?, + t => match self.cond(right)? { + Ct::Const(b) if b == stops => Ct::Const(b), + Ct::Const(_) => t, + u => join(t, u), + }, + }) + } + fn cond_here(&mut self, e: &Expr) -> R { Ok(match &e.kind { E::Unary("!", x) => match self.cond(x)? { Ct::Const(b) => Ct::Const(!b), t => Ct::Not(Box::new(t)), }, - E::Binary("&&", a, b) => match self.cond(a)? { - Ct::Const(false) => Ct::Const(false), - Ct::Const(true) => self.cond(b)?, - ta => match self.cond(b)? { - Ct::Const(true) => ta, - Ct::Const(false) => Ct::Const(false), - tb => Ct::And(Box::new(ta), Box::new(tb)), - }, - }, - E::Binary("||", a, b) => match self.cond(a)? { - Ct::Const(true) => Ct::Const(true), - Ct::Const(false) => self.cond(b)?, - ta => match self.cond(b)? { - Ct::Const(false) => ta, - Ct::Const(true) => Ct::Const(true), - tb => Ct::Or(Box::new(ta), Box::new(tb)), - }, - }, + E::Binary(op @ ("&&" | "||"), a, b) => { + let left = self.cond(a)?; + self.cond_then(op, left, b)? + } + E::Chain(first, rest) if matches!(rest[0].0, "&&" | "||") => { + let mut t = self.cond(first)?; + for (op, x) in rest { + self.tick(x.at)?; + t = self.cond_then(op, t, x)?; + } + t + } E::Binary(op @ ("==" | "!=" | "<" | "<=" | ">" | ">="), a, b) => { if matches!(*op, "==" | "!=") && let E::Unary("!", inner) = &a.kind @@ -2925,7 +3021,7 @@ impl<'a> Elab<'a> { let c = Ct::Atom(Cond { bits: bits[i..].to_vec(), value: (k >> i) & !1, equal: true }); t = Some(match t { None => c, - Some(p) => Ct::Or(Box::new(p), Box::new(c)), + Some(p) => either(p, c), }); } } @@ -2966,6 +3062,7 @@ impl<'a> Elab<'a> { E::Unary(op, x) => self.ev_unary(op, x, at), E::Binary(op @ ("&&" | "||"), a, b) => self.ev_logic(op, a, b), E::Binary(op, a, b) => self.ev_binary(op, a, b, at), + E::Chain(first, rest) => self.ev_chain(first, rest), E::Call(name, args) => self.call_ev(name, args, at), E::Cast(ty, x) => self.ev_cast(ty, x, at), E::Set(items) => self.ev_set(items), @@ -3006,7 +3103,7 @@ impl<'a> Elab<'a> { E::Measure(_) => return self.fail(diag::MEASURE_PLACE, at, "a measurement gives bits: assign it to a bit register (`c = measure q;`)"), E::Range(..) => return self.fail(diag::WRONG_TYPE, at, "a range is read only in an index or a `for` loop"), E::DurationOf(_) => return self.fail(diag::DURATION_OF, at, "`durationof` needs the duration of each gate, which the program does not give the reader"), - E::Name(_) | E::Index(..) | E::Unary(..) | E::Binary(..) | E::Call(..) | E::Cast(..) | E::Set(_) | E::Concat(..) => unreachable!("read by ev_here"), + E::Name(_) | E::Index(..) | E::Unary(..) | E::Binary(..) | E::Chain(..) | E::Call(..) | E::Cast(..) | E::Set(_) | E::Concat(..) => unreachable!("read by ev_here"), }) } @@ -3024,9 +3121,15 @@ impl<'a> Elab<'a> { #[inline(never)] fn ev_logic(&mut self, op: &str, a: &Expr, b: &Expr) -> R { let x = self.ev(a)?; + self.logic_then(op, x, a.at, b) + } + + /// `x`, read at `left`, `op` `b` (`&&` or `||`): `b` is read only where + /// `x` leaves it to decide. + fn logic_then(&mut self, op: &str, x: Ev, left: Loc, b: &Expr) -> R { Ok(match x { Ev::Val(v) => { - let Some(t) = v.truthy() else { return self.fail(diag::WRONG_TYPE, a.at, format!("`{op}` reads true or false, not {}", v.kind())) }; + let Some(t) = v.truthy() else { return self.fail(diag::WRONG_TYPE, left, format!("`{op}` reads true or false, not {}", v.kind())) }; if (op == "&&" && !t) || (op == "||" && t) { Ev::Val(Val::Bool(t)) } else { @@ -3040,7 +3143,7 @@ impl<'a> Elab<'a> { } } } - Ev::Bits(_, n) => return self.runtime_value(&n, a.at), + Ev::Bits(_, n) => return self.runtime_value(&n, left), o => o, }) } @@ -3049,6 +3152,32 @@ impl<'a> Elab<'a> { fn ev_binary(&mut self, op: &str, a: &Expr, b: &Expr, at: Loc) -> R { let x = self.evs(a)?; let y = self.evs(b)?; + self.binary_of(op, x, y, at) + } + + /// A run of operators of one precedence, left to right, in a loop: each + /// operand counts as a statement does, so what a run costs in a loop is + /// paid for. + #[inline(never)] + fn ev_chain(&mut self, first: &Expr, rest: &[(&'static str, Expr)]) -> R { + let logic = matches!(rest[0].0, "&&" | "||"); + let mut x = if logic { self.ev(first)? } else { self.evs(first)? }; + let mut whole = first.at; + for (op, e) in rest { + self.tick(e.at)?; + x = if logic { + self.logic_then(op, x, whole, e)? + } else { + let y = self.evs(e)?; + self.binary_of(op, x, y, whole.to(e.at))? + }; + whole = whole.to(e.at); + } + Ok(x) + } + + /// The operator `op` on two values. + fn binary_of(&mut self, op: &str, x: Ev, y: Ev, at: Loc) -> R { Ok(match (x, y) { (Ev::Val(x), Ev::Val(y)) => Ev::Val(self.binop(op, x, y, at)?), (Ev::Bits(_, n), _) | (_, Ev::Bits(_, n)) => return self.runtime_value(&n, at), @@ -3072,6 +3201,7 @@ impl<'a> Elab<'a> { fn ev_set(&mut self, items: &[Expr]) -> R { let mut out = Vec::with_capacity(items.len()); for i in items { + self.tick(i.at)?; match self.evs(i)? { Ev::Val(v) => out.push(v), Ev::Unknown(w) => return Ok(Ev::Unknown(w)), @@ -3429,17 +3559,22 @@ impl<'a> Elab<'a> { let saved = std::mem::replace(&mut self.trunc, false); let v = self.known(e); let trunc = std::mem::replace(&mut self.trunc, saved); - let v = v?; + self.real_of(v?, trunc, e.at) + } + + /// `v` as a real number, `trunc` saying that its value truncated an + /// integer quotient. + fn real_of(&self, v: Val, trunc: bool, at: Loc) -> R { if trunc && !matches!(v, Val::Angle(_)) { return self.fail( diag::TRUNCATING_DIVISION, - e.at, + at, "integers divided with a remainder where a real number is wanted: OpenQASM 3 truncates the quotient, OpenQASM 2 does not; write a decimal point (`1.0/2`)", ); } match v { - Val::Complex(_, im) if im != 0.0 => self.fail(diag::WRONG_TYPE, e.at, "a gate's parameter is a real number, not a complex one"), - v => v.real().ok_or_else(|| self.d(diag::WRONG_TYPE, e.at, format!("a gate's parameter is a number, not {}", v.kind()))), + Val::Complex(_, im) if im != 0.0 => self.fail(diag::WRONG_TYPE, at, "a gate's parameter is a real number, not a complex one"), + v => v.real().ok_or_else(|| self.d(diag::WRONG_TYPE, at, format!("a gate's parameter is a number, not {}", v.kind()))), } } @@ -3483,6 +3618,7 @@ impl<'a> Elab<'a> { E::Name(n) => matches!(self.lookup(n), Some(Binding::Var { ty: Ty::Stretch, .. })), E::Unary(_, x) | E::Cast(_, x) => self.mentions_stretch(x), E::Binary(_, a, b) => self.mentions_stretch(a) || self.mentions_stretch(b), + E::Chain(first, rest) => self.mentions_stretch(first) || rest.iter().any(|(_, x)| self.mentions_stretch(x)), E::Call(_, args) => args.iter().any(|a| self.mentions_stretch(a)), _ => false, } @@ -3994,6 +4130,47 @@ impl<'a> Elab<'a> { } } +/// `a`, and `b` after it, in one flat list. +fn both(a: Ct, b: Ct) -> Ct { + match a { + Ct::All(mut list) => { + list.push(b); + Ct::All(list) + } + a => Ct::All(vec![a, b]), + } +} + +/// `a`, or `b` after it, in one flat list. +fn either(a: Ct, b: Ct) -> Ct { + match a { + Ct::Any(mut list) => { + list.push(b); + Ct::Any(list) + } + a => Ct::Any(vec![a, b]), + } +} + +/// A list of conditions with those already decided taken out: `all` of them +/// must hold, or any one. +fn joined(list: impl Iterator, all: bool) -> Ct { + let mut kept = Vec::new(); + for t in list { + match t { + Ct::Const(b) if b == all => {} + Ct::Const(b) => return Ct::Const(b), + t => kept.push(t), + } + } + match (kept.len(), all) { + (0, _) => Ct::Const(all), + (1, _) => kept.pop().expect("one left"), + (_, true) => Ct::All(kept), + (_, false) => Ct::Any(kept), + } +} + fn negate(t: Ct) -> Ct { match t { Ct::Const(b) => Ct::Const(!b), diff --git a/wai/wai-quantum/src/qasm/parse.rs b/wai/wai-quantum/src/qasm/parse.rs index c46fcce0e..1314cf57f 100644 --- a/wai/wai-quantum/src/qasm/parse.rs +++ b/wai/wai-quantum/src/qasm/parse.rs @@ -34,10 +34,16 @@ type R = Result; /// the program is refused ([`diag::NESTING`], `Q006`) rather than read by a /// recursion that could run out of stack: on a thread of 1 MiB (a wasm /// module's) the reader and everything after it stay well inside it. -/// Programs people write nest a few levels; a long `else if` chain is one -/// level a branch, and reads as a `switch` at any length. +/// Programs people write nest a few levels. A run of operators of one +/// precedence (`a + b + c`) is one level however long it is, and so is an +/// `else if` chain: they are read and run in loops, so no recursion follows +/// their length, and only what nests counts. pub const MAX_NESTING: usize = 64; +/// The most dimensions an array has. Comma-separated, they cost the text no +/// level, but each is a level of the array's value. +pub const MAX_ARRAY_DIMS: usize = 7; + /// Words that start a declaration of a classical type. const TYPES: [&str; 10] = ["bit", "int", "uint", "float", "angle", "bool", "complex", "duration", "stretch", "array"]; @@ -50,7 +56,7 @@ impl<'a> Parser<'a> { fn too_deep(&self) -> R { self.err( diag::NESTING, - format!("this nests deeper than {MAX_NESTING} levels (statements, parentheses, operators, brackets and types together); write it in steps, with a `const` or a gate for a part, or a long `else if` chain as a `switch`"), + format!("this nests deeper than {MAX_NESTING} levels (statements, parentheses, operators, brackets and types together); write it in steps, with a `const` or a gate for a part"), ) } @@ -452,11 +458,20 @@ impl<'a> Parser<'a> { #[inline(never)] fn if_stmt(&mut self) -> R { - self.pos += 1; - let cond = self.condition()?; - let then = self.body()?; - let other = if self.eat_word("else") { Some(self.body()?) } else { None }; - Ok(S::If { cond, then, other }) + let mut arms = Vec::new(); + loop { + self.pos += 1; + let cond = self.condition()?; + let then = self.body()?; + arms.push((cond, then)); + if !self.eat_word("else") { + return Ok(S::If { arms, other: None }); + } + // An `else if` is the next arm, read in this loop. + if !matches!(self.peek(), Some(Tok::Ident(w)) if w == "if") { + return Ok(S::If { arms, other: Some(self.body()?) }); + } + } } #[inline(never)] @@ -642,6 +657,9 @@ impl<'a> Parser<'a> { continue; } dims.push(self.expr()?); + if dims.len() > MAX_ARRAY_DIMS { + return self.err(diag::NESTING, format!("an array has at most {MAX_ARRAY_DIMS} dimensions: each is a level of its value, read by what copies and writes it")); + } } self.expect("]")?; Type::Array(Box::new(t), dims) @@ -826,19 +844,43 @@ impl<'a> Parser<'a> { fn binary(&mut self, min: usize) -> R<(Expr, usize)> { const LEVELS: [&[&str]; 10] = [&["||"], &["&&"], &["|"], &["^"], &["&"], &["==", "!="], &["<", "<=", ">", ">="], &["<<", ">>"], &["+", "-"], &["*", "/", "%"]]; let (mut a, mut ha) = self.unary()?; + // The run being read: operators of one level, whose operands are read + // in this loop, so a run of any length holds no frame and no level. A + // later operator can only be of a lower level (the right operand took + // every higher one), and takes what was read as its first operand. + let (mut run, mut hr, mut at_level) = (Vec::new(), 0, usize::MAX); loop { let found = match self.peek() { Some(Tok::Sym(s)) if !(*s == "^" && self.version != Version::Three) => LEVELS.iter().position(|l| l.contains(s)).map(|level| (*s, level)), _ => None, }; - let Some((op, level)) = found.filter(|&(_, level)| level >= min) else { return Ok((a, ha)) }; + let Some((op, level)) = found.filter(|&(_, level)| level >= min) else { break }; self.pos += 1; let (b, hb) = self.nested(|p| p.binary(level + 1))?; - ha = 1 + ha.max(hb); - self.fits(ha)?; - let at = a.at.to(b.at); - a = Expr { kind: E::Binary(op, Box::new(a), Box::new(b)), at }; + if level != at_level { + (a, ha) = self.close_run(a, ha, std::mem::take(&mut run), hr)?; + (hr, at_level) = (0, level); + } + hr = hr.max(hb); + run.push((op, b)); } + self.close_run(a, ha, run, hr) + } + + /// `first` and the operators and operands of a run after it, as one + /// expression: a level of the tree however long the run is. + fn close_run(&self, first: Expr, h: usize, mut run: Vec<(&'static str, Expr)>, h_run: usize) -> R<(Expr, usize)> { + let Some(last) = run.last() else { return Ok((first, h)) }; + let at = first.at.to(last.1.at); + let h = 1 + h.max(h_run); + self.fits(h)?; + let kind = if run.len() == 1 { + let (op, b) = run.pop().expect("a run of one"); + E::Binary(op, Box::new(first), Box::new(b)) + } else { + E::Chain(Box::new(first), run) + }; + Ok((Expr { kind, at }, h)) } /// Prefix operators, then a power: `-x`, `!b`, `~k`, `+x`, `a ** b` diff --git a/wai/wai-quantum/src/qasm/pulse.rs b/wai/wai-quantum/src/qasm/pulse.rs index e020dcb65..06aab7d83 100644 --- a/wai/wai-quantum/src/qasm/pulse.rs +++ b/wai/wai-quantum/src/qasm/pulse.rs @@ -1469,6 +1469,16 @@ impl<'a> Lower<'a> { let (x, y) = (self.eval(a, locals)?, self.eval(b, locals)?); self.arith(op, x, y, at) } + E::Chain(first, rest) => { + let mut x = self.eval(first, locals)?; + let mut whole = first.at; + for (op, e) in rest { + let y = self.eval(e, locals)?; + whole = whole.to(e.at); + x = self.arith(op, x, y, whole)?; + } + Ok(x) + } E::Call(name, args) => self.call_value(&name.text, args, locals, at), E::Set(items) => { let mut s = Vec::with_capacity(items.len()); diff --git a/wai/wai-quantum/src/qasm/unroll.rs b/wai/wai-quantum/src/qasm/unroll.rs index 2044cd746..6128291e0 100644 --- a/wai/wai-quantum/src/qasm/unroll.rs +++ b/wai/wai-quantum/src/qasm/unroll.rs @@ -644,17 +644,20 @@ impl<'a> Unroller<'a> { // A condition on integers alone is decided here. let keep = self.value(i + 2, close - 1)? != 0; let ((tlo, thi), mut next) = self.body(close)?; + let else_if = self.ident(next) == Some("else") && self.ident(next + 1) == Some("if"); + if !keep && else_if { + // The rest of the chain is read here, in this loop, + // so a chain of any length holds no recursion. + i = next + 1; + continue; + } let mut other = None; - if self.ident(next) == Some("else") { - if self.ident(next + 1) == Some("if") { - let end = self.if_extent(next + 1)?; - other = Some((next + 1, end)); - next = end; - } else { - let (span, after) = self.body(next + 1)?; - other = Some(span); - next = after; - } + if else_if { + next = self.if_extent(next + 1)?; + } else if self.ident(next) == Some("else") { + let (span, after) = self.body(next + 1)?; + other = Some(span); + next = after; } if keep { self.seq(tlo, thi)?; diff --git a/wai/wai-quantum/src/quantum_frame.rs b/wai/wai-quantum/src/quantum_frame.rs index 708dfdc80..01ae158e5 100644 --- a/wai/wai-quantum/src/quantum_frame.rs +++ b/wai/wai-quantum/src/quantum_frame.rs @@ -1444,13 +1444,13 @@ mod tests { /// each checked once on one copy, not on a copy each. #[test] fn blocks_that_write_nothing_end_and_unrun_ones_are_checked_once() { - let t = std::time::Instant::now(); - for body in ["TICK\n", "SHIFT_COORDS(1)\n", ""] { - let text = format!("H 0\nREPEAT 18446744073709551615 {{\n{body}}}\n"); - let e = parse(&text).unwrap_err().to_string(); - assert!(e.contains("unrolls to more than"), "{e}"); - } - assert!(t.elapsed() < std::time::Duration::from_secs(120), "{:?}", t.elapsed()); + crate::quantum_qec_text::ends_within(120, || { + for body in ["TICK\n", "SHIFT_COORDS(1)\n", ""] { + let text = format!("H 0\nREPEAT 18446744073709551615 {{\n{body}}}\n"); + let e = parse(&text).unwrap_err().to_string(); + assert!(e.contains("unrolls to more than"), "{e}"); + } + }); let n = crate::quantum_qec_text::MAX_BLOCK_DEPTH; let unrun = format!("{}H 0\n{}{}M 0\n", "REPEAT 0 {\n".repeat(n), "}\n".repeat(n), "H 0\n".repeat(1000)); let c = parse(&unrun).unwrap(); diff --git a/wai/wai-quantum/src/quantum_qec_text.rs b/wai/wai-quantum/src/quantum_qec_text.rs index 1238b5679..5b53e02ea 100644 --- a/wai/wai-quantum/src/quantum_qec_text.rs +++ b/wai/wai-quantum/src/quantum_qec_text.rs @@ -33,7 +33,9 @@ //! instruction this module reads but that module does not implement is //! refused there, with its line, not skipped. //! -//! Blocks nest at most [`MAX_BLOCK_DEPTH`] deep; a deeper one is refused on +//! Qubits are named below [`MAX_QUBITS`] and detectors below +//! [`MAX_DETECTORS`], each refused on its line: what is made for an index is +//! as long as the index. Blocks nest at most [`MAX_BLOCK_DEPTH`] deep; a deeper one is refused on //! its line, so reading, unrolling and writing a text never runs out of //! stack. @@ -237,10 +239,26 @@ fn circuit_target(line: usize, tok: &str) -> Result { }; let first = body.chars().next().ok_or_else(|| err(line, "an empty target"))?; if matches!(first, 'X' | 'Y' | 'Z' | 'x' | 'y' | 'z') { - let q: u32 = number(line, &body[1..], "a qubit")?; + let q = qubit(line, number(line, &body[1..], "a qubit")?)?; return Ok(Target::Pauli { pauli: first.to_ascii_uppercase(), q, inverted }); } - Ok(Target::Qubit { q: number(line, body, "a qubit target")?, inverted }) + Ok(Target::Qubit { q: qubit(line, number(line, body, "a qubit target")?)?, inverted }) +} + +/// `q`, a qubit's index a reader of the text may take ([`MAX_QUBITS`]). +fn qubit(line: usize, q: u32) -> Result { + if q >= MAX_QUBITS { + return Err(err(line, format!("qubit {q} is past the {MAX_QUBITS} a circuit here holds"))); + } + Ok(q) +} + +/// `d`, a detector's index a reader of the text may take ([`MAX_DETECTORS`]). +fn detector(line: usize, d: u64) -> Result { + if d >= MAX_DETECTORS { + return Err(err(line, format!("detector {d} is past the {MAX_DETECTORS} a model here holds"))); + } + Ok(d) } /// The lines of a text, numbered from 1, comments and blank lines dropped. @@ -253,6 +271,16 @@ fn lines(text: &str) -> Vec<(usize, &str)> { /// of the blocks, which recurses into them, stays well inside its stack. pub const MAX_BLOCK_DEPTH: usize = 32; +/// The most qubits a circuit's text may name, and so the widest circuit it +/// makes. An index costs a few bytes of text and what is made for it is as +/// long as the index, so it is held where it is read, with its line. +pub const MAX_QUBITS: u32 = 1 << 24; + +/// The most detectors a model's text may name, offsets counted: what is made +/// for them (their coordinates, every decoder's tables) is as long as the +/// highest index, whichever instruction names it. +pub const MAX_DETECTORS: u64 = 1 << 24; + fn too_deep(line: usize) -> TextError { err(line, format!("blocks nested deeper than {MAX_BLOCK_DEPTH}: write the inner ones out, or fold them into one with the product of their counts")) } @@ -402,7 +430,8 @@ fn dem_block(ls: &[(usize, &str)], pos: &mut usize, open: Option, depth: if t == "^" { Ok(DemTarget::Separator) } else if let Some(k) = t.strip_prefix('D') { - number(line, k, "a detector").map(DemTarget::Detector) + let d: u32 = number(line, k, "a detector")?; + detector(line, u64::from(d)).map(|_| DemTarget::Detector(d)) } else if let Some(k) = t.strip_prefix('L') { number(line, k, "an observable").map(DemTarget::Observable) } else { @@ -527,10 +556,7 @@ impl DemWalk { let cur = parts.last_mut().expect("a part"); match *t { DemTarget::Detector(k) => { - let d = self.offset + k as u64; - if d >= u32::MAX as u64 { - return Err(err(line, "a detector index beyond 2³²")); - } + let d = detector(line, self.offset + k as u64)?; self.detectors = self.detectors.max(d + 1); let d = d as u32; match cur.0.binary_search(&d) { @@ -576,7 +602,7 @@ impl DemWalk { for t in &i.targets { match *t { DemTarget::Detector(k) => { - let d = self.offset + k as u64; + let d = detector(line, self.offset + k as u64)?; self.detectors = self.detectors.max(d + 1); // A declaration without coordinates only names the detector. if !i.args.is_empty() { @@ -605,7 +631,8 @@ impl DemWalk { [DemTarget::Number(n)] => *n, _ => return Err(err(line, "shift_detectors takes one count")), }; - self.offset += n; + // Held here, so that an offset never overflows the sum above. + self.offset = detector(line, self.offset.saturating_add(n))?; for (j, a) in i.args.iter().enumerate() { if self.coord_shift.len() <= j { self.coord_shift.resize(j + 1, 0.0); @@ -642,6 +669,20 @@ impl DemWalk { } } +/// `work` run within `secs` seconds, or the test fails then: what a bound +/// keeps from running forever fails here, by name, if the bound goes, rather +/// than holding the job until its own limit ends it. +#[cfg(test)] +pub(crate) fn ends_within(secs: u64, work: impl FnOnce() -> T + Send + 'static) -> T { + let (tx, rx) = std::sync::mpsc::channel(); + std::thread::Builder::new().stack_size(8 << 20).spawn(move || tx.send(std::panic::catch_unwind(std::panic::AssertUnwindSafe(work)))).unwrap(); + match rx.recv_timeout(std::time::Duration::from_secs(secs)) { + Ok(Ok(v)) => v, + Ok(Err(p)) => std::panic::resume_unwind(p), + Err(_) => panic!("not ended within {secs} s"), + } +} + #[cfg(test)] mod tests { use super::*; @@ -757,11 +798,42 @@ logical_observable L1 /// bound, so one repeated 2^64 − 1 times ends at once, refused. #[test] fn an_empty_block_repeated_without_end_is_refused_not_run() { - let t = std::time::Instant::now(); - let e = parse_dem("repeat 18446744073709551615 {\n}\nerror(0.1) D0\n").unwrap().to_model().unwrap_err(); - assert!(e.message.contains("unroll"), "{}", e.message); - let e = parse_dem("repeat 18446744073709551615 {\nrepeat 18446744073709551615 {\n}\n}\n").unwrap().to_model().unwrap_err(); - assert!(e.message.contains("unroll"), "{}", e.message); - assert!(t.elapsed() < std::time::Duration::from_secs(60), "{:?}", t.elapsed()); + ends_within(60, || { + let e = parse_dem("repeat 18446744073709551615 {\n}\nerror(0.1) D0\n").unwrap().to_model().unwrap_err(); + assert!(e.message.contains("unroll"), "{}", e.message); + let e = parse_dem("repeat 18446744073709551615 {\nrepeat 18446744073709551615 {\n}\n}\n").unwrap().to_model().unwrap_err(); + assert!(e.message.contains("unroll"), "{}", e.message); + }); + } + + /// An index in a few bytes of text would make what is as long as the + /// index: every way of naming one is refused where it is read, with its + /// line, before anything is made. The highest that reads still makes its + /// model, and the model counts it exactly. + #[test] + fn an_index_past_what_a_text_holds_is_refused_on_its_line() { + let past = MAX_DETECTORS; + let d = past - 1; + for model in [ + format!("shift_detectors {past}\ndetector(1) D0\n"), + format!("shift_detectors {past}\ndetector D0\n"), + format!("shift_detectors 100000000000000\ndetector(1) D0\n"), + format!("shift_detectors 18446744073709551615\nshift_detectors 18446744073709551615\ndetector D0\n"), + format!("shift_detectors {d}\nerror(0.1) D1\n"), + format!("shift_detectors {d}\nshift_detectors 1\n"), + ] { + let e = parse_dem(&model).unwrap().to_model().unwrap_err(); + assert!(e.message.contains("past the"), "{model}: {}", e.message); + assert!(e.line > 0, "{model}: {}", e.message); + } + for model in ["detector(1) D4294967294\n", "detector D4294967294\n", "error(0.1) D4294967294\n"] { + assert!(parse_dem(model).unwrap_err().message.contains("past the"), "{model}"); + } + let m = parse_dem(&format!("shift_detectors {d}\ndetector D0\n")).unwrap().to_model().unwrap(); + assert_eq!(m.detectors as u64, MAX_DETECTORS); + for circuit in ["M 4000000000\n".to_string(), format!("H {MAX_QUBITS}\n"), format!("MPP X{MAX_QUBITS}\n")] { + assert!(parse_circuit(&circuit).unwrap_err().message.contains("past the"), "{circuit}"); + } + assert!(parse_circuit(&format!("H {}\n", MAX_QUBITS - 1)).is_ok()); } } -- 2.54.0 (Apple Git-157) From de48547537674975c5a6558cc6aa676cc914eadf Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Sat, 7 Nov 2026 02:51:37 -0500 Subject: [PATCH 04/18] wai-quantum-service: a reading cut off for its answer is refused however its child ends Co-Authored-By: Claude Opus 5.5 --- wai/wai-quantum-service/src/offload.rs | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/wai/wai-quantum-service/src/offload.rs b/wai/wai-quantum-service/src/offload.rs index 40428ba53..ac60e7817 100644 --- a/wai/wai-quantum-service/src/offload.rs +++ b/wai/wai-quantum-service/src/offload.rs @@ -36,7 +36,7 @@ use crate::jobs::Config; use serde::{Deserialize, Serialize}; use serde_json::{Value, json}; use std::panic::{AssertUnwindSafe, catch_unwind}; -use std::process::Stdio; +use std::process::{ExitStatus, Stdio}; use std::collections::HashMap; use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; use std::sync::{Arc, Mutex}; @@ -276,7 +276,7 @@ async fn in_child(config: &Config, r: &Read) -> Result { if too_long.load(Ordering::SeqCst) { crate::jobs::kill(pid); let _ = child.wait().await; - break Err((422, tag(refusal::TOO_LARGE_HERE, format!("reading this request gave more than {} MiB, more than the service keeps of one; {LOCAL}", config.read_answer_mib)))); + break Ok(ExitStatus::default()); } if crate::jobs::rss_bytes(pid).is_some_and(|rss| rss > limit) { crate::jobs::kill(pid); @@ -292,6 +292,10 @@ async fn in_child(config: &Config, r: &Read) -> Result { let _ = writer.await; let out = reader.await.unwrap_or_default(); let err = errors.await.unwrap_or_default(); + // Asked here, as a child cut off may end by itself before the loop sees it. + if too_long.load(Ordering::SeqCst) { + return Err((422, tag(refusal::TOO_LARGE_HERE, format!("reading this request gave more than {} MiB, more than the service keeps of one; {LOCAL}", config.read_answer_mib)))); + } let status = ended?; let parsed = serde_json::from_slice::(&out).ok(); match parsed { -- 2.54.0 (Apple Git-157) From d0befb07a264aab999d5955acc25aa4eb9fa16fe Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Sat, 7 Nov 2026 07:58:13 -0500 Subject: [PATCH 05/18] wai-quantum-service: a client holds no whole pool, a check's shots are timed, a reading's answer is held to what it takes once read, and readers start from the service's own binary Co-Authored-By: Claude Opus 5.5 --- wai/changes/2026-11-06-quantum-parse-depth.md | 34 +- wai/wai-quantum-service/Cargo.toml | 5 +- wai/wai-quantum-service/README.md | 24 +- wai/wai-quantum-service/docs/scheduler.md | 2 +- wai/wai-quantum-service/docs/self-host.md | 25 +- wai/wai-quantum-service/src/analysis.rs | 26 +- wai/wai-quantum-service/src/http.rs | 26 +- wai/wai-quantum-service/src/jobs.rs | 150 ++++++-- wai/wai-quantum-service/src/main.rs | 8 + wai/wai-quantum-service/src/offload.rs | 354 ++++++++++++++---- wai/wai-quantum-service/tests/depth.rs | 154 +++++++- wai/wai-quantum/src/engine/qec.rs | 70 ++++ wai/wai-quantum/src/quantum_frame.rs | 21 +- 13 files changed, 741 insertions(+), 158 deletions(-) diff --git a/wai/changes/2026-11-06-quantum-parse-depth.md b/wai/changes/2026-11-06-quantum-parse-depth.md index 00cd48004..e1182f970 100644 --- a/wai/changes/2026-11-06-quantum-parse-depth.md +++ b/wai/changes/2026-11-06-quantum-parse-depth.md @@ -29,7 +29,8 @@ - **An index in a few bytes of text** (`quantum_qec_text::MAX_QUBITS`, `MAX_DETECTORS`, each 2²⁴). What is made for a detector or a qubit is as long as its index: `shift_detectors 100000000000000` and a `detector(1) D0` asked for 2.4 PB in 79 bytes, and an allocation that fails ends the process. Each index is refused where it is read, on - its line, offsets counted, however it is named. + its line, offsets counted, however it is named. A circuit's width with its correlated errors' carrier is held to + the same cap, refused at the first such error. - **The reader's memory** (`Q130`, new). A program whose reading would hold more than the reader takes is refused before it is made: one classical value (an array or a bit register, each element counted at its width, so `array[bit[1048576], 1000]` is a gigabyte) of more than 64 MiB, or more than 512 MiB in all of its values and @@ -63,15 +64,24 @@ made of it: an equivalence bound read a refusal as no work and let the check run. `/v1/verify` and MCP `verify_quantum_receipt` on a set are counted and read as a check is, as its sweep is written out again, and a seal that does not hold is not. A decode is held to 1 024 fired detectors, as matching grows faster than their - square. + square, and a memory experiment to 21 rounds. An error-correction decode or threshold measures its shots as they + run (`engine::qec::within_time`) and is refused as soon as the rest would take more than `WAI_QSVC_QEC_SECS` (25), + where one at the published bounds ran for many minutes and held its turn to the reading's cap. - **The job service: in a child.** Admission, a dry run, every check's bounds and answer, the error-correction, pulse and variational calls and the estimates, a set's receipt, and the verdicts the MCP tools state beside an answer are read by a child, the service's own binary run as `wai-quantum-service read`, sandboxed as a worker - is. Four requests are read at once and two checks beside them; a client has `WAI_QSVC_CLIENT_READS` (2) being read - or waiting; at most 32 wait, each for ten seconds; past any of these a request is refused, `503` and `J032`. A - child past `WAI_QSVC_READ_GB` of memory (4) or `WAI_QSVC_READ_SECS` (120) is stopped and its request refused, `422` - and `J031`; one that panics, overflows or aborts is answered `500`; the service runs on. A client that hangs up - ends its child. + is. Four requests are read at once and four checks beside them. A client has its share of each, + `WAI_QSVC_CLIENT_READS` (2) and `WAI_QSVC_CLIENT_CHECKS` (1), smaller than the pool, so one client holds no pool; + its requests past its share wait behind it in order (an agent's parallel calls are answered one after another), + four at most, 32 in all, each for ten seconds; past any of these a request is refused, `503` and `J032`. People + behind one address share one client's turns. A child is polled every 50 ms from a thread of its own (on macOS its + physical footprint, compressed pages included; on Linux resident and swapped pages) and killed past + `WAI_QSVC_READ_GB` (4) or `WAI_QSVC_READ_SECS` (45, so that its refusal reaches a client within a proxy's 60 s), its + request refused, `422` and `J031`. The poll is not a kernel limit; on Linux a child also holds its address space + to twice the cap and a gigabyte, and the kernel ends readers and workers before the service. What a reading + gives back is held to `WAI_QSVC_READ_ANSWER_MIB` (128) as it takes once read, refused in the child before it is + written, and read off the request threads; a dry run gets back its plans and points alone. One that panics, + overflows or aborts is answered `500`; the service runs on. A client that hangs up ends its child. - **The job service: MCP.** A tool that says nothing about a program takes no turn. A verdict reads the program only for a call that was counted; a poll states what its job holds, and reads nothing, however often it is asked; a call refused before it was counted (a bad key, a tool the service does not have, past the window) or told to try @@ -79,10 +89,12 @@ answer about a program has one, and follows its schema. A refused task is the same, where it read the program on a runtime worker with no budget. A run's MCP arguments that are not an object are refused, where they dropped the connection. -- **The job service: connections and restarts.** A request's header section is held to `WAI_QSVC_HEADER_SECS` and - its body to `WAI_QSVC_BODY_SECS` (30 each; past the second, `408`). A restart reads no program: a job keeps the - plan it was admitted with and a set its items, where a queue of slow programs held the port unbound for the sum of - their reads. +- **The job service: connections, updates and restarts.** A request's header section is held to + `WAI_QSVC_HEADER_SECS` (150, which also closes an idle connection, so it sits above a proxy's two-minute idle + limit) and its body to `WAI_QSVC_BODY_SECS` (30; past it, `408`). Readers and workers start from the service's own + binary as it was started (Linux: its running image; elsewhere a copy kept beside the signing key), so the binary + can be replaced before the restart. A restart reads no program: a job keeps the plan it was admitted with and a + set its items, where a queue of slow programs held the port unbound for the sum of their reads. - **Tests**: every nesting construct at thousands of levels on a 2 MiB thread, refused; runs, `else if` chains and case lists of a hundred thousand on 1 MiB, read to what the tree gives; a seeded property test over mixed nesting; the service binary started in a test and sent each of these through each door, answering `/v1/health` after each; diff --git a/wai/wai-quantum-service/Cargo.toml b/wai/wai-quantum-service/Cargo.toml index 83d7275df..9793d5534 100644 --- a/wai/wai-quantum-service/Cargo.toml +++ b/wai/wai-quantum-service/Cargo.toml @@ -24,8 +24,9 @@ blake3 = "=1.8.7" futures-util = { version = "=0.3.34", default-features = false } wai-quantum = { path = "../wai-quantum", default-features = false, features = ["quantum_engine", "quantum_receipt", "quantum_qec", "quantum_noise", "quantum_fabric", "quantum_sv_metal", "quantum_cutrank_metal"] } -# proc_pid_rusage and waitid, for each job's energy (src/energy.rs). -[target.'cfg(target_os = "macos")'.dependencies] +# proc_pid_rusage and waitid, for each job's energy (src/energy.rs) and a +# child's memory (src/jobs.rs); setrlimit, for a reader's (src/offload.rs). +[target.'cfg(unix)'.dependencies] libc = "0.2" [profile.release] diff --git a/wai/wai-quantum-service/README.md b/wai/wai-quantum-service/README.md index 82d9d260c..aa6b49c44 100644 --- a/wai/wai-quantum-service/README.md +++ b/wai/wai-quantum-service/README.md @@ -195,7 +195,15 @@ A submission is admitted in the order of what each step costs: `Retry-After` is the pace at which a refusal that counts would not spend your window (the window over its submissions, at least 5 s), or, past the window, when the next submission opens. A `503` carries it too. -Reading happens outside the service's own process, as each job's run does: each request is read by a child, this binary run as `wai-quantum-service read`, sandboxed as a worker is. Four are read at a time, and two checks beside them. A client has `WAI_QSVC_CLIENT_READS` (2) requests being read or waiting, across both; at most 32 requests wait, each for ten seconds; past any of these the request is refused, `503` and `J032`. A child that holds more than `WAI_QSVC_READ_GB` (4) of memory, reads for longer than `WAI_QSVC_READ_SECS` (120), or answers with more than `WAI_QSVC_READ_ANSWER_MIB` (256), is stopped and the request refused, `422` and `J031`. One that panics, overflows its stack or aborts is answered `500`, and the service runs on. A client that hangs up ends its child. A request that takes longer than `WAI_QSVC_HEADER_SECS` to send its header section is closed, and one longer than `WAI_QSVC_BODY_SECS` to send its body is refused, `408` (30 each). +Reading happens outside the service's own process, as each job's run does: each request is read by a child, this binary run as `wai-quantum-service read`, sandboxed as a worker is. + +- **Turns.** Four requests are read at a time (jobs, dry runs and the MCP tools' verdicts), and four checks beside them. A client has its share of each: `WAI_QSVC_CLIENT_READS` (2) of the first and `WAI_QSVC_CLIENT_CHECKS` (1) of the checks, so no client holds a whole pool. A client's requests past its share wait behind it, in order, so an agent's parallel calls are answered one after another; up to four of them may wait, at most 32 requests wait in all, each for ten seconds, and past any of these a request is refused, `503` and `J032`, having read and spent nothing. A client is its address, so people behind one address (an office, a carrier's NAT, a class) share one client's turns and its window: raise the shares, or give such a group an API key of its own on a private deployment. +- **Memory.** A child's memory is polled every 50 ms from a thread of its own, and the child is killed (`kill -9`) once it holds more than `WAI_QSVC_READ_GB` (4); the request is refused, `422` and `J031`. On macOS the poll reads the child's physical footprint, which counts its compressed and swapped pages; on Linux its resident and swapped pages. A poll is not a kernel limit: a child can pass the cap by what it touches in one interval before it is killed. On Linux the child also holds its own address space to twice the cap and a gigabyte, so an allocation past that fails at once (`422`, `J031`), and the kernel ends a reader or a worker before the service when the host runs short. macOS gives an unprivileged process neither, and has no limit that counts compressed memory. +- **Time.** A child that reads for longer than `WAI_QSVC_READ_SECS` (45) is killed and the request refused, `422` and `J031`. With the ten-second wait for a turn, an answer comes within the 60 s a proxy in front commonly waits for one; behind a proxy that waits less, lower it. An error-correction call's shots are measured as they run and refused as soon as the rest would take more than `WAI_QSVC_QEC_SECS` (25). +- **Its answer.** What a reading gives back comes into the service's memory, which every job shares: one that would take more than `WAI_QSVC_READ_ANSWER_MIB` (128) once read, counted value by value, is refused in the child before it is written, and the service reads it off its request threads. A dry run gets back the plans and points alone. +- **Ending.** A child that panics, overflows its stack or aborts is answered `500`, and the service runs on. A client that hangs up ends its child. +- **Connections.** A request whose header section takes longer than `WAI_QSVC_HEADER_SECS` (150) to arrive is closed; the same limit closes a connection left idle between requests, so it is set above a proxy's own idle limit (two minutes, commonly), and the proxy, not the service, closes its idle connections. A body that takes longer than `WAI_QSVC_BODY_SECS` (30) is refused, `408`. +- **Updating.** Readers and workers start from the service's own binary as it was started: on Linux its running image, elsewhere a copy it keeps beside its signing key, named by its digest. So the binary can be replaced before the restart (by a copy over it, or, on Linux, where a running binary cannot be written, by a rename), and the running service goes on with its own until it is restarted. ## Sending a job again @@ -221,13 +229,13 @@ A restart reads no program: a job keeps the plan it was admitted with, and a set Finished jobs are kept for `WAI_QSVC_RETAIN_SECS` (a day here), then dropped. A record names its client by a keyed hash of the address, never the address. [docs/self-host.md](docs/self-host.md#the-job-store) describes the store. -The six checks answer at once, each read and answered in a reading child as a job's admission is. Each counts against the client's submissions, as a job does, once it has its turn and before it reads anything (a check that waits ten seconds and finds no turn is refused, `503`, having read and spent nothing). At most two run at a time. Its bounds and its answer then read its programs within one budget, each read once: a check that runs the budget out is refused, `422` and `J031`, whatever it had made of it, so a repair of many fixes to a slow program ends at the budget. Some checks are refused (`422`) with the command that runs them on your own machine: +The six checks answer at once, each read and answered in a reading child as a job's admission is. Each counts against the client's submissions, as a job does, once it has its turn and before it reads anything (a check that waits ten seconds and finds no turn is refused, `503`, having read and spent nothing). At most four run at a time, one of them a client's, its others waiting behind it. Its bounds and its answer then read its programs within one budget, each read once: a check that runs the budget out is refused, `422` and `J031`, whatever it had made of it, so a repair of many fixes to a slow program ends at the budget. Some checks are refused (`422`) with the command that runs them on your own machine: - an equivalence check that would apply more than 4·10⁹ gate amplitudes; - an improvement whose candidates' checks would apply more than that; - a thorough search for shortcuts; - a compile of more than 20,000 gates, or with more than 256 rotations to approximate, or a budget that leaves a rotation less than 10⁻¹² (`wai-quantum compile`). A job that asks to be compiled first is held to the same bounds; -- an error-correction call past its bounds here (`wai-quantum qec`): a decode of more than 200,000 shots, of more than 1,024 fired detectors, of a memory experiment past distance 7, or of more than 2 MiB of circuit or model text; a threshold of more than 16 points or 200,000 shots in all, or at circuit level past distance 7; the distance certificate of `bb:288`; +- an error-correction call past its bounds here (`wai-quantum qec`): a decode of more than 200,000 shots, of more than 1,024 fired detectors, of a memory experiment past distance 7 or 21 rounds, or of more than 2 MiB of circuit or model text; a threshold of more than 16 points or 200,000 shots in all, or at circuit level past distance 7 or 21 rounds; a decode or a threshold whose shots, measured as they run, would take more than `WAI_QSVC_QEC_SECS` (25 s); the distance certificate of `bb:288`; - a gradient, an optimisation, a VQE or a QAOA run past its bounds here (`wai-quantum gradient`, `optimize`, `vqe`, `qaoa`): more than 16 qubits, more than 500 steps, or about 4·10⁹ gate amplitudes in all; phase estimation on more than 12 counting qubits, or amplitude estimation on more than 8 (`wai-quantum build`); - an estimate of more than 200,000 gates, or with more than 256 distinct rotation angles to synthesise, or a budget that leaves a rotation less than 10⁻¹² (`wai-quantum estimate`). Verifying an estimate re-runs it, within the same bounds. - a pulse check whose run takes more than 2·10¹⁰ of the simulator's multiply-adds, or a pulse call on more than 1 MiB of program (`wai-quantum pulse check`). A pulse job itself is held to the limits every job is. @@ -336,10 +344,12 @@ These variables set the limits: | `WAI_QSVC_CLIENT_BURST` | 30 | submissions one client may make per window | | `WAI_QSVC_CLIENT_WINDOW_SECS` | 600 | the window | | `WAI_QSVC_READ_GB` | 4 | the memory one request's reading may hold, in its child | -| `WAI_QSVC_READ_SECS` | 120 | the time one request's reading may take | -| `WAI_QSVC_READ_ANSWER_MIB` | 256 | the most one request's reading may give back to the service | -| `WAI_QSVC_CLIENT_READS` | 2 | how many of one client's requests may be read or wait to be, at once | -| `WAI_QSVC_HEADER_SECS` | 30 | how long a client may take to send a request's header section | +| `WAI_QSVC_READ_SECS` | 45 | the time one request's reading may take | +| `WAI_QSVC_READ_ANSWER_MIB` | 128 | the most one request's reading may give back to the service, once read | +| `WAI_QSVC_QEC_SECS` | 25 | the time an error-correction call's shots may take, measured as they run | +| `WAI_QSVC_CLIENT_READS` | 2 | how many of one client's jobs, dry runs and verdicts are read at once (more wait) | +| `WAI_QSVC_CLIENT_CHECKS` | 1 | how many of one client's checks run at once (more wait) | +| `WAI_QSVC_HEADER_SECS` | 150 | how long a client may take to send a request's header section, and how long an idle connection stays open | | `WAI_QSVC_BODY_SECS` | 30 | how long a client may take to send a request's body | | `WAI_QSVC_MAX_QUEUE` | 256 | jobs waiting at once | | `WAI_QSVC_RETAIN_SECS` | 86400 | how long finished jobs are kept, across restarts | diff --git a/wai/wai-quantum-service/docs/scheduler.md b/wai/wai-quantum-service/docs/scheduler.md index c388fdee2..679cd5e89 100644 --- a/wai/wai-quantum-service/docs/scheduler.md +++ b/wai/wai-quantum-service/docs/scheduler.md @@ -51,4 +51,4 @@ Before committing an allocation to it, a job can be planned without running: `wa ## Limits and keys -The public service counts a client by its address. A batch system submits from a few login or compute nodes on behalf of many people, so on a shared deployment give the scheduler its own API key (`WAI_QUANTUM_API_KEY`, see [self-host.md](self-host.md#api-keys-for-a-private-deployment)). Its limits are then counted by the key's name, and `WAI_QSVC_CLIENT_ACTIVE`, `WAI_QSVC_CLIENT_GB`, `WAI_QSVC_CLIENT_BURST` and `WAI_QSVC_CLIENT_READS` (how many of its submissions may be read at once) size them. Every answer carries the client's submission window in `RateLimit-Policy` and `RateLimit`, and a refusal past it carries `Retry-After`. The adapter exits 75 on such a refusal, which a scheduler can treat as "requeue". +The public service counts a client by its address. A batch system submits from a few login or compute nodes on behalf of many people, so on a shared deployment give the scheduler its own API key (`WAI_QUANTUM_API_KEY`, see [self-host.md](self-host.md#api-keys-for-a-private-deployment)). Its limits are then counted by the key's name, and `WAI_QSVC_CLIENT_ACTIVE`, `WAI_QSVC_CLIENT_GB`, `WAI_QSVC_CLIENT_BURST`, `WAI_QSVC_CLIENT_READS` and `WAI_QSVC_CLIENT_CHECKS` (how many of its submissions and checks are read at once; more wait behind them) size them. Every answer carries the client's submission window in `RateLimit-Policy` and `RateLimit`, and a refusal past it carries `Retry-After`. The adapter exits 75 on such a refusal, which a scheduler can treat as "requeue". diff --git a/wai/wai-quantum-service/docs/self-host.md b/wai/wai-quantum-service/docs/self-host.md index d58ab2fe8..309624e82 100644 --- a/wai/wai-quantum-service/docs/self-host.md +++ b/wai/wai-quantum-service/docs/self-host.md @@ -50,11 +50,13 @@ Everything is set by environment variables. | `WAI_QSVC_CLIENT_ACTIVE` | 3 | queued or running jobs one client may hold | | `WAI_QSVC_CLIENT_GB` | the job size | the memory one client's jobs may hold together | | `WAI_QSVC_CLIENT_BURST` | 30 | submissions one client may make in a window | -| `WAI_QSVC_READ_GB` | 4 | the memory one request's reading may take | -| `WAI_QSVC_READ_SECS` | 120 | the time one request's reading may take | -| `WAI_QSVC_READ_ANSWER_MIB` | 256 | the most one request's reading may give back to the service | -| `WAI_QSVC_CLIENT_READS` | 2 | how many of one client's requests may be read or wait to be, at once | -| `WAI_QSVC_HEADER_SECS` | 30 | how long a client may take to send a request's header section | +| `WAI_QSVC_READ_GB` | 4 | the memory one request's reading may take (polled; see the README's reading section) | +| `WAI_QSVC_READ_SECS` | 45 | the time one request's reading may take | +| `WAI_QSVC_READ_ANSWER_MIB` | 128 | the most one request's reading may give back to the service, once read | +| `WAI_QSVC_QEC_SECS` | 25 | the time an error-correction call's shots may take, measured as they run | +| `WAI_QSVC_CLIENT_READS` | 2 | how many of one client's jobs, dry runs and verdicts are read at once (more wait) | +| `WAI_QSVC_CLIENT_CHECKS` | 1 | how many of one client's checks run at once (more wait) | +| `WAI_QSVC_HEADER_SECS` | 150 | how long a client may take to send a request's header section, and how long an idle connection stays open | | `WAI_QSVC_BODY_SECS` | 30 | how long a client may take to send a request's body | | `WAI_QSVC_CLIENT_WINDOW_SECS` | 600 | the window | | `WAI_QSVC_MAX_QUEUE` | 256 | jobs waiting at once | @@ -138,6 +140,16 @@ Run the service on a loopback address and let the proxy take the public traffic. - **Event streams.** `GET /v1/jobs/{id}/events` is a server-sent event stream. The proxy must pass it through unbuffered (the service sends `X-Accel-Buffering: no` and a comment every 15 s), and allow it to stay open as long as `WAI_QSVC_JOB_SECS`. Polls with `?wait=` are held up to 25 s. - **Bodies.** A request body is at most 1 MiB, and `POST /v1/run` waits up to 25 s before answering 202. - **Rate-limit fields.** Pass the `RateLimit-Policy`, `RateLimit` and `Retry-After` headers through. +- **Timeouts.** A reading is cut off at `WAI_QSVC_READ_SECS` (45) after a wait for a turn of at most 10 s, so its refusal reaches the client within a proxy's 60 s wait for an answer's header; with a shorter wait in the proxy, lower `WAI_QSVC_READ_SECS` so that the two together fit. Keep the proxy's idle connections shorter than `WAI_QSVC_HEADER_SECS` (150), which also bounds an idle connection on the service's side: the proxy then closes its idle connections itself, and never sends a request on one as the service closes it. +- **One address, many people.** The service counts a client by the address the proxy gives it. People behind one address share one client's turns (`WAI_QSVC_CLIENT_READS`, `WAI_QSVC_CLIENT_CHECKS`) and window; raise those, or give such a group an API key. + +## Updating it + +Readers and workers start from the service's own binary as it was started: on Linux its running image (`/proc//exe`), elsewhere a copy kept beside the signing key (`bin/wai-quantum-service-`). So: + +1. Put the new binary in place: a copy over the old one, or a copy beside it renamed over it (on Linux a running binary cannot be written, so rename). The running service goes on reading and running jobs with its own binary. +2. Restart the service. It keeps a copy of the new binary on its first start (on Linux, nothing to keep). +3. Check `GET /v1/health`, then `smoke.sh`. Copies of earlier binaries beside the key are not removed; delete them once no service started from them runs. A minimal site, in one common proxy's configuration format: @@ -147,6 +159,9 @@ quantum.example.org { reverse_proxy 127.0.0.1:8241 { header_up X-Forwarded-For {remote_host} flush_interval -1 + transport http { + keepalive 60s + } } } } diff --git a/wai/wai-quantum-service/src/analysis.rs b/wai/wai-quantum-service/src/analysis.rs index 84811d119..2361519e2 100644 --- a/wai/wai-quantum-service/src/analysis.rs +++ b/wai/wai-quantum-service/src/analysis.rs @@ -197,6 +197,10 @@ pub const QEC_MAX_POINTS: usize = 16; pub const QEC_MAX_CIRCUIT_DISTANCE: u64 = 7; /// The most bytes of circuit or model text a decode reads here. pub const QEC_MAX_TEXT: usize = 2 << 20; +/// The most rounds of a memory experiment a decode or a threshold runs here: +/// three times the largest distance, past which a fit gains nothing a +/// distance does not. +pub const QEC_MAX_ROUNDS: u64 = 3 * QEC_MAX_CIRCUIT_DISTANCE; /// The most detectors a decode here is given as fired. Matching grows faster /// than the square of them, in time and in memory: a thousand take about a /// second and a hundred MiB. @@ -232,6 +236,9 @@ pub fn qec_bounds(op: &str, body: &Value) -> Result<(), (u16, String)> { if body.pointer("/memory/distance").and_then(Value::as_u64).is_some_and(|d| d > QEC_MAX_CIRCUIT_DISTANCE) { return far(format!("a memory experiment past distance {QEC_MAX_CIRCUIT_DISTANCE}; {}", local("decode"))); } + if body.pointer("/memory/rounds").and_then(Value::as_u64).is_some_and(|r| r > QEC_MAX_ROUNDS) { + return far(format!("a memory experiment of more than {QEC_MAX_ROUNDS} rounds; {}", local("decode"))); + } } _ => { let n = |k: &str, d: usize| body.get(k).and_then(Value::as_array).map_or(d, Vec::len); @@ -240,6 +247,9 @@ pub fn qec_bounds(op: &str, body: &Value) -> Result<(), (u16, String)> { if points > QEC_MAX_POINTS || shots.saturating_mul(points as u64) > QEC_MAX_SHOTS { return far(format!("{points} points of {shots} shots, more than a threshold here measures ({QEC_MAX_POINTS} points, {QEC_MAX_SHOTS} shots in all); {}", local("threshold"))); } + if body.get("rounds").and_then(Value::as_u64).is_some_and(|r| r > QEC_MAX_ROUNDS) { + return far(format!("more than {QEC_MAX_ROUNDS} rounds; {}", local("threshold"))); + } let wide = body.get("distances").and_then(Value::as_array).into_iter().flatten().filter_map(Value::as_u64).any(|d| d > QEC_MAX_CIRCUIT_DISTANCE); if wide && body.get("family").and_then(Value::as_str).unwrap_or("surface") == "surface" { return far(format!("a circuit-level threshold past distance {QEC_MAX_CIRCUIT_DISTANCE}; {}", local("threshold"))); @@ -368,7 +378,11 @@ pub fn call(kind: &str, op: &str, body: &Value, key: Option<&str>) -> Result { let op = ["code", "decode", "threshold"].into_iter().find(|o| *o == op).ok_or((400, format!("no error-correction call {op}")))?; qec_bounds(op, body)?; - engine(wai_quantum::engine::qec::call(op, body)) + // Its shots are measured as they run, and refused as soon as the + // rest would pass the time: a shot's cost is the noise's as much + // as the counts'. + let secs = crate::jobs::Config::from_env().qec_secs; + engine(wai_quantum::engine::qec::within_time(std::time::Duration::from_secs(secs), || wai_quantum::engine::qec::call(op, body))) } "pulse" => { let op = ["schedule", "check"].into_iter().find(|o| *o == op).ok_or((400, format!("no pulse call {op}")))?; @@ -554,6 +568,16 @@ mod tests { assert!(e.0 == 422 && e.1.ends_with("[J031]"), "{e:?}"); } + /// Rounds are held, in a decode's memory experiment and in a threshold. + #[test] + fn a_memory_experiment_is_held_to_its_rounds() { + for (op, body) in [("decode", serde_json::json!({ "memory": { "distance": 7, "rounds": 50 }, "shots": 10 })), ("threshold", serde_json::json!({ "distances": [3], "rounds": 50 }))] { + let e = qec_bounds(op, &body).unwrap_err(); + assert!(e.0 == 422 && e.1.contains("rounds") && e.1.ends_with("[J031]"), "{op}: {e:?}"); + } + assert!(qec_bounds("decode", &serde_json::json!({ "memory": { "distance": 7, "rounds": QEC_MAX_ROUNDS }, "shots": 10 })).is_ok()); + } + /// Matching grows faster than the square of the detectors fired: a decode /// is held to how many it is given, and told so with the local command. #[test] diff --git a/wai/wai-quantum-service/src/http.rs b/wai/wai-quantum-service/src/http.rs index 0fe8987d2..80cf74860 100644 --- a/wai/wai-quantum-service/src/http.rs +++ b/wai/wai-quantum-service/src/http.rs @@ -369,16 +369,18 @@ pub(crate) async fn admit(app: &App, who: &str, request: Value, key: Option); -fn admitted(v: Value) -> Result { +fn admitted(mut v: Value) -> Result { let bad = |e: serde_json::Error| (500, format!("the reading's answer did not read: {e}")); - let request = v["request"].clone(); - if let Some(r) = v["refused"].as_array() { - return Ok((request, Err((r[0].as_u64().unwrap_or(400) as u16, r[1].as_str().unwrap_or("refused").to_string())))); + // Each part taken, not copied: a set's items can be large. + let mut take = |k: &str| v.as_object_mut().and_then(|o| o.remove(k)).unwrap_or(Value::Null); + let request = take("request"); + if let Value::Array(r) = take("refused") { + return Ok((request, Err((r.first().and_then(Value::as_u64).unwrap_or(400) as u16, r.get(1).and_then(Value::as_str).unwrap_or("refused").to_string())))); } - let plan: catalog::Plan = serde_json::from_value(v["plan"].clone()).map_err(bad)?; - let work = match v["items"].as_array() { - None => Work::One, - Some(_) => Work::Set { items: crate::jobs::items_of(&v["items"]), plans: serde_json::from_value(v["plans"].clone()).map_err(bad)? }, + let plan: catalog::Plan = serde_json::from_value(take("plan")).map_err(bad)?; + let work = match take("items") { + Value::Null => Work::One, + items => Work::Set { items: crate::jobs::items_from(items), plans: serde_json::from_value(take("plans")).map_err(bad)? }, }; Ok((request, Ok((plan, work)))) } @@ -414,7 +416,7 @@ pub(crate) async fn admit_to(service: &Service, who: &str, request: Value, key: service.count(who).map_err(refused)?; cheap_bounds(&request)?; let received = crate::jobs::sourced(&request).then(|| crate::jobs::received_digest(&request)); - let read = crate::offload::read(&service.config, turn, crate::offload::Read::Admit { request, job_bytes: service.config.job_mib << 20 }).await?; + let read = crate::offload::read(&service.config, turn, crate::offload::Read::Admit { request, job_bytes: service.config.job_mib << 20, keep: true }).await?; let (request, planned) = admitted(read)?; // Sent in another form under a key: it may read as the key's job. if received.is_some() @@ -463,7 +465,7 @@ pub(crate) async fn dry_run(app: &App, who: &str, request: &Value) -> Result Config { let job_gb = env_num("WAI_QSVC_JOB_GB", 72u64); Config { @@ -146,10 +155,12 @@ impl Config { per_client_burst: env_num("WAI_QSVC_CLIENT_BURST", 30usize), per_client_window: Duration::from_secs(env_num("WAI_QSVC_CLIENT_WINDOW_SECS", 600u64)), read_mib: (env_num("WAI_QSVC_READ_GB", 4f64).max(0.0) * 1024.0) as u64, - read_timeout: Duration::from_secs(env_num("WAI_QSVC_READ_SECS", 120u64)), - read_answer_mib: env_num("WAI_QSVC_READ_ANSWER_MIB", 256u64), + read_timeout: Duration::from_secs(env_num("WAI_QSVC_READ_SECS", 45u64)), + read_answer_mib: env_num("WAI_QSVC_READ_ANSWER_MIB", 128u64), + qec_secs: env_num("WAI_QSVC_QEC_SECS", 25u64).max(1), per_client_reads: env_num("WAI_QSVC_CLIENT_READS", 2usize).max(1), - header_wait: Duration::from_secs(env_num("WAI_QSVC_HEADER_SECS", 30u64).max(1)), + per_client_checks: env_num("WAI_QSVC_CLIENT_CHECKS", 1usize).max(1), + header_wait: Duration::from_secs(env_num("WAI_QSVC_HEADER_SECS", 150u64).max(1)), body_wait: Duration::from_secs(env_num("WAI_QSVC_BODY_SECS", 30u64).max(1)), read_in_child: !cfg!(test), } @@ -276,6 +287,59 @@ pub async fn noting_count(work: F) -> (F::Output, bool) .await } +/// What the service starts its readers and workers from ([`self_exe`]). +static SELF_EXE: std::sync::OnceLock = std::sync::OnceLock::new(); + +/// Keeps what the service starts its readers and workers from, as the +/// running service is, whatever becomes of the file it was started from: on +/// Linux its own image (`/proc//exe`, which names the running binary +/// even once the file is replaced or removed); elsewhere a copy of the binary +/// in `dir`, named by its digest, made once. So the binary can be replaced +/// before the restart, by a copy over it or a rename, and the running service +/// goes on with its own until it is restarted. +pub fn keep_self(dir: &std::path::Path) -> Result { + let kept = if cfg!(target_os = "linux") { + std::path::PathBuf::from(format!("/proc/{}/exe", std::process::id())) + } else { + let exe = std::env::current_exe().map_err(|e| format!("the service could not find itself: {e}"))?; + let bytes = std::fs::read(&exe).map_err(|e| format!("{}: {e}", exe.display()))?; + let path = dir.join(format!("wai-quantum-service-{}", &blake3::hash(&bytes).to_hex()[..16])); + if std::fs::read(&path).map(|b| b != bytes).unwrap_or(true) { + std::fs::create_dir_all(dir).map_err(|e| format!("{}: {e}", dir.display()))?; + let part = dir.join(format!(".{}-{}", path.file_name().and_then(|n| n.to_str()).unwrap_or("copy"), std::process::id())); + std::fs::write(&part, &bytes).map_err(|e| format!("{}: {e}", part.display()))?; + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + std::fs::set_permissions(&part, std::fs::Permissions::from_mode(0o755)).map_err(|e| format!("{}: {e}", part.display()))?; + } + std::fs::rename(&part, &path).map_err(|e| format!("{}: {e}", path.display()))?; + } + path + }; + Ok(SELF_EXE.get_or_init(|| kept).clone()) +} + +/// What the service starts its readers and workers from: what +/// [`keep_self`] kept, or, where it was not asked to keep it (a test), the +/// binary it was started from. +pub fn self_exe() -> Result { + match SELF_EXE.get() { + Some(p) => Ok(p.clone()), + None => std::env::current_exe().map_err(|e| format!("the service could not find itself: {e}")), + } +} + +/// Makes this process the one a host short of memory ends first (Linux: +/// `oom_score_adj` 1000), so that a reader or a worker that grows between two +/// polls of its memory is ended before the service is. +pub fn expendable() { + #[cfg(target_os = "linux")] + { + let _ = std::fs::write("/proc/self/oom_score_adj", "1000"); + } +} + /// Whether `request` carries its program in another format, so that what it /// reads as is known only by reading it. pub fn sourced(request: &Value) -> bool { @@ -294,7 +358,18 @@ pub fn items_json(items: &[sets::Item]) -> Value { /// [`items_json`] read back. pub fn items_of(v: &Value) -> Vec { - v.as_array().into_iter().flatten().map(|i| sets::Item { request: i["request"].clone(), point: Some(i["point"].clone()).filter(|p| !p.is_null()) }).collect() + items_from(v.clone()) +} + +/// [`items_of`], taking the values rather than copying them. +pub fn items_from(v: Value) -> Vec { + let Value::Array(list) = v else { return Vec::new() }; + list.into_iter() + .map(|i| { + let Value::Object(mut o) = i else { return sets::Item { request: Value::Null, point: None } }; + sets::Item { request: o.remove("request").unwrap_or(Value::Null), point: o.remove("point").filter(|p| !p.is_null()) } + }) + .collect() } fn now_ms() -> u64 { @@ -500,7 +575,7 @@ impl Service { fn build(config: Config, store: Box, restored: Restored) -> Service { let memory = Arc::new(Semaphore::new(usize::try_from(config.budget_mib).unwrap_or(usize::MAX).min(Semaphore::MAX_PERMITS))); let workers = Arc::new(Semaphore::new(config.workers)); - let gate = crate::offload::Gate::new(config.per_client_reads); + let gate = crate::offload::Gate::new(config.per_client_reads, config.per_client_checks); Service { gate, config, @@ -1277,7 +1352,7 @@ impl Service { /// long it took. async fn supervise(&self, id: &str, input: Value, mib: u64, gpu: bool, others: u64, start: u64) -> Ran { let failed = |e: String| Ran { outcome: Err(e), peak: 0, energy: Value::Null, elapsed_ms: 0 }; - let exe = match std::env::current_exe() { + let exe = match self_exe() { Ok(e) => e, Err(e) => return failed(format!("the service could not find itself: {e}")), }; @@ -1465,26 +1540,27 @@ pub(crate) fn kill(pid: u32) { } } -/// A process's resident memory, bytes, the worker's child included where the -/// platform folds it in. +/// A process's memory, bytes, as its cap counts it: on Linux resident and +/// swapped, on macOS its physical footprint (compressed pages included). pub(crate) fn rss_bytes(pid: u32) -> Option { if pid == 0 { return None; } + // Linux: resident and swapped, so a page swapped out still counts. if cfg!(target_os = "linux") { let s = std::fs::read_to_string(format!("/proc/{pid}/status")).ok()?; - let kb: u64 = s.lines().find(|l| l.starts_with("VmRSS:"))?.split_whitespace().nth(1)?.parse().ok()?; - return Some(kb * 1024); + let kb = |field: &str| -> Option { s.lines().find(|l| l.starts_with(field))?.split_whitespace().nth(1)?.parse().ok() }; + return Some((kb("VmRSS:")? + kb("VmSwap:").unwrap_or(0)) * 1024); } - // Asked of the kernel: this runs on the runtime, for every child, many - // times a second, where a `ps` per reading held a worker for milliseconds. + // macOS: the physical footprint, what the kernel charges the process, + // compressed and swapped pages included, which resident pages are not. + // Asked of the kernel: this runs for every child, many times a second. #[cfg(target_os = "macos")] { - let mut info: libc::proc_taskinfo = unsafe { std::mem::zeroed() }; - let size = std::mem::size_of::() as i32; - // SAFETY: `info` is a `proc_taskinfo`, and `size` is its length. - let got = unsafe { libc::proc_pidinfo(pid as i32, libc::PROC_PIDTASKINFO, 0, (&raw mut info).cast(), size) }; - (got == size).then_some(info.pti_resident_size) + let mut info: libc::rusage_info_v2 = unsafe { std::mem::zeroed() }; + // SAFETY: `info` is a `rusage_info_v2`, the flavour asked for. + let got = unsafe { libc::proc_pid_rusage(pid as i32, libc::RUSAGE_INFO_V2, (&raw mut info).cast()) }; + (got == 0).then_some(info.ri_phys_footprint) } #[cfg(not(target_os = "macos"))] { @@ -1555,9 +1631,11 @@ mod tests { per_client_window: Duration::from_secs(60), read_mib: 4096, read_timeout: Duration::from_secs(300), - read_answer_mib: 256, + read_answer_mib: 128, + qec_secs: 25, read_in_child: false, per_client_reads: 2, + per_client_checks: 1, header_wait: Duration::from_secs(30), body_wait: Duration::from_secs(30), } @@ -1638,6 +1716,24 @@ mod tests { /// A key outlives the service: reopened, the store's job still answers /// for it, but for the job that was running, now interrupted. + /// What the service waits for sits inside what the proxy in front of it + /// waits for (the public deployment's: 60 s for an answer's header, two + /// minutes for an idle connection kept open): a reading and its wait for a + /// turn end before the proxy gives up on them, so its client hears why; + /// an idle connection is the proxy's to close, not the service's, so no + /// request sent on one is lost as it closes; an error-correction call's + /// shots end well inside the reading's time; and a client's share of each + /// pool is smaller than the pool. + #[test] + fn the_defaults_fit_inside_the_proxy_s() { + let c = Config::from_env(); + assert!(c.read_timeout + crate::offload::WAIT < Duration::from_secs(60), "{:?}", c.read_timeout); + assert!(c.header_wait > Duration::from_secs(120), "{:?}", c.header_wait); + assert!((c.qec_secs as f64) < 0.6 * c.read_timeout.as_secs_f64(), "{}", c.qec_secs); + assert!(c.per_client_reads < crate::offload::AT_ONCE && c.per_client_checks < crate::offload::CHECKS_AT_ONCE); + assert!(c.read_answer_mib <= 128); + } + /// A restart reads no program: what is queued keeps its plan, and a set /// its items, so a queue of slow programs delays no restart. These could /// not be read at all, and come back queued, as they were. @@ -2005,7 +2101,7 @@ mod tests { let s = Service::new(config()); let p = Plan { method: "mps".into(), why: "".into(), memory_bytes: 1, work: 1_000_000_000, qubits: 30, target: None, metered: true }; for _ in 0..3 { - let job = Job { id: job_id(), client: "x".into(), request: json!({}), plan: p.clone(), work: Work::One, status: Status::Done, submitted_ms: 0, started_ms: None, finished_ms: Some(1), result: Some(json!({ "elapsed_ms": 500, "energy": { "joules_micro": 5_000_000, "energy_uncertainty": { "window_us": 500_000 } } })), error: None, peak_rss: 0, digests: Digests::default(), key: None, interruptions: Vec::new(), child: None }; + let job = Job { id: job_id(), client: "x".into(), request: json!({}), plan: p.clone(), work: Work::One, status: Status::Done, submitted_ms: 0, started_ms: None, finished_ms: Some(1), result: Some(json!({ "elapsed_ms": 500, "energy": { "joules_micro": 5_000_000, "energy_uncertainty": { "window_us": 500_000 } } })), error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None }; s.time(&job); } let (t, _) = s.estimate(&p); diff --git a/wai/wai-quantum-service/src/main.rs b/wai/wai-quantum-service/src/main.rs index 6632f50ec..46516b41f 100644 --- a/wai/wai-quantum-service/src/main.rs +++ b/wai/wai-quantum-service/src/main.rs @@ -47,6 +47,7 @@ fn main() { Some("worker") => worker(), Some("read") => { watch_parent(); + jobs::expendable(); offload::child() } Some("run") => match args.get(2) { @@ -97,6 +98,7 @@ fn watch_parent() { /// `{"items": [{"result"} | {"error"}, ...]}`. fn worker() { watch_parent(); + jobs::expendable(); let mut input = String::new(); if let Err(e) = std::io::stdin().read_to_string(&mut input) { println!("{}", serde_json::json!({ "error": format!("the job did not arrive: {e}") })); @@ -173,6 +175,12 @@ fn run_file(path: &str) { fn serve() { let config = jobs::Config::from_env(); let key_path = receipt::default_path(); + // Readers and workers start from what this keeps, so the binary can be + // replaced before the restart. + let bin = key_path.parent().map(|d| d.join("bin")).unwrap_or_else(|| std::path::PathBuf::from("bin")); + if let Err(e) = jobs::keep_self(&bin) { + eprintln!("keeping the service's binary for its readers and workers: {e}; they start from the file it was started from"); + } let fail = |what: &str, e: String| -> ! { eprintln!("{what}: {e}"); std::process::exit(1) diff --git a/wai/wai-quantum-service/src/offload.rs b/wai/wai-quantum-service/src/offload.rs index ac60e7817..ad4492372 100644 --- a/wai/wai-quantum-service/src/offload.rs +++ b/wai/wai-quantum-service/src/offload.rs @@ -12,22 +12,34 @@ //! runs in a child, the service's own binary run as `wai-quantum-service //! read`, the same boundary each job runs behind: //! -//! - **Its memory is capped** ([`crate::jobs::Config::read_mib`]): polled as a -//! job's is, and killed past it; the request is refused, `422` and `J031`, -//! with the command that runs it on the client's own machine. **Its time is -//! capped** the same way ([`crate::jobs::Config::read_timeout`]): nothing a -//! request reads holds a turn past it. **Its answer is capped** -//! ([`crate::jobs::Config::read_answer_mib`]): it comes into this process. +//! - **Its memory is capped** ([`crate::jobs::Config::read_mib`]): polled +//! every 50 ms from a thread of its own ([`Watch`]; on macOS its physical +//! footprint, compressed and swapped pages included, on Linux its resident +//! and swapped pages), and killed (`kill -9`) past it; the request is +//! refused, `422` and `J031`, with the command that runs it on the client's +//! own machine. A poll is not a kernel limit: a child can pass the cap by +//! what it touches in one interval. On Linux the child also holds its own +//! address space ([`child`]) and is the process the kernel ends first when +//! the host runs short ([`crate::jobs::expendable`]); macOS offers an +//! unprivileged process neither. **Its time is capped** the same way +//! ([`crate::jobs::Config::read_timeout`]): nothing a request reads holds a +//! turn past it. **Its answer is capped** by what it takes once read +//! ([`crate::jobs::Config::read_answer_mib`], [`weight`]), in the child, +//! as it comes into this process, which reads it off the request threads. //! - **It ends alone**: a child that panics, overflows its stack or aborts is //! answered `500`, and the service runs on. //! - **It is sandboxed** where jobs are (macOS: no network, no file writes, -//! no further processes), and is given only what it reads ([`Read`]). +//! no further processes), is given only what it reads ([`Read`]), and is +//! started from the service's own binary as it was started +//! ([`crate::jobs::self_exe`]), whatever has become of the file since. //! - **It waits its turn** ([`Gate`]): [`AT_ONCE`] are read at once, and -//! [`CHECKS_AT_ONCE`] checks beside them. A client has -//! [`crate::jobs::Config::per_client_reads`] requests in all being read or -//! waiting, at most [`MAX_WAITING`] wait, each for at most [`WAIT`], and the -//! rest are refused (`503`, `J032`), having read and spent nothing. A -//! client that hangs up ends its child and frees its turn. +//! [`CHECKS_AT_ONCE`] checks beside them. A client has its share of each +//! ([`crate::jobs::Config::per_client_reads`], +//! [`crate::jobs::Config::per_client_checks`]), smaller than the pool; its +//! requests past that wait behind it, [`CLIENT_WAITING`] at most; at most +//! [`MAX_WAITING`] wait in all, each for at most [`WAIT`], and the rest are +//! refused (`503`, `J032`), having read and spent nothing. A client that +//! hangs up ends its child and frees its turn. //! //! The service's own tests read on a thread of their own ([`STACK`]) instead, //! as a test binary is not the service's (`Config::read_in_child`). @@ -50,14 +62,16 @@ pub const STACK: usize = 64 << 20; /// Requests read at once; the rest wait their turn. pub const AT_ONCE: usize = 4; /// Checks answered at once, beside them. -pub const CHECKS_AT_ONCE: usize = 2; +pub const CHECKS_AT_ONCE: usize = 4; /// How long a request waits for a turn before it is refused. pub const WAIT: Duration = Duration::from_secs(10); /// How many requests may wait for a turn: each holds its body meanwhile. pub const MAX_WAITING: usize = 32; +/// How many of one client's requests may wait behind its share of a pool. +pub const CLIENT_WAITING: usize = 4; /// What a request is read among. -#[derive(Clone, Copy, Debug, PartialEq, Eq)] +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] pub enum Pool { /// Jobs and dry runs, and the verdicts of MCP answers. Reading, @@ -65,36 +79,48 @@ pub enum Pool { Checks, } -type Clients = Arc>>; +/// Each client's share of each pool, and how many of its requests hold or +/// wait for one. +type Clients = Arc>>; + +struct Share { + turns: Arc, + requests: usize, +} /// The turns requests are read in: the service's two pools, each client's -/// share of them, and who is waiting. +/// share of each, and who is waiting. A client's requests past its share wait +/// for one of its own turns to come back, in order, so its parallel calls are +/// served one after another rather than refused, and no client holds more of +/// a pool than its share, however many it sends. pub struct Gate { reading: Arc, checks: Arc, waiting: Arc, clients: Clients, - per_client: usize, + /// A client's share of the reading pool and of the checks. + shares: (usize, usize), } /// A turn, held until its request has been read; dropped, the next goes. pub struct Turn { - _permit: OwnedSemaphorePermit, + _pool: OwnedSemaphorePermit, + _share: OwnedSemaphorePermit, _client: ClientHold, } struct ClientHold { clients: Clients, - who: String, + key: (Pool, String), } impl Drop for ClientHold { fn drop(&mut self) { let mut clients = self.clients.lock().unwrap(); - if let Some(n) = clients.get_mut(&self.who) { - *n -= 1; - if *n == 0 { - clients.remove(&self.who); + if let Some(s) = clients.get_mut(&self.key) { + s.requests -= 1; + if s.requests == 0 { + clients.remove(&self.key); } } } @@ -109,38 +135,54 @@ impl Drop for Waiting { } impl Gate { - pub fn new(per_client: usize) -> Gate { - Gate { reading: Arc::new(Semaphore::new(AT_ONCE)), checks: Arc::new(Semaphore::new(CHECKS_AT_ONCE)), waiting: Arc::new(AtomicUsize::new(0)), clients: Arc::default(), per_client } + /// The pools, a client having `reads` turns of the reading pool and + /// `checks` of the checks (each at least one, and at most the pool). + pub fn new(reads: usize, checks: usize) -> Gate { + Gate { + reading: Arc::new(Semaphore::new(AT_ONCE)), + checks: Arc::new(Semaphore::new(CHECKS_AT_ONCE)), + waiting: Arc::new(AtomicUsize::new(0)), + clients: Arc::default(), + shares: (reads.clamp(1, AT_ONCE), checks.clamp(1, CHECKS_AT_ONCE)), + } } - /// A turn for `who` in `pool`, or the refusal (`503`, `J032`): at once for - /// a client with its share in hand or with too many waiting, after - /// [`WAIT`] for one that found no turn. + /// A turn for `who` in `pool`, or the refusal (`503`, `J032`): at once + /// past [`CLIENT_WAITING`] of the client's own requests waiting, or past + /// [`MAX_WAITING`] in all; after [`WAIT`] for one that found no turn. pub async fn turn(&self, pool: Pool, who: &str) -> Result { let busy = |why: String| (503, tag(refusal::BUSY, why)); - let client = { + let (share, pool_turns, full) = match pool { + Pool::Reading => (self.shares.0, &self.reading, format!("{AT_ONCE} requests are being read; try again shortly")), + Pool::Checks => (self.shares.1, &self.checks, format!("{CHECKS_AT_ONCE} checks are running; try again shortly")), + }; + let key = (pool, who.to_string()); + let (mine, client) = { let mut clients = self.clients.lock().unwrap(); - let held = clients.entry(who.to_string()).or_insert(0); - if *held >= self.per_client { - return Err(busy(format!("{} of your requests are being read; send the next when one is answered", self.per_client))); + let s = clients.entry(key.clone()).or_insert_with(|| Share { turns: Arc::new(Semaphore::new(share)), requests: 0 }); + if s.requests >= share + CLIENT_WAITING { + return Err(busy(format!("{} of your requests are being read or waiting to be; send the next when one is answered", s.requests))); } - *held += 1; - ClientHold { clients: Arc::clone(&self.clients), who: who.to_string() } + s.requests += 1; + (Arc::clone(&s.turns), ClientHold { clients: Arc::clone(&self.clients), key }) }; - let (turns, full) = match pool { - Pool::Reading => (&self.reading, format!("{AT_ONCE} requests are being read; try again shortly")), - Pool::Checks => (&self.checks, format!("{CHECKS_AT_ONCE} checks are running; try again shortly")), - }; - if let Ok(permit) = Arc::clone(turns).try_acquire_owned() { - return Ok(Turn { _permit: permit, _client: client }); + if let Ok(m) = Arc::clone(&mine).try_acquire_owned() + && let Ok(p) = Arc::clone(pool_turns).try_acquire_owned() + { + return Ok(Turn { _pool: p, _share: m, _client: client }); } if self.waiting.fetch_add(1, Ordering::SeqCst) >= MAX_WAITING { self.waiting.fetch_sub(1, Ordering::SeqCst); return Err(busy(full)); } let _waiting = Waiting(Arc::clone(&self.waiting)); - match tokio::time::timeout(WAIT, Arc::clone(turns).acquire_owned()).await { - Ok(Ok(permit)) => Ok(Turn { _permit: permit, _client: client }), + let both = async { + let m = Arc::clone(&mine).acquire_owned().await.ok()?; + let p = Arc::clone(pool_turns).acquire_owned().await.ok()?; + Some((p, m)) + }; + match tokio::time::timeout(WAIT, both).await { + Ok(Some((p, m))) => Ok(Turn { _pool: p, _share: m, _client: client }), _ => Err(busy(full)), } } @@ -153,8 +195,10 @@ pub enum Read { /// A job or a set: its source read where it has one, held to the checks /// that read and planned within `job_bytes` (`http::plan_job`). The answer /// is `{"request", "plan", "items", "plans"}`, or `{"request", "refused": - /// [status, why]}` where the planning refused it. - Admit { request: Value, job_bytes: u64 }, + /// [status, why]}` where the planning refused it. With `keep` false (a + /// dry run, which keeps nothing) the request and each item's request are + /// left out: a dry run reads the plans and the points alone. + Admit { request: Value, job_bytes: u64, keep: bool }, /// A design check (`analysis::Check`, by name): its bounds and its /// answer, within one budget. Check { check: String, body: Value }, @@ -191,16 +235,21 @@ pub async fn read(config: &Config, turn: Turn, r: Read) -> Result Result { match r { - Read::Admit { request, job_bytes } => wai_quantum::qasm::within_steps(crate::http::READ_STEPS, || { + Read::Admit { request, job_bytes, keep } => wai_quantum::qasm::within_steps(crate::http::READ_STEPS, || { let request = crate::http::within_budget(|| crate::http::with_source(request))?; // A refusal of the plan is the answer's, beside the request as read. - let (p, work) = match crate::http::plan_job(job_bytes, &request) { + let planned = crate::http::plan_job(job_bytes, &request); + let request = if keep { request } else { Value::Null }; + let (p, work) = match planned { Ok(planned) => planned, Err((code, why)) => return Ok(json!({ "request": request, "refused": [code, why] })), }; let (items, plans) = match work { crate::jobs::Work::One => (Value::Null, Value::Null), - crate::jobs::Work::Set { items, plans } => (crate::jobs::items_json(&items), serde_json::to_value(plans).expect("plans are JSON")), + crate::jobs::Work::Set { items, plans } => { + let items = if keep { crate::jobs::items_json(&items) } else { items.iter().map(|i| json!({ "point": i.point })).collect() }; + (items, serde_json::to_value(plans).expect("plans are JSON")) + } }; Ok(json!({ "request": request, "plan": p, "items": items, "plans": plans })) }), @@ -219,12 +268,52 @@ pub fn run(r: Read) -> Result { } } +/// What `v` takes once read, about: each value and each string or key, +/// counted as `serde_json` holds them. A JSON text can take many times its +/// own length once read (`[0,0,…]` two bytes a value), so what a reading may +/// give back to the service is held to this, not to its length. +pub fn weight(v: &Value) -> u64 { + const VALUE: u64 = 32; + let mut stack = vec![v]; + let mut total = 0u64; + while let Some(v) = stack.pop() { + total += VALUE; + match v { + Value::String(s) => total += s.len() as u64, + Value::Array(a) => stack.extend(a.iter()), + Value::Object(o) => { + for (k, x) in o { + total += k.len() as u64 + VALUE; + stack.push(x); + } + } + _ => {} + } + } + total +} + /// `wai-quantum-service read`: one [`Read`] on standard input, its answer -/// on standard output as `{"ok": …}` or `{"refused": [status, why]}`. +/// on standard output as `{"ok": …}` or `{"refused": [status, why]}`. An +/// answer that would take more than the service takes back of one +/// ([`Config::read_answer_mib`], as [`weight`] counts it) is refused here, +/// before it is written. On Linux the child also holds its own address space +/// to twice its memory cap and a gigabyte, so that one that grows faster than +/// the service polls it fails its allocation rather than taking the host. pub fn child() { + let config = Config::from_env(); + #[cfg(target_os = "linux")] + { + let bytes = (config.read_mib << 20).saturating_mul(2).saturating_add(1 << 30); + let lim = libc::rlimit { rlim_cur: bytes as libc::rlim_t, rlim_max: bytes as libc::rlim_t }; + // SAFETY: a plain call on this process's own limit, with a valid struct. + unsafe { libc::setrlimit(libc::RLIMIT_AS, &lim) }; + } + let cap = config.read_answer_mib << 20; let mut input = String::new(); let out = match std::io::Read::read_to_string(&mut std::io::stdin(), &mut input).map_err(|e| e.to_string()).and_then(|_| serde_json::from_str::(&input).map_err(|e| e.to_string())) { Ok(r) => match run(r) { + Ok(v) if weight(&v) > cap => json!({ "refused": [422, too_much(config.read_answer_mib)] }), Ok(v) => json!({ "ok": v }), Err((code, why)) => json!({ "refused": [code, why] }), }, @@ -233,15 +322,77 @@ pub fn child() { println!("{out}"); } +/// The refusal of a reading whose answer is more than the service takes back. +fn too_much(mib: u64) -> String { + tag(refusal::TOO_LARGE_HERE, format!("reading this request gave more than {mib} MiB once read, more than the service takes back of one; {LOCAL}")) +} + /// The local command a refusal for size names. const LOCAL: &str = "run it on your own machine (`wai-quantum run`, `wai-quantum critique` and the rest read without these limits)"; +/// Why a watcher ended a child. +const OVER_MEMORY: u8 = 1; +const OVER_TIME: u8 = 2; + +/// A child's memory and time watched from a thread of its own, every 50 ms, +/// and the child killed (`kill -9`) past either: a runtime busy with other +/// requests does not delay the check. Dropped (the request answered, or its +/// client gone), it stops within one poll. +struct Watch { + done: Arc, + over: Arc, +} + +impl Watch { + fn start(pid: u32, limit: u64, deadline: Instant) -> Watch { + let done = Arc::new(AtomicBool::new(false)); + let over = Arc::new(std::sync::atomic::AtomicU8::new(0)); + let (d, o) = (Arc::clone(&done), Arc::clone(&over)); + let watch = move || { + while !d.load(Ordering::SeqCst) { + std::thread::sleep(Duration::from_millis(50)); + if d.load(Ordering::SeqCst) { + return; + } + let why = if crate::jobs::rss_bytes(pid).is_some_and(|rss| rss > limit) { + OVER_MEMORY + } else if Instant::now() > deadline { + OVER_TIME + } else { + continue; + }; + // Said before the kill, so whoever sees the child end sees why. + o.store(why, Ordering::SeqCst); + crate::jobs::kill(pid); + return; + } + }; + if std::thread::Builder::new().name("wai-qsvc-watch".into()).spawn(watch).is_err() { + // No thread to watch it: it is not run unwatched. + over.store(OVER_TIME, Ordering::SeqCst); + crate::jobs::kill(pid); + } + Watch { done, over } + } + + fn over(&self) -> u8 { + self.over.load(Ordering::SeqCst) + } +} + +impl Drop for Watch { + fn drop(&mut self) { + self.done.store(true, Ordering::SeqCst); + } +} + async fn in_child(config: &Config, r: &Read) -> Result { - let exe = std::env::current_exe().map_err(|e| (500, format!("the service could not find itself: {e}")))?; + let exe = crate::jobs::self_exe().map_err(|e| (500, e))?; let mut cmd = crate::jobs::sandboxed(&exe, config.sandbox, "read"); cmd.stdin(Stdio::piped()).stdout(Stdio::piped()).stderr(Stdio::piped()).kill_on_drop(true); let mut child = cmd.spawn().map_err(|e| (503, tag(refusal::BUSY, format!("the service could not start a reader ({e}); try again shortly"))))?; let pid = child.id().unwrap_or(0); + let watch = Watch::start(pid, config.read_mib << 20, Instant::now() + config.read_timeout); let text = serde_json::to_vec(r).expect("a reading is JSON"); let mut stdin = child.stdin.take().expect("piped"); let writer = tokio::spawn(async move { @@ -249,8 +400,9 @@ async fn in_child(config: &Config, r: &Read) -> Result { }); let mut stdout = child.stdout.take().expect("piped"); let mut stderr = child.stderr.take().expect("piped"); - // Its answer comes into this process, so it is held here as its memory - // is there: a sweep written out can be far larger than the request. + // Its answer comes into this process: the child holds it to what the + // service takes back once read; its length is held here too, past which + // the child is ended. let answer_limit = config.read_answer_mib << 20; let too_long = Arc::new(AtomicBool::new(false)); let over = Arc::clone(&too_long); @@ -265,8 +417,6 @@ async fn in_child(config: &Config, r: &Read) -> Result { let _ = (&mut stderr).take(1 << 20).read_to_end(&mut out).await; out }); - let limit = config.read_mib << 20; - let deadline = Instant::now() + config.read_timeout; let ended = loop { match tokio::time::timeout(Duration::from_millis(50), child.wait()).await { Ok(Ok(status)) => break Ok(status), @@ -278,31 +428,33 @@ async fn in_child(config: &Config, r: &Read) -> Result { let _ = child.wait().await; break Ok(ExitStatus::default()); } - if crate::jobs::rss_bytes(pid).is_some_and(|rss| rss > limit) { - crate::jobs::kill(pid); - let _ = child.wait().await; - break Err((422, tag(refusal::TOO_LARGE_HERE, format!("reading this request took more than {} MiB, more than the service reads with; {LOCAL}", config.read_mib)))); - } - if Instant::now() > deadline { - crate::jobs::kill(pid); - let _ = child.wait().await; - break Err((422, tag(refusal::TOO_LARGE_HERE, format!("reading this request took more than {} s, longer than the service reads for; {LOCAL}", config.read_timeout.as_secs())))); - } }; let _ = writer.await; let out = reader.await.unwrap_or_default(); let err = errors.await.unwrap_or_default(); + let refused = |why: String| Err((422, tag(refusal::TOO_LARGE_HERE, why))); // Asked here, as a child cut off may end by itself before the loop sees it. if too_long.load(Ordering::SeqCst) { - return Err((422, tag(refusal::TOO_LARGE_HERE, format!("reading this request gave more than {} MiB, more than the service keeps of one; {LOCAL}", config.read_answer_mib)))); + return Err((422, too_much(config.read_answer_mib))); } + let detail = String::from_utf8_lossy(&err); + match watch.over() { + OVER_MEMORY => return refused(format!("reading this request took more than {} MiB, more than the service reads with; {LOCAL}", config.read_mib)), + OVER_TIME => return refused(format!("reading this request took more than {} s, longer than the service reads for; {LOCAL}", config.read_timeout.as_secs())), + _ => {} + } + drop(watch); let status = ended?; - let parsed = serde_json::from_slice::(&out).ok(); + // An allocation its address-space limit refused: its memory, as above. + if !status.success() && detail.contains("memory allocation of") { + return refused(format!("reading this request took more than {} MiB, more than the service reads with; {LOCAL}", config.read_mib)); + } + // Read off the runtime: an answer of many megabytes takes a while. + let parsed = tokio::task::spawn_blocking(move || serde_json::from_slice::(&out).ok()).await.ok().flatten(); match parsed { - Some(v) if status.success() && v.get("ok").is_some() => Ok(v["ok"].clone()), + Some(Value::Object(mut v)) if status.success() && v.contains_key("ok") => Ok(v.remove("ok").unwrap_or(Value::Null)), Some(v) if status.success() && v["refused"][0].is_u64() => Err((v["refused"][0].as_u64().unwrap_or(500) as u16, v["refused"][1].as_str().unwrap_or("refused").to_string())), _ => { - let detail = String::from_utf8_lossy(&err); let last = detail.lines().rev().find(|l| !l.trim().is_empty()).unwrap_or("").trim(); Err((500, format!("reading this request failed inside the service ({status}{}{last}); nothing was admitted, and the service runs on", if last.is_empty() { "" } else { ": " }))) } @@ -360,29 +512,65 @@ mod tests { assert_eq!(on_thread(|| deep(8000)).await.unwrap(), want); } - /// A client has its share of the turns, waiting included, so one client - /// cannot take every turn: past it, it is told at once, and its share comes - /// back as its requests are answered. Other clients go on. + /// A client has its share of each pool, smaller than the pool: one that + /// sends more waits behind its own turns, in order, while every other + /// client is served at once; past its share and [`CLIENT_WAITING`] it is + /// told at once. As its turns come back, those waiting go. #[tokio::test] - async fn a_client_is_held_to_its_share_of_the_turns() { - let gate = Gate::new(2); - let (a, b) = (gate.turn(Pool::Reading, "a").await.unwrap(), gate.turn(Pool::Checks, "a").await.unwrap()); + async fn a_client_holds_no_more_of_a_pool_than_its_share() { + let gate = Gate::new(2, 1); + let soon = Duration::from_millis(300); + for (pool, share, size) in [(Pool::Reading, 2, AT_ONCE), (Pool::Checks, 1, CHECKS_AT_ONCE)] { + assert!(share < size); + let mut held = Vec::new(); + for _ in 0..share { + held.push(gate.turn(pool, "a").await.ok().unwrap()); + } + // Past its share, it waits; every other client is served at once. + let waiting = tokio::time::timeout(soon, gate.turn(pool, "a")).await; + assert!(waiting.is_err(), "{pool:?}: a client past its share waits"); + let t = Instant::now(); + let other = gate.turn(pool, "b").await.ok().unwrap(); + assert!(t.elapsed() < soon, "{pool:?}"); + drop(other); + // Its next waits, and goes once one of its turns comes back. + let g = &gate; + let next = async move { g.turn(pool, "a").await.ok().map(|_| ()) }; + let release = async { + tokio::time::sleep(soon).await; + held.pop(); + }; + let (got, ()) = tokio::join!(next, release); + assert!(got.is_some(), "{pool:?}"); + drop(held); + } + // Past its share and the waiting it is allowed, told at once. + let gate = std::sync::Arc::new(Gate::new(1, 1)); + let first = gate.turn(Pool::Checks, "a").await.ok().unwrap(); + let waiters: Vec<_> = (0..CLIENT_WAITING) + .map(|_| { + let g = std::sync::Arc::clone(&gate); + tokio::spawn(async move { g.turn(Pool::Checks, "a").await.err().map(|e| e.0) }) + }) + .collect(); + while gate.waiting.load(Ordering::SeqCst) < CLIENT_WAITING { + tokio::time::sleep(Duration::from_millis(5)).await; + } let t = Instant::now(); - let (code, why) = gate.turn(Pool::Reading, "a").await.err().unwrap(); - assert_eq!(code, 503); - assert!(why.contains("[J032]") && t.elapsed() < Duration::from_secs(1), "{why}"); - let other = gate.turn(Pool::Reading, "b").await; - assert!(other.is_ok()); - drop((a, b)); - assert!(gate.turn(Pool::Reading, "a").await.is_ok()); - assert!(gate.clients.lock().unwrap().get("a").is_none(), "a client that holds nothing is not kept"); + let (code, why) = gate.turn(Pool::Checks, "a").await.err().unwrap(); + assert!(code == 503 && why.ends_with("[J032]") && t.elapsed() < soon, "{why}"); + drop(first); + for w in waiters { + assert_eq!(w.await.unwrap(), None, "each waiting request had its turn in time"); + } + assert!(gate.clients.lock().unwrap().is_empty(), "a client that holds nothing is not kept"); } /// Past [`MAX_WAITING`] a request is told at once, and those that wait /// are told within [`WAIT`]: nothing waits without limit. #[tokio::test] async fn a_wait_for_a_turn_is_bounded_in_time_and_in_number() { - let gate = Arc::new(Gate::new(1)); + let gate = Arc::new(Gate::new(1, 1)); let mut held = Vec::new(); for i in 0..AT_ONCE { held.push(gate.turn(Pool::Reading, &format!("holder {i}")).await.ok().unwrap()); @@ -413,7 +601,7 @@ mod tests { let r = Read::Call { kind: "qec".into(), op: "decode".into(), body: json!({ "circuit": "H 0\n" }), key: None }; let back: Read = serde_json::from_value(serde_json::to_value(&r).unwrap()).unwrap(); assert!(matches!(back, Read::Call { ref kind, .. } if kind == "qec")); - let a = run(Read::Admit { request: json!({ "qasm": "OPENQASM 3.0; qubit q; h q;", "shots": 4 }), job_bytes: 1 << 30 }).unwrap(); + let a = run(Read::Admit { request: json!({ "qasm": "OPENQASM 3.0; qubit q; h q;", "shots": 4 }), job_bytes: 1 << 30, keep: true }).unwrap(); assert!(a["plan"]["method"].is_string() && a["items"].is_null(), "{a}"); } } diff --git a/wai/wai-quantum-service/tests/depth.rs b/wai/wai-quantum-service/tests/depth.rs index c37ea661f..cc9a8d215 100644 --- a/wai/wai-quantum-service/tests/depth.rs +++ b/wai/wai-quantum-service/tests/depth.rs @@ -47,7 +47,12 @@ fn start(dir: &Path, port: u16, burst: u32) -> Child { /// [`start`], with `env` set beside. fn start_with(dir: &Path, port: u16, burst: u32, env: &[(&str, &str)]) -> Child { - let child = Command::new(env!("CARGO_BIN_EXE_wai-quantum-service")) + start_from(Path::new(env!("CARGO_BIN_EXE_wai-quantum-service")), dir, port, burst, env) +} + +/// [`start_with`], the service started from the binary at `bin`. +fn start_from(bin: &Path, dir: &Path, port: u16, burst: u32, env: &[(&str, &str)]) -> Child { + let child = Command::new(bin) .envs(env.iter().copied()) .arg("serve") .env("WAI_QSVC_ADDR", format!("127.0.0.1:{port}")) @@ -523,14 +528,20 @@ fn reading_leaves_the_runtime_free_and_a_client_has_its_share() { }); }); assert!(worst < 1.0, "health took {worst} s"); - // One client, three at once: the third is told at once, its share in hand. + // One client's parallel calls, as an agent makes them: served one after + // another behind its share, every one answered. + at_once(5, |_| { + let out = from("10.0.1.1", "/v1/critique", &json!({ "qasm": "OPENQASM 3.0; qubit q; h q;" })); + assert!(out.starts_with("HTTP/1.1 200"), "{out}"); + }); + // Past its share and the requests it may have waiting, told at once. let answers = std::sync::Mutex::new(Vec::new()); - at_once(3, |k| { - if k == 2 { + at_once(7, |k| { + if k == 6 { std::thread::sleep(Duration::from_millis(300)); } let began = Instant::now(); - let out = from("10.0.1.1", "/v1/critique", &json!({ "qasm": slow(1000 + k) })); + let out = from("10.0.1.2", "/v1/critique", &json!({ "qasm": slow(1000 + k) })); answers.lock().unwrap().push((out.contains("J032"), began.elapsed().as_secs_f64())); }); let answers = answers.into_inner().unwrap(); @@ -641,3 +652,136 @@ fn a_stuck_connection_is_let_go() { svc.wait().unwrap(); let _ = std::fs::remove_dir_all(&d); } + +/// `body` sent to `path` as the client `who` (behind the trusted proxy): +/// the status and the whole answer. +fn post_as(port: u16, who: &str, path: &str, body: &Value) -> (u16, String) { + let mut s = TcpStream::connect(("127.0.0.1", port)).unwrap(); + s.set_read_timeout(Some(Duration::from_secs(150))).unwrap(); + let text = body.to_string(); + write!(s, "POST {path} HTTP/1.1\r\nHost: 127.0.0.1\r\nContent-Type: application/json\r\nX-Forwarded-For: {who}\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{text}", text.len()).unwrap(); + let mut out = String::new(); + let _ = s.read_to_string(&mut out); + (out.split_whitespace().nth(1).and_then(|c| c.parse().ok()).unwrap_or(0), out) +} + +/// An error-correction threshold at the service's bounds whose shots run for +/// minutes: what one client used to hold every check turn with. +fn heavy_threshold() -> Value { + json!({ "family": "surface", "distances": [7, 7, 7, 7], "p": [0.04, 0.045, 0.05, 0.055], "shots": 12_500 }) +} + +/// One client sending checks that each hold a turn for as long as a reading +/// may take holds no more than its share of the checks: every other client's +/// check is answered beside them, where it waited and was refused. +#[test] +fn one_client_cannot_hold_every_check_turn() { + let d = dir("checkshare"); + let port = free_port(); + let mut svc = start_with(&d, port, 1000, &[("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1"), ("WAI_QSVC_QEC_SECS", "60"), ("WAI_QSVC_READ_SECS", "40")]); + std::thread::scope(|t| { + for _ in 0..2 { + t.spawn(|| post_as(port, "198.51.100.66", "/v1/qec/threshold", &heavy_threshold())); + } + std::thread::sleep(Duration::from_millis(1500)); + for (path, body) in [("/v1/critique", json!({ "qasm": "OPENQASM 3.0; qubit q; h q;" })), ("/v1/qec/code", json!({ "code": "surface:3", "certify": false }))] { + let began = Instant::now(); + let (code, out) = post_as(port, "203.0.113.7", path, &body); + assert!(code == 200 && began.elapsed() < Duration::from_secs(5), "{path}: {code} after {:?}: {out}", began.elapsed()); + } + alive(port, &mut svc, "one client's checks"); + svc.kill().unwrap(); + }); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// An error-correction call at the bounds whose shots would run for minutes +/// ends well inside the reading's time: refused as soon as its first shots +/// say the rest would pass the call's time, with the command that runs it +/// locally. One that fits is answered. +#[test] +fn a_qec_call_at_its_bounds_ends_well_inside_the_reading_s_time() { + let d = dir("qectime"); + let port = free_port(); + let mut svc = start(&d, port, 1000); + for (path, body) in [("/v1/qec/threshold", heavy_threshold()), ("/v1/qec/decode", json!({ "memory": { "distance": 7, "rounds": 7, "p": 0.05 }, "shots": 200_000 }))] { + let began = Instant::now(); + let (code, b) = json_of(port, "POST", path, Some(&body)); + assert!(code == 422 && b.to_string().contains("J031") && b.to_string().contains("wai-quantum qec"), "{path}: {code} {b}"); + assert!(began.elapsed() < Duration::from_secs(30), "{path}: {:?}", began.elapsed()); + alive(port, &mut svc, path); + } + let (code, b) = json_of(port, "POST", "/v1/qec/threshold", Some(&json!({ "distances": [3], "p": [0.001], "shots": 1000 }))); + assert_eq!(code, 200, "{b}"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// The service's own peak memory, KiB, where the platform reports it. +fn peak_kib(svc: &Child) -> Option { + let s = std::fs::read_to_string(format!("/proc/{}/status", svc.id())).ok()?; + s.lines().find(|l| l.starts_with("VmHWM:"))?.split_whitespace().nth(1)?.parse().ok() +} + +/// A reading's answer comes back into the service, whose memory every job +/// shares: a sweep whose hundred items each copy a large field, a few hundred +/// kilobytes sent, is refused for what its answer takes once read, before the +/// service reads it, and the service's own memory stays small. A dry run +/// takes back the plans and the points alone. +#[test] +fn a_reading_s_answer_is_held_to_what_it_takes_once_read() { + let d = dir("answer-size"); + let port = free_port(); + let mut svc = start(&d, port, 1000); + let sweep = |pad: usize, dry: bool| json!({ "qasm": "OPENQASM 3.0;\ninput int n;\nqubit q;\nx q;\n", "sweep": { "n": (0..100).collect::>() }, "shots": 1, "dry_run": dry, "pad": vec![0; pad] }); + // A dry run takes back what it reports, however large its items are. + for pad in [0, 100_000] { + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&sweep(pad, true))); + assert!(code == 200 && b["items"].as_array().is_some_and(|i| i.len() == 100), "{code} {b}"); + } + // A job keeps its items: past what the service takes back, refused. + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&sweep(100_000, false))); + assert!(code == 422 && b.to_string().contains("J031") && b.to_string().contains("once read"), "{code} {b}"); + if let Some(kib) = peak_kib(&svc) { + assert!(kib < 400 << 10, "the service held {kib} KiB"); + } + alive(port, &mut svc, "a large answer"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// The binary replaced before the restart, by a rename (Linux, where a copy +/// over a running binary is refused) or a copy over it (elsewhere): the +/// running service goes on reading and running jobs with its own until it is +/// restarted, where every reading failed. +#[test] +fn the_binary_can_be_replaced_before_the_restart() { + let d = dir("replace"); + let bin = d.join("app/wai-quantum-service"); + std::fs::create_dir_all(bin.parent().unwrap()).unwrap(); + std::fs::copy(env!("CARGO_BIN_EXE_wai-quantum-service"), &bin).unwrap(); + let port = free_port(); + let mut svc = start_from(&bin, &d, port, 1000, &[]); + let stub = d.join("app/stub"); + std::fs::write(&stub, "#!/bin/sh\necho 'not this service' >&2\nexit 2\n").unwrap(); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + std::fs::set_permissions(&stub, std::fs::Permissions::from_mode(0o755)).unwrap(); + } + if cfg!(target_os = "linux") { + std::fs::rename(&stub, &bin).unwrap(); + } else { + std::fs::copy(&stub, &bin).unwrap(); + } + let (code, b) = json_of(port, "POST", "/v1/critique", Some(&json!({ "qasm": "OPENQASM 3.0; qubit q; h q;" }))); + assert_eq!(code, 200, "{b}"); + let (code, b) = json_of(port, "POST", "/v1/run", Some(&json!({ "qasm": "OPENQASM 3.0; qubit q; bit c; h q; c = measure q;", "shots": 10 }))); + assert!(code == 200 && b["status"] == "done", "{code} {b}"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} diff --git a/wai/wai-quantum/src/engine/qec.rs b/wai/wai-quantum/src/engine/qec.rs index a16493874..881209788 100644 --- a/wai/wai-quantum/src/engine/qec.rs +++ b/wai/wai-quantum/src/engine/qec.rs @@ -45,6 +45,49 @@ fn qerr(m: impl Into) -> String { tag(refusal::QEC, m.into()) } +thread_local! { + /// The time this thread's sampled decodes may take ([`within_time`]): + /// when the budget began, and how long it is. + static TIME: std::cell::Cell> = const { std::cell::Cell::new(None) }; +} + +/// Runs `f` with the shots it samples and decodes held to `limit` in all, on +/// this thread: a decode with `shots`, and a threshold, each refused +/// (`J031`, [`refusal::TOO_LARGE_HERE`]) as soon as what it has done says the +/// rest would take longer, or once it has. What a shot costs depends on the +/// noise and the decoder as much as on the counts, so a host that answers +/// within a time measures it as it goes rather than guessing it first. +pub fn within_time(limit: std::time::Duration, f: impl FnOnce() -> T) -> T { + struct Reset(Option<(std::time::Instant, std::time::Duration)>); + impl Drop for Reset { + fn drop(&mut self) { + TIME.with(|t| t.set(self.0)); + } + } + let _reset = Reset(TIME.with(|t| t.replace(Some((std::time::Instant::now(), limit))))); + f() +} + +/// Whether `done` of `total` shots, sampled and decoded so far, leave the +/// rest within this thread's time ([`within_time`]): the refusal, naming the +/// command that runs it locally, where they do not. +fn on_time(done: u64, total: u64, op: &str) -> Result<(), String> { + let Some((start, limit)) = TIME.with(std::cell::Cell::get) else { return Ok(()) }; + let spent = start.elapsed(); + let refuse = |secs: f64| Err(tag(refusal::TOO_LARGE_HERE, format!("its shots would take about {secs:.0} s to sample and decode here, more than the {} s a call here takes; run `wai-quantum qec {op}` on your machine", limit.as_secs()))); + if spent > limit { + return refuse(spent.as_secs_f64() * total.max(1) as f64 / done.max(1) as f64); + } + // Projected once enough is done to say: a hundredth, or a second. + if done > 0 && (done.saturating_mul(100) >= total || spent >= std::time::Duration::from_secs(1)) { + let projected = spent.as_secs_f64() * total as f64 / done as f64; + if projected > limit.as_secs_f64() { + return refuse(projected); + } + } + Ok(()) +} + fn hex(b: &[u8]) -> String { b.iter().map(|x| format!("{x:02x}")).collect() } @@ -361,6 +404,7 @@ pub fn decode(request: &Value) -> Result { let smp = frame::sample(&c, shots as usize, seed); let mut failures = 0u64; for i in 0..smp.shots { + on_time(i as u64, shots, "decode")?; if dec.run(&smp.fired(i), &mut ws)?.0 != smp.flips(i) { failures += 1; } @@ -519,6 +563,8 @@ pub fn threshold(request: &Value) -> Result { let mut memory = Vec::new(); let decoder; let noise; + let total = shots * points as u64; + let mut done = 0u64; match family { "surface" => { decoder = request.get("decoder").and_then(Value::as_str).unwrap_or("matching"); @@ -537,10 +583,12 @@ pub fn threshold(request: &Value) -> Result { let mut ws = None; let mut failures = 0u64; for i in 0..smp.shots { + on_time(done + i as u64, total, "threshold")?; if dec.run(&smp.fired(i), &mut ws)?.0 != smp.flips(i) { failures += 1; } } + done += shots; let pt = MemoryPoint { distance: d, rounds, p, shots, failures }; memory.push(pt); let rate = failures as f64 / shots as f64; @@ -559,6 +607,8 @@ pub fn threshold(request: &Value) -> Result { }; let c = read_code(&json!(code_name))?; for &p in &ps { + on_time(done, total, "threshold")?; + done += shots; let s = seed.wrapping_add(d * 1000 + (p * 1e6) as u64); let (trials, failures) = match &c.color { Some(cc) => crate::quantum_color::benchmark(cc, p, shots as u32, s), @@ -693,6 +743,26 @@ pub fn call(op: &str, request: &Value) -> Result { mod tests { use super::*; + /// Within a time, a decode and a threshold whose shots would take longer + /// are refused as soon as their first shots say so, not at the end of the + /// time; one that fits answers as it did; with no time set, none is. + #[test] + fn a_time_holds_the_shots_sampled_and_decoded() { + let heavy = [ + ("threshold", json!({ "family": "surface", "distances": [7, 7, 7, 7], "p": [0.04, 0.045, 0.05, 0.055], "shots": 12_500 })), + ("decode", json!({ "memory": { "distance": 7, "rounds": 7, "p": 0.05 }, "shots": 200_000 })), + ]; + for (op, req) in heavy { + let t = std::time::Instant::now(); + let e = within_time(std::time::Duration::from_secs(3), || call(op, &req)).unwrap_err(); + assert!(e.ends_with("[J031]") && e.contains("wai-quantum qec"), "{op}: {e}"); + assert!(t.elapsed() < std::time::Duration::from_secs(10), "{op}: {:?}", t.elapsed()); + } + let small = json!({ "family": "surface", "distances": [3], "p": [0.001], "shots": 200 }); + assert!(within_time(std::time::Duration::from_secs(30), || call("threshold", &small)).is_ok()); + assert!(TIME.with(std::cell::Cell::get).is_none()); + } + #[test] fn every_family_reads_and_certifies_its_distance() { for (name, n, k, d) in [("surface:5", 25, 1, 5), ("toric:4", 32, 2, 4), ("steane", 7, 1, 3), ("color:7", 37, 1, 7), ("bb:72", 72, 12, 6), ("gross", 144, 12, 12)] { diff --git a/wai/wai-quantum/src/quantum_frame.rs b/wai/wai-quantum/src/quantum_frame.rs index d1eae8690..fea2e0224 100644 --- a/wai/wai-quantum/src/quantum_frame.rs +++ b/wai/wai-quantum/src/quantum_frame.rs @@ -1653,14 +1653,25 @@ mod tests { } } - /// A text whose qubits reach 4294967294 leaves the carrier no index: a - /// circuit with a correlated error is refused at the error's line. + /// A text whose qubits reach 4294967294 is refused where it names one, as + /// past what a circuit here holds. One that names the last qubit a circuit + /// holds leaves its correlated errors' carrier no room within the cap: the + /// circuit is refused at the error's line, as the carrier would make it a + /// qubit wider than its text may name. #[test] fn a_correlated_error_with_no_index_left_for_its_carrier_is_refused() { - for (text, line) in [("R 0\nE(0.1) X1\nX_ERROR(0.1) 4294967294\nM 0\nDETECTOR rec[-1]\n", 2), ("R 0\nX 4294967294\nREPEAT 2 {\n CORRELATED_ERROR(0.1) Z0\n}\nM 0\n", 4)] { - let e = parse(text).map(|c| c.qubits).unwrap_err(); - assert!(e.to_string().starts_with(&format!("line {line}:")) && e.to_string().contains("4294967294"), "{text}: {e}"); + let last = crate::quantum_qec_text::MAX_QUBITS - 1; + for (text, line, names) in [ + ("R 0\nE(0.1) X1\nX_ERROR(0.1) 4294967294\nM 0\nDETECTOR rec[-1]\n".to_string(), 3, "4294967294".to_string()), + (format!("R 0\nE(0.1) X1\nX_ERROR(0.1) {last}\nM 0\nDETECTOR rec[-1]\n"), 2, (last + 2).to_string()), + (format!("R 0\nX {last}\nREPEAT 2 {{\n CORRELATED_ERROR(0.1) Z0\n}}\nM 0\n"), 4, (last + 2).to_string()), + ] { + let e = parse(&text).map(|c| c.qubits).unwrap_err(); + assert!(e.to_string().starts_with(&format!("line {line}:")) && e.to_string().contains(&names) && e.to_string().contains("past the"), "{text}: {e}"); } + // Without a correlated error the last qubit is the circuit's own. + assert_eq!(parse(&format!("R 0\nX_ERROR(0.1) {last}\nM 0\nDETECTOR rec[-1]\n")).unwrap().qubits, last + 1); + assert_eq!(parse(&format!("R 0\nE(0.1) X1\nX_ERROR(0.1) {}\nM 0\nDETECTOR rec[-1]\n", last - 1)).unwrap().qubits, last + 1); } /// The instructions this reader added are in the page that lists the text -- 2.54.0 (Apple Git-157) From 4f44194bf052df6952ed02126f7a5544e4f3dee0 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Sat, 7 Nov 2026 08:10:24 -0500 Subject: [PATCH 06/18] wai-quantum-qec workflow: name the test that holds a call's shots to its time Co-Authored-By: Claude Opus 5.5 --- .github/workflows/wai-quantum-qec.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/wai-quantum-qec.yml b/.github/workflows/wai-quantum-qec.yml index fce6549ff..bdf59b650 100644 --- a/.github/workflows/wai-quantum-qec.yml +++ b/.github/workflows/wai-quantum-qec.yml @@ -136,6 +136,7 @@ jobs: engine::mitigate::tests::a_job_carries_its_mitigation_and_its_refusals_name_their_code \ engine::mitigate::tests::refusals_say_why \ engine::qec::tests::every_family_reads_and_certifies_its_distance \ + engine::qec::tests::a_time_holds_the_shots_sampled_and_decoded \ engine::qec::tests::each_decoder_corrects_every_single_error \ engine::qec::tests::circuit_level_decoding_reads_every_model_form \ engine::qec::tests::a_threshold_is_measured_and_distance_helps_below_it \ -- 2.54.0 (Apple Git-157) From 62994dc04ed7e976c7235b8ca36c20a99e1a8cb5 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Sat, 7 Nov 2026 09:48:25 -0500 Subject: [PATCH 07/18] wai-quantum-service: ended jobs keep their request as text within caps, every answer comes within 55 s, a client's quick calls pass its own slow one, and a QEC call is timed apart from its setup Third review of #676, 14 findings: - A job keeps its request's canonical text, never its parse; a set lets go of its items when it ends; what ended jobs keep is capped per client (WAI_QSVC_CLIENT_KEPT_MIB, 128) and in all (WAI_QSVC_KEPT_MIB, 2048), oldest first. Bodies are parsed after the window check, reports are spliced from the text, memory is given back after large requests, and on macOS the service starts with the allocator's large cache off. - One deadline from arrival, WAI_QSVC_ANSWER_SECS (55): turns, readings, run waits and MCP verdicts end by it; self-host.md gives each route's arithmetic under the proxy's 60 s. - Past its share a client is lent one turn while two stay free, for 5 s; eight of its requests may wait; the refusal says whether its own requests or the pool held it up. - QEC calls time each point's setup apart from a pilot of its shots, every threshold kind included, so a call is refused at once or run. - Children are stopped by libc::kill on their process group, again every 100 ms until they end. - The Breaking lines for E::Chain and S::If; OpenAPI statuses on every check route; the poll's empty critique in the changelog. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/wai-quantum-qec.yml | 1 + wai/changes/2026-11-06-quantum-parse-depth.md | 53 +++- wai/wai-quantum-service/README.md | 16 +- wai/wai-quantum-service/docs/self-host.md | 5 +- wai/wai-quantum-service/src/discovery.rs | 24 +- wai/wai-quantum-service/src/http.rs | 139 ++++++--- wai/wai-quantum-service/src/jobs.rs | 266 +++++++++++++++-- wai/wai-quantum-service/src/main.rs | 24 +- wai/wai-quantum-service/src/mcp.rs | 12 +- wai/wai-quantum-service/src/offload.rs | 274 ++++++++++++++---- wai/wai-quantum-service/tests/depth.rs | 245 ++++++++++++---- wai/wai-quantum/src/engine/qec.rs | 193 +++++++++--- 12 files changed, 1010 insertions(+), 242 deletions(-) diff --git a/.github/workflows/wai-quantum-qec.yml b/.github/workflows/wai-quantum-qec.yml index bdf59b650..de235d728 100644 --- a/.github/workflows/wai-quantum-qec.yml +++ b/.github/workflows/wai-quantum-qec.yml @@ -137,6 +137,7 @@ jobs: engine::mitigate::tests::refusals_say_why \ engine::qec::tests::every_family_reads_and_certifies_its_distance \ engine::qec::tests::a_time_holds_the_shots_sampled_and_decoded \ + engine::qec::tests::a_time_is_measured_apart_from_setup_and_holds_every_threshold \ engine::qec::tests::each_decoder_corrects_every_single_error \ engine::qec::tests::circuit_level_decoding_reads_every_model_form \ engine::qec::tests::a_threshold_is_measured_and_distance_helps_below_it \ diff --git a/wai/changes/2026-11-06-quantum-parse-depth.md b/wai/changes/2026-11-06-quantum-parse-depth.md index e1182f970..3cdc734c8 100644 --- a/wai/changes/2026-11-06-quantum-parse-depth.md +++ b/wai/changes/2026-11-06-quantum-parse-depth.md @@ -65,30 +65,55 @@ `verify_quantum_receipt` on a set are counted and read as a check is, as its sweep is written out again, and a seal that does not hold is not. A decode is held to 1 024 fired detectors, as matching grows faster than their square, and a memory experiment to 21 rounds. An error-correction decode or threshold measures its shots as they - run (`engine::qec::within_time`) and is refused as soon as the rest would take more than `WAI_QSVC_QEC_SECS` (25), - where one at the published bounds ran for many minutes and held its turn to the reading's cap. + run (`engine::qec::within_time`): each point is set up and sampled, and a pilot of its own shots decoded, before the + rest of any, and the call is refused at once when what is measured says it would take more than + `WAI_QSVC_QEC_SECS` (25), stopped only if it runs a fifth past that; setup is not multiplied by the shots, so a call + that fits is run, and code-capacity thresholds are measured on a pilot of trials of each point too. One at the + published bounds ran for many minutes and held its turn to the reading's cap. - **The job service: in a child.** Admission, a dry run, every check's bounds and answer, the error-correction, pulse and variational calls and the estimates, a set's receipt, and the verdicts the MCP tools state beside an answer are read by a child, the service's own binary run as `wai-quantum-service read`, sandboxed as a worker is. Four requests are read at once and four checks beside them. A client has its share of each, - `WAI_QSVC_CLIENT_READS` (2) and `WAI_QSVC_CLIENT_CHECKS` (1), smaller than the pool, so one client holds no pool; - its requests past its share wait behind it in order (an agent's parallel calls are answered one after another), - four at most, 32 in all, each for ten seconds; past any of these a request is refused, `503` and `J032`. People - behind one address share one client's turns. A child is polled every 50 ms from a thread of its own (on macOS its + `WAI_QSVC_CLIENT_READS` (2) and `WAI_QSVC_CLIENT_CHECKS` (1), smaller than the pool, so one client holds no pool + for longer than a quick call; past its share one more turn is lent to it while the pool keeps two free, for at most + five seconds (so its quick calls pass its own slow one; a call that takes longer on a lent turn is refused, `503`), + and the rest wait behind its own turns in order (an agent's burst of eight is answered one after another), eight + at most, 32 in all, each for ten seconds; past any of these a request is refused, `503` and `J032`, saying whether + its own requests or the pool held it up. People behind one address share one client's turns. A child is polled every 50 ms from a thread of its own (on macOS its physical footprint, compressed pages included; on Linux resident and swapped pages) and killed past - `WAI_QSVC_READ_GB` (4) or `WAI_QSVC_READ_SECS` (45, so that its refusal reaches a client within a proxy's 60 s), its - request refused, `422` and `J031`. The poll is not a kernel limit; on Linux a child also holds its address space + `WAI_QSVC_READ_GB` (4) or `WAI_QSVC_READ_SECS` (45), its request refused, `422` and `J031`; the kill is a signal the + service sends itself, to the child's process group, again until it has ended, so it works when no new process can + be started. The poll is not a kernel limit; on Linux a child also holds its address space to twice the cap and a gigabyte, and the kernel ends readers and workers before the service. What a reading - gives back is held to `WAI_QSVC_READ_ANSWER_MIB` (128) as it takes once read, refused in the child before it is - written, and read off the request threads; a dry run gets back its plans and points alone. One that panics, + gives back is held to `WAI_QSVC_READ_ANSWER_MIB` (128) as it takes once read (about 80 bytes a value), refused in + the child before it is written, and read off the request threads; a dry run gets back its plans and points alone. + This newly refuses a check whose answer is that large: a critique of a program whose gates each draw a remark + (`qubit q; for int i in [1:N] { x q; }`) is refused, `422` and `J031`, at 200 000 gates, which main answered (150 000 + are answered). One that panics, overflows or aborts is answered `500`; the service runs on. A client that hangs up ends its child. - **The job service: MCP.** A tool that says nothing about a program takes no turn. A verdict reads the program only - for a call that was counted; a poll states what its job holds, and reads nothing, however often it is asked; a + for a call that was counted; a poll states what its job holds, and reads nothing, however often it is asked (so a + poll of a queued or running job, or of an ended one sent without `verdict` or `intent`, now gives a verdict with + empty `diagnostics` and `critique`, and a view on a poll no drawing, where they read the program before); a call refused before it was counted (a bad key, a tool the service does not have, past the window) or told to try again states a verdict that gives the reason. One that finds no turn for its verdict does the same, so every answer about a program has one, and follows its schema. A refused task is the same, where it read the program on a runtime worker with no budget. A run's MCP arguments that are not an object are refused, where they dropped the connection. +- **The job service: one answer within a proxy's wait.** Every request is answered within `WAI_QSVC_ANSWER_SECS` (55) + of its arrival: its waits for a turn and its readings end by then, a run waits for its job only for what is left + (`/v1/run` and the MCP run tools answer with the job, still running, to poll, rather than past a proxy's 60 s), and + an MCP answer's verdict is read only if time is left, saying so where it is not. +- **The job service: what ended jobs keep.** A job keeps its request as the canonical text the store keeps, where it + kept the parsed request (about 80 bytes a value, 37 times a body of numbers) for a day, across restarts and + counted against nothing; a set's items are let go once it ends. What ended jobs keep, their requests' text and + their results as read, is held to `WAI_QSVC_CLIENT_KEPT_MIB` (128) a client and `WAI_QSVC_KEPT_MIB` (2048) in all, + the oldest dropped first, before their `WAI_QSVC_RETAIN_SECS`. A request's body is parsed only once its client's + window has room (a resend under its key, answered before any limit, first), and a large MCP message from a client + past its window is refused unread. A job's report is written from its request's text, never parsed again, and the + memory a large request or answer was parsed into is given back once it is answered. On macOS the service starts + itself with the allocator's large-block cache off (`MallocLargeCache=0`), which kept every large block a parsed + request freed, dirty and charged to the service: on arm64, 30 requests of a megabyte kept 1 GiB; with it off, 2 MiB. - **The job service: connections, updates and restarts.** A request's header section is held to `WAI_QSVC_HEADER_SECS` (150, which also closes an idle connection, so it sits above a proxy's two-minute idle limit) and its body to `WAI_QSVC_BODY_SECS` (30; past it, `408`). Readers and workers start from the service's own @@ -101,3 +126,9 @@ refused MCP tasks that hold no worker; heavy requests from eight clients beside a health probe; a client's share of the turns; a reading stopped at its deadline; a hang-up that frees its turn; stuck connections let go; and a restart that reads nothing. Two tests that could only end by the job's own limit now fail by name. +- **Breaking:** `api:rust:wai_quantum::qasm::ast::E::Chain`: a run of left-associative operators of one precedence + is one node, `E::Chain(first, [(op, operand), …])`, so a run of any length holds no recursion; a match over every + variant of `E` outside the crate adds this one. +- **Breaking:** `api:rust:wai_quantum::qasm::ast::S::If`: an `if` and its `else if`s are one statement, + `S::If { arms: [(cond, then), …], other }`, where it was `S::If { cond, then, other }` with the `else if`s nested in + `other`; code that builds or destructures it reads each arm in turn. diff --git a/wai/wai-quantum-service/README.md b/wai/wai-quantum-service/README.md index aa6b49c44..bdd70517e 100644 --- a/wai/wai-quantum-service/README.md +++ b/wai/wai-quantum-service/README.md @@ -197,10 +197,11 @@ A submission is admitted in the order of what each step costs: Reading happens outside the service's own process, as each job's run does: each request is read by a child, this binary run as `wai-quantum-service read`, sandboxed as a worker is. -- **Turns.** Four requests are read at a time (jobs, dry runs and the MCP tools' verdicts), and four checks beside them. A client has its share of each: `WAI_QSVC_CLIENT_READS` (2) of the first and `WAI_QSVC_CLIENT_CHECKS` (1) of the checks, so no client holds a whole pool. A client's requests past its share wait behind it, in order, so an agent's parallel calls are answered one after another; up to four of them may wait, at most 32 requests wait in all, each for ten seconds, and past any of these a request is refused, `503` and `J032`, having read and spent nothing. A client is its address, so people behind one address (an office, a carrier's NAT, a class) share one client's turns and its window: raise the shares, or give such a group an API key of its own on a private deployment. -- **Memory.** A child's memory is polled every 50 ms from a thread of its own, and the child is killed (`kill -9`) once it holds more than `WAI_QSVC_READ_GB` (4); the request is refused, `422` and `J031`. On macOS the poll reads the child's physical footprint, which counts its compressed and swapped pages; on Linux its resident and swapped pages. A poll is not a kernel limit: a child can pass the cap by what it touches in one interval before it is killed. On Linux the child also holds its own address space to twice the cap and a gigabyte, so an allocation past that fails at once (`422`, `J031`), and the kernel ends a reader or a worker before the service when the host runs short. macOS gives an unprivileged process neither, and has no limit that counts compressed memory. -- **Time.** A child that reads for longer than `WAI_QSVC_READ_SECS` (45) is killed and the request refused, `422` and `J031`. With the ten-second wait for a turn, an answer comes within the 60 s a proxy in front commonly waits for one; behind a proxy that waits less, lower it. An error-correction call's shots are measured as they run and refused as soon as the rest would take more than `WAI_QSVC_QEC_SECS` (25). -- **Its answer.** What a reading gives back comes into the service's memory, which every job shares: one that would take more than `WAI_QSVC_READ_ANSWER_MIB` (128) once read, counted value by value, is refused in the child before it is written, and the service reads it off its request threads. A dry run gets back the plans and points alone. +- **Turns.** Four requests are read at a time (jobs, dry runs and the MCP tools' verdicts), and four checks beside them. A client has its share of each: `WAI_QSVC_CLIENT_READS` (2) of the first and `WAI_QSVC_CLIENT_CHECKS` (1) of the checks, so no client holds a whole pool. Past its share, one more turn is lent to a client while the pool keeps two free for others, for at most five seconds, so its quick calls (a critique, a lint, an estimate) pass its own slow one; a call that takes longer on a lent turn is refused, `503` and `J032`. Its other requests wait behind its own turns, in order, so an agent's burst of parallel calls is answered one after another: up to eight of them may wait, at most 32 requests wait in all, each for ten seconds, and past any of these a request is refused, `503` and `J032`, having read and spent nothing, saying whether its own requests or the pool held it up. A client is its address, so people behind one address (an office, a carrier's NAT, a class) share one client's turns and its window: raise the shares, or give such a group an API key of its own on a private deployment. +- **Memory.** A child's memory is polled every 50 ms from a thread of its own, and the child is killed once it holds more than `WAI_QSVC_READ_GB` (4); the request is refused, `422` and `J031`. The kill is a signal the service sends itself (`SIGKILL`, to the child's process group, and again every 100 ms until it has ended), so it needs no new process. On macOS the poll reads the child's physical footprint, which counts its compressed and swapped pages; on Linux its resident and swapped pages. A poll is not a kernel limit: a child can pass the cap by what it touches in one interval before it is killed. On Linux the child also holds its own address space to twice the cap and a gigabyte, so an allocation past that fails at once (`422`, `J031`), and the kernel ends a reader or a worker before the service when the host runs short. macOS gives an unprivileged process neither, and has no limit that counts compressed memory. +- **Time.** A child that reads for longer than `WAI_QSVC_READ_SECS` (45) is killed and the request refused, `422` and `J031`. Every request is answered within `WAI_QSVC_ANSWER_SECS` (55) of its arrival, under the 60 s a proxy in front commonly waits for an answer: its waits for a turn and its readings end by then, `POST /v1/run` and the MCP run tools wait for their job only for what is left and then answer with the job, still running, to poll, and an MCP answer's verdict is read again only if time is left (its verdict says so where it was not). An error-correction call's shots are measured as they run: each point is set up and sampled, and a pilot of its own shots decoded, before the rest of any; it is refused at once when what is measured says it would take more than `WAI_QSVC_QEC_SECS` (25), and stopped if it runs a fifth past that. +- **Its answer.** What a reading gives back comes into the service's memory, which every job shares: one that would take more than `WAI_QSVC_READ_ANSWER_MIB` (128) once read, about 80 bytes a value, is refused in the child before it is written (`422`, `J031`), and the service reads it off its request threads. A dry run gets back the plans and points alone. A check's answer can be that large: a critique of a program of 200,000 gates that each draw a remark (`qubit q; for int i in [1:200000] { x q; }`) is refused so; one of 150,000 is answered. +- **What ended jobs keep.** A job keeps its request as the canonical text the store keeps, and a set lets go of its items once it ends. What ended jobs keep in memory, their requests' text and their results as read, is held to `WAI_QSVC_CLIENT_KEPT_MIB` (128) a client and `WAI_QSVC_KEPT_MIB` (2048) in all; past either, the oldest ended jobs are dropped before their `WAI_QSVC_RETAIN_SECS`. A request's body is parsed only once its client's window has room (a resend under its key first, as it is answered before any limit), and a large MCP message from a client past its window is refused unread. A job's report is written from its request's text, never parsed again, and the memory a large request or answer was parsed into is given back to the system once it is answered. On macOS the service starts itself with the allocator's large-block cache off (`MallocLargeCache=0`): that cache kept every large block a parsed request had freed, dirty and charged to the service (on arm64, 30 requests of a megabyte kept 1 GiB; with it off, 2 MiB). - **Ending.** A child that panics, overflows its stack or aborts is answered `500`, and the service runs on. A client that hangs up ends its child. - **Connections.** A request whose header section takes longer than `WAI_QSVC_HEADER_SECS` (150) to arrive is closed; the same limit closes a connection left idle between requests, so it is set above a proxy's own idle limit (two minutes, commonly), and the proxy, not the service, closes its idle connections. A body that takes longer than `WAI_QSVC_BODY_SECS` (30) is refused, `408`. - **Updating.** Readers and workers start from the service's own binary as it was started: on Linux its running image, elsewhere a copy it keeps beside its signing key, named by its digest. So the binary can be replaced before the restart (by a copy over it, or, on Linux, where a running binary cannot be written, by a rename), and the running service goes on with its own until it is restarted. @@ -229,7 +230,7 @@ A restart reads no program: a job keeps the plan it was admitted with, and a set Finished jobs are kept for `WAI_QSVC_RETAIN_SECS` (a day here), then dropped. A record names its client by a keyed hash of the address, never the address. [docs/self-host.md](docs/self-host.md#the-job-store) describes the store. -The six checks answer at once, each read and answered in a reading child as a job's admission is. Each counts against the client's submissions, as a job does, once it has its turn and before it reads anything (a check that waits ten seconds and finds no turn is refused, `503`, having read and spent nothing). At most four run at a time, one of them a client's, its others waiting behind it. Its bounds and its answer then read its programs within one budget, each read once: a check that runs the budget out is refused, `422` and `J031`, whatever it had made of it, so a repair of many fixes to a slow program ends at the budget. Some checks are refused (`422`) with the command that runs them on your own machine: +The six checks answer at once, each read and answered in a reading child as a job's admission is. Each counts against the client's submissions, as a job does, once it has its turn and before it reads anything (a check that waits ten seconds and finds no turn is refused, `503`, having read and spent nothing). At most four run at a time, one of them a client's (and one more lent for a quick call), its others waiting behind it. Its bounds and its answer then read its programs within one budget, each read once: a check that runs the budget out is refused, `422` and `J031`, whatever it had made of it, so a repair of many fixes to a slow program ends at the budget. Some checks are refused (`422`) with the command that runs them on your own machine: - an equivalence check that would apply more than 4·10⁹ gate amplitudes; - an improvement whose candidates' checks would apply more than that; @@ -247,7 +248,7 @@ The six checks answer at once, each read and answered in a reading child as a jo - resources: the language the engine reads (`wai-quantum://docs/language`), its diagnostics, the methods with their accuracy classes (their reach in a job's memory here), and worked examples, each with what it must do; - prompts: writing and verifying a circuit for an intent, explaining a result and its receipt, and repairing a program from its diagnostics. Each names this server's own tools. -Every answer about a program states a verdict. Reading the program again for it costs what reading it cost, so it is read, in a reading child, only for a call that was counted. A poll (`get_quantum_job`) states what its job holds: the result's verdict, or, for a job admitted and not yet ended, that it reads. A tool that says nothing about a program takes no turn. A call refused before it was counted, told to try again, or that finds no turn for its verdict states a verdict that gives the reason and nothing of the program. +Every answer about a program states a verdict. Reading the program again for it costs what reading it cost, so it is read, in a reading child, only for a call that was counted. A poll (`get_quantum_job`) states what its job holds: the result's verdict where the job asked for one (`verdict` or `intent`); otherwise, and for a job not yet ended, that it reads, with empty `diagnostics` and `critique`, and a view on a poll draws nothing. A tool that says nothing about a program takes no turn. A call refused before it was counted, told to try again, or that finds no turn for its verdict states a verdict that gives the reason and nothing of the program. A client whose `initialize` says its host draws interactive views (the extension `io.modelcontextprotocol/ui`, version 2026-01-26) gets the engine's answers as views, as from the command line's server: each tool that has one names its `ui://wai-quantum/*` resource, the six views are listed and read, and each answer carries what its view draws in its `_meta`. The service keeps no session, so the client's word is carried in the session id it is given at `initialize` (`wq-views-` and 32 random hex digits, kept nowhere), which a client sends back with every request (`Mcp-Session-Id`); a client that draws no views gets no session id and none of this. For a job still running when the run's wait ends, the result view follows it with `poll_quantum_job`, which the host keeps from the model (`visibility: ["app"]`); a model reads a job with `get_quantum_job`. @@ -347,6 +348,9 @@ These variables set the limits: | `WAI_QSVC_READ_SECS` | 45 | the time one request's reading may take | | `WAI_QSVC_READ_ANSWER_MIB` | 128 | the most one request's reading may give back to the service, once read | | `WAI_QSVC_QEC_SECS` | 25 | the time an error-correction call's shots may take, measured as they run | +| `WAI_QSVC_ANSWER_SECS` | 55 | the longest any answer takes, its waits and readings included | +| `WAI_QSVC_CLIENT_KEPT_MIB` | 128 | what one client's ended jobs may keep in memory | +| `WAI_QSVC_KEPT_MIB` | 2048 | what all ended jobs may keep in memory | | `WAI_QSVC_CLIENT_READS` | 2 | how many of one client's jobs, dry runs and verdicts are read at once (more wait) | | `WAI_QSVC_CLIENT_CHECKS` | 1 | how many of one client's checks run at once (more wait) | | `WAI_QSVC_HEADER_SECS` | 150 | how long a client may take to send a request's header section, and how long an idle connection stays open | diff --git a/wai/wai-quantum-service/docs/self-host.md b/wai/wai-quantum-service/docs/self-host.md index 309624e82..8dcc8fc2d 100644 --- a/wai/wai-quantum-service/docs/self-host.md +++ b/wai/wai-quantum-service/docs/self-host.md @@ -54,6 +54,9 @@ Everything is set by environment variables. | `WAI_QSVC_READ_SECS` | 45 | the time one request's reading may take | | `WAI_QSVC_READ_ANSWER_MIB` | 128 | the most one request's reading may give back to the service, once read | | `WAI_QSVC_QEC_SECS` | 25 | the time an error-correction call's shots may take, measured as they run | +| `WAI_QSVC_ANSWER_SECS` | 55 | the longest any answer takes, its waits and readings included | +| `WAI_QSVC_CLIENT_KEPT_MIB` | 128 | what one client's ended jobs may keep in memory | +| `WAI_QSVC_KEPT_MIB` | 2048 | what all ended jobs may keep in memory | | `WAI_QSVC_CLIENT_READS` | 2 | how many of one client's jobs, dry runs and verdicts are read at once (more wait) | | `WAI_QSVC_CLIENT_CHECKS` | 1 | how many of one client's checks run at once (more wait) | | `WAI_QSVC_HEADER_SECS` | 150 | how long a client may take to send a request's header section, and how long an idle connection stays open | @@ -140,7 +143,7 @@ Run the service on a loopback address and let the proxy take the public traffic. - **Event streams.** `GET /v1/jobs/{id}/events` is a server-sent event stream. The proxy must pass it through unbuffered (the service sends `X-Accel-Buffering: no` and a comment every 15 s), and allow it to stay open as long as `WAI_QSVC_JOB_SECS`. Polls with `?wait=` are held up to 25 s. - **Bodies.** A request body is at most 1 MiB, and `POST /v1/run` waits up to 25 s before answering 202. - **Rate-limit fields.** Pass the `RateLimit-Policy`, `RateLimit` and `Retry-After` headers through. -- **Timeouts.** A reading is cut off at `WAI_QSVC_READ_SECS` (45) after a wait for a turn of at most 10 s, so its refusal reaches the client within a proxy's 60 s wait for an answer's header; with a shorter wait in the proxy, lower `WAI_QSVC_READ_SECS` so that the two together fit. Keep the proxy's idle connections shorter than `WAI_QSVC_HEADER_SECS` (150), which also bounds an idle connection on the service's side: the proxy then closes its idle connections itself, and never sends a request on one as the service closes it. +- **Timeouts.** Every answer comes within `WAI_QSVC_ANSWER_SECS` (55) of the request's arrival, so within a proxy's 60 s wait for an answer's header. The worst case of each route, from arrival: a check, a dry run or a submission waits at most 10 s for a turn and reads for at most `WAI_QSVC_READ_SECS` (45), 55 s in all; `POST /v1/run` and the MCP run tools then wait for their job only for what is left of the 55 s (less 5 s, on MCP, for its verdict) and answer with the job, still running, to poll; an MCP answer's verdict waits for a turn and is read only within what is left, and says where no time was left; `tasks/result` holds a reply 50 s at most. With a shorter wait in the proxy, lower `WAI_QSVC_ANSWER_SECS` below it. Keep the proxy's idle connections shorter than `WAI_QSVC_HEADER_SECS` (150), which also bounds an idle connection on the service's side: the proxy then closes its idle connections itself, and never sends a request on one as the service closes it. - **One address, many people.** The service counts a client by the address the proxy gives it. People behind one address share one client's turns (`WAI_QSVC_CLIENT_READS`, `WAI_QSVC_CLIENT_CHECKS`) and window; raise those, or give such a group an API key. ## Updating it diff --git a/wai/wai-quantum-service/src/discovery.rs b/wai/wai-quantum-service/src/discovery.rs index d061fb018..7b69ba15c 100644 --- a/wai/wai-quantum-service/src/discovery.rs +++ b/wai/wai-quantum-service/src/discovery.rs @@ -291,27 +291,27 @@ pub fn openapi(server: &str) -> Value { "/v1/critique": { "post": { "summary": "A program's measures and smells, each with its lines", "requestBody": { "required": true, "content": { "application/json": { "schema": prog.clone() } } }, - "responses": { "200": { "description": "The critique, or {\"ok\": false, \"error\", \"line\"} for a program that does not read" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" } } + "responses": { "200": { "description": "The critique, or {\"ok\": false, \"error\", \"line\"} for a program that does not read" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" }, "422": { "description": "Larger than the service takes (J031): its bounds, the budget of reading, the reading child's memory or time, or an answer past what the service takes back" }, "500": { "description": "Its reading failed inside the service: nothing was answered, and the service runs on" } } } }, "/v1/equivalent": { "post": { "summary": "Whether two programs are the same circuit up to a global phase, and how that was decided", "requestBody": { "required": true, "content": { "application/json": { "schema": { "type": "object", "required": ["a", "b"], "properties": { "a": { "type": "string" }, "b": { "type": "string" } } } } } }, - "responses": { "200": { "description": "{\"verdict\": \"equivalent\" | \"different\" | \"unchecked\", \"how\", \"why\"}" }, "422": { "description": "A check longer than the service takes; the reason names the command that runs it locally" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" } } + "responses": { "200": { "description": "{\"verdict\": \"equivalent\" | \"different\" | \"unchecked\", \"how\", \"why\"}" }, "422": { "description": "A check longer than the service takes; the reason names the command that runs it locally" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" }, "500": { "description": "Its reading failed inside the service: nothing was answered, and the service runs on" } } } }, "/v1/convert": { "post": { "summary": "A program carried between formats (OpenQASM 3, QIR as LLVM text, jeff as base64), the output read back and held to it by the equivalence ladder", "requestBody": { "required": true, "content": { "application/json": { "schema": { "type": "object", "required": ["to"], "properties": { "program": { "type": "string", "description": "The program's text: OpenQASM or QIR." }, "program_base64": { "type": "string", "description": "The program's bytes as base64: jeff." }, "from": { "enum": ["qasm", "qir", "jeff"] }, "to": { "enum": ["qasm", "qir", "jeff"] }, "profile": { "enum": ["base", "adaptive"] }, "check": { "type": "boolean" } } } } } }, - "responses": { "200": { "description": "{\"ok\": true, \"format\", \"from\", \"to\", \"program\" | \"program_base64\", \"qasm\", \"notes\", \"read\", \"written\", \"check\"}, or the refusal {\"error\", \"code\", \"codes\", \"outcome\", \"meaning\"} with the reader's code (Q700 to Q721) and place" }, "422": { "description": "A program or a check larger than the service takes" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" } } + "responses": { "200": { "description": "{\"ok\": true, \"format\", \"from\", \"to\", \"program\" | \"program_base64\", \"qasm\", \"notes\", \"read\", \"written\", \"check\"}, or the refusal {\"error\", \"code\", \"codes\", \"outcome\", \"meaning\"} with the reader's code (Q700 to Q721) and place" }, "422": { "description": "A program or a check larger than the service takes" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" }, "500": { "description": "Its reading failed inside the service: nothing was answered, and the service runs on" } } } }, "/v1/improve": { "post": { "summary": "The program with its smells fixed, each candidate checked the same circuit", "requestBody": { "required": true, "content": { "application/json": { "schema": prog.clone() } } }, - "responses": { "200": { "description": "{\"base\", \"candidates\": [{\"label\", \"qasm\", \"changes\", \"measures\", \"equivalence\"}]}" }, "422": { "description": "Checks longer than the service takes" }, "429": { "description": "Past the per-client limits" } } + "responses": { "200": { "description": "{\"base\", \"candidates\": [{\"label\", \"qasm\", \"changes\", \"measures\", \"equivalence\"}]}" }, "422": { "description": "Checks longer than the service takes" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" }, "500": { "description": "Its reading failed inside the service: nothing was answered, and the service runs on" } } } }, "/v1/shortcuts": { "post": { "summary": "Each exact classical route to simulating a program, priced on it, and the cheapest for each question", "requestBody": { "required": true, "content": { "application/json": { "schema": prog } } }, - "responses": { "200": { "description": "{\"routes\", \"cheapest\", \"summary\", \"note\"}" }, "422": { "description": "A thorough search, which runs locally" }, "429": { "description": "Past the per-client limits" } } + "responses": { "200": { "description": "{\"routes\", \"cheapest\", \"summary\", \"note\"}" }, "422": { "description": "A thorough search, which runs locally" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" }, "500": { "description": "Its reading failed inside the service: nothing was answered, and the service runs on" } } } }, "/v1/compile": { "post": { "summary": "The program compiled through checked passes, with a certificate of each pass and the distance, sealed with the service's key", @@ -321,7 +321,8 @@ pub fn openapi(server: &str) -> Value { "400": { "description": "A program or options that do not read" }, "422": { "description": "A compile the service does not take (too many gates or rotations, or ε too fine for them: the reason names the local command), or one the engine refuses" }, "429": { "description": "Past the per-client limits" }, - "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" } + "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" }, + "500": { "description": "Its reading failed inside the service: nothing was answered, and the service runs on" } } } }, "/v1/estimate": { "post": { @@ -332,13 +333,14 @@ pub fn openapi(server: &str) -> Value { "400": { "description": "A request, program or document that does not read" }, "422": { "description": "An estimate the service does not take (too many gates or distinct rotation angles, or a budget too fine for its rotations: the reason names the local command), or one the engine refuses (its code, QE02 to QE08)" }, "429": { "description": "Past the per-client limits" }, - "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" } + "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" }, + "500": { "description": "Its reading failed inside the service: nothing was answered, and the service runs on" } } } }, "/v1/repair": { "post": { "summary": "The program repaired on the lines its diagnostics flag alone: the mechanical fixes, a proposed repair held to those lines, and what needs a person", "requestBody": { "required": true, "content": { "application/json": { "schema": { "type": "object", "required": ["qasm"], "properties": { "qasm": { "type": "string" }, "proposed": { "type": "string", "description": "A repaired version: taken only if it changes no line the diagnostics do not flag." }, "max_iterations": { "type": "integer", "minimum": 1 }, "dyadic_bits": { "type": "integer", "minimum": 1, "maximum": 32 }, "sizes": { "type": "object", "additionalProperties": { "type": "integer" } }, "diagnostics": { "type": "array" }, "rules": { "type": "boolean" } } } } } }, - "responses": { "200": { "description": "{\"ok\", \"changed\", \"qasm\", \"repairs\": [{\"code\", \"rule\", \"by\", \"flagged\", \"edits\"}], \"needs_person\", \"verdict\"}, or {\"ok\": false, \"error\", \"refusal\"} for a proposal rejected (J021)" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" } } + "responses": { "200": { "description": "{\"ok\", \"changed\", \"qasm\", \"repairs\": [{\"code\", \"rule\", \"by\", \"flagged\", \"edits\"}], \"needs_person\", \"verdict\"}, or {\"ok\": false, \"error\", \"refusal\"} for a proposal rejected (J021)" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" }, "422": { "description": "Larger than the service takes (J031): its bounds, the budget of reading, the reading child's memory or time, or an answer past what the service takes back" }, "500": { "description": "Its reading failed inside the service: nothing was answered, and the service runs on" } } } }, "/mcp": { "post": { "summary": "The Model Context Protocol endpoint (streamable HTTP, JSON responses): tools to run circuits and sets, plan a job without running it, read jobs, verify receipts, list examples, and critique, compare, improve, price, compile, estimate and repair programs", "responses": { "200": { "description": "A JSON-RPC response" } } } } } @@ -351,7 +353,7 @@ pub fn openapi(server: &str) -> Value { /// The pulse calls' paths. fn pulse_paths() -> Vec<(&'static str, Value)> { - let responses = |ok: &str| json!({ "200": { "description": ok }, "400": { "description": "A body that does not read" }, "422": { "description": "A program the pulse reader refuses (its Q code), a request the engine refuses (J040, J041), or a check larger than the service takes (J031, naming the local command)" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" } }); + let responses = |ok: &str| json!({ "200": { "description": ok }, "400": { "description": "A body that does not read" }, "422": { "description": "A program the pulse reader refuses (its Q code), a request the engine refuses (J040, J041), or a check larger than the service takes (J031, naming the local command)" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" }, "500": { "description": "Its reading failed inside the service: nothing was answered, and the service runs on" } }); vec![ ("/v1/pulse/schedule", json!({ "post": { "summary": "A pulse program's schedule on frames: every play, delay, phase and frequency change, capture and barrier at its sample, each waveform's samples, the gate calls, and the digests a pulse run's receipt binds; sealed with the service's key", @@ -369,7 +371,7 @@ fn pulse_paths() -> Vec<(&'static str, Value)> { /// The variational and algorithm library's paths, apart, so the document's /// macro stays within its depth. fn variational_paths() -> Vec<(&'static str, Value)> { - let responses = |ok: &str| json!({ "200": { "description": ok }, "400": { "description": "A body that does not read" }, "422": { "description": "A request the engine refuses (J050 params, J051 a gradient, J052 an optimisation, J053 a builder, Q129 a parameter it cannot follow) or larger than the service takes (J031, naming the local command)" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" } }); + let responses = |ok: &str| json!({ "200": { "description": ok }, "400": { "description": "A body that does not read" }, "422": { "description": "A request the engine refuses (J050 params, J051 a gradient, J052 an optimisation, J053 a builder, Q129 a parameter it cannot follow) or larger than the service takes (J031, naming the local command)" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" }, "500": { "description": "Its reading failed inside the service: nothing was answered, and the service runs on" } }); let body = || json!({ "required": true, "content": { "application/json": { "schema": { "type": "object" } } } }); vec![ ("/v1/gradient", json!({ "post": { @@ -403,7 +405,7 @@ fn variational_paths() -> Vec<(&'static str, Value)> { /// The error-correction calls' paths, apart, so the document's macro stays /// within its depth. fn qec_paths() -> Vec<(&'static str, Value)> { - let responses = |ok: &str| json!({ "200": { "description": ok }, "400": { "description": "A body that does not read" }, "422": { "description": "A request the engine refuses (J009) or larger than the service takes (J031, naming the local command)" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" } }); + let responses = |ok: &str| json!({ "200": { "description": ok }, "400": { "description": "A body that does not read" }, "422": { "description": "A request the engine refuses (J009) or larger than the service takes (J031, naming the local command)" }, "429": { "description": "Past the per-client limits" }, "503": { "description": "Every check is running, or your share of them (J032): nothing was read, and nothing counted" }, "500": { "description": "Its reading failed inside the service: nothing was answered, and the service runs on" } }); vec![ ("/v1/qec/code", json!({ "post": { "summary": "An error-correcting code (surface:D, toric:L, steane, color:D, bb:72 to bb:288, gross): its checks, stabilizers, k, layout, the decoders it takes, and its distance certificate (a witness and every lighter support ruled out); with a certificate, its check; sealed with the service's key", diff --git a/wai/wai-quantum-service/src/http.rs b/wai/wai-quantum-service/src/http.rs index 80cf74860..cacf2786e 100644 --- a/wai/wai-quantum-service/src/http.rs +++ b/wai/wai-quantum-service/src/http.rs @@ -141,6 +141,14 @@ fn reply(status: u16, body: Value) -> Response { r } +/// [`reply`], its body JSON text already written. +fn reply_text(status: u16, body: String) -> Response { + let mut r = (StatusCode::from_u16(status).unwrap_or(StatusCode::INTERNAL_SERVER_ERROR), [("content-type", "application/json")], body).into_response(); + r.headers_mut().insert("access-control-allow-origin", HeaderValue::from_static("*")); + r.headers_mut().insert("access-control-expose-headers", HeaderValue::from_static("ratelimit, ratelimit-policy, retry-after")); + r +} + /// A refusal: its reason, and the codes it ends with (`engine::refusal`). fn error(status: u16, why: &str) -> Response { let r = wai_quantum::engine::refusal::refusal_json(why); @@ -551,12 +559,13 @@ fn dry_run_now(service: &Service, who: &str, (_, planned): Admitted) -> Result, ConnectInfo(peer): ConnectInfo, headers: HeaderMap, body: Result, axum::extract::rejection::JsonRejection>) -> Response { - let Json(request) = match body { - Ok(b) => b, - Err(e) => return error(400, &format!("the body is not a JSON job: {e}")), - }; +async fn submit(State(app): State, ConnectInfo(peer): ConnectInfo, headers: HeaderMap, body: axum::body::Bytes) -> Response { let who = client(&app, peer, &headers); + // A resend under its key is answered before any limit, so read first. + let request = match json_body(&app, &who, &body, headers.contains_key("idempotency-key"), "the body is not a JSON job") { + Ok(v) => v, + Err((code, why)) => return error(code, &why), + }; if is_dry_run(&request) { return match dry_run(&app, &who, &request).await { Ok(plan) => reply(200, plan), @@ -568,17 +577,18 @@ async fn submit(State(app): State, ConnectInfo(peer): ConnectInfo return error(code, &why), }; match admit(&app, &who, request, key).await { - Ok(id) => reply(202, app.service.get(&id).unwrap_or(json!({ "id": id }))), + Ok(id) => reply_text(202, app.service.report(&id).unwrap_or_else(|| json!({ "id": id }).to_string())), Err((code, why)) => error(code, &why), } } -async fn run(State(app): State, ConnectInfo(peer): ConnectInfo, headers: HeaderMap, body: Result, axum::extract::rejection::JsonRejection>) -> Response { - let Json(request) = match body { - Ok(b) => b, - Err(e) => return error(400, &format!("the body is not a JSON job: {e}")), - }; +async fn run(State(app): State, ConnectInfo(peer): ConnectInfo, headers: HeaderMap, body: axum::body::Bytes) -> Response { let who = client(&app, peer, &headers); + // A resend under its key is answered before any limit, so read first. + let request = match json_body(&app, &who, &body, headers.contains_key("idempotency-key"), "the body is not a JSON job") { + Ok(v) => v, + Err((code, why)) => return error(code, &why), + }; if is_dry_run(&request) { return match dry_run(&app, &who, &request).await { Ok(plan) => reply(200, plan), @@ -593,14 +603,33 @@ async fn run(State(app): State, ConnectInfo(peer): ConnectInfo id, Err((code, why)) => return error(code, &why), }; - let status = app.service.wait(&id, RUN_WAIT).await; - let body = app.service.get(&id).unwrap_or(json!({ "id": id })); + // Within the answer's time, whatever admission took. + let status = app.service.wait(&id, crate::offload::left().map_or(RUN_WAIT, |l| l.min(RUN_WAIT))).await; + let body = app.service.report(&id).unwrap_or_else(|| json!({ "id": id }).to_string()); match status { - Some(s) if !s.active() => reply(200, body), - _ => reply(202, body), + Some(s) if !s.active() => reply_text(200, body), + _ => reply_text(202, body), } } +/// A request or an answer past this many bytes has the memory it was parsed +/// into given back to the system once it is answered ([`crate::jobs::give_back`]). +const LARGE: usize = 256 << 10; + +/// The largest MCP message read from a client past its window. +const MCP_UNREAD: usize = 64 << 10; + +/// A request's body read as JSON, once its client's window has room, so that +/// a request refused for its window is never parsed: a parsed body takes many +/// times its length. With `first` (a resend under its key, answered before any +/// limit) it is read whatever the window says. +fn json_body(app: &App, who: &str, body: &[u8], first: bool, bad: &str) -> Result { + if !first && let Err(Refused(code, why)) = app.service.window_open(who) { + return Err((code, why)); + } + serde_json::from_slice(body).map_err(|e| (400, format!("{bad}: {e}"))) +} + /// The longest `GET /v1/jobs/{id}?wait=S` holds its answer. pub const POLL_WAIT: Duration = Duration::from_secs(25); @@ -625,8 +654,8 @@ async fn job(State(app): State, Path(id): Path, Query(poll): Que } } } - match app.service.get(&id) { - Some(j) => reply(200, j), + match app.service.report(&id) { + Some(j) => reply_text(200, j), None => error(404, &format!("no such job (finished jobs are kept for {} s)", app.service.config.retain.as_secs())), } } @@ -736,9 +765,12 @@ async fn verify(State(app): State, ConnectInfo(peer): ConnectInfo, axum::extract::rejection::JsonRejection>, check: crate::analysis::Check) -> Response { - let Ok(Json(v)) = body else { return error(400, "the body is not JSON") }; +async fn analysis(app: Shared, peer: SocketAddr, headers: HeaderMap, body: axum::body::Bytes, check: crate::analysis::Check) -> Response { let who = client(&app, peer, &headers); + let v = match json_body(&app, &who, &body, false, "the body is not JSON") { + Ok(v) => v, + Err((code, why)) => return error(code, &why), + }; match crate::analysis::analyse(&app, &who, check, v).await { Ok(answer) => reply(200, answer), Err((code, why)) => error(code, &why), @@ -797,18 +829,24 @@ pub(crate) async fn pulse(app: &App, who: &str, op: &'static str, request: Value Ok(json!({ "request": request, "result": result })) } -async fn pulse_route(app: Shared, peer: SocketAddr, headers: HeaderMap, body: Result, axum::extract::rejection::JsonRejection>, op: &'static str) -> Response { - let Ok(Json(v)) = body else { return error(400, "the body is not JSON") }; +async fn pulse_route(app: Shared, peer: SocketAddr, headers: HeaderMap, body: axum::body::Bytes, op: &'static str) -> Response { let who = client(&app, peer, &headers); + let v = match json_body(&app, &who, &body, false, "the body is not JSON") { + Ok(v) => v, + Err((code, why)) => return error(code, &why), + }; match pulse(&app, &who, op, v).await { Ok(sealed) => reply(200, sealed), Err((code, why)) => error(code, &why), } } -async fn qec_route(app: Shared, peer: SocketAddr, headers: HeaderMap, body: Result, axum::extract::rejection::JsonRejection>, op: &'static str) -> Response { - let Ok(Json(v)) = body else { return error(400, "the body is not JSON") }; +async fn qec_route(app: Shared, peer: SocketAddr, headers: HeaderMap, body: axum::body::Bytes, op: &'static str) -> Response { let who = client(&app, peer, &headers); + let v = match json_body(&app, &who, &body, false, "the body is not JSON") { + Ok(v) => v, + Err((code, why)) => return error(code, &why), + }; match qec(&app, &who, op, v).await { Ok(sealed) => reply(200, sealed), Err((code, why)) => error(code, &why), @@ -828,27 +866,36 @@ pub(crate) async fn variational(app: &App, who: &str, op: &'static str, request: Ok(json!({ "request": request, "result": result })) } -async fn variational_route(app: Shared, peer: SocketAddr, headers: HeaderMap, body: Result, axum::extract::rejection::JsonRejection>, op: &'static str) -> Response { - let Ok(Json(v)) = body else { return error(400, "the body is not JSON") }; +async fn variational_route(app: Shared, peer: SocketAddr, headers: HeaderMap, body: axum::body::Bytes, op: &'static str) -> Response { let who = client(&app, peer, &headers); + let v = match json_body(&app, &who, &body, false, "the body is not JSON") { + Ok(v) => v, + Err((code, why)) => return error(code, &why), + }; match variational(&app, &who, op, v).await { Ok(sealed) => reply(200, sealed), Err((code, why)) => error(code, &why), } } -async fn estimate_route(State(app): State, ConnectInfo(peer): ConnectInfo, headers: HeaderMap, body: Result, axum::extract::rejection::JsonRejection>) -> Response { - let Ok(Json(v)) = body else { return error(400, "the body is not JSON") }; +async fn estimate_route(State(app): State, ConnectInfo(peer): ConnectInfo, headers: HeaderMap, body: axum::body::Bytes) -> Response { let who = client(&app, peer, &headers); + let v = match json_body(&app, &who, &body, false, "the body is not JSON") { + Ok(v) => v, + Err((code, why)) => return error(code, &why), + }; match estimate(&app, &who, v).await { Ok(sealed) => reply(200, sealed), Err((code, why)) => error(code, &why), } } -async fn compile_route(State(app): State, ConnectInfo(peer): ConnectInfo, headers: HeaderMap, body: Result, axum::extract::rejection::JsonRejection>) -> Response { - let Ok(Json(v)) = body else { return error(400, "the body is not JSON") }; +async fn compile_route(State(app): State, ConnectInfo(peer): ConnectInfo, headers: HeaderMap, body: axum::body::Bytes) -> Response { let who = client(&app, peer, &headers); + let v = match json_body(&app, &who, &body, false, "the body is not JSON") { + Ok(v) => v, + Err((code, why)) => return error(code, &why), + }; match compile(&app, &who, v).await { Ok(sealed) => reply(200, sealed), Err((code, why)) => error(code, &why), @@ -886,14 +933,22 @@ async fn index(State(app): State, headers: HeaderMap) -> Response { /// (`wq-views-…`, random, held nowhere), and every request that carries it is /// answered with the views (see `mcp`); any other client gets no session id /// and no views. -async fn mcp(State(app): State, ConnectInfo(peer): ConnectInfo, headers: HeaderMap, body: Result, axum::extract::rejection::JsonRejection>) -> Response { - let Ok(Json(msg)) = body else { +async fn mcp(State(app): State, ConnectInfo(peer): ConnectInfo, headers: HeaderMap, body: axum::body::Bytes) -> Response { + let who = client(&app, peer, &headers); + // A large message from a client past its window can only be a call it + // would be refused: refused unread (a parsed message takes many times its + // length). A small one is read, for what costs nothing (a list, a poll). + if body.len() > MCP_UNREAD + && let Err(Refused(code, why)) = app.service.window_open(&who) + { + return reply(code, json!({ "jsonrpc": "2.0", "id": null, "error": { "code": -32000, "message": why } })); + } + let Ok(msg) = serde_json::from_slice::(&body) else { return reply(400, json!({ "jsonrpc": "2.0", "id": null, "error": { "code": -32700, "message": "not JSON" } })); }; if !msg.is_object() { return reply(400, json!({ "jsonrpc": "2.0", "id": null, "error": { "code": -32600, "message": "one JSON-RPC message a request" } })); } - let who = client(&app, peer, &headers); let opening = msg["method"] == "initialize" && wai_quantum::engine::views::negotiated(&msg["params"]); let views = opening || headers.get("mcp-session-id").and_then(|v| v.to_str().ok()).is_some_and(crate::mcp::is_views_session); // tasks/result waits until its task ends: to a client that takes an @@ -1001,10 +1056,21 @@ fn rate_headers(app: &App, who: &str, r: &mut Response) { /// A request with its body read, within `wait`: a client that stops sending /// holds its connection no longer than that (`408`). Only then is it handled. -async fn body_in_time(State(wait): State, request: axum::extract::Request, next: axum::middleware::Next) -> Response { +/// The request is answered within `answer` of its arrival +/// ([`crate::offload::answering_by`]). +async fn body_in_time(State((wait, answer)): State<(Duration, Duration)>, request: axum::extract::Request, next: axum::middleware::Next) -> Response { + let due = std::time::Instant::now() + answer; let (parts, body) = request.into_parts(); match tokio::time::timeout(wait, axum::body::to_bytes(body, MAX_BODY + 1)).await { - Ok(Ok(bytes)) => next.run(axum::extract::Request::from_parts(parts, axum::body::Body::from(bytes))).await, + Ok(Ok(bytes)) => { + let large = bytes.len() > LARGE; + let r = crate::offload::answering_by(due, next.run(axum::extract::Request::from_parts(parts, axum::body::Body::from(bytes)))).await; + // What a large request or answer was parsed into, given back. + if large || axum::body::HttpBody::size_hint(r.body()).exact().is_some_and(|n| n > LARGE as u64) { + crate::jobs::give_back(); + } + r + } Ok(Err(_)) => error(413, &format!("the body is larger than the {MAX_BODY} bytes a request may be, or it broke off")), Err(_) => error(408, &format!("the body did not arrive within {} s", wait.as_secs())), } @@ -1073,7 +1139,7 @@ pub fn router(app: Shared) -> Router { .route("/mcp", post(mcp).get(mcp_get)) .route("/", get(index)) .fallback(|| async { error(404, "no such endpoint; GET /v1 lists them") }) - .layer(axum::middleware::from_fn_with_state(app.service.config.body_wait, body_in_time)) + .layer(axum::middleware::from_fn_with_state((app.service.config.body_wait, Duration::from_secs(app.service.config.answer_secs)), body_in_time)) .layer(axum::middleware::from_fn_with_state(Arc::clone(&app), guard)) .layer(DefaultBodyLimit::max(MAX_BODY)) .with_state(app) @@ -1106,6 +1172,9 @@ mod tests { read_timeout: Duration::from_secs(300), read_answer_mib: 128, qec_secs: 25, + answer_secs: 55, + client_kept_mib: 128, + kept_mib: 2048, read_in_child: false, per_client_reads: 2, per_client_checks: 1, diff --git a/wai/wai-quantum-service/src/jobs.rs b/wai/wai-quantum-service/src/jobs.rs index 5b64602e7..efd00ff0e 100644 --- a/wai/wai-quantum-service/src/jobs.rs +++ b/wai/wai-quantum-service/src/jobs.rs @@ -103,6 +103,14 @@ pub struct Config { /// once read (each value and each string): a sweep's items, written out, /// come into the service's own memory. pub read_answer_mib: u64, + /// What ended jobs may keep in memory, MiB: one client's, and all of + /// them; past either, its oldest ended jobs are dropped before their time. + pub client_kept_mib: u64, + pub kept_mib: u64, + /// How long the service takes to answer a request at most, seconds: every + /// wait for a turn, every reading and every wait for a job within it end + /// by then, so an answer comes within a proxy's wait for one. + pub answer_secs: u64, /// The seconds an error-correction call's sampled shots may take, measured /// as they run (`wai_quantum::engine::qec::within_time`). pub qec_secs: u64, @@ -136,6 +144,9 @@ impl Config { /// what an error-correction call's shots may take, /// `WAI_QSVC_CLIENT_READS` (2) and `WAI_QSVC_CLIENT_CHECKS` (1), how many /// of a client's requests are read at once in each pool, + /// `WAI_QSVC_ANSWER_SECS` (55), the longest any answer takes, + /// `WAI_QSVC_CLIENT_KEPT_MIB` (128) and `WAI_QSVC_KEPT_MIB` (2048), what + /// ended jobs may keep in memory, one client's and all, /// `WAI_QSVC_HEADER_SECS` (150) and `WAI_QSVC_BODY_SECS` (30), how long a /// client may take to send a request (the first also bounds an idle /// connection kept open between requests). @@ -158,6 +169,9 @@ impl Config { read_timeout: Duration::from_secs(env_num("WAI_QSVC_READ_SECS", 45u64)), read_answer_mib: env_num("WAI_QSVC_READ_ANSWER_MIB", 128u64), qec_secs: env_num("WAI_QSVC_QEC_SECS", 25u64).max(1), + answer_secs: env_num("WAI_QSVC_ANSWER_SECS", 55u64).max(1), + client_kept_mib: env_num("WAI_QSVC_CLIENT_KEPT_MIB", 128u64), + kept_mib: env_num("WAI_QSVC_KEPT_MIB", 2048u64), per_client_reads: env_num("WAI_QSVC_CLIENT_READS", 2usize).max(1), per_client_checks: env_num("WAI_QSVC_CLIENT_CHECKS", 1usize).max(1), header_wait: Duration::from_secs(env_num("WAI_QSVC_HEADER_SECS", 150u64).max(1)), @@ -247,7 +261,10 @@ pub struct Job { pub id: String, /// The client's name as the store keeps it ([`store::pseudonym`]). pub client: String, - pub request: Value, + /// The request as its canonical text, the bytes the store keeps and the + /// receipt seals: a parsed request takes many times its text, and a job is + /// kept for a day, so it is read only where it is used ([`Job::request`]). + pub request: std::sync::Arc, pub plan: Plan, pub work: Work, pub status: Status, @@ -269,6 +286,10 @@ pub struct Job { /// 1970, each time it did. pub interruptions: Vec, child: Option, + /// What the job keeps in memory once it has ended, bytes: its request's + /// text and its result as read ([`crate::offload::weight`]), counted + /// against its client's and the service's caps on what ended jobs keep. + kept: u64, } tokio::task_local! { @@ -330,6 +351,28 @@ pub fn self_exe() -> Result { } } +/// Gives the memory freed since back to the system: a parsed request of a +/// megabyte takes tens of megabytes, and an allocator keeps what it freed, +/// which on macOS stays charged to the service's footprint. Called after a +/// large request or answer. +pub fn give_back() { + #[cfg(all(target_os = "linux", target_env = "gnu"))] + // SAFETY: a plain call into the C allocator. + unsafe { + libc::malloc_trim(0); + } + #[cfg(target_os = "macos")] + { + unsafe extern "C" { + fn malloc_zone_pressure_relief(zone: *mut std::ffi::c_void, goal: usize) -> usize; + } + // SAFETY: every zone (a null zone), as much as can be given back. + unsafe { + malloc_zone_pressure_relief(std::ptr::null_mut(), 0); + } + } +} + /// Makes this process the one a host short of memory ends first (Linux: /// `oom_score_adj` 1000), so that a reader or a worker that grows between two /// polls of its memory is ended before the service is. @@ -380,17 +423,44 @@ fn now_ms() -> u64 { pub const RECORD: &str = "wai.quantum.job-record/1"; impl Job { + /// The job has ended: what it needed only to run is let go (a set's + /// items, each a copy of its fields), and what it keeps is counted. + fn ended(&mut self) { + if let Work::Set { items, .. } = &mut self.work { + *items = Vec::new(); + } + self.kept = self.request.len() as u64 + self.result.as_ref().map_or(0, crate::offload::weight) + self.error.as_ref().map_or(0, |e| e.len() as u64); + } + + /// The request, read from its text. + pub fn request(&self) -> Value { + serde_json::from_str(&self.request).unwrap_or(Value::Null) + } + + /// The job as the API reports it, but its request. + fn to_json_without_request(&self) -> Value { + let mut o = self.to_json_with(Value::Null); + if let Some(m) = o.as_object_mut() { + m.shift_remove("request"); + } + o + } + /// The job as the API reports it. pub fn to_json(&self) -> Value { + self.to_json_with(self.request()) + } + + fn to_json_with(&self, request: Value) -> Value { let mut o = json!({ "id": self.id, "status": self.status.name(), "method": self.plan.method, "why": self.plan.why, "estimate": { "memory_bytes": self.plan.memory_bytes, "work": self.plan.work }, - // The request as submitted, so whoever holds the id can check the + // The request as sealed, so whoever holds the id can check the // result's receipt against it. - "request": self.request, + "request": request, "submitted_ms": self.submitted_ms, }); if let Some(t) = self.started_ms { @@ -680,6 +750,8 @@ impl Service { queued.sort(); s.queue.get_mut().unwrap().extend(queued.into_iter().map(|(_, id)| id)); let s = Arc::new(s); + // What ended jobs keep, held to its caps as they were on starting. + s.trim(); s.sweep(); Ok(s) } @@ -694,17 +766,23 @@ impl Service { self.store.get(d)?.ok_or_else(|| format!("the object {d} is not in the store")) }; let json_of = |d: &Value| -> Result { serde_json::from_slice(&text(d)?).map_err(|e| format!("an object does not read: {e}")) }; - let request = json_of(&rec["request"])?; + let request_text = String::from_utf8(text(&rec["request"])?).map_err(|e| format!("its request is not text: {e}"))?; + // Read only where it is needed (a set from before sets kept their + // items): a parsed request takes many times its text. + let request = || serde_json::from_str::(&request_text).map_err(|e| format!("its request does not read: {e}")); let plan: Plan = serde_json::from_value(rec["plan"].clone()).map_err(|e| format!("its plan does not read: {e}"))?; let status = rec["status"].as_str().and_then(Status::from_name).ok_or("its status does not read")?; let work = match rec.get("set_plans") { Some(p) => { let plans: Vec = serde_json::from_value(p.clone()).map_err(|e| format!("its set's plans do not read: {e}"))?; let items = match rec.get("set_items") { - Some(d) => items_of(&json_of(d)?), + Some(d) if status.active() => items_of(&json_of(d)?), // Kept before sets kept their items; what has ended needs none. - None if status.active() => wai_quantum::qasm::within_steps(crate::http::READ_STEPS, || sets::expand(&request))?, - None => Vec::new(), + None if status.active() => { + let request = request()?; + wai_quantum::qasm::within_steps(crate::http::READ_STEPS, || sets::expand(&request))? + } + _ => Vec::new(), }; if status.active() && items.len() != plans.len() { return Err("its set's plans do not match its items".into()); @@ -722,10 +800,10 @@ impl Service { result = Some(r); } let s = |k: &str| rec[k].as_str().map(str::to_string); - Ok(Job { + let mut job = Job { id: s("id").ok_or("no id")?, client: s("client").unwrap_or_default(), - request, + request: request_text.into(), plan, work, status, @@ -746,7 +824,12 @@ impl Service { received: s("received"), interruptions: rec["interruptions"].as_array().into_iter().flatten().filter_map(Value::as_u64).collect(), child: None, - }) + kept: 0, + }; + if job.finished_ms.is_some() { + job.ended(); + } + Ok(job) } /// The store, for the service's description. @@ -961,7 +1044,7 @@ impl Service { let job = Job { id: id.clone(), client: name, - request, + request: String::from_utf8_lossy(&canonical).into(), plan, work, status: Status::Queued, @@ -976,6 +1059,7 @@ impl Service { received, interruptions: Vec::new(), child: None, + kept: 0, }; { let mut queue = self.queue.lock().unwrap(); @@ -1000,6 +1084,20 @@ impl Service { self.jobs.lock().unwrap().get(id).map(Job::to_json) } + /// The job's report as JSON text, its request written in as the text it + /// is kept as, never read: what the HTTP routes answer with, however + /// often a client polls a job whose request is large. + pub fn report(&self, id: &str) -> Option { + let (lean, request) = { + let jobs = self.jobs.lock().unwrap(); + let j = jobs.get(id)?; + (j.to_json_without_request(), std::sync::Arc::clone(&j.request)) + }; + let text = lean.to_string(); + let open = text.strip_suffix('}')?; + Some(format!("{open}{}\"request\":{request}}}", if open.ends_with('{') { "" } else { "," })) + } + /// Whether the job `id` is `client`'s. pub fn owned_by(&self, id: &str, client: &str) -> bool { let name = self.name(client); @@ -1217,7 +1315,7 @@ impl Service { j.started_ms = Some(now_ms()); let _ = self.keep(j); let input = match &j.work { - Work::One => json!({ "request": j.request, "plan": j.plan }), + Work::One => json!({ "request": j.request(), "plan": j.plan }), Work::Set { items, plans } => json!({ "set": items.iter().zip(plans).map(|(i, p)| json!({ "request": i.request, "plan": p })).collect::>() }), }; Some((input, matches!(j.plan.method.as_str(), "statevector_single" | "cutrank_single") || matches!(&j.work, Work::Set { plans, .. } if plans.iter().any(|p| matches!(p.method.as_str(), "statevector_single" | "cutrank_single"))), others)) @@ -1243,7 +1341,7 @@ impl Service { /// the store's writes are done outside the lock on the jobs. fn finish(&self, id: &str, ran: Ran) { let Ran { outcome, peak, energy: figure, elapsed_ms } = ran; - let (request, work) = { + let (request, work, request_len) = { let mut jobs = self.jobs.lock().unwrap(); let Some(j) = jobs.get_mut(id) else { return }; j.child = None; @@ -1252,7 +1350,7 @@ impl Service { let _ = self.keep(j); return; } - (j.request.clone(), j.work.clone()) + (j.request(), j.work.clone(), j.request.len()) }; let finished_ms = now_ms(); let mut kept: (Option, Option) = (None, None); @@ -1312,8 +1410,53 @@ impl Service { j.error = Some(e); } } + j.ended(); } self.changed(id); + self.trim(); + // A large request was read to run and to seal it: given back. + if request_len > 256 << 10 { + give_back(); + } + } + + /// Ended jobs past what they may keep, dropped, the oldest first: each + /// client's past [`Config::client_kept_mib`], then all past + /// [`Config::kept_mib`]. A job is kept for [`Config::retain`], or until + /// these say otherwise. + fn trim(&self) { + let gone: Vec = { + let mut jobs = self.jobs.lock().unwrap(); + let mut ended: Vec<(u64, String, String, u64)> = jobs.values().filter(|j| j.finished_ms.is_some()).map(|j| (j.finished_ms.unwrap_or(0), j.id.clone(), j.client.clone(), j.kept)).collect(); + ended.sort(); + let mut by_client: HashMap = HashMap::new(); + let mut total = 0u64; + for (_, _, c, k) in &ended { + *by_client.entry(c.clone()).or_default() += k; + total += k; + } + let (client_cap, cap) = (self.config.client_kept_mib << 20, self.config.kept_mib << 20); + let mut gone = Vec::new(); + for (_, id, c, k) in &ended { + let mine = by_client.get(c).copied().unwrap_or(0); + if mine > client_cap || total > cap { + by_client.insert(c.clone(), mine - k); + total -= k; + gone.push(id.clone()); + } + } + for id in &gone { + jobs.remove(id); + } + gone + }; + for id in &gone { + let _ = self.store.remove_record(id); + } + if !gone.is_empty() { + self.keys.lock().unwrap().retain(|_, id| !gone.contains(id)); + self.sweep(); + } } /// A finished worker's output sealed: the result without its receipt, @@ -1373,6 +1516,10 @@ impl Service { j.child = Some(pid); } let text = input.to_string(); + drop(input); + if text.len() > 256 << 10 { + give_back(); + } if let Some(mut stdin) = child.stdin.take() { let _ = stdin.write_all(text.as_bytes()).await; } @@ -1418,14 +1565,12 @@ impl Service { if let Some(rss) = rss_bytes(pid) { peak = peak.max(rss); if rss > limit { - kill(pid); - let _ = child.wait().await; + stop(&mut child, pid).await; break Err(format!("the job used {} MiB, over the {} MiB it was admitted with; it was stopped", rss >> 20, mib)); } } if Instant::now() > deadline { - kill(pid); - let _ = child.wait().await; + stop(&mut child, pid).await; break Err(format!("the job ran past {} s; it was stopped", self.config.job_timeout.as_secs())); } if exited.is_none() { @@ -1495,6 +1640,9 @@ pub(crate) fn sandboxed(exe: &std::path::Path, sandbox: bool, mode: &str) -> tok tokio::process::Command::new(exe) }; cmd.arg(mode); + // A group of its own, so that killing it kills what it started too. + #[cfg(unix)] + cmd.process_group(0); cmd } @@ -1517,6 +1665,8 @@ fn low_priority(exe: &std::path::Path, sandbox: bool) -> tokio::process::Command c }; cmd.arg("worker"); + #[cfg(unix)] + cmd.process_group(0); cmd } @@ -1534,9 +1684,38 @@ pub fn worker_profile(exe: &std::path::Path) -> String { ) } -pub(crate) fn kill(pid: u32) { - if pid > 0 { - let _ = std::process::Command::new("kill").args(["-9", &pid.to_string()]).status(); +/// Kills a reader or a worker, and whatever it started: a signal sent by the +/// service itself, which needs no process of its own, so it works when no new +/// process can be started. The child leads a process group of its own +/// ([`sandboxed`]), so the group is sent the signal, then the child: whether +/// either was sent (or the child is already gone). +pub(crate) fn kill(pid: u32) -> bool { + if pid == 0 { + return false; + } + #[cfg(unix)] + { + let Ok(pid) = i32::try_from(pid) else { return false }; + // SAFETY: plain signals to a child of this process and its group. + let group = unsafe { libc::kill(-pid, libc::SIGKILL) } == 0; + let one = unsafe { libc::kill(pid, libc::SIGKILL) } == 0; + group || one || std::io::Error::last_os_error().raw_os_error() == Some(libc::ESRCH) + } + #[cfg(not(unix))] + { + std::process::Command::new("kill").args(["-9", &pid.to_string()]).status().is_ok_and(|s| s.success()) + } +} + +/// Stops `child`: killed, and killed again every 200 ms until it has ended, +/// so a signal that did not take is sent again rather than waited out. +pub(crate) async fn stop(child: &mut tokio::process::Child, pid: u32) { + loop { + kill(pid); + let _ = child.start_kill(); + if let Ok(Ok(_)) = tokio::time::timeout(Duration::from_millis(200), child.wait()).await { + return; + } } } @@ -1633,6 +1812,9 @@ mod tests { read_timeout: Duration::from_secs(300), read_answer_mib: 128, qec_secs: 25, + answer_secs: 55, + client_kept_mib: 128, + kept_mib: 2048, read_in_child: false, per_client_reads: 2, per_client_checks: 1, @@ -1641,6 +1823,39 @@ mod tests { } } + /// What ended jobs keep is their request's text, not its parsed tree, and + /// it is held to its caps: past a client's, its oldest ended jobs go first, + /// and past the service's, the oldest of any; jobs still running are kept. + #[test] + fn ended_jobs_keep_their_request_as_text_within_their_caps() { + let s = Service::new(Config { client_kept_mib: 1, kept_mib: 2, ..config() }); + let pad = |k: usize| serde_json::json!({ "qasm": "OPENQASM 3.0; qubit q; h q;", "pad": vec![k; 300_000] }); + let job = |client: &str, k: usize, ended: Option| { + let request = pad(k); + let text = String::from_utf8(canon::canonical(&request)).unwrap(); + let mut j = Job { id: job_id(), client: s.name(client), request: text.into(), plan: plan(1 << 20), work: Work::One, status: if ended.is_some() { Status::Done } else { Status::Running }, submitted_ms: 0, started_ms: None, finished_ms: ended, result: Some(serde_json::json!({ "counts": { "0": 1 } })), error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0 }; + if ended.is_some() { + j.ended(); + } + assert!(j.kept < 700_000, "kept as text: {}", j.kept); + assert_eq!(j.request()["pad"].as_array().map(Vec::len), Some(300_000)); + j + }; + let mut ids = Vec::new(); + for (client, k, ended) in [("a", 1, Some(1)), ("a", 2, Some(2)), ("a", 3, Some(3)), ("a", 4, None), ("b", 5, Some(4)), ("c", 6, Some(5)), ("d", 7, Some(6))] { + let j = job(client, k, ended); + ids.push(j.id.clone()); + s.jobs.lock().unwrap().insert(j.id.clone(), j); + } + s.trim(); + let have: Vec = ids.iter().map(|id| s.get(id).is_some()).collect(); + // a's two oldest ended jobs go for its client's cap, then the oldest + // of all for the service's; the running one stays. + assert_eq!(have, [false, false, false, true, true, true, true], "{have:?}"); + let kept: u64 = s.jobs.lock().unwrap().values().filter(|j| j.finished_ms.is_some()).map(|j| j.kept).sum(); + assert!(kept <= 2 << 20, "{kept}"); + } + fn ids_of(s: &Service, client: &str) -> Vec { let name = s.name(client); s.jobs.lock().unwrap().values().filter(|j| j.client == name).map(|j| j.id.clone()).collect() @@ -1728,6 +1943,7 @@ mod tests { fn the_defaults_fit_inside_the_proxy_s() { let c = Config::from_env(); assert!(c.read_timeout + crate::offload::WAIT < Duration::from_secs(60), "{:?}", c.read_timeout); + assert!(c.answer_secs < 60 && c.read_timeout + crate::offload::WAIT <= Duration::from_secs(c.answer_secs), "{}", c.answer_secs); assert!(c.header_wait > Duration::from_secs(120), "{:?}", c.header_wait); assert!((c.qec_secs as f64) < 0.6 * c.read_timeout.as_secs_f64(), "{}", c.qec_secs); assert!(c.per_client_reads < crate::offload::AT_ONCE && c.per_client_checks < crate::offload::CHECKS_AT_ONCE); @@ -1962,7 +2178,7 @@ mod tests { let key = wai_quantum::ed25519_dalek::SigningKey::from_bytes(&[19; 32]); let request = json!({ "qasm": "OPENQASM 3.0; qubit[2] q; h q[0]; cx q[0], q[1];", "shots": 20, "observables": ["Z0 Z1"] }); let (p, mut result) = wai_quantum::engine::job::plan_and_run(&request, 1 << 30).unwrap(); - let mut job = Job { id: job_id(), client: "test".into(), request: request.clone(), plan: p, work: Work::One, status: Status::Queued, submitted_ms: now_ms(), started_ms: None, finished_ms: None, result: None, error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None }; + let mut job = Job { id: job_id(), client: "test".into(), request: request.to_string().into(), plan: p, work: Work::One, status: Status::Queued, submitted_ms: now_ms(), started_ms: None, finished_ms: None, result: None, error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0 }; let r = crate::mcp::schemas(); let check = |j: &Job| { let v = j.to_json(); @@ -1992,7 +2208,7 @@ mod tests { let items = wai_quantum::engine::sets::expand(&set_request).unwrap(); let plans = wai_quantum::engine::sets::plan(&items, 1 << 30).unwrap(); let body = wai_quantum::engine::sets::run(&set_request, 1 << 30, &key, json!({ "unmetered": "a test" })).unwrap(); - let set = Job { id: job_id(), client: "test".into(), request: set_request, plan: wai_quantum::engine::sets::summary(&plans), work: Work::Set { items, plans }, status: Status::Done, submitted_ms: now_ms(), started_ms: Some(now_ms()), finished_ms: Some(now_ms()), result: Some(body["result"].clone()), error: None, peak_rss: 1 << 20, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None }; + let set = Job { id: job_id(), client: "test".into(), request: set_request.to_string().into(), plan: wai_quantum::engine::sets::summary(&plans), work: Work::Set { items, plans }, status: Status::Done, submitted_ms: now_ms(), started_ms: Some(now_ms()), finished_ms: Some(now_ms()), result: Some(body["result"].clone()), error: None, peak_rss: 1 << 20, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0 }; let done = check(&set); assert_eq!(wai_quantum::engine::sets::verify(&done)["valid"], true, "{done}"); } @@ -2101,7 +2317,7 @@ mod tests { let s = Service::new(config()); let p = Plan { method: "mps".into(), why: "".into(), memory_bytes: 1, work: 1_000_000_000, qubits: 30, target: None, metered: true }; for _ in 0..3 { - let job = Job { id: job_id(), client: "x".into(), request: json!({}), plan: p.clone(), work: Work::One, status: Status::Done, submitted_ms: 0, started_ms: None, finished_ms: Some(1), result: Some(json!({ "elapsed_ms": 500, "energy": { "joules_micro": 5_000_000, "energy_uncertainty": { "window_us": 500_000 } } })), error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None }; + let job = Job { id: job_id(), client: "x".into(), request: "{}".into(), plan: p.clone(), work: Work::One, status: Status::Done, submitted_ms: 0, started_ms: None, finished_ms: Some(1), result: Some(json!({ "elapsed_ms": 500, "energy": { "joules_micro": 5_000_000, "energy_uncertainty": { "window_us": 500_000 } } })), error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0 }; s.time(&job); } let (t, _) = s.estimate(&p); @@ -2120,7 +2336,7 @@ mod tests { assert!(t["seconds"].is_null() && t["at_most"] == 5, "{t}"); // Timed jobs here take over, and their metered power gives an energy estimate. for _ in 0..3 { - let job = Job { id: job_id(), client: "x".into(), request: json!({}), plan: Plan { work: 1_000_000_000, ..p.clone() }, work: Work::One, status: Status::Done, submitted_ms: 0, started_ms: None, finished_ms: Some(1), result: Some(json!({ "elapsed_ms": 500, "energy": { "joules_micro": 5_000_000, "energy_uncertainty": { "window_us": 500_000 } } })), error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None }; + let job = Job { id: job_id(), client: "x".into(), request: "{}".into(), plan: Plan { work: 1_000_000_000, ..p.clone() }, work: Work::One, status: Status::Done, submitted_ms: 0, started_ms: None, finished_ms: Some(1), result: Some(json!({ "elapsed_ms": 500, "energy": { "joules_micro": 5_000_000, "energy_uncertainty": { "window_us": 500_000 } } })), error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0 }; s.time(&job); } let (t, e) = s.estimate(&p); diff --git a/wai/wai-quantum-service/src/main.rs b/wai/wai-quantum-service/src/main.rs index 46516b41f..6a37566b8 100644 --- a/wai/wai-quantum-service/src/main.rs +++ b/wai/wai-quantum-service/src/main.rs @@ -172,7 +172,27 @@ fn run_file(path: &str) { println!("{}", serde_json::to_string_pretty(&serde_json::json!({ "plan": plan, "result": result })).expect("JSON")); } +/// On macOS, the allocator keeps large blocks it freed, dirty, in a cache +/// charged to the process: a parsed request of a megabyte takes tens of +/// megabytes in blocks that large, so each one read stayed in the service's +/// footprint (measured on arm64: 30 requests of a megabyte, 1 GiB kept; with +/// the cache off, 2 MiB). The allocator reads its settings as the process +/// starts, so the service starts itself again with the cache off, as the same +/// process (`exec`); its readers and workers inherit the setting. +#[cfg(target_os = "macos")] +fn without_large_cache() { + use std::os::unix::process::CommandExt; + if std::env::var_os("MallocLargeCache").is_some() { + return; + } + let Ok(exe) = std::env::current_exe() else { return }; + let e = std::process::Command::new(exe).args(std::env::args_os().skip(1)).env("MallocLargeCache", "0").exec(); + eprintln!("starting again with the allocator's large cache off: {e}; going on with it on"); +} + fn serve() { + #[cfg(target_os = "macos")] + without_large_cache(); let config = jobs::Config::from_env(); let key_path = receipt::default_path(); // Readers and workers start from what this keeps, so the binary can be @@ -226,7 +246,9 @@ fn serve() { eprintln!("binding {addr}: {e}"); std::process::exit(1) }); - eprintln!("wai-quantum-service {} on {addr}, receipt key {}", env!("CARGO_PKG_VERSION"), app.signer.public_key()); + // The address bound, which names its port where port 0 asked for any. + let bound = listener.local_addr().unwrap_or(addr); + eprintln!("wai-quantum-service {} on {bound}, receipt key {}", env!("CARGO_PKG_VERSION"), app.signer.public_key()); let router = http::router(Arc::clone(&app)); let header_wait = app.service.config.header_wait; http::serve(listener, router, header_wait).await; diff --git a/wai/wai-quantum-service/src/mcp.rs b/wai/wai-quantum-service/src/mcp.rs index 3a4e8e4ef..9621b23f5 100644 --- a/wai/wai-quantum-service/src/mcp.rs +++ b/wai/wai-quantum-service/src/mcp.rs @@ -147,6 +147,9 @@ pub fn versions() -> Value { /// How long a `run_quantum_circuit` call waits before handing back the job id. pub const CALL_WAIT: Duration = Duration::from_secs(50); +/// What an answer keeps, of its time, to read its verdict after a run. +pub const VERDICT_TIME: Duration = Duration::from_secs(5); + const INSTRUCTIONS: &str = "Run quantum programs written in OpenQASM 3 (or 2). Call run_quantum_circuit with the program and what you want back: shots for measurement counts, observables (Pauli strings such as \"Z0 Z1\") for expectation values, amplitudes (bit strings, qubit n-1 first) for amplitudes, and `intent` for what the program must do. The service picks the strongest method the circuit allows and says which and why; every result names its accuracy, states its verdict (how far the engine checked: proof, evidence, executed, parsed or refused) and carries a signature verify_quantum_receipt can check. Counts are keyed by the classical bits, the highest bit first. If a call answers with status \"running\", call get_quantum_job with its id. A client that speaks tasks (MCP 2025-11-25) may call the three runs as tasks: the task is the job, its id the job's, and tasks/get, tasks/result and tasks/cancel follow it. run_quantum_set runs several programs, or one over a grid of its inputs, as one set with one receipt; dry_run_quantum_job says what a job or a set would take (method, memory, time, energy, bounds) and whether the service would take it, running nothing. quantum_circuit_examples gives ready-to-run programs."; /// What `initialize` tells an agent: how to use the service, then the @@ -640,6 +643,10 @@ async fn tool_answer(app: &App, who: &str, name: &str, args: Value, base: &str, let why = error.unwrap_or_else(|| "this answer was not read again".to_string()); return finish(name, &args, answer, views, Reading::Unread(&why)); } + // No time left within the answer to read the program again: said so. + if crate::offload::left().is_some_and(|l| l < Duration::from_secs(1)) { + return finish(name, &args, answer, views, Reading::Unread("no time was left within this answer to read the program again; poll the job for its verdict")); + } let finished = match app.service.gate.turn(crate::offload::Pool::Reading, who).await { Ok(turn) => crate::offload::read(&app.service.config, turn, crate::offload::Read::Finish { name: name.to_string(), args: args.clone(), answer: answer.clone(), views }).await, Err(e) => Err(e), @@ -869,7 +876,10 @@ async fn call(app: &App, who: &str, name: &str, args: Value, base: &str) -> Valu Ok(id) => id, Err(refused) => return refused, }; - app.service.wait(&id, CALL_WAIT).await; + // Within the answer's time, whatever admission took; a verdict + // read after it needs time too. + let wait = crate::offload::left().map_or(CALL_WAIT, |l| l.saturating_sub(VERDICT_TIME).min(CALL_WAIT)); + app.service.wait(&id, wait).await; let job = app.service.get(&id).unwrap_or(json!({ "id": id })); let failed = job["status"] == "failed"; tool_result(summary(&job), job, failed) diff --git a/wai/wai-quantum-service/src/offload.rs b/wai/wai-quantum-service/src/offload.rs index ad4492372..0e66d1a58 100644 --- a/wai/wai-quantum-service/src/offload.rs +++ b/wai/wai-quantum-service/src/offload.rs @@ -67,8 +67,37 @@ pub const CHECKS_AT_ONCE: usize = 4; pub const WAIT: Duration = Duration::from_secs(10); /// How many requests may wait for a turn: each holds its body meanwhile. pub const MAX_WAITING: usize = 32; -/// How many of one client's requests may wait behind its share of a pool. -pub const CLIENT_WAITING: usize = 4; +/// How many of one client's requests may wait behind its share of a pool: an +/// agent's burst of parallel calls waits, rather than being refused. +pub const CLIENT_WAITING: usize = 8; +/// How long a turn lent to a client past its share lasts: long enough for a +/// quick call (a critique, a lint, an estimate) beside its own slow one. +pub const LENT_FOR: Duration = Duration::from_secs(5); +/// A turn is lent past a client's share only while this many of the pool's +/// turns stay free for other clients after it. +pub const LEND_LEAVES: usize = 2; + +tokio::task_local! { + /// When the request being served must be answered by. + static DUE: Instant; +} + +/// `work`, its answer due at `due`: every turn it waits for and every +/// reading it makes ends by then ([`Gate::turn`], [`read`]). +pub async fn answering_by(due: Instant, work: F) -> F::Output { + DUE.scope(due, work).await +} + +/// When the request being served must be answered by, where that was said. +pub fn due() -> Option { + DUE.try_with(|d| *d).ok() +} + +/// The time left until the request being served must be answered, if any is +/// said. +pub fn left() -> Option { + due().map(|d| d.saturating_duration_since(Instant::now())) +} /// What a request is read among. #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] @@ -86,13 +115,17 @@ type Clients = Arc>>; struct Share { turns: Arc, requests: usize, + lent: usize, } /// The turns requests are read in: the service's two pools, each client's -/// share of each, and who is waiting. A client's requests past its share wait -/// for one of its own turns to come back, in order, so its parallel calls are -/// served one after another rather than refused, and no client holds more of -/// a pool than its share, however many it sends. +/// share of each, and who is waiting. A client holds at most its share of a +/// pool for as long as a reading may take. Past its share, one more turn may +/// be lent to it while the pool keeps [`LEND_LEAVES`] free for others, for at +/// most [`LENT_FOR`], so its quick calls pass its own slow one; the rest wait +/// behind its own turns, in order, so its parallel calls are answered one +/// after another rather than refused. No client holds more than its share of +/// a pool for longer than a quick call takes, however many it sends. pub struct Gate { reading: Arc, checks: Arc, @@ -105,13 +138,23 @@ pub struct Gate { /// A turn, held until its request has been read; dropped, the next goes. pub struct Turn { _pool: OwnedSemaphorePermit, - _share: OwnedSemaphorePermit, + _share: Option, _client: ClientHold, + /// Whether it was lent past the client's share, and so lasts at most + /// [`LENT_FOR`]. + pub lent: bool, } struct ClientHold { clients: Clients, key: (Pool, String), + lent: bool, +} + +impl ClientHold { + fn lend(&mut self) { + self.lent = true; + } } impl Drop for ClientHold { @@ -119,6 +162,9 @@ impl Drop for ClientHold { let mut clients = self.clients.lock().unwrap(); if let Some(s) = clients.get_mut(&self.key) { s.requests -= 1; + if self.lent { + s.lent -= 1; + } if s.requests == 0 { clients.remove(&self.key); } @@ -147,44 +193,101 @@ impl Gate { } } + /// A turn for `who` in `pool`, waiting at most [`WAIT`], and no later + /// than the request's answer is due ([`due`]). + pub async fn turn(&self, pool: Pool, who: &str) -> Result { + let until = Instant::now() + WAIT; + self.turn_until(pool, who, due().map_or(until, |d| d.min(until))).await + } + /// A turn for `who` in `pool`, or the refusal (`503`, `J032`): at once /// past [`CLIENT_WAITING`] of the client's own requests waiting, or past - /// [`MAX_WAITING`] in all; after [`WAIT`] for one that found no turn. - pub async fn turn(&self, pool: Pool, who: &str) -> Result { + /// [`MAX_WAITING`] in all; at `until` (and at most [`WAIT`]) for one that + /// found none, saying whether the client's own requests or the pool held + /// it up. + pub async fn turn_until(&self, pool: Pool, who: &str, until: Instant) -> Result { let busy = |why: String| (503, tag(refusal::BUSY, why)); let (share, pool_turns, full) = match pool { Pool::Reading => (self.shares.0, &self.reading, format!("{AT_ONCE} requests are being read; try again shortly")), Pool::Checks => (self.shares.1, &self.checks, format!("{CHECKS_AT_ONCE} checks are running; try again shortly")), }; let key = (pool, who.to_string()); - let (mine, client) = { + let (mine, mut client) = { let mut clients = self.clients.lock().unwrap(); - let s = clients.entry(key.clone()).or_insert_with(|| Share { turns: Arc::new(Semaphore::new(share)), requests: 0 }); - if s.requests >= share + CLIENT_WAITING { + let s = clients.entry(key.clone()).or_insert_with(|| Share { turns: Arc::new(Semaphore::new(share)), requests: 0, lent: 0 }); + if s.requests >= share + 1 + CLIENT_WAITING { return Err(busy(format!("{} of your requests are being read or waiting to be; send the next when one is answered", s.requests))); } s.requests += 1; - (Arc::clone(&s.turns), ClientHold { clients: Arc::clone(&self.clients), key }) + (Arc::clone(&s.turns), ClientHold { clients: Arc::clone(&self.clients), key, lent: false }) }; + // A turn of the client's own, and the pool's. if let Ok(m) = Arc::clone(&mine).try_acquire_owned() && let Ok(p) = Arc::clone(pool_turns).try_acquire_owned() { - return Ok(Turn { _pool: p, _share: m, _client: client }); + return Ok(Turn { _pool: p, _share: Some(m), _client: client, lent: false }); + } + if let Some(p) = self.lend(pool_turns, &mut client) { + return Ok(Turn { _pool: p, _share: None, _client: client, lent: true }); } if self.waiting.fetch_add(1, Ordering::SeqCst) >= MAX_WAITING { self.waiting.fetch_sub(1, Ordering::SeqCst); return Err(busy(full)); } let _waiting = Waiting(Arc::clone(&self.waiting)); - let both = async { - let m = Arc::clone(&mine).acquire_owned().await.ok()?; - let p = Arc::clone(pool_turns).acquire_owned().await.ok()?; - Some((p, m)) - }; - match tokio::time::timeout(WAIT, both).await { - Ok(Some((p, m))) => Ok(Turn { _pool: p, _share: m, _client: client }), - _ => Err(busy(full)), + let until = until.min(Instant::now() + WAIT); + // Each acquisition kept across the loop, so its place in line is kept. + let mut own: Option = None; + let mut own_wait = Box::pin(Arc::clone(&mine).acquire_owned()); + let mut pool_wait = Box::pin(Arc::clone(pool_turns).acquire_owned()); + loop { + let now = Instant::now(); + if now >= until { + break; + } + let step = tokio::time::sleep((until - now).min(Duration::from_millis(20))); + if own.is_none() { + tokio::select! { + m = &mut own_wait => match m { + Ok(m) => own = Some(m), + Err(_) => break, + }, + () = step => {} + } + if own.is_none() + && let Some(p) = self.lend(pool_turns, &mut client) + { + return Ok(Turn { _pool: p, _share: None, _client: client, lent: true }); + } + } else { + tokio::select! { + p = &mut pool_wait => match p { + Ok(p) => return Ok(Turn { _pool: p, _share: own, _client: client, lent: false }), + Err(_) => break, + }, + () = step => {} + } + } } + Err(busy(if own.is_some() { + full + } else { + format!("your own {share} turn{} of this pool {} still taken by your other requests after {} s; send this when one is answered", if share == 1 { "" } else { "s" }, if share == 1 { "is" } else { "are" }, WAIT.as_secs()) + })) + } + + /// A turn lent to `client` past its share, if the pool can spare one and + /// still keep [`LEND_LEAVES`] free, and the client holds none lent. + fn lend(&self, pool_turns: &Arc, client: &mut ClientHold) -> Option { + let mut clients = self.clients.lock().unwrap(); + let s = clients.get_mut(&client.key)?; + if s.lent > 0 || pool_turns.available_permits() <= LEND_LEAVES { + return None; + } + let p = Arc::clone(pool_turns).try_acquire_owned().ok()?; + s.lent += 1; + client.lend(); + Some(p) } } @@ -220,14 +323,34 @@ pub enum Read { /// `config` (or on a thread of its own, where `config` says so): its answer, /// or the refusal. pub async fn read(config: &Config, turn: Turn, r: Read) -> Result { + read_until(config, turn, r, due()).await +} + +/// [`read`], ended by `until` too, where the request has to be answered by +/// then (an answer due within a proxy's wait). +pub async fn read_until(config: &Config, turn: Turn, r: Read, until: Option) -> Result { + let now = Instant::now(); + // A turn lent past the client's share lasts a quick call's time. + let lent = turn.lent.then(|| now + LENT_FOR); + let by = [Some(now + config.read_timeout), lent, until].into_iter().flatten().min().unwrap_or(now + config.read_timeout); + let why = if lent == Some(by) { + Some(tag(refusal::BUSY, format!("this was read on a turn lent beside your own requests, which lasts {} s, and it took longer; send it again when your others are answered", LENT_FOR.as_secs()))) + } else if until == Some(by) && until < Some(now + config.read_timeout) { + Some(tag(refusal::BUSY, "no time was left to read this within one answer; send it again shortly".to_string())) + } else { + None + }; if !config.read_in_child { - return on_thread(move || { + let read = on_thread(move || { let _turn = turn; run(r) - }) - .await?; + }); + return match tokio::time::timeout(by.saturating_duration_since(Instant::now()), read).await { + Ok(r) => r?, + Err(_) => Err((if why.is_some() { 503 } else { 422 }, why.unwrap_or_else(|| tag(refusal::TOO_LARGE_HERE, format!("reading this request took more than {} s, longer than the service reads for; {LOCAL}", config.read_timeout.as_secs()))))), + }; } - let out = in_child(config, &r).await; + let out = in_child(config, &r, by, why).await; drop(turn); out } @@ -273,7 +396,9 @@ pub fn run(r: Read) -> Result { /// own length once read (`[0,0,…]` two bytes a value), so what a reading may /// give back to the service is held to this, not to its length. pub fn weight(v: &Value) -> u64 { - const VALUE: u64 = 32; + // A value with its place in an array, or an object's entry with its + // key and index (objects keep their order): measured, about 80 bytes. + const VALUE: u64 = 80; let mut stack = vec![v]; let mut total = 0u64; while let Some(v) = stack.pop() { @@ -363,7 +488,12 @@ impl Watch { }; // Said before the kill, so whoever sees the child end sees why. o.store(why, Ordering::SeqCst); - crate::jobs::kill(pid); + // Killed again until it has ended: a signal that did not take + // is sent again rather than waited out. + while !d.load(Ordering::SeqCst) { + crate::jobs::kill(pid); + std::thread::sleep(Duration::from_millis(100)); + } return; } }; @@ -378,6 +508,11 @@ impl Watch { fn over(&self) -> u8 { self.over.load(Ordering::SeqCst) } + + /// The child has ended: no signal is sent to its pid again. + fn ended(&self) { + self.done.store(true, Ordering::SeqCst); + } } impl Drop for Watch { @@ -386,13 +521,13 @@ impl Drop for Watch { } } -async fn in_child(config: &Config, r: &Read) -> Result { +async fn in_child(config: &Config, r: &Read, by: Instant, short: Option) -> Result { let exe = crate::jobs::self_exe().map_err(|e| (500, e))?; let mut cmd = crate::jobs::sandboxed(&exe, config.sandbox, "read"); cmd.stdin(Stdio::piped()).stdout(Stdio::piped()).stderr(Stdio::piped()).kill_on_drop(true); let mut child = cmd.spawn().map_err(|e| (503, tag(refusal::BUSY, format!("the service could not start a reader ({e}); try again shortly"))))?; let pid = child.id().unwrap_or(0); - let watch = Watch::start(pid, config.read_mib << 20, Instant::now() + config.read_timeout); + let watch = Watch::start(pid, config.read_mib << 20, by); let text = serde_json::to_vec(r).expect("a reading is JSON"); let mut stdin = child.stdin.take().expect("piped"); let writer = tokio::spawn(async move { @@ -419,13 +554,16 @@ async fn in_child(config: &Config, r: &Read) -> Result { }); let ended = loop { match tokio::time::timeout(Duration::from_millis(50), child.wait()).await { - Ok(Ok(status)) => break Ok(status), + Ok(Ok(status)) => { + watch.ended(); + break Ok(status); + } Ok(Err(e)) => break Err((500, format!("the reader could not be watched: {e}"))), Err(_) => {} } if too_long.load(Ordering::SeqCst) { - crate::jobs::kill(pid); - let _ = child.wait().await; + crate::jobs::stop(&mut child, pid).await; + watch.ended(); break Ok(ExitStatus::default()); } }; @@ -440,7 +578,13 @@ async fn in_child(config: &Config, r: &Read) -> Result { let detail = String::from_utf8_lossy(&err); match watch.over() { OVER_MEMORY => return refused(format!("reading this request took more than {} MiB, more than the service reads with; {LOCAL}", config.read_mib)), - OVER_TIME => return refused(format!("reading this request took more than {} s, longer than the service reads for; {LOCAL}", config.read_timeout.as_secs())), + OVER_TIME => { + // Cut short for its turn or its answer's time: to try again. + if let Some(why) = short { + return Err((503, why)); + } + return refused(format!("reading this request took more than {} s, longer than the service reads for; {LOCAL}", config.read_timeout.as_secs())); + } _ => {} } drop(watch); @@ -449,8 +593,18 @@ async fn in_child(config: &Config, r: &Read) -> Result { if !status.success() && detail.contains("memory allocation of") { return refused(format!("reading this request took more than {} MiB, more than the service reads with; {LOCAL}", config.read_mib)); } - // Read off the runtime: an answer of many megabytes takes a while. - let parsed = tokio::task::spawn_blocking(move || serde_json::from_slice::(&out).ok()).await.ok().flatten(); + // Read off the runtime: an answer of many megabytes takes a while. Read + // here where it is small, or where no thread can be started for it. + let parsed = if out.len() < 256 << 10 { + serde_json::from_slice::(&out).ok() + } else { + let out = Arc::new(out); + let theirs = Arc::clone(&out); + match catch_unwind(AssertUnwindSafe(|| tokio::task::spawn_blocking(move || serde_json::from_slice::(&theirs).ok()))) { + Ok(task) => task.await.ok().flatten(), + Err(_) => serde_json::from_slice::(&out).ok(), + } + }; match parsed { Some(Value::Object(mut v)) if status.success() && v.contains_key("ok") => Ok(v.remove("ok").unwrap_or(Value::Null)), Some(v) if status.success() && v["refused"][0].is_u64() => Err((v["refused"][0].as_u64().unwrap_or(500) as u16, v["refused"][1].as_str().unwrap_or("refused").to_string())), @@ -512,28 +666,39 @@ mod tests { assert_eq!(on_thread(|| deep(8000)).await.unwrap(), want); } - /// A client has its share of each pool, smaller than the pool: one that - /// sends more waits behind its own turns, in order, while every other - /// client is served at once; past its share and [`CLIENT_WAITING`] it is - /// told at once. As its turns come back, those waiting go. + /// A client has its share of each pool, smaller than the pool. Past it, + /// one more turn is lent to it while the pool keeps [`LEND_LEAVES`] free, + /// so its quick call passes its own slow one; the rest wait behind its own + /// turns, and every other client is served at once. As turns come back, + /// those waiting go. Past its share, one lent turn and [`CLIENT_WAITING`] + /// waiting, it is told at once. #[tokio::test] - async fn a_client_holds_no_more_of_a_pool_than_its_share() { + async fn a_client_holds_no_more_of_a_pool_than_its_share_and_a_quick_call_passes() { let gate = Gate::new(2, 1); let soon = Duration::from_millis(300); for (pool, share, size) in [(Pool::Reading, 2, AT_ONCE), (Pool::Checks, 1, CHECKS_AT_ONCE)] { assert!(share < size); let mut held = Vec::new(); for _ in 0..share { - held.push(gate.turn(pool, "a").await.ok().unwrap()); + let t = gate.turn(pool, "a").await.ok().unwrap(); + assert!(!t.lent); + held.push(t); } - // Past its share, it waits; every other client is served at once. - let waiting = tokio::time::timeout(soon, gate.turn(pool, "a")).await; - assert!(waiting.is_err(), "{pool:?}: a client past its share waits"); + // Lent only while the pool keeps two free for others. + let lent = if size - share > LEND_LEAVES { + let t = gate.turn(pool, "a").await.ok().unwrap(); + assert!(t.lent, "{pool:?}"); + Some(t) + } else { + None + }; + // Past that, it waits; every other client is served at once. + assert!(tokio::time::timeout(soon, gate.turn(pool, "a")).await.is_err(), "{pool:?}: a client past its share waits"); let t = Instant::now(); let other = gate.turn(pool, "b").await.ok().unwrap(); - assert!(t.elapsed() < soon, "{pool:?}"); + assert!(!other.lent && t.elapsed() < soon, "{pool:?}"); drop(other); - // Its next waits, and goes once one of its turns comes back. + // Its next goes once one of its turns comes back. let g = &gate; let next = async move { g.turn(pool, "a").await.ok().map(|_| ()) }; let release = async { @@ -542,11 +707,20 @@ mod tests { }; let (got, ()) = tokio::join!(next, release); assert!(got.is_some(), "{pool:?}"); - drop(held); + drop((held, lent)); } - // Past its share and the waiting it is allowed, told at once. + // No turn is lent where the pool would keep fewer than two free. + let gate = Gate::new(2, 1); + let (b, c) = (gate.turn(Pool::Checks, "b").await.ok().unwrap(), gate.turn(Pool::Checks, "c").await.ok().unwrap()); + let a = gate.turn(Pool::Checks, "a").await.ok().unwrap(); + let (code, why) = gate.turn_until(Pool::Checks, "a", Instant::now() + soon).await.err().unwrap(); + assert!(code == 503 && why.contains("your own 1 turn"), "{why}"); + drop((a, b, c)); + // Past its share, one lent and the waiting it is allowed, told at once. let gate = std::sync::Arc::new(Gate::new(1, 1)); let first = gate.turn(Pool::Checks, "a").await.ok().unwrap(); + let lent = gate.turn(Pool::Checks, "a").await.ok().unwrap(); + assert!(lent.lent); let waiters: Vec<_> = (0..CLIENT_WAITING) .map(|_| { let g = std::sync::Arc::clone(&gate); @@ -559,7 +733,7 @@ mod tests { let t = Instant::now(); let (code, why) = gate.turn(Pool::Checks, "a").await.err().unwrap(); assert!(code == 503 && why.ends_with("[J032]") && t.elapsed() < soon, "{why}"); - drop(first); + drop((first, lent)); for w in waiters { assert_eq!(w.await.unwrap(), None, "each waiting request had its turn in time"); } diff --git a/wai/wai-quantum-service/tests/depth.rs b/wai/wai-quantum-service/tests/depth.rs index cc9a8d215..84ac4c3ab 100644 --- a/wai/wai-quantum-service/tests/depth.rs +++ b/wai/wai-quantum-service/tests/depth.rs @@ -12,14 +12,11 @@ use serde_json::{Value, json}; use std::io::{Read, Write}; -use std::net::{TcpListener, TcpStream}; +use std::net::TcpStream; use std::path::Path; use std::process::{Child, Command, Stdio}; use std::time::{Duration, Instant}; -fn free_port() -> u16 { - TcpListener::bind("127.0.0.1:0").unwrap().local_addr().unwrap().port() -} /// One HTTP/1.1 exchange: the status, the headers and the body. fn http(port: u16, method: &str, path: &str, body: Option<&Value>) -> (u16, String, String) { @@ -40,22 +37,26 @@ fn json_of(port: u16, method: &str, path: &str, body: Option<&Value>) -> (u16, V } /// The service, its store at `dir`, `burst` submissions a window, once it -/// answers. -fn start(dir: &Path, port: u16, burst: u32) -> Child { - start_with(dir, port, burst, &[]) +/// answers: its process and the port it bound. +fn start(dir: &Path, burst: u32) -> (Child, u16) { + start_with(dir, burst, &[]) } /// [`start`], with `env` set beside. -fn start_with(dir: &Path, port: u16, burst: u32, env: &[(&str, &str)]) -> Child { - start_from(Path::new(env!("CARGO_BIN_EXE_wai-quantum-service")), dir, port, burst, env) +fn start_with(dir: &Path, burst: u32, env: &[(&str, &str)]) -> (Child, u16) { + start_from(Path::new(env!("CARGO_BIN_EXE_wai-quantum-service")), dir, burst, env) } -/// [`start_with`], the service started from the binary at `bin`. -fn start_from(bin: &Path, dir: &Path, port: u16, burst: u32, env: &[(&str, &str)]) -> Child { - let child = Command::new(bin) +/// [`start_with`], the service started from the binary at `bin`. It binds a +/// port the system chooses, and says which in its log, so no two services +/// started at once (by these tests or anything else on the host) race for one. +fn start_from(bin: &Path, dir: &Path, burst: u32, env: &[(&str, &str)]) -> (Child, u16) { + static STARTS: std::sync::atomic::AtomicUsize = std::sync::atomic::AtomicUsize::new(0); + let path = dir.join(format!("service-{}.log", STARTS.fetch_add(1, std::sync::atomic::Ordering::SeqCst))); + let mut child = Command::new(bin) .envs(env.iter().copied()) .arg("serve") - .env("WAI_QSVC_ADDR", format!("127.0.0.1:{port}")) + .env("WAI_QSVC_ADDR", "127.0.0.1:0") .env("WAI_QSVC_STORE", dir.join("store")) .env("WAI_QSVC_KEY", dir.join("signing.key")) .env("WAI_QSVC_JOB_GB", "2") @@ -64,15 +65,29 @@ fn start_from(bin: &Path, dir: &Path, port: u16, burst: u32, env: &[(&str, &str) .env("WAI_QSVC_ENERGY", "0") .env("WAI_QSVC_CLIENT_BURST", burst.to_string()) .stdout(Stdio::null()) - .stderr(Stdio::null()) + .stderr(std::fs::File::create(&path).map(Stdio::from).unwrap_or_else(|_| Stdio::null())) .spawn() .expect("the service starts"); + let log = || std::fs::read_to_string(&path).unwrap_or_default(); let t = Instant::now(); + let port = loop { + if let Some(p) = log().lines().find_map(|l| l.split(" on 127.0.0.1:").nth(1)?.split(',').next()?.trim().parse::().ok()) { + break p; + } + if let Ok(Some(status)) = child.try_wait() { + panic!("the service ended as it started ({status}): {}", log()); + } + assert!(t.elapsed() < Duration::from_secs(120), "the service did not start in 120 s: {}", log()); + std::thread::sleep(Duration::from_millis(50)); + }; while TcpStream::connect(("127.0.0.1", port)).is_err() || http(port, "GET", "/v1/health", None).0 != 200 { - assert!(t.elapsed() < Duration::from_secs(60), "the service did not answer"); + if let Ok(Some(status)) = child.try_wait() { + panic!("the service ended ({status}): {}", log()); + } + assert!(t.elapsed() < Duration::from_secs(120), "the service did not answer in 120 s: {}", log()); std::thread::sleep(Duration::from_millis(50)); } - child + (child, port) } /// The service is up and answering, and its process has not ended. @@ -99,8 +114,7 @@ fn parens(n: usize) -> String { #[test] fn programs_nested_thousands_deep_are_refused_and_the_service_runs_on() { let d = dir("depth"); - let port = free_port(); - let mut svc = start(&d, port, 1000); + let (mut svc, port) = start(&d, 1000); let deep = parens(5_000); let blocks = format!("{HEAD}{}x q[0];{}\nc = measure q;\n", "{ ".repeat(5_000), " }".repeat(5_000)); let power = format!("{HEAD}rz(1{}) q[0];\nc = measure q;\n", " ** 1".repeat(50_000)); @@ -191,8 +205,7 @@ fn programs_nested_thousands_deep_are_refused_and_the_service_runs_on() { #[test] fn a_request_with_heavy_references_is_refused_in_bounded_time_and_counted() { let d = dir("budget"); - let port = free_port(); - let mut svc = start(&d, port, 4); + let (mut svc, port) = start(&d, 4); // Each reference loops five million times on reading, and each differs, // so none is read from another's reading. let heavy = |k: usize| format!("OPENQASM 3.0;\nqubit[1] q;\nint a = 0;\nfor int i in [0:4999999] {{ a += 1; }}\n// {k}\nx q[0];\n"); @@ -244,8 +257,7 @@ fn a_request_with_heavy_references_is_refused_in_bounded_time_and_counted() { #[test] fn programs_that_would_hold_without_limit_are_refused_and_the_service_runs_on() { let d = dir("memory"); - let port = free_port(); - let mut svc = start(&d, port, 1000); + let (mut svc, port) = start(&d, 1000); let program = |body: &str| format!("OPENQASM 3.0;\nqubit[1] q;\n{body}x q[0];\n"); for body in [ "array[int[8], 4294967295] a;\n", @@ -279,14 +291,13 @@ fn programs_that_would_hold_without_limit_are_refused_and_the_service_runs_on() } /// The reading child's own caps, what holds where a limit of the reader's -/// might not: given 256 MiB, a set whose programs each read within the +/// might not: given 100 MiB, a set whose programs each read within the /// reader's limits but together hold more is refused for its memory, as is an error-correction circuit that unrolls past it; /// the service runs on. #[test] fn a_reading_past_its_memory_is_stopped_and_refused_and_the_service_runs_on() { let d = dir("readcap"); - let port = free_port(); - let mut svc = start_with(&d, port, 1000, &[("WAI_QSVC_READ_GB", "0.25"), ("WAI_QSVC_READ_SECS", "120")]); + let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_READ_GB", "0.1"), ("WAI_QSVC_READ_SECS", "120")]); let big = |k: u32| format!("OPENQASM 3.0;\nqubit[2] q;\nfor int i in [0:999998] {{ x q[0]; x q[1]; }}\n// {k}\n"); let t = Instant::now(); let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&json!({ "programs": (0..2).map(big).collect::>(), "shots": 10 }))); @@ -303,8 +314,7 @@ fn a_reading_past_its_memory_is_stopped_and_refused_and_the_service_runs_on() { svc.wait().unwrap(); // A sweep written out comes back into the service: held too, however // small its request, as that memory is the one every job shares. - let port = free_port(); - let mut svc = start_with(&d, port, 1000, &[("WAI_QSVC_READ_ANSWER_MIB", "1")]); + let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_READ_ANSWER_MIB", "1")]); let body = "x q;\n".repeat(500); let sweep = json!({ "qasm": format!("OPENQASM 3.0;\ninput int n;\nqubit q;\nfor int i in [1:n] {{\n{body}}}\n"), "sweep": { "n": (0..100).map(|k| 100 + k).collect::>() }, "shots": 1 }); let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&sweep)); @@ -323,8 +333,7 @@ fn a_reading_past_its_memory_is_stopped_and_refused_and_the_service_runs_on() { #[test] fn a_client_told_to_retry_spends_nothing_and_its_key_s_job_comes_back_past_the_window() { let d = dir("retry"); - let port = free_port(); - let mut svc = start_with(&d, port, 4, &[("WAI_QSVC_CLIENT_ACTIVE", "1")]); + let (mut svc, port) = start_with(&d, 4, &[("WAI_QSVC_CLIENT_ACTIVE", "1")]); let left = |head: &str| head.to_ascii_lowercase().lines().find_map(|l| l.strip_prefix("ratelimit: \"submissions\";r=").map(|r| r.split(';').next().unwrap_or("").to_string())).unwrap_or_default(); let post = |path: &str, body: &Value, key: Option<&str>| { let mut s = TcpStream::connect(("127.0.0.1", port)).unwrap(); @@ -381,8 +390,7 @@ fn a_client_told_to_retry_spends_nothing_and_its_key_s_job_comes_back_past_the_w #[test] fn an_index_past_what_a_model_holds_is_refused_and_the_service_runs_on() { let d = dir("indices"); - let port = free_port(); - let mut svc = start(&d, port, 1000); + let (mut svc, port) = start(&d, 1000); for body in [ json!({ "model": "shift_detectors 100000000000000\ndetector(1) D0\n", "detectors": [] }), json!({ "model": "shift_detectors 100000000000000\ndetector D0\n", "detectors": [] }), @@ -459,8 +467,7 @@ fn health_during(port: u16, during: impl FnOnce() + Send) -> f64 { #[test] fn a_refused_mcp_task_reads_nothing_and_holds_no_worker() { let d = dir("tasks"); - let port = free_port(); - let mut svc = start(&d, port, 1000); + let (mut svc, port) = start(&d, 1000); let worst = health_during(port, || { let took = at_once(4, |k| { let (code, b) = json_of(port, "POST", "/mcp", Some(&json!({ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "run_quantum_circuit", "task": {}, "arguments": { "qasm": slow(k), "idempotency_key": 5 } } }))); @@ -480,8 +487,7 @@ fn a_refused_mcp_task_reads_nothing_and_holds_no_worker() { #[test] fn a_check_and_a_set_s_receipt_count_before_they_read() { let d = dir("counted"); - let port = free_port(); - let mut svc = start(&d, port, 5); + let (mut svc, port) = start(&d, 5); let left = |head: &str| head.to_ascii_lowercase().lines().find_map(|l| l.strip_prefix("ratelimit: \"submissions\";r=").map(|r| r.split(';').next().unwrap_or("").to_string())).unwrap_or_default(); let wide = format!("OPENQASM 3.0;\nqubit[22] q;\nh q;\n{}", "rz(0.3) q;\n".repeat(200)); let (code, head, body) = http(port, "POST", "/v1/equivalent", Some(&json!({ "a": wide, "b": wide }))); @@ -509,8 +515,7 @@ fn a_check_and_a_set_s_receipt_count_before_they_read() { #[test] fn reading_leaves_the_runtime_free_and_a_client_has_its_share() { let d = dir("runtime"); - let port = free_port(); - let mut svc = start_with(&d, port, 1000, &[("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1")]); + let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1")]); let from = |who: &str, path: &str, body: &Value| { let mut s = TcpStream::connect(("127.0.0.1", port)).unwrap(); s.set_read_timeout(Some(Duration::from_secs(120))).unwrap(); @@ -534,10 +539,11 @@ fn reading_leaves_the_runtime_free_and_a_client_has_its_share() { let out = from("10.0.1.1", "/v1/critique", &json!({ "qasm": "OPENQASM 3.0; qubit q; h q;" })); assert!(out.starts_with("HTTP/1.1 200"), "{out}"); }); - // Past its share and the requests it may have waiting, told at once. + // Past its share, a lent turn and the requests it may have waiting, told + // at once. let answers = std::sync::Mutex::new(Vec::new()); - at_once(7, |k| { - if k == 6 { + at_once(11, |k| { + if k == 10 { std::thread::sleep(Duration::from_millis(300)); } let began = Instant::now(); @@ -558,8 +564,7 @@ fn reading_leaves_the_runtime_free_and_a_client_has_its_share() { #[test] fn a_reading_past_its_time_is_stopped_and_a_hang_up_ends_it() { let d = dir("deadline"); - let port = free_port(); - let mut svc = start_with(&d, port, 1000, &[("WAI_QSVC_READ_SECS", "1"), ("WAI_QSVC_CLIENT_READS", "1")]); + let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_READ_SECS", "1"), ("WAI_QSVC_CLIENT_READS", "1")]); let long = "OPENQASM 3.0;\nqubit q;\nint a = 0;\nfor int i in [0:9899999] { a += 1; }\nx q;\n"; let t = Instant::now(); let (code, b) = json_of(port, "POST", "/v1/critique", Some(&json!({ "qasm": long }))); @@ -568,8 +573,7 @@ fn a_reading_past_its_time_is_stopped_and_a_hang_up_ends_it() { alive(port, &mut svc, "a reading past its time"); svc.kill().unwrap(); svc.wait().unwrap(); - let port = free_port(); - let mut svc = start_with(&d, port, 1000, &[("WAI_QSVC_CLIENT_READS", "1")]); + let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_CLIENT_READS", "1")]); let text = json!({ "qasm": long }).to_string(); let mut s = TcpStream::connect(("127.0.0.1", port)).unwrap(); write!(s, "POST /v1/critique HTTP/1.1\r\nHost: 127.0.0.1\r\nContent-Type: application/json\r\nContent-Length: {}\r\n\r\n{text}", text.len()).unwrap(); @@ -595,8 +599,7 @@ fn a_reading_past_its_time_is_stopped_and_a_hang_up_ends_it() { #[test] fn a_check_s_answer_is_held_to_its_budget() { let d = dir("answer"); - let port = free_port(); - let mut svc = start(&d, port, 1000); + let (mut svc, port) = start(&d, 1000); let program = format!("OPENQASM 3.0;\nqubit[1] q;\nbit[1] c;\nint x = 0;\nfor int i in [0:9899999] {{ x = i; }}\n{}", "measure q[0];\n".repeat(40)); let t = Instant::now(); let (code, b) = json_of(port, "POST", "/v1/repair", Some(&json!({ "qasm": program }))); @@ -612,8 +615,7 @@ fn a_check_s_answer_is_held_to_its_budget() { #[test] fn a_job_within_the_reader_s_bounds_reads_here() { let d = dir("own"); - let port = free_port(); - let mut svc = start(&d, port, 1000); + let (mut svc, port) = start(&d, 1000); let program = "OPENQASM 3.0;\nqubit q;\nint a = 0;\nfor int i in [0:8499999] { a += 1; }\nfor int i in [0:1399999] { x q; }\n"; let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&json!({ "qasm": program, "shots": 1, "dry_run": true }))); assert!(code == 200 && b["accepted"] == true, "{code} {b}"); @@ -628,8 +630,7 @@ fn a_job_within_the_reader_s_bounds_reads_here() { #[test] fn a_stuck_connection_is_let_go() { let d = dir("stuck"); - let port = free_port(); - let mut svc = start_with(&d, port, 1000, &[("WAI_QSVC_HEADER_SECS", "1"), ("WAI_QSVC_BODY_SECS", "1")]); + let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_HEADER_SECS", "1"), ("WAI_QSVC_BODY_SECS", "1")]); let open = |text: &str| { let mut s = TcpStream::connect(("127.0.0.1", port)).unwrap(); s.set_read_timeout(Some(Duration::from_secs(10))).unwrap(); @@ -677,8 +678,7 @@ fn heavy_threshold() -> Value { #[test] fn one_client_cannot_hold_every_check_turn() { let d = dir("checkshare"); - let port = free_port(); - let mut svc = start_with(&d, port, 1000, &[("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1"), ("WAI_QSVC_QEC_SECS", "60"), ("WAI_QSVC_READ_SECS", "40")]); + let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1"), ("WAI_QSVC_QEC_SECS", "60"), ("WAI_QSVC_READ_SECS", "40")]); std::thread::scope(|t| { for _ in 0..2 { t.spawn(|| post_as(port, "198.51.100.66", "/v1/qec/threshold", &heavy_threshold())); @@ -703,8 +703,7 @@ fn one_client_cannot_hold_every_check_turn() { #[test] fn a_qec_call_at_its_bounds_ends_well_inside_the_reading_s_time() { let d = dir("qectime"); - let port = free_port(); - let mut svc = start(&d, port, 1000); + let (mut svc, port) = start(&d, 1000); for (path, body) in [("/v1/qec/threshold", heavy_threshold()), ("/v1/qec/decode", json!({ "memory": { "distance": 7, "rounds": 7, "p": 0.05 }, "shots": 200_000 }))] { let began = Instant::now(); let (code, b) = json_of(port, "POST", path, Some(&body)); @@ -733,8 +732,7 @@ fn peak_kib(svc: &Child) -> Option { #[test] fn a_reading_s_answer_is_held_to_what_it_takes_once_read() { let d = dir("answer-size"); - let port = free_port(); - let mut svc = start(&d, port, 1000); + let (mut svc, port) = start(&d, 1000); let sweep = |pad: usize, dry: bool| json!({ "qasm": "OPENQASM 3.0;\ninput int n;\nqubit q;\nx q;\n", "sweep": { "n": (0..100).collect::>() }, "shots": 1, "dry_run": dry, "pad": vec![0; pad] }); // A dry run takes back what it reports, however large its items are. for pad in [0, 100_000] { @@ -763,8 +761,7 @@ fn the_binary_can_be_replaced_before_the_restart() { let bin = d.join("app/wai-quantum-service"); std::fs::create_dir_all(bin.parent().unwrap()).unwrap(); std::fs::copy(env!("CARGO_BIN_EXE_wai-quantum-service"), &bin).unwrap(); - let port = free_port(); - let mut svc = start_from(&bin, &d, port, 1000, &[]); + let (mut svc, port) = start_from(&bin, &d, 1000, &[]); let stub = d.join("app/stub"); std::fs::write(&stub, "#!/bin/sh\necho 'not this service' >&2\nexit 2\n").unwrap(); #[cfg(unix)] @@ -785,3 +782,135 @@ fn the_binary_can_be_replaced_before_the_restart() { svc.wait().unwrap(); let _ = std::fs::remove_dir_all(&d); } + +/// The service's memory now, KiB: resident and swapped on Linux, resident +/// elsewhere. +fn mem_kib(svc: &Child) -> u64 { + if let Ok(s) = std::fs::read_to_string(format!("/proc/{}/status", svc.id())) { + let kb = |f: &str| s.lines().find(|l| l.starts_with(f)).and_then(|l| l.split_whitespace().nth(1)?.parse::().ok()).unwrap_or(0); + return kb("VmRSS:") + kb("VmSwap:"); + } + let o = Command::new("ps").args(["-o", "rss=", "-p", &svc.id().to_string()]).output().unwrap(); + String::from_utf8_lossy(&o.stdout).trim().parse().unwrap_or(0) +} + +/// What ended jobs keep: thirty jobs from one client, each carrying a field +/// of a few hundred thousand numbers (a parsed request took tens of MiB, kept +/// for a day and reloaded on restart), keep their requests' text alone, and +/// the service stays small, after a restart too. +#[test] +fn ended_jobs_keep_little_and_a_restart_keeps_it_little() { + let d = dir("kept"); + let (mut svc, port) = start(&d, 1000); + let job = json!({ "qasm": "OPENQASM 3.0; qubit q; bit c; h q; c = measure q;", "shots": 8, "pad": vec![0; 400_000] }); + let mut last = String::new(); + for _ in 0..30 { + let (code, b) = json_of(port, "POST", "/v1/run", Some(&job)); + assert!(code == 200 && b["status"] == "done", "{code} {b}"); + last = b["id"].as_str().unwrap().to_string(); + } + let kib = mem_kib(&svc); + assert!(kib < 200 << 10, "after thirty ended jobs the service holds {kib} KiB"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let (mut svc, port) = start(&d, 1000); + let (code, b) = json_of(port, "GET", &format!("/v1/jobs/{last}"), None); + assert!(code == 200 && b["request"]["pad"].as_array().is_some_and(|p| p.len() == 400_000), "{code}"); + let kib = mem_kib(&svc); + assert!(kib < 200 << 10, "after a restart the service holds {kib} KiB"); + alive(port, &mut svc, "the jobs kept"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// A program that runs for minutes. +fn long_job() -> Value { + let mut long = String::from("OPENQASM 3.0;\nqubit[24] q;\nh q;\n"); + for k in 0..6000 { + long += &format!("rx(0.{}) q[{}];\ncx q[{}], q[{}];\n", k % 9 + 1, k % 24, k % 24, (k + 1) % 24); + } + json!({ "qasm": long, "observables": ["Z0"] }) +} + +/// Every answer comes within the answer's time from the request's arrival: +/// a run whose job is still running answers with the job to poll, by REST +/// and by MCP, where it waited 25 s and 50 s and passed a proxy's wait. +#[test] +fn a_run_s_answer_comes_within_the_answer_s_time() { + let d = dir("answer-time"); + let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_ANSWER_SECS", "15")]); + let began = Instant::now(); + let (code, b) = json_of(port, "POST", "/v1/run", Some(&long_job())); + assert!(code == 202 && b["id"].is_string() && began.elapsed() < Duration::from_secs(20), "{code} after {:?}", began.elapsed()); + let first = b["id"].as_str().unwrap().to_string(); + let began = Instant::now(); + let (code, b) = json_of(port, "POST", "/mcp", Some(&json!({ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "run_quantum_circuit", "arguments": long_job() } }))); + let sc = &b["result"]["structuredContent"]; + assert!(code == 200 && sc["id"].is_string() && began.elapsed() < Duration::from_secs(20), "{code} after {:?}: {}", began.elapsed(), sc["status"]); + assert!(sc["verdict"].is_object(), "{sc}"); + for id in [first.as_str(), sc["id"].as_str().unwrap()] { + let _ = http(port, "DELETE", &format!("/v1/jobs/{id}"), None); + } + alive(port, &mut svc, "runs answered in time"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// One client's quick calls pass its own slow, in-bounds check, on a turn +/// lent beside it, and its burst of eight quick calls is answered in full, +/// where they waited behind it and were refused. +#[test] +fn a_client_s_quick_calls_pass_its_own_slow_one_and_its_burst_is_answered() { + let d = dir("ownqueue"); + let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1"), ("WAI_QSVC_QEC_SECS", "60"), ("WAI_QSVC_READ_SECS", "40")]); + let me = "198.51.100.23"; + std::thread::scope(|t| { + t.spawn(|| post_as(port, me, "/v1/qec/threshold", &heavy_threshold())); + std::thread::sleep(Duration::from_millis(1500)); + let began = Instant::now(); + let (code, out) = post_as(port, me, "/v1/critique", &json!({ "qasm": "OPENQASM 3.0; qubit q; h q;" })); + // Well inside the slow call's 40 s, and inside the lent turn's 5 s. + assert!(code == 200 && began.elapsed() < Duration::from_secs(5), "{code} after {:?}: {out}", began.elapsed()); + let answers = at_once(8, |_| { + let (code, out) = post_as(port, me, "/v1/critique", &json!({ "qasm": "OPENQASM 3.0; qubit q; h q;" })); + assert_eq!(code, 200, "{out}"); + }); + assert!(answers.iter().all(|&s| s < 10.0), "{answers:?}"); + alive(port, &mut svc, "a client's own queue"); + svc.kill().unwrap(); + }); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// The caps stop a reading when no new process can be started: the kill is a +/// signal the service sends itself. Linux, where a process's limits can be +/// lowered while it runs (`prlimit`). +#[cfg(target_os = "linux")] +#[test] +fn a_reading_is_stopped_when_no_process_can_be_started() { + if Command::new("prlimit").arg("--version").output().is_err() { + eprintln!("no prlimit here: not run"); + return; + } + let d = dir("nproc"); + // No time of its own for the call: only the reading's cap stops it. + let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_READ_SECS", "4"), ("WAI_QSVC_QEC_SECS", "100000")]); + let pid = svc.id().to_string(); + let answer = std::thread::spawn(move || { + let began = Instant::now(); + let (code, b) = json_of(port, "POST", "/v1/qec/threshold", Some(&heavy_threshold())); + (code, b, began.elapsed()) + }); + std::thread::sleep(Duration::from_millis(1500)); + let ok = Command::new("prlimit").args(["--pid", &pid, "--nproc=1:1"]).status().unwrap().success(); + assert!(ok, "prlimit"); + let (code, b, took) = answer.join().unwrap(); + assert!(code == 422 && b.to_string().contains("J031") && took < Duration::from_secs(15), "{code} after {took:?}: {b}"); + assert!(svc.try_wait().unwrap().is_none(), "the service ended"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} diff --git a/wai/wai-quantum/src/engine/qec.rs b/wai/wai-quantum/src/engine/qec.rs index 881209788..f274cf0b1 100644 --- a/wai/wai-quantum/src/engine/qec.rs +++ b/wai/wai-quantum/src/engine/qec.rs @@ -52,11 +52,15 @@ thread_local! { } /// Runs `f` with the shots it samples and decodes held to `limit` in all, on -/// this thread: a decode with `shots`, and a threshold, each refused -/// (`J031`, [`refusal::TOO_LARGE_HERE`]) as soon as what it has done says the -/// rest would take longer, or once it has. What a shot costs depends on the -/// noise and the decoder as much as on the counts, so a host that answers -/// within a time measures it as it goes rather than guessing it first. +/// this thread: a decode with `shots`, and a threshold of any kind, each +/// refused (`J031`, [`refusal::TOO_LARGE_HERE`]) at once when what it has +/// measured says it would take longer, and stopped if it runs a fifth past +/// that. What a shot costs depends on the noise, the distance and the decoder +/// as much as on the counts, so each point is measured before its shots are +/// run: its setup and sampling as they are, and a pilot of its own shots +/// (at most one in a hundred, and at least 32) for its rate. The rest is +/// projected from those, so setup is never multiplied by the shots, and a +/// call whose measured rate says it fits is run. pub fn within_time(limit: std::time::Duration, f: impl FnOnce() -> T) -> T { struct Reset(Option<(std::time::Instant, std::time::Duration)>); impl Drop for Reset { @@ -68,24 +72,42 @@ pub fn within_time(limit: std::time::Duration, f: impl FnOnce() -> T) -> T { f() } -/// Whether `done` of `total` shots, sampled and decoded so far, leave the -/// rest within this thread's time ([`within_time`]): the refusal, naming the -/// command that runs it locally, where they do not. -fn on_time(done: u64, total: u64, op: &str) -> Result<(), String> { +/// The shots of a point decoded first to measure its rate: at most one in a +/// hundred (and at least 32), for at most half a second, and at least 8. +fn pilot_of(shots: u64) -> u64 { + shots.min((shots / 100).max(32)) +} + +/// Whether a pilot begun at `began` with `done` of its `most` shots decoded +/// goes on. +fn piloting(done: usize, most: usize, began: std::time::Instant) -> bool { + done < most && (done < 8 || began.elapsed() < std::time::Duration::from_millis(500)) +} + +/// The refusal of a call whose shots would take `secs`. +fn too_slow(secs: f64, limit: std::time::Duration, op: &str) -> String { + tag(refusal::TOO_LARGE_HERE, format!("its shots would take about {secs:.0} s to sample and decode here, more than the {} s a call here takes; run `wai-quantum qec {op}` on your machine", limit.as_secs())) +} + +/// Whether the call, `rest` seconds of measured work from done, fits this +/// thread's time ([`within_time`]): the refusal where it does not. +fn fits(rest: f64, op: &str) -> Result<(), String> { + let Some((start, limit)) = TIME.with(std::cell::Cell::get) else { return Ok(()) }; + let projected = start.elapsed().as_secs_f64() + rest; + if projected > limit.as_secs_f64() { Err(too_slow(projected, limit, op)) } else { Ok(()) } +} + +/// Whether the call has run a fifth past this thread's time: the refusal +/// where it has, its projection having been short. +fn still_on_time(op: &str) -> Result<(), String> { let Some((start, limit)) = TIME.with(std::cell::Cell::get) else { return Ok(()) }; let spent = start.elapsed(); - let refuse = |secs: f64| Err(tag(refusal::TOO_LARGE_HERE, format!("its shots would take about {secs:.0} s to sample and decode here, more than the {} s a call here takes; run `wai-quantum qec {op}` on your machine", limit.as_secs()))); - if spent > limit { - return refuse(spent.as_secs_f64() * total.max(1) as f64 / done.max(1) as f64); - } - // Projected once enough is done to say: a hundredth, or a second. - if done > 0 && (done.saturating_mul(100) >= total || spent >= std::time::Duration::from_secs(1)) { - let projected = spent.as_secs_f64() * total as f64 / done as f64; - if projected > limit.as_secs_f64() { - return refuse(projected); - } - } - Ok(()) + if spent > limit + limit / 5 { Err(too_slow(spent.as_secs_f64(), limit, op)) } else { Ok(()) } +} + +/// Whether a time is set on this thread: only then is a pilot measured. +fn timed() -> bool { + TIME.with(std::cell::Cell::get).is_some() } fn hex(b: &[u8]) -> String { @@ -403,8 +425,23 @@ pub fn decode(request: &Value) -> Result { let seed = request.get("seed").and_then(Value::as_u64).unwrap_or(1); let smp = frame::sample(&c, shots as usize, seed); let mut failures = 0u64; - for i in 0..smp.shots { - on_time(i as u64, shots, "decode")?; + // The pilot's shots measure the rate; the rest are projected from it. + let most = if timed() { pilot_of(shots) as usize } else { 0 }; + let began = std::time::Instant::now(); + let mut i = 0; + while piloting(i, most, began) { + if dec.run(&smp.fired(i), &mut ws)?.0 != smp.flips(i) { + failures += 1; + } + i += 1; + } + if i > 0 { + fits(began.elapsed().as_secs_f64() / i as f64 * (smp.shots - i) as f64, "decode")?; + } + for i in i..smp.shots { + if i % 64 == 0 { + still_on_time("decode")?; + } if dec.run(&smp.fired(i), &mut ws)?.0 != smp.flips(i) { failures += 1; } @@ -563,12 +600,26 @@ pub fn threshold(request: &Value) -> Result { let mut memory = Vec::new(); let decoder; let noise; - let total = shots * points as u64; - let mut done = 0u64; match family { "surface" => { decoder = request.get("decoder").and_then(Value::as_str).unwrap_or("matching"); noise = "circuit level: depolarizing noise after every gate and on idle data, flips after resets and before measurements, all at p"; + // Each point set up, sampled and its pilot decoded first, so the + // whole call is projected from what every point costs before the + // rest of any is run; then each point's other shots, in order. + struct Point { + d: u32, + rounds: u32, + p: f64, + smp: frame::Detections, + dec: Dec, + ws: Option, + failures: u64, + piloted: usize, + } + let most = if timed() { pilot_of(shots) as usize } else { 0 }; + let mut run: Vec = Vec::with_capacity(points); + let mut rest = 0f64; for &d in &dists { let d = d as u32; if !(3..=MAX_CIRCUIT_DISTANCE).contains(&d) || d.is_multiple_of(2) { @@ -580,40 +631,75 @@ pub fn threshold(request: &Value) -> Result { let model = frame::error_model(&c).map_err(|e| qerr(e.to_string()))?; let dec = decoder_for(decoder, &model, request)?; let smp = frame::sample(&c, shots as usize, seed.wrapping_add(u64::from(d) * 1000 + (p * 1e6) as u64)); - let mut ws = None; - let mut failures = 0u64; - for i in 0..smp.shots { - on_time(done + i as u64, total, "threshold")?; - if dec.run(&smp.fired(i), &mut ws)?.0 != smp.flips(i) { - failures += 1; + let mut pt = Point { d, rounds, p, smp, dec, ws: None, failures: 0, piloted: 0 }; + let began = std::time::Instant::now(); + while piloting(pt.piloted, most, began) { + let i = pt.piloted; + if pt.dec.run(&pt.smp.fired(i), &mut pt.ws)?.0 != pt.smp.flips(i) { + pt.failures += 1; } + pt.piloted += 1; } - done += shots; - let pt = MemoryPoint { distance: d, rounds, p, shots, failures }; - memory.push(pt); - let rate = failures as f64 / shots as f64; - out_points.push(json!({ "distance": d, "p": p, "rounds": rounds, "shots": shots, "failures": failures, "rate": rate, "stderr": (rate * (1.0 - rate) / shots as f64).sqrt(), "per_round": pt.per_round() })); + if pt.piloted > 0 { + rest += began.elapsed().as_secs_f64() / pt.piloted as f64 * (pt.smp.shots - pt.piloted) as f64; + // Refused as soon as what is measured already passes it. + fits(rest, "threshold")?; + } + run.push(pt); } } + for mut pt in run { + for i in pt.piloted..pt.smp.shots { + if i % 64 == 0 { + still_on_time("threshold")?; + } + if pt.dec.run(&pt.smp.fired(i), &mut pt.ws)?.0 != pt.smp.flips(i) { + pt.failures += 1; + } + } + let (d, rounds, p, failures) = (pt.d, pt.rounds, pt.p, pt.failures); + let pt = MemoryPoint { distance: d, rounds, p, shots, failures }; + memory.push(pt); + let rate = failures as f64 / shots as f64; + out_points.push(json!({ "distance": d, "p": p, "rounds": rounds, "shots": shots, "failures": failures, "rate": rate, "stderr": (rate * (1.0 - rate) / shots as f64).sqrt(), "per_round": pt.per_round() })); + } } "color" | "toric" | "surface_capacity" => { decoder = if family == "color" { "color" } else { "union_find" }; noise = "code capacity: independent X flips on the data at p, perfect syndromes"; + let mut codes = Vec::with_capacity(dists.len()); for &d in &dists { let code_name = match family { "color" => format!("color:{d}"), "toric" => format!("toric:{d}"), _ => format!("surface:{d}"), }; - let c = read_code(&json!(code_name))?; + codes.push((d, read_code(&json!(code_name))?)); + } + let bench = |c: &Code, p: f64, trials: u32, s: u64| match &c.color { + Some(cc) => crate::quantum_color::benchmark(cc, p, trials, s), + None => crate::quantum_qec::benchmark_uf(&c.css, p, trials, s), + }; + // A point runs whole, so each is measured on a pilot of its own + // trials first, and the call projected from them all. + if timed() { + let pilot = pilot_of(shots); + let mut rest = 0f64; + for (d, c) in &codes { + for &p in &ps { + let began = std::time::Instant::now(); + bench(c, p, pilot as u32, seed.wrapping_add(d * 1000 + (p * 1e6) as u64)); + rest += began.elapsed().as_secs_f64() / pilot as f64 * shots as f64; + fits(rest, "threshold")?; + } + } + } + for (d, c) in &codes { + let d = *d; for &p in &ps { - on_time(done, total, "threshold")?; - done += shots; + still_on_time("threshold")?; let s = seed.wrapping_add(d * 1000 + (p * 1e6) as u64); - let (trials, failures) = match &c.color { - Some(cc) => crate::quantum_color::benchmark(cc, p, shots as u32, s), - None => crate::quantum_qec::benchmark_uf(&c.css, p, shots as u32, s), - }; + let (trials, failures) = bench(c, p, shots as u32, s); let rate = f64::from(failures) / f64::from(trials); out_points.push(json!({ "distance": d, "p": p, "shots": trials, "failures": failures, "rate": rate, "stderr": (rate * (1.0 - rate) / f64::from(trials)).sqrt() })); } @@ -743,6 +829,27 @@ pub fn call(op: &str, request: &Value) -> Result { mod tests { use super::*; + /// A call whose setup and sampling are most of its time, and its shots + /// cheap, is run within a time it fits: what is measured before the first + /// shot is not multiplied by the shots. A code-capacity threshold, whose + /// points run whole, is measured on a pilot of each point and refused at + /// once where it would not fit. A documented default answers, or is + /// refused within a few seconds, not at the end of its time. + #[test] + fn a_time_is_measured_apart_from_setup_and_holds_every_threshold() { + let circuit = crate::quantum_frame::surface_code_memory(11, 50, 1e-5); + let req = json!({ "circuit": circuit, "shots": 20_000 }); + let t = std::time::Instant::now(); + let r = within_time(std::time::Duration::from_secs(15), || call("decode", &req)); + assert!(r.is_ok(), "{r:?} after {:?}", t.elapsed()); + let t = std::time::Instant::now(); + let e = within_time(std::time::Duration::from_secs(3), || call("threshold", &json!({ "family": "color", "distances": [15], "p": [0.08], "shots": 200_000 }))).unwrap_err(); + assert!(e.ends_with("[J031]") && t.elapsed() < std::time::Duration::from_secs(10), "{e} after {:?}", t.elapsed()); + let t = std::time::Instant::now(); + let r = within_time(std::time::Duration::from_secs(25), || call("threshold", &json!({ "family": "surface", "decoder": "correlated" }))); + assert!(r.is_ok() || t.elapsed() < std::time::Duration::from_secs(12), "{r:?} after {:?}", t.elapsed()); + } + /// Within a time, a decode and a threshold whose shots would take longer /// are refused as soon as their first shots say so, not at the end of the /// time; one that fits answers as it did; with no time set, none is. -- 2.54.0 (Apple Git-157) From 381392897cc7aa1e1d8a1c5f3a291fbc5ab51d7f Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Sat, 7 Nov 2026 11:02:35 -0500 Subject: [PATCH 08/18] jouleclaw public-surface audit: the report goes to a file and the log keeps the verdict The scan writes an 800 KB report at once. On 2026-11-07 three runs of it (main at 208ffd374, #659 and #676), on three runners, kept a log that stopped at 256 KiB mid-report, and each job was failed twelve minutes later, though the scan's summary read "public surface only: 0". The summary stays in the log, and on a failure so does the report's list of what is missing; a pass logs 393 bytes. The error lines are single-quoted, so their backticks are no longer run as commands. Co-Authored-By: Claude Opus 5.5 --- .../jouleclaw-public-surface-documented.yml | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/.github/workflows/jouleclaw-public-surface-documented.yml b/.github/workflows/jouleclaw-public-surface-documented.yml index b94d6b6dd..e10766ec0 100644 --- a/.github/workflows/jouleclaw-public-surface-documented.yml +++ b/.github/workflows/jouleclaw-public-surface-documented.yml @@ -81,11 +81,21 @@ jobs: - name: Run brownfield self-scan with --fail-on-lost working-directory: jouleclaw/jouleclaw-rs + # The report (stdout) goes to a file, not the log: it is about 800 KB + # written at once, and on 2026-11-07 three runs of it, on three + # runners, kept a log that stopped at 256 KiB, mid-report, and the job + # was failed twelve minutes later, though the scan's own summary read + # "public surface only: 0". The summary (stderr) stays in the log, and + # on a failure so does the report's list of what is missing, which + # comes before its module purposes. run: | set -euo pipefail - if ! cargo run -q -p jouleclaw-brownfield --example scan_self -- --fail-on-lost; then - echo "::error::Public-surface Lost > 0 — jouleclaw-rs has undocumented public functions." - echo "::error::Fix by adding a `///` doc comment above each flagged `pub fn`, OR by downgrading to `pub(crate)` if it isn't consumer-visible." - echo "::error::Run locally with: cargo run -p jouleclaw-brownfield --example scan_self" + report="$(mktemp)" + if ! cargo run -q -p jouleclaw-brownfield --example scan_self -- --fail-on-lost > "$report"; then + sed -n '/^## Module purposes/q;p' "$report" + echo '::error::Public-surface Lost > 0 — jouleclaw-rs has undocumented public functions.' + echo '::error::Fix by adding a `///` doc comment above each flagged `pub fn`, OR by downgrading to `pub(crate)` if it is not consumer-visible.' + echo '::error::Run locally with: cargo run -p jouleclaw-brownfield --example scan_self' exit 1 fi + echo "public-surface report: $(wc -c < "$report") bytes, not printed; run the scan locally to read it" -- 2.54.0 (Apple Git-157) From 53676a3e51e1e7f58a728f4b5d3d400c33f40c4a Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Sun, 8 Nov 2026 08:43:42 -0500 Subject: [PATCH 09/18] wai-quantum qec: a code-capacity threshold's pilot is held to half a second, and the timing test is held to its own build A code-capacity point runs whole, so its pilot was one in a hundred of its trials, run whole: 2,000 trials of colour:15 took 11 to 14 s in a debug build beside the library's other tests (the macOS energy-receipts leg at 7586335a7, and the Studio twice), before the call could be refused for its 3 s. The pilot now runs in batches, each twice the one before, while piloting() holds it, as a circuit's is: at least 8 trials, then up to half a second. A trial has no setup of its own, so a batch's time over its trials is the point's rate. The test's refusal bound is 5 s, from 10. The setup-heavy decode in the same test is timed first with no time set, then given twice that and a second: a fixed 15 s was a release build's figure and refused it in a debug build under load. Before, on the Studio, the whole library in debug: the test failed in two runs of two (at the decode, and at the colour call after 14.5 s). After: 1,411 passed, 0 failed, twice; Thor release 1,416 passed, and its seven QEC tests in debug. Co-Authored-By: Claude Opus 5.5 --- wai/wai-quantum/src/engine/qec.rs | 36 +++++++++++++++++++++++++------ 1 file changed, 29 insertions(+), 7 deletions(-) diff --git a/wai/wai-quantum/src/engine/qec.rs b/wai/wai-quantum/src/engine/qec.rs index f274cf0b1..7556b04d4 100644 --- a/wai/wai-quantum/src/engine/qec.rs +++ b/wai/wai-quantum/src/engine/qec.rs @@ -681,15 +681,28 @@ pub fn threshold(request: &Value) -> Result { None => crate::quantum_qec::benchmark_uf(&c.css, p, trials, s), }; // A point runs whole, so each is measured on a pilot of its own - // trials first, and the call projected from them all. + // trials first, and the call projected from them all. The pilot + // runs in batches, each twice the one before, held as a circuit's + // is to half a second ([`piloting`]): run whole, a point's one in + // a hundred of 200,000 trials of a large code took 11 s in a debug + // build before the call could be refused. A batch has no setup of + // its own (each trial is drawn and decoded alone), so its time + // over its trials is the point's rate. if timed() { - let pilot = pilot_of(shots); + let pilot = pilot_of(shots) as usize; let mut rest = 0f64; for (d, c) in &codes { for &p in &ps { + let s = seed.wrapping_add(d * 1000 + (p * 1e6) as u64); let began = std::time::Instant::now(); - bench(c, p, pilot as u32, seed.wrapping_add(d * 1000 + (p * 1e6) as u64)); - rest += began.elapsed().as_secs_f64() / pilot as f64 * shots as f64; + let (mut done, mut batch) = (0usize, 8usize); + while piloting(done, pilot, began) { + let n = batch.min(pilot - done); + bench(c, p, n as u32, s.wrapping_add(done as u64)); + done += n; + batch *= 2; + } + rest += began.elapsed().as_secs_f64() / done.max(1) as f64 * shots as f64; fits(rest, "threshold")?; } } @@ -837,14 +850,23 @@ mod tests { /// refused within a few seconds, not at the end of its time. #[test] fn a_time_is_measured_apart_from_setup_and_holds_every_threshold() { + // Its time measured here first, with no time set; then, given twice + // that and a second, it is run, not refused: a time here is a build's + // and a host's (a debug build beside the library's other tests took + // past 15 s, where a release build alone takes a second or two). let circuit = crate::quantum_frame::surface_code_memory(11, 50, 1e-5); let req = json!({ "circuit": circuit, "shots": 20_000 }); let t = std::time::Instant::now(); - let r = within_time(std::time::Duration::from_secs(15), || call("decode", &req)); - assert!(r.is_ok(), "{r:?} after {:?}", t.elapsed()); + call("decode", &req).unwrap(); + let took = t.elapsed(); + let t = std::time::Instant::now(); + let r = within_time(took * 2 + std::time::Duration::from_secs(1), || call("decode", &req)); + assert!(r.is_ok(), "{r:?} after {:?}, where it took {took:?} with no time set", t.elapsed()); + // Refused on its pilot, which is held to about half a second, not on + // its one in a hundred run whole (11 s in a debug build on macOS). let t = std::time::Instant::now(); let e = within_time(std::time::Duration::from_secs(3), || call("threshold", &json!({ "family": "color", "distances": [15], "p": [0.08], "shots": 200_000 }))).unwrap_err(); - assert!(e.ends_with("[J031]") && t.elapsed() < std::time::Duration::from_secs(10), "{e} after {:?}", t.elapsed()); + assert!(e.ends_with("[J031]") && t.elapsed() < std::time::Duration::from_secs(5), "{e} after {:?}", t.elapsed()); let t = std::time::Instant::now(); let r = within_time(std::time::Duration::from_secs(25), || call("threshold", &json!({ "family": "surface", "decoder": "correlated" }))); assert!(r.is_ok() || t.elapsed() < std::time::Duration::from_secs(12), "{r:?} after {:?}", t.elapsed()); -- 2.54.0 (Apple Git-157) From 10fdebd147b7e1dac10a0f2040a0248006e3a285 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Sun, 8 Nov 2026 11:05:51 -0500 Subject: [PATCH 10/18] wai-quantum-service: connections held under the open-file limit, results kept until read, a lent turn moves onto the client's own, and one RFC 8785 report on every route Fourth review of #676, 11 findings: - The service raises its open-file limit to its hard one as it starts (on macOS to kern.maxfilesperproc and 10240 at most) and logs it. It holds at most WAI_QSVC_CONNECTIONS (4096) connections, fewer under that limit; past them the connection idle longest, or the one whose body has been arriving longest, is closed for the new one, so a health check is always answered, and with none to close the new one gets 503 at once. A client may have WAI_QSVC_CLIENT_REQUESTS (32) requests open, refused 503 past them before a body is read. - A job keeps its result as RFC 8785 text, counted at its length. An ended job not yet read is kept for WAI_QSVC_KEPT_MIN_SECS (600) whatever it weighs; while a client's unread results hold its cap its next job is refused (429), and while everyone's hold the service's, 503. Past the service's cap the client keeping the most loses its results first, the newest between equals. An MCP run whose job is no longer known is an error, never a success without a result. - A reading on a lent turn moves onto the client's own turn when one comes free, and runs past 5 s while no other request waits. - Every route writes a job as one RFC 8785 text, its request and result spliced in as kept, the receipt's keys sorted, the same after a restart; MCP answers are written as RFC 8785 too. - A large MCP message past the window is read only for its id, method, tool and key, and answered under its own id; a keyed resend of a job the client holds and a receipt check are read whole. - A body is read before its window only for a resend under a key the client holds a job under; receipt checks have their own window, WAI_QSVC_CLIENT_VERIFIES (120), asked before the body is read. - tasks/result holds its answer within what is left of the answer's time; the OpenAPI document lists 408, 413 and the pre-handler 503 on every route. Co-Authored-By: Claude Opus 5.5 --- wai/changes/2026-11-06-quantum-parse-depth.md | 16 +- wai/wai-quantum-service/README.md | 12 +- wai/wai-quantum-service/docs/self-host.md | 10 +- wai/wai-quantum-service/src/discovery.rs | 28 ++ wai/wai-quantum-service/src/http.rs | 202 +++++++++-- wai/wai-quantum-service/src/jobs.rs | 313 +++++++++++++--- wai/wai-quantum-service/src/main.rs | 48 ++- wai/wai-quantum-service/src/mcp.rs | 59 +++- wai/wai-quantum-service/src/offload.rs | 139 ++++++-- wai/wai-quantum-service/tests/depth.rs | 334 +++++++++++++++++- 10 files changed, 1033 insertions(+), 128 deletions(-) diff --git a/wai/changes/2026-11-06-quantum-parse-depth.md b/wai/changes/2026-11-06-quantum-parse-depth.md index 3cdc734c8..9f91ef3b0 100644 --- a/wai/changes/2026-11-06-quantum-parse-depth.md +++ b/wai/changes/2026-11-06-quantum-parse-depth.md @@ -76,7 +76,7 @@ is. Four requests are read at once and four checks beside them. A client has its share of each, `WAI_QSVC_CLIENT_READS` (2) and `WAI_QSVC_CLIENT_CHECKS` (1), smaller than the pool, so one client holds no pool for longer than a quick call; past its share one more turn is lent to it while the pool keeps two free, for at most - five seconds (so its quick calls pass its own slow one; a call that takes longer on a lent turn is refused, `503`), + five seconds once others wait (so its quick calls pass its own slow one; a call on a lent turn moves onto the client's own turn as one comes free, and is refused, `503`, only if others wait for a turn after five seconds), and the rest wait behind its own turns in order (an agent's burst of eight is answered one after another), eight at most, 32 in all, each for ten seconds; past any of these a request is refused, `503` and `J032`, saying whether its own requests or the pool held it up. People behind one address share one client's turns. A child is polled every 50 ms from a thread of its own (on macOS its @@ -110,13 +110,23 @@ their results as read, is held to `WAI_QSVC_CLIENT_KEPT_MIB` (128) a client and `WAI_QSVC_KEPT_MIB` (2048) in all, the oldest dropped first, before their `WAI_QSVC_RETAIN_SECS`. A request's body is parsed only once its client's window has room (a resend under its key, answered before any limit, first), and a large MCP message from a client - past its window is refused unread. A job's report is written from its request's text, never parsed again, and the + past its window is answered under its own id with a tool result that says why, read only for its id, method, tool + and key (a resend under a key the client holds, and a receipt check, are read whole). A body is read before its + window only for a resend under a key its client holds a job under. A receipt check has a window of its own, + `WAI_QSVC_CLIENT_VERIFIES` (120). An ended job not yet read is kept for `WAI_QSVC_KEPT_MIN_SECS` (600) whatever it + weighs, and past the service's cap the client keeping the most loses its results first. A job's report is one + RFC 8785 text on every route and after a restart, its request and result written in as kept, never parsed again, and the memory a large request or answer was parsed into is given back once it is answered. On macOS the service starts itself with the allocator's large-block cache off (`MallocLargeCache=0`), which kept every large block a parsed request freed, dirty and charged to the service: on arm64, 30 requests of a megabyte kept 1 GiB; with it off, 2 MiB. - **The job service: connections, updates and restarts.** A request's header section is held to `WAI_QSVC_HEADER_SECS` (150, which also closes an idle connection, so it sits above a proxy's two-minute idle - limit) and its body to `WAI_QSVC_BODY_SECS` (30; past it, `408`). Readers and workers start from the service's own + limit) and its body to `WAI_QSVC_BODY_SECS` (30; past it, `408`). It raises its open-file limit to its hard + limit as it starts (on macOS to `kern.maxfilesperproc` and 10240 at most) and holds at most `WAI_QSVC_CONNECTIONS` + (4096) connections under it: past them the one idle longest is closed for the new one, so a health check is always + answered, or the new one is answered `503` at once; a client may have `WAI_QSVC_CLIENT_REQUESTS` (32) requests open, + refused `503` past them before a body is read. `tasks/result` holds its answer within what is left of the answer's + time. Readers and workers start from the service's own binary as it was started (Linux: its running image; elsewhere a copy kept beside the signing key), so the binary can be replaced before the restart. A restart reads no program: a job keeps the plan it was admitted with and a set its items, where a queue of slow programs held the port unbound for the sum of their reads. diff --git a/wai/wai-quantum-service/README.md b/wai/wai-quantum-service/README.md index bdd70517e..dc86394d2 100644 --- a/wai/wai-quantum-service/README.md +++ b/wai/wai-quantum-service/README.md @@ -142,7 +142,7 @@ Counts are always seeded samples. | `GET /v1` | the service: its methods, limits, load and receipt key | | `POST /v1/run` | submit a job and wait up to 25 s: the finished job, or `202` and the job if it is still running | | `POST /v1/jobs` | submit a job or a set: `202` and the job, with its `id`, once it is in the store; with `dry_run`, `200` and the plan | -| `GET /v1/jobs/{id}` | the job: its status, the request as submitted, its result when it is done, and the digests it is kept under; `?wait=S` (at most 25) holds the answer until its status changes | +| `GET /v1/jobs/{id}` | the job, as RFC 8785 text, the same on every route and after a restart: its status, the request as sealed (the canonical form of the one submitted), its result when it is done, and the digests it is kept under; `?wait=S` (at most 25) holds the answer until its status changes | | `GET /v1/jobs/{id}/events` | the job's status as server-sent events: a `status` event at each change, then one `job` event with the whole job when it ends | | `DELETE /v1/jobs/{id}` | cancel a queued or running job | | `POST /v1/verify` | `{"request": ..., "result": ...}` (a set's is counted and read as a check is, since its sweep is written out again; one whose seal does not hold is not): whether the result's receipt seals that request and that result and every rule of [the provider protocol's section 8.4](../extensions/quantum-providers.md#84-what-a-verifier-checks) holds, read from the body's own text: `valid`, `reasons`, `key`, `form`, and what a provider's result says of itself | @@ -197,13 +197,13 @@ A submission is admitted in the order of what each step costs: Reading happens outside the service's own process, as each job's run does: each request is read by a child, this binary run as `wai-quantum-service read`, sandboxed as a worker is. -- **Turns.** Four requests are read at a time (jobs, dry runs and the MCP tools' verdicts), and four checks beside them. A client has its share of each: `WAI_QSVC_CLIENT_READS` (2) of the first and `WAI_QSVC_CLIENT_CHECKS` (1) of the checks, so no client holds a whole pool. Past its share, one more turn is lent to a client while the pool keeps two free for others, for at most five seconds, so its quick calls (a critique, a lint, an estimate) pass its own slow one; a call that takes longer on a lent turn is refused, `503` and `J032`. Its other requests wait behind its own turns, in order, so an agent's burst of parallel calls is answered one after another: up to eight of them may wait, at most 32 requests wait in all, each for ten seconds, and past any of these a request is refused, `503` and `J032`, having read and spent nothing, saying whether its own requests or the pool held it up. A client is its address, so people behind one address (an office, a carrier's NAT, a class) share one client's turns and its window: raise the shares, or give such a group an API key of its own on a private deployment. +- **Turns.** Four requests are read at a time (jobs, dry runs and the MCP tools' verdicts), and four checks beside them. A client has its share of each: `WAI_QSVC_CLIENT_READS` (2) of the first and `WAI_QSVC_CLIENT_CHECKS` (1) of the checks, so no client holds a whole pool. Past its share, one more turn is lent to a client while the pool keeps two free for others, so its quick calls (a critique, a lint, an estimate) pass its own slow one. A call on a lent turn moves onto the client's own turn as soon as one comes free, and then has the whole of its time; until then it runs past five seconds only while no other request waits for a turn, and once one does it is refused, `503` and `J032`, to send again. Its other requests wait behind its own turns, in order, so an agent's burst of parallel calls is answered one after another: up to eight of them may wait, at most 32 requests wait in all, each for ten seconds, and past any of these a request is refused, `503` and `J032`, having read and spent nothing, saying whether its own requests or the pool held it up. A client is its address, so people behind one address (an office, a carrier's NAT, a class) share one client's turns and its window: raise the shares, or give such a group an API key of its own on a private deployment. - **Memory.** A child's memory is polled every 50 ms from a thread of its own, and the child is killed once it holds more than `WAI_QSVC_READ_GB` (4); the request is refused, `422` and `J031`. The kill is a signal the service sends itself (`SIGKILL`, to the child's process group, and again every 100 ms until it has ended), so it needs no new process. On macOS the poll reads the child's physical footprint, which counts its compressed and swapped pages; on Linux its resident and swapped pages. A poll is not a kernel limit: a child can pass the cap by what it touches in one interval before it is killed. On Linux the child also holds its own address space to twice the cap and a gigabyte, so an allocation past that fails at once (`422`, `J031`), and the kernel ends a reader or a worker before the service when the host runs short. macOS gives an unprivileged process neither, and has no limit that counts compressed memory. - **Time.** A child that reads for longer than `WAI_QSVC_READ_SECS` (45) is killed and the request refused, `422` and `J031`. Every request is answered within `WAI_QSVC_ANSWER_SECS` (55) of its arrival, under the 60 s a proxy in front commonly waits for an answer: its waits for a turn and its readings end by then, `POST /v1/run` and the MCP run tools wait for their job only for what is left and then answer with the job, still running, to poll, and an MCP answer's verdict is read again only if time is left (its verdict says so where it was not). An error-correction call's shots are measured as they run: each point is set up and sampled, and a pilot of its own shots decoded, before the rest of any; it is refused at once when what is measured says it would take more than `WAI_QSVC_QEC_SECS` (25), and stopped if it runs a fifth past that. - **Its answer.** What a reading gives back comes into the service's memory, which every job shares: one that would take more than `WAI_QSVC_READ_ANSWER_MIB` (128) once read, about 80 bytes a value, is refused in the child before it is written (`422`, `J031`), and the service reads it off its request threads. A dry run gets back the plans and points alone. A check's answer can be that large: a critique of a program of 200,000 gates that each draw a remark (`qubit q; for int i in [1:200000] { x q; }`) is refused so; one of 150,000 is answered. -- **What ended jobs keep.** A job keeps its request as the canonical text the store keeps, and a set lets go of its items once it ends. What ended jobs keep in memory, their requests' text and their results as read, is held to `WAI_QSVC_CLIENT_KEPT_MIB` (128) a client and `WAI_QSVC_KEPT_MIB` (2048) in all; past either, the oldest ended jobs are dropped before their `WAI_QSVC_RETAIN_SECS`. A request's body is parsed only once its client's window has room (a resend under its key first, as it is answered before any limit), and a large MCP message from a client past its window is refused unread. A job's report is written from its request's text, never parsed again, and the memory a large request or answer was parsed into is given back to the system once it is answered. On macOS the service starts itself with the allocator's large-block cache off (`MallocLargeCache=0`): that cache kept every large block a parsed request had freed, dirty and charged to the service (on arm64, 30 requests of a megabyte kept 1 GiB; with it off, 2 MiB). +- **What ended jobs keep.** A job keeps its request as the canonical text the store keeps, and a set lets go of its items once it ends. What ended jobs keep in memory, their requests' text and their results as read, is held to `WAI_QSVC_CLIENT_KEPT_MIB` (128) a client and `WAI_QSVC_KEPT_MIB` (2048) in all; past a client's cap its oldest ended jobs are dropped before their `WAI_QSVC_RETAIN_SECS`, and past the service's, those of the client keeping the most (between two keeping as much, the newer results), so many clients together cannot drop another's. An ended job whose result has not been read is kept whatever it weighs, for `WAI_QSVC_KEPT_MIN_SECS` (600) or until it is read: a result is never dropped before it could be read. While a client's unread results hold its whole cap its next job is refused, `429`, until it reads them, and while all unread results hold the service's, `503`. A request's body is parsed only once its client's window has room (or, for a resend under a key the client holds a job under, first, as it is answered before any limit). A receipt check has a window of its own, `WAI_QSVC_CLIENT_VERIFIES` (120), asked before its body is read. An MCP message over 64 KiB from a client past its window is read only for its id, method, tool and key, and answered under its own id with a tool result that says why; a resend under a key the client holds, and a receipt check, are read whole. A job's report is one RFC 8785 text on every route (a poll, its events, a cancel, the MCP job tools, whose answers are written as RFC 8785 too) and after a restart: its request and its result are written in as the text each is kept as, never parsed again, and the memory a large request or answer was parsed into is given back to the system once it is answered. On macOS the service starts itself with the allocator's large-block cache off (`MallocLargeCache=0`): that cache kept every large block a parsed request had freed, dirty and charged to the service (on arm64, 30 requests of a megabyte kept 1 GiB; with it off, 2 MiB). - **Ending.** A child that panics, overflows its stack or aborts is answered `500`, and the service runs on. A client that hangs up ends its child. -- **Connections.** A request whose header section takes longer than `WAI_QSVC_HEADER_SECS` (150) to arrive is closed; the same limit closes a connection left idle between requests, so it is set above a proxy's own idle limit (two minutes, commonly), and the proxy, not the service, closes its idle connections. A body that takes longer than `WAI_QSVC_BODY_SECS` (30) is refused, `408`. +- **Connections.** A request whose header section takes longer than `WAI_QSVC_HEADER_SECS` (150) to arrive is closed; the same limit closes a connection left idle between requests, so it is set above a proxy's own idle limit (two minutes, commonly), and the proxy, not the service, closes its idle connections. A body that takes longer than `WAI_QSVC_BODY_SECS` (30) is refused, `408`. A client may have `WAI_QSVC_CLIENT_REQUESTS` (32) requests open at once; past it a request is refused, `503`, before its body is read (a health check is answered whatever that is). The service raises its open-file limit to its hard limit as it starts (on macOS no further than `kern.maxfilesperproc` and 10240), says so in its log, and holds at most `WAI_QSVC_CONNECTIONS` (4096) connections, fewer where that limit leaves less room: past them the connection idle longest (or the one whose body has been arriving longest) is closed for the new one, so a health check is always answered, and with none to close a new connection is answered `503` at once. - **Updating.** Readers and workers start from the service's own binary as it was started: on Linux its running image, elsewhere a copy it keeps beside its signing key, named by its digest. So the binary can be replaced before the restart (by a copy over it, or, on Linux, where a running binary cannot be written, by a rename), and the running service goes on with its own until it is restarted. ## Sending a job again @@ -351,6 +351,10 @@ These variables set the limits: | `WAI_QSVC_ANSWER_SECS` | 55 | the longest any answer takes, its waits and readings included | | `WAI_QSVC_CLIENT_KEPT_MIB` | 128 | what one client's ended jobs may keep in memory | | `WAI_QSVC_KEPT_MIB` | 2048 | what all ended jobs may keep in memory | +| `WAI_QSVC_KEPT_MIN_SECS` | 600 | how long an ended job not yet read is kept whatever the two caps above say | +| `WAI_QSVC_CLIENT_VERIFIES` | 120 | receipts one client may have checked in its window (`POST /v1/verify`, `verify_quantum_receipt`) | +| `WAI_QSVC_CLIENT_REQUESTS` | 32 | requests one client may have open at once; past it, `503` before the body is read | +| `WAI_QSVC_CONNECTIONS` | 4096 | connections held at once, at most; lowered to what the open-file limit leaves room for | | `WAI_QSVC_CLIENT_READS` | 2 | how many of one client's jobs, dry runs and verdicts are read at once (more wait) | | `WAI_QSVC_CLIENT_CHECKS` | 1 | how many of one client's checks run at once (more wait) | | `WAI_QSVC_HEADER_SECS` | 150 | how long a client may take to send a request's header section, and how long an idle connection stays open | diff --git a/wai/wai-quantum-service/docs/self-host.md b/wai/wai-quantum-service/docs/self-host.md index 8dcc8fc2d..66d2f2d21 100644 --- a/wai/wai-quantum-service/docs/self-host.md +++ b/wai/wai-quantum-service/docs/self-host.md @@ -57,6 +57,10 @@ Everything is set by environment variables. | `WAI_QSVC_ANSWER_SECS` | 55 | the longest any answer takes, its waits and readings included | | `WAI_QSVC_CLIENT_KEPT_MIB` | 128 | what one client's ended jobs may keep in memory | | `WAI_QSVC_KEPT_MIB` | 2048 | what all ended jobs may keep in memory | +| `WAI_QSVC_KEPT_MIN_SECS` | 600 | how long an ended job not yet read is kept whatever the two caps above say | +| `WAI_QSVC_CLIENT_VERIFIES` | 120 | receipts one client may have checked in its window (`POST /v1/verify`, `verify_quantum_receipt`) | +| `WAI_QSVC_CLIENT_REQUESTS` | 32 | requests one client may have open at once; past it, `503` before the body is read | +| `WAI_QSVC_CONNECTIONS` | 4096 | connections held at once, at most; lowered to what the open-file limit leaves room for | | `WAI_QSVC_CLIENT_READS` | 2 | how many of one client's jobs, dry runs and verdicts are read at once (more wait) | | `WAI_QSVC_CLIENT_CHECKS` | 1 | how many of one client's checks run at once (more wait) | | `WAI_QSVC_HEADER_SECS` | 150 | how long a client may take to send a request's header section, and how long an idle connection stays open | @@ -134,6 +138,10 @@ Each finished job carries its energy inside the signed result: a counter reading - **Linux:** the processor packages' energy counters, read through the kernel's power-capping interface before the worker starts and after it exits. They count whole packages, so the figure is the packages' energy over the job's window, every process on the host included, with no attribution to the job (`attribution: none`). It names how many other jobs ran in the window (`concurrent_jobs`). The counter is used only when every package zone reads and its count is live; its declared uncertainty covers the two reads and the counter's update edges, measured at start. - **Permissions:** many Linux hosts let only root, or a group such as `energy`, read the count. Add the service's user to that group. In a container the interface is usually hidden; a container that should meter needs it visible and readable, which widens what the container sees of the host. Without it, jobs say they are unmetered and why. +## Open files + +Each connection takes a file. The service raises its soft open-file limit to its hard one as it starts and logs the result (`open files: …`), and holds connections under it with room for its readers' and workers' pipes and its store's files (`connections: at most …`). On macOS a launchd service inherits `launchctl limit maxfiles`, 256 by default; raise it in the service's plist (`SoftResourceLimits` and `HardResourceLimits`, `NumberOfFiles`, say 10240), or the service holds at most 192 connections at once. + ## Behind a reverse proxy Run the service on a loopback address and let the proxy take the public traffic. @@ -143,7 +151,7 @@ Run the service on a loopback address and let the proxy take the public traffic. - **Event streams.** `GET /v1/jobs/{id}/events` is a server-sent event stream. The proxy must pass it through unbuffered (the service sends `X-Accel-Buffering: no` and a comment every 15 s), and allow it to stay open as long as `WAI_QSVC_JOB_SECS`. Polls with `?wait=` are held up to 25 s. - **Bodies.** A request body is at most 1 MiB, and `POST /v1/run` waits up to 25 s before answering 202. - **Rate-limit fields.** Pass the `RateLimit-Policy`, `RateLimit` and `Retry-After` headers through. -- **Timeouts.** Every answer comes within `WAI_QSVC_ANSWER_SECS` (55) of the request's arrival, so within a proxy's 60 s wait for an answer's header. The worst case of each route, from arrival: a check, a dry run or a submission waits at most 10 s for a turn and reads for at most `WAI_QSVC_READ_SECS` (45), 55 s in all; `POST /v1/run` and the MCP run tools then wait for their job only for what is left of the 55 s (less 5 s, on MCP, for its verdict) and answer with the job, still running, to poll; an MCP answer's verdict waits for a turn and is read only within what is left, and says where no time was left; `tasks/result` holds a reply 50 s at most. With a shorter wait in the proxy, lower `WAI_QSVC_ANSWER_SECS` below it. Keep the proxy's idle connections shorter than `WAI_QSVC_HEADER_SECS` (150), which also bounds an idle connection on the service's side: the proxy then closes its idle connections itself, and never sends a request on one as the service closes it. +- **Timeouts.** Every answer comes within `WAI_QSVC_ANSWER_SECS` (55) of the request's arrival, so within a proxy's 60 s wait for an answer's header. The worst case of each route, from arrival: a check, a dry run or a submission waits at most 10 s for a turn and reads for at most `WAI_QSVC_READ_SECS` (45), 55 s in all; `POST /v1/run` and the MCP run tools then wait for their job only for what is left of the 55 s (less 5 s, on MCP, for its verdict) and answer with the job, still running, to poll; an MCP answer's verdict waits for a turn and is read only within what is left, and says where no time was left; `tasks/result` holds a reply for what is left of the 55 s, 50 s at most. With a shorter wait in the proxy, lower `WAI_QSVC_ANSWER_SECS` below it. Keep the proxy's idle connections shorter than `WAI_QSVC_HEADER_SECS` (150), which also bounds an idle connection on the service's side: the proxy then closes its idle connections itself, and never sends a request on one as the service closes it. - **One address, many people.** The service counts a client by the address the proxy gives it. People behind one address share one client's turns (`WAI_QSVC_CLIENT_READS`, `WAI_QSVC_CLIENT_CHECKS`) and window; raise those, or give such a group an API key. ## Updating it diff --git a/wai/wai-quantum-service/src/discovery.rs b/wai/wai-quantum-service/src/discovery.rs index 7b69ba15c..cc946f419 100644 --- a/wai/wai-quantum-service/src/discovery.rs +++ b/wai/wai-quantum-service/src/discovery.rs @@ -348,6 +348,25 @@ pub fn openapi(server: &str) -> Value { for (path, item) in qec_paths().into_iter().chain(pulse_paths()).chain(variational_paths()) { doc["paths"][path] = item; } + // What every route can answer before its handler runs: a body that did + // not arrive in time, one past the size a request may be, and a client + // with as many requests open as it may have, or a service with as many + // connections as it holds. + let before = [ + ("408", "The body did not arrive within WAI_QSVC_BODY_SECS (30 s)"), + ("413", "The body is larger than the 1 MiB a request may be"), + ("503", "The client has WAI_QSVC_CLIENT_REQUESTS (32) requests open at once, or the service holds as many connections as it may (J032); Retry-After says when to try again"), + ]; + if let Some(paths) = doc["paths"].as_object_mut() { + for item in paths.values_mut() { + for op in item.as_object_mut().into_iter().flat_map(|o| o.values_mut()) { + let Some(responses) = op.get_mut("responses").and_then(Value::as_object_mut) else { continue }; + for (code, why) in before { + responses.entry(code).or_insert_with(|| json!({ "description": why, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } })); + } + } + } + } doc } @@ -505,5 +524,14 @@ mod tests { assert!(d["paths"].get(p).is_some(), "{p}"); } assert_eq!(d["openapi"], "3.1.0"); + // What any route answers before its handler: a body late, too large, + // or a client or the service with as many open as it may have. + for (path, item) in d["paths"].as_object().unwrap() { + for (method, op) in item.as_object().unwrap() { + for code in ["408", "413", "503"] { + assert!(op["responses"].get(code).is_some(), "{method} {path} lists no {code}"); + } + } + } } } diff --git a/wai/wai-quantum-service/src/http.rs b/wai/wai-quantum-service/src/http.rs index cacf2786e..df4c1f279 100644 --- a/wai/wai-quantum-service/src/http.rs +++ b/wai/wai-quantum-service/src/http.rs @@ -96,7 +96,8 @@ use axum::routing::{get, post}; use axum::{Json, Router}; use serde_json::{Value, json}; use std::net::{IpAddr, SocketAddr}; -use std::sync::Arc; +use std::collections::HashMap; +use std::sync::{Arc, Mutex}; use std::time::Duration; use wai_quantum::engine::refusal::{self, tag}; use wai_quantum::engine::sets; @@ -142,6 +143,11 @@ fn reply(status: u16, body: Value) -> Response { } /// [`reply`], its body JSON text already written. +/// `v` as RFC 8785 text: how a job's report is written on every route. +pub fn canonical_text(v: &Value) -> String { + String::from_utf8(wai_quantum::engine::receipt::canonical(v)).unwrap_or_else(|_| v.to_string()) +} + fn reply_text(status: u16, body: String) -> Response { let mut r = (StatusCode::from_u16(status).unwrap_or(StatusCode::INTERNAL_SERVER_ERROR), [("content-type", "application/json")], body).into_response(); r.headers_mut().insert("access-control-allow-origin", HeaderValue::from_static("*")); @@ -561,8 +567,8 @@ fn dry_run_now(service: &Service, who: &str, (_, planned): Admitted) -> Result, ConnectInfo(peer): ConnectInfo, headers: HeaderMap, body: axum::body::Bytes) -> Response { let who = client(&app, peer, &headers); - // A resend under its key is answered before any limit, so read first. - let request = match json_body(&app, &who, &body, headers.contains_key("idempotency-key"), "the body is not a JSON job") { + // A resend under a key this client holds is answered before any limit, so read first. + let request = match json_body(&app, &who, &body, resend(&app, &who, &headers), "the body is not a JSON job") { Ok(v) => v, Err((code, why)) => return error(code, &why), }; @@ -584,8 +590,8 @@ async fn submit(State(app): State, ConnectInfo(peer): ConnectInfo, ConnectInfo(peer): ConnectInfo, headers: HeaderMap, body: axum::body::Bytes) -> Response { let who = client(&app, peer, &headers); - // A resend under its key is answered before any limit, so read first. - let request = match json_body(&app, &who, &body, headers.contains_key("idempotency-key"), "the body is not a JSON job") { + // A resend under a key this client holds is answered before any limit, so read first. + let request = match json_body(&app, &who, &body, resend(&app, &who, &headers), "the body is not a JSON job") { Ok(v) => v, Err((code, why)) => return error(code, &why), }; @@ -619,6 +625,12 @@ const LARGE: usize = 256 << 10; /// The largest MCP message read from a client past its window. const MCP_UNREAD: usize = 64 << 10; +/// Whether the request is a resend under an idempotency key that names a job +/// `who` holds: only then is its body read before its window is asked. +fn resend(app: &App, who: &str, headers: &HeaderMap) -> bool { + headers.get("idempotency-key").and_then(|v| v.to_str().ok()).is_some_and(|k| app.service.holds_key(who, k)) +} + /// A request's body read as JSON, once its client's window has room, so that /// a request refused for its window is never parsed: a parsed body takes many /// times its length. With `first` (a resend under its key, answered before any @@ -680,15 +692,14 @@ async fn events(State(app): State, Path(id): Path) -> Response { return None; } loop { - let job = f.app.service.get(&f.id)?; - let status = job["status"].as_str().unwrap_or("").to_string(); + let status = f.app.service.status(&f.id)?.name().to_string(); if f.last.as_deref() != Some(status.as_str()) { f.last = Some(status.clone()); let ev = if matches!(status.as_str(), "queued" | "running") { Event::default().event("status").data(json!({ "id": f.id, "status": status }).to_string()) } else { f.ended = true; - Event::default().event("job").data(job.to_string()) + Event::default().event("job").data(f.app.service.report(&f.id)?) }; return Some((Ok::(ev), f)); } @@ -716,7 +727,7 @@ async fn receipt_keys(State(app): State) -> Response { async fn cancel(State(app): State, Path(id): Path) -> Response { match app.service.cancel(&id) { - Some(true) => reply(200, app.service.get(&id).unwrap_or(json!({ "id": id }))), + Some(true) => reply_text(200, app.service.report(&id).unwrap_or_else(|| json!({ "id": id }).to_string())), Some(false) => error(409, "the job has already finished"), None => error(404, "no such job"), } @@ -730,6 +741,10 @@ async fn cancel(State(app): State, Path(id): Path) -> Response { /// resource receipt in it checked. async fn verify(State(app): State, ConnectInfo(peer): ConnectInfo, headers: HeaderMap, body: Result) -> Response { let Ok(bytes) = body else { return error(400, "the body is not JSON") }; + // Counted against the client's own window for checks, before it is read. + if let Err(Refused(code, why)) = app.service.verify_counted(&client(&app, peer, &headers)) { + return error(code, &why); + } let Some(text) = std::str::from_utf8(&bytes).ok() else { return error(400, "the body is not JSON") }; let Ok(v) = serde_json::from_str::(text) else { return error(400, "the body is not JSON") }; // Read as every job's text is, whatever kind of job it is. @@ -939,9 +954,10 @@ async fn mcp(State(app): State, ConnectInfo(peer): ConnectInfo MCP_UNREAD - && let Err(Refused(code, why)) = app.service.window_open(&who) + && let Err(Refused(_, why)) = app.service.window_open(&who) + && let Some(answer) = crate::mcp::past_window(&app, &who, &body, &why) { - return reply(code, json!({ "jsonrpc": "2.0", "id": null, "error": { "code": -32000, "message": why } })); + return reply_text(200, canonical_text(&answer)); } let Ok(msg) = serde_json::from_slice::(&body) else { return reply(400, json!({ "jsonrpc": "2.0", "id": null, "error": { "code": -32700, "message": "not JSON" } })); @@ -967,7 +983,7 @@ async fn mcp(State(app): State, ConnectInfo(peer): ConnectInfo(Event::default().data(a.to_string())) + Ok::(Event::default().data(canonical_text(&a))) }); let mut r = Sse::new(answer).keep_alive(KeepAlive::new().interval(Duration::from_secs(15))).into_response(); r.headers_mut().insert("access-control-allow-origin", HeaderValue::from_static("*")); @@ -976,7 +992,8 @@ async fn mcp(State(app): State, ConnectInfo(peer): ConnectInfo { - let mut r = reply(200, answer); + // Written as RFC 8785, so a job in an answer has its report's bytes. + let mut r = reply_text(200, canonical_text(&answer)); if opening && let Ok(id) = HeaderValue::from_str(&crate::mcp::views_session()) { r.headers_mut().insert("mcp-session-id", id); r.headers_mut().insert("access-control-expose-headers", HeaderValue::from_static("mcp-session-id")); @@ -1058,11 +1075,33 @@ fn rate_headers(app: &App, who: &str, r: &mut Response) { /// holds its connection no longer than that (`408`). Only then is it handled. /// The request is answered within `answer` of its arrival /// ([`crate::offload::answering_by`]). -async fn body_in_time(State((wait, answer)): State<(Duration, Duration)>, request: axum::extract::Request, next: axum::middleware::Next) -> Response { +async fn body_in_time(State(app): State, request: axum::extract::Request, next: axum::middleware::Next) -> Response { + let (wait, answer) = (app.service.config.body_wait, Duration::from_secs(app.service.config.answer_secs)); let due = std::time::Instant::now() + answer; let (parts, body) = request.into_parts(); + // Counted open before its body is read: one client's held requests are + // held to their number. A health check is answered whatever that is. + let _open = if parts.uri.path() == "/v1/health" { + None + } else { + let peer = parts.extensions.get::>().map_or(SocketAddr::from(([0, 0, 0, 0], 0)), |c| c.0); + match app.service.open_request(&client(&app, peer, &parts.headers)) { + Ok(open) => Some(open), + Err(Refused(code, why)) => { + let mut r = error(code, &why); + r.headers_mut().insert("retry-after", HeaderValue::from_static("5")); + // The body is not read: the connection goes with the answer. + r.headers_mut().insert("connection", HeaderValue::from_static("close")); + return r; + } + } + }; match tokio::time::timeout(wait, axum::body::to_bytes(body, MAX_BODY + 1)).await { Ok(Ok(bytes)) => { + // Read: its connection is answering now, no longer to be closed for room. + if let Some(OnConn(c)) = parts.extensions.get::() { + c.set(Phase::Answer); + } let large = bytes.len() > LARGE; let r = crate::offload::answering_by(due, next.run(axum::extract::Request::from_parts(parts, axum::body::Body::from(bytes)))).await; // What a large request or answer was parsed into, given back. @@ -1076,30 +1115,143 @@ async fn body_in_time(State((wait, answer)): State<(Duration, Duration)>, reques } } +/// Where a connection is: waiting for a request's header section (or idle +/// between requests), reading a request's body, answering, or being closed. +#[derive(Clone, Copy, PartialEq, Eq)] +enum Phase { + Idle, + Body, + Answer, + Closing, +} + +/// One connection's phase and since when, and how to close it. +struct Conn { + phase: Mutex<(Phase, std::time::Instant)>, + close: tokio::sync::Notify, +} + +impl Conn { + fn set(&self, phase: Phase) { + let mut p = self.phase.lock().unwrap(); + if p.0 != Phase::Closing { + *p = (phase, std::time::Instant::now()); + } + } +} + +/// The connection a request came on, so that its body read moves it on. +#[derive(Clone)] +struct OnConn(Arc); + +/// The connections the service holds. +#[derive(Default)] +struct Conns { + held: Mutex>>, + next: std::sync::atomic::AtomicU64, +} + +impl Conns { + /// One to close so another can be held: the one idle longest, or else + /// the one whose body has been arriving longest, past a second. + fn evict(&self) -> bool { + let held = self.held.lock().unwrap(); + let pick = |phase: Phase, after: Duration| { + held.values().filter_map(|c| { + let p = *c.phase.lock().unwrap(); + (p.0 == phase && p.1.elapsed() >= after).then_some((p.1, Arc::clone(c))) + }).min_by_key(|(since, _)| *since).map(|(_, c)| c) + }; + let Some(c) = pick(Phase::Idle, Duration::ZERO).or_else(|| pick(Phase::Body, Duration::from_secs(1))) else { return false }; + c.phase.lock().unwrap().0 = Phase::Closing; + c.close.notify_one(); + true + } +} + +/// A response's body, its connection idle again once the body is sent. +struct Tracked { + inner: axum::body::Body, + conn: Arc, +} + +impl hyper::body::Body for Tracked { + type Data = axum::body::Bytes; + type Error = axum::Error; + + fn poll_frame(self: std::pin::Pin<&mut Self>, cx: &mut std::task::Context<'_>) -> std::task::Poll, Self::Error>>> { + hyper::body::Body::poll_frame(std::pin::Pin::new(&mut self.get_mut().inner), cx) + } + + fn is_end_stream(&self) -> bool { + hyper::body::Body::is_end_stream(&self.inner) + } + + fn size_hint(&self) -> hyper::body::SizeHint { + hyper::body::Body::size_hint(&self.inner) + } +} + +impl Drop for Tracked { + fn drop(&mut self) { + self.conn.set(Phase::Idle); + } +} + +/// The answer to a connection past [`serve`]'s cap, sent before anything is read. +const FULL: &str = "{\"error\":\"the service holds as many connections as it may; try again shortly [J032]\"}"; + /// Serves `router` on `listener` until the service stops. Each connection is -/// held to `config.header_wait` for a request's header section, and the -/// router to `body_wait` for its body, so a client that stops sending holds a +/// held to `header_wait` for a request's header section, and the router to +/// its body's time for its body, so a client that stops sending holds a /// connection for no longer than those (axum's own loop takes no such limit). -pub async fn serve(listener: tokio::net::TcpListener, router: Router, header_wait: Duration) { +/// At most `max` connections are held: past it the one idle longest is +/// closed (or the one whose body has been arriving longest), so a new +/// connection, a health check say, is always answered; with none to close it +/// is answered `503` at once and closed. `max` is set under the open-file +/// limit, so accepting never runs out of descriptors. +pub async fn serve(listener: tokio::net::TcpListener, router: Router, header_wait: Duration, max: usize) { let service = hyper_util::service::TowerToHyperService::new(router); + let conns = Arc::new(Conns::default()); loop { - let (stream, peer) = match listener.accept().await { + let (mut stream, peer) = match listener.accept().await { Ok(accepted) => accepted, Err(e) => { // Out of descriptors, say: those held free themselves. eprintln!("accepting a connection: {e}"); - tokio::time::sleep(Duration::from_secs(1)).await; + tokio::time::sleep(Duration::from_millis(100)).await; continue; } }; - let service = service.clone(); + if conns.held.lock().unwrap().len() >= max && !conns.evict() { + tokio::spawn(async move { + use tokio::io::AsyncWriteExt; + let answer = format!("HTTP/1.1 503 Service Unavailable\r\ncontent-type: application/json\r\nretry-after: 5\r\nconnection: close\r\ncontent-length: {}\r\n\r\n{FULL}", FULL.len()); + let _ = tokio::time::timeout(Duration::from_secs(2), async { stream.write_all(answer.as_bytes()).await.and(stream.shutdown().await) }).await; + }); + continue; + } + let conn = Arc::new(Conn { phase: Mutex::new((Phase::Idle, std::time::Instant::now())), close: tokio::sync::Notify::new() }); + let id = conns.next.fetch_add(1, std::sync::atomic::Ordering::Relaxed); + conns.held.lock().unwrap().insert(id, Arc::clone(&conn)); + let (service, conns) = (service.clone(), Arc::clone(&conns)); tokio::spawn(async move { + let on = Arc::clone(&conn); let with_peer = hyper::service::service_fn(move |mut r: hyper::Request| { + on.set(Phase::Body); r.extensions_mut().insert(ConnectInfo(peer)); - hyper::service::Service::call(&service, r) + r.extensions_mut().insert(OnConn(Arc::clone(&on))); + let answered = hyper::service::Service::call(&service, r); + let on = Arc::clone(&on); + async move { answered.await.map(|r| r.map(|inner| Tracked { inner, conn: on })) } }); let io = hyper_util::rt::TokioIo::new(stream); - let _ = hyper::server::conn::http1::Builder::new().timer(hyper_util::rt::TokioTimer::new()).header_read_timeout(header_wait).serve_connection(io, with_peer).await; + let served = hyper::server::conn::http1::Builder::new().timer(hyper_util::rt::TokioTimer::new()).header_read_timeout(header_wait).serve_connection(io, with_peer); + tokio::select! { + _ = served => {} + () = conn.close.notified() => {} + } + conns.held.lock().unwrap().remove(&id); }); } } @@ -1139,7 +1291,7 @@ pub fn router(app: Shared) -> Router { .route("/mcp", post(mcp).get(mcp_get)) .route("/", get(index)) .fallback(|| async { error(404, "no such endpoint; GET /v1 lists them") }) - .layer(axum::middleware::from_fn_with_state((app.service.config.body_wait, Duration::from_secs(app.service.config.answer_secs)), body_in_time)) + .layer(axum::middleware::from_fn_with_state(Arc::clone(&app), body_in_time)) .layer(axum::middleware::from_fn_with_state(Arc::clone(&app), guard)) .layer(DefaultBodyLimit::max(MAX_BODY)) .with_state(app) @@ -1175,6 +1327,10 @@ mod tests { answer_secs: 55, client_kept_mib: 128, kept_mib: 2048, + kept_min: Duration::from_secs(600), + client_verifies: 120, + client_requests: 32, + connections: 4096, read_in_child: false, per_client_reads: 2, per_client_checks: 1, diff --git a/wai/wai-quantum-service/src/jobs.rs b/wai/wai-quantum-service/src/jobs.rs index efd00ff0e..7884d5358 100644 --- a/wai/wai-quantum-service/src/jobs.rs +++ b/wai/wai-quantum-service/src/jobs.rs @@ -107,6 +107,18 @@ pub struct Config { /// them; past either, its oldest ended jobs are dropped before their time. pub client_kept_mib: u64, pub kept_mib: u64, + /// How long an ended job that has not been read is kept whatever the + /// caps above say ([`Service::trim`]): a result is never dropped before + /// its client could read it. + pub kept_min: Duration, + /// Receipts one client may have checked in its window + /// (`POST /v1/verify`, `verify_quantum_receipt`), before any is read. + pub client_verifies: usize, + /// Requests one client may have open at once, before any body is read. + pub client_requests: usize, + /// Connections the service holds at once, at most: lowered to what its + /// open-file limit leaves room for ([`crate::http::serve`]). + pub connections: usize, /// How long the service takes to answer a request at most, seconds: every /// wait for a turn, every reading and every wait for a job within it end /// by then, so an answer comes within a proxy's wait for one. @@ -172,6 +184,10 @@ impl Config { answer_secs: env_num("WAI_QSVC_ANSWER_SECS", 55u64).max(1), client_kept_mib: env_num("WAI_QSVC_CLIENT_KEPT_MIB", 128u64), kept_mib: env_num("WAI_QSVC_KEPT_MIB", 2048u64), + kept_min: Duration::from_secs(env_num("WAI_QSVC_KEPT_MIN_SECS", 600u64)), + client_verifies: env_num("WAI_QSVC_CLIENT_VERIFIES", 120usize), + client_requests: env_num("WAI_QSVC_CLIENT_REQUESTS", 32usize).max(1), + connections: env_num("WAI_QSVC_CONNECTIONS", 4096usize).max(16), per_client_reads: env_num("WAI_QSVC_CLIENT_READS", 2usize).max(1), per_client_checks: env_num("WAI_QSVC_CLIENT_CHECKS", 1usize).max(1), header_wait: Duration::from_secs(env_num("WAI_QSVC_HEADER_SECS", 150u64).max(1)), @@ -271,7 +287,9 @@ pub struct Job { pub submitted_ms: u64, pub started_ms: Option, pub finished_ms: Option, - pub result: Option, + /// The result as served, its receipt in it: RFC 8785 text, the same + /// before a restart and after, read only where it is used. + pub result: Option>, pub error: Option, pub peak_rss: u64, pub digests: Digests, @@ -290,6 +308,9 @@ pub struct Job { /// text and its result as read ([`crate::offload::weight`]), counted /// against its client's and the service's caps on what ended jobs keep. kept: u64, + /// Whether its result has been served since it ended: until then the + /// caps on what ended jobs keep pass it by, for [`Config::kept_min`]. + read: bool, } tokio::task_local! { @@ -419,6 +440,40 @@ fn now_ms() -> u64 { SystemTime::now().duration_since(UNIX_EPOCH).map(|d| d.as_millis() as u64).unwrap_or(0) } +/// Where a report's request and result are written in ([`Service::report`]): +/// no request or result holds these, as no JSON text holds a bare NUL. +const REQUEST_HERE: &str = "\u{0}wai.quantum.request\u{0}"; +const RESULT_HERE: &str = "\u{0}wai.quantum.result\u{0}"; + +/// A string as RFC 8785 writes it. +fn canonical_str(s: &str) -> String { + String::from_utf8(canon::canonical(&json!(s))).unwrap_or_default() +} + +/// A result as served, its receipt in it, as RFC 8785 text: written the +/// same way as a job ends and as it is read back after a restart. +fn served_text(result: &Value) -> std::sync::Arc { + String::from_utf8(canon::canonical(result)).unwrap_or_default().into() +} + +/// A client's request open ([`Service::open_request`]); dropped, it is not. +pub struct OpenRequest { + open: Arc>>, + client: String, +} + +impl Drop for OpenRequest { + fn drop(&mut self) { + let mut open = self.open.lock().unwrap(); + if let Some(n) = open.get_mut(&self.client) { + *n -= 1; + if *n == 0 { + open.remove(&self.client); + } + } + } +} + /// The record format a store keeps a job in. pub const RECORD: &str = "wai.quantum.job-record/1"; @@ -429,7 +484,7 @@ impl Job { if let Work::Set { items, .. } = &mut self.work { *items = Vec::new(); } - self.kept = self.request.len() as u64 + self.result.as_ref().map_or(0, crate::offload::weight) + self.error.as_ref().map_or(0, |e| e.len() as u64); + self.kept = self.request.len() as u64 + self.result.as_ref().map_or(0, |r| r.len() as u64) + self.error.as_ref().map_or(0, |e| e.len() as u64); } /// The request, read from its text. @@ -438,8 +493,10 @@ impl Job { } /// The job as the API reports it, but its request. - fn to_json_without_request(&self) -> Value { - let mut o = self.to_json_with(Value::Null); + /// The report without its request and result, each to be written in + /// as the text it is kept as ([`Service::report`]). + fn lean(&self) -> Value { + let mut o = self.to_json_with(Value::Null, false); if let Some(m) = o.as_object_mut() { m.shift_remove("request"); } @@ -447,11 +504,12 @@ impl Job { } /// The job as the API reports it. + #[cfg(test)] pub fn to_json(&self) -> Value { - self.to_json_with(self.request()) + self.to_json_with(self.request(), true) } - fn to_json_with(&self, request: Value) -> Value { + fn to_json_with(&self, request: Value, with_result: bool) -> Value { let mut o = json!({ "id": self.id, "status": self.status.name(), @@ -475,8 +533,8 @@ impl Job { if !self.interruptions.is_empty() { o["interruptions"] = json!(self.interruptions); } - if let Some(r) = &self.result { - o["result"] = r.clone(); + if with_result && let Some(r) = &self.result { + o["result"] = serde_json::from_str(r).unwrap_or(Value::Null); } if let Some(e) = &self.error { o["error"] = json!(e); @@ -551,6 +609,7 @@ pub struct Refused(pub u16, pub String); #[derive(Default)] struct Client { submissions: VecDeque, + verifies: VecDeque, } /// One timed job of a method: its work, the time the engine took over it, @@ -594,6 +653,8 @@ pub struct Service { clients: Mutex>, /// The job each key's digest names. Taken before `jobs` or `queue`. keys: Mutex>, + /// Each client's requests open now ([`Service::open_request`]). + open: Arc>>, /// The turns requests are read in. pub gate: crate::offload::Gate, wake: Notify, @@ -654,6 +715,7 @@ impl Service { queue: Mutex::new(VecDeque::new()), clients: Mutex::new(HashMap::new()), keys: Mutex::new(HashMap::new()), + open: Arc::default(), wake: Notify::new(), memory, workers, @@ -736,7 +798,9 @@ impl Service { _ => { if job.status == Status::Done { s.restored.finished += 1; - s.time(&job); + if let Some(r) = job.result.as_deref().and_then(|t| serde_json::from_str::(t).ok()) { + s.time(&job, &r); + } } } } @@ -797,7 +861,7 @@ impl Service { if rec.get("receipt").is_some() { r["receipt"] = json_of(&rec["receipt"])?; } - result = Some(r); + result = Some(served_text(&r)); } let s = |k: &str| rec[k].as_str().map(str::to_string); let mut job = Job { @@ -824,7 +888,7 @@ impl Service { received: s("received"), interruptions: rec["interruptions"].as_array().into_iter().flatten().filter_map(Value::as_u64).collect(), child: None, - kept: 0, + kept: 0, read: false, }; if job.finished_ms.is_some() { job.ended(); @@ -870,6 +934,55 @@ impl Service { self.in_window(client, true) } + /// One receipt check counted against `client`'s own window for them + /// ([`Config::client_verifies`]), before its body is read: the refusal + /// where it is spent. + pub fn verify_counted(&self, client: &str) -> Result<(), Refused> { + self.verify_window(client, true) + } + + /// Whether `client`'s window for receipt checks has room, counting one + /// where `spend` says so. + pub fn verify_window(&self, client: &str, spend: bool) -> Result<(), Refused> { + let now = Instant::now(); + let name = self.name(client); + let mut clients = self.clients.lock().unwrap(); + let c = clients.entry(name).or_default(); + while c.verifies.front().is_some_and(|t| now.duration_since(*t) > self.config.per_client_window) { + c.verifies.pop_front(); + } + if c.verifies.len() >= self.config.client_verifies { + return Err(Refused(429, tag(refusal::RATE, format!("at most {} receipts checked in {} s; try again shortly", self.config.client_verifies, self.config.per_client_window.as_secs())))); + } + if spend { + c.verifies.push_back(now); + } + Ok(()) + } + + /// A request of `client`'s open, held until it is answered: refused at + /// once, before its body is read, past [`Config::client_requests`] of + /// them, so one client's held connections cannot take every one the + /// service has. + pub fn open_request(&self, client: &str) -> Result { + let mut open = self.open.lock().unwrap(); + let n = open.entry(client.to_string()).or_default(); + if *n >= self.config.client_requests { + return Err(Refused(503, tag(refusal::BUSY, format!("{n} of your requests are open at once, all one client may have; send this when one is answered")))); + } + *n += 1; + Ok(OpenRequest { open: Arc::clone(&self.open), client: client.to_string() }) + } + + /// Whether `client` holds a job under the idempotency key `key`: a + /// resend of it is answered before any limit, so its body is read first; + /// a request under any other key waits for the window like the rest. + pub fn holds_key(&self, client: &str, key: &str) -> bool { + let digest = Self::key_digest(&self.name(client), Key(key)); + let Some(id) = self.keys.lock().unwrap().get(&digest).cloned() else { return false }; + self.jobs.lock().unwrap().get(&id).is_some_and(|j| j.status.holds_key()) + } + /// Whether `client`'s window has room, spending none of it: what a /// request is asked before it waits for a turn. It is counted once it has /// its turn ([`crate::offload`]), so a request told to try again while @@ -906,6 +1019,15 @@ impl Service { Window { limit: self.config.per_client_burst, remaining: self.config.per_client_burst.saturating_sub(live.len()), reset_secs } } + /// What ended jobs not yet read keep, bytes, that the caps pass by + /// ([`Service::trim`]): `name`'s, and everyone's. + fn unread(&self, name: &str) -> (u64, u64) { + let now = now_ms(); + let min = self.config.kept_min.as_millis() as u64; + let jobs = self.jobs.lock().unwrap(); + jobs.values().filter(|j| !j.read && j.finished_ms.is_some_and(|f| now.saturating_sub(f) < min)).fold((0, 0), |(m, a), j| (m + if j.client == name { j.kept } else { 0 }, a + j.kept)) + } + /// The client's active jobs and the memory they hold, MiB. fn held(&self, name: &str) -> (usize, u64) { let jobs = self.jobs.lock().unwrap(); @@ -917,6 +1039,13 @@ impl Service { /// its jobs at once, or a full queue. Checked before a submission is /// counted, so a client told to try again shortly spends nothing doing so. pub fn refused_before_reading(&self, client: &str) -> Option { + let (mine, all) = self.unread(&self.name(client)); + if mine >= self.config.client_kept_mib << 20 { + return Some(Refused(429, tag(refusal::RATE, format!("results of yours not yet read hold about {} MiB, all one client's ended jobs keep; read them (GET /v1/jobs/{{id}}) before sending more, or wait {} s", mine >> 20, self.config.kept_min.as_secs())))); + } + if all >= self.config.kept_mib << 20 { + return Some(Refused(503, tag(refusal::BUSY, format!("results not yet read hold about {} MiB, all the service's ended jobs keep; try again in a few minutes", all >> 20)))); + } let (active, held) = self.held(&self.name(client)); if active >= self.config.per_client_active { return Some(Refused(429, tag(refusal::RATE, format!("{active} of your jobs are queued or running; wait for one to finish (at most {} at once)", self.config.per_client_active)))); @@ -1059,7 +1188,7 @@ impl Service { received, interruptions: Vec::new(), child: None, - kept: 0, + kept: 0, read: false, }; { let mut queue = self.queue.lock().unwrap(); @@ -1079,23 +1208,46 @@ impl Service { Ok(id) } - /// The job's report, if it is known. + /// The job's report, if it is known: [`Service::report`] read, the job + /// counted as read. pub fn get(&self, id: &str) -> Option { - self.jobs.lock().unwrap().get(id).map(Job::to_json) + self.report(id).and_then(|t| serde_json::from_str(&t).ok()) } - /// The job's report as JSON text, its request written in as the text it - /// is kept as, never read: what the HTTP routes answer with, however - /// often a client polls a job whose request is large. + /// The job's report, as [`Service::get`], but not counted as read: for + /// what states a job's status and not its result (a task's state). + pub fn peek(&self, id: &str) -> Option { + self.report_as(id, false).and_then(|t| serde_json::from_str(&t).ok()) + } + + /// The job's report as RFC 8785 text, its request and its result written + /// in as the text each is kept as, never read: what every route answers + /// with, so a job's bytes are the same on each, and before a restart and + /// after. An ended job served counts as read ([`Service::trim`]). pub fn report(&self, id: &str) -> Option { - let (lean, request) = { - let jobs = self.jobs.lock().unwrap(); - let j = jobs.get(id)?; - (j.to_json_without_request(), std::sync::Arc::clone(&j.request)) + self.report_as(id, true) + } + + fn report_as(&self, id: &str, read: bool) -> Option { + let (mut lean, request, result) = { + let mut jobs = self.jobs.lock().unwrap(); + let j = jobs.get_mut(id)?; + if read && j.finished_ms.is_some() { + j.read = true; + } + (j.lean(), std::sync::Arc::clone(&j.request), j.result.clone()) }; - let text = lean.to_string(); - let open = text.strip_suffix('}')?; - Some(format!("{open}{}\"request\":{request}}}", if open.ends_with('{') { "" } else { "," })) + lean["request"] = json!(REQUEST_HERE); + if result.is_some() { + lean["result"] = json!(RESULT_HERE); + } + let mut out = String::from_utf8(canon::canonical(&lean)).ok()?; + // Each where RFC 8785 sorts it among the report's keys. + if let Some(r) = &result { + out = out.replacen(&format!("\"result\":{}", canonical_str(RESULT_HERE)), &format!("\"result\":{r}"), 1); + } + out = out.replacen(&format!("\"request\":{}", canonical_str(REQUEST_HERE)), &format!("\"request\":{request}"), 1); + Some(out) } /// Whether the job `id` is `client`'s. @@ -1143,8 +1295,7 @@ impl Service { } /// Keeps a finished job's timing for the dry run's estimates. - fn time(&self, job: &Job) { - let Some(r) = &job.result else { return }; + fn time(&self, job: &Job, r: &Value) { let mut t = self.timings.lock().unwrap(); let mut push = |method: &str, work: u64, r: &Value, energy: &Value| { let Some(elapsed_ms) = r["elapsed_ms"].as_u64() else { return }; @@ -1354,16 +1505,15 @@ impl Service { }; let finished_ms = now_ms(); let mut kept: (Option, Option) = (None, None); - let outcome = outcome.and_then(|out| self.seal(&request, &work, out, figure, elapsed_ms)).map(|(result, receipt)| { - // Served as the store has it, so it reads the same before a restart and after. - let bare = canon::canonical(&result); - match (self.store.put(&bare), self.store.put(&canon::canonical(&receipt))) { + let outcome = outcome.and_then(|out| self.seal(&request, &work, out, figure, elapsed_ms)).map(|(mut result, receipt)| { + match (self.store.put(&canon::canonical(&result)), self.store.put(&canon::canonical(&receipt))) { (Ok(r), Ok(s)) => kept = (Some(r), Some(s)), (Err(e), _) | (_, Err(e)) => eprintln!("the job store: {id}: its result: {e}"), } - let mut served: Value = serde_json::from_slice(&bare).unwrap_or(result); - served["receipt"] = receipt; - served + // Served as RFC 8785 text, receipt and all, as a restart reads it back. + result["receipt"] = receipt; + let text = served_text(&result); + (result, text) }); // The record as the job ends, written before the end is seen. let record = { @@ -1398,12 +1548,12 @@ impl Service { } j.finished_ms = Some(finished_ms); match outcome { - Ok(served) => { + Ok((served, text)) => { j.status = Status::Done; - j.result = Some(served); + self.time(j, &served); + j.result = Some(text); j.digests.result = kept.0; j.digests.receipt = kept.1; - self.time(j); } Err(e) => { j.status = Status::Failed; @@ -1420,31 +1570,57 @@ impl Service { } } - /// Ended jobs past what they may keep, dropped, the oldest first: each - /// client's past [`Config::client_kept_mib`], then all past - /// [`Config::kept_mib`]. A job is kept for [`Config::retain`], or until - /// these say otherwise. + /// Ended jobs past what they may keep, dropped: each client's past + /// [`Config::client_kept_mib`], its oldest first, then all past + /// [`Config::kept_mib`], the oldest of the client that keeps the most + /// first, so many clients together drop the largest holder's results, + /// not another's. An ended job not yet read is passed by for + /// [`Config::kept_min`] whatever it weighs: a result is never dropped + /// before its client could read it. While such jobs alone are past a cap, + /// new jobs are refused instead ([`Service::refused_before_reading`]). A + /// job is kept for [`Config::retain`], or until these say otherwise. fn trim(&self) { let gone: Vec = { let mut jobs = self.jobs.lock().unwrap(); - let mut ended: Vec<(u64, String, String, u64)> = jobs.values().filter(|j| j.finished_ms.is_some()).map(|j| (j.finished_ms.unwrap_or(0), j.id.clone(), j.client.clone(), j.kept)).collect(); + let now = now_ms(); + let min = self.config.kept_min.as_millis() as u64; + // (ended, id, client, kept, passed by) + let mut ended: Vec<(u64, String, String, u64, bool)> = jobs + .values() + .filter_map(|j| j.finished_ms.map(|f| (f, j.id.clone(), j.client.clone(), j.kept, !j.read && now.saturating_sub(f) < min))) + .collect(); ended.sort(); let mut by_client: HashMap = HashMap::new(); let mut total = 0u64; - for (_, _, c, k) in &ended { + for (_, _, c, k, _) in &ended { *by_client.entry(c.clone()).or_default() += k; total += k; } let (client_cap, cap) = (self.config.client_kept_mib << 20, self.config.kept_mib << 20); let mut gone = Vec::new(); - for (_, id, c, k) in &ended { + let mut left: HashMap> = HashMap::new(); + for (f, id, c, k, passed) in &ended { + if *passed { + continue; + } let mine = by_client.get(c).copied().unwrap_or(0); - if mine > client_cap || total > cap { + if mine > client_cap { by_client.insert(c.clone(), mine - k); total -= k; gone.push(id.clone()); + } else { + left.entry(c.clone()).or_default().push_back((*f, id.clone(), *k)); } } + // The client keeping the most goes first; between two keeping as + // much, the one whose results are newest, so those who came first keep theirs. + while total > cap { + let Some(c) = left.iter().filter_map(|(c, q)| Some((by_client.get(c).copied().unwrap_or(0), q.back()?.0, c))).max().map(|(_, _, c)| c.clone()) else { break }; + let Some((_, id, k)) = left.get_mut(&c).and_then(VecDeque::pop_front) else { break }; + *by_client.entry(c).or_default() -= k; + total -= k; + gone.push(id); + } for id in &gone { jobs.remove(id); } @@ -1815,6 +1991,10 @@ mod tests { answer_secs: 55, client_kept_mib: 128, kept_mib: 2048, + kept_min: Duration::from_secs(600), + client_verifies: 120, + client_requests: 32, + connections: 4096, read_in_child: false, per_client_reads: 2, per_client_checks: 1, @@ -1825,7 +2005,9 @@ mod tests { /// What ended jobs keep is their request's text, not its parsed tree, and /// it is held to its caps: past a client's, its oldest ended jobs go first, - /// and past the service's, the oldest of any; jobs still running are kept. + /// and past the service's, the largest holder's, the newest between equals, + /// so many clients cannot drop an earlier one's; jobs still running are + /// kept, and so is an ended job not yet read, whatever it weighs. #[test] fn ended_jobs_keep_their_request_as_text_within_their_caps() { let s = Service::new(Config { client_kept_mib: 1, kept_mib: 2, ..config() }); @@ -1833,7 +2015,7 @@ mod tests { let job = |client: &str, k: usize, ended: Option| { let request = pad(k); let text = String::from_utf8(canon::canonical(&request)).unwrap(); - let mut j = Job { id: job_id(), client: s.name(client), request: text.into(), plan: plan(1 << 20), work: Work::One, status: if ended.is_some() { Status::Done } else { Status::Running }, submitted_ms: 0, started_ms: None, finished_ms: ended, result: Some(serde_json::json!({ "counts": { "0": 1 } })), error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0 }; + let mut j = Job { id: job_id(), client: s.name(client), request: text.into(), plan: plan(1 << 20), work: Work::One, status: if ended.is_some() { Status::Done } else { Status::Running }, submitted_ms: 0, started_ms: None, finished_ms: ended, result: Some(served_text(&serde_json::json!({ "counts": { "0": 1 } }))), error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0, read: false }; if ended.is_some() { j.ended(); } @@ -1847,13 +2029,30 @@ mod tests { ids.push(j.id.clone()); s.jobs.lock().unwrap().insert(j.id.clone(), j); } + for j in s.jobs.lock().unwrap().values_mut() { + j.read = true; + } s.trim(); - let have: Vec = ids.iter().map(|id| s.get(id).is_some()).collect(); - // a's two oldest ended jobs go for its client's cap, then the oldest - // of all for the service's; the running one stays. - assert_eq!(have, [false, false, false, true, true, true, true], "{have:?}"); + let have: Vec = ids.iter().map(|id| s.jobs.lock().unwrap().contains_key(id)).collect(); + // a's two oldest ended jobs go for its client's cap; then, all four + // clients keeping as much, the newest for the service's (d's); the + // running one stays. + assert_eq!(have, [false, false, true, true, true, true, false], "{have:?}"); let kept: u64 = s.jobs.lock().unwrap().values().filter(|j| j.finished_ms.is_some()).map(|j| j.kept).sum(); assert!(kept <= 2 << 20, "{kept}"); + // An ended job past its client's cap, not yet read, stays until it is. + let big = job("e", 8, Some(now_ms())); + let (big_id, mut bigger) = (big.id.clone(), job("e", 9, Some(now_ms()))); + bigger.kept = 2 << 20; + let bigger_id = bigger.id.clone(); + s.jobs.lock().unwrap().insert(big_id.clone(), big); + s.jobs.lock().unwrap().insert(bigger_id.clone(), bigger); + s.trim(); + assert!(s.jobs.lock().unwrap().contains_key(&bigger_id), "kept until it is read"); + assert!(s.refused_before_reading("e").is_some(), "and the client's next job waits for it to be read"); + assert!(s.report(&bigger_id).is_some()); + s.trim(); + assert!(!s.jobs.lock().unwrap().contains_key(&bigger_id), "read, it goes"); } fn ids_of(s: &Service, client: &str) -> Vec { @@ -2178,7 +2377,7 @@ mod tests { let key = wai_quantum::ed25519_dalek::SigningKey::from_bytes(&[19; 32]); let request = json!({ "qasm": "OPENQASM 3.0; qubit[2] q; h q[0]; cx q[0], q[1];", "shots": 20, "observables": ["Z0 Z1"] }); let (p, mut result) = wai_quantum::engine::job::plan_and_run(&request, 1 << 30).unwrap(); - let mut job = Job { id: job_id(), client: "test".into(), request: request.to_string().into(), plan: p, work: Work::One, status: Status::Queued, submitted_ms: now_ms(), started_ms: None, finished_ms: None, result: None, error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0 }; + let mut job = Job { id: job_id(), client: "test".into(), request: request.to_string().into(), plan: p, work: Work::One, status: Status::Queued, submitted_ms: now_ms(), started_ms: None, finished_ms: None, result: None, error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0, read: false }; let r = crate::mcp::schemas(); let check = |j: &Job| { let v = j.to_json(); @@ -2195,7 +2394,7 @@ mod tests { job.status = Status::Done; job.finished_ms = Some(now_ms()); job.peak_rss = 1 << 20; - job.result = Some(result); + job.result = Some(served_text(&result)); let done = check(&job); assert!(wai_quantum::engine::job::verify(&done).unwrap().0, "{done}"); assert_eq!(done["result"]["schema"], wai_quantum::engine::schema::JOB_RESULT); @@ -2208,7 +2407,7 @@ mod tests { let items = wai_quantum::engine::sets::expand(&set_request).unwrap(); let plans = wai_quantum::engine::sets::plan(&items, 1 << 30).unwrap(); let body = wai_quantum::engine::sets::run(&set_request, 1 << 30, &key, json!({ "unmetered": "a test" })).unwrap(); - let set = Job { id: job_id(), client: "test".into(), request: set_request.to_string().into(), plan: wai_quantum::engine::sets::summary(&plans), work: Work::Set { items, plans }, status: Status::Done, submitted_ms: now_ms(), started_ms: Some(now_ms()), finished_ms: Some(now_ms()), result: Some(body["result"].clone()), error: None, peak_rss: 1 << 20, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0 }; + let set = Job { id: job_id(), client: "test".into(), request: set_request.to_string().into(), plan: wai_quantum::engine::sets::summary(&plans), work: Work::Set { items, plans }, status: Status::Done, submitted_ms: now_ms(), started_ms: Some(now_ms()), finished_ms: Some(now_ms()), result: Some(served_text(&body["result"])), error: None, peak_rss: 1 << 20, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0, read: false }; let done = check(&set); assert_eq!(wai_quantum::engine::sets::verify(&done)["valid"], true, "{done}"); } @@ -2317,8 +2516,8 @@ mod tests { let s = Service::new(config()); let p = Plan { method: "mps".into(), why: "".into(), memory_bytes: 1, work: 1_000_000_000, qubits: 30, target: None, metered: true }; for _ in 0..3 { - let job = Job { id: job_id(), client: "x".into(), request: "{}".into(), plan: p.clone(), work: Work::One, status: Status::Done, submitted_ms: 0, started_ms: None, finished_ms: Some(1), result: Some(json!({ "elapsed_ms": 500, "energy": { "joules_micro": 5_000_000, "energy_uncertainty": { "window_us": 500_000 } } })), error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0 }; - s.time(&job); + let job = Job { id: job_id(), client: "x".into(), request: "{}".into(), plan: p.clone(), work: Work::One, status: Status::Done, submitted_ms: 0, started_ms: None, finished_ms: Some(1), result: Some(served_text(&json!({ "elapsed_ms": 500, "energy": { "joules_micro": 5_000_000, "energy_uncertainty": { "window_us": 500_000 } } }))), error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0, read: false }; + s.time(&job, &serde_json::from_str(job.result.as_deref().unwrap_or("null")).unwrap()); } let (t, _) = s.estimate(&p); assert!(t["basis"].as_str().unwrap().contains("nominal"), "{t}"); @@ -2336,8 +2535,8 @@ mod tests { assert!(t["seconds"].is_null() && t["at_most"] == 5, "{t}"); // Timed jobs here take over, and their metered power gives an energy estimate. for _ in 0..3 { - let job = Job { id: job_id(), client: "x".into(), request: "{}".into(), plan: Plan { work: 1_000_000_000, ..p.clone() }, work: Work::One, status: Status::Done, submitted_ms: 0, started_ms: None, finished_ms: Some(1), result: Some(json!({ "elapsed_ms": 500, "energy": { "joules_micro": 5_000_000, "energy_uncertainty": { "window_us": 500_000 } } })), error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0 }; - s.time(&job); + let job = Job { id: job_id(), client: "x".into(), request: "{}".into(), plan: Plan { work: 1_000_000_000, ..p.clone() }, work: Work::One, status: Status::Done, submitted_ms: 0, started_ms: None, finished_ms: Some(1), result: Some(served_text(&json!({ "elapsed_ms": 500, "energy": { "joules_micro": 5_000_000, "energy_uncertainty": { "window_us": 500_000 } } }))), error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0, read: false }; + s.time(&job, &serde_json::from_str(job.result.as_deref().unwrap_or("null")).unwrap()); } let (t, e) = s.estimate(&p); assert!((t["seconds"].as_f64().unwrap() - 1.0).abs() < 1e-9 && t["basis"].as_str().unwrap().contains("this service timed"), "{t}"); diff --git a/wai/wai-quantum-service/src/main.rs b/wai/wai-quantum-service/src/main.rs index 6a37566b8..c040740a2 100644 --- a/wai/wai-quantum-service/src/main.rs +++ b/wai/wai-quantum-service/src/main.rs @@ -190,9 +190,51 @@ fn without_large_cache() { eprintln!("starting again with the allocator's large cache off: {e}; going on with it on"); } +/// The open-file limit raised as far as the system lets this process: its +/// soft limit to its hard one (on macOS no further than `kern.maxfilesperproc` +/// and `OPEN_MAX`, as an infinite limit is refused there). Logged, and the +/// limit it ends at returned: the connections the service holds are set +/// under it ([`http::serve`]). +fn open_files() -> u64 { + // SAFETY: getrlimit and setrlimit read and write the struct given, and + // sysctlbyname writes at most the size it is told into the integer given. + unsafe { + let mut l = libc::rlimit { rlim_cur: 0, rlim_max: 0 }; + if libc::getrlimit(libc::RLIMIT_NOFILE, &mut l) != 0 { + eprintln!("open files: the limit could not be read; connections are held under 256"); + return 256; + } + let before = l.rlim_cur; + #[allow(unused_mut)] + let mut want = l.rlim_max; + #[cfg(target_os = "macos")] + { + const OPEN_MAX: libc::rlim_t = 10240; + let mut per: libc::c_int = 0; + let mut size = std::mem::size_of::(); + if libc::sysctlbyname(c"kern.maxfilesperproc".as_ptr(), (&raw mut per).cast(), &mut size, std::ptr::null_mut(), 0) == 0 && per > 0 { + want = want.min(per as libc::rlim_t); + } + want = want.min(OPEN_MAX); + } + if want > before { + let raised = libc::rlimit { rlim_cur: want, rlim_max: l.rlim_max }; + if libc::setrlimit(libc::RLIMIT_NOFILE, &raised) == 0 { + eprintln!("open files: {want} (raised from {before}; the hard limit is {})", l.rlim_max); + return want; + } + eprintln!("open files: {before}; raising it to {want} was refused: {}", std::io::Error::last_os_error()); + return before; + } + eprintln!("open files: {before}, the hard limit"); + before + } +} + fn serve() { #[cfg(target_os = "macos")] without_large_cache(); + let files = open_files(); let config = jobs::Config::from_env(); let key_path = receipt::default_path(); // Readers and workers start from what this keeps, so the binary can be @@ -251,6 +293,10 @@ fn serve() { eprintln!("wai-quantum-service {} on {bound}, receipt key {}", env!("CARGO_PKG_VERSION"), app.signer.public_key()); let router = http::router(Arc::clone(&app)); let header_wait = app.service.config.header_wait; - http::serve(listener, router, header_wait).await; + // Connections held under the open-file limit, with room beside them + // for readers' and workers' pipes, the store's files and the listener. + let max = app.service.config.connections.min(files.saturating_sub((files / 4).max(64)) as usize).max(16); + eprintln!("connections: at most {max} at once, {} open requests a client", app.service.config.client_requests); + http::serve(listener, router, header_wait, max).await; }); } diff --git a/wai/wai-quantum-service/src/mcp.rs b/wai/wai-quantum-service/src/mcp.rs index 9621b23f5..05d1ed393 100644 --- a/wai/wai-quantum-service/src/mcp.rs +++ b/wai/wai-quantum-service/src/mcp.rs @@ -491,6 +491,46 @@ fn fail(id: &Value, code: i64, message: &str) -> Value { json!({ "jsonrpc": "2.0", "id": id, "error": { "code": code, "message": message } }) } +/// The answer to a message over 64 KiB from a client past its window, read +/// only for its id, its method, and its tool's name and key (the rest passed +/// over, unread): a tool result under the message's own id, stating why. A +/// resend under a key the client holds, and a receipt checked within the +/// client's own window for checks, are read whole as any message is: `None`. +pub fn past_window(app: &App, who: &str, body: &[u8], why: &str) -> Option { + #[derive(serde::Deserialize)] + struct Args { + idempotency_key: Option, + } + #[derive(serde::Deserialize)] + struct Params { + name: Option, + arguments: Option, + } + #[derive(serde::Deserialize)] + struct Message { + id: Option, + method: Option, + params: Option, + } + // Not a message: read whole, it is answered as one that is not. + let m: Message = serde_json::from_slice(body).ok()?; + let id = m.id.unwrap_or(Value::Null); + if m.method.as_deref() != Some("tools/call") { + return Some(fail(&id, -32000, why)); + } + let (name, key) = m.params.map(|p| (p.name.unwrap_or_default(), p.arguments.and_then(|a| a.idempotency_key))).unwrap_or_default(); + if key.is_some_and(|k| app.service.holds_key(who, &k)) { + return None; + } + if name == "verify_quantum_receipt" { + return match app.service.verify_window(who, false) { + Ok(()) => None, + Err(crate::jobs::Refused(_, w)) => Some(ok(&id, tool_result(format!("refused: {w}"), json!({ "error": w }), true))), + }; + } + Some(ok(&id, finish(&name, &Value::Null, tool_result(format!("refused: {why}"), json!({ "error": why }), true), false, Reading::Unread(why)))) +} + /// A tool's answer: text for the model, the whole object as structured content. fn tool_result(text: String, structured: Value, is_error: bool) -> Value { json!({ "content": [{ "type": "text", "text": text }], "structuredContent": structured, "isError": is_error }) @@ -548,7 +588,7 @@ fn task_job(app: &App, who: &str, task: &str) -> Option { if app.keys.is_some() && !app.service.owned_by(task, who) { return None; } - app.service.get(task) + app.service.peek(task) } fn no_task(id: &Value, app: &App) -> Value { @@ -780,7 +820,7 @@ pub async fn handle(app: &App, who: &str, msg: &Value, base: &str, views: bool) return Some(fail(&id, -32602, "a dry run runs nothing and answers at once: call it without a task")); } match crate::jobs::noting_count(admit_task(app, who, name, args.clone())).await { - (Ok(job), _) => ok(&id, json!({ "task": task_of(&app.service.get(&job).unwrap_or(json!({ "id": job })), app.service.config.retain.as_millis() as u64) })), + (Ok(job), _) => ok(&id, json!({ "task": task_of(&app.service.peek(&job).unwrap_or(json!({ "id": job })), app.service.config.retain.as_millis() as u64) })), (Err(why), counted) => { let mut data = wai_quantum::engine::refusal::refusal_json(&why); data["verdict"] = task_verdict(app, who, args, why.clone(), counted).await; @@ -803,14 +843,16 @@ pub async fn handle(app: &App, who: &str, msg: &Value, base: &str, views: bool) "tasks/cancel" if !active => fail(&id, -32602, &format!("the task has ended ({}): a task that has ended is not cancelled", task_of(&job, ttl)["status"].as_str().unwrap_or(""))), "tasks/cancel" => { app.service.cancel(task); - ok(&id, task_of(&app.service.get(task).unwrap_or(job), ttl)) + ok(&id, task_of(&app.service.peek(task).unwrap_or(job), ttl)) } _ => { // An answer held until the task ends, as long as a reply // may be held here; the HTTP endpoint streams it to a // client that takes an event stream, for as long as it takes. - if active && app.service.wait(task, CALL_WAIT).await.is_some_and(|s| s.active()) { - return Some(fail(&id, -32603, &format!("the task is still working after {} s, longer than one reply is held here: ask tasks/get until it has ended, then tasks/result, or ask for tasks/result with Accept: text/event-stream and the answer waits until it ends", CALL_WAIT.as_secs()))); + // Held within the answer's time from the request's arrival. + let hold = crate::offload::left().map_or(CALL_WAIT, |l| l.min(CALL_WAIT)); + if active && app.service.wait(task, hold).await.is_some_and(|s| s.active()) { + return Some(fail(&id, -32603, &format!("the task is still working after {} s, as long as one reply is held here: ask tasks/get until it has ended, then tasks/result, or ask for tasks/result with Accept: text/event-stream and the answer waits until it ends", hold.as_secs()))); } ok(&id, task_result(app, who, task, base, views).await) } @@ -880,7 +922,9 @@ async fn call(app: &App, who: &str, name: &str, args: Value, base: &str) -> Valu // read after it needs time too. let wait = crate::offload::left().map_or(CALL_WAIT, |l| l.saturating_sub(VERDICT_TIME).min(CALL_WAIT)); app.service.wait(&id, wait).await; - let job = app.service.get(&id).unwrap_or(json!({ "id": id })); + let Some(job) = app.service.get(&id) else { + return tool_result(format!("the job {id} is no longer kept; send the run again"), json!({ "id": id, "error": "no such job" }), true); + }; let failed = job["status"] == "failed"; tool_result(summary(&job), job, failed) } @@ -938,6 +982,9 @@ async fn call(app: &App, who: &str, name: &str, args: Value, base: &str) -> Valu tool_result(line, v, false) } "verify_quantum_receipt" => { + if let Err(crate::jobs::Refused(_, why)) = app.service.verify_counted(who) { + return tool_result(format!("refused: {why}"), json!({ "error": why }), true); + } let (request, result) = (&args["request"], &args["result"]); if result.get("receipt").is_none() { return tool_result("the result carries no receipt".into(), json!({ "valid": false }), true); diff --git a/wai/wai-quantum-service/src/offload.rs b/wai/wai-quantum-service/src/offload.rs index 0e66d1a58..3a378a9c9 100644 --- a/wai/wai-quantum-service/src/offload.rs +++ b/wai/wai-quantum-service/src/offload.rs @@ -140,9 +140,18 @@ pub struct Turn { _pool: OwnedSemaphorePermit, _share: Option, _client: ClientHold, + /// For a lent turn: the client's own share, which the reading moves onto + /// when one of its turns comes free, and how many others are waiting. + lend: Option<(Arc, Arc)>, +} + +#[cfg(test)] +impl Turn { /// Whether it was lent past the client's share, and so lasts at most - /// [`LENT_FOR`]. - pub lent: bool, + /// [`LENT_FOR`] once others wait, unless the client's own turn comes free. + pub fn lent(&self) -> bool { + self.lend.is_some() + } } struct ClientHold { @@ -225,10 +234,10 @@ impl Gate { if let Ok(m) = Arc::clone(&mine).try_acquire_owned() && let Ok(p) = Arc::clone(pool_turns).try_acquire_owned() { - return Ok(Turn { _pool: p, _share: Some(m), _client: client, lent: false }); + return Ok(Turn { _pool: p, _share: Some(m), _client: client, lend: None }); } if let Some(p) = self.lend(pool_turns, &mut client) { - return Ok(Turn { _pool: p, _share: None, _client: client, lent: true }); + return Ok(Turn { _pool: p, _share: None, _client: client, lend: Some((Arc::clone(&mine), Arc::clone(&self.waiting))) }); } if self.waiting.fetch_add(1, Ordering::SeqCst) >= MAX_WAITING { self.waiting.fetch_sub(1, Ordering::SeqCst); @@ -257,12 +266,12 @@ impl Gate { if own.is_none() && let Some(p) = self.lend(pool_turns, &mut client) { - return Ok(Turn { _pool: p, _share: None, _client: client, lent: true }); + return Ok(Turn { _pool: p, _share: None, _client: client, lend: Some((Arc::clone(&mine), Arc::clone(&self.waiting))) }); } } else { tokio::select! { p = &mut pool_wait => match p { - Ok(p) => return Ok(Turn { _pool: p, _share: own, _client: client, lent: false }), + Ok(p) => return Ok(Turn { _pool: p, _share: own, _client: client, lend: None }), Err(_) => break, }, () = step => {} @@ -330,31 +339,99 @@ pub async fn read(config: &Config, turn: Turn, r: Read) -> Result) -> Result { let now = Instant::now(); - // A turn lent past the client's share lasts a quick call's time. - let lent = turn.lent.then(|| now + LENT_FOR); - let by = [Some(now + config.read_timeout), lent, until].into_iter().flatten().min().unwrap_or(now + config.read_timeout); - let why = if lent == Some(by) { - Some(tag(refusal::BUSY, format!("this was read on a turn lent beside your own requests, which lasts {} s, and it took longer; send it again when your others are answered", LENT_FOR.as_secs()))) - } else if until == Some(by) && until < Some(now + config.read_timeout) { - Some(tag(refusal::BUSY, "no time was left to read this within one answer; send it again shortly".to_string())) + let own_by = [Some(now + config.read_timeout), until].into_iter().flatten().min().unwrap_or(now + config.read_timeout); + let own_why = (until == Some(own_by) && until < Some(now + config.read_timeout)).then(|| tag(refusal::BUSY, "no time was left to read this within one answer; send it again shortly".to_string())); + let due = Due::new(own_by, own_why.clone()); + // A turn lent past the client's share lasts a quick call's time, unless + // one of the client's own turns comes free first: the reading then moves + // onto it and has the whole of its time. Past that time it goes on while + // no other request waits for a turn, and is stopped when one does. + let mut keeper = None; + if let Some((own, waiting)) = turn.lend.clone() { + let lent_why = tag(refusal::BUSY, format!("this was read on a turn lent beside your own requests, which lasts {} s once others wait, and it took longer; send it again when your others are answered", LENT_FOR.as_secs())); + let lent_by = { + let own_why = own_why.clone(); + move |t: Instant| if t < own_by { Some(lent_why.clone()) } else { own_why.clone() } + }; + let first = (now + LENT_FOR).min(own_by); + due.set(first, lent_by(first)); + let d = due.clone(); + let held: Arc>> = Arc::default(); + let keep = Arc::clone(&held); + let own_why = own_why.clone(); + keeper = Some(( + tokio::spawn(async move { + let mut until = first; + let mut acquire = Box::pin(own.acquire_owned()); + loop { + tokio::select! { + p = &mut acquire => { + if let Ok(p) = p { + *keep.lock().unwrap() = Some(p); + d.set(own_by, own_why); + } + return; + } + () = tokio::time::sleep_until(until.into()) => { + if until >= own_by || waiting.load(Ordering::SeqCst) > 0 { + return; + } + until = (until + Duration::from_millis(250)).min(own_by); + d.set(until, lent_by(until)); + } + } + } + }), + held, + )); + } + let mut turn = Some(turn); + let out = if config.read_in_child { + in_child(config, &r, &due).await } else { - None - }; - if !config.read_in_child { + // The turn held as long as the thread reads, however long that is. + let held = turn.take(); let read = on_thread(move || { - let _turn = turn; + let _turn = held; run(r) }); - return match tokio::time::timeout(by.saturating_duration_since(Instant::now()), read).await { - Ok(r) => r?, - Err(_) => Err((if why.is_some() { 503 } else { 422 }, why.unwrap_or_else(|| tag(refusal::TOO_LARGE_HERE, format!("reading this request took more than {} s, longer than the service reads for; {LOCAL}", config.read_timeout.as_secs()))))), - }; + tokio::pin!(read); + loop { + let (by, why) = due.get(); + match tokio::time::timeout(by.saturating_duration_since(Instant::now()).min(Duration::from_millis(100)), &mut read).await { + Ok(r) => break r.and_then(|r| r), + Err(_) if Instant::now() < due.get().0 => {} + Err(_) => break Err((if why.is_some() { 503 } else { 422 }, why.unwrap_or_else(|| tag(refusal::TOO_LARGE_HERE, format!("reading this request took more than {} s, longer than the service reads for; {LOCAL}", config.read_timeout.as_secs()))))), + } + } + }; + if let Some((k, held)) = keeper { + k.abort(); + drop(held); } - let out = in_child(config, &r, by, why).await; drop(turn); out } +/// When a reading must end by, and why it is cut short if that is before +/// its own time: moved while it runs, as a lent turn becomes the client's own. +#[derive(Clone)] +struct Due(Arc)>>); + +impl Due { + fn new(by: Instant, why: Option) -> Due { + Due(Arc::new(Mutex::new((by, why)))) + } + + fn set(&self, by: Instant, why: Option) { + *self.0.lock().unwrap() = (by, why); + } + + fn get(&self) -> (Instant, Option) { + self.0.lock().unwrap().clone() + } +} + /// `read`, carried out here: what the child does. pub fn run(r: Read) -> Result { match r { @@ -469,7 +546,7 @@ struct Watch { } impl Watch { - fn start(pid: u32, limit: u64, deadline: Instant) -> Watch { + fn start(pid: u32, limit: u64, due: Due) -> Watch { let done = Arc::new(AtomicBool::new(false)); let over = Arc::new(std::sync::atomic::AtomicU8::new(0)); let (d, o) = (Arc::clone(&done), Arc::clone(&over)); @@ -481,7 +558,7 @@ impl Watch { } let why = if crate::jobs::rss_bytes(pid).is_some_and(|rss| rss > limit) { OVER_MEMORY - } else if Instant::now() > deadline { + } else if Instant::now() > due.get().0 { OVER_TIME } else { continue; @@ -521,13 +598,13 @@ impl Drop for Watch { } } -async fn in_child(config: &Config, r: &Read, by: Instant, short: Option) -> Result { +async fn in_child(config: &Config, r: &Read, due: &Due) -> Result { let exe = crate::jobs::self_exe().map_err(|e| (500, e))?; let mut cmd = crate::jobs::sandboxed(&exe, config.sandbox, "read"); cmd.stdin(Stdio::piped()).stdout(Stdio::piped()).stderr(Stdio::piped()).kill_on_drop(true); let mut child = cmd.spawn().map_err(|e| (503, tag(refusal::BUSY, format!("the service could not start a reader ({e}); try again shortly"))))?; let pid = child.id().unwrap_or(0); - let watch = Watch::start(pid, config.read_mib << 20, by); + let watch = Watch::start(pid, config.read_mib << 20, due.clone()); let text = serde_json::to_vec(r).expect("a reading is JSON"); let mut stdin = child.stdin.take().expect("piped"); let writer = tokio::spawn(async move { @@ -580,7 +657,7 @@ async fn in_child(config: &Config, r: &Read, by: Instant, short: Option) OVER_MEMORY => return refused(format!("reading this request took more than {} MiB, more than the service reads with; {LOCAL}", config.read_mib)), OVER_TIME => { // Cut short for its turn or its answer's time: to try again. - if let Some(why) = short { + if let Some(why) = due.get().1 { return Err((503, why)); } return refused(format!("reading this request took more than {} s, longer than the service reads for; {LOCAL}", config.read_timeout.as_secs())); @@ -681,13 +758,13 @@ mod tests { let mut held = Vec::new(); for _ in 0..share { let t = gate.turn(pool, "a").await.ok().unwrap(); - assert!(!t.lent); + assert!(!t.lent()); held.push(t); } // Lent only while the pool keeps two free for others. let lent = if size - share > LEND_LEAVES { let t = gate.turn(pool, "a").await.ok().unwrap(); - assert!(t.lent, "{pool:?}"); + assert!(t.lent(), "{pool:?}"); Some(t) } else { None @@ -696,7 +773,7 @@ mod tests { assert!(tokio::time::timeout(soon, gate.turn(pool, "a")).await.is_err(), "{pool:?}: a client past its share waits"); let t = Instant::now(); let other = gate.turn(pool, "b").await.ok().unwrap(); - assert!(!other.lent && t.elapsed() < soon, "{pool:?}"); + assert!(!other.lent() && t.elapsed() < soon, "{pool:?}"); drop(other); // Its next goes once one of its turns comes back. let g = &gate; @@ -720,7 +797,7 @@ mod tests { let gate = std::sync::Arc::new(Gate::new(1, 1)); let first = gate.turn(Pool::Checks, "a").await.ok().unwrap(); let lent = gate.turn(Pool::Checks, "a").await.ok().unwrap(); - assert!(lent.lent); + assert!(lent.lent()); let waiters: Vec<_> = (0..CLIENT_WAITING) .map(|_| { let g = std::sync::Arc::clone(&gate); diff --git a/wai/wai-quantum-service/tests/depth.rs b/wai/wai-quantum-service/tests/depth.rs index 84ac4c3ab..73c083d09 100644 --- a/wai/wai-quantum-service/tests/depth.rs +++ b/wai/wai-quantum-service/tests/depth.rs @@ -51,11 +51,42 @@ fn start_with(dir: &Path, burst: u32, env: &[(&str, &str)]) -> (Child, u16) { /// port the system chooses, and says which in its log, so no two services /// started at once (by these tests or anything else on the host) race for one. fn start_from(bin: &Path, dir: &Path, burst: u32, env: &[(&str, &str)]) -> (Child, u16) { + let mut cmd = Command::new(bin); + cmd.arg("serve"); + launch(cmd, dir, burst, env) +} + +/// [`start_with`], the service started by a shell that runs `limit` first +/// (`ulimit -n 256`, say). +#[cfg(unix)] +fn start_limited(dir: &Path, limit: &str, env: &[(&str, &str)]) -> (Child, u16) { + let mut cmd = Command::new("sh"); + cmd.arg("-c").arg(format!("{limit} && exec \"$0\" serve")).arg(env!("CARGO_BIN_EXE_wai-quantum-service")); + launch(cmd, dir, burst_of(env), env) +} + +#[cfg(unix)] +fn burst_of(env: &[(&str, &str)]) -> u32 { + env.iter().find(|(k, _)| *k == "WAI_QSVC_CLIENT_BURST").and_then(|(_, v)| v.parse().ok()).unwrap_or(1000) +} + +/// Every service log written under `dir`. +fn logs(dir: &Path) -> String { + let mut out = String::new(); + for e in std::fs::read_dir(dir).into_iter().flatten().flatten() { + if e.file_name().to_string_lossy().starts_with("service-") { + out += &std::fs::read_to_string(e.path()).unwrap_or_default(); + } + } + out +} + +/// `cmd`, the service, started over the store at `dir`. +fn launch(mut cmd: Command, dir: &Path, burst: u32, env: &[(&str, &str)]) -> (Child, u16) { static STARTS: std::sync::atomic::AtomicUsize = std::sync::atomic::AtomicUsize::new(0); let path = dir.join(format!("service-{}.log", STARTS.fetch_add(1, std::sync::atomic::Ordering::SeqCst))); - let mut child = Command::new(bin) + let mut child = cmd .envs(env.iter().copied()) - .arg("serve") .env("WAI_QSVC_ADDR", "127.0.0.1:0") .env("WAI_QSVC_STORE", dir.join("store")) .env("WAI_QSVC_KEY", dir.join("signing.key")) @@ -914,3 +945,302 @@ fn a_reading_is_stopped_when_no_process_can_be_started() { svc.wait().unwrap(); let _ = std::fs::remove_dir_all(&d); } + +/// This test process's own open-file limit raised to `n`, as far as its hard +/// limit lets it: it holds hundreds of connections to the service. +#[cfg(unix)] +fn room_for(n: u64) { + // SAFETY: getrlimit and setrlimit read and write the struct given. + unsafe { + let mut l = libc::rlimit { rlim_cur: 0, rlim_max: 0 }; + if libc::getrlimit(libc::RLIMIT_NOFILE, &mut l) == 0 && l.rlim_cur < n as libc::rlim_t { + l.rlim_cur = (n as libc::rlim_t).min(l.rlim_max); + libc::setrlimit(libc::RLIMIT_NOFILE, &l); + } + } +} + +/// Whether a health check on a new connection is answered `200` within 3 s. +fn healthy(port: u16) -> bool { + let Ok(mut s) = TcpStream::connect_timeout(&std::net::SocketAddr::from(([127, 0, 0, 1], port)), Duration::from_secs(3)) else { return false }; + let _ = s.set_read_timeout(Some(Duration::from_secs(3))); + let mut out = String::new(); + write!(s, "GET /v1/health HTTP/1.1\r\nHost: 127.0.0.1\r\nConnection: close\r\n\r\n").is_ok() && s.read_to_string(&mut out).is_ok() && out.split_whitespace().nth(1) == Some("200") +} + +/// Connections held open, from one client and from many, partway through a +/// header section or a body, take no more of the service's open files than it +/// has, and a health check is answered throughout: where its limit is 256 and +/// cannot be raised (a launchd service's), and where only its soft limit is +/// 256, which the service raises. Past a client's open requests, the rest +/// are answered `503` at once, unread. +#[cfg(unix)] +#[test] +fn held_connections_leave_room_and_health_answers() { + room_for(8192); + for (name, limit) in [("hard", "ulimit -Sn 256 && ulimit -Hn 256"), ("soft", "ulimit -Sn 256")] { + let d = dir(&format!("files-{name}")); + let (mut svc, port) = start_limited(&d, limit, &[("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1")]); + let answered = |what: &str| { + let ok = (0..15).filter(|_| healthy(port)).count(); + assert_eq!(ok, 15, "{name}: health answered {ok} of 15 {what}: {}", logs(&d)); + }; + // One client, 400 header sections that never end. + let held: Vec = (0..400) + .filter_map(|_| { + let mut s = TcpStream::connect(("127.0.0.1", port)).ok()?; + let _ = s.write_all(b"POST /v1/run HTTP/1.1\r\nHost: 127.0.0.1\r\n"); + Some(s) + }) + .collect(); + answered("beside 400 header sections held open"); + drop(held); + // 400 clients, a body each that stops arriving. + let held: Vec = (0..400) + .filter_map(|i| { + let mut s = TcpStream::connect(("127.0.0.1", port)).ok()?; + let _ = write!(s, "POST /v1/run HTTP/1.1\r\nHost: 127.0.0.1\r\nX-Forwarded-For: 198.51.{}.{}\r\nContent-Type: application/json\r\nContent-Length: 1048576\r\n\r\n{{\"qasm\": \"", i / 200, i % 200 + 1); + Some(s) + }) + .collect(); + std::thread::sleep(Duration::from_millis(1500)); + answered("beside 400 clients' bodies held open"); + drop(held); + // One client, 400 bodies that stop arriving: past its open requests, + // refused at once. + let mut held: Vec = (0..400) + .filter_map(|_| { + let mut s = TcpStream::connect(("127.0.0.1", port)).ok()?; + let _ = write!(s, "POST /v1/run HTTP/1.1\r\nHost: 127.0.0.1\r\nContent-Type: application/json\r\nContent-Length: 1048576\r\n\r\n{{\"qasm\": \""); + Some(s) + }) + .collect(); + answered("beside one client's 400 bodies held open"); + let refused = held + .iter_mut() + .rev() + .take(50) + .map(|s| { + let _ = s.set_read_timeout(Some(Duration::from_secs(2))); + let mut out = [0u8; 512]; + let n = s.read(&mut out).unwrap_or(0); + let text = String::from_utf8_lossy(&out[..n]); + n == 0 || (text.starts_with("HTTP/1.1 503") && text.contains("J032")) + }) + .filter(|refused| *refused) + .count(); + assert_eq!(refused, 50, "{name}: a client's requests past its open ones are refused or closed at once"); + drop(held); + assert!(!logs(&d).contains("Too many open files"), "{name}: {}", logs(&d)); + // The limit, raised where it can be, and logged. + if name == "soft" { + assert!(logs(&d).contains("(raised from 256"), "{}", logs(&d)); + } else { + assert!(logs(&d).contains("open files: 256"), "{}", logs(&d)); + } + alive(port, &mut svc, "connections held open"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); + } +} + +/// A sweep whose result is more than its client's ended jobs may keep, +/// sent within every published limit, is read in full: by a poll, from +/// `/v1/run` and from the MCP set tool. It is kept until it has been read, +/// where it was dropped as it ended and the client was told no such job. +#[test] +fn a_result_larger_than_its_client_keeps_is_kept_until_it_is_read() { + let d = dir("large-result"); + let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_CLIENT_KEPT_MIB", "1")]); + let sweep = |points: usize| json!({ "qasm": "OPENQASM 3.0;\ninput float theta;\nqubit[16] q;\nbit[16] c;\nh q;\nrz(theta) q[0];\nc = measure q;\n", "sweep": { "theta": (0..points).map(|i| 0.01 * (i + 1) as f64).collect::>() }, "shots": 20000, "seed": 7 }); + let done = |id: &str| { + let t = Instant::now(); + loop { + let (code, b) = json_of(port, "GET", &format!("/v1/jobs/{id}?wait=25"), None); + assert_eq!(code, 200, "{id} after {:?}: {b}", t.elapsed()); + if b["status"] == "done" { + return b; + } + assert!(b["status"] == "queued" || b["status"] == "running", "{b}"); + assert!(t.elapsed() < Duration::from_secs(120), "{id}"); + } + }; + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&sweep(10))); + assert_eq!(code, 202, "{b}"); + let job = done(b["id"].as_str().unwrap()); + assert_eq!(job["result"]["items"].as_array().map(Vec::len), Some(10)); + let (code, b) = json_of(port, "POST", "/v1/run", Some(&sweep(10))); + assert!(code == 200 || code == 202, "{code}: {b}"); + let job = done(b["id"].as_str().unwrap()); + assert_eq!(job["result"]["items"].as_array().map(Vec::len), Some(10)); + let (code, b) = json_of(port, "POST", "/mcp", Some(&json!({ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "run_quantum_set", "arguments": sweep(10) } }))); + let sc = &b["result"]["structuredContent"]; + assert_eq!(code, 200, "{b}"); + assert!(b["result"]["isError"] == false && sc["id"].is_string(), "{}", b["result"]["content"]); + let job = done(sc["id"].as_str().unwrap()); + assert_eq!(job["result"]["items"].as_array().map(Vec::len), Some(10)); + alive(port, &mut svc, "large results"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// A job's report is the same bytes on every route that serves it (a poll, +/// its events, the MCP job tool) and after a restart: one RFC 8785 text, +/// its receipt's keys in their order as well as its request's. +#[test] +fn a_job_reads_the_same_on_every_route_and_after_a_restart() { + let d = dir("same-bytes"); + let (mut svc, port) = start(&d, 1000); + let request = json!({ "qasm": "OPENQASM 3.0;\nqubit[2] q;\nh q[0];\ncx q[0], q[1];\n", "shots": 200, "seed": 9, "noise": { "depolarizing_1q": 0.000001, "readout": 1e-7, "depolarizing_2q": 0.0 }, "observables": ["Z0", "0.5 Z1"] }); + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&request)); + assert_eq!(code, 202, "{b}"); + let id = b["id"].as_str().unwrap().to_string(); + let get = || { + let t = Instant::now(); + loop { + let (code, _, text) = http(port, "GET", &format!("/v1/jobs/{id}?wait=25"), None); + assert_eq!(code, 200, "{text}"); + if text.contains("\"status\":\"done\"") { + return text; + } + assert!(t.elapsed() < Duration::from_secs(60), "{text}"); + } + }; + let text = get(); + let v: Value = serde_json::from_str(&text).unwrap(); + assert_eq!(text.as_bytes(), wai_quantum::engine::receipt::canonical(&v), "a report is RFC 8785"); + let (_, _, events) = http(port, "GET", &format!("/v1/jobs/{id}/events"), None); + let event = events.lines().find_map(|l| l.strip_prefix("data: ").filter(|d| d.contains("\"result\""))).unwrap_or_else(|| panic!("{events}")); + assert_eq!(event, text, "the job event"); + let (_, _, mcp) = http(port, "POST", "/mcp", Some(&json!({ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "get_quantum_job", "arguments": { "id": id } } }))); + // Written as RFC 8785 too, so the job in it, beside what MCP adds to it + // (its verdict), is the report's own bytes. + let answer: Value = serde_json::from_str(&mcp).unwrap(); + assert_eq!(mcp.as_bytes(), wai_quantum::engine::receipt::canonical(&answer), "an MCP answer is RFC 8785"); + let mut job = answer["result"]["structuredContent"].clone(); + job.as_object_mut().unwrap().retain(|k, _| v.get(k).is_some()); + assert_eq!(String::from_utf8(wai_quantum::engine::receipt::canonical(&job)).unwrap(), text, "the MCP job tool"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let (mut svc, port) = start(&d, 1000); + let (code, _, again) = http(port, "GET", &format!("/v1/jobs/{id}"), None); + assert_eq!(code, 200, "{again}"); + assert_eq!(again, text, "after a restart"); + alive(port, &mut svc, "a report on every route"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// An MCP message over 64 KiB from a client past its window is answered +/// under its own id, as a tool result that says why; a resend under its key +/// of a job it holds is answered with that job. +#[test] +fn a_large_mcp_message_past_the_window_is_answered_under_its_own_id() { + let d = dir("mcp-large"); + let (mut svc, port) = start(&d, 2); + let big = format!("OPENQASM 3.0;\nqubit[2] q;\n{}", "h q[0];\n".repeat(9000)); + let call = |id: u32, name: &str, args: Value| json_of(port, "POST", "/mcp", Some(&json!({ "jsonrpc": "2.0", "id": id, "method": "tools/call", "params": { "name": name, "arguments": args } }))); + let (code, first) = call(1, "run_quantum_circuit", json!({ "qasm": big, "shots": 10, "idempotency_key": "large-1" })); + assert_eq!(code, 200, "{first}"); + let job = first["result"]["structuredContent"]["id"].as_str().unwrap_or_else(|| panic!("{first}")).to_string(); + let (code, b) = call(2, "critique_qasm", json!({ "qasm": "OPENQASM 3.0; qubit q; h q;" })); + assert_eq!(code, 200, "{b}"); + let (code, b) = call(3, "critique_qasm", json!({ "qasm": big })); + assert_eq!((code, b["id"].as_u64(), b["result"]["isError"].as_bool()), (200, Some(3), Some(true)), "{b}"); + assert!(b["result"]["content"][0]["text"].as_str().is_some_and(|t| t.contains("J030")), "{b}"); + let (code, b) = call(4, "run_quantum_circuit", json!({ "qasm": big, "shots": 10, "idempotency_key": "large-1" })); + assert_eq!((code, b["id"].as_u64()), (200, Some(4)), "{b}"); + assert_eq!(b["result"]["structuredContent"]["id"].as_str(), Some(job.as_str()), "{b}"); + alive(port, &mut svc, "a large MCP message past the window"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// Past its window a client's body is not read, whatever key it carries +/// where the client holds no job under that key; a receipt check has a +/// window of its own, asked before its body is read. +#[test] +fn a_body_is_read_only_once_its_window_or_its_key_allows() { + let d = dir("unread-key"); + let (mut svc, port) = start_with(&d, 0, &[("WAI_QSVC_CLIENT_VERIFIES", "2")]); + let raw = |path: &str, extra: &str, body: &str| { + let mut s = TcpStream::connect(("127.0.0.1", port)).unwrap(); + s.set_read_timeout(Some(Duration::from_secs(30))).unwrap(); + write!(s, "POST {path} HTTP/1.1\r\nHost: 127.0.0.1\r\nContent-Type: application/json\r\n{extra}Content-Length: {}\r\nConnection: close\r\n\r\n{body}", body.len()).unwrap(); + let mut out = String::new(); + let _ = s.read_to_string(&mut out); + out.split_whitespace().nth(1).and_then(|c| c.parse::().ok()).unwrap_or(0) + }; + for path in ["/v1/jobs", "/v1/run"] { + assert_eq!(raw(path, "Idempotency-Key: fresh-key\r\n", "{not json"), 429, "{path}: a fresh key past the window"); + } + let codes: Vec = (0..3).map(|_| raw("/v1/verify", "", "{not json")).collect(); + assert_eq!(codes, [400, 400, 429], "verify's own window"); + alive(port, &mut svc, "bodies left unread"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// `tasks/result` holds its answer within the answer's time from arrival, +/// as every other route does, where it held a fixed 50 s. +#[test] +fn a_task_s_result_is_held_within_the_answer_s_time() { + let d = dir("task-wait"); + let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_ANSWER_SECS", "10"), ("WAI_QSVC_MIN_FREE_PCT", "100")]); + let program = format!("OPENQASM 3.0;\nqubit[25] q;\nh q;\n{}", (0..25).map(|i| format!("t q[{i}];\n")).collect::()); + let (code, made) = json_of(port, "POST", "/mcp", Some(&json!({ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "run_quantum_circuit", "task": {}, "arguments": { "qasm": program, "shots": 10 } } }))); + assert_eq!(code, 200, "{made}"); + let task = made["result"]["task"]["taskId"].as_str().unwrap_or_else(|| panic!("{made}")).to_string(); + let began = Instant::now(); + let (code, b) = json_of(port, "POST", "/mcp", Some(&json!({ "jsonrpc": "2.0", "id": 2, "method": "tasks/result", "params": { "taskId": task } }))); + let took = began.elapsed(); + assert_eq!(code, 200, "{b}"); + assert!(took < Duration::from_secs(12), "answered after {took:?}: {b}"); + assert!(b["error"]["message"].as_str().is_some_and(|m| m.contains("still working")), "{b}"); + let _ = http(port, "DELETE", &format!("/v1/jobs/{task}"), None); + alive(port, &mut svc, "a task's result held"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// A documented QEC call sent beside another check from the same client is +/// answered: a lent turn moves onto the client's own as that comes free, and +/// runs past its 5 s while no one waits, where it was stopped at 5 s. +#[test] +fn a_documented_qec_call_beside_another_check_from_its_client_is_answered() { + let d = dir("lent-qec"); + let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1")]); + let me = "198.51.100.41"; + let decode = json!({ "memory": { "distance": 5, "rounds": 5, "p": 0.005 }, "shots": 200_000, "seed": 1 }); + let critique = json!({ "qasm": "OPENQASM 3.0;\nqubit[2] q;\nh q[0];\ncx q[0], q[1];\n" }); + for pair in [[("/v1/critique", &critique), ("/v1/qec/decode", &decode)], [("/v1/qec/decode", &decode), ("/v1/qec/decode", &decode)]] { + let answers: Vec<(u16, String, f64)> = std::thread::scope(|t| { + let calls: Vec<_> = pair + .iter() + .enumerate() + .map(|(i, (path, body))| { + t.spawn(move || { + std::thread::sleep(Duration::from_millis(30 * i as u64)); + let began = Instant::now(); + let (code, out) = post_as(port, me, path, body); + (code, out, began.elapsed().as_secs_f64()) + }) + }) + .collect(); + calls.into_iter().map(|c| c.join().unwrap()).collect() + }); + for ((path, _), (code, out, took)) in pair.iter().zip(&answers) { + assert_eq!(*code, 200, "{path} after {took:.1} s: {}", out.lines().last().unwrap_or("")); + } + } + alive(port, &mut svc, "QEC calls beside others"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} -- 2.54.0 (Apple Git-157) From a0804db72a255bf57a6a7a7e41a8bdd66e2599cf Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Sun, 8 Nov 2026 11:50:54 -0500 Subject: [PATCH 11/18] wai-quantum-service: no more accepted than the descriptors let go, a lent deadline moved ahead of its watcher, and a job trimmed before its waiters read it - Past the connection cap, a few connections at a time are answered 503 and the rest closed at once, and nothing more is accepted until those closed for room have let their descriptors go: under a limit of 256, 400 connections at once never ran the service out of them. - A lent reading's deadline is moved a second at a time, half a second before it is due, so the watcher never stops it at a time about to move. - A finished job is trimmed before its waiters are woken, so a run's or a tool's own read of it does not leave it to be dropped as read. - The report's placeholders are named outside the registry's name space; the receipt corpus test has a window for its receipts; the OpenAPI test reads operations, not path parameters; the lent-turn test gives a slow host the time its decode takes. Co-Authored-By: Claude Opus 5.5 --- wai/wai-quantum-service/src/discovery.rs | 2 +- wai/wai-quantum-service/src/http.rs | 21 +++++++++++++++++++++ wai/wai-quantum-service/src/jobs.rs | 8 +++++--- wai/wai-quantum-service/src/offload.rs | 6 ++++-- wai/wai-quantum-service/tests/depth.rs | 7 +++++-- wai/wai-quantum-service/tests/verify.rs | 1 + 6 files changed, 37 insertions(+), 8 deletions(-) diff --git a/wai/wai-quantum-service/src/discovery.rs b/wai/wai-quantum-service/src/discovery.rs index cc946f419..076940f1d 100644 --- a/wai/wai-quantum-service/src/discovery.rs +++ b/wai/wai-quantum-service/src/discovery.rs @@ -527,7 +527,7 @@ mod tests { // What any route answers before its handler: a body late, too large, // or a client or the service with as many open as it may have. for (path, item) in d["paths"].as_object().unwrap() { - for (method, op) in item.as_object().unwrap() { + for (method, op) in item.as_object().unwrap().iter().filter(|(m, _)| ["get", "post", "delete"].contains(&m.as_str())) { for code in ["408", "413", "503"] { assert!(op["responses"].get(code).is_some(), "{method} {path} lists no {code}"); } diff --git a/wai/wai-quantum-service/src/http.rs b/wai/wai-quantum-service/src/http.rs index df4c1f279..17194ba06 100644 --- a/wai/wai-quantum-service/src/http.rs +++ b/wai/wai-quantum-service/src/http.rs @@ -1149,8 +1149,13 @@ struct OnConn(Arc); struct Conns { held: Mutex>>, next: std::sync::atomic::AtomicU64, + /// Connections being answered `503` past the cap. + refusing: std::sync::atomic::AtomicUsize, } +/// How many connections past the cap are answered at once; the rest are closed. +const REFUSING: usize = 8; + impl Conns { /// One to close so another can be held: the one idle longest, or else /// the one whose body has been arriving longest, past a second. @@ -1214,6 +1219,12 @@ pub async fn serve(listener: tokio::net::TcpListener, router: Router, header_wai let service = hyper_util::service::TowerToHyperService::new(router); let conns = Arc::new(Conns::default()); loop { + // Connections closed for room close as their tasks next run: no more + // is accepted until they have, so the descriptors held stay under + // the cap and a few beside it, however fast connections arrive. + while conns.held.lock().unwrap().len() + conns.refusing.load(std::sync::atomic::Ordering::SeqCst) >= max + REFUSING { + tokio::time::sleep(Duration::from_millis(5)).await; + } let (mut stream, peer) = match listener.accept().await { Ok(accepted) => accepted, Err(e) => { @@ -1224,10 +1235,20 @@ pub async fn serve(listener: tokio::net::TcpListener, router: Router, header_wai } }; if conns.held.lock().unwrap().len() >= max && !conns.evict() { + // Answered by a few at a time, each for 2 s at most; past them + // closed at once, so refusing takes no more descriptors either. + if conns.refusing.fetch_add(1, std::sync::atomic::Ordering::SeqCst) >= REFUSING { + conns.refusing.fetch_sub(1, std::sync::atomic::Ordering::SeqCst); + drop(stream); + continue; + } + let conns = Arc::clone(&conns); tokio::spawn(async move { use tokio::io::AsyncWriteExt; let answer = format!("HTTP/1.1 503 Service Unavailable\r\ncontent-type: application/json\r\nretry-after: 5\r\nconnection: close\r\ncontent-length: {}\r\n\r\n{FULL}", FULL.len()); let _ = tokio::time::timeout(Duration::from_secs(2), async { stream.write_all(answer.as_bytes()).await.and(stream.shutdown().await) }).await; + drop(stream); + conns.refusing.fetch_sub(1, std::sync::atomic::Ordering::SeqCst); }); continue; } diff --git a/wai/wai-quantum-service/src/jobs.rs b/wai/wai-quantum-service/src/jobs.rs index 7884d5358..5f572cbe6 100644 --- a/wai/wai-quantum-service/src/jobs.rs +++ b/wai/wai-quantum-service/src/jobs.rs @@ -442,8 +442,8 @@ fn now_ms() -> u64 { /// Where a report's request and result are written in ([`Service::report`]): /// no request or result holds these, as no JSON text holds a bare NUL. -const REQUEST_HERE: &str = "\u{0}wai.quantum.request\u{0}"; -const RESULT_HERE: &str = "\u{0}wai.quantum.result\u{0}"; +const REQUEST_HERE: &str = "\u{0}the request here\u{0}"; +const RESULT_HERE: &str = "\u{0}the result here\u{0}"; /// A string as RFC 8785 writes it. fn canonical_str(s: &str) -> String { @@ -1562,8 +1562,10 @@ impl Service { } j.ended(); } - self.changed(id); + // Trimmed before the end is seen: the job is not yet read, so it is + // kept, and whoever waits for it reads it. self.trim(); + self.changed(id); // A large request was read to run and to seal it: given back. if request_len > 256 << 10 { give_back(); diff --git a/wai/wai-quantum-service/src/offload.rs b/wai/wai-quantum-service/src/offload.rs index 3a378a9c9..448c708ab 100644 --- a/wai/wai-quantum-service/src/offload.rs +++ b/wai/wai-quantum-service/src/offload.rs @@ -372,11 +372,13 @@ pub async fn read_until(config: &Config, turn: Turn, r: Read, until: Option { + // Asked half a second before it is due, so the + // watcher never sees a time about to be moved. + () = tokio::time::sleep_until((until - Duration::from_millis(500)).into()) => { if until >= own_by || waiting.load(Ordering::SeqCst) > 0 { return; } - until = (until + Duration::from_millis(250)).min(own_by); + until = (until + Duration::from_secs(1)).min(own_by); d.set(until, lent_by(until)); } } diff --git a/wai/wai-quantum-service/tests/depth.rs b/wai/wai-quantum-service/tests/depth.rs index 73c083d09..11e179966 100644 --- a/wai/wai-quantum-service/tests/depth.rs +++ b/wai/wai-quantum-service/tests/depth.rs @@ -989,10 +989,11 @@ fn held_connections_leave_room_and_health_answers() { let held: Vec = (0..400) .filter_map(|_| { let mut s = TcpStream::connect(("127.0.0.1", port)).ok()?; - let _ = s.write_all(b"POST /v1/run HTTP/1.1\r\nHost: 127.0.0.1\r\n"); + let _ = s.write_all(b"GET /v1/health HTTP/1.1\r\nHost: 127.0.0.1\r\n"); Some(s) }) .collect(); + std::thread::sleep(Duration::from_secs(1)); answered("beside 400 header sections held open"); drop(held); // 400 clients, a body each that stops arriving. @@ -1215,7 +1216,9 @@ fn a_task_s_result_is_held_within_the_answer_s_time() { #[test] fn a_documented_qec_call_beside_another_check_from_its_client_is_answered() { let d = dir("lent-qec"); - let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1")]); + // The QEC, reading and answer times raised, so a slow or loaded host + // runs the decode, however long it takes there, rather than refusing it. + let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1"), ("WAI_QSVC_QEC_SECS", "100"), ("WAI_QSVC_READ_SECS", "100"), ("WAI_QSVC_ANSWER_SECS", "110")]); let me = "198.51.100.41"; let decode = json!({ "memory": { "distance": 5, "rounds": 5, "p": 0.005 }, "shots": 200_000, "seed": 1 }); let critique = json!({ "qasm": "OPENQASM 3.0;\nqubit[2] q;\nh q[0];\ncx q[0], q[1];\n" }); diff --git a/wai/wai-quantum-service/tests/verify.rs b/wai/wai-quantum-service/tests/verify.rs index 9a793465b..cf00c1a93 100644 --- a/wai/wai-quantum-service/tests/verify.rs +++ b/wai/wai-quantum-service/tests/verify.rs @@ -37,6 +37,7 @@ fn start(dir: &Path, port: u16) -> Child { .env("WAI_QSVC_ENERGY", "0") // A set's receipt counts, as a check does: the corpus holds more than a window's. .env("WAI_QSVC_CLIENT_BURST", "100000") + .env("WAI_QSVC_CLIENT_VERIFIES", "100000") .stdout(Stdio::null()) .stderr(Stdio::null()) .spawn() -- 2.54.0 (Apple Git-157) From e02f5b79af9850adb0b9293e17da20172ae70d26 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Sun, 8 Nov 2026 15:52:37 -0500 Subject: [PATCH 12/18] wai-quantum-service: health checks held past the connection cap, long polls and streams capped, a moved lent turn given back, read and unread results counted apart, and MCP ids echoed exactly Fifth review of #676: - Four connections past the cap are held for health checks alone, one request each for 2 s; long polls and event streams are held to WAI_QSVC_CLIENT_STREAMS (4) a client and WAI_QSVC_STREAMS (512, a quarter of the connections at most) in all; past them a long poll is answered at once with the job as it is and a stream is refused 503. - A reading that moves from a lent turn onto its client's own gives the lent one back, and is stopped past 5 s only when other clients wait. - Read and unread results are counted apart: only read ones are dropped before their time, the largest holder's first; a client's next job waits while its unread results, with WAI_QSVC_RESULT_MIB (64) for each running job and the new one, would pass WAI_QSVC_CLIENT_UNREAD_MIB (256), and every next job while all unread hold WAI_QSVC_UNREAD_MIB (8192); a result past 64 MiB fails its job, saying so; KEPT_MIB 4096. - A task's state is read from the lean report; MCP answers are written by serde with RFC 8785's numbers, so ids are echoed exactly and a job keeps its report's bytes. - self-host.md: the 192 cap holds only at a hard limit of 256. Co-Authored-By: Claude Opus 5.5 --- wai/changes/2026-11-06-quantum-parse-depth.md | 9 +- wai/wai-quantum-service/README.md | 13 +- wai/wai-quantum-service/docs/self-host.md | 9 +- wai/wai-quantum-service/src/http.rs | 80 ++++++++- wai/wai-quantum-service/src/jobs.rs | 122 +++++++++++--- wai/wai-quantum-service/src/main.rs | 5 +- wai/wai-quantum-service/src/offload.rs | 81 +++++++-- wai/wai-quantum-service/tests/depth.rs | 159 ++++++++++++++++-- 8 files changed, 419 insertions(+), 59 deletions(-) diff --git a/wai/changes/2026-11-06-quantum-parse-depth.md b/wai/changes/2026-11-06-quantum-parse-depth.md index 9f91ef3b0..48c09d58d 100644 --- a/wai/changes/2026-11-06-quantum-parse-depth.md +++ b/wai/changes/2026-11-06-quantum-parse-depth.md @@ -126,7 +126,14 @@ (4096) connections under it: past them the one idle longest is closed for the new one, so a health check is always answered, or the new one is answered `503` at once; a client may have `WAI_QSVC_CLIENT_REQUESTS` (32) requests open, refused `503` past them before a body is read. `tasks/result` holds its answer within what is left of the answer's - time. Readers and workers start from the service's own + time. Four connections past the cap are held for health checks alone, and long polls and event streams are held to + `WAI_QSVC_CLIENT_STREAMS` (4) a client and `WAI_QSVC_STREAMS` (512, a quarter of the connections at most); past them + a long poll is answered at once with the job as it is. A reading that moves from a lent turn onto its client's own + gives the lent one back. Results read and not yet read are counted apart: only read ones are dropped before their + time, the largest holder's first; a client's next job waits while its unread results, with + `WAI_QSVC_RESULT_MIB` (64) for each job running, would pass `WAI_QSVC_CLIENT_UNREAD_MIB` (256), and every next job + while all unread hold `WAI_QSVC_UNREAD_MIB` (8192); a result past `WAI_QSVC_RESULT_MIB` fails its job, saying so. + MCP answers echo a JSON-RPC id exactly. Readers and workers start from the service's own binary as it was started (Linux: its running image; elsewhere a copy kept beside the signing key), so the binary can be replaced before the restart. A restart reads no program: a job keeps the plan it was admitted with and a set its items, where a queue of slow programs held the port unbound for the sum of their reads. diff --git a/wai/wai-quantum-service/README.md b/wai/wai-quantum-service/README.md index dc86394d2..c082ba614 100644 --- a/wai/wai-quantum-service/README.md +++ b/wai/wai-quantum-service/README.md @@ -197,13 +197,13 @@ A submission is admitted in the order of what each step costs: Reading happens outside the service's own process, as each job's run does: each request is read by a child, this binary run as `wai-quantum-service read`, sandboxed as a worker is. -- **Turns.** Four requests are read at a time (jobs, dry runs and the MCP tools' verdicts), and four checks beside them. A client has its share of each: `WAI_QSVC_CLIENT_READS` (2) of the first and `WAI_QSVC_CLIENT_CHECKS` (1) of the checks, so no client holds a whole pool. Past its share, one more turn is lent to a client while the pool keeps two free for others, so its quick calls (a critique, a lint, an estimate) pass its own slow one. A call on a lent turn moves onto the client's own turn as soon as one comes free, and then has the whole of its time; until then it runs past five seconds only while no other request waits for a turn, and once one does it is refused, `503` and `J032`, to send again. Its other requests wait behind its own turns, in order, so an agent's burst of parallel calls is answered one after another: up to eight of them may wait, at most 32 requests wait in all, each for ten seconds, and past any of these a request is refused, `503` and `J032`, having read and spent nothing, saying whether its own requests or the pool held it up. A client is its address, so people behind one address (an office, a carrier's NAT, a class) share one client's turns and its window: raise the shares, or give such a group an API key of its own on a private deployment. +- **Turns.** Four requests are read at a time (jobs, dry runs and the MCP tools' verdicts), and four checks beside them. A client has its share of each: `WAI_QSVC_CLIENT_READS` (2) of the first and `WAI_QSVC_CLIENT_CHECKS` (1) of the checks, so no client holds a whole pool. Past its share, one more turn is lent to a client while the pool keeps two free for others, so its quick calls (a critique, a lint, an estimate) pass its own slow one. A call on a lent turn moves onto the client's own turn as soon as one comes free, and then has the whole of its time and gives the lent turn back, so the client's quick calls pass it on that one; until then it runs past five seconds only while no other request waits for a turn, and once one does it is refused, `503` and `J032`, to send again. Its other requests wait behind its own turns, in order, so an agent's burst of parallel calls is answered one after another: up to eight of them may wait, at most 32 requests wait in all, each for ten seconds, and past any of these a request is refused, `503` and `J032`, having read and spent nothing, saying whether its own requests or the pool held it up. A client is its address, so people behind one address (an office, a carrier's NAT, a class) share one client's turns and its window: raise the shares, or give such a group an API key of its own on a private deployment. - **Memory.** A child's memory is polled every 50 ms from a thread of its own, and the child is killed once it holds more than `WAI_QSVC_READ_GB` (4); the request is refused, `422` and `J031`. The kill is a signal the service sends itself (`SIGKILL`, to the child's process group, and again every 100 ms until it has ended), so it needs no new process. On macOS the poll reads the child's physical footprint, which counts its compressed and swapped pages; on Linux its resident and swapped pages. A poll is not a kernel limit: a child can pass the cap by what it touches in one interval before it is killed. On Linux the child also holds its own address space to twice the cap and a gigabyte, so an allocation past that fails at once (`422`, `J031`), and the kernel ends a reader or a worker before the service when the host runs short. macOS gives an unprivileged process neither, and has no limit that counts compressed memory. - **Time.** A child that reads for longer than `WAI_QSVC_READ_SECS` (45) is killed and the request refused, `422` and `J031`. Every request is answered within `WAI_QSVC_ANSWER_SECS` (55) of its arrival, under the 60 s a proxy in front commonly waits for an answer: its waits for a turn and its readings end by then, `POST /v1/run` and the MCP run tools wait for their job only for what is left and then answer with the job, still running, to poll, and an MCP answer's verdict is read again only if time is left (its verdict says so where it was not). An error-correction call's shots are measured as they run: each point is set up and sampled, and a pilot of its own shots decoded, before the rest of any; it is refused at once when what is measured says it would take more than `WAI_QSVC_QEC_SECS` (25), and stopped if it runs a fifth past that. - **Its answer.** What a reading gives back comes into the service's memory, which every job shares: one that would take more than `WAI_QSVC_READ_ANSWER_MIB` (128) once read, about 80 bytes a value, is refused in the child before it is written (`422`, `J031`), and the service reads it off its request threads. A dry run gets back the plans and points alone. A check's answer can be that large: a critique of a program of 200,000 gates that each draw a remark (`qubit q; for int i in [1:200000] { x q; }`) is refused so; one of 150,000 is answered. -- **What ended jobs keep.** A job keeps its request as the canonical text the store keeps, and a set lets go of its items once it ends. What ended jobs keep in memory, their requests' text and their results as read, is held to `WAI_QSVC_CLIENT_KEPT_MIB` (128) a client and `WAI_QSVC_KEPT_MIB` (2048) in all; past a client's cap its oldest ended jobs are dropped before their `WAI_QSVC_RETAIN_SECS`, and past the service's, those of the client keeping the most (between two keeping as much, the newer results), so many clients together cannot drop another's. An ended job whose result has not been read is kept whatever it weighs, for `WAI_QSVC_KEPT_MIN_SECS` (600) or until it is read: a result is never dropped before it could be read. While a client's unread results hold its whole cap its next job is refused, `429`, until it reads them, and while all unread results hold the service's, `503`. A request's body is parsed only once its client's window has room (or, for a resend under a key the client holds a job under, first, as it is answered before any limit). A receipt check has a window of its own, `WAI_QSVC_CLIENT_VERIFIES` (120), asked before its body is read. An MCP message over 64 KiB from a client past its window is read only for its id, method, tool and key, and answered under its own id with a tool result that says why; a resend under a key the client holds, and a receipt check, are read whole. A job's report is one RFC 8785 text on every route (a poll, its events, a cancel, the MCP job tools, whose answers are written as RFC 8785 too) and after a restart: its request and its result are written in as the text each is kept as, never parsed again, and the memory a large request or answer was parsed into is given back to the system once it is answered. On macOS the service starts itself with the allocator's large-block cache off (`MallocLargeCache=0`): that cache kept every large block a parsed request had freed, dirty and charged to the service (on arm64, 30 requests of a megabyte kept 1 GiB; with it off, 2 MiB). +- **What ended jobs keep.** A job keeps its request as the canonical text the store keeps, and a set lets go of its items once it ends. What ended jobs keep in memory, their requests' text and their results as RFC 8785 text, is counted in two parts. Results that have been read (or ended more than `WAI_QSVC_KEPT_MIN_SECS`, 600 s, ago) are held to `WAI_QSVC_CLIENT_KEPT_MIB` (128) a client, its oldest dropped first, and `WAI_QSVC_KEPT_MIB` (4096) in all, those of the client keeping the most dropped first (between two keeping as much, the newer); only these are ever dropped before `WAI_QSVC_RETAIN_SECS`. Results not yet read are dropped by neither cap: a client's next job is refused, `429`, while its unread results, with `WAI_QSVC_RESULT_MIB` (64) for each job it has running and the new one, would pass `WAI_QSVC_CLIENT_UNREAD_MIB` (256), and every next job is refused, `503`, while all unread results hold `WAI_QSVC_UNREAD_MIB` (8192). A result larger than `WAI_QSVC_RESULT_MIB` fails its job, saying so. The bounds that remain: memory for ended jobs stays under the two caps, 12 GiB at the defaults, with the results of jobs still running; reaching the unread cap takes 32 clients each holding its whole allowance unread, and holds every next job for at most 600 s; a client is its address, so people behind one address share its allowances, and one of them who leaves 256 MiB unread holds the others' next jobs until it is read or 600 s pass; and an honest client's read result is dropped early only where it, of all clients, keeps the most. A request's body is parsed only once its client's window has room (or, for a resend under a key the client holds a job under, first, as it is answered before any limit). A receipt check has a window of its own, `WAI_QSVC_CLIENT_VERIFIES` (120), asked before its body is read. An MCP message over 64 KiB from a client past its window is read only for its id, method, tool and key, and answered under its own id with a tool result that says why; a resend under a key the client holds, and a receipt check, are read whole. A job's report is one RFC 8785 text on every route (a poll, its events, a cancel, and in the MCP job tools' answers, written with RFC 8785's numbers, the job's own bytes, with each JSON-RPC id echoed exactly) and after a restart: its request and its result are written in as the text each is kept as, never parsed again, and the memory a large request or answer was parsed into is given back to the system once it is answered. On macOS the service starts itself with the allocator's large-block cache off (`MallocLargeCache=0`): that cache kept every large block a parsed request had freed, dirty and charged to the service (on arm64, 30 requests of a megabyte kept 1 GiB; with it off, 2 MiB). - **Ending.** A child that panics, overflows its stack or aborts is answered `500`, and the service runs on. A client that hangs up ends its child. -- **Connections.** A request whose header section takes longer than `WAI_QSVC_HEADER_SECS` (150) to arrive is closed; the same limit closes a connection left idle between requests, so it is set above a proxy's own idle limit (two minutes, commonly), and the proxy, not the service, closes its idle connections. A body that takes longer than `WAI_QSVC_BODY_SECS` (30) is refused, `408`. A client may have `WAI_QSVC_CLIENT_REQUESTS` (32) requests open at once; past it a request is refused, `503`, before its body is read (a health check is answered whatever that is). The service raises its open-file limit to its hard limit as it starts (on macOS no further than `kern.maxfilesperproc` and 10240), says so in its log, and holds at most `WAI_QSVC_CONNECTIONS` (4096) connections, fewer where that limit leaves less room: past them the connection idle longest (or the one whose body has been arriving longest) is closed for the new one, so a health check is always answered, and with none to close a new connection is answered `503` at once. +- **Connections.** A request whose header section takes longer than `WAI_QSVC_HEADER_SECS` (150) to arrive is closed; the same limit closes a connection left idle between requests, so it is set above a proxy's own idle limit (two minutes, commonly), and the proxy, not the service, closes its idle connections. A body that takes longer than `WAI_QSVC_BODY_SECS` (30) is refused, `408`. A client may have `WAI_QSVC_CLIENT_REQUESTS` (32) requests open at once; past it a request is refused, `503`, before its body is read (a health check is answered whatever that is). Long polls and event streams are held to `WAI_QSVC_CLIENT_STREAMS` (4) a client and `WAI_QSVC_STREAMS` (512, at most a quarter of the connections) in all: past them a long poll is answered at once with the job as it is, and an event stream is refused, `503`. Four connections past the connection cap are held for health checks alone, one request each, for 2 s. The service raises its open-file limit to its hard limit as it starts (on macOS no further than `kern.maxfilesperproc` and 10240), says so in its log, and holds at most `WAI_QSVC_CONNECTIONS` (4096) connections, fewer where that limit leaves less room: past them the connection idle longest (or the one whose body has been arriving longest) is closed for the new one, so a health check is always answered, and with none to close a new connection is answered `503` at once. - **Updating.** Readers and workers start from the service's own binary as it was started: on Linux its running image, elsewhere a copy it keeps beside its signing key, named by its digest. So the binary can be replaced before the restart (by a copy over it, or, on Linux, where a running binary cannot be written, by a rename), and the running service goes on with its own until it is restarted. ## Sending a job again @@ -350,7 +350,12 @@ These variables set the limits: | `WAI_QSVC_QEC_SECS` | 25 | the time an error-correction call's shots may take, measured as they run | | `WAI_QSVC_ANSWER_SECS` | 55 | the longest any answer takes, its waits and readings included | | `WAI_QSVC_CLIENT_KEPT_MIB` | 128 | what one client's ended jobs may keep in memory | -| `WAI_QSVC_KEPT_MIB` | 2048 | what all ended jobs may keep in memory | +| `WAI_QSVC_KEPT_MIB` | 4096 | what all ended jobs whose results have been read may keep in memory | +| `WAI_QSVC_CLIENT_UNREAD_MIB` | 256 | what one client's results not yet read may hold, its running jobs counted at `WAI_QSVC_RESULT_MIB` each; past it, its next job is refused (`429`) | +| `WAI_QSVC_UNREAD_MIB` | 8192 | what all results not yet read may hold; past it, every next job is refused (`503`) | +| `WAI_QSVC_RESULT_MIB` | 64 | the largest result one job may give, as served; past it the job fails, saying so | +| `WAI_QSVC_CLIENT_STREAMS` | 4 | long polls (`?wait=`) and event streams one client may hold at once | +| `WAI_QSVC_STREAMS` | 512 | those of all clients, at most a quarter of the connections held | | `WAI_QSVC_KEPT_MIN_SECS` | 600 | how long an ended job not yet read is kept whatever the two caps above say | | `WAI_QSVC_CLIENT_VERIFIES` | 120 | receipts one client may have checked in its window (`POST /v1/verify`, `verify_quantum_receipt`) | | `WAI_QSVC_CLIENT_REQUESTS` | 32 | requests one client may have open at once; past it, `503` before the body is read | diff --git a/wai/wai-quantum-service/docs/self-host.md b/wai/wai-quantum-service/docs/self-host.md index 66d2f2d21..95398f51b 100644 --- a/wai/wai-quantum-service/docs/self-host.md +++ b/wai/wai-quantum-service/docs/self-host.md @@ -56,7 +56,12 @@ Everything is set by environment variables. | `WAI_QSVC_QEC_SECS` | 25 | the time an error-correction call's shots may take, measured as they run | | `WAI_QSVC_ANSWER_SECS` | 55 | the longest any answer takes, its waits and readings included | | `WAI_QSVC_CLIENT_KEPT_MIB` | 128 | what one client's ended jobs may keep in memory | -| `WAI_QSVC_KEPT_MIB` | 2048 | what all ended jobs may keep in memory | +| `WAI_QSVC_KEPT_MIB` | 4096 | what all ended jobs whose results have been read may keep in memory | +| `WAI_QSVC_CLIENT_UNREAD_MIB` | 256 | what one client's results not yet read may hold, its running jobs counted at `WAI_QSVC_RESULT_MIB` each; past it, its next job is refused (`429`) | +| `WAI_QSVC_UNREAD_MIB` | 8192 | what all results not yet read may hold; past it, every next job is refused (`503`) | +| `WAI_QSVC_RESULT_MIB` | 64 | the largest result one job may give, as served; past it the job fails, saying so | +| `WAI_QSVC_CLIENT_STREAMS` | 4 | long polls (`?wait=`) and event streams one client may hold at once | +| `WAI_QSVC_STREAMS` | 512 | those of all clients, at most a quarter of the connections held | | `WAI_QSVC_KEPT_MIN_SECS` | 600 | how long an ended job not yet read is kept whatever the two caps above say | | `WAI_QSVC_CLIENT_VERIFIES` | 120 | receipts one client may have checked in its window (`POST /v1/verify`, `verify_quantum_receipt`) | | `WAI_QSVC_CLIENT_REQUESTS` | 32 | requests one client may have open at once; past it, `503` before the body is read | @@ -140,7 +145,7 @@ Each finished job carries its energy inside the signed result: a counter reading ## Open files -Each connection takes a file. The service raises its soft open-file limit to its hard one as it starts and logs the result (`open files: …`), and holds connections under it with room for its readers' and workers' pipes and its store's files (`connections: at most …`). On macOS a launchd service inherits `launchctl limit maxfiles`, 256 by default; raise it in the service's plist (`SoftResourceLimits` and `HardResourceLimits`, `NumberOfFiles`, say 10240), or the service holds at most 192 connections at once. +Each connection takes a file. The service raises its soft open-file limit to its hard one as it starts and logs the result (`open files: …`), and holds connections under it with room for its readers' and workers' pipes and its store's files (`connections: at most …`). On macOS a launchd service inherits `launchctl limit maxfiles`, soft 256 and hard unlimited by default: the service raises its soft limit itself (to 10240, or `kern.maxfilesperproc` if lower) and holds `WAI_QSVC_CONNECTIONS` (4096). Only where the hard limit is low does the cap fall: at a hard limit of 256 it holds 192 connections, so a plist that sets `HardResourceLimits` (`NumberOfFiles`) should set it to 10240 or more. Whatever the cap, a few connections past it are held for health checks alone, and long polls and event streams take at most a quarter of it. ## Behind a reverse proxy diff --git a/wai/wai-quantum-service/src/http.rs b/wai/wai-quantum-service/src/http.rs index 17194ba06..327f98527 100644 --- a/wai/wai-quantum-service/src/http.rs +++ b/wai/wai-quantum-service/src/http.rs @@ -143,9 +143,26 @@ fn reply(status: u16, body: Value) -> Response { } /// [`reply`], its body JSON text already written. -/// `v` as RFC 8785 text: how a job's report is written on every route. +/// `v` as JSON text, its keys in their order and each fractional number as +/// RFC 8785 writes it (an integer exactly, as read): a job read back from its +/// report, which is RFC 8785, is written as the same bytes, and a JSON-RPC id +/// is echoed exactly, past 2^53 too. pub fn canonical_text(v: &Value) -> String { - String::from_utf8(wai_quantum::engine::receipt::canonical(v)).unwrap_or_else(|_| v.to_string()) + struct Es; + impl serde_json::ser::Formatter for Es { + fn write_f64(&mut self, w: &mut W, x: f64) -> std::io::Result<()> { + w.write_all(wai_quantum::engine::receipt::es_number(x).as_bytes()) + } + fn write_f32(&mut self, w: &mut W, x: f32) -> std::io::Result<()> { + w.write_all(wai_quantum::engine::receipt::es_number(f64::from(x)).as_bytes()) + } + } + let mut out = Vec::new(); + let mut ser = serde_json::Serializer::with_formatter(&mut out, Es); + match serde::Serialize::serialize(v, &mut ser) { + Ok(()) => String::from_utf8(out).unwrap_or_else(|_| v.to_string()), + Err(_) => v.to_string(), + } } fn reply_text(status: u16, body: String) -> Response { @@ -652,11 +669,15 @@ struct Poll { /// A job's report; with `?wait=S` (at most 25 s), held until its status /// changes from what it was, or the wait passes. -async fn job(State(app): State, Path(id): Path, Query(poll): Query) -> Response { +async fn job(State(app): State, ConnectInfo(peer): ConnectInfo, headers: HeaderMap, Path(id): Path, Query(poll): Query) -> Response { let Some(first) = app.service.status(&id) else { return error(404, &format!("no such job (finished jobs are kept for {} s)", app.service.config.retain.as_secs())); }; - if let Some(w) = poll.wait.filter(|&w| w > 0 && first.active()) { + // Held only while its client's and everyone's long polls leave room; + // past them it is answered at once with what the job is now, as a long + // poll may always be. + let held = poll.wait.filter(|&w| w > 0 && first.active()).and_then(|w| Some((w, app.service.hold_stream(&client(&app, peer, &headers))?))); + if let Some((w, _held)) = held { let mut rx = app.service.subscribe(); let deadline = tokio::time::Instant::now() + Duration::from_secs(w).min(POLL_WAIT); while app.service.status(&id) == Some(first) { @@ -675,18 +696,24 @@ async fn job(State(app): State, Path(id): Path, Query(poll): Que /// A job's status as server-sent events: `status` events (`{"id", "status"}`) /// as it changes, then one `job` event with the whole report when it ends, /// and the stream closes. A comment every 15 s keeps an idle stream open. -async fn events(State(app): State, Path(id): Path) -> Response { +async fn events(State(app): State, ConnectInfo(peer): ConnectInfo, headers: HeaderMap, Path(id): Path) -> Response { if app.service.status(&id).is_none() { return error(404, &format!("no such job (finished jobs are kept for {} s)", app.service.config.retain.as_secs())); } + let Some(held) = app.service.hold_stream(&client(&app, peer, &headers)) else { + let mut r = error(503, &tag(refusal::BUSY, "as many event streams and long polls are open as may be, yours or everyone's; poll GET /v1/jobs/{id} instead")); + r.headers_mut().insert("retry-after", HeaderValue::from_static("5")); + return r; + }; struct Follow { app: Shared, id: String, rx: tokio::sync::broadcast::Receiver, last: Option, ended: bool, + _held: crate::jobs::OpenRequest, } - let start = Follow { rx: app.service.subscribe(), app, id, last: None, ended: false }; + let start = Follow { rx: app.service.subscribe(), app, id, last: None, ended: false, _held: held }; let stream = futures_util::stream::unfold(start, |mut f| async move { if f.ended { return None; @@ -1151,8 +1178,15 @@ struct Conns { next: std::sync::atomic::AtomicU64, /// Connections being answered `503` past the cap. refusing: std::sync::atomic::AtomicUsize, + /// Connections past the cap held for a health check alone. + health: std::sync::atomic::AtomicUsize, } +/// Connections past the cap held for a health check alone, each for one +/// request and 2 s at most: a health check is answered however many +/// connections answer long requests. +const HEALTH_ROOM: usize = 4; + /// How many connections past the cap are answered at once; the rest are closed. const REFUSING: usize = 8; @@ -1205,6 +1239,7 @@ impl Drop for Tracked { /// The answer to a connection past [`serve`]'s cap, sent before anything is read. const FULL: &str = "{\"error\":\"the service holds as many connections as it may; try again shortly [J032]\"}"; +const FULL_WHY: &str = "the service holds as many connections as it may; try again shortly [J032]"; /// Serves `router` on `listener` until the service stops. Each connection is /// held to `header_wait` for a request's header section, and the router to @@ -1222,7 +1257,7 @@ pub async fn serve(listener: tokio::net::TcpListener, router: Router, header_wai // Connections closed for room close as their tasks next run: no more // is accepted until they have, so the descriptors held stay under // the cap and a few beside it, however fast connections arrive. - while conns.held.lock().unwrap().len() + conns.refusing.load(std::sync::atomic::Ordering::SeqCst) >= max + REFUSING { + while conns.held.lock().unwrap().len() + conns.refusing.load(std::sync::atomic::Ordering::SeqCst) + conns.health.load(std::sync::atomic::Ordering::SeqCst) >= max + REFUSING + HEALTH_ROOM { tokio::time::sleep(Duration::from_millis(5)).await; } let (mut stream, peer) = match listener.accept().await { @@ -1234,7 +1269,31 @@ pub async fn serve(listener: tokio::net::TcpListener, router: Router, header_wai continue; } }; - if conns.held.lock().unwrap().len() >= max && !conns.evict() { + let full = conns.held.lock().unwrap().len() >= max && !conns.evict(); + if full && conns.health.fetch_add(1, std::sync::atomic::Ordering::SeqCst) < HEALTH_ROOM { + // Held for a health check alone, one request, in 2 s. + let (service, conns) = (service.clone(), Arc::clone(&conns)); + tokio::spawn(async move { + let health_only = hyper::service::service_fn(move |mut r: hyper::Request| { + let call = (r.method() == hyper::Method::GET && r.uri().path() == "/v1/health").then(|| { + r.extensions_mut().insert(ConnectInfo(peer)); + hyper::service::Service::call(&service, r) + }); + async move { + match call { + Some(answered) => answered.await, + None => Ok(error(503, FULL_WHY)), + } + } + }); + let io = hyper_util::rt::TokioIo::new(stream); + let _ = tokio::time::timeout(Duration::from_secs(2), hyper::server::conn::http1::Builder::new().timer(hyper_util::rt::TokioTimer::new()).keep_alive(false).header_read_timeout(Duration::from_secs(2)).serve_connection(io, health_only)).await; + conns.health.fetch_sub(1, std::sync::atomic::Ordering::SeqCst); + }); + continue; + } + if full { + conns.health.fetch_sub(1, std::sync::atomic::Ordering::SeqCst); // Answered by a few at a time, each for 2 s at most; past them // closed at once, so refusing takes no more descriptors either. if conns.refusing.fetch_add(1, std::sync::atomic::Ordering::SeqCst) >= REFUSING { @@ -1349,9 +1408,14 @@ mod tests { client_kept_mib: 128, kept_mib: 2048, kept_min: Duration::from_secs(600), + client_unread_mib: 256, + unread_mib: 8192, + result_mib: 64, client_verifies: 120, client_requests: 32, connections: 4096, + client_streams: 4, + streams: 512, read_in_child: false, per_client_reads: 2, per_client_checks: 1, diff --git a/wai/wai-quantum-service/src/jobs.rs b/wai/wai-quantum-service/src/jobs.rs index 5f572cbe6..3803ba8c4 100644 --- a/wai/wai-quantum-service/src/jobs.rs +++ b/wai/wai-quantum-service/src/jobs.rs @@ -58,7 +58,7 @@ use crate::store::{self, Store}; use serde_json::{Value, json}; use std::collections::{BTreeSet, HashMap, VecDeque}; use std::process::Stdio; -use std::sync::atomic::{AtomicU64, Ordering}; +use std::sync::atomic::{AtomicU64, AtomicUsize, Ordering}; use std::sync::{Arc, Mutex}; use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH}; use tokio::io::{AsyncReadExt, AsyncWriteExt}; @@ -111,6 +111,15 @@ pub struct Config { /// caps above say ([`Service::trim`]): a result is never dropped before /// its client could read it. pub kept_min: Duration, + /// What one client's results not yet read may keep, MiB, its jobs still + /// running counted at [`Config::result_mib`] each: past it its next job + /// is refused until it reads them. Everyone's, [`Config::unread_mib`]: + /// past it every next job is refused. + pub client_unread_mib: u64, + pub unread_mib: u64, + /// The largest result a job may give, MiB, as served: past it the job + /// fails, saying so. + pub result_mib: u64, /// Receipts one client may have checked in its window /// (`POST /v1/verify`, `verify_quantum_receipt`), before any is read. pub client_verifies: usize, @@ -119,6 +128,10 @@ pub struct Config { /// Connections the service holds at once, at most: lowered to what its /// open-file limit leaves room for ([`crate::http::serve`]). pub connections: usize, + /// Long polls (`?wait=`) and event streams one client may hold at once, + /// and all clients: lowered to a quarter of the connections held. + pub client_streams: usize, + pub streams: usize, /// How long the service takes to answer a request at most, seconds: every /// wait for a turn, every reading and every wait for a job within it end /// by then, so an answer comes within a proxy's wait for one. @@ -183,11 +196,16 @@ impl Config { qec_secs: env_num("WAI_QSVC_QEC_SECS", 25u64).max(1), answer_secs: env_num("WAI_QSVC_ANSWER_SECS", 55u64).max(1), client_kept_mib: env_num("WAI_QSVC_CLIENT_KEPT_MIB", 128u64), - kept_mib: env_num("WAI_QSVC_KEPT_MIB", 2048u64), + kept_mib: env_num("WAI_QSVC_KEPT_MIB", 4096u64), + client_unread_mib: env_num("WAI_QSVC_CLIENT_UNREAD_MIB", 256u64), + unread_mib: env_num("WAI_QSVC_UNREAD_MIB", 8192u64), + result_mib: env_num("WAI_QSVC_RESULT_MIB", 64u64), kept_min: Duration::from_secs(env_num("WAI_QSVC_KEPT_MIN_SECS", 600u64)), client_verifies: env_num("WAI_QSVC_CLIENT_VERIFIES", 120usize), client_requests: env_num("WAI_QSVC_CLIENT_REQUESTS", 32usize).max(1), connections: env_num("WAI_QSVC_CONNECTIONS", 4096usize).max(16), + client_streams: env_num("WAI_QSVC_CLIENT_STREAMS", 4usize).max(1), + streams: env_num("WAI_QSVC_STREAMS", 512usize).max(1), per_client_reads: env_num("WAI_QSVC_CLIENT_READS", 2usize).max(1), per_client_checks: env_num("WAI_QSVC_CLIENT_CHECKS", 1usize).max(1), header_wait: Duration::from_secs(env_num("WAI_QSVC_HEADER_SECS", 150u64).max(1)), @@ -655,6 +673,10 @@ pub struct Service { keys: Mutex>, /// Each client's requests open now ([`Service::open_request`]). open: Arc>>, + /// Each client's long polls and event streams held now, and how many all + /// may hold ([`Service::hold_stream`]). + streams: Arc>>, + pub stream_cap: AtomicUsize, /// The turns requests are read in. pub gate: crate::offload::Gate, wake: Notify, @@ -707,6 +729,7 @@ impl Service { let memory = Arc::new(Semaphore::new(usize::try_from(config.budget_mib).unwrap_or(usize::MAX).min(Semaphore::MAX_PERMITS))); let workers = Arc::new(Semaphore::new(config.workers)); let gate = crate::offload::Gate::new(config.per_client_reads, config.per_client_checks); + let streams = config.streams; Service { gate, config, @@ -716,6 +739,8 @@ impl Service { clients: Mutex::new(HashMap::new()), keys: Mutex::new(HashMap::new()), open: Arc::default(), + streams: Arc::default(), + stream_cap: AtomicUsize::new(streams), wake: Notify::new(), memory, workers, @@ -974,6 +999,21 @@ impl Service { Ok(OpenRequest { open: Arc::clone(&self.open), client: client.to_string() }) } + /// A long poll or an event stream of `client`'s held, until it is + /// dropped: none past [`Config::client_streams`] of the client's, or + /// past [`Service::stream_cap`] of everyone's, so they cannot hold every + /// connection the service has. + pub fn hold_stream(&self, client: &str) -> Option { + let mut held = self.streams.lock().unwrap(); + let all: usize = held.values().sum(); + let n = held.entry(client.to_string()).or_default(); + if *n >= self.config.client_streams || all >= self.stream_cap.load(Ordering::SeqCst) { + return None; + } + *n += 1; + Some(OpenRequest { open: Arc::clone(&self.streams), client: client.to_string() }) + } + /// Whether `client` holds a job under the idempotency key `key`: a /// resend of it is answered before any limit, so its body is read first; /// a request under any other key waits for the window like the rest. @@ -1040,11 +1080,15 @@ impl Service { /// counted, so a client told to try again shortly spends nothing doing so. pub fn refused_before_reading(&self, client: &str) -> Option { let (mine, all) = self.unread(&self.name(client)); - if mine >= self.config.client_kept_mib << 20 { - return Some(Refused(429, tag(refusal::RATE, format!("results of yours not yet read hold about {} MiB, all one client's ended jobs keep; read them (GET /v1/jobs/{{id}}) before sending more, or wait {} s", mine >> 20, self.config.kept_min.as_secs())))); + // A job still running may give a result as large as a job may give. + let (running, _) = self.held(&self.name(client)); + let room = mine + (running as u64 + 1) * (self.config.result_mib << 20); + if room > self.config.client_unread_mib << 20 { + let room = room.div_ceil(1 << 20); + return Some(Refused(429, tag(refusal::RATE, format!("results of yours not yet read hold about {} MiB, and with your jobs running and this one they could hold {room} MiB, more than the {} MiB one client's unread results may; read them (GET /v1/jobs/{{id}}) before sending more, or wait {} s", mine >> 20, self.config.client_unread_mib, self.config.kept_min.as_secs())))); } - if all >= self.config.kept_mib << 20 { - return Some(Refused(503, tag(refusal::BUSY, format!("results not yet read hold about {} MiB, all the service's ended jobs keep; try again in a few minutes", all >> 20)))); + if all >= self.config.unread_mib << 20 { + return Some(Refused(503, tag(refusal::BUSY, format!("results not yet read hold about {} MiB, all the service keeps unread; try again in a few minutes", all >> 20)))); } let (active, held) = self.held(&self.name(client)); if active >= self.config.per_client_active { @@ -1214,10 +1258,10 @@ impl Service { self.report(id).and_then(|t| serde_json::from_str(&t).ok()) } - /// The job's report, as [`Service::get`], but not counted as read: for - /// what states a job's status and not its result (a task's state). + /// The job's report but its request and result, not counted as read: + /// for what states a job's status and not its result (a task's state). pub fn peek(&self, id: &str) -> Option { - self.report_as(id, false).and_then(|t| serde_json::from_str(&t).ok()) + self.jobs.lock().unwrap().get(id).map(Job::lean) } /// The job's report as RFC 8785 text, its request and its result written @@ -1505,15 +1549,19 @@ impl Service { }; let finished_ms = now_ms(); let mut kept: (Option, Option) = (None, None); - let outcome = outcome.and_then(|out| self.seal(&request, &work, out, figure, elapsed_ms)).map(|(mut result, receipt)| { - match (self.store.put(&canon::canonical(&result)), self.store.put(&canon::canonical(&receipt))) { - (Ok(r), Ok(s)) => kept = (Some(r), Some(s)), - (Err(e), _) | (_, Err(e)) => eprintln!("the job store: {id}: its result: {e}"), - } + let outcome = outcome.and_then(|out| self.seal(&request, &work, out, figure, elapsed_ms)).and_then(|(mut result, receipt)| { + let bare = canon::canonical(&result); // Served as RFC 8785 text, receipt and all, as a restart reads it back. result["receipt"] = receipt; let text = served_text(&result); - (result, text) + if text.len() as u64 > self.config.result_mib << 20 { + return Err(tag(refusal::TOO_LARGE_HERE, format!("its result is {} MiB as served, more than the {} MiB one job here may give; send fewer points or fewer shots, or run it on your own machine (`wai-quantum run`)", text.len() >> 20, self.config.result_mib))); + } + match (self.store.put(&bare), self.store.put(&canon::canonical(&result["receipt"]))) { + (Ok(r), Ok(s)) => kept = (Some(r), Some(s)), + (Err(e), _) | (_, Err(e)) => eprintln!("the job store: {id}: its result: {e}"), + } + Ok((result, text)) }); // The record as the job ends, written before the end is seen. let record = { @@ -1592,11 +1640,15 @@ impl Service { .filter_map(|j| j.finished_ms.map(|f| (f, j.id.clone(), j.client.clone(), j.kept, !j.read && now.saturating_sub(f) < min))) .collect(); ended.sort(); + // Read results are counted and dropped apart from those not yet + // read, which are dropped by none of this ([`Config::unread_mib`]). let mut by_client: HashMap = HashMap::new(); let mut total = 0u64; - for (_, _, c, k, _) in &ended { - *by_client.entry(c.clone()).or_default() += k; - total += k; + for (_, _, c, k, passed) in &ended { + if !*passed { + *by_client.entry(c.clone()).or_default() += k; + total += k; + } } let (client_cap, cap) = (self.config.client_kept_mib << 20, self.config.kept_mib << 20); let mut gone = Vec::new(); @@ -1994,9 +2046,14 @@ mod tests { client_kept_mib: 128, kept_mib: 2048, kept_min: Duration::from_secs(600), + client_unread_mib: 256, + unread_mib: 8192, + result_mib: 64, client_verifies: 120, client_requests: 32, connections: 4096, + client_streams: 4, + streams: 512, read_in_child: false, per_client_reads: 2, per_client_checks: 1, @@ -2012,7 +2069,7 @@ mod tests { /// kept, and so is an ended job not yet read, whatever it weighs. #[test] fn ended_jobs_keep_their_request_as_text_within_their_caps() { - let s = Service::new(Config { client_kept_mib: 1, kept_mib: 2, ..config() }); + let s = Service::new(Config { client_kept_mib: 1, kept_mib: 2, client_unread_mib: 64, ..config() }); let pad = |k: usize| serde_json::json!({ "qasm": "OPENQASM 3.0; qubit q; h q;", "pad": vec![k; 300_000] }); let job = |client: &str, k: usize, ended: Option| { let request = pad(k); @@ -2057,6 +2114,33 @@ mod tests { assert!(!s.jobs.lock().unwrap().contains_key(&bigger_id), "read, it goes"); } + /// Past the service's cap only read results go, the largest holder's + /// first, so results not yet read survive however many clients press + /// on it; and a client's next job is refused while its unread results, + /// with what its running jobs may give, would pass its allowance. + #[test] + fn read_results_go_before_unread_ones_and_unread_ones_hold_their_client_s_next_job() { + let s = Service::new(Config { client_kept_mib: 4, kept_mib: 2, client_unread_mib: 64, result_mib: 64, ..config() }); + let job = |client: &str, ended: u64, read: bool| { + let text = String::from_utf8(canon::canonical(&serde_json::json!({ "qasm": "OPENQASM 3.0; qubit q; h q;", "pad": vec![1; 300_000] }))).unwrap(); + let mut j = Job { id: job_id(), client: s.name(client), request: text.into(), plan: plan(1 << 20), work: Work::One, status: Status::Done, submitted_ms: 0, started_ms: None, finished_ms: Some(ended), result: None, error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0, read }; + j.ended(); + let id = j.id.clone(); + s.jobs.lock().unwrap().insert(id.clone(), j); + id + }; + let now = now_ms(); + // An honest client's unread result, then four others' read ones. + let honest = job("honest", now, false); + let others: Vec = (0..4).map(|i| job(&format!("other-{i}"), now + 1 + i, true)).collect(); + s.trim(); + assert!(s.jobs.lock().unwrap().contains_key(&honest), "an unread result is not dropped for others' read ones"); + let left = others.iter().filter(|id| s.jobs.lock().unwrap().contains_key(*id)).count(); + assert!(left < 4, "read results go past the service's cap"); + assert!(s.refused_before_reading("honest").is_some(), "its unread result and a next job's room pass its allowance"); + assert!(s.refused_before_reading("fresh").is_none(), "another client is not refused for it"); + } + fn ids_of(s: &Service, client: &str) -> Vec { let name = s.name(client); s.jobs.lock().unwrap().values().filter(|j| j.client == name).map(|j| j.id.clone()).collect() diff --git a/wai/wai-quantum-service/src/main.rs b/wai/wai-quantum-service/src/main.rs index c040740a2..3ee295f74 100644 --- a/wai/wai-quantum-service/src/main.rs +++ b/wai/wai-quantum-service/src/main.rs @@ -296,7 +296,10 @@ fn serve() { // Connections held under the open-file limit, with room beside them // for readers' and workers' pipes, the store's files and the listener. let max = app.service.config.connections.min(files.saturating_sub((files / 4).max(64)) as usize).max(16); - eprintln!("connections: at most {max} at once, {} open requests a client", app.service.config.client_requests); + // Long polls and event streams held to a quarter of the connections. + let streams = app.service.config.streams.min(max / 4).max(1); + app.service.stream_cap.store(streams, std::sync::atomic::Ordering::SeqCst); + eprintln!("connections: at most {max} at once, {} open requests a client; {streams} long polls and event streams, {} a client", app.service.config.client_requests, app.service.config.client_streams); http::serve(listener, router, header_wait, max).await; }); } diff --git a/wai/wai-quantum-service/src/offload.rs b/wai/wai-quantum-service/src/offload.rs index 448c708ab..327175d68 100644 --- a/wai/wai-quantum-service/src/offload.rs +++ b/wai/wai-quantum-service/src/offload.rs @@ -116,6 +116,8 @@ struct Share { turns: Arc, requests: usize, lent: usize, + /// Of its requests, those waiting for a turn. + waiting: usize, } /// The turns requests are read in: the service's two pools, each client's @@ -142,7 +144,7 @@ pub struct Turn { _client: ClientHold, /// For a lent turn: the client's own share, which the reading moves onto /// when one of its turns comes free, and how many others are waiting. - lend: Option<(Arc, Arc)>, + lend: Option<(Arc, Others)>, } #[cfg(test)] @@ -157,12 +159,35 @@ impl Turn { struct ClientHold { clients: Clients, key: (Pool, String), - lent: bool, + lent: Arc, } impl ClientHold { fn lend(&mut self) { - self.lent = true; + self.lent.store(true, Ordering::SeqCst); + } + + /// How to give its lent turn back while it is held ([`Unlend`]). + fn unlend(&self) -> Unlend { + Unlend { clients: Arc::clone(&self.clients), key: self.key.clone(), lent: Arc::clone(&self.lent) } + } +} + +/// A lent turn given back once its reading has moved onto the client's own +/// turn: the client may be lent another, so its quick calls pass the long one. +struct Unlend { + clients: Clients, + key: (Pool, String), + lent: Arc, +} + +impl Unlend { + fn run(&self) { + if self.lent.swap(false, Ordering::SeqCst) + && let Some(s) = self.clients.lock().unwrap().get_mut(&self.key) + { + s.lent -= 1; + } } } @@ -171,7 +196,7 @@ impl Drop for ClientHold { let mut clients = self.clients.lock().unwrap(); if let Some(s) = clients.get_mut(&self.key) { s.requests -= 1; - if self.lent { + if self.lent.load(Ordering::SeqCst) { s.lent -= 1; } if s.requests == 0 { @@ -181,11 +206,30 @@ impl Drop for ClientHold { } } -struct Waiting(Arc); +struct Waiting(Arc, Clients, (Pool, String)); impl Drop for Waiting { fn drop(&mut self) { self.0.fetch_sub(1, Ordering::SeqCst); + if let Some(s) = self.1.lock().unwrap().get_mut(&self.2) { + s.waiting -= 1; + } + } +} + +/// Whether requests of other clients than one are waiting for a turn: a +/// turn lent to it goes back once they are, not for its own requests. +#[derive(Clone)] +pub struct Others { + waiting: Arc, + clients: Clients, + key: (Pool, String), +} + +impl Others { + fn waiting(&self) -> bool { + let mine = self.clients.lock().unwrap().get(&self.key).map_or(0, |s| s.waiting); + self.waiting.load(Ordering::SeqCst) > mine } } @@ -223,12 +267,12 @@ impl Gate { let key = (pool, who.to_string()); let (mine, mut client) = { let mut clients = self.clients.lock().unwrap(); - let s = clients.entry(key.clone()).or_insert_with(|| Share { turns: Arc::new(Semaphore::new(share)), requests: 0, lent: 0 }); + let s = clients.entry(key.clone()).or_insert_with(|| Share { turns: Arc::new(Semaphore::new(share)), requests: 0, lent: 0, waiting: 0 }); if s.requests >= share + 1 + CLIENT_WAITING { return Err(busy(format!("{} of your requests are being read or waiting to be; send the next when one is answered", s.requests))); } s.requests += 1; - (Arc::clone(&s.turns), ClientHold { clients: Arc::clone(&self.clients), key, lent: false }) + (Arc::clone(&s.turns), ClientHold { clients: Arc::clone(&self.clients), key, lent: Arc::default() }) }; // A turn of the client's own, and the pool's. if let Ok(m) = Arc::clone(&mine).try_acquire_owned() @@ -237,13 +281,19 @@ impl Gate { return Ok(Turn { _pool: p, _share: Some(m), _client: client, lend: None }); } if let Some(p) = self.lend(pool_turns, &mut client) { - return Ok(Turn { _pool: p, _share: None, _client: client, lend: Some((Arc::clone(&mine), Arc::clone(&self.waiting))) }); + { + let others = self.others(&client.key); + return Ok(Turn { _pool: p, _share: None, _client: client, lend: Some((Arc::clone(&mine), others)) }); + } } if self.waiting.fetch_add(1, Ordering::SeqCst) >= MAX_WAITING { self.waiting.fetch_sub(1, Ordering::SeqCst); return Err(busy(full)); } - let _waiting = Waiting(Arc::clone(&self.waiting)); + if let Some(s) = self.clients.lock().unwrap().get_mut(&client.key) { + s.waiting += 1; + } + let _waiting = Waiting(Arc::clone(&self.waiting), Arc::clone(&self.clients), client.key.clone()); let until = until.min(Instant::now() + WAIT); // Each acquisition kept across the loop, so its place in line is kept. let mut own: Option = None; @@ -266,7 +316,10 @@ impl Gate { if own.is_none() && let Some(p) = self.lend(pool_turns, &mut client) { - return Ok(Turn { _pool: p, _share: None, _client: client, lend: Some((Arc::clone(&mine), Arc::clone(&self.waiting))) }); + { + let others = self.others(&client.key); + return Ok(Turn { _pool: p, _share: None, _client: client, lend: Some((Arc::clone(&mine), others)) }); + } } } else { tokio::select! { @@ -285,6 +338,10 @@ impl Gate { })) } + fn others(&self, key: &(Pool, String)) -> Others { + Others { waiting: Arc::clone(&self.waiting), clients: Arc::clone(&self.clients), key: key.clone() } + } + /// A turn lent to `client` past its share, if the pool can spare one and /// still keep [`LEND_LEAVES`] free, and the client holds none lent. fn lend(&self, pool_turns: &Arc, client: &mut ClientHold) -> Option { @@ -348,6 +405,7 @@ pub async fn read_until(config: &Config, turn: Turn, r: Read, until: Option { - if until >= own_by || waiting.load(Ordering::SeqCst) > 0 { + if until >= own_by || waiting.waiting() { return; } until = (until + Duration::from_secs(1)).min(own_by); diff --git a/wai/wai-quantum-service/tests/depth.rs b/wai/wai-quantum-service/tests/depth.rs index 11e179966..ba010a11e 100644 --- a/wai/wai-quantum-service/tests/depth.rs +++ b/wai/wai-quantum-service/tests/depth.rs @@ -960,6 +960,31 @@ fn room_for(n: u64) { } } +/// How many connections this test process can hold open beside what it +/// needs for itself and its health checks: `want`, or fewer where its own +/// open-file limit is low (a whole suite run under `ulimit -n 256`). +#[cfg(unix)] +fn can_hold(want: usize) -> usize { + // SAFETY: getrlimit writes the struct given. + let soft = unsafe { + let mut l = libc::rlimit { rlim_cur: 0, rlim_max: 0 }; + if libc::getrlimit(libc::RLIMIT_NOFILE, &mut l) == 0 { l.rlim_cur as usize } else { 256 } + }; + // Under a low limit of its own the suite's tests share it, four at a time. + if soft < 1024 { want.min(soft / 4) } else { want } +} + +/// Whether this test process's hard open-file limit lets a child raise its +/// soft one past 256. +#[cfg(unix)] +fn can_raise() -> bool { + // SAFETY: getrlimit writes the struct given. + unsafe { + let mut l = libc::rlimit { rlim_cur: 0, rlim_max: 0 }; + libc::getrlimit(libc::RLIMIT_NOFILE, &mut l) == 0 && l.rlim_max > 256 + } +} + /// Whether a health check on a new connection is answered `200` within 3 s. fn healthy(port: u16) -> bool { let Ok(mut s) = TcpStream::connect_timeout(&std::net::SocketAddr::from(([127, 0, 0, 1], port)), Duration::from_secs(3)) else { return false }; @@ -986,7 +1011,8 @@ fn held_connections_leave_room_and_health_answers() { assert_eq!(ok, 15, "{name}: health answered {ok} of 15 {what}: {}", logs(&d)); }; // One client, 400 header sections that never end. - let held: Vec = (0..400) + let n = can_hold(400); + let held: Vec = (0..n) .filter_map(|_| { let mut s = TcpStream::connect(("127.0.0.1", port)).ok()?; let _ = s.write_all(b"GET /v1/health HTTP/1.1\r\nHost: 127.0.0.1\r\n"); @@ -997,7 +1023,7 @@ fn held_connections_leave_room_and_health_answers() { answered("beside 400 header sections held open"); drop(held); // 400 clients, a body each that stops arriving. - let held: Vec = (0..400) + let held: Vec = (0..n) .filter_map(|i| { let mut s = TcpStream::connect(("127.0.0.1", port)).ok()?; let _ = write!(s, "POST /v1/run HTTP/1.1\r\nHost: 127.0.0.1\r\nX-Forwarded-For: 198.51.{}.{}\r\nContent-Type: application/json\r\nContent-Length: 1048576\r\n\r\n{{\"qasm\": \"", i / 200, i % 200 + 1); @@ -1009,7 +1035,7 @@ fn held_connections_leave_room_and_health_answers() { drop(held); // One client, 400 bodies that stop arriving: past its open requests, // refused at once. - let mut held: Vec = (0..400) + let mut held: Vec = (0..n) .filter_map(|_| { let mut s = TcpStream::connect(("127.0.0.1", port)).ok()?; let _ = write!(s, "POST /v1/run HTTP/1.1\r\nHost: 127.0.0.1\r\nContent-Type: application/json\r\nContent-Length: 1048576\r\n\r\n{{\"qasm\": \""); @@ -1020,7 +1046,7 @@ fn held_connections_leave_room_and_health_answers() { let refused = held .iter_mut() .rev() - .take(50) + .take(n.saturating_sub(40).min(50)) .map(|s| { let _ = s.set_read_timeout(Some(Duration::from_secs(2))); let mut out = [0u8; 512]; @@ -1030,11 +1056,11 @@ fn held_connections_leave_room_and_health_answers() { }) .filter(|refused| *refused) .count(); - assert_eq!(refused, 50, "{name}: a client's requests past its open ones are refused or closed at once"); + assert_eq!(refused, n.saturating_sub(40).min(50), "{name}: a client's requests past its open ones are refused or closed at once"); drop(held); assert!(!logs(&d).contains("Too many open files"), "{name}: {}", logs(&d)); // The limit, raised where it can be, and logged. - if name == "soft" { + if name == "soft" && can_raise() { assert!(logs(&d).contains("(raised from 256"), "{}", logs(&d)); } else { assert!(logs(&d).contains("open files: 256"), "{}", logs(&d)); @@ -1116,13 +1142,9 @@ fn a_job_reads_the_same_on_every_route_and_after_a_restart() { let event = events.lines().find_map(|l| l.strip_prefix("data: ").filter(|d| d.contains("\"result\""))).unwrap_or_else(|| panic!("{events}")); assert_eq!(event, text, "the job event"); let (_, _, mcp) = http(port, "POST", "/mcp", Some(&json!({ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "get_quantum_job", "arguments": { "id": id } } }))); - // Written as RFC 8785 too, so the job in it, beside what MCP adds to it - // (its verdict), is the report's own bytes. - let answer: Value = serde_json::from_str(&mcp).unwrap(); - assert_eq!(mcp.as_bytes(), wai_quantum::engine::receipt::canonical(&answer), "an MCP answer is RFC 8785"); - let mut job = answer["result"]["structuredContent"].clone(); - job.as_object_mut().unwrap().retain(|k, _| v.get(k).is_some()); - assert_eq!(String::from_utf8(wai_quantum::engine::receipt::canonical(&job)).unwrap(), text, "the MCP job tool"); + // The job in the MCP answer is the report's own bytes, what MCP adds to + // it (its verdict) after them. + assert!(mcp.contains(&text[..text.len() - 1]), "the MCP job tool: {mcp}"); svc.kill().unwrap(); svc.wait().unwrap(); let (mut svc, port) = start(&d, 1000); @@ -1247,3 +1269,114 @@ fn a_documented_qec_call_beside_another_check_from_its_client_is_answered() { svc.wait().unwrap(); let _ = std::fs::remove_dir_all(&d); } + +/// Long polls and event streams, 210 of them from 8 clients, where the +/// service's open-file limit is 256 and cannot be raised, hold no more than +/// their share of its connections: a health check is answered throughout, +/// and a long poll past them is answered at once with the job as it is. +#[cfg(unix)] +#[test] +fn long_polls_and_streams_leave_a_health_check_answered() { + room_for(8192); + let d = dir("long-polls"); + let (mut svc, port) = start_limited(&d, "ulimit -Sn 256 && ulimit -Hn 256", &[("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1")]); + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&long_job())); + assert_eq!(code, 202, "{b}"); + let id = b["id"].as_str().unwrap().to_string(); + let held: Vec = (0..can_hold(210)) + .filter_map(|i| { + let mut s = TcpStream::connect(("127.0.0.1", port)).ok()?; + let path = if i % 3 == 0 { format!("/v1/jobs/{id}/events") } else { format!("/v1/jobs/{id}?wait=25") }; + let _ = write!(s, "GET {path} HTTP/1.1\r\nHost: 127.0.0.1\r\nX-Forwarded-For: 198.51.100.{}\r\n\r\n", i % 8 + 1); + Some(s) + }) + .collect(); + std::thread::sleep(Duration::from_secs(1)); + let ok = (0..25).filter(|_| healthy(port)).count(); + assert_eq!(ok, 25, "health answered {ok} of 25 beside 210 long polls and streams: {}", logs(&d)); + drop(held); + let _ = http(port, "DELETE", &format!("/v1/jobs/{id}"), None); + alive(port, &mut svc, "long polls and streams"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// The README's own burst from one client: a decode, a threshold and six +/// critiques at once, every one answered; a long check that moves onto the +/// client's own turn gives back its lent one, so the critiques pass it. +#[test] +fn the_readme_s_burst_from_one_client_is_answered() { + let d = dir("readme-burst"); + let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1"), ("WAI_QSVC_QEC_SECS", "100"), ("WAI_QSVC_READ_SECS", "100"), ("WAI_QSVC_ANSWER_SECS", "110")]); + let me = "198.51.100.77"; + let decode = json!({ "memory": { "distance": 5, "rounds": 5, "p": 0.003 }, "shots": 20000, "decoder": "correlated" }); + let threshold = json!({ "distances": [3, 5, 7], "p": [0.002, 0.004], "shots": 20000 }); + let critique = json!({ "qasm": "OPENQASM 3.0;\nqubit[2] q;\nh q[0];\ncx q[0], q[1];\n" }); + let mut calls: Vec<(&str, &Value)> = vec![("/v1/qec/decode", &decode), ("/v1/qec/threshold", &threshold)]; + calls.extend(std::iter::repeat_n(("/v1/critique", &critique), 6)); + let answers: Vec<(u16, String)> = std::thread::scope(|t| { + let each: Vec<_> = calls + .iter() + .enumerate() + .map(|(i, (path, body))| { + t.spawn(move || { + std::thread::sleep(Duration::from_millis(30 * i as u64)); + post_as(port, me, path, body) + }) + }) + .collect(); + each.into_iter().map(|c| c.join().unwrap()).collect() + }); + for ((path, _), (code, out)) in calls.iter().zip(&answers) { + assert_eq!(*code, 200, "{path}: {}", out.lines().last().unwrap_or("")); + } + alive(port, &mut svc, "the README's burst"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// A JSON-RPC id is echoed exactly, an integer past 2^53 included. +#[test] +fn an_mcp_id_is_echoed_exactly() { + let d = dir("mcp-ids"); + let (mut svc, port) = start(&d, 1000); + for id in ["9007199254740993", "-9007199254740993", "9223372036854775807", "12345678901234567890", "1730000000123456789"] { + let body: Value = serde_json::from_str(&format!("{{\"jsonrpc\":\"2.0\",\"id\":{id},\"method\":\"ping\"}}")).unwrap(); + let (code, _, text) = http(port, "POST", "/mcp", Some(&body)); + assert_eq!(code, 200, "{text}"); + assert!(text.contains(&format!("\"id\":{id},")) || text.contains(&format!("\"id\":{id}}}")), "{id}: {text}"); + } + alive(port, &mut svc, "ids echoed"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// A job whose result is larger than one job may give fails, saying so, +/// rather than being kept past every cap or dropped unread. +#[test] +fn a_result_past_what_a_job_may_give_fails_and_says_so() { + let d = dir("result-cap"); + let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_RESULT_MIB", "1")]); + let sweep = json!({ "qasm": "OPENQASM 3.0;\ninput float theta;\nqubit[16] q;\nbit[16] c;\nh q;\nrz(theta) q[0];\nc = measure q;\n", "sweep": { "theta": (0..10).map(|i| 0.01 * (i + 1) as f64).collect::>() }, "shots": 20000, "seed": 7 }); + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&sweep)); + assert_eq!(code, 202, "{b}"); + let id = b["id"].as_str().unwrap(); + let t = Instant::now(); + let job = loop { + let (code, b) = json_of(port, "GET", &format!("/v1/jobs/{id}?wait=25"), None); + assert_eq!(code, 200, "{b}"); + if b["status"] != "queued" && b["status"] != "running" { + break b; + } + assert!(t.elapsed() < Duration::from_secs(120)); + }; + assert_eq!(job["status"], "failed", "{}", job["status"]); + assert!(job["error"].as_str().is_some_and(|e| e.contains("MiB as served") && e.contains("J031")), "{}", job["error"]); + alive(port, &mut svc, "a result past its cap"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} -- 2.54.0 (Apple Git-157) From 34270f6aecc5a71934f6d976adacfbec3cf81dbf Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Sun, 8 Nov 2026 16:42:29 -0500 Subject: [PATCH 13/18] wai-quantum-service: a result held to its cap before it runs, and the round-5 probes as tests A job's or a set's result is bounded before it runs, from the request as read (catalog::plan_bounded and sets::plan_bounded, new in the engine): each item's counts at most its shots, 2^bits and 65,536 outcomes, each its bit string, quotes, a colon, a comma and its count's digits; expectation values, amplitudes, diagnostics, outputs, a repair, a compile, a target, a mitigation and a verdict by what each may hold or at their size as read; a set's every item request and frame beside. A request whose bound passes WAI_QSVC_RESULT_MIB (64) is refused at once, 422 and J031, with the bound and the cap in the reason, on every route that takes a job; a dry run gives the bound as result_bytes, and GET /v1, the dry run's bounds, the MCP description and the OpenAPI document publish the cap beside the set limits. At 64 MiB, 100 points of 65,536 outcomes (about 150 MB) are refused and 100 points of 16 bits at 30,000 shots are taken. A client's running jobs are counted against its unread allowance at their own bounds, not the cap. The check as a result ends is kept only as a backstop. The round-5 verifier's probes are the tests for its first two findings: 210 long polls on a running job from 210 addresses, 3 ms apart, under a hard limit of 256 (health 0 of 25 on a0804db72), and the README's decode, a threshold made long enough to outlast a waiting call's 10 s, and six critiques at the service's own limits (refused J032 at 10.07 s on a0804db72). Both pass here. Co-Authored-By: Claude Opus 5.5 --- wai/changes/2026-11-06-quantum-parse-depth.md | 13 +- wai/wai-quantum-service/README.md | 13 +- wai/wai-quantum-service/docs/self-host.md | 6 +- wai/wai-quantum-service/src/discovery.rs | 13 +- wai/wai-quantum-service/src/http.rs | 81 ++++- wai/wai-quantum-service/src/jobs.rs | 96 ++++-- wai/wai-quantum-service/src/mcp.rs | 2 +- wai/wai-quantum-service/src/offload.rs | 7 +- wai/wai-quantum-service/tests/depth.rs | 288 +++++++++++++++--- wai/wai-quantum/src/engine/catalog.rs | 93 +++++- wai/wai-quantum/src/engine/run.rs | 82 +++++ wai/wai-quantum/src/engine/sets.rs | 8 +- 12 files changed, 595 insertions(+), 107 deletions(-) diff --git a/wai/changes/2026-11-06-quantum-parse-depth.md b/wai/changes/2026-11-06-quantum-parse-depth.md index 48c09d58d..20d6f14be 100644 --- a/wai/changes/2026-11-06-quantum-parse-depth.md +++ b/wai/changes/2026-11-06-quantum-parse-depth.md @@ -130,9 +130,16 @@ `WAI_QSVC_CLIENT_STREAMS` (4) a client and `WAI_QSVC_STREAMS` (512, a quarter of the connections at most); past them a long poll is answered at once with the job as it is. A reading that moves from a lent turn onto its client's own gives the lent one back. Results read and not yet read are counted apart: only read ones are dropped before their - time, the largest holder's first; a client's next job waits while its unread results, with - `WAI_QSVC_RESULT_MIB` (64) for each job running, would pass `WAI_QSVC_CLIENT_UNREAD_MIB` (256), and every next job - while all unread hold `WAI_QSVC_UNREAD_MIB` (8192); a result past `WAI_QSVC_RESULT_MIB` fails its job, saying so. + time, the largest holder's first; a client's next job waits while its unread results, with each job running at + its result's bound and the new one at `WAI_QSVC_RESULT_MIB` (64), would pass `WAI_QSVC_CLIENT_UNREAD_MIB` (256), + and every next job while all unread hold `WAI_QSVC_UNREAD_MIB` (8192). A job's or a set's result is bounded before + it runs (`catalog::plan_bounded`, `sets::plan_bounded`, new: each item's counts at most its shots, `2^bits` and + 65,536 outcomes, every other part by what it may hold or at its size as read), and a request whose bound passes + `WAI_QSVC_RESULT_MIB` is refused at once (`422`, `J031`), its bound and the cap in the reason; a dry run states the + bound (`result_bytes`), and `GET /v1`, the dry run's bounds and the OpenAPI document the cap. At 64 MiB, 100 + points of 65,536 outcomes (about 150 MB) are refused, and 100 points of 16 bits at 30,000 shots are taken. The + check as a result ends stays as a backstop, for a job kept across a restart that lowered the cap and for a pulse + run, which is not bounded before it runs. MCP answers echo a JSON-RPC id exactly. Readers and workers start from the service's own binary as it was started (Linux: its running image; elsewhere a copy kept beside the signing key), so the binary can be replaced before the restart. A restart reads no program: a job keeps the plan it was admitted with and a diff --git a/wai/wai-quantum-service/README.md b/wai/wai-quantum-service/README.md index c082ba614..e9e5791b0 100644 --- a/wai/wai-quantum-service/README.md +++ b/wai/wai-quantum-service/README.md @@ -49,6 +49,8 @@ One request can run many programs, as one job: A set holds at most 100 items. They run one after another in one worker, inside one job's time limit, holding the memory of the largest item. Each item is a plain job: its request is complete in itself, and it runs, plans and verifies as it would alone. A set with an item that does not plan is refused, naming the item. +A job's result, or a set's with every item's request and result in it, may take at most `WAI_QSVC_RESULT_MIB` (64 MiB) as served. That is checked before anything runs, from a bound of the result read from the request: each item's counts list at most its shots, `2^bits` and 65,536 outcomes, each its bit string, quotes, a colon, a comma and its count's digits; each other part is counted by what it may hold or at its size as read (a verdict asking for one counts 512 bytes for each gate its critique may remark on). A request whose bound passes the cap is refused at once, `422` and `J031`, with the bound and the cap in the reason, and a dry run gives the bound as `result_bytes`. The cap is less than a set at its most can give: at about 22 bytes a 16-bit outcome, 100 points of 65,536 outcomes each are about 150 MB, and are refused; 100 points of 16 measured bits are taken up to about 30,000 shots a point. Send fewer points or shots, split the set, or run it on your own machine (`wai-quantum run`), where no cap applies. A method that reads its own program (a pulse run) is not bounded before it runs: it is counted at the cap, and a result past the cap fails its job, saying so, as one kept across a restart that lowered the cap does. + The result is a set (`"format": "wai.quantum.set/1"`). Each item has its own result and its own receipt. Each item's leaf is the digest its receipt signs (a failed item's: the digest of its request and its error). The head holds the Merkle root of the leaves, under the JWP Merkle domains, with the count, and the set's receipt seals the set's request and its head. Each item carries its `path` to the root, so one item can be shown sealed by its set without the others. `POST /v1/verify` checks the whole set: its receipt, every item as the request gives it, each item's receipt, the root, and each path. ```sh runnable needs=service @@ -67,7 +69,8 @@ curl -s https://wai.transaction.science/quantum-jobs/v1/run -H 'content-type: ap | `memory_bytes`, `work` | the memory it would be admitted with, and the planner's work | | `time` | `seconds` with its `basis`: the median rate of the last jobs of the method this service timed, or, before it has timed three, the planner's nominal rate; or `seconds: null` with why, where the planner's work does not predict the time | | `energy` | `joules_micro_estimate`, class `Estimator`, from the median power of the last metered jobs of the method here; or `unestimated` with why. Never a reading | -| `bounds` | the limits that apply: memory and time per job, jobs and memory per client, submissions per window, the queue, the most items in a set, the most intents, the steps a request's programs may take to read, the largest body | +| `bounds` | the limits that apply: memory and time per job, jobs and memory per client, submissions per window, the queue, the most items in a set, the most a result may take (`result_bytes`), the most intents, the steps a request's programs may take to read, the largest body | +| `result_bytes` | the most its result takes as served, bounded before it runs; past `bounds.result_bytes` it is refused, `422` and `J031` | | `client` | where you stand: active jobs, memory held, submissions left | | `accepted` | whether the service would take the job now, by every limit; if not, `status` and `refusal` (its message and code) say why | | `items` | for a set: each item's plan and estimates | @@ -191,7 +194,7 @@ A submission is admitted in the order of what each step costs: 4. With its turn the submission is counted against your window. Every refusal after this counts. 5. What reads nothing: every whole number exact, at most 16 intents (each of a set's items too), at most 100 items in a set. 6. Its programs are read: the job's own, a set's items and a sweep's points, and every intent's reference, each once however often admission reads it, and all of them within one budget of 22,000,000 steps (statements the reader runs, gates it writes, a generator's loop passes and what it writes out, and every 128 bytes of a value it makes or copies). That is as many as one program may take, its statements and its gates each to the reader's bound, so a job whose own program reads within the reader's bounds reads here. A request whose programs take more is refused, `422` and `J031`, with the command that runs it on your machine. -7. The job is kept and queued, within your and the queue's limits. +7. The job is kept and queued, within your and the queue's limits, and within the most a result may take: one whose result's bound passes `WAI_QSVC_RESULT_MIB` is refused, `422` and `J031`, before it runs. `Retry-After` is the pace at which a refusal that counts would not spend your window (the window over its submissions, at least 5 s), or, past the window, when the next submission opens. A `503` carries it too. @@ -201,7 +204,7 @@ Reading happens outside the service's own process, as each job's run does: each - **Memory.** A child's memory is polled every 50 ms from a thread of its own, and the child is killed once it holds more than `WAI_QSVC_READ_GB` (4); the request is refused, `422` and `J031`. The kill is a signal the service sends itself (`SIGKILL`, to the child's process group, and again every 100 ms until it has ended), so it needs no new process. On macOS the poll reads the child's physical footprint, which counts its compressed and swapped pages; on Linux its resident and swapped pages. A poll is not a kernel limit: a child can pass the cap by what it touches in one interval before it is killed. On Linux the child also holds its own address space to twice the cap and a gigabyte, so an allocation past that fails at once (`422`, `J031`), and the kernel ends a reader or a worker before the service when the host runs short. macOS gives an unprivileged process neither, and has no limit that counts compressed memory. - **Time.** A child that reads for longer than `WAI_QSVC_READ_SECS` (45) is killed and the request refused, `422` and `J031`. Every request is answered within `WAI_QSVC_ANSWER_SECS` (55) of its arrival, under the 60 s a proxy in front commonly waits for an answer: its waits for a turn and its readings end by then, `POST /v1/run` and the MCP run tools wait for their job only for what is left and then answer with the job, still running, to poll, and an MCP answer's verdict is read again only if time is left (its verdict says so where it was not). An error-correction call's shots are measured as they run: each point is set up and sampled, and a pilot of its own shots decoded, before the rest of any; it is refused at once when what is measured says it would take more than `WAI_QSVC_QEC_SECS` (25), and stopped if it runs a fifth past that. - **Its answer.** What a reading gives back comes into the service's memory, which every job shares: one that would take more than `WAI_QSVC_READ_ANSWER_MIB` (128) once read, about 80 bytes a value, is refused in the child before it is written (`422`, `J031`), and the service reads it off its request threads. A dry run gets back the plans and points alone. A check's answer can be that large: a critique of a program of 200,000 gates that each draw a remark (`qubit q; for int i in [1:200000] { x q; }`) is refused so; one of 150,000 is answered. -- **What ended jobs keep.** A job keeps its request as the canonical text the store keeps, and a set lets go of its items once it ends. What ended jobs keep in memory, their requests' text and their results as RFC 8785 text, is counted in two parts. Results that have been read (or ended more than `WAI_QSVC_KEPT_MIN_SECS`, 600 s, ago) are held to `WAI_QSVC_CLIENT_KEPT_MIB` (128) a client, its oldest dropped first, and `WAI_QSVC_KEPT_MIB` (4096) in all, those of the client keeping the most dropped first (between two keeping as much, the newer); only these are ever dropped before `WAI_QSVC_RETAIN_SECS`. Results not yet read are dropped by neither cap: a client's next job is refused, `429`, while its unread results, with `WAI_QSVC_RESULT_MIB` (64) for each job it has running and the new one, would pass `WAI_QSVC_CLIENT_UNREAD_MIB` (256), and every next job is refused, `503`, while all unread results hold `WAI_QSVC_UNREAD_MIB` (8192). A result larger than `WAI_QSVC_RESULT_MIB` fails its job, saying so. The bounds that remain: memory for ended jobs stays under the two caps, 12 GiB at the defaults, with the results of jobs still running; reaching the unread cap takes 32 clients each holding its whole allowance unread, and holds every next job for at most 600 s; a client is its address, so people behind one address share its allowances, and one of them who leaves 256 MiB unread holds the others' next jobs until it is read or 600 s pass; and an honest client's read result is dropped early only where it, of all clients, keeps the most. A request's body is parsed only once its client's window has room (or, for a resend under a key the client holds a job under, first, as it is answered before any limit). A receipt check has a window of its own, `WAI_QSVC_CLIENT_VERIFIES` (120), asked before its body is read. An MCP message over 64 KiB from a client past its window is read only for its id, method, tool and key, and answered under its own id with a tool result that says why; a resend under a key the client holds, and a receipt check, are read whole. A job's report is one RFC 8785 text on every route (a poll, its events, a cancel, and in the MCP job tools' answers, written with RFC 8785's numbers, the job's own bytes, with each JSON-RPC id echoed exactly) and after a restart: its request and its result are written in as the text each is kept as, never parsed again, and the memory a large request or answer was parsed into is given back to the system once it is answered. On macOS the service starts itself with the allocator's large-block cache off (`MallocLargeCache=0`): that cache kept every large block a parsed request had freed, dirty and charged to the service (on arm64, 30 requests of a megabyte kept 1 GiB; with it off, 2 MiB). +- **What ended jobs keep.** A job keeps its request as the canonical text the store keeps, and a set lets go of its items once it ends. What ended jobs keep in memory, their requests' text and their results as RFC 8785 text, is counted in two parts. Results that have been read (or ended more than `WAI_QSVC_KEPT_MIN_SECS`, 600 s, ago) are held to `WAI_QSVC_CLIENT_KEPT_MIB` (128) a client, its oldest dropped first, and `WAI_QSVC_KEPT_MIB` (4096) in all, those of the client keeping the most dropped first (between two keeping as much, the newer); only these are ever dropped before `WAI_QSVC_RETAIN_SECS`. Results not yet read are dropped by neither cap: a client's next job is refused, `429`, while its unread results, with each job it has running at its result's bound and the new one at `WAI_QSVC_RESULT_MIB` (64), would pass `WAI_QSVC_CLIENT_UNREAD_MIB` (256), and every next job is refused, `503`, while all unread results hold `WAI_QSVC_UNREAD_MIB` (8192). A job whose result's bound passes `WAI_QSVC_RESULT_MIB` is refused before it runs (see [Sets and sweeps](#sets-and-sweeps)). The bounds that remain: memory for ended jobs stays under the two caps, 12 GiB at the defaults, with the results of jobs still running; reaching the unread cap takes 32 clients each holding its whole allowance unread, and holds every next job for at most 600 s; a client is its address, so people behind one address share its allowances, and one of them who leaves 256 MiB unread holds the others' next jobs until it is read or 600 s pass; and an honest client's read result is dropped early only where it, of all clients, keeps the most. A request's body is parsed only once its client's window has room (or, for a resend under a key the client holds a job under, first, as it is answered before any limit). A receipt check has a window of its own, `WAI_QSVC_CLIENT_VERIFIES` (120), asked before its body is read. An MCP message over 64 KiB from a client past its window is read only for its id, method, tool and key, and answered under its own id with a tool result that says why; a resend under a key the client holds, and a receipt check, are read whole. A job's report is one RFC 8785 text on every route (a poll, its events, a cancel, and in the MCP job tools' answers, written with RFC 8785's numbers, the job's own bytes, with each JSON-RPC id echoed exactly) and after a restart: its request and its result are written in as the text each is kept as, never parsed again, and the memory a large request or answer was parsed into is given back to the system once it is answered. On macOS the service starts itself with the allocator's large-block cache off (`MallocLargeCache=0`): that cache kept every large block a parsed request had freed, dirty and charged to the service (on arm64, 30 requests of a megabyte kept 1 GiB; with it off, 2 MiB). - **Ending.** A child that panics, overflows its stack or aborts is answered `500`, and the service runs on. A client that hangs up ends its child. - **Connections.** A request whose header section takes longer than `WAI_QSVC_HEADER_SECS` (150) to arrive is closed; the same limit closes a connection left idle between requests, so it is set above a proxy's own idle limit (two minutes, commonly), and the proxy, not the service, closes its idle connections. A body that takes longer than `WAI_QSVC_BODY_SECS` (30) is refused, `408`. A client may have `WAI_QSVC_CLIENT_REQUESTS` (32) requests open at once; past it a request is refused, `503`, before its body is read (a health check is answered whatever that is). Long polls and event streams are held to `WAI_QSVC_CLIENT_STREAMS` (4) a client and `WAI_QSVC_STREAMS` (512, at most a quarter of the connections) in all: past them a long poll is answered at once with the job as it is, and an event stream is refused, `503`. Four connections past the connection cap are held for health checks alone, one request each, for 2 s. The service raises its open-file limit to its hard limit as it starts (on macOS no further than `kern.maxfilesperproc` and 10240), says so in its log, and holds at most `WAI_QSVC_CONNECTIONS` (4096) connections, fewer where that limit leaves less room: past them the connection idle longest (or the one whose body has been arriving longest) is closed for the new one, so a health check is always answered, and with none to close a new connection is answered `503` at once. - **Updating.** Readers and workers start from the service's own binary as it was started: on Linux its running image, elsewhere a copy it keeps beside its signing key, named by its digest. So the binary can be replaced before the restart (by a copy over it, or, on Linux, where a running binary cannot be written, by a rename), and the running service goes on with its own until it is restarted. @@ -351,9 +354,9 @@ These variables set the limits: | `WAI_QSVC_ANSWER_SECS` | 55 | the longest any answer takes, its waits and readings included | | `WAI_QSVC_CLIENT_KEPT_MIB` | 128 | what one client's ended jobs may keep in memory | | `WAI_QSVC_KEPT_MIB` | 4096 | what all ended jobs whose results have been read may keep in memory | -| `WAI_QSVC_CLIENT_UNREAD_MIB` | 256 | what one client's results not yet read may hold, its running jobs counted at `WAI_QSVC_RESULT_MIB` each; past it, its next job is refused (`429`) | +| `WAI_QSVC_CLIENT_UNREAD_MIB` | 256 | what one client's results not yet read may hold, each of its running jobs counted at its result's bound and the next at `WAI_QSVC_RESULT_MIB`; past it, its next job is refused (`429`) | | `WAI_QSVC_UNREAD_MIB` | 8192 | what all results not yet read may hold; past it, every next job is refused (`503`) | -| `WAI_QSVC_RESULT_MIB` | 64 | the largest result one job may give, as served; past it the job fails, saying so | +| `WAI_QSVC_RESULT_MIB` | 64 | the most one job's or set's result may take, as served: a request whose result's bound passes it is refused before it runs (`422`, `J031`) | | `WAI_QSVC_CLIENT_STREAMS` | 4 | long polls (`?wait=`) and event streams one client may hold at once | | `WAI_QSVC_STREAMS` | 512 | those of all clients, at most a quarter of the connections held | | `WAI_QSVC_KEPT_MIN_SECS` | 600 | how long an ended job not yet read is kept whatever the two caps above say | diff --git a/wai/wai-quantum-service/docs/self-host.md b/wai/wai-quantum-service/docs/self-host.md index 95398f51b..e77636156 100644 --- a/wai/wai-quantum-service/docs/self-host.md +++ b/wai/wai-quantum-service/docs/self-host.md @@ -57,9 +57,9 @@ Everything is set by environment variables. | `WAI_QSVC_ANSWER_SECS` | 55 | the longest any answer takes, its waits and readings included | | `WAI_QSVC_CLIENT_KEPT_MIB` | 128 | what one client's ended jobs may keep in memory | | `WAI_QSVC_KEPT_MIB` | 4096 | what all ended jobs whose results have been read may keep in memory | -| `WAI_QSVC_CLIENT_UNREAD_MIB` | 256 | what one client's results not yet read may hold, its running jobs counted at `WAI_QSVC_RESULT_MIB` each; past it, its next job is refused (`429`) | +| `WAI_QSVC_CLIENT_UNREAD_MIB` | 256 | what one client's results not yet read may hold, each of its running jobs counted at its result's bound and the next at `WAI_QSVC_RESULT_MIB`; past it, its next job is refused (`429`) | | `WAI_QSVC_UNREAD_MIB` | 8192 | what all results not yet read may hold; past it, every next job is refused (`503`) | -| `WAI_QSVC_RESULT_MIB` | 64 | the largest result one job may give, as served; past it the job fails, saying so | +| `WAI_QSVC_RESULT_MIB` | 64 | the most one job's or set's result may take, as served: a request whose result's bound passes it is refused before it runs (`422`, `J031`) | | `WAI_QSVC_CLIENT_STREAMS` | 4 | long polls (`?wait=`) and event streams one client may hold at once | | `WAI_QSVC_STREAMS` | 512 | those of all clients, at most a quarter of the connections held | | `WAI_QSVC_KEPT_MIN_SECS` | 600 | how long an ended job not yet read is kept whatever the two caps above say | @@ -154,7 +154,7 @@ Run the service on a loopback address and let the proxy take the public traffic. - **The client's address.** The proxy sets `X-Forwarded-For`, and the service trusts it only from `WAI_QSVC_TRUSTED_PROXIES`. Have the proxy set the header from the connection itself, not pass on what the request carried, or a client could choose its own limits. - **A path prefix.** Mount the service under a prefix (the public one is `/quantum-jobs`) by stripping the prefix before passing the request on. Set `WAI_QSVC_PUBLIC_BASE` to the prefixed URL, so the links the service hands out are right. A well-known path belongs at the origin's root: map `/.well-known/wai-quantum-receipt-keys` there to the service's own. - **Event streams.** `GET /v1/jobs/{id}/events` is a server-sent event stream. The proxy must pass it through unbuffered (the service sends `X-Accel-Buffering: no` and a comment every 15 s), and allow it to stay open as long as `WAI_QSVC_JOB_SECS`. Polls with `?wait=` are held up to 25 s. -- **Bodies.** A request body is at most 1 MiB, and `POST /v1/run` waits up to 25 s before answering 202. +- **Bodies.** A request body is at most 1 MiB, and `POST /v1/run` waits up to 25 s before answering 202. An answer's body can be as large as a result may be as served, `WAI_QSVC_RESULT_MIB` (64 MiB), with the job's request beside it: a proxy that buffers or caps answers must allow that. A request whose result could be larger is refused before it runs (`422`, `J031`), its bound in the reason. - **Rate-limit fields.** Pass the `RateLimit-Policy`, `RateLimit` and `Retry-After` headers through. - **Timeouts.** Every answer comes within `WAI_QSVC_ANSWER_SECS` (55) of the request's arrival, so within a proxy's 60 s wait for an answer's header. The worst case of each route, from arrival: a check, a dry run or a submission waits at most 10 s for a turn and reads for at most `WAI_QSVC_READ_SECS` (45), 55 s in all; `POST /v1/run` and the MCP run tools then wait for their job only for what is left of the 55 s (less 5 s, on MCP, for its verdict) and answer with the job, still running, to poll; an MCP answer's verdict waits for a turn and is read only within what is left, and says where no time was left; `tasks/result` holds a reply for what is left of the 55 s, 50 s at most. With a shorter wait in the proxy, lower `WAI_QSVC_ANSWER_SECS` below it. Keep the proxy's idle connections shorter than `WAI_QSVC_HEADER_SECS` (150), which also bounds an idle connection on the service's side: the proxy then closes its idle connections itself, and never sends a request on one as the service closes it. - **One address, many people.** The service counts a client by the address the proxy gives it. People behind one address share one client's turns (`WAI_QSVC_CLIENT_READS`, `WAI_QSVC_CLIENT_CHECKS`) and window; raise those, or give such a group an API key. diff --git a/wai/wai-quantum-service/src/discovery.rs b/wai/wai-quantum-service/src/discovery.rs index 076940f1d..2ac0e7696 100644 --- a/wai/wai-quantum-service/src/discovery.rs +++ b/wai/wai-quantum-service/src/discovery.rs @@ -104,7 +104,7 @@ pub fn examples() -> Value { } /// The API as an OpenAPI 3.1 document. -pub fn openapi(server: &str) -> Value { +pub fn openapi(server: &str, result_bytes: u64) -> Value { let prog = json!({ "type": "object", "required": ["qasm"], "properties": { "qasm": { "type": "string", "description": "The program, OpenQASM 3 or 2." } } }); let pauli = json!({ "oneOf": [ { "type": "string", "description": "A Pauli string: factors such as `Z0`, `X3`, `Y12`, separated by spaces; qubit indices count from 0.", "examples": ["Z0 Z1", "X3 Y4"] }, @@ -139,7 +139,7 @@ pub fn openapi(server: &str) -> Value { "intent": { "description": "What the program must do, checked with the run: one intent or a list, each {\"distribution\": {\"00\": 0.5, \"11\": 0.5}, \"tolerance\"}, {\"equivalent\": \"\"} or {\"prepares\": \"\"}, with an optional note and \"require\": \"proof\". The result's `verdict` states the level the engine checked to (proof, evidence, executed, parsed, refused) and whether the intents hold, and the receipt seals it." }, "verdict": { "type": "boolean", "description": "true: the result carries the verdict with no intent (how far the engine checked)." }, "repair": { "description": "true, or the choices {\"max_iterations\", \"dyadic_bits\", \"sizes\"}: the program is repaired first on the lines its diagnostics flag (as POST /v1/repair does), and the result's `repaired` holds the program that ran and each repair." }, - "programs": { "type": "array", "minItems": 1, "maxItems": wai_quantum::engine::sets::MAX_ITEMS, "items": { "oneOf": [{ "type": "string" }, { "type": "object", "required": ["qasm"] }] }, "description": "A set: in place of qasm, the programs to run as one job, each a program's text or a job whose own fields win over the set's. Every other field applies to each item. The result is a set (SetResult)." }, + "programs": { "type": "array", "minItems": 1, "maxItems": wai_quantum::engine::sets::MAX_ITEMS, "items": { "oneOf": [{ "type": "string" }, { "type": "object", "required": ["qasm"] }] }, "description": format!("A set: in place of qasm, the programs to run as one job, each a program's text or a job whose own fields win over the set's. Every other field applies to each item. The result is a set (SetResult). A job or set whose result could take more than {result_bytes} bytes as served (result_bytes), by its bound before it runs (each item's counts at most its shots, 2^bits or 65,536 outcomes, and its other parts as read), is refused at once (422, J031): at about 20 bytes and a count a 16-bit outcome, 100 items of 65,536 outcomes each is about 145 MB.") }, "sweep": { "type": "object", "additionalProperties": { "type": "array", "minItems": 1, "items": { "type": "number" } }, "description": "A set over qasm's inputs: each input's values, every combination run (the names in sorted order, the first varying slowest). An integer input (`input uint n;`) takes whole numbers and the program is written out at each; a float or angle input (`input float theta;`) is bound to the double given. Every input the program declares is given values." }, "dry_run": { "type": "boolean", "description": "true: run nothing, and answer 200 with the plan (DryRun): the method, memory, estimated time and energy with their basis, the bounds, and whether the service would take it now." }, "pulse": wai_quantum::engine::pulse::options_schema(), @@ -161,7 +161,8 @@ pub fn openapi(server: &str) -> Value { "time": { "type": "object", "description": "{\"seconds\", \"basis\"}: an estimate from this service's own timings of the method, or the planner's nominal rate; or {\"seconds\": null, \"why\", \"at_most\"} where the work does not predict the time." }, "energy": { "type": "object", "description": "{\"joules_micro_estimate\", \"energy_provenance\": \"Estimator\", \"basis\"} from the power of past metered jobs of the method here, or {\"unestimated\": why}. Never a reading." }, "within_time_limit": { "type": ["boolean", "null"] }, - "bounds": { "type": "object", "description": "The limits that apply: job_memory_bytes, job_seconds, active_jobs_per_client, memory_per_client_bytes, submissions_per_window, window_seconds, queue, set_items, intents (the most a job states), read_steps (the steps all of a request's programs may take to read together), body_bytes." }, + "bounds": { "type": "object", "description": "The limits that apply: job_memory_bytes, job_seconds, active_jobs_per_client, memory_per_client_bytes, submissions_per_window, window_seconds, queue, set_items, result_bytes (the most a job's or a set's result may take as served, by its bound before it runs), intents (the most a job states), read_steps (the steps all of a request's programs may take to read together), body_bytes." }, + "result_bytes": { "type": ["integer", "object"], "description": "The most its result takes as served, bytes, bounded before it runs; past bounds.result_bytes the submission is refused (422, J031). {\"bytes\": null, \"why\"} for a method that reads its own program (a pulse run), counted at bounds.result_bytes." }, "client": { "type": "object", "description": "Where the client stands: active_jobs, memory_held_bytes, submissions_left, window_resets_in_seconds." }, "items": { "type": "array", "description": "A set's items, each with its index, point, method, memory, work, time and energy." } } @@ -248,7 +249,7 @@ pub fn openapi(server: &str) -> Value { "200": { "description": "The finished job, or with dry_run the plan (DryRun)", "headers": { "RateLimit": { "$ref": "#/components/headers/RateLimit" }, "RateLimit-Policy": { "$ref": "#/components/headers/RateLimit-Policy" } }, "content": { "application/json": { "schema": { "oneOf": [{ "$ref": "#/components/schemas/Job" }, { "$ref": "#/components/schemas/DryRun" }] } } } }, "202": { "description": "Still running: poll GET /v1/jobs/{id}", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Job" } } } }, "400": { "description": "A program, job or Idempotency-Key that does not read", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, - "422": { "description": "Larger than one job may be (J031: its plan, or its reading past read_steps, the reading child's memory or its time), or an Idempotency-Key sent before with another request (J034)" }, + "422": { "description": "Larger than one job may be (J031: its plan, its result's bound past result_bytes, or its reading past read_steps, the reading child's memory or its time), or an Idempotency-Key sent before with another request (J034)" }, "401": { "description": "A private deployment's key missing or not one it holds (J033)" }, "429": { "description": "Past the per-client limits (with Retry-After)" }, "503": { "description": "The queue is full, or no turn to read it came (J032): nothing was read, and nothing counted" }, @@ -263,7 +264,7 @@ pub fn openapi(server: &str) -> Value { "202": { "description": "Accepted, and kept in the service's store before this answer; sent again with its Idempotency-Key, the key's job", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Job" } } } }, "200": { "description": "With dry_run: the plan, nothing run", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/DryRun" } } } }, "400": { "description": "A program, job or Idempotency-Key that does not read", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } } }, - "422": { "description": "Larger than one job may be (J031: its plan, or its reading past read_steps, the reading child's memory or its time), or an Idempotency-Key sent before with another request (J034)" }, + "422": { "description": "Larger than one job may be (J031: its plan, its result's bound past result_bytes, or its reading past read_steps, the reading child's memory or its time), or an Idempotency-Key sent before with another request (J034)" }, "401": { "description": "A private deployment's key missing or not one it holds (J033)" }, "429": { "description": "Past the per-client limits (with Retry-After)" }, "503": { "description": "The queue is full, or no turn to read it came (J032): nothing was read, and nothing counted" }, @@ -517,7 +518,7 @@ mod tests { #[test] fn the_openapi_document_names_every_route() { - let d = openapi("https://example.org/q"); + let d = openapi("https://example.org/q", 64 << 20); for p in ["/v1", "/v1/methods", "/v1/examples", "/v1/targets", "/v1/run", "/v1/jobs", "/v1/jobs/{id}", "/v1/jobs/{id}/events", "/.well-known/wai-quantum-receipt-keys", "/v1/verify", "/v1/compile", "/v1/estimate", "/v1/convert", "/v1/qec/code", "/v1/qec/decode", "/v1/qec/threshold", "/v1/gradient", "/v1/optimize", "/v1/vqe", "/v1/qaoa", "/v1/build", "/v1/pulse/schedule", "/v1/pulse/check", "/mcp"] { diff --git a/wai/wai-quantum-service/src/http.rs b/wai/wai-quantum-service/src/http.rs index 327f98527..94caeb99c 100644 --- a/wai/wai-quantum-service/src/http.rs +++ b/wai/wai-quantum-service/src/http.rs @@ -244,6 +244,10 @@ async fn describe(State(app): State) -> Response { "queue": c.max_queue, "result_retention_seconds": c.retain.as_secs(), "set_items": sets::MAX_ITEMS, + // A job or set whose result could take more, by its bound + // before it runs, is refused at once (422, J031). + "result_bytes": c.result_mib << 20, + "unread_results_per_client_bytes": c.client_unread_mib << 20, "intents": crate::analysis::MAX_INTENTS, "read_steps": READ_STEPS, }, @@ -279,6 +283,7 @@ fn bounds(c: &crate::jobs::Config) -> Value { "window_seconds": c.per_client_window.as_secs(), "queue": c.max_queue, "set_items": sets::MAX_ITEMS, + "result_bytes": c.result_mib << 20, "intents": crate::analysis::MAX_INTENTS, "read_steps": READ_STEPS, "body_bytes": MAX_BODY, @@ -369,11 +374,42 @@ pub(crate) fn within_budget(read: impl FnOnce() -> Result) if spent { Err(read_too_long()) } else { r } } +/// What a job's result takes as served beside what the engine bounds +/// ([`catalog::plan_bounded`]): its receipt and its energy figure. +const HOST_BYTES: u64 = 4 << 10; +/// What a set's item takes as the set serves it beside its result and its +/// request: its index, status, method, point and path to the root, and its +/// receipt and energy. +const ITEM_BYTES: u64 = 4 << 10; +/// A set's head as served: its format, counts, root, energy and receipt. +const SET_HEAD_BYTES: u64 = 4 << 10; + +/// `v`'s length as JSON text, written nowhere. RFC 8785 writes the same +/// characters but for a number's form, which a sixteenth more allows for. +fn json_len(v: &Value) -> u64 { + struct Count(u64); + impl std::io::Write for Count { + fn write(&mut self, b: &[u8]) -> std::io::Result { + self.0 += b.len() as u64; + Ok(b.len()) + } + fn flush(&mut self) -> std::io::Result<()> { + Ok(()) + } + } + let mut c = Count(0); + let _ = serde_json::to_writer(&mut c, v); + c.0 + c.0 / 16 +} + /// `request` planned: one job's plan, or a set's plan with its items' (each -/// held to a job's bounds), or the refusal's status and reason. The checks -/// that read nothing come first; then every program the request gives is -/// read, once, within one budget ([`READ_STEPS`]). -pub(crate) fn plan_job(cap: u64, request: &Value) -> Result<(catalog::Plan, Work), (u16, String)> { +/// held to a job's bounds), and the most its result takes as served, bytes +/// (`None` where a method that reads its own program leaves it unbounded), +/// or the refusal's status and reason. The checks that read nothing come +/// first; then every program the request gives is read, once, within one +/// budget ([`READ_STEPS`]). A set's result serves each item's request, its +/// result and its own fields, and the set's head. +pub(crate) fn plan_job(cap: u64, request: &Value) -> Result<(catalog::Plan, Work, Option), (u16, String)> { cheap_bounds(request)?; within_budget(|| { if catalog::is_set(request) { @@ -381,11 +417,17 @@ pub(crate) fn plan_job(cap: u64, request: &Value) -> Result<(catalog::Plan, Work for (i, it) in items.iter().enumerate() { job_bounds(&it.request).map_err(|(c, e)| (c, format!("item {i}: {e}")))?; } - let plans = sets::plan(&items, cap).map_err(|e| (400, e))?; - return Ok((sets::summary(&plans), Work::Set { items, plans })); + let planned = sets::plan_bounded(&items, cap).map_err(|e| (400, e))?; + let bound = planned.iter().zip(&items).try_fold(SET_HEAD_BYTES, |all, ((p, result), it)| { + let item = ITEM_BYTES + json_len(&it.request) + json_len(&json!(p.why)) + it.point.as_ref().map_or(0, json_len); + Some(all.saturating_add((*result)?).saturating_add(item)) + }); + let plans: Vec = planned.into_iter().map(|(p, _)| p).collect(); + return Ok((sets::summary(&plans), Work::Set { items, plans }, bound)); } job_bounds(request)?; - Ok((catalog::plan(request, cap).map_err(|e| (400, e))?, Work::One)) + let (plan, bound) = catalog::plan_bounded(request, cap).map_err(|e| (400, e))?; + Ok((plan, Work::One, bound.map(|b| b.saturating_add(HOST_BYTES)))) }) } @@ -396,9 +438,10 @@ pub(crate) async fn admit(app: &App, who: &str, request: Value, key: Option); +/// the request as read (from its source, where it has one), and its plan, +/// work and the most its result takes as served ([`plan_job`]), or the +/// refusal its planning gave. +type Admitted = (Value, Result<(catalog::Plan, Work, Option), (u16, String)>); fn admitted(mut v: Value) -> Result { let bad = |e: serde_json::Error| (500, format!("the reading's answer did not read: {e}")); @@ -409,11 +452,12 @@ fn admitted(mut v: Value) -> Result { return Ok((request, Err((r.first().and_then(Value::as_u64).unwrap_or(400) as u16, r.get(1).and_then(Value::as_str).unwrap_or("refused").to_string())))); } let plan: catalog::Plan = serde_json::from_value(take("plan")).map_err(bad)?; + let bound = take("result_bytes").as_u64(); let work = match take("items") { Value::Null => Work::One, items => Work::Set { items: crate::jobs::items_from(items), plans: serde_json::from_value(take("plans")).map_err(bad)? }, }; - Ok((request, Ok((plan, work)))) + Ok((request, Ok((plan, work, bound)))) } /// [`admit`], in the order of what each step costs: @@ -456,8 +500,8 @@ pub(crate) async fn admit_to(service: &Service, who: &str, request: Value, key: { return Ok(id); } - let (plan, work) = planned?; - service.submit_counted(who, request, plan, work, key, received).map_err(refused) + let (plan, work, bound) = planned?; + service.submit_counted(who, request, plan, work, key, received, bound).map_err(refused) } /// The `Idempotency-Key` a request gives, if it gives one: a structured @@ -510,7 +554,7 @@ fn dry_run_now(service: &Service, who: &str, (_, planned): Admitted) -> Result p, Err((status, why)) => { out["accepted"] = json!(false); @@ -553,7 +597,7 @@ fn dry_run_now(service: &Service, who: &str, (_, planned): Admitted) -> Result Result json!(b), + None => json!({ "bytes": null, "why": "a method that reads its own program (a pulse run) is not bounded before it runs: it is counted at the most a job's result may take, and held to it as it ends" }), + }; if let Some(t) = &plan.target { out["target"] = t.clone(); } @@ -953,7 +1002,7 @@ async fn examples() -> Response { } async fn openapi(State(app): State) -> Response { - reply(200, crate::discovery::openapi(&app.base)) + reply(200, crate::discovery::openapi(&app.base, app.service.config.result_mib << 20)) } /// The root: a browser goes to the console page; a program gets an index. diff --git a/wai/wai-quantum-service/src/jobs.rs b/wai/wai-quantum-service/src/jobs.rs index 3803ba8c4..e1a86513c 100644 --- a/wai/wai-quantum-service/src/jobs.rs +++ b/wai/wai-quantum-service/src/jobs.rs @@ -329,6 +329,11 @@ pub struct Job { /// Whether its result has been served since it ended: until then the /// caps on what ended jobs keep pass it by, for [`Config::kept_min`]. read: bool, + /// The most its result takes as served, bytes, known as it was admitted + /// (`http::plan_job`): what its client's unread results are counted to + /// hold while it runs. `u64::MAX` where it is not known (a job kept + /// across a restart, a pulse run), counted at [`Config::result_mib`]. + bound: u64, } tokio::task_local! { @@ -914,6 +919,7 @@ impl Service { interruptions: rec["interruptions"].as_array().into_iter().flatten().filter_map(Value::as_u64).collect(), child: None, kept: 0, read: false, + bound: u64::MAX, }; if job.finished_ms.is_some() { job.ended(); @@ -1068,6 +1074,15 @@ impl Service { jobs.values().filter(|j| !j.read && j.finished_ms.is_some_and(|f| now.saturating_sub(f) < min)).fold((0, 0), |(m, a), j| (m + if j.client == name { j.kept } else { 0 }, a + j.kept)) } + /// What the results of `name`'s jobs still running may hold once they + /// end, bytes: each its bound ([`Job::bound`]), one not known at the + /// most a job's result may take ([`Config::result_mib`]). + fn running_results(&self, name: &str) -> u64 { + let cap = self.config.result_mib << 20; + let jobs = self.jobs.lock().unwrap(); + jobs.values().filter(|j| j.client == name && j.status.active()).map(|j| j.bound.min(cap)).sum() + } + /// The client's active jobs and the memory they hold, MiB. fn held(&self, name: &str) -> (usize, u64) { let jobs = self.jobs.lock().unwrap(); @@ -1080,9 +1095,9 @@ impl Service { /// counted, so a client told to try again shortly spends nothing doing so. pub fn refused_before_reading(&self, client: &str) -> Option { let (mine, all) = self.unread(&self.name(client)); - // A job still running may give a result as large as a job may give. - let (running, _) = self.held(&self.name(client)); - let room = mine + (running as u64 + 1) * (self.config.result_mib << 20); + // Each job still running counted at its bound, and this one, not yet + // read, at the most a job's result may take. + let room = mine + self.running_results(&self.name(client)) + (self.config.result_mib << 20); if room > self.config.client_unread_mib << 20 { let room = room.div_ceil(1 << 20); return Some(Refused(429, tag(refusal::RATE, format!("results of yours not yet read hold about {} MiB, and with your jobs running and this one they could hold {room} MiB, more than the {} MiB one client's unread results may; read them (GET /v1/jobs/{{id}}) before sending more, or wait {} s", mine >> 20, self.config.client_unread_mib, self.config.kept_min.as_secs())))); @@ -1106,11 +1121,23 @@ impl Service { /// Whether `client` may submit a job planned as `plan` now, by every /// limit but its submission window: the refusal it would get, or nothing. - pub fn would_refuse(&self, client: &str, plan: &Plan) -> Option { + /// `bound` is the most its result takes as served (`http::plan_job`): + /// past [`Config::result_mib`] it is refused at once, before it runs. + pub fn would_refuse(&self, client: &str, plan: &Plan, bound: Option) -> Option { let job_mib = plan.memory_bytes.div_ceil(1 << 20); if job_mib > self.config.job_mib { return Some(Refused(422, tag(refusal::TOO_LARGE_HERE, format!("the {} method would need about {} MiB for this circuit; one job may take at most {} MiB", plan.method, job_mib, self.config.job_mib)))); } + let cap = self.config.result_mib << 20; + if let Some(b) = bound.filter(|&b| b > cap) { + return Some(Refused(422, tag(refusal::TOO_LARGE_HERE, format!("its result could take up to {:.1} MiB as served (its counts listing up to the shots, 2^bits or 65,536 outcomes an item, for every item), more than the {} MiB one job's result here may take; send fewer points, fewer shots or fewer measured bits, split it into several jobs, or run it on your own machine (`wai-quantum run`)", b as f64 / f64::from(1 << 20), self.config.result_mib)))); + } + let name = self.name(client); + let (mine, _) = self.unread(&name); + let room = mine + self.running_results(&name) + bound.unwrap_or(cap).min(cap); + if room > self.config.client_unread_mib << 20 { + return Some(Refused(429, tag(refusal::RATE, format!("results of yours not yet read hold about {} MiB, and with your jobs running and this one they could hold {} MiB, more than the {} MiB one client's unread results may; read them (GET /v1/jobs/{{id}}) before sending more, or wait {} s", mine >> 20, room.div_ceil(1 << 20), self.config.client_unread_mib, self.config.kept_min.as_secs())))); + } let (active, held) = self.held(&self.name(client)); if active >= self.config.per_client_active { return Some(Refused(429, tag(refusal::RATE, format!("{active} of your jobs are queued or running; wait for one to finish (at most {} at once)", self.config.per_client_active)))); @@ -1168,19 +1195,21 @@ impl Service { /// the key's job, before any limit, and counts as nothing. #[cfg(test)] pub fn submit_work(&self, client: &str, request: Value, plan: Plan, work: Work, key: Option) -> Result { - self.keep_work(client, request, plan, work, key, None, true) + self.keep_work(client, request, plan, work, key, None, None, true) } /// [`Service::submit_work`] for a submission already counted against /// its client's window (`http::admit` counts before it reads anything), /// `received` being the digest of the request as it came, where that is - /// not the request as it read. - pub fn submit_counted(&self, client: &str, request: Value, plan: Plan, work: Work, key: Option, received: Option) -> Result { - self.keep_work(client, request, plan, work, key, received, false) + /// not the request as it read, and `bound` the most its result takes as + /// served (`http::plan_job`). + #[allow(clippy::too_many_arguments)] + pub fn submit_counted(&self, client: &str, request: Value, plan: Plan, work: Work, key: Option, received: Option, bound: Option) -> Result { + self.keep_work(client, request, plan, work, key, received, bound, false) } #[allow(clippy::too_many_arguments)] - fn keep_work(&self, client: &str, request: Value, plan: Plan, work: Work, key: Option, received: Option, count: bool) -> Result { + fn keep_work(&self, client: &str, request: Value, plan: Plan, work: Work, key: Option, received: Option, bound: Option, count: bool) -> Result { let name = self.name(client); let canonical = canon::canonical(&request); let request_digest = store::digest(&canonical); @@ -1193,7 +1222,7 @@ impl Service { return Ok(id); } // A full queue is checked again below, as the job is queued. - if let Some(r) = self.would_refuse(client, &plan) + if let Some(r) = self.would_refuse(client, &plan, bound) && r.0 != 503 { return Err(r); @@ -1233,6 +1262,7 @@ impl Service { interruptions: Vec::new(), child: None, kept: 0, read: false, + bound: bound.unwrap_or(u64::MAX), }; { let mut queue = self.queue.lock().unwrap(); @@ -1554,8 +1584,12 @@ impl Service { // Served as RFC 8785 text, receipt and all, as a restart reads it back. result["receipt"] = receipt; let text = served_text(&result); + // A backstop: a job is admitted only on a bound of its result + // within the cap ([`Service::would_refuse`]), so this is a job + // kept across a restart that lowered the cap, or one whose + // method left its result unbounded (a pulse run). if text.len() as u64 > self.config.result_mib << 20 { - return Err(tag(refusal::TOO_LARGE_HERE, format!("its result is {} MiB as served, more than the {} MiB one job here may give; send fewer points or fewer shots, or run it on your own machine (`wai-quantum run`)", text.len() >> 20, self.config.result_mib))); + return Err(tag(refusal::TOO_LARGE_HERE, format!("its result is {} MiB as served, more than the {} MiB one job's result here may take; send fewer points or fewer shots, or run it on your own machine (`wai-quantum run`)", text.len() >> 20, self.config.result_mib))); } match (self.store.put(&bare), self.store.put(&canon::canonical(&result["receipt"]))) { (Ok(r), Ok(s)) => kept = (Some(r), Some(s)), @@ -2074,7 +2108,7 @@ mod tests { let job = |client: &str, k: usize, ended: Option| { let request = pad(k); let text = String::from_utf8(canon::canonical(&request)).unwrap(); - let mut j = Job { id: job_id(), client: s.name(client), request: text.into(), plan: plan(1 << 20), work: Work::One, status: if ended.is_some() { Status::Done } else { Status::Running }, submitted_ms: 0, started_ms: None, finished_ms: ended, result: Some(served_text(&serde_json::json!({ "counts": { "0": 1 } }))), error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0, read: false }; + let mut j = Job { id: job_id(), client: s.name(client), request: text.into(), plan: plan(1 << 20), work: Work::One, status: if ended.is_some() { Status::Done } else { Status::Running }, submitted_ms: 0, started_ms: None, finished_ms: ended, result: Some(served_text(&serde_json::json!({ "counts": { "0": 1 } }))), error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0, read: false, bound: u64::MAX }; if ended.is_some() { j.ended(); } @@ -2123,7 +2157,7 @@ mod tests { let s = Service::new(Config { client_kept_mib: 4, kept_mib: 2, client_unread_mib: 64, result_mib: 64, ..config() }); let job = |client: &str, ended: u64, read: bool| { let text = String::from_utf8(canon::canonical(&serde_json::json!({ "qasm": "OPENQASM 3.0; qubit q; h q;", "pad": vec![1; 300_000] }))).unwrap(); - let mut j = Job { id: job_id(), client: s.name(client), request: text.into(), plan: plan(1 << 20), work: Work::One, status: Status::Done, submitted_ms: 0, started_ms: None, finished_ms: Some(ended), result: None, error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0, read }; + let mut j = Job { id: job_id(), client: s.name(client), request: text.into(), plan: plan(1 << 20), work: Work::One, status: Status::Done, submitted_ms: 0, started_ms: None, finished_ms: Some(ended), result: None, error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0, read, bound: u64::MAX }; j.ended(); let id = j.id.clone(); s.jobs.lock().unwrap().insert(id.clone(), j); @@ -2141,6 +2175,34 @@ mod tests { assert!(s.refused_before_reading("fresh").is_none(), "another client is not refused for it"); } + /// A job whose result's bound passes the cap on a job's result is + /// refused before it runs (`422`, `J031`), naming the bound and the cap; + /// one at the cap is taken. A client's jobs still running are counted + /// against its unread allowance at their own bounds, where each was + /// counted at the cap: seven small jobs running leave room for an + /// eighth, which seven caps had refused. + #[test] + fn a_result_s_bound_is_held_to_the_cap_and_running_jobs_count_their_own() { + let s = Service::new(Config { result_mib: 1, client_unread_mib: 4, per_client_active: 8, max_queue: 16, ..config() }); + let p = plan(1 << 20); + let Refused(code, why) = s.would_refuse("a", &p, Some((1 << 20) + 1)).expect("past the cap"); + assert!(code == 422 && why.contains("J031") && why.contains("1.0 MiB") && why.contains("the 1 MiB"), "{code} {why}"); + assert!(s.would_refuse("a", &p, Some(1 << 20)).is_none()); + for k in 0..7 { + s.keep_work("a", json!({ "qasm": "OPENQASM 3.0; qubit q; h q;", "shots": k + 1 }), p.clone(), Work::One, None, None, Some(4096), false).unwrap(); + } + assert!(s.refused_before_reading("a").is_none(), "seven small results and a next job's room fit in 4 MiB"); + assert!(s.would_refuse("a", &p, Some(4096)).is_none()); + // Not known (a pulse run), a job running is counted at the cap. + let t = Service::new(Config { result_mib: 1, client_unread_mib: 4, per_client_active: 8, max_queue: 16, ..config() }); + for k in 0..3 { + t.keep_work("a", json!({ "qasm": "OPENQASM 3.0; qubit q; h q;", "shots": k + 1 }), p.clone(), Work::One, None, None, None, false).unwrap(); + } + assert_eq!(t.refused_before_reading("a").map(|r| r.0), None, "three caps and a next job's fit in 4 MiB"); + t.keep_work("a", json!({ "qasm": "OPENQASM 3.0; qubit q; h q;", "shots": 9 }), p.clone(), Work::One, None, None, None, false).unwrap(); + assert_eq!(t.refused_before_reading("a").map(|r| r.0), Some(429), "four caps and a next job's do not"); + } + fn ids_of(s: &Service, client: &str) -> Vec { let name = s.name(client); s.jobs.lock().unwrap().values().filter(|j| j.client == name).map(|j| j.id.clone()).collect() @@ -2463,7 +2525,7 @@ mod tests { let key = wai_quantum::ed25519_dalek::SigningKey::from_bytes(&[19; 32]); let request = json!({ "qasm": "OPENQASM 3.0; qubit[2] q; h q[0]; cx q[0], q[1];", "shots": 20, "observables": ["Z0 Z1"] }); let (p, mut result) = wai_quantum::engine::job::plan_and_run(&request, 1 << 30).unwrap(); - let mut job = Job { id: job_id(), client: "test".into(), request: request.to_string().into(), plan: p, work: Work::One, status: Status::Queued, submitted_ms: now_ms(), started_ms: None, finished_ms: None, result: None, error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0, read: false }; + let mut job = Job { id: job_id(), client: "test".into(), request: request.to_string().into(), plan: p, work: Work::One, status: Status::Queued, submitted_ms: now_ms(), started_ms: None, finished_ms: None, result: None, error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0, read: false, bound: u64::MAX }; let r = crate::mcp::schemas(); let check = |j: &Job| { let v = j.to_json(); @@ -2493,7 +2555,7 @@ mod tests { let items = wai_quantum::engine::sets::expand(&set_request).unwrap(); let plans = wai_quantum::engine::sets::plan(&items, 1 << 30).unwrap(); let body = wai_quantum::engine::sets::run(&set_request, 1 << 30, &key, json!({ "unmetered": "a test" })).unwrap(); - let set = Job { id: job_id(), client: "test".into(), request: set_request.to_string().into(), plan: wai_quantum::engine::sets::summary(&plans), work: Work::Set { items, plans }, status: Status::Done, submitted_ms: now_ms(), started_ms: Some(now_ms()), finished_ms: Some(now_ms()), result: Some(served_text(&body["result"])), error: None, peak_rss: 1 << 20, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0, read: false }; + let set = Job { id: job_id(), client: "test".into(), request: set_request.to_string().into(), plan: wai_quantum::engine::sets::summary(&plans), work: Work::Set { items, plans }, status: Status::Done, submitted_ms: now_ms(), started_ms: Some(now_ms()), finished_ms: Some(now_ms()), result: Some(served_text(&body["result"])), error: None, peak_rss: 1 << 20, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0, read: false, bound: u64::MAX }; let done = check(&set); assert_eq!(wai_quantum::engine::sets::verify(&done)["valid"], true, "{done}"); } @@ -2602,7 +2664,7 @@ mod tests { let s = Service::new(config()); let p = Plan { method: "mps".into(), why: "".into(), memory_bytes: 1, work: 1_000_000_000, qubits: 30, target: None, metered: true }; for _ in 0..3 { - let job = Job { id: job_id(), client: "x".into(), request: "{}".into(), plan: p.clone(), work: Work::One, status: Status::Done, submitted_ms: 0, started_ms: None, finished_ms: Some(1), result: Some(served_text(&json!({ "elapsed_ms": 500, "energy": { "joules_micro": 5_000_000, "energy_uncertainty": { "window_us": 500_000 } } }))), error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0, read: false }; + let job = Job { id: job_id(), client: "x".into(), request: "{}".into(), plan: p.clone(), work: Work::One, status: Status::Done, submitted_ms: 0, started_ms: None, finished_ms: Some(1), result: Some(served_text(&json!({ "elapsed_ms": 500, "energy": { "joules_micro": 5_000_000, "energy_uncertainty": { "window_us": 500_000 } } }))), error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0, read: false, bound: u64::MAX }; s.time(&job, &serde_json::from_str(job.result.as_deref().unwrap_or("null")).unwrap()); } let (t, _) = s.estimate(&p); @@ -2621,7 +2683,7 @@ mod tests { assert!(t["seconds"].is_null() && t["at_most"] == 5, "{t}"); // Timed jobs here take over, and their metered power gives an energy estimate. for _ in 0..3 { - let job = Job { id: job_id(), client: "x".into(), request: "{}".into(), plan: Plan { work: 1_000_000_000, ..p.clone() }, work: Work::One, status: Status::Done, submitted_ms: 0, started_ms: None, finished_ms: Some(1), result: Some(served_text(&json!({ "elapsed_ms": 500, "energy": { "joules_micro": 5_000_000, "energy_uncertainty": { "window_us": 500_000 } } }))), error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0, read: false }; + let job = Job { id: job_id(), client: "x".into(), request: "{}".into(), plan: Plan { work: 1_000_000_000, ..p.clone() }, work: Work::One, status: Status::Done, submitted_ms: 0, started_ms: None, finished_ms: Some(1), result: Some(served_text(&json!({ "elapsed_ms": 500, "energy": { "joules_micro": 5_000_000, "energy_uncertainty": { "window_us": 500_000 } } }))), error: None, peak_rss: 0, digests: Digests::default(), key: None, received: None, interruptions: Vec::new(), child: None, kept: 0, read: false, bound: u64::MAX }; s.time(&job, &serde_json::from_str(job.result.as_deref().unwrap_or("null")).unwrap()); } let (t, e) = s.estimate(&p); diff --git a/wai/wai-quantum-service/src/mcp.rs b/wai/wai-quantum-service/src/mcp.rs index 05d1ed393..63f800722 100644 --- a/wai/wai-quantum-service/src/mcp.rs +++ b/wai/wai-quantum-service/src/mcp.rs @@ -1006,7 +1006,7 @@ async fn call(app: &App, who: &str, name: &str, args: Value, base: &str) -> Valu }, "limits": { "job_seconds": c.job_timeout.as_secs(), "active_jobs_per_client": c.per_client_active, "memory_per_client_bytes": c.per_client_mib << 20, "submissions_per_window": c.per_client_burst, "window_seconds": c.per_client_window.as_secs() }, "load": { "queued": queued, "running": running }, - "sets": { "most_items": wai_quantum::engine::sets::MAX_ITEMS }, + "sets": { "most_items": wai_quantum::engine::sets::MAX_ITEMS, "result_bytes": c.result_mib << 20, "note": "a job or set whose result could take more as served, by its bound before it runs, is refused at once (J031)" }, "store": { "kind": app.service.store().kind(), "durable": app.service.store().durable(), "retention_seconds": c.retain.as_secs() }, "energy": crate::energy::meter().describe(), "receipt_keys": format!("{base}/.well-known/wai-quantum-receipt-keys"), diff --git a/wai/wai-quantum-service/src/offload.rs b/wai/wai-quantum-service/src/offload.rs index 327175d68..c8b7fee1d 100644 --- a/wai/wai-quantum-service/src/offload.rs +++ b/wai/wai-quantum-service/src/offload.rs @@ -363,7 +363,8 @@ impl Gate { pub enum Read { /// A job or a set: its source read where it has one, held to the checks /// that read and planned within `job_bytes` (`http::plan_job`). The answer - /// is `{"request", "plan", "items", "plans"}`, or `{"request", "refused": + /// is `{"request", "plan", "items", "plans", "result_bytes"}` (the most + /// its result takes as served, or null), or `{"request", "refused": /// [status, why]}` where the planning refused it. With `keep` false (a /// dry run, which keeps nothing) the request and each item's request are /// left out: a dry run reads the plans and the points alone. @@ -501,7 +502,7 @@ pub fn run(r: Read) -> Result { // A refusal of the plan is the answer's, beside the request as read. let planned = crate::http::plan_job(job_bytes, &request); let request = if keep { request } else { Value::Null }; - let (p, work) = match planned { + let (p, work, bound) = match planned { Ok(planned) => planned, Err((code, why)) => return Ok(json!({ "request": request, "refused": [code, why] })), }; @@ -512,7 +513,7 @@ pub fn run(r: Read) -> Result { (items, serde_json::to_value(plans).expect("plans are JSON")) } }; - Ok(json!({ "request": request, "plan": p, "items": items, "plans": plans })) + Ok(json!({ "request": request, "plan": p, "items": items, "plans": plans, "result_bytes": bound })) }), Read::Check { check, body } => { let check = crate::analysis::Check::named(&check).ok_or((400, format!("no check {check}")))?; diff --git a/wai/wai-quantum-service/tests/depth.rs b/wai/wai-quantum-service/tests/depth.rs index ba010a11e..7384d6b28 100644 --- a/wai/wai-quantum-service/tests/depth.rs +++ b/wai/wai-quantum-service/tests/depth.rs @@ -1270,31 +1270,93 @@ fn a_documented_qec_call_beside_another_check_from_its_client_is_answered() { let _ = std::fs::remove_dir_all(&d); } -/// Long polls and event streams, 210 of them from 8 clients, where the -/// service's open-file limit is 256 and cannot be raised, hold no more than -/// their share of its connections: a health check is answered throughout, -/// and a long poll past them is answered at once with the job as it is. +/// The program the round-5 verifier kept the service busy with: a layer of +/// T gates and a CX ladder, thirty times over, one shot. It runs for tens of +/// seconds (the verifier's 22 qubits took about 35 s on its host; 24 here, so +/// a faster host still runs it past every wait below), so a `?wait=25` poll +/// on it holds its whole wait. +fn pump() -> Value { + let n = 24; + let mut q = format!("OPENQASM 3.0;\nqubit[{n}] q;\nbit[{n}] c;\n"); + for _ in 0..30 { + q += "t q;\n"; + for i in 0..n - 1 { + q += &format!("cx q[{i}], q[{}];\n", i + 1); + } + } + q += "c = measure q;\n"; + json!({ "qasm": q, "method": "auto", "shots": 1 }) +} + +/// Whether a connection is still open, and what has been answered on it so +/// far, read without waiting. +fn so_far(s: &mut TcpStream) -> (bool, Vec) { + let _ = s.set_nonblocking(true); + let (mut got, mut b) = (Vec::new(), [0u8; 4096]); + let open = loop { + match s.read(&mut b) { + Ok(0) => break false, + Ok(n) => got.extend_from_slice(&b[..n]), + Err(e) if e.kind() == std::io::ErrorKind::WouldBlock => break true, + Err(_) => break false, + } + }; + let _ = s.set_nonblocking(false); + (open, got) +} + +/// The round-5 verifier's flood, as its probe sent it: where the service's +/// open-file limit is 256 and cannot be raised (192 connections), 210 long +/// polls (`?wait=25`) on a running job, each from an address of its own +/// (`10.0.k.m`), opened 3 ms apart, then health checked 25 times. They held +/// every connection, none could be closed for room, and every health check +/// was answered `503`. Long polls are held to a share of the connections, +/// past it answered at once with the job as it is, and four connections past +/// the cap are kept for health checks: each is answered `200`. The same for +/// event streams, which held their connections as long polls did. Where this +/// test process holds fewer connections (a suite run under `ulimit -n 256`), +/// the service's cap is lowered as far, so they still outnumber it. #[cfg(unix)] #[test] fn long_polls_and_streams_leave_a_health_check_answered() { room_for(8192); + let n = can_hold(210); + let cap = (n * 192 / 210).max(16).to_string(); + let mut env = vec![("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1")]; + if n < 210 { + env.push(("WAI_QSVC_CONNECTIONS", cap.as_str())); + } let d = dir("long-polls"); - let (mut svc, port) = start_limited(&d, "ulimit -Sn 256 && ulimit -Hn 256", &[("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1")]); - let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&long_job())); + let (mut svc, port) = start_limited(&d, "ulimit -Sn 256 && ulimit -Hn 256", &env); + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&pump())); assert_eq!(code, 202, "{b}"); let id = b["id"].as_str().unwrap().to_string(); - let held: Vec = (0..can_hold(210)) - .filter_map(|i| { - let mut s = TcpStream::connect(("127.0.0.1", port)).ok()?; - let path = if i % 3 == 0 { format!("/v1/jobs/{id}/events") } else { format!("/v1/jobs/{id}?wait=25") }; - let _ = write!(s, "GET {path} HTTP/1.1\r\nHost: 127.0.0.1\r\nX-Forwarded-For: 198.51.100.{}\r\n\r\n", i % 8 + 1); - Some(s) - }) - .collect(); - std::thread::sleep(Duration::from_secs(1)); - let ok = (0..25).filter(|_| healthy(port)).count(); - assert_eq!(ok, 25, "health answered {ok} of 25 beside 210 long polls and streams: {}", logs(&d)); - drop(held); + for (what, path) in [("long polls", format!("/v1/jobs/{id}?wait=25")), ("event streams", format!("/v1/jobs/{id}/events"))] { + let mut held: Vec = (0..n) + .filter_map(|i| { + let mut s = TcpStream::connect(("127.0.0.1", port)).ok()?; + let _ = write!(s, "GET {path} HTTP/1.1\r\nHost: x\r\nX-Forwarded-For: 10.0.{}.{}\r\n\r\n", i % 8, i / 8 + 1); + std::thread::sleep(Duration::from_millis(3)); + Some(s) + }) + .collect(); + std::thread::sleep(Duration::from_secs(3)); + let ok = (0..25).filter(|_| healthy(port)).count(); + // The flood was real: the job still runs, and a long poll is held + // with nothing answered, a stream with its answer begun. + let waiting = held + .iter_mut() + .map(so_far) + .filter(|(open, got)| *open && if what == "long polls" { got.is_empty() } else { got.starts_with(b"HTTP/1.1 200") }) + .count(); + let sent = held.len(); + drop(held); + std::thread::sleep(Duration::from_millis(500)); + let (_, now) = json_of(port, "GET", &format!("/v1/jobs/{id}"), None); + assert!(now["status"] == "queued" || now["status"] == "running", "{what}: the job ended before the checks, so nothing was held for it: {now}"); + assert!(waiting > 0, "{what}: none of {sent} held"); + assert_eq!(ok, 25, "{what}: health answered {ok} of 25 beside {sent} {what}, {waiting} of them held: {}", logs(&d)); + } let _ = http(port, "DELETE", &format!("/v1/jobs/{id}"), None); alive(port, &mut svc, "long polls and streams"); svc.kill().unwrap(); @@ -1302,34 +1364,110 @@ fn long_polls_and_streams_leave_a_health_check_answered() { let _ = std::fs::remove_dir_all(&d); } -/// The README's own burst from one client: a decode, a threshold and six -/// critiques at once, every one answered; a long check that moves onto the -/// client's own turn gives back its lent one, so the critiques pass it. +/// [`post_as`] for the MCP endpoint: a tool call, and whether it was answered +/// with no error. +fn mcp_as(port: u16, who: &str, tool: &str, args: &Value) -> (bool, String) { + let body = json!({ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": tool, "arguments": args } }); + let mut s = TcpStream::connect(("127.0.0.1", port)).unwrap(); + s.set_read_timeout(Some(Duration::from_secs(150))).unwrap(); + let text = body.to_string(); + write!(s, "POST /mcp HTTP/1.1\r\nHost: 127.0.0.1\r\nContent-Type: application/json\r\nAccept: application/json\r\nX-Forwarded-For: {who}\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{text}", text.len()).unwrap(); + let mut out = String::new(); + let _ = s.read_to_string(&mut out); + let answer: Value = out.split_once("\r\n\r\n").and_then(|(_, b)| serde_json::from_str(b).ok()).unwrap_or(Value::Null); + (out.starts_with("HTTP/1.1 200") && answer["result"].is_object() && answer["result"]["isError"] != true, out) +} + +/// The README's threshold (`qec threshold --distances 3,5,7 --p +/// 0.002,0.004 --shots 20000`) made longer, as the round-5 verifier made it +/// longer (30,000 shots on its host, where that took about 15 s), until it +/// takes 11 s alone here, past the 10 s a request waits for a turn: its +/// shots raised half again at a time to the 200,000 a threshold here takes +/// in all, then its rounds a point raised from the distance to 9, 11, 13 (21 +/// at most here), for as long as the service still takes it within the 25 s +/// an error-correction call may take. The longest call it took, and how +/// long that took. +fn slow_threshold(port: u16) -> (Value, f64) { + let run = |t: &Value| -> Option { + let began = Instant::now(); + let (code, out) = post_as(port, "203.0.113.99", "/v1/qec/threshold", t); + let secs = began.elapsed().as_secs_f64(); + eprintln!("the threshold alone, {} shots, rounds {}: {code} in {secs:.1} s", t["shots"], t["rounds"]); + match code { + 200 => Some(secs), + 422 if out.contains("J031") => None, + _ => panic!("the threshold alone: {code}: {}", out.lines().last().unwrap_or("")), + } + }; + let mut t = json!({ "distances": [3, 5, 7], "p": [0.002, 0.004], "shots": 20000 }); + let mut best = (t.clone(), run(&t).expect("the README's threshold is taken")); + while best.1 < 11.0 { + let shots = t["shots"].as_u64().unwrap_or(20000); + let rounds = t["rounds"].as_u64().unwrap_or(7); + if shots < 33_333 { + t["shots"] = json!((shots * 3 / 2).min(33_333)); + } else if rounds < 13 { + t["rounds"] = json!(rounds + 2); + } else { + break; + } + match run(&t) { + Some(secs) => best = (t.clone(), secs), + None => break, + } + } + best +} + +/// The round-5 verifier's burst, at the service's own limits: one client +/// sends the README's decode (`--memory 5,5,0.003 --shots 20000 --decoder +/// correlated`), a threshold, and six critiques of a Bell pair, 30 ms apart, +/// over HTTP and then over MCP. The threshold, on a turn lent beside the +/// decode, moved onto the client's own turn when the decode ended and kept +/// the lent one, so no critique could have either: each waited out its 10 s +/// and was refused (`J032`), or, with the threshold under 10 s, was answered +/// only as it ended. A moved call gives its lent turn back: every call is +/// answered, each critique well before the threshold. #[test] fn the_readme_s_burst_from_one_client_is_answered() { let d = dir("readme-burst"); - let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1"), ("WAI_QSVC_QEC_SECS", "100"), ("WAI_QSVC_READ_SECS", "100"), ("WAI_QSVC_ANSWER_SECS", "110")]); - let me = "198.51.100.77"; + let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1")]); let decode = json!({ "memory": { "distance": 5, "rounds": 5, "p": 0.003 }, "shots": 20000, "decoder": "correlated" }); - let threshold = json!({ "distances": [3, 5, 7], "p": [0.002, 0.004], "shots": 20000 }); + let (threshold, alone) = slow_threshold(port); let critique = json!({ "qasm": "OPENQASM 3.0;\nqubit[2] q;\nh q[0];\ncx q[0], q[1];\n" }); - let mut calls: Vec<(&str, &Value)> = vec![("/v1/qec/decode", &decode), ("/v1/qec/threshold", &threshold)]; - calls.extend(std::iter::repeat_n(("/v1/critique", &critique), 6)); - let answers: Vec<(u16, String)> = std::thread::scope(|t| { - let each: Vec<_> = calls - .iter() - .enumerate() - .map(|(i, (path, body))| { - t.spawn(move || { - std::thread::sleep(Duration::from_millis(30 * i as u64)); - post_as(port, me, path, body) + let mut calls: Vec<(&str, &str, &Value)> = vec![("/v1/qec/decode", "qec_decode", &decode), ("/v1/qec/threshold", "qec_threshold", &threshold)]; + calls.extend(std::iter::repeat_n(("/v1/critique", "critique_qasm", &critique), 6)); + for (route, me) in [("HTTP", "198.51.100.77"), ("MCP", "198.51.100.78")] { + let began = Instant::now(); + let answers: Vec<(bool, String, f64)> = std::thread::scope(|t| { + let each: Vec<_> = calls + .iter() + .enumerate() + .map(|(i, (path, tool, body))| { + let began = &began; + t.spawn(move || { + std::thread::sleep(Duration::from_millis(30 * i as u64)); + let (ok, out) = if route == "HTTP" { + let (code, out) = post_as(port, me, path, body); + (code == 200, out) + } else { + mcp_as(port, me, tool, body) + }; + (ok, out, began.elapsed().as_secs_f64()) + }) }) - }) - .collect(); - each.into_iter().map(|c| c.join().unwrap()).collect() - }); - for ((path, _), (code, out)) in calls.iter().zip(&answers) { - assert_eq!(*code, 200, "{path}: {}", out.lines().last().unwrap_or("")); + .collect(); + each.into_iter().map(|c| c.join().unwrap()).collect() + }); + let at: Vec = calls.iter().zip(&answers).map(|((path, tool, _), (ok, _, s))| format!("{} {} at {s:.2} s", if route == "HTTP" { path } else { tool }, if *ok { "answered" } else { "refused" })).collect(); + eprintln!("{route}, the threshold alone {alone:.1} s: {}", at.join(", ")); + for ((path, ..), (ok, out, s)) in calls.iter().zip(&answers) { + assert!(*ok, "{route}: {path} at {s:.2} s (the threshold alone {alone:.1} s): {}\n{at:#?}", out.lines().last().unwrap_or("")); + } + let threshold_at = answers[1].2; + for (_, _, s) in &answers[2..] { + assert!(*s + 1.0 < threshold_at, "{route}: a critique answered at {s:.2} s, the threshold at {threshold_at:.2} s (alone {alone:.1} s): the critiques waited for it\n{at:#?}"); + } } alive(port, &mut svc, "the README's burst"); svc.kill().unwrap(); @@ -1354,16 +1492,55 @@ fn an_mcp_id_is_echoed_exactly() { let _ = std::fs::remove_dir_all(&d); } -/// A job whose result is larger than one job may give fails, saying so, -/// rather than being kept past every cap or dropped unread. +/// A job's result is held to `WAI_QSVC_RESULT_MIB` before it runs, where a +/// result past it was run and then failed: a sweep whose result's bound +/// passes the cap is refused at once (`422`, `J031`), naming its bound and +/// the cap, on every route that takes a job, and its dry run says so; the +/// largest sweep under the cap is taken and ends done, its result within its +/// bound. At the defaults, the published set at its most, 100 items of +/// 65,536 outcomes each (about 150 MB as served), is refused, and 100 +/// points of 30,000 shots of 16 bits each are taken. #[test] -fn a_result_past_what_a_job_may_give_fails_and_says_so() { +fn a_result_past_what_a_job_may_give_is_refused_before_it_runs() { + let sweep = |points: usize, shots: u64| json!({ "qasm": "OPENQASM 3.0;\ninput float theta;\nqubit[16] q;\nbit[16] c;\nh q;\nrz(theta) q[0];\nc = measure q;\n", "sweep": { "theta": (0..points).map(|i| 0.01 * (i + 1) as f64).collect::>() }, "shots": shots, "seed": 7 }); + let dry = |port: u16, v: &Value| { + let mut v = v.clone(); + v["dry_run"] = json!(true); + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&v)); + assert_eq!(code, 200, "{b}"); + b + }; let d = dir("result-cap"); let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_RESULT_MIB", "1")]); - let sweep = json!({ "qasm": "OPENQASM 3.0;\ninput float theta;\nqubit[16] q;\nbit[16] c;\nh q;\nrz(theta) q[0];\nc = measure q;\n", "sweep": { "theta": (0..10).map(|i| 0.01 * (i + 1) as f64).collect::>() }, "shots": 20000, "seed": 7 }); - let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&sweep)); + let cap = 1u64 << 20; + assert_eq!(dry(port, &sweep(1, 2000))["bounds"]["result_bytes"].as_u64(), Some(cap)); + // The most points of 2,000 shots a point under the cap, by the dry runs' bounds. + let bound = |n: usize| dry(port, &sweep(n, 2000))["result_bytes"].as_u64().unwrap(); + let (one, two) = (bound(1), bound(2)); + let mut points = ((cap - (2 * one - two)) / (two - one)) as usize; + while bound(points) > cap { + points -= 1; + } + while bound(points + 1) <= cap { + points += 1; + } + let past = dry(port, &sweep(points + 1, 2000)); + assert!(past["accepted"] == false && past["status"] == 422 && past["refusal"]["code"] == "J031", "{past}"); + // Refused at once, on every route that takes a job. + let mcp = |args: Value| json!({ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "run_quantum_set", "arguments": args } }); + for (path, body) in [("/v1/jobs", sweep(points + 1, 2000)), ("/v1/run", sweep(points + 1, 2000)), ("/mcp", mcp(sweep(points + 1, 2000)))] { + let began = Instant::now(); + let (code, b) = json_of(port, "POST", path, Some(&body)); + let why = if path == "/mcp" { b["result"]["content"][0]["text"].as_str().unwrap_or("").to_string() } else { b["error"].as_str().unwrap_or("").to_string() }; + assert!(if path == "/mcp" { code == 200 && b["result"]["isError"] == true } else { code == 422 }, "{path}: {code} {b}"); + assert!(why.contains("J031") && why.contains("could take up to") && why.contains("the 1 MiB"), "{path}: {why}"); + assert!(began.elapsed() < Duration::from_secs(10), "{path}: answered after {:?}", began.elapsed()); + } + // At the bound: taken, and done, its result within its bound. + let at = bound(points); + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&sweep(points, 2000))); assert_eq!(code, 202, "{b}"); - let id = b["id"].as_str().unwrap(); + let id = b["id"].as_str().unwrap().to_string(); let t = Instant::now(); let job = loop { let (code, b) = json_of(port, "GET", &format!("/v1/jobs/{id}?wait=25"), None); @@ -1373,9 +1550,22 @@ fn a_result_past_what_a_job_may_give_fails_and_says_so() { } assert!(t.elapsed() < Duration::from_secs(120)); }; - assert_eq!(job["status"], "failed", "{}", job["status"]); - assert!(job["error"].as_str().is_some_and(|e| e.contains("MiB as served") && e.contains("J031")), "{}", job["error"]); - alive(port, &mut svc, "a result past its cap"); + assert_eq!(job["status"], "done", "{}", job["error"]); + let served = wai_quantum::engine::receipt::canonical(&job["result"]).len() as u64; + eprintln!("{points} points of 2,000 shots: bound {at} bytes, served {served}"); + assert!(served <= at && at <= cap, "{points} points: served {served} bytes, bound {at}"); + alive(port, &mut svc, "results held to their cap"); + svc.kill().unwrap(); + svc.wait().unwrap(); + // The defaults. + let (mut svc, port) = start(&d, 1000); + let most = dry(port, &sweep(100, 1_000_000)); + let bytes = most["result_bytes"].as_u64().unwrap_or(0); + eprintln!("100 points of 65,536 outcomes: bound {bytes} bytes"); + assert!(most["accepted"] == false && most["refusal"]["code"] == "J031" && (130_000_000..180_000_000).contains(&bytes), "{most}"); + let taken = dry(port, &sweep(100, 30_000)); + eprintln!("100 points of 30,000 shots: bound {} bytes", taken["result_bytes"]); + assert!(taken["accepted"] == true, "{taken}"); svc.kill().unwrap(); svc.wait().unwrap(); let _ = std::fs::remove_dir_all(&d); diff --git a/wai/wai-quantum/src/engine/catalog.rs b/wai/wai-quantum/src/engine/catalog.rs index 5d2cf4dc8..98e40d93f 100644 --- a/wai/wai-quantum/src/engine/catalog.rs +++ b/wai/wai-quantum/src/engine/catalog.rs @@ -1249,6 +1249,83 @@ pub fn plan(v: &Value, job_bytes: u64) -> Result { plan_in(Registry::builtins(), v, job_bytes) } +/// [`plan`], with the most its result may take as JSON text, in bytes, +/// beside what a host adds to it (a receipt, an energy figure), known before +/// it runs ([`result_bound`]); `None` for a method that reads its program +/// itself (a pulse run), whose result this reads nothing of. +pub fn plan_bounded(v: &Value, job_bytes: u64) -> Result<(Plan, Option), String> { + plan_bounded_in(Registry::builtins(), v, job_bytes) +} + +/// What a result holds beside the parts [`result_bound`] counts one by one: +/// its method, qubits and gates, its accuracy and its notes, its state's +/// hash, its schema, its timing and its provenance, the plan's reason apart. +const RESULT_REST: u64 = 16 << 10; +/// An expectation value's numbers and names, beside its observable's text +/// (a bound, a ladder of thresholds, a standard error). +const EXPECTATION_BYTES: u64 = 1 << 10; +/// An amplitude's numbers and names, beside its bit string. +const AMPLITUDE_BYTES: u64 = 1 << 9; +/// A verdict's own fields, beside its critique's remarks and its intents'. +const VERDICT_BYTES: u64 = 16 << 10; +/// One intent's check, beside the intent as given and the counts it may +/// list ([`super::verdict::MAX_LISTED`]). +const INTENT_BYTES: u64 = 16 << 10; +/// A critique's remarks on one gate of the program in standard gates, at +/// most: a gate draws one that it does nothing, and is one of a pair that +/// cancels or merges, each naming its lines and its qubits. +const REMARKS_PER_GATE: u64 = 512; +/// A mitigation's own fields, beside a value for each observable and a +/// count for each outcome. +const MITIGATION_BYTES: u64 = 64 << 10; + +/// The most the result of the job `v`, read as `r` and planned as `p`, takes +/// as JSON text, bytes: each part that grows with what the job asks bounded +/// by what it may list, from the job as read. Its counts (and a +/// mitigation's) by [`super::run::counts_bound`]; each expectation value by +/// its observable's text, escaped, and a fixed allowance; each amplitude by +/// its bit string and one; what its reading put in it (its diagnostics, its +/// outputs, its repair, its compile and its target) at its size as read; a +/// verdict by its fields, each intent as given and the counts its check may +/// list, and its critique's remarks at [`REMARKS_PER_GATE`] a gate; and the +/// rest at [`RESULT_REST`]. The counts are written exactly so; RFC 8785 +/// writes another number at most a few characters longer than serde does, so +/// a sixteenth more is allowed for the rest. +pub(crate) fn result_bound(v: &Value, r: &Request, p: &Plan) -> u64 { + let size = |x: &Value| serde_json::to_string(x).map_or(0, |s| s.len() as u64); + // A string of the job's own, every character escaped at its longest. + let text = |t: &str| 6 * t.len() as u64 + 2; + let bits = r.program.n_bits; + let digits = r.shots.max(1).to_string().len() as u64; + let counts = super::run::counts_bound(bits, r.shots, None); + let mut b = RESULT_REST + 2 * size(&json!(p.why)); + b += r.observables.iter().map(|(t, _, _)| EXPECTATION_BYTES + text(t)).sum::(); + b += r.amplitudes.len() as u64 * (AMPLITUDE_BYTES + u64::from(r.program.n_qubits)); + b += r.diagnostics.iter().map(|d| size(&d.to_json()) + 1).sum::(); + b += r.program.extra.outputs.iter().map(|o| 64 + text(&o.name) + text(&o.ty) + match &o.value { + qasm::OutputValue::Bits(bits) => 11 * bits.len() as u64, + qasm::OutputValue::Value(_, x) => size(x), + }).sum::(); + let repaired = r.repaired.as_ref().map_or(0, size); + b += repaired + r.compiled.as_ref().map_or(0, size) + r.compile_passes.as_ref().map_or(0, size); + b += p.target.as_ref().map_or(0, size); + if r.mitigate.is_some() { + b += MITIGATION_BYTES + r.observables.len() as u64 * EXPECTATION_BYTES + super::run::counts_bound(bits, r.shots, Some(24)); + } + if super::verdict::asked(v) { + let intents: Vec<&Value> = match v.get("intent") { + Some(Value::Array(a)) => a.iter().collect(), + Some(i) if !i.is_null() => vec![i], + _ => Vec::new(), + }; + let listed = super::verdict::MAX_LISTED as u64 * (u64::from(bits) + 4 + digits); + b += VERDICT_BYTES + repaired + intents.iter().map(|i| INTENT_BYTES + 2 * size(i) + listed).sum::(); + let gates = u64::try_from(r.program.standard_gates()).unwrap_or(u64::MAX); + b = b.saturating_add(gates.saturating_mul(REMARKS_PER_GATE)).saturating_add(u64::from(r.program.n_qubits) * REMARKS_PER_GATE); + } + b.saturating_add(b / 16).saturating_add(counts) +} + /// The method `v` names, when it reads its job's program itself (a pulse /// run: the pulse reader reads it, and the pulse simulator integrates it). pub(crate) fn reads_own<'a>(reg: &'a Registry, v: &Value) -> Option<&'a dyn Method> { @@ -1257,8 +1334,13 @@ pub(crate) fn reads_own<'a>(reg: &'a Registry, v: &Value) -> Option<&'a dyn Meth /// [`plan`] through `reg`: its methods, channels and passes. pub(crate) fn plan_in(reg: &Registry, v: &Value, job_bytes: u64) -> Result { + plan_bounded_in(reg, v, job_bytes).map(|(p, _)| p) +} + +/// [`plan_bounded`] through `reg`. +pub(crate) fn plan_bounded_in(reg: &Registry, v: &Value, job_bytes: u64) -> Result<(Plan, Option), String> { if let Some(m) = reads_own(reg, v) { - return m.plan(&Job::new(reg, v, job_bytes, None, &[])); + return Ok((m.plan(&Job::new(reg, v, job_bytes, None, &[]))?, None)); } // A job that names a device target is compiled, placed and simulated as // the device would run it (engine::targets::job). @@ -1268,7 +1350,10 @@ pub(crate) fn plan_in(reg: &Registry, v: &Value, job_bytes: u64) -> Result Result String { /// The most characters of bit strings a counts map lists. const MAX_COUNTS_CHARS: usize = 16 << 20; +/// The most a counts map ([`counts`]) of `shots` samples of `width`-bit +/// outcomes takes as JSON text, bytes: as many outcomes as it may list (no +/// more than the shots, `2^width`, [`MAX_OUTCOMES`] and what +/// [`MAX_COUNTS_CHARS`] of bit strings hold), each its bit string, two +/// quotes, a colon and a comma, and the map's own fields; with `value`, that +/// many bytes for each outcome's value, and without, its count's digits. `k` +/// counts that sum to `shots` have at most `k·(log10(shots/k) + 1)` digits +/// in all, the most where they are equal. +pub(crate) fn counts_bound(width: u32, shots: u64, value: Option) -> u64 { + if shots == 0 { + return 0; + } + let w = u64::from(width).max(1); + let distinct = if width >= 63 { u64::MAX } else { 1u64 << width }; + let keep = (MAX_OUTCOMES as u64).min((MAX_COUNTS_CHARS as u64 / w).max(1)); + let k = shots.min(distinct).min(keep); + let values = match value { + Some(v) => k * v, + None => ((k as f64) * ((shots as f64 / k as f64).log10() + 1.0)).ceil() as u64 + 16, + }; + k * (u64::from(width) + 4) + values + 128 +} + /// A counts map: the most frequent outcomes (at most [`MAX_OUTCOMES`], and /// at most [`MAX_COUNTS_CHARS`] characters of bit strings), and how many /// shots the omitted ones held. @@ -1068,6 +1091,65 @@ mod tests { run(&job, &p).unwrap() } + /// No result is larger than the bound its plan gave before it ran + /// (`catalog::plan_bounded`), as RFC 8785 writes it with its provenance: + /// the worked examples with their verdicts, and jobs that ask for each + /// part a result grows with (counts at the most a map lists, wide bit + /// strings, measurements in the middle, observables, amplitudes, a + /// critique's remarks on every gate, a repair, a compile, noise and a + /// mitigation, a device target). Where counts make the result, the bound + /// is within a quarter of it. + #[test] + #[cfg(all(feature = "quantum_qasm", feature = "quantum_noise", feature = "quantum_receipt"))] + fn no_result_passes_the_bound_its_plan_gave() { + let wide = format!("OPENQASM 3.0;\nqubit[16] q;\nbit[16] c;\nh q;\nc = measure q;\n"); + let observables: Vec = (0..256).map(|i| format!("{}.5 Z{} X{} Y{}", i % 7, i % 16, (i + 1) % 16, (i + 2) % 16)).collect(); + let amplitudes: Vec = (0..64).map(|i| format!("{:010b}", i * 13)).collect(); + let mut jobs: Vec<(String, Value)> = crate::engine::guide::examples() + .iter() + .map(|e| { + let mut intent = e.intent.to_string(); + if let Some(r) = e.reference { + intent = intent.replace("\"reference.qasm\"", &json!(r).to_string()); + } + let mut j = json!({ "qasm": e.program, "verdict": true, "intent": serde_json::from_str::(&intent).unwrap() }); + j.as_object_mut().unwrap().extend(e.run.as_object().cloned().unwrap_or_default()); + (format!("example {}", e.name), j) + }) + .collect(); + jobs.extend([ + ("65,536 outcomes of 16 bits".to_string(), json!({ "qasm": wide, "shots": 1_000_000, "seed": 3 })), + ("10,000 outcomes of 16 bits".to_string(), json!({ "qasm": wide, "shots": 10_000, "seed": 3 })), + ("outcomes of 300 bits".to_string(), json!({ "qasm": "OPENQASM 3.0;\nqubit[12] q;\nbit[300] c;\nh q;\nfor uint i in [0:11] { c[i * 25] = measure q[i]; }\n", "shots": 20000 })), + ("measured in the middle".to_string(), json!({ "qasm": "OPENQASM 3.0;\nqubit[3] q;\nbit[6] c;\nh q;\nc[0] = measure q[0];\nc[1] = measure q[1];\nc[2] = measure q[2];\nreset q;\nh q;\nc[3] = measure q[0];\nc[4] = measure q[1];\nc[5] = measure q[2];\n", "shots": 5000 })), + ("256 observables".to_string(), json!({ "qasm": "OPENQASM 3.0;\nqubit[16] q;\nh q;\nrx(0.3) q[2];\ncx q[0], q[1];\n", "observables": observables })), + ("64 amplitudes".to_string(), json!({ "qasm": "OPENQASM 3.0;\nqubit[10] q;\nh q;\nrz(0.7) q[3];\n", "amplitudes": amplitudes })), + ("a remark on every gate".to_string(), json!({ "qasm": "OPENQASM 3.0;\nqubit[2] q;\nbit[2] c;\nfor int i in [1:4000] { x q[0]; h q[1]; h q[1]; }\nc = measure q;\n", "shots": 100, "verdict": true })), + ("repaired".to_string(), json!({ "qasm": "OPENQASM 3.0;\nqubit q;\nbit c;\nh q;\nh q;\nx q;\nc = measure q;\n", "shots": 100, "repair": true, "verdict": true })), + ("compiled".to_string(), json!({ "qasm": "OPENQASM 3.0;\nqubit[2] q;\nbit[2] c;\nrz(0.1234) q[0];\nrx(0.77) q[1];\ncx q[0], q[1];\nc = measure q;\n", "shots": 500, "compile": { "epsilon": 1e-3, "level": 2 } })), + ("noisy".to_string(), json!({ "qasm": "OPENQASM 3.0;\nqubit[8] q;\nbit[8] c;\nh q;\ncx q[0], q[1];\nc = measure q;\n", "shots": 30000, "noise": { "depolarizing_1q": 0.01, "readout": 0.02 } })), + ("mitigated".to_string(), json!({ "qasm": "OPENQASM 3.0;\nqubit[3] q;\nh q[0];\ncx q[0], q[1];\ncx q[1], q[2];\n", "observables": ["Z0 Z1", "Z1 Z2"], "noise": { "depolarizing_2q": 0.02, "depolarizing_1q": 0.004 }, "mitigate": { "method": "zne", "shots": 4000 } })), + ("on a device target".to_string(), json!({ "qasm": "OPENQASM 3.0; qubit[3] q; bit[3] c; h q[0]; cx q[0], q[1]; cx q[1], q[2]; c = measure q;", "target": "synthetic-ring-8", "shots": 3000, "seed": 2 })), + ]); + let mut checked = 0; + for (name, job) in &jobs { + let (p, bound) = match crate::engine::catalog::plan_bounded(job, CAP) { + Ok((p, Some(b))) => (p, b), + Ok((_, None)) => panic!("{name}: no bound"), + Err(e) => panic!("{name}: {e}"), + }; + let result = run_with_provenance(job, &p, "a host 1.0").unwrap_or_else(|e| panic!("{name}: {e}")); + let served = crate::engine::receipt::canonical(&result).len() as u64; + eprintln!("{name}: {served} bytes, bound {bound}"); + assert!(served <= bound, "{name}: its result is {served} bytes, past its bound of {bound}"); + if name.contains("outcomes of 16 bits") { + assert!(bound <= served + served / 4 + (64 << 10), "{name}: {served} bytes, bound {bound}"); + } + checked += 1; + } + assert_eq!(checked, jobs.len()); + } + #[test] #[cfg(feature = "quantum_toolchain")] fn grover_as_the_ide_exports_it_finds_the_marked_item() { diff --git a/wai/wai-quantum/src/engine/sets.rs b/wai/wai-quantum/src/engine/sets.rs index 001de11f0..92381147c 100644 --- a/wai/wai-quantum/src/engine/sets.rs +++ b/wai/wai-quantum/src/engine/sets.rs @@ -213,6 +213,12 @@ fn sweep_of(request: &Value, sweep: &Value) -> Result, String> { /// Each item planned within `job_bytes`, or the first that does not plan, /// with its index. pub fn plan(items: &[Item], job_bytes: u64) -> Result, String> { + plan_bounded(items, job_bytes).map(|planned| planned.into_iter().map(|(p, _)| p).collect()) +} + +/// [`plan`], each item's plan with the most its result may take +/// ([`catalog::plan_bounded`]). +pub fn plan_bounded(items: &[Item], job_bytes: u64) -> Result)>, String> { items .iter() .enumerate() @@ -220,7 +226,7 @@ pub fn plan(items: &[Item], job_bytes: u64) -> Result, String if let Some(at) = receipt::inexact_number(&it.request) { return Err(super::refusal::tag(super::refusal::LIMIT, format!("item {i}: `{at}`: a whole number here must be within ±9007199254740991 (2^53 − 1)"))); } - catalog::plan(&it.request, job_bytes).map_err(|e| format!("item {i}: {e}")) + catalog::plan_bounded(&it.request, job_bytes).map_err(|e| format!("item {i}: {e}")) }) .collect() } -- 2.54.0 (Apple Git-157) From 462ba8b377a443153bf9757ff0bb50042c443d8f Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Mon, 9 Nov 2026 07:58:37 -0500 Subject: [PATCH 14/18] wai-quantum-service: a pulse run's result bounded from its schedule, the dry run's result_bytes in the tool schema, and a QEC pilot held to 200 ms A pulse run reads its own program, so its result was not bounded before it ran: it was counted at WAI_QSVC_RESULT_MIB and held to it only as it ended. It is bounded now from its schedule as read (pulse::plan_bounded, through Method::plan_bounded, a default method on the registry's trait that gives no bound): the populations by the qubits it simulates, the distribution by the captures it reads, each keyed as wide as the program's bits, the counts by its shots (run::counts_bound_among), and each capture with its frame's and bit's names. A pulse run of 100,000 bits declared and one captured, about 1.4 MB as served, is refused at once under a 1 MiB cap, where it had been taken and failed as it ended; 100 bits are taken. No built-in method is now unbounded, so the check as a result ends is only a backstop, for a job kept across a restart that lowered the cap. engine::pulse::tests::no_pulse_result_passes_the_bound_its_plan_gave runs one capture, one bit of 3,000 captured, nothing captured and two qubits captured, each within its bound and within a few times it. The dry run's result_bytes, an integer or {"bytes": null, "why"}, is named in schemas/service-tools.v1.json and the CLI's snapshot of the service's tools; every_tool_s_answer_follows_its_output_schema failed on it on both hosts at 15a1bc9ed. A circuit's QEC pilot goes on past its one in a hundred until 200 ms, its shots allowing (PILOT_FLOOR). Its shots are the call's own, decoded once and counted in its result, so it costs nothing; 20 ms of a light decode, read through one pause of a loaded host's scheduler, projected a decode that took 2.7 s alone at 7 s against a 6 s limit and refused it (a_time_is_measured_apart_from_setup_and_holds_every_threshold, the M3 Ultra at load 40). The workflows name the new tests, and the pulse tests are 28, native and wasip2. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/wai-quantum-pulse.yml | 7 +- .github/workflows/wai-quantum-qec.yml | 1 + .github/workflows/wai-quantum-service.yml | 8 +- wai/changes/2026-11-06-quantum-parse-depth.md | 11 +- wai/wai-quantum-cli/docs/service-mcp.json | 135 +++++++++++++++++- wai/wai-quantum-service/README.md | 2 +- .../schemas/service-tools.v1.json | 27 +++- wai/wai-quantum-service/src/discovery.rs | 2 +- wai/wai-quantum-service/src/http.rs | 2 +- wai/wai-quantum-service/src/jobs.rs | 17 ++- wai/wai-quantum-service/tests/depth.rs | 27 +++- wai/wai-quantum/src/engine/catalog.rs | 8 +- wai/wai-quantum/src/engine/pulse.rs | 75 ++++++++++ wai/wai-quantum/src/engine/pulse/tests.rs | 37 +++++ wai/wai-quantum/src/engine/qec.rs | 39 ++++- .../src/engine/registry/builtin.rs | 3 + wai/wai-quantum/src/engine/registry/mod.rs | 8 ++ wai/wai-quantum/src/engine/run.rs | 9 +- 18 files changed, 383 insertions(+), 35 deletions(-) diff --git a/.github/workflows/wai-quantum-pulse.yml b/.github/workflows/wai-quantum-pulse.yml index eca332c5f..89289cad3 100644 --- a/.github/workflows/wai-quantum-pulse.yml +++ b/.github/workflows/wai-quantum-pulse.yml @@ -74,7 +74,7 @@ jobs: run: | set -euo pipefail cargo test --locked --release --features full --lib -- qasm::pulse:: quantum_pulse:: engine::pulse:: 2>&1 | tee "$RUNNER_TEMP/pulse.log" - grep -qE '^test result: ok\. 27 passed; 0 failed' "$RUNNER_TEMP/pulse.log" || { echo "::error::the pulse tests did not all pass (27 expected)"; exit 1; } + grep -qE '^test result: ok\. 28 passed; 0 failed' "$RUNNER_TEMP/pulse.log" || { echo "::error::the pulse tests did not all pass (28 expected)"; exit 1; } for t in qasm::pulse::tests::a_program_reads_to_its_schedule_on_frames \ qasm::pulse::tests::barriers_align_frames_and_a_gate_starts_its_frames_together \ qasm::pulse::tests::phase_and_frequency_follow_the_frame_and_the_carrier_runs_on \ @@ -101,7 +101,8 @@ jobs: engine::pulse::tests::a_pulse_job_is_checked_by_running_it_again \ engine::pulse::tests::a_schedule_is_drawn_with_its_digests \ engine::pulse::tests::the_request_schema_holds_the_options_the_engine_reads \ - engine::pulse::tests::a_pulse_job_is_the_same_bits_on_every_platform; do + engine::pulse::tests::a_pulse_job_is_the_same_bits_on_every_platform \ + engine::pulse::tests::no_pulse_result_passes_the_bound_its_plan_gave; do grep -q "^test $t \.\.\. ok$" "$RUNNER_TEMP/pulse.log" || { echo "::error::$t did not run and pass"; exit 1; } done # Without warnings: a feature gate that leaves code unreachable or a @@ -207,4 +208,4 @@ jobs: run: | set -euo pipefail cargo test --locked --target wasm32-wasip2 --features full --lib -- qasm::pulse:: quantum_pulse:: engine::pulse:: 2>&1 | tee "$RUNNER_TEMP/wasip2.log" - grep -qE '^test result: ok\. 27 passed; 0 failed' "$RUNNER_TEMP/wasip2.log" || { echo "::error::the pulse tests did not all pass under wasip2 (27 expected)"; exit 1; } + grep -qE '^test result: ok\. 28 passed; 0 failed' "$RUNNER_TEMP/wasip2.log" || { echo "::error::the pulse tests did not all pass under wasip2 (28 expected)"; exit 1; } diff --git a/.github/workflows/wai-quantum-qec.yml b/.github/workflows/wai-quantum-qec.yml index de235d728..e6fa38419 100644 --- a/.github/workflows/wai-quantum-qec.yml +++ b/.github/workflows/wai-quantum-qec.yml @@ -138,6 +138,7 @@ jobs: engine::qec::tests::every_family_reads_and_certifies_its_distance \ engine::qec::tests::a_time_holds_the_shots_sampled_and_decoded \ engine::qec::tests::a_time_is_measured_apart_from_setup_and_holds_every_threshold \ + engine::qec::tests::a_circuit_s_pilot_runs_for_its_floor_within_its_shots \ engine::qec::tests::each_decoder_corrects_every_single_error \ engine::qec::tests::circuit_level_decoding_reads_every_model_form \ engine::qec::tests::a_threshold_is_measured_and_distance_helps_below_it \ diff --git a/.github/workflows/wai-quantum-service.yml b/.github/workflows/wai-quantum-service.yml index 1b2cefde9..5460c630f 100644 --- a/.github/workflows/wai-quantum-service.yml +++ b/.github/workflows/wai-quantum-service.yml @@ -85,18 +85,22 @@ jobs: # sweeps and their Merkle paths, a result's provenance, the package # counter's reader and probe (on fake trees; a container hides the real # one), and a real-valued input bound by the reader. - - name: The engine's sets, provenance and package energy + # The bound on a result the service admits a job by (a circuit's, a + # set's and a pulse run's): no result passes it. + - name: The engine's sets, provenance, result bounds and package energy shell: bash working-directory: wai/wai-quantum run: | set -euo pipefail - cargo test --locked --release --features full --lib -- engine::sets:: engine::energy:: merkle:: engine::run::tests::a_result_s_provenance qasm::tests::a_bound_parameter 2>&1 | tee "$RUNNER_TEMP/engine.log" + cargo test --locked --release --features full --lib -- engine::sets:: engine::energy:: merkle:: engine::run::tests::a_result_s_provenance engine::run::tests::no_result_passes_the_bound engine::pulse::tests::no_pulse_result_passes_the_bound qasm::tests::a_bound_parameter 2>&1 | tee "$RUNNER_TEMP/engine.log" grep -qE '^test result: ok\. [1-9][0-9]* passed; 0 failed' "$RUNNER_TEMP/engine.log" || { echo "::error::the engine's tests did not pass"; exit 1; } for t in engine::sets::tests::one_receipt_binds_every_item_and_any_change_breaks_it \ engine::sets::tests::a_sweep_is_its_grid_each_point_the_program_written_out \ engine::sets::tests::a_failed_item_is_bound_by_its_error \ merkle::tests::every_leaf_s_path_leads_to_the_root_and_nothing_else_does \ engine::run::tests::a_result_s_provenance_names_the_passes_its_certificate_ran \ + engine::run::tests::no_result_passes_the_bound_its_plan_gave \ + engine::pulse::tests::no_pulse_result_passes_the_bound_its_plan_gave \ engine::energy::tests::the_probe_finds_a_live_counter_s_cadence_and_refuses_a_stalled_one \ engine::energy::tests::only_top_level_package_zones_are_read_and_a_frozen_one_is_not_live \ engine::energy::tests::a_package_figure_names_its_scope_and_declares_its_edges \ diff --git a/wai/changes/2026-11-06-quantum-parse-depth.md b/wai/changes/2026-11-06-quantum-parse-depth.md index 20d6f14be..356326a71 100644 --- a/wai/changes/2026-11-06-quantum-parse-depth.md +++ b/wai/changes/2026-11-06-quantum-parse-depth.md @@ -65,8 +65,9 @@ `verify_quantum_receipt` on a set are counted and read as a check is, as its sweep is written out again, and a seal that does not hold is not. A decode is held to 1 024 fired detectors, as matching grows faster than their square, and a memory experiment to 21 rounds. An error-correction decode or threshold measures its shots as they - run (`engine::qec::within_time`): each point is set up and sampled, and a pilot of its own shots decoded, before the - rest of any, and the call is refused at once when what is measured says it would take more than + run (`engine::qec::within_time`): each point is set up and sampled, and a pilot of its own shots decoded (one in a + hundred, and on for at least 200 ms where its shots allow: 20 ms of a light decode read three times slow on a + loaded host and refused a call that fit), before the rest of any, and the call is refused at once when what is measured says it would take more than `WAI_QSVC_QEC_SECS` (25), stopped only if it runs a fifth past that; setup is not multiplied by the shots, so a call that fits is run, and code-capacity thresholds are measured on a pilot of trials of each point too. One at the published bounds ran for many minutes and held its turn to the reading's cap. @@ -134,12 +135,12 @@ its result's bound and the new one at `WAI_QSVC_RESULT_MIB` (64), would pass `WAI_QSVC_CLIENT_UNREAD_MIB` (256), and every next job while all unread hold `WAI_QSVC_UNREAD_MIB` (8192). A job's or a set's result is bounded before it runs (`catalog::plan_bounded`, `sets::plan_bounded`, new: each item's counts at most its shots, `2^bits` and - 65,536 outcomes, every other part by what it may hold or at its size as read), and a request whose bound passes + 65,536 outcomes, every other part by what it may hold or at its size as read; a pulse run's from its schedule, + `pulse::plan_bounded`, through `Method::plan_bounded`), and a request whose bound passes `WAI_QSVC_RESULT_MIB` is refused at once (`422`, `J031`), its bound and the cap in the reason; a dry run states the bound (`result_bytes`), and `GET /v1`, the dry run's bounds and the OpenAPI document the cap. At 64 MiB, 100 points of 65,536 outcomes (about 150 MB) are refused, and 100 points of 16 bits at 30,000 shots are taken. The - check as a result ends stays as a backstop, for a job kept across a restart that lowered the cap and for a pulse - run, which is not bounded before it runs. + check as a result ends stays as a backstop, for a job kept across a restart that lowered the cap. MCP answers echo a JSON-RPC id exactly. Readers and workers start from the service's own binary as it was started (Linux: its running image; elsewhere a copy kept beside the signing key), so the binary can be replaced before the restart. A restart reads no program: a job keeps the plan it was admitted with and a diff --git a/wai/wai-quantum-cli/docs/service-mcp.json b/wai/wai-quantum-cli/docs/service-mcp.json index 98e4a5cc6..a9fec9aeb 100644 --- a/wai/wai-quantum-cli/docs/service-mcp.json +++ b/wai/wai-quantum-cli/docs/service-mcp.json @@ -8636,9 +8636,34 @@ }, "bounds": { "type": "object", - "description": "The bounds that apply here: memory, time, queue and items of a set.", + "description": "The bounds that apply here: memory, time, queue, items of a set, and result_bytes, the most a job's or a set's result may take as served.", "additionalProperties": true }, + "result_bytes": { + "description": "The most its result takes as served, bytes, bounded before it runs: past bounds.result_bytes the job is refused (422, J031). A pulse run is bounded from its schedule. For a method that gives no bound before it runs (no built-in is such): {\"bytes\": null, \"why\"}, and it is counted at bounds.result_bytes.", + "oneOf": [ + { + "type": "integer", + "minimum": 0 + }, + { + "type": "object", + "required": [ + "bytes", + "why" + ], + "properties": { + "bytes": { + "type": "null" + }, + "why": { + "type": "string" + } + }, + "additionalProperties": false + } + ] + }, "client": { "type": "object", "description": "Where the client stands: its active jobs, the memory they hold, the submissions left in its window and when the window resets.", @@ -18894,9 +18919,34 @@ }, "bounds": { "type": "object", - "description": "The bounds that apply here: memory, time, queue and items of a set.", + "description": "The bounds that apply here: memory, time, queue, items of a set, and result_bytes, the most a job's or a set's result may take as served.", "additionalProperties": true }, + "result_bytes": { + "description": "The most its result takes as served, bytes, bounded before it runs: past bounds.result_bytes the job is refused (422, J031). A pulse run is bounded from its schedule. For a method that gives no bound before it runs (no built-in is such): {\"bytes\": null, \"why\"}, and it is counted at bounds.result_bytes.", + "oneOf": [ + { + "type": "integer", + "minimum": 0 + }, + { + "type": "object", + "required": [ + "bytes", + "why" + ], + "properties": { + "bytes": { + "type": "null" + }, + "why": { + "type": "string" + } + }, + "additionalProperties": false + } + ] + }, "client": { "type": "object", "description": "Where the client stands: its active jobs, the memory they hold, the submissions left in its window and when the window resets.", @@ -21221,9 +21271,34 @@ }, "bounds": { "type": "object", - "description": "The bounds that apply here: memory, time, queue and items of a set.", + "description": "The bounds that apply here: memory, time, queue, items of a set, and result_bytes, the most a job's or a set's result may take as served.", "additionalProperties": true }, + "result_bytes": { + "description": "The most its result takes as served, bytes, bounded before it runs: past bounds.result_bytes the job is refused (422, J031). A pulse run is bounded from its schedule. For a method that gives no bound before it runs (no built-in is such): {\"bytes\": null, \"why\"}, and it is counted at bounds.result_bytes.", + "oneOf": [ + { + "type": "integer", + "minimum": 0 + }, + { + "type": "object", + "required": [ + "bytes", + "why" + ], + "properties": { + "bytes": { + "type": "null" + }, + "why": { + "type": "string" + } + }, + "additionalProperties": false + } + ] + }, "client": { "type": "object", "description": "Where the client stands: its active jobs, the memory they hold, the submissions left in its window and when the window resets.", @@ -30172,9 +30247,34 @@ }, "bounds": { "type": "object", - "description": "The bounds that apply here: memory, time, queue and items of a set.", + "description": "The bounds that apply here: memory, time, queue, items of a set, and result_bytes, the most a job's or a set's result may take as served.", "additionalProperties": true }, + "result_bytes": { + "description": "The most its result takes as served, bytes, bounded before it runs: past bounds.result_bytes the job is refused (422, J031). A pulse run is bounded from its schedule. For a method that gives no bound before it runs (no built-in is such): {\"bytes\": null, \"why\"}, and it is counted at bounds.result_bytes.", + "oneOf": [ + { + "type": "integer", + "minimum": 0 + }, + { + "type": "object", + "required": [ + "bytes", + "why" + ], + "properties": { + "bytes": { + "type": "null" + }, + "why": { + "type": "string" + } + }, + "additionalProperties": false + } + ] + }, "client": { "type": "object", "description": "Where the client stands: its active jobs, the memory they hold, the submissions left in its window and when the window resets.", @@ -52701,9 +52801,34 @@ }, "bounds": { "type": "object", - "description": "The bounds that apply here: memory, time, queue and items of a set.", + "description": "The bounds that apply here: memory, time, queue, items of a set, and result_bytes, the most a job's or a set's result may take as served.", "additionalProperties": true }, + "result_bytes": { + "description": "The most its result takes as served, bytes, bounded before it runs: past bounds.result_bytes the job is refused (422, J031). A pulse run is bounded from its schedule. For a method that gives no bound before it runs (no built-in is such): {\"bytes\": null, \"why\"}, and it is counted at bounds.result_bytes.", + "oneOf": [ + { + "type": "integer", + "minimum": 0 + }, + { + "type": "object", + "required": [ + "bytes", + "why" + ], + "properties": { + "bytes": { + "type": "null" + }, + "why": { + "type": "string" + } + }, + "additionalProperties": false + } + ] + }, "client": { "type": "object", "description": "Where the client stands: its active jobs, the memory they hold, the submissions left in its window and when the window resets.", diff --git a/wai/wai-quantum-service/README.md b/wai/wai-quantum-service/README.md index e9e5791b0..8d24dfba2 100644 --- a/wai/wai-quantum-service/README.md +++ b/wai/wai-quantum-service/README.md @@ -49,7 +49,7 @@ One request can run many programs, as one job: A set holds at most 100 items. They run one after another in one worker, inside one job's time limit, holding the memory of the largest item. Each item is a plain job: its request is complete in itself, and it runs, plans and verifies as it would alone. A set with an item that does not plan is refused, naming the item. -A job's result, or a set's with every item's request and result in it, may take at most `WAI_QSVC_RESULT_MIB` (64 MiB) as served. That is checked before anything runs, from a bound of the result read from the request: each item's counts list at most its shots, `2^bits` and 65,536 outcomes, each its bit string, quotes, a colon, a comma and its count's digits; each other part is counted by what it may hold or at its size as read (a verdict asking for one counts 512 bytes for each gate its critique may remark on). A request whose bound passes the cap is refused at once, `422` and `J031`, with the bound and the cap in the reason, and a dry run gives the bound as `result_bytes`. The cap is less than a set at its most can give: at about 22 bytes a 16-bit outcome, 100 points of 65,536 outcomes each are about 150 MB, and are refused; 100 points of 16 measured bits are taken up to about 30,000 shots a point. Send fewer points or shots, split the set, or run it on your own machine (`wai-quantum run`), where no cap applies. A method that reads its own program (a pulse run) is not bounded before it runs: it is counted at the cap, and a result past the cap fails its job, saying so, as one kept across a restart that lowered the cap does. +A job's result, or a set's with every item's request and result in it, may take at most `WAI_QSVC_RESULT_MIB` (64 MiB) as served. That is checked before anything runs, from a bound of the result read from the request: each item's counts list at most its shots, `2^bits` and 65,536 outcomes, each its bit string, quotes, a colon, a comma and its count's digits; each other part is counted by what it may hold or at its size as read (a verdict asking for one counts 512 bytes for each gate its critique may remark on). A request whose bound passes the cap is refused at once, `422` and `J031`, with the bound and the cap in the reason, and a dry run gives the bound as `result_bytes`. The cap is less than a set at its most can give: at about 22 bytes a 16-bit outcome, 100 points of 65,536 outcomes each are about 150 MB, and are refused; 100 points of 16 measured bits are taken up to about 30,000 shots a point. Send fewer points or shots, split the set, or run it on your own machine (`wai-quantum run`), where no cap applies. A pulse run is bounded the same way, from its schedule as read: its populations and its distribution by the qubits it simulates and the captures it reads, each keyed as wide as the program's bits, its counts by its shots, and each capture with its frame's and bit's names. The check as a result ends stays only as a backstop, for a job kept across a restart that lowered the cap. The result is a set (`"format": "wai.quantum.set/1"`). Each item has its own result and its own receipt. Each item's leaf is the digest its receipt signs (a failed item's: the digest of its request and its error). The head holds the Merkle root of the leaves, under the JWP Merkle domains, with the count, and the set's receipt seals the set's request and its head. Each item carries its `path` to the root, so one item can be shown sealed by its set without the others. `POST /v1/verify` checks the whole set: its receipt, every item as the request gives it, each item's receipt, the root, and each path. diff --git a/wai/wai-quantum-service/schemas/service-tools.v1.json b/wai/wai-quantum-service/schemas/service-tools.v1.json index e06c0aa2e..1b198eb5e 100644 --- a/wai/wai-quantum-service/schemas/service-tools.v1.json +++ b/wai/wai-quantum-service/schemas/service-tools.v1.json @@ -390,9 +390,34 @@ }, "bounds": { "type": "object", - "description": "The bounds that apply here: memory, time, queue and items of a set.", + "description": "The bounds that apply here: memory, time, queue, items of a set, and result_bytes, the most a job's or a set's result may take as served.", "additionalProperties": true }, + "result_bytes": { + "description": "The most its result takes as served, bytes, bounded before it runs: past bounds.result_bytes the job is refused (422, J031). A pulse run is bounded from its schedule. For a method that gives no bound before it runs (no built-in is such): {\"bytes\": null, \"why\"}, and it is counted at bounds.result_bytes.", + "oneOf": [ + { + "type": "integer", + "minimum": 0 + }, + { + "type": "object", + "required": [ + "bytes", + "why" + ], + "properties": { + "bytes": { + "type": "null" + }, + "why": { + "type": "string" + } + }, + "additionalProperties": false + } + ] + }, "client": { "type": "object", "description": "Where the client stands: its active jobs, the memory they hold, the submissions left in its window and when the window resets.", diff --git a/wai/wai-quantum-service/src/discovery.rs b/wai/wai-quantum-service/src/discovery.rs index 2ac0e7696..f260b35bc 100644 --- a/wai/wai-quantum-service/src/discovery.rs +++ b/wai/wai-quantum-service/src/discovery.rs @@ -162,7 +162,7 @@ pub fn openapi(server: &str, result_bytes: u64) -> Value { "energy": { "type": "object", "description": "{\"joules_micro_estimate\", \"energy_provenance\": \"Estimator\", \"basis\"} from the power of past metered jobs of the method here, or {\"unestimated\": why}. Never a reading." }, "within_time_limit": { "type": ["boolean", "null"] }, "bounds": { "type": "object", "description": "The limits that apply: job_memory_bytes, job_seconds, active_jobs_per_client, memory_per_client_bytes, submissions_per_window, window_seconds, queue, set_items, result_bytes (the most a job's or a set's result may take as served, by its bound before it runs), intents (the most a job states), read_steps (the steps all of a request's programs may take to read together), body_bytes." }, - "result_bytes": { "type": ["integer", "object"], "description": "The most its result takes as served, bytes, bounded before it runs; past bounds.result_bytes the submission is refused (422, J031). {\"bytes\": null, \"why\"} for a method that reads its own program (a pulse run), counted at bounds.result_bytes." }, + "result_bytes": { "type": ["integer", "object"], "description": "The most its result takes as served, bytes, bounded before it runs; past bounds.result_bytes the submission is refused (422, J031). {\"bytes\": null, \"why\"} for a method that gives no bound before it runs (no built-in), counted at bounds.result_bytes. A pulse run is bounded from its schedule." }, "client": { "type": "object", "description": "Where the client stands: active_jobs, memory_held_bytes, submissions_left, window_resets_in_seconds." }, "items": { "type": "array", "description": "A set's items, each with its index, point, method, memory, work, time and energy." } } diff --git a/wai/wai-quantum-service/src/http.rs b/wai/wai-quantum-service/src/http.rs index 94caeb99c..bf75b66f8 100644 --- a/wai/wai-quantum-service/src/http.rs +++ b/wai/wai-quantum-service/src/http.rs @@ -612,7 +612,7 @@ fn dry_run_now(service: &Service, who: &str, (_, planned): Admitted) -> Result json!(b), - None => json!({ "bytes": null, "why": "a method that reads its own program (a pulse run) is not bounded before it runs: it is counted at the most a job's result may take, and held to it as it ends" }), + None => json!({ "bytes": null, "why": "its method gives no bound of its result before it runs (no built-in method is such): it is counted at the most a job's result may take, and held to it as it ends" }), }; if let Some(t) = &plan.target { out["target"] = t.clone(); diff --git a/wai/wai-quantum-service/src/jobs.rs b/wai/wai-quantum-service/src/jobs.rs index e1a86513c..b62d0f73f 100644 --- a/wai/wai-quantum-service/src/jobs.rs +++ b/wai/wai-quantum-service/src/jobs.rs @@ -112,13 +112,16 @@ pub struct Config { /// its client could read it. pub kept_min: Duration, /// What one client's results not yet read may keep, MiB, its jobs still - /// running counted at [`Config::result_mib`] each: past it its next job + /// running counted at their results' bounds ([`Job::bound`]) and its next + /// at [`Config::result_mib`]: past it its next job /// is refused until it reads them. Everyone's, [`Config::unread_mib`]: /// past it every next job is refused. pub client_unread_mib: u64, pub unread_mib: u64, - /// The largest result a job may give, MiB, as served: past it the job - /// fails, saying so. + /// The largest result a job may give, MiB, as served: a job whose + /// result's bound passes it is refused before it runs + /// ([`Service::would_refuse`]); one whose result passes it as it ends (a + /// backstop) fails, saying so. pub result_mib: u64, /// Receipts one client may have checked in its window /// (`POST /v1/verify`, `verify_quantum_receipt`), before any is read. @@ -332,7 +335,7 @@ pub struct Job { /// The most its result takes as served, bytes, known as it was admitted /// (`http::plan_job`): what its client's unread results are counted to /// hold while it runs. `u64::MAX` where it is not known (a job kept - /// across a restart, a pulse run), counted at [`Config::result_mib`]. + /// across a restart), counted at [`Config::result_mib`]. bound: u64, } @@ -1586,8 +1589,8 @@ impl Service { let text = served_text(&result); // A backstop: a job is admitted only on a bound of its result // within the cap ([`Service::would_refuse`]), so this is a job - // kept across a restart that lowered the cap, or one whose - // method left its result unbounded (a pulse run). + // kept across a restart that lowered the cap (every built-in + // method's result is bounded before it runs, a pulse run's too). if text.len() as u64 > self.config.result_mib << 20 { return Err(tag(refusal::TOO_LARGE_HERE, format!("its result is {} MiB as served, more than the {} MiB one job's result here may take; send fewer points or fewer shots, or run it on your own machine (`wai-quantum run`)", text.len() >> 20, self.config.result_mib))); } @@ -2193,7 +2196,7 @@ mod tests { } assert!(s.refused_before_reading("a").is_none(), "seven small results and a next job's room fit in 4 MiB"); assert!(s.would_refuse("a", &p, Some(4096)).is_none()); - // Not known (a pulse run), a job running is counted at the cap. + // Not known (a job kept across a restart), a job running is counted at the cap. let t = Service::new(Config { result_mib: 1, client_unread_mib: 4, per_client_active: 8, max_queue: 16, ..config() }); for k in 0..3 { t.keep_work("a", json!({ "qasm": "OPENQASM 3.0; qubit q; h q;", "shots": k + 1 }), p.clone(), Work::One, None, None, None, false).unwrap(); diff --git a/wai/wai-quantum-service/tests/depth.rs b/wai/wai-quantum-service/tests/depth.rs index 7384d6b28..9b7bc34ca 100644 --- a/wai/wai-quantum-service/tests/depth.rs +++ b/wai/wai-quantum-service/tests/depth.rs @@ -1497,7 +1497,8 @@ fn an_mcp_id_is_echoed_exactly() { /// passes the cap is refused at once (`422`, `J031`), naming its bound and /// the cap, on every route that takes a job, and its dry run says so; the /// largest sweep under the cap is taken and ends done, its result within its -/// bound. At the defaults, the published set at its most, 100 items of +/// bound; a pulse run past the cap is refused at once as well, by its +/// schedule. At the defaults, the published set at its most, 100 items of /// 65,536 outcomes each (about 150 MB as served), is refused, and 100 /// points of 30,000 shots of 16 bits each are taken. #[test] @@ -1554,6 +1555,30 @@ fn a_result_past_what_a_job_may_give_is_refused_before_it_runs() { let served = wai_quantum::engine::receipt::canonical(&job["result"]).len() as u64; eprintln!("{points} points of 2,000 shots: bound {at} bytes, served {served}"); assert!(served <= at && at <= cap, "{points} points: served {served} bytes, bound {at}"); + // A pulse run is bounded from its schedule as well: 100,000 bits + // declared (in registers of 64, as the pulse reader takes them) and one + // captured, each bit named and every outcome's key as wide (about 1.4 MB + // as served), are refused at once, where the run had been taken and + // failed only as it ended; 100 bits are taken. + let pulse = |n: usize| json!({ "method": "pulse", "shots": 100, "qasm": format!("OPENQASM 3.0;\ndefcalgrammar \"openpulse\";\ncal {{\n extern port d0;\n extern port a0;\n frame q0 = newframe(d0, 5.0e9, 0);\n frame c0 = newframe(a0, 7.1e9, 0);\n}}\ndefcal x $0 {{ play(q0, gaussian(0.4, 64dt, 16dt)); }}\ndefcal measure $0 -> bit {{ return capture_v2(c0, 4dt); }}\n{}x $0;\nb{}[{}] = measure $0;\n", (0..n.div_ceil(64)).map(|r| format!("bit[{}] b{r};\n", 64.min(n - 64 * r))).collect::(), (n - 1) / 64, (n - 1) % 64) }); + let began = Instant::now(); + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&pulse(100_000))); + if code == 202 { + let id = b["id"].as_str().unwrap_or_default().to_string(); + let job = loop { + let (_, b) = json_of(port, "GET", &format!("/v1/jobs/{id}?wait=25"), None); + if b["status"] != "queued" && b["status"] != "running" { + break b; + } + assert!(began.elapsed() < Duration::from_secs(300)); + }; + panic!("a pulse run of 100,000 bits was taken, and then after {:?}: {} {}", began.elapsed(), job["status"], job["error"]); + } + let why = b["error"].as_str().unwrap_or_default(); + assert!(code == 422 && why.contains("J031") && why.contains("could take up to") && why.contains("the 1 MiB"), "a pulse run past the cap: {code} {b}"); + assert!(began.elapsed() < Duration::from_secs(10), "answered after {:?}", began.elapsed()); + let small = dry(port, &pulse(100)); + assert!(small["accepted"] == true && small["result_bytes"].as_u64().is_some_and(|b| b < cap), "{small}"); alive(port, &mut svc, "results held to their cap"); svc.kill().unwrap(); svc.wait().unwrap(); diff --git a/wai/wai-quantum/src/engine/catalog.rs b/wai/wai-quantum/src/engine/catalog.rs index 98e40d93f..69d051a5b 100644 --- a/wai/wai-quantum/src/engine/catalog.rs +++ b/wai/wai-quantum/src/engine/catalog.rs @@ -1251,8 +1251,10 @@ pub fn plan(v: &Value, job_bytes: u64) -> Result { /// [`plan`], with the most its result may take as JSON text, in bytes, /// beside what a host adds to it (a receipt, an energy figure), known before -/// it runs ([`result_bound`]); `None` for a method that reads its program -/// itself (a pulse run), whose result this reads nothing of. +/// it runs ([`result_bound`]); a method that reads its program itself bounds +/// its own ([`Method::plan_bounded`]: a pulse run's is +/// [`super::pulse::plan_bounded`]), and `None` is one that gives no bound, +/// which no built-in is. pub fn plan_bounded(v: &Value, job_bytes: u64) -> Result<(Plan, Option), String> { plan_bounded_in(Registry::builtins(), v, job_bytes) } @@ -1340,7 +1342,7 @@ pub(crate) fn plan_in(reg: &Registry, v: &Value, job_bytes: u64) -> Result Result<(Plan, Option), String> { if let Some(m) = reads_own(reg, v) { - return Ok((m.plan(&Job::new(reg, v, job_bytes, None, &[]))?, None)); + return m.plan_bounded(&Job::new(reg, v, job_bytes, None, &[])); } // A job that names a device target is compiled, placed and simulated as // the device would run it (engine::targets::job). diff --git a/wai/wai-quantum/src/engine/pulse.rs b/wai/wai-quantum/src/engine/pulse.rs index 18c62938f..00dd6e625 100644 --- a/wai/wai-quantum/src/engine/pulse.rs +++ b/wai/wai-quantum/src/engine/pulse.rs @@ -436,6 +436,81 @@ pub fn lint(request: &Value) -> Value { /// memory and the work, and why. pub fn plan(v: &Value, job_bytes: u64) -> Result { let (_, p) = prepare(v)?; + plan_of(&p, job_bytes) +} + +/// [`plan`], with the most the run's result ([`run`]) takes as JSON text, +/// bytes, known before it runs, from the same reading: as +/// [`super::catalog::plan_bounded`] gives a circuit's. +pub fn plan_bounded(v: &Value, job_bytes: u64) -> Result<(Plan, u64), String> { + let (_, p) = prepare(v)?; + let plan = plan_of(&p, job_bytes)?; + let bound = result_bound(&p, &plan); + Ok((plan, bound)) +} + +/// What a pulse result holds beside the parts [`result_bound`] counts one by +/// one: its accuracy, its integrator, its notes, its digests, its schema, +/// its verdict's own fields and its provenance, the plan's reason apart. +const RESULT_REST: u64 = 16 << 10; +/// A number as JSON text at its longest (`-2.2250738585072014e-308` is 24 +/// characters), with room. +const NUMBER: u64 = 32; +/// One capture as the result lists it, beside its frame's and its bit's +/// names: its qubit, its start in samples and in ns, and its two +/// probabilities. +const CAPTURE_BYTES: u64 = 192; + +/// The most [`run`]'s result of the job read as `p` and planned as `plan` +/// takes as JSON text, bytes, each part that grows with the program bounded +/// by what it may list: the populations, `2^n` outcomes of the `n` simulated +/// qubits; the distribution, the populations' where nothing is captured, +/// or the joint reading's, `2^d` outcomes of the `d` captures where each is +/// on its own qubit (else none), each keyed as wide as the program's bits +/// or the captures; the counts sampled from it +/// ([`super::run::counts_bound_among`]); the outcome bits' labels; each +/// capture with its frame's and bit's names; the leakage; the diagnostics, +/// in the result and in its verdict; the device as read and the target; and +/// the rest at [`RESULT_REST`], a sixteenth more for the forms RFC 8785 +/// writes a number in. +fn result_bound(p: &Prepared, plan: &Plan) -> u64 { + let size = |x: &Value| serde_json::to_string(x).map_or(0, |s| s.len() as u64); + // A name of the program's own as JSON text: RFC 8785 escapes a string's + // characters as serde does (`receipt::write_str`). + let text = |t: &str| serde_json::to_string(t).map_or(6 * t.len() as u64 + 2, |s| s.len() as u64); + let pow2 = |e: u64| if e >= 63 { u64::MAX } else { 1u64 << e }; + let s = &p.reading.schedule; + let n = p.plan.qubits.len() as u64; + let caps: Vec<&ir::Instruction> = s.instructions.iter().filter(|i| matches!(i.op, Op::Capture { .. })).collect(); + let mut read: Vec = caps.iter().map(|i| s.frame_qubit(i.frame)).collect(); + read.sort_unstable(); + read.dedup(); + let bits = s.bits.len() as u64; + let (outcomes, width) = if caps.is_empty() { + (pow2(n), n) + } else if read.len() == caps.len() { + (pow2(read.len() as u64), bits.max(read.len() as u64)) + } else { + (0, 0) + }; + let entry = |w: u64| w + 4 + NUMBER; + let mut b = RESULT_REST + 2 * size(&json!(plan.why)); + b = b.saturating_add(pow2(n).saturating_mul(entry(n))); + b = b.saturating_add(outcomes.saturating_mul(entry(width))); + b = b.saturating_add(super::run::counts_bound_among(u32::try_from(width).unwrap_or(u32::MAX), p.shots, outcomes, None)); + b = b.saturating_add(16 * n + s.bits.iter().map(|x| text(x) + 1).sum::() + 16 * caps.len() as u64); + b = b.saturating_add(caps.iter().map(|i| CAPTURE_BYTES + text(&s.frames[i.frame].name) + match &i.op { + Op::Capture { bit: Some(k), .. } => s.bits.get(*k).map_or(0, |x| text(x)), + _ => 0, + }).sum::()); + b = b.saturating_add(n * (NUMBER + 32)); + b = b.saturating_add(2 * p.reading.diagnostics.iter().map(|d| size(&d.to_json()) + 1).sum::()); + b = b.saturating_add(size(&p.device_json) + p.target.as_ref().map_or(0, size)); + b.saturating_add(b / 16) +} + +/// [`plan`] of the job read as `p`. +fn plan_of(p: &Prepared, job_bytes: u64) -> Result { let s = &p.reading.schedule; let held: usize = s.waveforms.iter().map(|w| w.samples.len()).sum(); let memory = (32u64 << 20) + 16 * held as u64 + 96 * s.instructions.len() as u64 + 64 * (p.plan.dim * p.plan.dim) as u64 + 64 * p.plan.samples; diff --git a/wai/wai-quantum/src/engine/pulse/tests.rs b/wai/wai-quantum/src/engine/pulse/tests.rs index 5e5ac1d38..41fd57a24 100644 --- a/wai/wai-quantum/src/engine/pulse/tests.rs +++ b/wai/wai-quantum/src/engine/pulse/tests.rs @@ -197,3 +197,40 @@ fn a_pulse_job_is_the_same_bits_on_every_platform() { let r = crate::engine::run::run(&req, &p).unwrap(); assert_eq!(r["pulse"]["density_hash"].as_str(), Some(XM_DENSITY), "the bits differ from the pinned ones: {}", r["pulse"]["density_hash"]); } + +/// No pulse result is larger than the bound its plan gave before it ran +/// (`catalog::plan_bounded`, through the pulse method's own), as RFC 8785 +/// writes it with its provenance: one capture; 3,000 bits declared and one +/// captured (every key of the distribution and the counts as wide as the +/// program's bits, and each bit named); nothing captured (the populations as +/// the distribution); and two qubits captured at once. A qubit is captured +/// once at most (`J040`), so what grows is the bits. Each is bounded, and +/// within a few times what it gives. +#[test] +#[cfg(feature = "quantum_receipt")] +fn no_pulse_result_passes_the_bound_its_plan_gave() { + let cal = "OPENQASM 3.0;\ndefcalgrammar \"openpulse\";\ncal {\n extern port d0;\n extern port m0;\n extern port a0;\n extern port d1;\n extern port a1;\n frame q0 = newframe(d0, 5.0e9, 0);\n frame r0 = newframe(m0, 7.1e9, 0);\n frame c0 = newframe(a0, 7.1e9, 0);\n frame q1 = newframe(d1, 5.2e9, 0);\n frame c1 = newframe(a1, 7.2e9, 0);\n}\ndefcal x $0 { play(q0, gaussian(0.4, 64dt, 16dt)); }\ndefcal x $1 { play(q1, gaussian(0.4, 64dt, 16dt)); }\ndefcal measure $0 -> bit { return capture_v2(c0, 4dt); }\ndefcal measure $1 -> bit { return capture_v2(c1, 4dt); }\n"; + // Registers of 64 bits at most (Q460), as many as `n` bits take, and the + // name of bit `i`. + let regs = |n: usize| (0..n.div_ceil(64)).map(|r| format!("bit[{}] b{r};\n", 64.min(n - 64 * r))).collect::(); + let bit = |i: usize| format!("b{}[{}]", i / 64, i % 64); + let jobs = [ + ("one capture", XM.to_string(), 500), + ("one bit of 3,000 captured", format!("{cal}{}x $0;\n{} = measure $0;\n", regs(3000), bit(2999)), 2000), + ("nothing captured", format!("{cal}x $0;\n"), 500), + ("two qubits captured", format!("{cal}bit[2] c;\nx $0;\nx $1;\nc[0] = measure $0;\nc[1] = measure $1;\n"), 4000), + ]; + for (name, qasm, shots) in jobs { + let job = json!({ "qasm": qasm, "method": "pulse", "shots": shots, "seed": 3 }); + let (p, bound) = match catalog::plan_bounded(&job, 1 << 30) { + Ok((p, Some(b))) => (p, b), + Ok((_, None)) => panic!("{name}: no bound"), + Err(e) => panic!("{name}: {e}"), + }; + let result = crate::engine::run::run_with_provenance(&job, &p, "a host 1.0").unwrap_or_else(|e| panic!("{name}: {e}")); + let served = crate::engine::receipt::canonical(&result).len() as u64; + eprintln!("{name}: {served} bytes, bound {bound}"); + assert!(served <= bound, "{name}: its result is {served} bytes, past its bound of {bound}"); + assert!(bound <= 4 * served + (32 << 10), "{name}: {served} bytes, bound {bound}"); + } +} diff --git a/wai/wai-quantum/src/engine/qec.rs b/wai/wai-quantum/src/engine/qec.rs index 7556b04d4..a644cf37d 100644 --- a/wai/wai-quantum/src/engine/qec.rs +++ b/wai/wai-quantum/src/engine/qec.rs @@ -58,7 +58,8 @@ thread_local! { /// that. What a shot costs depends on the noise, the distance and the decoder /// as much as on the counts, so each point is measured before its shots are /// run: its setup and sampling as they are, and a pilot of its own shots -/// (at most one in a hundred, and at least 32) for its rate. The rest is +/// (one in a hundred, at least 32, and on for [`PILOT_FLOOR`] where its shots +/// allow) for its rate. The rest is /// projected from those, so setup is never multiplied by the shots, and a /// call whose measured rate says it fits is run. pub fn within_time(limit: std::time::Duration, f: impl FnOnce() -> T) -> T { @@ -84,6 +85,21 @@ fn piloting(done: usize, most: usize, began: std::time::Instant) -> bool { done < most && (done < 8 || began.elapsed() < std::time::Duration::from_millis(500)) } +/// The least time a circuit's pilot runs, its shots allowing. Its shots are +/// the call's own, decoded once and counted in its result, so measuring for +/// longer costs nothing; one in a hundred of a light decode is 20 ms, and +/// one pause of a loaded host's scheduler in so short a time read the rate +/// three times slow and refused a call that fit (a decode that took 2.7 s +/// alone, projected at 7 s against 6, at load 40 on the M3 Ultra). +const PILOT_FLOOR: std::time::Duration = std::time::Duration::from_millis(200); + +/// [`piloting`] for a pilot of a call's own shots, `total` in all: past its +/// one in a hundred it goes on until [`PILOT_FLOOR`]. With no time set +/// (`most` 0) there is no pilot. +fn piloting_own(done: usize, most: usize, total: usize, began: std::time::Instant) -> bool { + most > 0 && done < total && (piloting(done, most, began) || (done >= most && began.elapsed() < PILOT_FLOOR)) +} + /// The refusal of a call whose shots would take `secs`. fn too_slow(secs: f64, limit: std::time::Duration, op: &str) -> String { tag(refusal::TOO_LARGE_HERE, format!("its shots would take about {secs:.0} s to sample and decode here, more than the {} s a call here takes; run `wai-quantum qec {op}` on your machine", limit.as_secs())) @@ -429,7 +445,7 @@ pub fn decode(request: &Value) -> Result { let most = if timed() { pilot_of(shots) as usize } else { 0 }; let began = std::time::Instant::now(); let mut i = 0; - while piloting(i, most, began) { + while piloting_own(i, most, smp.shots, began) { if dec.run(&smp.fired(i), &mut ws)?.0 != smp.flips(i) { failures += 1; } @@ -633,7 +649,7 @@ pub fn threshold(request: &Value) -> Result { let smp = frame::sample(&c, shots as usize, seed.wrapping_add(u64::from(d) * 1000 + (p * 1e6) as u64)); let mut pt = Point { d, rounds, p, smp, dec, ws: None, failures: 0, piloted: 0 }; let began = std::time::Instant::now(); - while piloting(pt.piloted, most, began) { + while piloting_own(pt.piloted, most, pt.smp.shots, began) { let i = pt.piloted; if pt.dec.run(&pt.smp.fired(i), &mut pt.ws)?.0 != pt.smp.flips(i) { pt.failures += 1; @@ -842,6 +858,23 @@ pub fn call(op: &str, request: &Value) -> Result { mod tests { use super::*; + /// A circuit's pilot goes on past its one in a hundred until + /// [`PILOT_FLOOR`], so its rate is not read from 20 ms of a light decode; + /// it never passes its shots, nor half a second once one in a hundred is + /// not done, and there is none with no time set. + #[test] + fn a_circuit_s_pilot_runs_for_its_floor_within_its_shots() { + let now = std::time::Instant::now(); + let ago = |ms: u64| now.checked_sub(std::time::Duration::from_millis(ms)).expect("an instant that long ago"); + assert!(piloting_own(200, 200, 20_000, now), "past one in a hundred, within the floor"); + assert!(!piloting_own(200, 200, 20_000, ago(PILOT_FLOOR.as_millis() as u64 + 1)), "past the floor"); + assert!(!piloting_own(20_000, 200, 20_000, now), "every shot done"); + assert!(!piloting_own(0, 0, 20_000, now), "no time set"); + assert!(piloting_own(100, 200, 20_000, ago(300)), "short of one in a hundred, within half a second"); + assert!(!piloting_own(100, 200, 20_000, ago(600)), "past half a second"); + assert!(piloting_own(3, 200, 20_000, ago(600)), "fewer than 8 done"); + } + /// A call whose setup and sampling are most of its time, and its shots /// cheap, is run within a time it fits: what is measured before the first /// shot is not multiplied by the shots. A code-capacity threshold, whose diff --git a/wai/wai-quantum/src/engine/registry/builtin.rs b/wai/wai-quantum/src/engine/registry/builtin.rs index e960c3a84..066a6aa68 100644 --- a/wai/wai-quantum/src/engine/registry/builtin.rs +++ b/wai/wai-quantum/src/engine/registry/builtin.rs @@ -147,6 +147,9 @@ impl Method for PulseMethod { fn plan(&self, job: &Job<'_>) -> Result { crate::engine::pulse::plan(job.json(), job.job_bytes()) } + fn plan_bounded(&self, job: &Job<'_>) -> Result<(Plan, Option), String> { + crate::engine::pulse::plan_bounded(job.json(), job.job_bytes()).map(|(p, b)| (p, Some(b))) + } fn run(&self, job: &Job<'_>, plan: &Plan, out: &mut Value) -> Result<(), String> { *out = crate::engine::pulse::run(job.json(), plan)?; Ok(()) diff --git a/wai/wai-quantum/src/engine/registry/mod.rs b/wai/wai-quantum/src/engine/registry/mod.rs index 9a83c3468..1b9398d55 100644 --- a/wai/wai-quantum/src/engine/registry/mod.rs +++ b/wai/wai-quantum/src/engine/registry/mod.rs @@ -259,6 +259,14 @@ pub trait Method: Send + Sync { } /// The job admitted, with the memory and work it will take, or why not. fn plan(&self, job: &Job<'_>) -> Result; + /// [`Method::plan`], with the most its result takes as JSON text, bytes, + /// known before it runs, for a method that reads its own program + /// ([`Method::reads_program`]): the engine bounds every other method's + /// result itself. `None`, the default, where it does not say; a host + /// then holds the result to its cap only as it ends. + fn plan_bounded(&self, job: &Job<'_>) -> Result<(Plan, Option), String> { + Ok((self.plan(job)?, None)) + } /// The plan carried out: the method writes its answers into `out`, which /// holds the method, the qubits and the gates (and a compile's or a /// repair's account) already. It must write `accuracy`, with its diff --git a/wai/wai-quantum/src/engine/run.rs b/wai/wai-quantum/src/engine/run.rs index c52580657..5a50ad2d0 100644 --- a/wai/wai-quantum/src/engine/run.rs +++ b/wai/wai-quantum/src/engine/run.rs @@ -23,11 +23,16 @@ const MAX_COUNTS_CHARS: usize = 16 << 20; /// counts that sum to `shots` have at most `k·(log10(shots/k) + 1)` digits /// in all, the most where they are equal. pub(crate) fn counts_bound(width: u32, shots: u64, value: Option) -> u64 { - if shots == 0 { + counts_bound_among(width, shots, if width >= 63 { u64::MAX } else { 1u64 << width }, value) +} + +/// [`counts_bound`] where at most `distinct` outcomes can occur (a pulse +/// run's, which reads fewer qubits than its keys are wide). +pub(crate) fn counts_bound_among(width: u32, shots: u64, distinct: u64, value: Option) -> u64 { + if shots == 0 || distinct == 0 { return 0; } let w = u64::from(width).max(1); - let distinct = if width >= 63 { u64::MAX } else { 1u64 << width }; let keep = (MAX_OUTCOMES as u64).min((MAX_COUNTS_CHARS as u64 / w).max(1)); let k = shots.min(distinct).min(keep); let values = match value { -- 2.54.0 (Apple Git-157) From d5564748e5842c3db5f17eca717dd8501ef8fac5 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Mon, 9 Nov 2026 08:22:34 -0500 Subject: [PATCH 15/18] wai-quantum-service tests: a just-written binary started again while busy, the README's burst sent again when its threshold could not be measured, and a QEC refusal's time held to the host's Two depth tests failed on the Framework in the whole suite under `ulimit -n 256`, at load 43, beside the other tests: - the_binary_can_be_replaced_before_the_restart: the copy it had just written could not be started, "Text file busy" (ETXTBSY). A child that another test forked between the copy's open and its close holds the file open for writing until it execs, close-on-exec or not. launch() now starts it again, every 50 ms for up to 10 s, on that error alone. - the_readme_s_burst_from_one_client_is_answered: its threshold, 18.8 s alone on the loaded host, was measured beside the burst at 58 s and refused at once (J031, past the 25 s an error-correction call may take), so the burst tested nothing. On that refusal alone the threshold is measured again and the burst sent again, three times at most; every other answer is asserted as before. On a0804db72 the threshold is answered and the critiques refused (J032), so this changes nothing there. And one library test, in the same run's --features full --lib (1,418 passed, 1 failed), at load 77 and 302 s where the M3 Ultra took 167 s: a_time_is_measured_apart_from_setup_and_holds_every_threshold asked a colour-code threshold to be refused within 5 s and it was refused after 8.2 s, on its pilot (the call refused, J031). The code-capacity pilot is unchanged this round. The test's first half already measures its own decode with no time set, because a time here is a build's and a host's; the refusal is now asked within 5 s or twice that decode's time, whichever is longer. Co-Authored-By: Claude Opus 5.5 --- wai/wai-quantum-service/tests/depth.rs | 88 ++++++++++++++++---------- wai/wai-quantum/src/engine/qec.rs | 8 ++- 2 files changed, 61 insertions(+), 35 deletions(-) diff --git a/wai/wai-quantum-service/tests/depth.rs b/wai/wai-quantum-service/tests/depth.rs index 9b7bc34ca..be5268a39 100644 --- a/wai/wai-quantum-service/tests/depth.rs +++ b/wai/wai-quantum-service/tests/depth.rs @@ -85,8 +85,7 @@ fn logs(dir: &Path) -> String { fn launch(mut cmd: Command, dir: &Path, burst: u32, env: &[(&str, &str)]) -> (Child, u16) { static STARTS: std::sync::atomic::AtomicUsize = std::sync::atomic::AtomicUsize::new(0); let path = dir.join(format!("service-{}.log", STARTS.fetch_add(1, std::sync::atomic::Ordering::SeqCst))); - let mut child = cmd - .envs(env.iter().copied()) + cmd.envs(env.iter().copied()) .env("WAI_QSVC_ADDR", "127.0.0.1:0") .env("WAI_QSVC_STORE", dir.join("store")) .env("WAI_QSVC_KEY", dir.join("signing.key")) @@ -96,9 +95,18 @@ fn launch(mut cmd: Command, dir: &Path, burst: u32, env: &[(&str, &str)]) -> (Ch .env("WAI_QSVC_ENERGY", "0") .env("WAI_QSVC_CLIENT_BURST", burst.to_string()) .stdout(Stdio::null()) - .stderr(std::fs::File::create(&path).map(Stdio::from).unwrap_or_else(|_| Stdio::null())) - .spawn() - .expect("the service starts"); + .stderr(std::fs::File::create(&path).map(Stdio::from).unwrap_or_else(|_| Stdio::null())); + // A binary this test process has just written (a copy to replace) can + // be busy for a moment: a child another test forked between the write's + // open and its close holds the file open for writing until it execs + // (`ETXTBSY`, Linux), whatever close-on-exec says. Started again shortly. + let tried = Instant::now(); + let mut child = loop { + match cmd.spawn() { + Err(e) if e.kind() == std::io::ErrorKind::ExecutableFileBusy && tried.elapsed() < Duration::from_secs(10) => std::thread::sleep(Duration::from_millis(50)), + started => break started.expect("the service starts"), + } + }; let log = || std::fs::read_to_string(&path).unwrap_or_default(); let t = Instant::now(); let port = loop { @@ -1427,42 +1435,56 @@ fn slow_threshold(port: u16) -> (Value, f64) { /// the lent one, so no critique could have either: each waited out its 10 s /// and was refused (`J032`), or, with the threshold under 10 s, was answered /// only as it ended. A moved call gives its lent turn back: every call is -/// answered, each critique well before the threshold. +/// answered, each critique well before the threshold. On a host so loaded +/// (a whole suite beside it) that the threshold, measured beside the burst, +/// would pass the 25 s an error-correction call may take, it is refused at +/// once (`J031`) and nothing was tested: the threshold is measured again and +/// the burst sent again, three times at most. #[test] fn the_readme_s_burst_from_one_client_is_answered() { let d = dir("readme-burst"); let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1")]); let decode = json!({ "memory": { "distance": 5, "rounds": 5, "p": 0.003 }, "shots": 20000, "decoder": "correlated" }); - let (threshold, alone) = slow_threshold(port); let critique = json!({ "qasm": "OPENQASM 3.0;\nqubit[2] q;\nh q[0];\ncx q[0], q[1];\n" }); - let mut calls: Vec<(&str, &str, &Value)> = vec![("/v1/qec/decode", "qec_decode", &decode), ("/v1/qec/threshold", "qec_threshold", &threshold)]; - calls.extend(std::iter::repeat_n(("/v1/critique", "critique_qasm", &critique), 6)); for (route, me) in [("HTTP", "198.51.100.77"), ("MCP", "198.51.100.78")] { - let began = Instant::now(); - let answers: Vec<(bool, String, f64)> = std::thread::scope(|t| { - let each: Vec<_> = calls - .iter() - .enumerate() - .map(|(i, (path, tool, body))| { - let began = &began; - t.spawn(move || { - std::thread::sleep(Duration::from_millis(30 * i as u64)); - let (ok, out) = if route == "HTTP" { - let (code, out) = post_as(port, me, path, body); - (code == 200, out) - } else { - mcp_as(port, me, tool, body) - }; - (ok, out, began.elapsed().as_secs_f64()) + let mut tries = 0; + let (answers, at, alone) = loop { + tries += 1; + let (threshold, alone) = slow_threshold(port); + let mut calls: Vec<(&str, &str, &Value)> = vec![("/v1/qec/decode", "qec_decode", &decode), ("/v1/qec/threshold", "qec_threshold", &threshold)]; + calls.extend(std::iter::repeat_n(("/v1/critique", "critique_qasm", &critique), 6)); + let began = Instant::now(); + let answers: Vec<(bool, String, f64)> = std::thread::scope(|t| { + let each: Vec<_> = calls + .iter() + .enumerate() + .map(|(i, (path, tool, body))| { + let began = &began; + t.spawn(move || { + std::thread::sleep(Duration::from_millis(30 * i as u64)); + let (ok, out) = if route == "HTTP" { + let (code, out) = post_as(port, me, path, body); + (code == 200, out) + } else { + mcp_as(port, me, tool, body) + }; + (ok, out, began.elapsed().as_secs_f64()) + }) }) - }) - .collect(); - each.into_iter().map(|c| c.join().unwrap()).collect() - }); - let at: Vec = calls.iter().zip(&answers).map(|((path, tool, _), (ok, _, s))| format!("{} {} at {s:.2} s", if route == "HTTP" { path } else { tool }, if *ok { "answered" } else { "refused" })).collect(); - eprintln!("{route}, the threshold alone {alone:.1} s: {}", at.join(", ")); - for ((path, ..), (ok, out, s)) in calls.iter().zip(&answers) { - assert!(*ok, "{route}: {path} at {s:.2} s (the threshold alone {alone:.1} s): {}\n{at:#?}", out.lines().last().unwrap_or("")); + .collect(); + each.into_iter().map(|c| c.join().unwrap()).collect() + }); + let at: Vec = calls.iter().zip(&answers).map(|((path, tool, _), (ok, _, s))| format!("{} {} at {s:.2} s", if route == "HTTP" { path } else { tool }, if *ok { "answered" } else { "refused" })).collect(); + eprintln!("{route}, try {tries}, the threshold alone {alone:.1} s: {}", at.join(", ")); + let (ok, out, _) = &answers[1]; + if !ok && out.contains("J031") && out.contains("would take about") && tries < 3 { + eprintln!("{route}: the threshold, measured beside the burst, was past the time a call may take, so nothing was tested; again"); + continue; + } + break (answers, at, alone); + }; + for ((ok, out, _), name) in answers.iter().zip(&at) { + assert!(*ok, "{route}: {name} (the threshold alone {alone:.1} s): {}\n{at:#?}", out.lines().last().unwrap_or("")); } let threshold_at = answers[1].2; for (_, _, s) in &answers[2..] { diff --git a/wai/wai-quantum/src/engine/qec.rs b/wai/wai-quantum/src/engine/qec.rs index a644cf37d..7fe43fb07 100644 --- a/wai/wai-quantum/src/engine/qec.rs +++ b/wai/wai-quantum/src/engine/qec.rs @@ -896,10 +896,14 @@ mod tests { let r = within_time(took * 2 + std::time::Duration::from_secs(1), || call("decode", &req)); assert!(r.is_ok(), "{r:?} after {:?}, where it took {took:?} with no time set", t.elapsed()); // Refused on its pilot, which is held to about half a second, not on - // its one in a hundred run whole (11 s in a debug build on macOS). + // its one in a hundred run whole (11 s in a debug build on macOS): + // within 5 s, or twice the decode's time where the host makes that + // longer (8.2 s in a release build beside the library's other tests, + // the Framework at load 77, where 5 s was asked). let t = std::time::Instant::now(); let e = within_time(std::time::Duration::from_secs(3), || call("threshold", &json!({ "family": "color", "distances": [15], "p": [0.08], "shots": 200_000 }))).unwrap_err(); - assert!(e.ends_with("[J031]") && t.elapsed() < std::time::Duration::from_secs(5), "{e} after {:?}", t.elapsed()); + let most = std::time::Duration::from_secs(5).max(took * 2); + assert!(e.ends_with("[J031]") && t.elapsed() < most, "{e} after {:?}, where {most:?} was asked (the decode took {took:?})", t.elapsed()); let t = std::time::Instant::now(); let r = within_time(std::time::Duration::from_secs(25), || call("threshold", &json!({ "family": "surface", "decoder": "correlated" }))); assert!(r.is_ok() || t.elapsed() < std::time::Duration::from_secs(12), "{r:?} after {:?}", t.elapsed()); -- 2.54.0 (Apple Git-157) From f45df2f1d0b2dc2be7a50b67fcea84ad7597b855 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Mon, 9 Nov 2026 13:51:45 -0500 Subject: [PATCH 16/18] wai-quantum-service: a result's bound holds for every job, a dry run says what its run takes, a client's quick calls pass its own long checks, and an MCP id comes back as it was sent Sixth review of #676: - The bound on a result is a true upper bound, part by part (catalog::Bound: counts, verdict, notes, requests, rest). A sampled intent's fit names at most 64 of the outcomes it rules out and says how many (impossible_outcomes), and its reason says them once; the bound counts each intent's lists and the statement's reasons. A trajectories run's counts are bounded by its trajectories. A verdict counts the reader's notes and the repair again. Every number is counted at the length RFC 8785 writes it (receipt::canonical_len), so a set's items' requests and the provenance's shots and seed are counted as served. - Where the method knows better, the bound is close: at most 2^b outcomes for the b bits measured, and on the noiseless stabilizer method 2^d, d its measured qubits and its branching gates (a GHZ state's two). A verdict's critique is bounded by the smells its program's gates can draw (critique::smells_bound, the critique's own walk), not 512 bytes a gate: 100 GHZ24 at 30,000 shots, and a 124,700-gate loop with its verdict, are taken. A refusal names the largest part and what makes it smaller. - A dry run applies the run's checks: over MCP it plans the job as the run tools run it, with its verdict, and its reading is held to what the submission's would take back (refused as accepted: false). - A set's worker writes each item's result as text as it ends, so the README's 100 points of 30,000 shots run to the end within their memory. - A turn lent to a client goes back once one of its own requests has waited 7 s, as once another client's waits, so two long checks never hold its quick calls past their wait. A client's eleventh request at once is refused, of readings as of checks, and no turn is lent while one of its own is free or held by a request waiting for the pool. - Every MCP answer echoes the JSON-RPC id as the text it was sent as. - A long poll past its client's holds is answered 429 (503 past everyone's) with Retry-After and the job's status; the command line's follow waits and polls again. - Every job queued or running counts against the service-wide unread cap at its bound, checked again under the queue's lock. - README, OpenAPI, the tool schema and the changelog say what holds; the MCP tool snapshot and the command line's docs are regenerated. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/wai-quantum-service.yml | 8 +- wai/changes/2026-11-06-quantum-parse-depth.md | 45 +- wai/wai-quantum-cli/docs/llms-full.txt | 2 +- wai/wai-quantum-cli/docs/local-mcp.json | 280 ++++++++++-- wai/wai-quantum-cli/docs/service-mcp.json | 327 +++++++++++--- wai/wai-quantum-cli/src/service.rs | 9 +- wai/wai-quantum-service/Cargo.toml | 3 +- wai/wai-quantum-service/README.md | 12 +- .../schemas/service-tools.v1.json | 2 +- wai/wai-quantum-service/src/discovery.rs | 8 +- wai/wai-quantum-service/src/http.rs | 134 ++++-- wai/wai-quantum-service/src/jobs.rs | 89 +++- wai/wai-quantum-service/src/main.rs | 26 +- wai/wai-quantum-service/src/mcp.rs | 21 +- wai/wai-quantum-service/src/offload.rs | 237 ++++++++-- wai/wai-quantum-service/tests/depth.rs | 412 +++++++++++++++++- wai/wai-quantum/schemas/verdict.v1.json | 7 +- wai/wai-quantum/src/engine/bound_tests.rs | 344 +++++++++++++++ wai/wai-quantum/src/engine/catalog.rs | 210 +++++++-- wai/wai-quantum/src/engine/critique.rs | 156 ++++--- wai/wai-quantum/src/engine/mod.rs | 2 + wai/wai-quantum/src/engine/pulse.rs | 21 +- wai/wai-quantum/src/engine/receipt.rs | 48 ++ wai/wai-quantum/src/engine/run.rs | 20 +- wai/wai-quantum/src/engine/sets.rs | 8 +- wai/wai-quantum/src/engine/stats.rs | 22 +- wai/wai-quantum/src/engine/verdict.rs | 6 +- 27 files changed, 2108 insertions(+), 351 deletions(-) create mode 100644 wai/wai-quantum/src/engine/bound_tests.rs diff --git a/.github/workflows/wai-quantum-service.yml b/.github/workflows/wai-quantum-service.yml index 5460c630f..a1a93ce71 100644 --- a/.github/workflows/wai-quantum-service.yml +++ b/.github/workflows/wai-quantum-service.yml @@ -86,13 +86,14 @@ jobs: # counter's reader and probe (on fake trees; a container hides the real # one), and a real-valued input bound by the reader. # The bound on a result the service admits a job by (a circuit's, a - # set's and a pulse run's): no result passes it. + # set's and a pulse run's): no result passes it, for jobs of every kind + # drawn at random, and it is close where the method knows its outcomes. - name: The engine's sets, provenance, result bounds and package energy shell: bash working-directory: wai/wai-quantum run: | set -euo pipefail - cargo test --locked --release --features full --lib -- engine::sets:: engine::energy:: merkle:: engine::run::tests::a_result_s_provenance engine::run::tests::no_result_passes_the_bound engine::pulse::tests::no_pulse_result_passes_the_bound qasm::tests::a_bound_parameter 2>&1 | tee "$RUNNER_TEMP/engine.log" + cargo test --locked --release --features full --lib -- engine::sets:: engine::energy:: merkle:: engine::run::tests::a_result_s_provenance engine::run::tests::no_result_passes_the_bound engine::pulse::tests::no_pulse_result_passes_the_bound engine::bound_tests:: qasm::tests::a_bound_parameter 2>&1 | tee "$RUNNER_TEMP/engine.log" grep -qE '^test result: ok\. [1-9][0-9]* passed; 0 failed' "$RUNNER_TEMP/engine.log" || { echo "::error::the engine's tests did not pass"; exit 1; } for t in engine::sets::tests::one_receipt_binds_every_item_and_any_change_breaks_it \ engine::sets::tests::a_sweep_is_its_grid_each_point_the_program_written_out \ @@ -101,6 +102,9 @@ jobs: engine::run::tests::a_result_s_provenance_names_the_passes_its_certificate_ran \ engine::run::tests::no_result_passes_the_bound_its_plan_gave \ engine::pulse::tests::no_pulse_result_passes_the_bound_its_plan_gave \ + engine::bound_tests::no_random_job_s_result_passes_its_bound \ + engine::bound_tests::a_failing_intent_trajectories_and_notes_stay_within_their_bounds \ + engine::bound_tests::a_ghz_state_and_a_long_loop_are_bounded_by_what_they_can_list \ engine::energy::tests::the_probe_finds_a_live_counter_s_cadence_and_refuses_a_stalled_one \ engine::energy::tests::only_top_level_package_zones_are_read_and_a_frozen_one_is_not_live \ engine::energy::tests::a_package_figure_names_its_scope_and_declares_its_edges \ diff --git a/wai/changes/2026-11-06-quantum-parse-depth.md b/wai/changes/2026-11-06-quantum-parse-depth.md index 356326a71..7843ac685 100644 --- a/wai/changes/2026-11-06-quantum-parse-depth.md +++ b/wai/changes/2026-11-06-quantum-parse-depth.md @@ -129,19 +129,34 @@ refused `503` past them before a body is read. `tasks/result` holds its answer within what is left of the answer's time. Four connections past the cap are held for health checks alone, and long polls and event streams are held to `WAI_QSVC_CLIENT_STREAMS` (4) a client and `WAI_QSVC_STREAMS` (512, a quarter of the connections at most); past them - a long poll is answered at once with the job as it is. A reading that moves from a lent turn onto its client's own - gives the lent one back. Results read and not yet read are counted apart: only read ones are dropped before their + a long poll of a job still queued or running is not held, and says so: `429` (its client's) or `503` (everyone's), + `J032`, with `Retry-After: 2` and the job's status (the command line's `follow` waits and polls again). A reading + that moves from a lent turn onto its client's own gives the lent one back; one still on a lent turn past 5 s goes + back once another client's request waits for a turn or one of its own client's has waited 7 s, so two long checks + of one client never hold its quick calls past their wait. A client's eleventh request at once is refused, of + readings (the reading pool lends no turn at the default share of two) and of checks alike, with eight waiting. Results read and not yet read are counted apart: only read ones are dropped before their time, the largest holder's first; a client's next job waits while its unread results, with each job running at its result's bound and the new one at `WAI_QSVC_RESULT_MIB` (64), would pass `WAI_QSVC_CLIENT_UNREAD_MIB` (256), - and every next job while all unread hold `WAI_QSVC_UNREAD_MIB` (8192). A job's or a set's result is bounded before - it runs (`catalog::plan_bounded`, `sets::plan_bounded`, new: each item's counts at most its shots, `2^bits` and - 65,536 outcomes, every other part by what it may hold or at its size as read; a pulse run's from its schedule, - `pulse::plan_bounded`, through `Method::plan_bounded`), and a request whose bound passes - `WAI_QSVC_RESULT_MIB` is refused at once (`422`, `J031`), its bound and the cap in the reason; a dry run states the - bound (`result_bytes`), and `GET /v1`, the dry run's bounds and the OpenAPI document the cap. At 64 MiB, 100 - points of 65,536 outcomes (about 150 MB) are refused, and 100 points of 16 bits at 30,000 shots are taken. The - check as a result ends stays as a backstop, for a job kept across a restart that lowered the cap. - MCP answers echo a JSON-RPC id exactly. Readers and workers start from the service's own + and every next job while all unread, with every job queued or running at its result's bound and the new one, would + pass `WAI_QSVC_UNREAD_MIB` (8192), so what ended jobs keep stays under the two caps. A job's or a set's result is + bounded before it runs, part by part (`catalog::plan_bounded`, `catalog::plan_bound` and `catalog::Bound`, + `sets::plan_bounded`, new): each item's counts at most the samples it lists (its shots, or a `trajectories` run's + trajectories) among the outcomes its measured bits can give (a noiseless `stabilizer` run's, two to the gates + that branch: a GHZ state's two) and 65,536; a verdict's critique by the remarks its program's gates can draw + (`critique::smells_bound`, new: the walk the critique makes), each intent's check with the 64 outcomes it lists + at a time, and the reader's notes and repair again; every other part by what it may hold or at its length as + RFC 8785 writes it, each number in its own form (`receipt::canonical_len`, new); a pulse run's from its schedule, + `pulse::plan_bounded`, through `Method::plan_bounded`. A sampled intent's fit names at most 64 of the outcomes it + rules out and says how many there were (`impossible_outcomes`, new), and its reason says them once. A request + whose bound passes `WAI_QSVC_RESULT_MIB` is refused at once (`422`, `J031`), its bound, its largest part and what + makes that part smaller in the reason; a dry run states the bound (`result_bytes`), is held to what its + submission's reading would take back, and over MCP plans the job as the run tools run it, its verdict asked; and + `GET /v1`, the dry run's bounds and the OpenAPI document the cap. At 64 MiB, 100 points of 65,536 outcomes (about + 150 MB) are refused, and 100 points of 16 bits at 30,000 shots are taken and run to the end: a set's worker keeps + each item's result as text as it ends, where it kept every item's as a tree and passed the memory it was admitted + with. 100 copies of a 24-qubit GHZ state at 30,000 shots, and a loop of 124,700 gates with its verdict, are taken. + The check as a result ends stays as a backstop, for a job kept across a restart that lowered the cap. + MCP answers echo each JSON-RPC id as the text it was sent as (`1.0`, `1e3`, an integer past 64 bits). Readers and workers start from the service's own binary as it was started (Linux: its running image; elsewhere a copy kept beside the signing key), so the binary can be replaced before the restart. A restart reads no program: a job keeps the plan it was admitted with and a set its items, where a queue of slow programs held the port unbound for the sum of their reads. @@ -150,7 +165,13 @@ the service binary started in a test and sent each of these through each door, answering `/v1/health` after each; refused MCP tasks that hold no worker; heavy requests from eight clients beside a health probe; a client's share of the turns; a reading stopped at its deadline; a hang-up that frees its turn; stuck connections let go; and a - restart that reads nothing. Two tests that could only end by the job's own limit now fail by name. + restart that reads nothing. Two tests that could only end by the job's own limit now fail by name. A property test + draws jobs of every kind (failing sampled intents, trajectories, verdicts over notes and smells, random Clifford + circuits, noise, observables, amplitudes, mitigations, repairs, compiles, measurements in the middle, device + targets, pulse runs, intents carrying long numbers) and holds each result to its bound; the service's tests run + the README's largest sweep to the end, a GHZ set, the long loop, an MCP dry run against its run, a dry run against + its submission, a set's long numbers, two long checks beside quick calls, raw ids, a long poll past its holds and + the unread cap. - **Breaking:** `api:rust:wai_quantum::qasm::ast::E::Chain`: a run of left-associative operators of one precedence is one node, `E::Chain(first, [(op, operand), …])`, so a run of any length holds no recursion; a match over every variant of `E` outside the crate adds this one. diff --git a/wai/wai-quantum-cli/docs/llms-full.txt b/wai/wai-quantum-cli/docs/llms-full.txt index 6fa9f152d..68f86c8a3 100644 --- a/wai/wai-quantum-cli/docs/llms-full.txt +++ b/wai/wai-quantum-cli/docs/llms-full.txt @@ -672,7 +672,7 @@ Parameters: ### `dry_run_quantum_job`: Plan a quantum job or set without running it -What the service would do with a job or a set, running nothing: the method and why, the memory, the estimated time and energy (each with its basis: this service's own timings and readings where it has them, otherwise the planner's nominal rate, and no energy figure without readings), the bounds that apply, where you stand against your limits, and whether the service would take it now, or the refusal it would give, with its code. Takes what run_quantum_circuit or run_quantum_set takes. It counts as one submission. +What the service would do with a job or a set, running nothing: the method and why, the memory, the estimated time and energy (each with its basis: this service's own timings and readings where it has them, otherwise the planner's nominal rate, and no energy figure without readings), the bounds that apply, where you stand against your limits, and whether the service would take it now, or the refusal it would give, with its code. Takes what run_quantum_circuit or run_quantum_set takes, and plans it as they run it, its verdict asked: what it says they take, they take. It counts as one submission. Parameters: - `qasm` (string): The program, OpenQASM 3 (or 2), read as the whole language: registers, aliases and slices; the standard gates, `U` and `gphase`; user `gate` definitions and `def` subroutines, inlined; every modifier (`ctrl @`, `negctrl @`, `inv @`, `pow(k) @`) on any gate, lowered exactly; classical types (int, uint, float, angle, complex, bool, bit, arrays), their arithmetic and built-in functions, computed on reading; `for`, `while`, `switch`, `break`, `continue`; measurement and `reset` anywhere; classical control on measured bits (`if (c[0]) x q[1];`, `if (c >= 2) { … } else { … }`, `&&`, `||`); a `while` on measured bits takes `@wai.max_iterations N` before it. `output` declarations come back in the result; `delay`, `box` and `stretch` are carried and labelled as not applied; `defcal` and `cal` are kept, not run. A program that measures in the middle or acts on what it measured runs on statevector or stabilizer: counts split their shots at each measurement; expectation values are exact, of the state before the final measurements. What the reader refuses comes back with a code, as `[Q103]`; qasm_diagnostics says what each code means. diff --git a/wai/wai-quantum-cli/docs/local-mcp.json b/wai/wai-quantum-cli/docs/local-mcp.json index 08198bceb..2e6c17e14 100644 --- a/wai/wai-quantum-cli/docs/local-mcp.json +++ b/wai/wai-quantum-cli/docs/local-mcp.json @@ -2368,7 +2368,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -6212,7 +6217,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -8401,7 +8411,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -8809,7 +8824,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -9804,7 +9824,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -10212,7 +10237,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -10700,7 +10730,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -11101,7 +11136,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -11511,7 +11551,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -12044,7 +12089,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -12452,7 +12502,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -12994,7 +13049,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -13395,7 +13455,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -13805,7 +13870,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -14253,7 +14323,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -14661,7 +14736,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -15742,7 +15822,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -16143,7 +16228,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -16553,7 +16643,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -17066,7 +17161,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -17467,7 +17567,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -17877,7 +17982,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -18869,7 +18979,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -19277,7 +19392,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -20515,7 +20635,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -20923,7 +21048,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -22724,7 +22854,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -23125,7 +23260,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -23535,7 +23675,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -24237,7 +24382,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -24617,7 +24767,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -24997,7 +25152,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -25401,7 +25561,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -25777,7 +25942,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -26216,7 +26386,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -26626,7 +26801,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -32834,7 +33014,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -36678,7 +36863,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -52883,7 +53073,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -56727,7 +56922,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" diff --git a/wai/wai-quantum-cli/docs/service-mcp.json b/wai/wai-quantum-cli/docs/service-mcp.json index b605dda1c..52c825bb7 100644 --- a/wai/wai-quantum-cli/docs/service-mcp.json +++ b/wai/wai-quantum-cli/docs/service-mcp.json @@ -2384,7 +2384,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -6228,7 +6233,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -8417,7 +8427,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -8516,7 +8531,7 @@ }, "accepted": { "type": "boolean", - "description": "Whether the service would take it now, by every limit." + "description": "Whether the service would take it now, by every limit, its reading held to what the submission's would take; the MCP tools plan it as their runs run it, its verdict asked." }, "status": { "type": "integer", @@ -9011,7 +9026,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -9419,7 +9439,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -9796,7 +9821,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -10206,7 +10236,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -12667,7 +12702,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -16511,7 +16551,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -18700,7 +18745,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -18799,7 +18849,7 @@ }, "accepted": { "type": "boolean", - "description": "Whether the service would take it now, by every limit." + "description": "Whether the service would take it now, by every limit, its reading held to what the submission's would take; the MCP tools plan it as their runs run it, its verdict asked." }, "status": { "type": "integer", @@ -19294,7 +19344,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -19702,7 +19757,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -20079,7 +20139,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -20489,7 +20554,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -20578,7 +20648,7 @@ { "name": "dry_run_quantum_job", "title": "Plan a quantum job or set without running it", - "description": "What the service would do with a job or a set, running nothing: the method and why, the memory, the estimated time and energy (each with its basis: this service's own timings and readings where it has them, otherwise the planner's nominal rate, and no energy figure without readings), the bounds that apply, where you stand against your limits, and whether the service would take it now, or the refusal it would give, with its code. Takes what run_quantum_circuit or run_quantum_set takes. It counts as one submission.", + "description": "What the service would do with a job or a set, running nothing: the method and why, the memory, the estimated time and energy (each with its basis: this service's own timings and readings where it has them, otherwise the planner's nominal rate, and no energy figure without readings), the bounds that apply, where you stand against your limits, and whether the service would take it now, or the refusal it would give, with its code. Takes what run_quantum_circuit or run_quantum_set takes, and plans it as they run it, its verdict asked: what it says they take, they take. It counts as one submission.", "inputSchema": { "type": "object", "properties": { @@ -21151,7 +21221,7 @@ }, "accepted": { "type": "boolean", - "description": "Whether the service would take it now, by every limit." + "description": "Whether the service would take it now, by every limit, its reading held to what the submission's would take; the MCP tools plan it as their runs run it, its verdict asked." }, "status": { "type": "integer", @@ -21646,7 +21716,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -22054,7 +22129,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -23995,7 +24075,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -27839,7 +27924,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -30028,7 +30118,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -30127,7 +30222,7 @@ }, "accepted": { "type": "boolean", - "description": "Whether the service would take it now, by every limit." + "description": "Whether the service would take it now, by every limit, its reading held to what the submission's would take; the MCP tools plan it as their runs run it, its verdict asked." }, "status": { "type": "integer", @@ -30622,7 +30717,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -31030,7 +31130,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -31407,7 +31512,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -31817,7 +31927,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -32575,7 +32690,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -32976,7 +33096,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -33386,7 +33511,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -33796,7 +33926,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -34240,7 +34375,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -34648,7 +34788,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -35058,7 +35203,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -36186,7 +36336,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -36587,7 +36742,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -36997,7 +37157,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -37407,7 +37572,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -37913,7 +38083,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -38314,7 +38489,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -38724,7 +38904,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -39134,7 +39319,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -40113,7 +40303,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -40521,7 +40716,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -40931,7 +41131,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -42261,7 +42466,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -42669,7 +42879,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -47043,7 +47258,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -50887,7 +51107,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" @@ -52810,7 +53035,7 @@ }, "accepted": { "type": "boolean", - "description": "Whether the service would take it now, by every limit." + "description": "Whether the service would take it now, by every limit, its reading held to what the submission's would take; the MCP tools plan it as their runs run it, its verdict asked." }, "status": { "type": "integer", diff --git a/wai/wai-quantum-cli/src/service.rs b/wai/wai-quantum-cli/src/service.rs index 14e9ba94c..6094eb5b6 100644 --- a/wai/wai-quantum-cli/src/service.rs +++ b/wai/wai-quantum-cli/src/service.rs @@ -163,7 +163,11 @@ impl Client { } loop { let (code, v) = self.get(&format!("/v1/jobs/{id}?wait=25"))?; - if code >= 400 { + // A poll past the client's holds (or everyone's) is not held: it + // says so (429 or 503, J032) with the job's status, and is sent + // again after the pause below. + let unheld = matches!(code, 429 | 503) && v["code"] == "J032" && v["status"].is_string(); + if code >= 400 && !unheld { return Err(Self::refused(code, &v)); } let s = v["status"].as_str().unwrap_or("").to_string(); @@ -174,7 +178,8 @@ impl Client { if start.elapsed() > FOLLOW_FOR { return Err(format!("the job {id} is still {s}; read it later with `wai-quantum task status {id}`")); } - // A service that does not hold the poll answers at once. + // A service that does not hold the poll answers at once (and one + // past its holds says to wait 2 s). std::thread::sleep(Duration::from_secs(2)); } } diff --git a/wai/wai-quantum-service/Cargo.toml b/wai/wai-quantum-service/Cargo.toml index 9793d5534..022b747d6 100644 --- a/wai/wai-quantum-service/Cargo.toml +++ b/wai/wai-quantum-service/Cargo.toml @@ -16,7 +16,8 @@ tokio = { version = "1", features = ["rt-multi-thread", "macros", "process", "ti serde = { version = "1", features = ["derive"] } # float_roundtrip: a number a client sends back reads as the double it was # (the default reader rounds twice), so its receipt still verifies. -serde_json = { version = "1", features = ["float_roundtrip"] } +# raw_value: a JSON-RPC id is echoed as the text it was sent as (src/http.rs). +serde_json = { version = "1", features = ["float_roundtrip", "raw_value"] } getrandom = "0.4" # The job store's content addresses (src/store.rs), as the engine's receipts use. blake3 = "=1.8.7" diff --git a/wai/wai-quantum-service/README.md b/wai/wai-quantum-service/README.md index b72d69457..d33de0ef7 100644 --- a/wai/wai-quantum-service/README.md +++ b/wai/wai-quantum-service/README.md @@ -47,9 +47,9 @@ One request can run many programs, as one job: - **A set:** `"programs"`, a list, each item a program's text or a job (`{"qasm", …}`). The request's other fields (`shots`, `observables`, `seed`, `intent`, …) apply to every item that does not set them itself. - **A sweep:** `"qasm"` with `"sweep"`, each of the program's inputs and its values. Every combination runs, the names in sorted order and the first varying slowest. An integer input (`input uint n;`) takes whole numbers, and the program is written out at each. A float or angle input (`input float theta;`, `input angle phi;`) takes any number, bound to exactly the double given. Every input the program declares needs values; a list of one fixes it. -A set holds at most 100 items. They run one after another in one worker, inside one job's time limit, holding the memory of the largest item. Each item is a plain job: its request is complete in itself, and it runs, plans and verifies as it would alone. A set with an item that does not plan is refused, naming the item. +A set holds at most 100 items. They run one after another in one worker, inside one job's time limit, holding the memory of the largest item; each item's result is kept as text as it ends, so the results of earlier items take no more than they will as served. Each item is a plain job: its request is complete in itself, and it runs, plans and verifies as it would alone. A set with an item that does not plan is refused, naming the item. -A job's result, or a set's with every item's request and result in it, may take at most `WAI_QSVC_RESULT_MIB` (64 MiB) as served. That is checked before anything runs, from a bound of the result read from the request: each item's counts list at most its shots, `2^bits` and 65,536 outcomes, each its bit string, quotes, a colon, a comma and its count's digits; each other part is counted by what it may hold or at its size as read (a verdict asking for one counts 512 bytes for each gate its critique may remark on). A request whose bound passes the cap is refused at once, `422` and `J031`, with the bound and the cap in the reason, and a dry run gives the bound as `result_bytes`. The cap is less than a set at its most can give: at about 22 bytes a 16-bit outcome, 100 points of 65,536 outcomes each are about 150 MB, and are refused; 100 points of 16 measured bits are taken up to about 30,000 shots a point. Send fewer points or shots, split the set, or run it on your own machine (`wai-quantum run`), where no cap applies. A pulse run is bounded the same way, from its schedule as read: its populations and its distribution by the qubits it simulates and the captures it reads, each keyed as wide as the program's bits, its counts by its shots, and each capture with its frame's and bit's names. The check as a result ends stays only as a backstop, for a job kept across a restart that lowered the cap. +A job's result, or a set's with every item's request and result in it, may take at most `WAI_QSVC_RESULT_MIB` (64 MiB) as served. That is checked before anything runs, from a bound of the result read from the request: each item's counts list at most its shots (a `trajectories` run's, its trajectories), the outcomes its measured bits can give (a noiseless `stabilizer` run's, two to the power of the gates that branch, `h`, `sx` and `sxdg`: a GHZ state's two, however wide) and 65,536 outcomes, each its bit string, quotes, a colon, a comma and its count's digits; a verdict counts the critique's remarks the program can draw (one for each gate that does nothing and each pair that cancels or merges, not one for every gate), each intent's check with the 64 outcomes it lists at a time, and the reader's notes again; each other part is counted by what it may hold or at its length as RFC 8785 writes it, every number in its own form (`1e20` is 21 characters). A request whose bound passes the cap is refused at once, `422` and `J031`, with the bound, its largest part (the counts, the verdict, the reader's notes, the items' requests or the rest) and what makes that part smaller in the reason, and a dry run gives the bound as `result_bytes`. The cap is less than a set at its most can give: at about 22 bytes a 16-bit outcome, 100 points of 65,536 outcomes each are about 150 MB, and are refused; 100 points of 16 measured bits are taken, and run to the end, up to about 30,000 shots a point (30,878 without a verdict; 30,003 over MCP, whose runs ask for one). Send fewer points or shots, split the set, or run it on your own machine (`wai-quantum run`), where no cap applies. A pulse run is bounded the same way, from its schedule as read: its populations and its distribution by the qubits it simulates and the captures it reads, each keyed as wide as the program's bits, its counts by its shots, and each capture with its frame's and bit's names. The check as a result ends stays only as a backstop, for a job kept across a restart that lowered the cap. The result is a set (`"format": "wai.quantum.set/1"`). Each item has its own result and its own receipt. Each item's leaf is the digest its receipt signs (a failed item's: the digest of its request and its error). The head holds the Merkle root of the leaves, under the JWP Merkle domains, with the count, and the set's receipt seals the set's request and its head. Each item carries its `path` to the root, so one item can be shown sealed by its set without the others. `POST /v1/verify` checks the whole set: its receipt, every item as the request gives it, each item's receipt, the root, and each path. @@ -182,7 +182,7 @@ Every refusal says why and ends with its code, which the body also carries as `c ## Status and limits -A job is `queued`, `running`, `done`, `failed`, `cancelled`, or `interrupted`: the service stopped while it ran, after it had run it again once, so it has no result, and `error` says so. Follow a job by its event stream (`GET /v1/jobs/{id}/events`), or by polling `GET /v1/jobs/{id}`, with `?wait=S` to hold each poll until the status changes. +A job is `queued`, `running`, `done`, `failed`, `cancelled`, or `interrupted`: the service stopped while it ran, after it had run it again once, so it has no result, and `error` says so. Follow a job by its event stream (`GET /v1/jobs/{id}/events`), or by polling `GET /v1/jobs/{id}`, with `?wait=S` to hold each poll until the status changes. A client holds at most `WAI_QSVC_CLIENT_STREAMS` (4) long polls and event streams at once: a long poll past them is not held, and says so, `429` (`503` past everyone's), `J032`, with `Retry-After: 2` and the job's `status`; wait that long before polling again, or poll without `?wait`. Every answer carries the client's submission window in the rate-limit fields of the IETF HTTP API working group: `RateLimit-Policy: "submissions";q=30;w=600` (at most 30 submissions in 600 s) and `RateLimit: "submissions";r=12;t=34` (12 left, one more in 34 s). A refusal past the limits carries `Retry-After`. @@ -200,13 +200,13 @@ A submission is admitted in the order of what each step costs: Reading happens outside the service's own process, as each job's run does: each request is read by a child, this binary run as `wai-quantum-service read`, sandboxed as a worker is. -- **Turns.** Four requests are read at a time (jobs, dry runs and the MCP tools' verdicts), and four checks beside them. A client has its share of each: `WAI_QSVC_CLIENT_READS` (2) of the first and `WAI_QSVC_CLIENT_CHECKS` (1) of the checks, so no client holds a whole pool. Past its share, one more turn is lent to a client while the pool keeps two free for others, so its quick calls (a critique, a lint, an estimate) pass its own slow one. A call on a lent turn moves onto the client's own turn as soon as one comes free, and then has the whole of its time and gives the lent turn back, so the client's quick calls pass it on that one; until then it runs past five seconds only while no other request waits for a turn, and once one does it is refused, `503` and `J032`, to send again. Its other requests wait behind its own turns, in order, so an agent's burst of parallel calls is answered one after another: up to eight of them may wait, at most 32 requests wait in all, each for ten seconds, and past any of these a request is refused, `503` and `J032`, having read and spent nothing, saying whether its own requests or the pool held it up. A client is its address, so people behind one address (an office, a carrier's NAT, a class) share one client's turns and its window: raise the shares, or give such a group an API key of its own on a private deployment. +- **Turns.** Four requests are read at a time (jobs, dry runs and the MCP tools' verdicts), and four checks beside them. A client has its share of each: `WAI_QSVC_CLIENT_READS` (2) of the first and `WAI_QSVC_CLIENT_CHECKS` (1) of the checks, so no client holds a whole pool. Past its share, one more turn is lent to a client while the pool keeps two free for others, so its quick calls (a critique, a lint, an estimate) pass its own slow one: of the checks, at their share of one; the reading pool's four turns, at its share of two, leave none to lend. A call on a lent turn moves onto the client's own turn as soon as one comes free, and then has the whole of its time and gives the lent turn back, so the client's quick calls pass it on that one; until then it runs past five seconds only while no other client's request waits for a turn and none of its own client's has waited seven seconds, and once one has it is refused, `503` and `J032`, to send again: two long calls of one client never hold its quick ones past their wait. Its other requests wait behind its own turns, in order, so an agent's burst of parallel calls is answered one after another: up to eight of them may wait (so its eleventh request at once, of readings or of checks, is refused), at most 32 requests wait in all, each for ten seconds, and past any of these a request is refused, `503` and `J032`, having read and spent nothing, saying whether its own requests or the pool held it up. A client is its address, so people behind one address (an office, a carrier's NAT, a class) share one client's turns and its window: raise the shares, or give such a group an API key of its own on a private deployment. - **Memory.** A child's memory is polled every 50 ms from a thread of its own, and the child is killed once it holds more than `WAI_QSVC_READ_GB` (4); the request is refused, `422` and `J031`. The kill is a signal the service sends itself (`SIGKILL`, to the child's process group, and again every 100 ms until it has ended), so it needs no new process. On macOS the poll reads the child's physical footprint, which counts its compressed and swapped pages; on Linux its resident and swapped pages. A poll is not a kernel limit: a child can pass the cap by what it touches in one interval before it is killed. On Linux the child also holds its own address space to twice the cap and a gigabyte, so an allocation past that fails at once (`422`, `J031`), and the kernel ends a reader or a worker before the service when the host runs short. macOS gives an unprivileged process neither, and has no limit that counts compressed memory. - **Time.** A child that reads for longer than `WAI_QSVC_READ_SECS` (45) is killed and the request refused, `422` and `J031`. Every request is answered within `WAI_QSVC_ANSWER_SECS` (55) of its arrival, under the 60 s a proxy in front commonly waits for an answer: its waits for a turn and its readings end by then, `POST /v1/run` and the MCP run tools wait for their job only for what is left and then answer with the job, still running, to poll, and an MCP answer's verdict is read again only if time is left (its verdict says so where it was not). An error-correction call's shots are measured as they run: each point is set up and sampled, and a pilot of its own shots decoded, before the rest of any; it is refused at once when what is measured says it would take more than `WAI_QSVC_QEC_SECS` (25), and stopped if it runs a fifth past that. - **Its answer.** What a reading gives back comes into the service's memory, which every job shares: one that would take more than `WAI_QSVC_READ_ANSWER_MIB` (128) once read, about 80 bytes a value, is refused in the child before it is written (`422`, `J031`), and the service reads it off its request threads. A dry run gets back the plans and points alone. A check's answer can be that large: a critique of a program of 200,000 gates that each draw a remark (`qubit q; for int i in [1:200000] { x q; }`) is refused so; one of 150,000 is answered. -- **What ended jobs keep.** A job keeps its request as the canonical text the store keeps, and a set lets go of its items once it ends. What ended jobs keep in memory, their requests' text and their results as RFC 8785 text, is counted in two parts. Results that have been read (or ended more than `WAI_QSVC_KEPT_MIN_SECS`, 600 s, ago) are held to `WAI_QSVC_CLIENT_KEPT_MIB` (128) a client, its oldest dropped first, and `WAI_QSVC_KEPT_MIB` (4096) in all, those of the client keeping the most dropped first (between two keeping as much, the newer); only these are ever dropped before `WAI_QSVC_RETAIN_SECS`. Results not yet read are dropped by neither cap: a client's next job is refused, `429`, while its unread results, with each job it has running at its result's bound and the new one at `WAI_QSVC_RESULT_MIB` (64), would pass `WAI_QSVC_CLIENT_UNREAD_MIB` (256), and every next job is refused, `503`, while all unread results hold `WAI_QSVC_UNREAD_MIB` (8192). A job whose result's bound passes `WAI_QSVC_RESULT_MIB` is refused before it runs (see [Sets and sweeps](#sets-and-sweeps)). The bounds that remain: memory for ended jobs stays under the two caps, 12 GiB at the defaults, with the results of jobs still running; reaching the unread cap takes 32 clients each holding its whole allowance unread, and holds every next job for at most 600 s; a client is its address, so people behind one address share its allowances, and one of them who leaves 256 MiB unread holds the others' next jobs until it is read or 600 s pass; and an honest client's read result is dropped early only where it, of all clients, keeps the most. A request's body is parsed only once its client's window has room (or, for a resend under a key the client holds a job under, first, as it is answered before any limit). A receipt check has a window of its own, `WAI_QSVC_CLIENT_VERIFIES` (120), asked before its body is read. An MCP message over 64 KiB from a client past its window is read only for its id, method, tool and key, and answered under its own id with a tool result that says why; a resend under a key the client holds, and a receipt check, are read whole. A job's report is one RFC 8785 text on every route (a poll, its events, a cancel, and in the MCP job tools' answers, written with RFC 8785's numbers, the job's own bytes, with each JSON-RPC id echoed exactly) and after a restart: its request and its result are written in as the text each is kept as, never parsed again, and the memory a large request or answer was parsed into is given back to the system once it is answered. On macOS the service starts itself with the allocator's large-block cache off (`MallocLargeCache=0`): that cache kept every large block a parsed request had freed, dirty and charged to the service (on arm64, 30 requests of a megabyte kept 1 GiB; with it off, 2 MiB). +- **What ended jobs keep.** A job keeps its request as the canonical text the store keeps, and a set lets go of its items once it ends. What ended jobs keep in memory, their requests' text and their results as RFC 8785 text, is counted in two parts. Results that have been read (or ended more than `WAI_QSVC_KEPT_MIN_SECS`, 600 s, ago) are held to `WAI_QSVC_CLIENT_KEPT_MIB` (128) a client, its oldest dropped first, and `WAI_QSVC_KEPT_MIB` (4096) in all, those of the client keeping the most dropped first (between two keeping as much, the newer); only these are ever dropped before `WAI_QSVC_RETAIN_SECS`. Results not yet read are dropped by neither cap: a client's next job is refused, `429`, while its unread results, with each job it has running at its result's bound and the new one at `WAI_QSVC_RESULT_MIB` (64), would pass `WAI_QSVC_CLIENT_UNREAD_MIB` (256), and every next job is refused, `503`, while all unread results, with every job queued or running at its result's bound and the new one, would pass `WAI_QSVC_UNREAD_MIB` (8192). A job whose result's bound passes `WAI_QSVC_RESULT_MIB` is refused before it runs (see [Sets and sweeps](#sets-and-sweeps)). The bounds that remain: memory for ended jobs stays under the two caps, 12 GiB at the defaults, the results of jobs still queued or running counted within the unread cap at their bounds; reaching the unread cap takes 32 clients each holding its whole allowance unread, and holds every next job for at most 600 s; a client is its address, so people behind one address share its allowances, and one of them who leaves 256 MiB unread holds the others' next jobs until it is read or 600 s pass; and an honest client's read result is dropped early only where it, of all clients, keeps the most. A request's body is parsed only once its client's window has room (or, for a resend under a key the client holds a job under, first, as it is answered before any limit). A receipt check has a window of its own, `WAI_QSVC_CLIENT_VERIFIES` (120), asked before its body is read. An MCP message over 64 KiB from a client past its window is read only for its id, method, tool and key, and answered under its own id with a tool result that says why; a resend under a key the client holds, and a receipt check, are read whole. A job's report is one RFC 8785 text on every route (a poll, its events, a cancel, and in the MCP job tools' answers, written with RFC 8785's numbers, the job's own bytes, with each JSON-RPC id echoed as the text it was sent as: `1.0`, `1e3` and an integer past 64 bits as written) and after a restart: its request and its result are written in as the text each is kept as, never parsed again, and the memory a large request or answer was parsed into is given back to the system once it is answered. On macOS the service starts itself with the allocator's large-block cache off (`MallocLargeCache=0`): that cache kept every large block a parsed request had freed, dirty and charged to the service (on arm64, 30 requests of a megabyte kept 1 GiB; with it off, 2 MiB). - **Ending.** A child that panics, overflows its stack or aborts is answered `500`, and the service runs on. A client that hangs up ends its child. -- **Connections.** A request whose header section takes longer than `WAI_QSVC_HEADER_SECS` (150) to arrive is closed; the same limit closes a connection left idle between requests, so it is set above a proxy's own idle limit (two minutes, commonly), and the proxy, not the service, closes its idle connections. A body that takes longer than `WAI_QSVC_BODY_SECS` (30) is refused, `408`. A client may have `WAI_QSVC_CLIENT_REQUESTS` (32) requests open at once; past it a request is refused, `503`, before its body is read (a health check is answered whatever that is). Long polls and event streams are held to `WAI_QSVC_CLIENT_STREAMS` (4) a client and `WAI_QSVC_STREAMS` (512, at most a quarter of the connections) in all: past them a long poll is answered at once with the job as it is, and an event stream is refused, `503`. Four connections past the connection cap are held for health checks alone, one request each, for 2 s. The service raises its open-file limit to its hard limit as it starts (on macOS no further than `kern.maxfilesperproc` and 10240), says so in its log, and holds at most `WAI_QSVC_CONNECTIONS` (4096) connections, fewer where that limit leaves less room: past them the connection idle longest (or the one whose body has been arriving longest) is closed for the new one, so a health check is always answered, and with none to close a new connection is answered `503` at once. +- **Connections.** A request whose header section takes longer than `WAI_QSVC_HEADER_SECS` (150) to arrive is closed; the same limit closes a connection left idle between requests, so it is set above a proxy's own idle limit (two minutes, commonly), and the proxy, not the service, closes its idle connections. A body that takes longer than `WAI_QSVC_BODY_SECS` (30) is refused, `408`. A client may have `WAI_QSVC_CLIENT_REQUESTS` (32) requests open at once; past it a request is refused, `503`, before its body is read (a health check is answered whatever that is). Long polls and event streams are held to `WAI_QSVC_CLIENT_STREAMS` (4) a client and `WAI_QSVC_STREAMS` (512, at most a quarter of the connections) in all: past them a long poll is answered at once, `429` (its client's) or `503` (everyone's), `J032`, with `Retry-After: 2` and the job's status, and an event stream is refused, `503`. Four connections past the connection cap are held for health checks alone, one request each, for 2 s. The service raises its open-file limit to its hard limit as it starts (on macOS no further than `kern.maxfilesperproc` and 10240), says so in its log, and holds at most `WAI_QSVC_CONNECTIONS` (4096) connections, fewer where that limit leaves less room: past them the connection idle longest (or the one whose body has been arriving longest) is closed for the new one, so a health check is always answered, and with none to close a new connection is answered `503` at once. - **Updating.** Readers and workers start from the service's own binary as it was started: on Linux its running image, elsewhere a copy it keeps beside its signing key, named by its digest. So the binary can be replaced before the restart (by a copy over it, or, on Linux, where a running binary cannot be written, by a rename), and the running service goes on with its own until it is restarted. ## Sending a job again diff --git a/wai/wai-quantum-service/schemas/service-tools.v1.json b/wai/wai-quantum-service/schemas/service-tools.v1.json index 1b198eb5e..83a355d02 100644 --- a/wai/wai-quantum-service/schemas/service-tools.v1.json +++ b/wai/wai-quantum-service/schemas/service-tools.v1.json @@ -304,7 +304,7 @@ }, "accepted": { "type": "boolean", - "description": "Whether the service would take it now, by every limit." + "description": "Whether the service would take it now, by every limit, its reading held to what the submission's would take; the MCP tools plan it as their runs run it, its verdict asked." }, "status": { "type": "integer", diff --git a/wai/wai-quantum-service/src/discovery.rs b/wai/wai-quantum-service/src/discovery.rs index f260b35bc..77081f1fd 100644 --- a/wai/wai-quantum-service/src/discovery.rs +++ b/wai/wai-quantum-service/src/discovery.rs @@ -139,7 +139,7 @@ pub fn openapi(server: &str, result_bytes: u64) -> Value { "intent": { "description": "What the program must do, checked with the run: one intent or a list, each {\"distribution\": {\"00\": 0.5, \"11\": 0.5}, \"tolerance\"}, {\"equivalent\": \"\"} or {\"prepares\": \"\"}, with an optional note and \"require\": \"proof\". The result's `verdict` states the level the engine checked to (proof, evidence, executed, parsed, refused) and whether the intents hold, and the receipt seals it." }, "verdict": { "type": "boolean", "description": "true: the result carries the verdict with no intent (how far the engine checked)." }, "repair": { "description": "true, or the choices {\"max_iterations\", \"dyadic_bits\", \"sizes\"}: the program is repaired first on the lines its diagnostics flag (as POST /v1/repair does), and the result's `repaired` holds the program that ran and each repair." }, - "programs": { "type": "array", "minItems": 1, "maxItems": wai_quantum::engine::sets::MAX_ITEMS, "items": { "oneOf": [{ "type": "string" }, { "type": "object", "required": ["qasm"] }] }, "description": format!("A set: in place of qasm, the programs to run as one job, each a program's text or a job whose own fields win over the set's. Every other field applies to each item. The result is a set (SetResult). A job or set whose result could take more than {result_bytes} bytes as served (result_bytes), by its bound before it runs (each item's counts at most its shots, 2^bits or 65,536 outcomes, and its other parts as read), is refused at once (422, J031): at about 20 bytes and a count a 16-bit outcome, 100 items of 65,536 outcomes each is about 145 MB.") }, + "programs": { "type": "array", "minItems": 1, "maxItems": wai_quantum::engine::sets::MAX_ITEMS, "items": { "oneOf": [{ "type": "string" }, { "type": "object", "required": ["qasm"] }] }, "description": format!("A set: in place of qasm, the programs to run as one job, each a program's text or a job whose own fields win over the set's. Every other field applies to each item. The result is a set (SetResult). A job or set whose result could take more than {result_bytes} bytes as served (result_bytes), by its bound before it runs (each item's counts at most its shots or a trajectories run's trajectories, the outcomes its measured bits can give and 65,536 outcomes; a verdict's critique by the remarks its gates can draw; its other parts at their length as read, every number as RFC 8785 writes it), is refused at once (422, J031), naming its largest part: at about 22 bytes a 16-bit outcome, 100 items of 65,536 outcomes each is about 150 MB.") }, "sweep": { "type": "object", "additionalProperties": { "type": "array", "minItems": 1, "items": { "type": "number" } }, "description": "A set over qasm's inputs: each input's values, every combination run (the names in sorted order, the first varying slowest). An integer input (`input uint n;`) takes whole numbers and the program is written out at each; a float or angle input (`input float theta;`) is bound to the double given. Every input the program declares is given values." }, "dry_run": { "type": "boolean", "description": "true: run nothing, and answer 200 with the plan (DryRun): the method, memory, estimated time and energy with their basis, the bounds, and whether the service would take it now." }, "pulse": wai_quantum::engine::pulse::options_schema(), @@ -152,7 +152,7 @@ pub fn openapi(server: &str, result_bytes: u64) -> Value { "properties": { "dry_run": { "const": true }, "ran": { "const": false }, - "accepted": { "type": "boolean", "description": "Whether the service would take it now, by every limit." }, + "accepted": { "type": "boolean", "description": "Whether the service would take it now, by every limit, its reading held to what the submission's would take; the MCP tools plan it as their runs run it, its verdict asked." }, "status": { "type": "integer", "description": "With accepted false: the HTTP status the submission would get." }, "refusal": { "type": "object", "description": "With accepted false: the refusal as data (message, code, codes, outcome, meaning)." }, "method": { "type": "string" }, "why": { "type": "string" }, "qubits": { "type": "integer" }, @@ -275,8 +275,8 @@ pub fn openapi(server: &str, result_bytes: u64) -> Value { "parameters": [{ "name": "id", "in": "path", "required": true, "schema": { "type": "string" } }], "get": { "summary": "A job: its status, and its result when done. Kept across the service's restarts for its retention", - "parameters": [{ "name": "wait", "in": "query", "required": false, "schema": { "type": "integer", "minimum": 1, "maximum": 25 }, "description": "Hold the answer until the job's status changes, or this many seconds pass." }], - "responses": { "200": { "description": "The job", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Job" } } } }, "404": { "description": "No such job" } } + "parameters": [{ "name": "wait", "in": "query", "required": false, "schema": { "type": "integer", "minimum": 1, "maximum": 25 }, "description": "Hold the answer until the job's status changes, or this many seconds pass. A client holds at most WAI_QSVC_CLIENT_STREAMS (4) long polls and event streams at once: past them a poll of a job still queued or running is not held, and is answered 429 (503 past everyone's) with Retry-After; wait that long, or poll without wait." }], + "responses": { "200": { "description": "The job", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Job" } } } }, "404": { "description": "No such job" }, "429": { "description": "A long poll past its client's holds, not held (J032): {\"error\", \"code\", \"id\", \"status\"}, with Retry-After" }, "503": { "description": "A long poll past everyone's holds, not held (J032): {\"error\", \"code\", \"id\", \"status\"}, with Retry-After" } } }, "delete": { "summary": "Cancel a queued or running job", "responses": { "200": { "description": "Cancelled" }, "409": { "description": "Already finished" } } } }, diff --git a/wai/wai-quantum-service/src/http.rs b/wai/wai-quantum-service/src/http.rs index bf75b66f8..97334ec7f 100644 --- a/wai/wai-quantum-service/src/http.rs +++ b/wai/wai-quantum-service/src/http.rs @@ -142,11 +142,9 @@ fn reply(status: u16, body: Value) -> Response { r } -/// [`reply`], its body JSON text already written. /// `v` as JSON text, its keys in their order and each fractional number as /// RFC 8785 writes it (an integer exactly, as read): a job read back from its -/// report, which is RFC 8785, is written as the same bytes, and a JSON-RPC id -/// is echoed exactly, past 2^53 too. +/// report, which is RFC 8785, is written as the same bytes. pub fn canonical_text(v: &Value) -> String { struct Es; impl serde_json::ser::Formatter for Es { @@ -165,6 +163,36 @@ pub fn canonical_text(v: &Value) -> String { } } +/// The text of a JSON-RPC message's `id` as it was sent, where it has one: +/// `1.0`, `1e3` and an integer past 64 bits as written, which a number read +/// and written again would not keep. +fn raw_id(body: &[u8]) -> Option { + #[derive(serde::Deserialize)] + struct Message<'a> { + #[serde(borrow)] + id: Option<&'a serde_json::value::RawValue>, + } + let m: Message<'_> = serde_json::from_slice(body).ok()?; + m.id.map(|r| r.get().to_string()) +} + +/// An MCP answer as text ([`canonical_text`]), its `id` the request's exactly +/// as it was sent (`raw`, [`raw_id`]): each JSON-RPC id is echoed as its +/// text, whatever number it is. The id is written in its place through a +/// mark no answer can hold (random, made for this answer). +fn mcp_text(mut answer: Value, raw: Option<&str>) -> String { + let Some(raw) = raw.filter(|_| answer.get("id").is_some_and(|i| !i.is_null())) else { + return canonical_text(&answer); + }; + let mut b = [0u8; 16]; + if getrandom::fill(&mut b).is_err() { + return canonical_text(&answer); + } + let mark = format!("wai-mcp-id-{}", b.iter().map(|x| format!("{x:02x}")).collect::()); + answer["id"] = json!(mark); + canonical_text(&answer).replacen(&format!("\"{mark}\""), raw, 1) +} + fn reply_text(status: u16, body: String) -> Response { let mut r = (StatusCode::from_u16(status).unwrap_or(StatusCode::INTERNAL_SERVER_ERROR), [("content-type", "application/json")], body).into_response(); r.headers_mut().insert("access-control-allow-origin", HeaderValue::from_static("*")); @@ -384,22 +412,11 @@ const ITEM_BYTES: u64 = 4 << 10; /// A set's head as served: its format, counts, root, energy and receipt. const SET_HEAD_BYTES: u64 = 4 << 10; -/// `v`'s length as JSON text, written nowhere. RFC 8785 writes the same -/// characters but for a number's form, which a sixteenth more allows for. +/// `v`'s length as RFC 8785 text, written nowhere, each number in its own +/// form ([`wai_quantum::engine::receipt::canonical_len`]): what a set serves +/// of each item's request. fn json_len(v: &Value) -> u64 { - struct Count(u64); - impl std::io::Write for Count { - fn write(&mut self, b: &[u8]) -> std::io::Result { - self.0 += b.len() as u64; - Ok(b.len()) - } - fn flush(&mut self) -> std::io::Result<()> { - Ok(()) - } - } - let mut c = Count(0); - let _ = serde_json::to_writer(&mut c, v); - c.0 + c.0 / 16 + wai_quantum::engine::receipt::canonical_len(v) } /// `request` planned: one job's plan, or a set's plan with its items' (each @@ -409,7 +426,7 @@ fn json_len(v: &Value) -> u64 { /// first; then every program the request gives is read, once, within one /// budget ([`READ_STEPS`]). A set's result serves each item's request, its /// result and its own fields, and the set's head. -pub(crate) fn plan_job(cap: u64, request: &Value) -> Result<(catalog::Plan, Work, Option), (u16, String)> { +pub(crate) fn plan_job(cap: u64, request: &Value) -> Result<(catalog::Plan, Work, Option), (u16, String)> { cheap_bounds(request)?; within_budget(|| { if catalog::is_set(request) { @@ -418,16 +435,17 @@ pub(crate) fn plan_job(cap: u64, request: &Value) -> Result<(catalog::Plan, Work job_bounds(&it.request).map_err(|(c, e)| (c, format!("item {i}: {e}")))?; } let planned = sets::plan_bounded(&items, cap).map_err(|e| (400, e))?; - let bound = planned.iter().zip(&items).try_fold(SET_HEAD_BYTES, |all, ((p, result), it)| { - let item = ITEM_BYTES + json_len(&it.request) + json_len(&json!(p.why)) + it.point.as_ref().map_or(0, json_len); - Some(all.saturating_add((*result)?).saturating_add(item)) + let head = catalog::Bound { rest: SET_HEAD_BYTES, ..Default::default() }; + let bound = planned.iter().zip(&items).try_fold(head, |all, ((p, result), it)| { + let item = catalog::Bound { requests: json_len(&it.request), rest: ITEM_BYTES + json_len(&json!(p.why)) + it.point.as_ref().map_or(0, json_len), ..Default::default() }; + Some(all.plus((*result)?).plus(item)) }); let plans: Vec = planned.into_iter().map(|(p, _)| p).collect(); return Ok((sets::summary(&plans), Work::Set { items, plans }, bound)); } job_bounds(request)?; - let (plan, bound) = catalog::plan_bounded(request, cap).map_err(|e| (400, e))?; - Ok((plan, Work::One, bound.map(|b| b.saturating_add(HOST_BYTES)))) + let (plan, bound) = catalog::plan_bound(request, cap).map_err(|e| (400, e))?; + Ok((plan, Work::One, bound.map(|b| b.plus(catalog::Bound { rest: HOST_BYTES, ..Default::default() })))) }) } @@ -441,7 +459,7 @@ pub(crate) async fn admit(app: &App, who: &str, request: Value, key: Option), (u16, String)>); +type Admitted = (Value, Result<(catalog::Plan, Work, Option), (u16, String)>); fn admitted(mut v: Value) -> Result { let bad = |e: serde_json::Error| (500, format!("the reading's answer did not read: {e}")); @@ -452,7 +470,7 @@ fn admitted(mut v: Value) -> Result { return Ok((request, Err((r.first().and_then(Value::as_u64).unwrap_or(400) as u16, r.get(1).and_then(Value::as_str).unwrap_or("refused").to_string())))); } let plan: catalog::Plan = serde_json::from_value(take("plan")).map_err(bad)?; - let bound = take("result_bytes").as_u64(); + let bound: Option = serde_json::from_value(take("result_parts")).map_err(bad)?; let work = match take("items") { Value::Null => Work::One, items => Work::Set { items: crate::jobs::items_from(items), plans: serde_json::from_value(take("plans")).map_err(bad)? }, @@ -540,8 +558,14 @@ pub(crate) async fn dry_run(app: &App, who: &str, request: &Value) -> Result dry_run_now(service, who, admitted(read)?), + Err((422, why)) => dry_run_now(service, who, (Value::Null, Err((422, why)))), + Err(e) => Err(e), + } } /// [`dry_run`]'s answer, its submission counted and its request read. @@ -611,7 +635,7 @@ fn dry_run_now(service: &Service, who: &str, (_, planned): Admitted) -> Result json!(b), + Some(b) => json!(b.bytes()), None => json!({ "bytes": null, "why": "its method gives no bound of its result before it runs (no built-in method is such): it is counted at the most a job's result may take, and held to it as it ends" }), }; if let Some(t) = &plan.target { @@ -716,16 +740,38 @@ struct Poll { wait: Option, } +/// How long a long poll past the holds is told to wait before it polls +/// again, seconds ([`job`]). +pub const POLL_AGAIN: u64 = 2; + /// A job's report; with `?wait=S` (at most 25 s), held until its status -/// changes from what it was, or the wait passes. +/// changes from what it was, or the wait passes. Past its client's holds (or +/// everyone's), a long poll of a job still queued or running is answered at +/// once `429` (its client's) or `503` (everyone's), `J032`, with +/// `Retry-After` and the job's status: a poll that is not held says so, and +/// when to poll again, so a client that polls in a loop waits. async fn job(State(app): State, ConnectInfo(peer): ConnectInfo, headers: HeaderMap, Path(id): Path, Query(poll): Query) -> Response { let Some(first) = app.service.status(&id) else { return error(404, &format!("no such job (finished jobs are kept for {} s)", app.service.config.retain.as_secs())); }; - // Held only while its client's and everyone's long polls leave room; - // past them it is answered at once with what the job is now, as a long - // poll may always be. - let held = poll.wait.filter(|&w| w > 0 && first.active()).and_then(|w| Some((w, app.service.hold_stream(&client(&app, peer, &headers))?))); + let held = match poll.wait.filter(|&w| w > 0 && first.active()) { + None => None, + Some(w) => match app.service.hold_stream_or(&client(&app, peer, &headers)) { + Ok(h) => Some((w, h)), + Err(mine) => { + let why = if mine { + format!("your {} long polls and event streams are held, all one client may hold; this poll was not held: poll again after {POLL_AGAIN} s, or without ?wait", app.service.config.client_streams) + } else { + format!("as many long polls and event streams are held as the service may; this poll was not held: poll again after {POLL_AGAIN} s, or without ?wait") + }; + let why = tag(refusal::BUSY, why); + let r = wai_quantum::engine::refusal::refusal_json(&why); + let mut resp = reply(if mine { 429 } else { 503 }, json!({ "error": why, "code": r["code"], "codes": r["codes"], "id": id, "status": first.name() })); + resp.headers_mut().insert("retry-after", HeaderValue::from(POLL_AGAIN)); + return resp; + } + }, + }; if let Some((w, _held)) = held { let mut rx = app.service.subscribe(); let deadline = tokio::time::Instant::now() + Duration::from_secs(w).min(POLL_WAIT); @@ -1029,11 +1075,13 @@ async fn mcp(State(app): State, ConnectInfo(peer): ConnectInfo MCP_UNREAD && let Err(Refused(_, why)) = app.service.window_open(&who) && let Some(answer) = crate::mcp::past_window(&app, &who, &body, &why) { - return reply_text(200, canonical_text(&answer)); + return reply_text(200, mcp_text(answer, raw.as_deref())); } let Ok(msg) = serde_json::from_slice::(&body) else { return reply(400, json!({ "jsonrpc": "2.0", "id": null, "error": { "code": -32700, "message": "not JSON" } })); @@ -1059,7 +1107,7 @@ async fn mcp(State(app): State, ConnectInfo(peer): ConnectInfo(Event::default().data(canonical_text(&a))) + Ok::(Event::default().data(mcp_text(a, raw.as_deref()))) }); let mut r = Sse::new(answer).keep_alive(KeepAlive::new().interval(Duration::from_secs(15))).into_response(); r.headers_mut().insert("access-control-allow-origin", HeaderValue::from_static("*")); @@ -1068,8 +1116,9 @@ async fn mcp(State(app): State, ConnectInfo(peer): ConnectInfo { - // Written as RFC 8785, so a job in an answer has its report's bytes. - let mut r = reply_text(200, canonical_text(&answer)); + // Written as RFC 8785, so a job in an answer has its report's + // bytes, and its id as it was sent. + let mut r = reply_text(200, mcp_text(answer, raw.as_deref())); if opening && let Ok(id) = HeaderValue::from_str(&crate::mcp::views_session()) { r.headers_mut().insert("mcp-session-id", id); r.headers_mut().insert("access-control-expose-headers", HeaderValue::from_static("mcp-session-id")); @@ -1136,13 +1185,16 @@ fn rate_headers(app: &App, who: &str, r: &mut Response) { if let Ok(v) = HeaderValue::from_str(&format!("\"submissions\";r={};t={}", w.remaining, w.reset_secs)) { h.insert("ratelimit", v); } - // Sent again at this pace, a refusal that counts never spends the window. + // Sent again at this pace, a refusal that counts never spends the window; + // a handler that says when to come back (a long poll not held) is kept. let pace = (window / w.limit.max(1) as u64).max(5); - if r.status() == StatusCode::TOO_MANY_REQUESTS + let said = r.headers().contains_key("retry-after"); + if !said + && r.status() == StatusCode::TOO_MANY_REQUESTS && let Ok(v) = HeaderValue::from_str(&(if w.remaining == 0 { w.reset_secs.max(1) } else { pace }).to_string()) { r.headers_mut().insert("retry-after", v); - } else if r.status() == StatusCode::SERVICE_UNAVAILABLE { + } else if !said && r.status() == StatusCode::SERVICE_UNAVAILABLE { r.headers_mut().insert("retry-after", HeaderValue::from_static("5")); } } diff --git a/wai/wai-quantum-service/src/jobs.rs b/wai/wai-quantum-service/src/jobs.rs index b62d0f73f..7389227c4 100644 --- a/wai/wai-quantum-service/src/jobs.rs +++ b/wai/wai-quantum-service/src/jobs.rs @@ -52,7 +52,7 @@ //! job the service interrupted: that key runs the request again. A record //! keeps the key's digest with the client's name, never the key. -use crate::catalog::Plan; +use crate::catalog::{Bound, Plan}; use crate::energy; use crate::store::{self, Store}; use serde_json::{Value, json}; @@ -1013,14 +1013,23 @@ impl Service { /// past [`Service::stream_cap`] of everyone's, so they cannot hold every /// connection the service has. pub fn hold_stream(&self, client: &str) -> Option { + self.hold_stream_or(client).ok() + } + + /// [`Service::hold_stream`], or whose holds are full: the client's own + /// (`true`), or everyone's (`false`). + pub fn hold_stream_or(&self, client: &str) -> Result { let mut held = self.streams.lock().unwrap(); let all: usize = held.values().sum(); let n = held.entry(client.to_string()).or_default(); - if *n >= self.config.client_streams || all >= self.stream_cap.load(Ordering::SeqCst) { - return None; + if *n >= self.config.client_streams { + return Err(true); + } + if all >= self.stream_cap.load(Ordering::SeqCst) { + return Err(false); } *n += 1; - Some(OpenRequest { open: Arc::clone(&self.streams), client: client.to_string() }) + Ok(OpenRequest { open: Arc::clone(&self.streams), client: client.to_string() }) } /// Whether `client` holds a job under the idempotency key `key`: a @@ -1077,15 +1086,40 @@ impl Service { jobs.values().filter(|j| !j.read && j.finished_ms.is_some_and(|f| now.saturating_sub(f) < min)).fold((0, 0), |(m, a), j| (m + if j.client == name { j.kept } else { 0 }, a + j.kept)) } - /// What the results of `name`'s jobs still running may hold once they - /// end, bytes: each its bound ([`Job::bound`]), one not known at the - /// most a job's result may take ([`Config::result_mib`]). + /// What the results of `name`'s jobs still queued or running may hold + /// once they end, bytes: each its bound ([`Job::bound`]), one not known + /// at the most a job's result may take ([`Config::result_mib`]). fn running_results(&self, name: &str) -> u64 { let cap = self.config.result_mib << 20; let jobs = self.jobs.lock().unwrap(); jobs.values().filter(|j| j.client == name && j.status.active()).map(|j| j.bound.min(cap)).sum() } + /// What every result not yet read may hold, bytes: those of ended jobs + /// as they are kept ([`Service::unread`]), and those of every job still + /// queued or running at its bound, as [`Service::running_results`] + /// counts a client's. What the service-wide cap on unread results holds + /// ([`Config::unread_mib`]): a job is taken only while this and its own + /// bound stay under it, so ended jobs' unread results never pass it. + fn unread_all(&self) -> u64 { + let cap = self.config.result_mib << 20; + let now = now_ms(); + let min = self.config.kept_min.as_millis() as u64; + let jobs = self.jobs.lock().unwrap(); + jobs.values() + .map(|j| if j.status.active() { j.bound.min(cap) } else if !j.read && j.finished_ms.is_some_and(|f| now.saturating_sub(f) < min) { j.kept } else { 0 }) + .sum() + } + + /// The service-wide unread cap's refusal of a job whose result may take + /// `bound` bytes, if it is past the cap ([`Service::unread_all`]). + fn past_unread_cap(&self, bound: u64) -> Option { + let all = self.unread_all(); + (all.saturating_add(bound) > self.config.unread_mib << 20).then(|| { + Refused(503, tag(refusal::BUSY, format!("results not yet read, with those of the jobs queued and running, could hold about {} MiB, and this one {:.1} MiB more; all the service keeps unread is {} MiB: try again in a few minutes", all >> 20, bound as f64 / f64::from(1 << 20), self.config.unread_mib))) + }) + } + /// The client's active jobs and the memory they hold, MiB. fn held(&self, name: &str) -> (usize, u64) { let jobs = self.jobs.lock().unwrap(); @@ -1097,7 +1131,7 @@ impl Service { /// its jobs at once, or a full queue. Checked before a submission is /// counted, so a client told to try again shortly spends nothing doing so. pub fn refused_before_reading(&self, client: &str) -> Option { - let (mine, all) = self.unread(&self.name(client)); + let (mine, _) = self.unread(&self.name(client)); // Each job still running counted at its bound, and this one, not yet // read, at the most a job's result may take. let room = mine + self.running_results(&self.name(client)) + (self.config.result_mib << 20); @@ -1105,8 +1139,9 @@ impl Service { let room = room.div_ceil(1 << 20); return Some(Refused(429, tag(refusal::RATE, format!("results of yours not yet read hold about {} MiB, and with your jobs running and this one they could hold {room} MiB, more than the {} MiB one client's unread results may; read them (GET /v1/jobs/{{id}}) before sending more, or wait {} s", mine >> 20, self.config.client_unread_mib, self.config.kept_min.as_secs())))); } - if all >= self.config.unread_mib << 20 { - return Some(Refused(503, tag(refusal::BUSY, format!("results not yet read hold about {} MiB, all the service keeps unread; try again in a few minutes", all >> 20)))); + // Every result not yet read, and this one at the most a job's may take. + if let Some(r) = self.past_unread_cap(self.config.result_mib << 20) { + return Some(r); } let (active, held) = self.held(&self.name(client)); if active >= self.config.per_client_active { @@ -1126,14 +1161,21 @@ impl Service { /// limit but its submission window: the refusal it would get, or nothing. /// `bound` is the most its result takes as served (`http::plan_job`): /// past [`Config::result_mib`] it is refused at once, before it runs. - pub fn would_refuse(&self, client: &str, plan: &Plan, bound: Option) -> Option { + pub fn would_refuse(&self, client: &str, plan: &Plan, bound: Option) -> Option { let job_mib = plan.memory_bytes.div_ceil(1 << 20); if job_mib > self.config.job_mib { return Some(Refused(422, tag(refusal::TOO_LARGE_HERE, format!("the {} method would need about {} MiB for this circuit; one job may take at most {} MiB", plan.method, job_mib, self.config.job_mib)))); } let cap = self.config.result_mib << 20; - if let Some(b) = bound.filter(|&b| b > cap) { - return Some(Refused(422, tag(refusal::TOO_LARGE_HERE, format!("its result could take up to {:.1} MiB as served (its counts listing up to the shots, 2^bits or 65,536 outcomes an item, for every item), more than the {} MiB one job's result here may take; send fewer points, fewer shots or fewer measured bits, split it into several jobs, or run it on your own machine (`wai-quantum run`)", b as f64 / f64::from(1 << 20), self.config.result_mib)))); + let mib = |b: u64| b as f64 / f64::from(1 << 20); + if let Some(b) = bound.filter(|b| b.bytes() > cap) { + // Named by its largest part, and what makes that part smaller. + let (part, what, fix) = b.largest(); + return Some(Refused(422, tag(refusal::TOO_LARGE_HERE, format!("its result could take up to {:.1} MiB as served, {:.1} MiB of it {what}: more than the {} MiB one job's result here may take; {fix}, split it into several jobs, or run it on your own machine (`wai-quantum run`)", mib(b.bytes()), mib(part), self.config.result_mib)))); + } + let bound = bound.map(|b| b.bytes()); + if let Some(r) = self.past_unread_cap(bound.unwrap_or(cap).min(cap)) { + return Some(r); } let name = self.name(client); let (mine, _) = self.unread(&name); @@ -1207,12 +1249,12 @@ impl Service { /// not the request as it read, and `bound` the most its result takes as /// served (`http::plan_job`). #[allow(clippy::too_many_arguments)] - pub fn submit_counted(&self, client: &str, request: Value, plan: Plan, work: Work, key: Option, received: Option, bound: Option) -> Result { + pub fn submit_counted(&self, client: &str, request: Value, plan: Plan, work: Work, key: Option, received: Option, bound: Option) -> Result { self.keep_work(client, request, plan, work, key, received, bound, false) } #[allow(clippy::too_many_arguments)] - fn keep_work(&self, client: &str, request: Value, plan: Plan, work: Work, key: Option, received: Option, bound: Option, count: bool) -> Result { + fn keep_work(&self, client: &str, request: Value, plan: Plan, work: Work, key: Option, received: Option, bound: Option, count: bool) -> Result { let name = self.name(client); let canonical = canon::canonical(&request); let request_digest = store::digest(&canonical); @@ -1265,17 +1307,22 @@ impl Service { interruptions: Vec::new(), child: None, kept: 0, read: false, - bound: bound.unwrap_or(u64::MAX), + bound: bound.map_or(u64::MAX, |b| b.bytes()), }; { + // Under the queue's lock, so no two jobs pass the caps on one + // count: the queue's, and every unread result's with this one's. let mut queue = self.queue.lock().unwrap(); if queue.len() >= self.config.max_queue { return Err(Refused(503, tag(refusal::BUSY, format!("the queue is full ({} jobs); try again later", self.config.max_queue)))); } + if let Some(r) = self.past_unread_cap(job.bound.min(self.config.result_mib << 20)) { + return Err(r); + } self.store.put_record(&id, &job.record()).map_err(unkept)?; + self.jobs.lock().unwrap().insert(id.clone(), job); queue.push_back(id.clone()); } - self.jobs.lock().unwrap().insert(id.clone(), job); if let (Some(keys), Some(k)) = (keys.as_mut(), key) { keys.insert(k, id.clone()); } @@ -2188,14 +2235,14 @@ mod tests { fn a_result_s_bound_is_held_to_the_cap_and_running_jobs_count_their_own() { let s = Service::new(Config { result_mib: 1, client_unread_mib: 4, per_client_active: 8, max_queue: 16, ..config() }); let p = plan(1 << 20); - let Refused(code, why) = s.would_refuse("a", &p, Some((1 << 20) + 1)).expect("past the cap"); + let Refused(code, why) = s.would_refuse("a", &p, Some(Bound { counts: (1 << 20) + 1, ..Bound::default() })).expect("past the cap"); assert!(code == 422 && why.contains("J031") && why.contains("1.0 MiB") && why.contains("the 1 MiB"), "{code} {why}"); - assert!(s.would_refuse("a", &p, Some(1 << 20)).is_none()); + assert!(s.would_refuse("a", &p, Some(Bound { counts: 1 << 20, ..Bound::default() })).is_none()); for k in 0..7 { - s.keep_work("a", json!({ "qasm": "OPENQASM 3.0; qubit q; h q;", "shots": k + 1 }), p.clone(), Work::One, None, None, Some(4096), false).unwrap(); + s.keep_work("a", json!({ "qasm": "OPENQASM 3.0; qubit q; h q;", "shots": k + 1 }), p.clone(), Work::One, None, None, Some(Bound { rest: 4096, ..Bound::default() }), false).unwrap(); } assert!(s.refused_before_reading("a").is_none(), "seven small results and a next job's room fit in 4 MiB"); - assert!(s.would_refuse("a", &p, Some(4096)).is_none()); + assert!(s.would_refuse("a", &p, Some(Bound { rest: 4096, ..Bound::default() })).is_none()); // Not known (a job kept across a restart), a job running is counted at the cap. let t = Service::new(Config { result_mib: 1, client_unread_mib: 4, per_client_active: 8, max_queue: 16, ..config() }); for k in 0..3 { diff --git a/wai/wai-quantum-service/src/main.rs b/wai/wai-quantum-service/src/main.rs index 3ee295f74..b6dd988d7 100644 --- a/wai/wai-quantum-service/src/main.rs +++ b/wai/wai-quantum-service/src/main.rs @@ -108,11 +108,29 @@ fn worker() { let out = match serde_json::from_str::(&input) { Ok(v) if v.get("set").is_some() => { let pairs: Option> = v["set"].as_array().and_then(|a| a.iter().map(|it| Some((it.get("request")?.clone(), plan_of(&it["plan"])?))).collect()); + drop(v); match pairs { - Some(pairs) => serde_json::json!({ "items": sets::run_items(&pairs, HOST).into_iter().map(|o| match o { - Ok(r) => serde_json::json!({ "result": r }), - Err(e) => serde_json::json!({ "error": e }), - }).collect::>() }), + // Each item's result written out as it ends, and its tree + // dropped: a counted outcome takes about 21 bytes as text and + // ten times that as a tree, so a set of a hundred items held + // as trees passed the memory its largest item was admitted + // with. The set holds one item's tree at a time, and the text. + Some(pairs) => { + let mut text = String::from("{\"items\":["); + for (i, (request, plan)) in pairs.iter().enumerate() { + if i > 0 { + text.push(','); + } + let item = match run::run_with_provenance(request, plan, HOST) { + Ok(r) => serde_json::json!({ "result": r }), + Err(e) => serde_json::json!({ "error": e }), + }; + text.push_str(&item.to_string()); + } + text.push_str("]}"); + println!("{text}"); + return; + } None => serde_json::json!({ "error": "the set lacks an item's request or plan" }), } } diff --git a/wai/wai-quantum-service/src/mcp.rs b/wai/wai-quantum-service/src/mcp.rs index 63f800722..971a481d2 100644 --- a/wai/wai-quantum-service/src/mcp.rs +++ b/wai/wai-quantum-service/src/mcp.rs @@ -308,7 +308,7 @@ fn tool_list() -> Value { { "name": "dry_run_quantum_job", "title": "Plan a quantum job or set without running it", - "description": "What the service would do with a job or a set, running nothing: the method and why, the memory, the estimated time and energy (each with its basis: this service's own timings and readings where it has them, otherwise the planner's nominal rate, and no energy figure without readings), the bounds that apply, where you stand against your limits, and whether the service would take it now, or the refusal it would give, with its code. Takes what run_quantum_circuit or run_quantum_set takes. It counts as one submission.", + "description": "What the service would do with a job or a set, running nothing: the method and why, the memory, the estimated time and energy (each with its basis: this service's own timings and readings where it has them, otherwise the planner's nominal rate, and no energy figure without readings), the bounds that apply, where you stand against your limits, and whether the service would take it now, or the refusal it would give, with its code. Takes what run_quantum_circuit or run_quantum_set takes, and plans it as they run it, its verdict asked: what it says they take, they take. It counts as one submission.", "inputSchema": { "type": "object", "properties": plan_props }, "annotations": { "readOnlyHint": true } }, @@ -647,16 +647,22 @@ fn run_refusal(name: &str, args: &Value) -> Option { /// A run's job admitted as the run tools admit it, its verdict asked: its /// id, or the tool's refusal. It is admitted as `POST /v1/jobs` admits one /// (`http::admit`): counted before anything is read, and read in a child. -async fn admit_run(app: &App, who: &str, mut args: Value, key: Option) -> Result { +async fn admit_run(app: &App, who: &str, args: Value, key: Option) -> Result { let refused = |why: String| tool_result(format!("refused: {why}"), json!({ "error": why }), true); if !args.is_object() { return Err(refused(wai_quantum::engine::refusal::tag(wai_quantum::engine::refusal::BAD_FIELD, "a run's arguments are an object: {\"qasm\": …, \"shots\": …}"))); } - // Every run an agent asks for states its verdict. - if !verdict::asked(&args) { + crate::http::admit(app, who, as_run(args), key).await.map_err(|(_, why)| refused(why)) +} + +/// A run tool's arguments as the job it admits: every run an agent asks for +/// states its verdict. A dry run of the tools plans the same job +/// ([`dry_run_answer`]), so what it says is taken is taken. +fn as_run(mut args: Value) -> Value { + if args.is_object() && !verdict::asked(&args) { args["verdict"] = json!(true); } - crate::http::admit(app, who, args, key).await.map_err(|(_, why)| refused(why)) + args } /// A tool's answer as `tools/call` gives it: the call's, the verdict of @@ -1136,9 +1142,10 @@ async fn call(app: &App, who: &str, name: &str, args: Value, base: &str) -> Valu } } -/// A dry run's answer, as a tool gives it. +/// A dry run's answer, as a tool gives it: of the job the run tools would +/// admit ([`as_run`]), its verdict asked. async fn dry_run_answer(app: &App, who: &str, args: &Value) -> Value { - match crate::http::dry_run(app, who, args).await { + match crate::http::dry_run(app, who, &as_run(args.clone())).await { Ok(v) => { let secs = v["time"]["seconds"].as_f64().map(|t| format!("about {t:.3} s ({})", v["time"]["basis"].as_str().unwrap_or(""))).unwrap_or_else(|| format!("time not estimated ({})", v["time"]["why"].as_str().unwrap_or(""))); let energy = v["energy"]["joules_micro_estimate"].as_u64().map(|u| format!("about {:.3} J, an estimate", u as f64 / 1e6)).unwrap_or_else(|| format!("energy not estimated ({})", v["energy"]["unestimated"].as_str().unwrap_or(""))); diff --git a/wai/wai-quantum-service/src/offload.rs b/wai/wai-quantum-service/src/offload.rs index c8b7fee1d..fba393bc7 100644 --- a/wai/wai-quantum-service/src/offload.rs +++ b/wai/wai-quantum-service/src/offload.rs @@ -116,8 +116,12 @@ struct Share { turns: Arc, requests: usize, lent: usize, - /// Of its requests, those waiting for a turn. - waiting: usize, + /// Of its requests, those waiting for a turn: when each began to wait, + /// by a number of its own. + waiting: HashMap, + /// Of its requests, those holding one of its own turns while they wait + /// for the pool's: each will run on its own turn, so none is lent one. + between: usize, } /// The turns requests are read in: the service's two pools, each client's @@ -139,8 +143,11 @@ pub struct Gate { /// A turn, held until its request has been read; dropped, the next goes. pub struct Turn { - _pool: OwnedSemaphorePermit, + // Dropped in this order: the client's own turn first, so a request of + // the same client waiting for one takes it before the pool's turn comes + // free, and is never lent a turn past its share in between. _share: Option, + _pool: OwnedSemaphorePermit, _client: ClientHold, /// For a lent turn: the client's own share, which the reading moves onto /// when one of its turns comes free, and how many others are waiting. @@ -206,19 +213,50 @@ impl Drop for ClientHold { } } -struct Waiting(Arc, Clients, (Pool, String)); +struct Waiting(Arc, Clients, (Pool, String), u64); + +/// A request of the client holding its own turn and waiting for the pool's +/// ([`Share::between`]), until it is dropped. +struct Between(Clients, (Pool, String)); + +impl Between { + fn new(clients: &Clients, key: &(Pool, String)) -> Between { + if let Some(s) = clients.lock().unwrap().get_mut(key) { + s.between += 1; + } + Between(Arc::clone(clients), key.clone()) + } +} + +impl Drop for Between { + fn drop(&mut self) { + if let Some(s) = self.0.lock().unwrap().get_mut(&self.1) { + s.between -= 1; + } + } +} impl Drop for Waiting { fn drop(&mut self) { self.0.fetch_sub(1, Ordering::SeqCst); if let Some(s) = self.1.lock().unwrap().get_mut(&self.2) { - s.waiting -= 1; + s.waiting.remove(&self.3); } } } -/// Whether requests of other clients than one are waiting for a turn: a -/// turn lent to it goes back once they are, not for its own requests. +/// How long one of a client's own requests may wait behind a turn lent to +/// that client before the lent turn goes back: as long as a call of its own +/// that ends sooner (a decode beside a threshold) may take to free one of its +/// turns, and short enough that the waiting call takes the lent turn with +/// time to spare before its [`WAIT`] ends (the lent reading is stopped within +/// a second and a half of it being due). +pub const OWN_PATIENCE: Duration = Duration::from_secs(WAIT.as_secs() - 3); + +/// Whether a turn lent to a client should go back: requests of other +/// clients are waiting for a turn, or one of its own has waited +/// [`OWN_PATIENCE`]. Its own requests that have waited less wait on: the +/// call on its own turn may end first and free it. #[derive(Clone)] pub struct Others { waiting: Arc, @@ -228,11 +266,15 @@ pub struct Others { impl Others { fn waiting(&self) -> bool { - let mine = self.clients.lock().unwrap().get(&self.key).map_or(0, |s| s.waiting); - self.waiting.load(Ordering::SeqCst) > mine + let clients = self.clients.lock().unwrap(); + let (count, oldest) = clients.get(&self.key).map_or((0, None), |s| (s.waiting.len(), s.waiting.values().min().copied())); + self.waiting.load(Ordering::SeqCst) > count || oldest.is_some_and(|t| t.elapsed() >= OWN_PATIENCE) } } +/// A number for each request that waits, so it is told from the others. +static TICKETS: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0); + impl Gate { /// The pools, a client having `reads` turns of the reading pool and /// `checks` of the checks (each at least one, and at most the pool). @@ -260,15 +302,19 @@ impl Gate { /// it up. pub async fn turn_until(&self, pool: Pool, who: &str, until: Instant) -> Result { let busy = |why: String| (503, tag(refusal::BUSY, why)); - let (share, pool_turns, full) = match pool { - Pool::Reading => (self.shares.0, &self.reading, format!("{AT_ONCE} requests are being read; try again shortly")), - Pool::Checks => (self.shares.1, &self.checks, format!("{CHECKS_AT_ONCE} checks are running; try again shortly")), + let (share, pool_turns, pool_size, full) = match pool { + Pool::Reading => (self.shares.0, &self.reading, AT_ONCE, format!("{AT_ONCE} requests are being read; try again shortly")), + Pool::Checks => (self.shares.1, &self.checks, CHECKS_AT_ONCE, format!("{CHECKS_AT_ONCE} checks are running; try again shortly")), }; let key = (pool, who.to_string()); let (mine, mut client) = { let mut clients = self.clients.lock().unwrap(); - let s = clients.entry(key.clone()).or_insert_with(|| Share { turns: Arc::new(Semaphore::new(share)), requests: 0, lent: 0, waiting: 0 }); - if s.requests >= share + 1 + CLIENT_WAITING { + let s = clients.entry(key.clone()).or_insert_with(|| Share { turns: Arc::new(Semaphore::new(share)), requests: 0, lent: 0, waiting: HashMap::new(), between: 0 }); + // Its share, a turn lent past it where the pool can ever lend one + // to a client holding its share ([`LEND_LEAVES`]), and + // [`CLIENT_WAITING`] waiting behind them. + let lendable = usize::from(pool_size.saturating_sub(share) > LEND_LEAVES); + if s.requests >= share + lendable + CLIENT_WAITING { return Err(busy(format!("{} of your requests are being read or waiting to be; send the next when one is answered", s.requests))); } s.requests += 1; @@ -280,7 +326,7 @@ impl Gate { { return Ok(Turn { _pool: p, _share: Some(m), _client: client, lend: None }); } - if let Some(p) = self.lend(pool_turns, &mut client) { + if let Some(p) = self.lend(pool_turns, &mine, &mut client) { { let others = self.others(&client.key); return Ok(Turn { _pool: p, _share: None, _client: client, lend: Some((Arc::clone(&mine), others)) }); @@ -290,13 +336,15 @@ impl Gate { self.waiting.fetch_sub(1, Ordering::SeqCst); return Err(busy(full)); } + let ticket = TICKETS.fetch_add(1, Ordering::SeqCst); if let Some(s) = self.clients.lock().unwrap().get_mut(&client.key) { - s.waiting += 1; + s.waiting.insert(ticket, Instant::now()); } - let _waiting = Waiting(Arc::clone(&self.waiting), Arc::clone(&self.clients), client.key.clone()); + let _waiting = Waiting(Arc::clone(&self.waiting), Arc::clone(&self.clients), client.key.clone(), ticket); let until = until.min(Instant::now() + WAIT); // Each acquisition kept across the loop, so its place in line is kept. let mut own: Option = None; + let mut _between: Option = None; let mut own_wait = Box::pin(Arc::clone(&mine).acquire_owned()); let mut pool_wait = Box::pin(Arc::clone(pool_turns).acquire_owned()); loop { @@ -308,13 +356,16 @@ impl Gate { if own.is_none() { tokio::select! { m = &mut own_wait => match m { - Ok(m) => own = Some(m), + Ok(m) => { + own = Some(m); + _between = Some(Between::new(&self.clients, &client.key)); + } Err(_) => break, }, () = step => {} } if own.is_none() - && let Some(p) = self.lend(pool_turns, &mut client) + && let Some(p) = self.lend(pool_turns, &mine, &mut client) { { let others = self.others(&client.key); @@ -343,11 +394,16 @@ impl Gate { } /// A turn lent to `client` past its share, if the pool can spare one and - /// still keep [`LEND_LEAVES`] free, and the client holds none lent. - fn lend(&self, pool_turns: &Arc, client: &mut ClientHold) -> Option { + /// still keep [`LEND_LEAVES`] free, and the client holds none lent and + /// has every one of its own turns running: none free (`mine`), and none + /// taken by a request still waiting for the pool's turn, as each of those + /// is about to run on its own (a turn's pool turn may come free a moment + /// before such a request takes it, and the pool then looks to have one + /// to spare). + fn lend(&self, pool_turns: &Arc, mine: &Semaphore, client: &mut ClientHold) -> Option { let mut clients = self.clients.lock().unwrap(); let s = clients.get_mut(&client.key)?; - if s.lent > 0 || pool_turns.available_permits() <= LEND_LEAVES { + if s.lent > 0 || s.between > 0 || mine.available_permits() > 0 || pool_turns.available_permits() <= LEND_LEAVES { return None; } let p = Arc::clone(pool_turns).try_acquire_owned().ok()?; @@ -363,8 +419,9 @@ impl Gate { pub enum Read { /// A job or a set: its source read where it has one, held to the checks /// that read and planned within `job_bytes` (`http::plan_job`). The answer - /// is `{"request", "plan", "items", "plans", "result_bytes"}` (the most - /// its result takes as served, or null), or `{"request", "refused": + /// is `{"request", "plan", "items", "plans", "result_bytes", + /// "result_parts"}` (the most its result takes as served, in all and by + /// part, or null), or `{"request", "refused": /// [status, why]}` where the planning refused it. With `keep` false (a /// dry run, which keeps nothing) the request and each item's request are /// left out: a dry run reads the plans and the points alone. @@ -403,11 +460,13 @@ pub async fn read_until(config: &Config, turn: Turn, r: Read, until: Option Result { (items, serde_json::to_value(plans).expect("plans are JSON")) } }; - Ok(json!({ "request": request, "plan": p, "items": items, "plans": plans, "result_bytes": bound })) + Ok(json!({ "request": request, "plan": p, "items": items, "plans": plans, "result_bytes": bound.map(|b| b.bytes()), "result_parts": bound })) }), Read::Check { check, body } => { let check = crate::analysis::Check::named(&check).ok_or((400, format!("no check {check}")))?; @@ -576,16 +635,59 @@ pub fn child() { let cap = config.read_answer_mib << 20; let mut input = String::new(); let out = match std::io::Read::read_to_string(&mut std::io::stdin(), &mut input).map_err(|e| e.to_string()).and_then(|_| serde_json::from_str::(&input).map_err(|e| e.to_string())) { - Ok(r) => match run(r) { - Ok(v) if weight(&v) > cap => json!({ "refused": [422, too_much(config.read_answer_mib)] }), - Ok(v) => json!({ "ok": v }), - Err((code, why)) => json!({ "refused": [code, why] }), - }, + Ok(r) => { + // A dry run reads as its submission reads, and is held to what the + // submission's answer would take, here and as the service reads it + // ([`in_child`]): it is given back its plans and points alone. + let dry = matches!(r, Read::Admit { keep: false, .. }); + let r = match r { + Read::Admit { request, job_bytes, keep: false } => Read::Admit { request, job_bytes, keep: true }, + r => r, + }; + match run(r) { + Ok(v) if weight(&v) > cap || (dry && text_len(&json!({ "ok": v })) + 1 > cap) => json!({ "refused": [422, too_much(config.read_answer_mib)] }), + Ok(v) if dry => json!({ "ok": plans_alone(v) }), + Ok(v) => json!({ "ok": v }), + Err((code, why)) => json!({ "refused": [code, why] }), + } + } Err(e) => json!({ "refused": [500, format!("the reading did not arrive: {e}")] }), }; println!("{out}"); } +/// `v`'s length as the child writes it, bytes, written nowhere. +fn text_len(v: &Value) -> u64 { + struct Count(u64); + impl std::io::Write for Count { + fn write(&mut self, b: &[u8]) -> std::io::Result { + self.0 += b.len() as u64; + Ok(b.len()) + } + fn flush(&mut self) -> std::io::Result<()> { + Ok(()) + } + } + let mut c = Count(0); + let _ = serde_json::to_writer(&mut c, v); + c.0 +} + +/// An admission's answer as a dry run takes it ([`Read::Admit`] with `keep` +/// false): the request and each item's request left out, the plans and the +/// points kept. +fn plans_alone(mut v: Value) -> Value { + if v.get("request").is_some() { + v["request"] = Value::Null; + } + if let Some(items) = v.get_mut("items").and_then(Value::as_array_mut) { + for it in items.iter_mut() { + *it = json!({ "point": it.get("point").cloned().unwrap_or(Value::Null) }); + } + } + v +} + /// The refusal of a reading whose answer is more than the service takes back. fn too_much(mib: u64) -> String { tag(refusal::TOO_LARGE_HERE, format!("reading this request gave more than {mib} MiB once read, more than the service takes back of one; {LOCAL}")) @@ -879,6 +981,77 @@ mod tests { assert!(gate.clients.lock().unwrap().is_empty(), "a client that holds nothing is not kept"); } + /// At the default shares, a client's readings run two at a time and at + /// most eight wait: no reading turn is lent to a client holding its share + /// of two (the pool would keep fewer than two free for others), so its + /// eleventh request at once is refused, at once. A check's share of one is + /// lent one more turn, and eight wait behind both: its eleventh is + /// refused too. As turns come back, every one that waited goes. + #[tokio::test] + async fn a_client_s_eleventh_request_at_once_is_refused_and_eight_wait() { + for (pool, share, lent) in [(Pool::Reading, 2, false), (Pool::Checks, 1, true)] { + let gate = Arc::new(Gate::new(2, 1)); + let mut held = Vec::new(); + for _ in 0..share { + held.push(gate.turn(pool, "a").await.ok().unwrap()); + } + if lent { + let t = gate.turn(pool, "a").await.ok().unwrap(); + assert!(t.lent(), "{pool:?}"); + held.push(t); + } + let waiters: Vec<_> = (0..CLIENT_WAITING) + .map(|_| { + let g = Arc::clone(&gate); + tokio::spawn(async move { g.turn(pool, "a").await.map(|t| t.lent()).map_err(|e| e.0) }) + }) + .collect(); + while gate.waiting.load(Ordering::SeqCst) < CLIENT_WAITING { + tokio::time::sleep(Duration::from_millis(5)).await; + } + let t = Instant::now(); + let (code, why) = gate.turn(pool, "a").await.err().expect("an eleventh request at once is refused"); + assert!(code == 503 && why.contains(&format!("{} of your requests", held.len() + CLIENT_WAITING)) && t.elapsed() < Duration::from_millis(300), "{pool:?}: {why}"); + drop(held); + for w in waiters { + let r = w.await.unwrap(); + assert!(r.is_ok(), "{pool:?}: a waiting request had its turn: {r:?}"); + } + } + } + + /// A client at its share of the readings is never lent one, even in the + /// moment between a turn of its own coming free and the request waiting + /// for it taking the pool's: through fifty bursts of eight readings each, + /// every one runs on the client's own turn, and the pool always keeps two + /// free for others. + #[tokio::test] + async fn a_client_at_its_share_of_the_readings_is_never_lent_one() { + let gate = Arc::new(Gate::new(2, 1)); + for _ in 0..50 { + let held = vec![gate.turn(Pool::Reading, "a").await.ok().unwrap(), gate.turn(Pool::Reading, "a").await.ok().unwrap()]; + let burst: Vec<_> = (0..CLIENT_WAITING) + .map(|_| { + let g = Arc::clone(&gate); + tokio::spawn(async move { + let t = g.turn(Pool::Reading, "a").await.ok().unwrap(); + let free = g.reading.available_permits(); + tokio::time::sleep(Duration::from_millis(1)).await; + (t.lent(), free) + }) + }) + .collect(); + while gate.waiting.load(Ordering::SeqCst) < CLIENT_WAITING { + tokio::time::sleep(Duration::from_millis(1)).await; + } + drop(held); + for b in burst { + let (lent, free) = b.await.unwrap(); + assert!(!lent && free >= LEND_LEAVES, "a reading lent past the share, or the pool left with {free} free"); + } + } + } + /// Past [`MAX_WAITING`] a request is told at once, and those that wait /// are told within [`WAIT`]: nothing waits without limit. #[tokio::test] diff --git a/wai/wai-quantum-service/tests/depth.rs b/wai/wai-quantum-service/tests/depth.rs index be5268a39..1c2e87639 100644 --- a/wai/wai-quantum-service/tests/depth.rs +++ b/wai/wai-quantum-service/tests/depth.rs @@ -767,17 +767,17 @@ fn peak_kib(svc: &Child) -> Option { /// shares: a sweep whose hundred items each copy a large field, a few hundred /// kilobytes sent, is refused for what its answer takes once read, before the /// service reads it, and the service's own memory stays small. A dry run -/// takes back the plans and the points alone. +/// takes back the plans and the points alone, and is held to what its +/// submission's answer would take: it says the same sweep is refused. #[test] fn a_reading_s_answer_is_held_to_what_it_takes_once_read() { let d = dir("answer-size"); let (mut svc, port) = start(&d, 1000); let sweep = |pad: usize, dry: bool| json!({ "qasm": "OPENQASM 3.0;\ninput int n;\nqubit q;\nx q;\n", "sweep": { "n": (0..100).collect::>() }, "shots": 1, "dry_run": dry, "pad": vec![0; pad] }); - // A dry run takes back what it reports, however large its items are. - for pad in [0, 100_000] { - let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&sweep(pad, true))); - assert!(code == 200 && b["items"].as_array().is_some_and(|i| i.len() == 100), "{code} {b}"); - } + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&sweep(0, true))); + assert!(code == 200 && b["accepted"] == true && b["items"].as_array().is_some_and(|i| i.len() == 100), "{code} {b}"); + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&sweep(100_000, true))); + assert!(code == 200 && b["accepted"] == false && b["refusal"]["code"] == "J031" && b.to_string().contains("once read"), "{code} {b}"); // A job keeps its items: past what the service takes back, refused. let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&sweep(100_000, false))); assert!(code == 422 && b.to_string().contains("J031") && b.to_string().contains("once read"), "{code} {b}"); @@ -1319,7 +1319,7 @@ fn so_far(s: &mut TcpStream) -> (bool, Vec) { /// (`10.0.k.m`), opened 3 ms apart, then health checked 25 times. They held /// every connection, none could be closed for room, and every health check /// was answered `503`. Long polls are held to a share of the connections, -/// past it answered at once with the job as it is, and four connections past +/// past it answered at once (`503`, with `Retry-After`), and four connections past /// the cap are kept for health checks: each is answered `200`. The same for /// event streams, which held their connections as long polls did. Where this /// test process holds fewer connections (a suite run under `ulimit -n 256`), @@ -1617,3 +1617,401 @@ fn a_result_past_what_a_job_may_give_is_refused_before_it_runs() { svc.wait().unwrap(); let _ = std::fs::remove_dir_all(&d); } + +/// A job's report once it has ended, polled. +fn ended(port: u16, id: &str, within: Duration) -> Value { + let t = Instant::now(); + loop { + let (code, b) = json_of(port, "GET", &format!("/v1/jobs/{id}?wait=5"), None); + assert_eq!(code, 200, "{b}"); + if b["status"] != "queued" && b["status"] != "running" { + return b; + } + assert!(t.elapsed() < within, "{id} still {} after {within:?}", b["status"]); + } +} + +/// [`json_of`] as the client `who` (through a trusted proxy). +fn json_as(port: u16, who: &str, path: &str, body: &Value) -> (u16, Value) { + let (code, out) = post_as(port, who, path, body); + let text = out.split_once("\r\n\r\n").map(|(_, b)| b).unwrap_or(""); + (code, serde_json::from_str(text).unwrap_or(Value::Null)) +} + +/// An MCP tool's answer: its result's `structuredContent`, and whether it +/// was an error. +fn mcp_tool(port: u16, tool: &str, args: &Value) -> (bool, Value) { + let body = json!({ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": tool, "arguments": args } }); + let (code, b) = json_of(port, "POST", "/mcp", Some(&body)); + assert_eq!(code, 200, "{tool}: {b}"); + (b["result"]["isError"] == true, b["result"]["structuredContent"].clone()) +} + +/// A 16-qubit sweep: `points` angles, `shots` a point, 16 measured bits. +fn sweep16(points: usize, shots: u64) -> Value { + json!({ "qasm": "OPENQASM 3.0;\ninput float theta;\nqubit[16] q;\nbit[16] c;\nh q;\nrz(theta) q[0];\nc = measure q;\n", "sweep": { "theta": (0..points).map(|i| 0.01 * (i + 1) as f64).collect::>() }, "shots": shots, "seed": 7 }) +} + +/// An MCP dry run says what the MCP run tools take: it plans the job as they +/// run it, its verdict asked, so the most points of a sweep its dry run +/// accepts under a cap of 1 MiB are taken by `run_quantum_set`, where the +/// dry run planned the job without its verdict and the run then refused it +/// at once (`J031`). +#[test] +fn an_mcp_dry_run_says_what_the_run_tools_take() { + let d = dir("mcp-dry"); + let (mut svc, port) = start_with(&d, 10_000, &[("WAI_QSVC_RESULT_MIB", "1")]); + let accepted = |n: usize| -> bool { + let (err, v) = mcp_tool(port, "dry_run_quantum_job", &sweep16(n, 2000)); + assert!(!err, "{v}"); + v["accepted"] == true + }; + let mut n = 1; + while accepted(n + 1) { + n += 1; + assert!(n < 100); + } + let (err, v) = mcp_tool(port, "run_quantum_set", &sweep16(n, 2000)); + let why = v["error"].as_str().unwrap_or("").to_string(); + assert!(!(err && why.contains("J031")), "{n} points, accepted by the MCP dry run, refused by the run: {why}"); + let id = v["id"].as_str().unwrap_or_default().to_string(); + let job = if v["status"] == "done" { v } else { ended(port, &id, Duration::from_secs(120)) }; + assert_eq!(job["status"], "done", "{}", job["error"]); + let (err, v) = mcp_tool(port, "run_quantum_set", &sweep16(n + 1, 2000)); + assert!(err && v["error"].as_str().is_some_and(|w| w.contains("J031")), "{} points: {v}", n + 1); + alive(port, &mut svc, "MCP dry runs"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// A dry run that says a request is taken is taken: a sweep whose items +/// each carry a large shared field read, kept, past what the service takes +/// back of one reading (`J031`) is refused by its dry run too, as the dry +/// run's reading is held to what the submission's would take; one under it +/// is taken by both. +#[test] +fn a_dry_run_that_says_taken_is_taken_by_its_submission() { + let d = dir("dry-weight"); + let (mut svc, port) = start(&d, 10_000); + for pad in [12_000usize, 20_000] { + let pads = vec!["1e20"; pad].join(","); + let text = format!("{{\"qasm\":\"OPENQASM 3.0;\\ninput float theta;\\nqubit q;\\nbit c;\\nrx(theta) q;\\nc = measure q;\\n\",\"sweep\":{{\"theta\":[{}]}},\"shots\":10,\"pad\":[{pads}]}}", (0..100).map(|i| format!("0.{}", i + 1)).collect::>().join(",")); + let send = |dry: bool| -> (u16, String) { + let body = if dry { text.replacen("{", "{\"dry_run\":true,", 1) } else { text.clone() }; + let mut s = TcpStream::connect(("127.0.0.1", port)).unwrap(); + s.set_read_timeout(Some(Duration::from_secs(120))).unwrap(); + write!(s, "POST /v1/jobs HTTP/1.1\r\nHost: 127.0.0.1\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}", body.len()).unwrap(); + let mut out = String::new(); + let _ = s.read_to_string(&mut out); + (out.split_whitespace().nth(1).and_then(|c| c.parse().ok()).unwrap_or(0), out.split_once("\r\n\r\n").map(|(_, b)| b.to_string()).unwrap_or_default()) + }; + let (code, dry) = send(true); + assert_eq!(code, 200, "{dry}"); + let dry: Value = serde_json::from_str(&dry).unwrap(); + let (code, sent) = send(false); + eprintln!("pad {pad}: dry run accepted {}, submission {code}", dry["accepted"]); + assert_eq!(dry["accepted"] == true, code == 202, "pad {pad}: the dry run said {} ({}), the submission {code} {sent}", dry["accepted"], dry["refusal"]); + if code == 202 { + let id = serde_json::from_str::(&sent).unwrap()["id"].as_str().unwrap().to_string(); + let _ = http(port, "DELETE", &format!("/v1/jobs/{id}"), None); + } else { + assert!(sent.contains("J031") && dry["refusal"]["code"] == "J031", "pad {pad}: {sent} / {dry}"); + } + } + alive(port, &mut svc, "dry runs held to the submission's reading"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// A set's bound counts each item's request as it is served: a number RFC +/// 8785 writes longer than serde (`1e20`, 21 characters there) is counted at +/// its served length, so the result of a sweep sharing thousands of them is +/// within its dry run's `result_bytes`. +#[test] +fn a_set_s_bound_counts_its_items_requests_as_served() { + let d = dir("set-numbers"); + let (mut svc, port) = start(&d, 10_000); + let pads = vec!["1e20"; 20_000].join(","); + let text = format!("{{\"qasm\":\"OPENQASM 3.0;\\ninput float theta;\\nqubit q;\\nbit c;\\nrx(theta) q;\\nc = measure q;\\n\",\"sweep\":{{\"theta\":[0.1,0.2,0.3]}},\"shots\":10,\"pad\":[{pads}]}}"); + let req: Value = serde_json::from_str(&text).unwrap(); + let mut dry = req.clone(); + dry["dry_run"] = json!(true); + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&dry)); + assert_eq!(code, 200, "{b}"); + let bound = b["result_bytes"].as_u64().unwrap(); + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&req)); + assert_eq!(code, 202, "{b}"); + let job = ended(port, b["id"].as_str().unwrap(), Duration::from_secs(120)); + assert_eq!(job["status"], "done", "{}", job["error"]); + let served = wai_quantum::engine::receipt::canonical(&job["result"]).len() as u64; + eprintln!("3 points sharing 20,000 numbers: served {served} bytes, bound {bound}"); + assert!(served <= bound, "served {served} bytes, past the bound of {bound}"); + alive(port, &mut svc, "a set's numbers"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// A GHZ state's sweep is bounded by the two outcomes the stabilizer method +/// can give, not by a distinct outcome a shot: 100 copies of a 24-qubit GHZ +/// state at 30,000 shots each, refused before at a bound of 89 MB for a +/// result of about 250 KB, are taken and run, their result within its bound. +#[test] +fn an_honest_clifford_set_is_taken_by_the_outcomes_it_can_give() { + let d = dir("ghz-set"); + let (mut svc, port) = start(&d, 10_000); + let mut ghz = "OPENQASM 3.0;\ninclude \"stdgates.inc\";\nqubit[24] q;\nbit[24] c;\nh q[0];\n".to_string(); + for i in 1..24 { + ghz += &format!("cx q[{}], q[{i}];\n", i - 1); + } + ghz += "c = measure q;\n"; + let set = json!({ "programs": vec![ghz; 100], "shots": 30_000, "seed": 3 }); + let mut dry = set.clone(); + dry["dry_run"] = json!(true); + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&dry)); + assert_eq!(code, 200, "{b}"); + assert!(b["accepted"] == true, "{}", b["refusal"]); + let bound = b["result_bytes"].as_u64().unwrap(); + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&set)); + assert_eq!(code, 202, "{b}"); + let job = ended(port, b["id"].as_str().unwrap(), Duration::from_secs(300)); + assert_eq!(job["status"], "done", "{}", job["error"]); + let served = wai_quantum::engine::receipt::canonical(&job["result"]).len() as u64; + eprintln!("100 GHZ24 at 30,000 shots: served {served} bytes, bound {bound}"); + assert!(served <= bound && bound < 8 << 20, "served {served}, bound {bound}"); + alive(port, &mut svc, "a GHZ set"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// A run with its verdict is bounded by what its critique can say, not by a +/// remark on every gate: a loop of 124,700 gates with nothing to remark on is +/// run by an MCP run tool and by `POST /v1/run` with `"verdict": true` (the +/// Studio's request), where both were refused for a 64.7 MiB bound. A job +/// past the cap is refused naming its largest part and what makes it +/// smaller: a verdict's critique of thousands of gates that cancel, the +/// reader's notes, or the counts. +#[test] +fn a_long_loop_s_verdict_is_taken_and_a_refusal_names_its_part() { + let lp = "OPENQASM 3.0;\ninclude \"stdgates.inc\";\nqubit[8] q;\nbit[8] c;\nfor int s in [1:4300] {\n cx q[0], q[1]; rz(0.1) q[1]; cx q[0], q[1];\n cx q[1], q[2]; rz(0.2) q[2]; cx q[1], q[2];\n cx q[2], q[3]; rz(0.3) q[3]; cx q[2], q[3];\n cx q[3], q[4]; rz(0.4) q[4]; cx q[3], q[4];\n cx q[4], q[5]; rz(0.5) q[5]; cx q[4], q[5];\n cx q[5], q[6]; rz(0.6) q[6]; cx q[5], q[6];\n cx q[6], q[7]; rz(0.7) q[7]; cx q[6], q[7];\n rx(0.1) q[0]; rx(0.1) q[1]; rx(0.1) q[2]; rx(0.1) q[3]; rx(0.1) q[4]; rx(0.1) q[5]; rx(0.1) q[6]; rx(0.1) q[7];\n}\nc = measure q;\n"; + let d = dir("loop-verdict"); + let (mut svc, port) = start(&d, 10_000); + let (err, v) = mcp_tool(port, "run_quantum_circuit", &json!({ "qasm": lp, "shots": 1000 })); + assert!(!err, "the MCP run: {v}"); + let (code, b) = json_of(port, "POST", "/v1/run", Some(&json!({ "qasm": lp, "shots": 1000, "verdict": true }))); + assert!(code == 200 && b["status"] == "done", "{code} {}", b["error"]); + svc.kill().unwrap(); + svc.wait().unwrap(); + // Past a cap of 1 MiB: each refusal names the part that passes it. + let (mut svc, port) = start_with(&d, 10_000, &[("WAI_QSVC_RESULT_MIB", "1")]); + let cancels = "OPENQASM 3.0;\nqubit q;\nbit c;\nfor int i in [1:10000] { h q; h q; }\nc = measure q;\n"; + let noted = format!("OPENQASM 3.0;\nqubit q;\nbit c;\n{}h q;\nc = measure q;\n", (0..7000).map(|i| format!("pragma note{i} {}\n", "x".repeat(120))).collect::()); + for (what, job, says) in [ + ("a critique of 10,000 pairs", json!({ "qasm": cancels, "shots": 10, "verdict": true }), "its verdict"), + ("7,000 notes", json!({ "qasm": noted, "shots": 10 }), "the reader's warnings and notes"), + ("a sweep's counts", sweep16(40, 2000), "its counts"), + ] { + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&job)); + let why = b["error"].as_str().unwrap_or(""); + eprintln!("{what}: {code} {why}"); + assert!(code == 422 && why.contains("J031") && why.contains(says), "{what}: {code} {why}"); + } + alive(port, &mut svc, "verdicts bounded by their critique"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// The README's largest sweep, 100 points of 16 measured bits at 30,000 +/// shots a point, is taken and ends done at the service's defaults: the set's +/// worker keeps each item's result as text as it ends, where it kept every +/// item's as a tree and passed the memory it was admitted with (on Linux +/// every time, `the job used 693 MiB, over the 258 MiB`). +#[test] +fn the_readme_s_largest_sweep_is_taken_and_ends_done() { + let d = dir("readme-sweep"); + let (mut svc, port) = start(&d, 10_000); + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&sweep16(100, 30_000))); + assert_eq!(code, 202, "{b}"); + let job = ended(port, b["id"].as_str().unwrap(), Duration::from_secs(600)); + assert_eq!(job["status"], "done", "{}", job["error"]); + assert_eq!(job["result"]["items"].as_array().map(Vec::len), Some(100)); + alive(port, &mut svc, "the README's sweep"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// A client's quick calls are never stuck behind two of its own long checks: +/// one client sends two thresholds that each run past the 10 s a request +/// waits for a turn, then four critiques. The first threshold holds the +/// client's own turn and the second a lent one; the lent turn goes back once +/// a critique of its own has waited 7 s, so every critique is answered (the +/// second threshold may be refused, `J032`, to send again), where every +/// critique waited out its 10 s and was refused. +#[test] +fn a_client_s_quick_calls_pass_two_of_its_own_long_checks() { + let d = dir("two-long"); + let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1")]); + let me = "198.51.100.91"; + let critique = json!({ "qasm": "OPENQASM 3.0;\nqubit[2] q;\nh q[0];\ncx q[0], q[1];\n" }); + let (threshold, alone) = slow_threshold(port); + if alone < 11.0 { + eprintln!("the threshold took {alone:.1} s alone, under the 10 s a request waits for a turn: nothing to test on this host"); + } + let began = Instant::now(); + let answers: Vec<(String, u16, String, f64)> = std::thread::scope(|t| { + let mut calls = Vec::new(); + for (i, (name, path, body, after)) in [("threshold 1", "/v1/qec/threshold", &threshold, 0u64), ("threshold 2", "/v1/qec/threshold", &threshold, 30)].into_iter().chain((0..4).map(|k| ("critique", "/v1/critique", &critique, 4000 + 30 * k))).enumerate() { + let began = &began; + calls.push(t.spawn(move || { + std::thread::sleep(Duration::from_millis(after)); + let (code, out) = post_as(port, me, path, body); + (format!("{name} #{i}"), code, out.lines().last().unwrap_or("").to_string(), began.elapsed().as_secs_f64()) + })); + } + calls.into_iter().map(|c| c.join().unwrap()).collect() + }); + eprintln!("{answers:#?}"); + for (name, code, out, at) in &answers { + if name.starts_with("critique") || name.starts_with("threshold 1") { + assert_eq!(*code, 200, "{name} at {at:.2} s: {out}"); + } else { + assert!(*code == 200 || (*code == 503 && out.contains("lent")), "{name} at {at:.2} s: {code} {out}"); + } + } + alive(port, &mut svc, "two long checks and quick calls"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// Each JSON-RPC id is echoed as it was sent: a fraction or an exponent +/// (`1.0`, `1e3`), and an integer past 64 bits, which read as a number and +/// written again came back as `1`, `1000` and `1.2345678901234568e+29`. +#[test] +fn an_mcp_id_is_echoed_as_it_was_sent() { + let d = dir("mcp-raw-ids"); + let (mut svc, port) = start(&d, 1000); + for id in ["1.0", "1e3", "1E3", "-0.0", "1.50", "18446744073709551616", "123456789012345678901234567890", "\"1.0\"", "7"] { + for method in ["ping", "tools/list"] { + let body = format!("{{\"jsonrpc\":\"2.0\",\"id\":{id},\"method\":\"{method}\"}}"); + let mut s = TcpStream::connect(("127.0.0.1", port)).unwrap(); + s.set_read_timeout(Some(Duration::from_secs(60))).unwrap(); + write!(s, "POST /mcp HTTP/1.1\r\nHost: 127.0.0.1\r\nContent-Type: application/json\r\nAccept: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}", body.len()).unwrap(); + let mut out = String::new(); + let _ = s.read_to_string(&mut out); + assert!(out.starts_with("HTTP/1.1 200"), "{id} {method}: {out}"); + assert!(out.contains(&format!("\"id\":{id},")) || out.contains(&format!("\"id\":{id}}}")), "{method}: sent the id {id}, answered {}", out.split_once("\r\n\r\n").map(|(_, b)| b).unwrap_or("").chars().take(120).collect::()); + } + } + alive(port, &mut svc, "ids echoed as sent"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// A long poll past its client's holds is not held, and says so: `429`, +/// `J032`, `Retry-After` and the job's status, so a client polling in a +/// loop waits, where it was answered `200` at once, unchanged, and a client +/// that polled again at once spun. Another client's long poll is held. +#[test] +fn a_long_poll_past_its_client_s_holds_says_when_to_poll_again() { + let d = dir("poll-again"); + let (mut svc, port) = start_with(&d, 1000, &[("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1")]); + let (code, b) = json_of(port, "POST", "/v1/jobs", Some(&pump())); + assert_eq!(code, 202, "{b}"); + let id = b["id"].as_str().unwrap().to_string(); + let me = "198.51.100.93"; + let streams: Vec = (0..4) + .map(|_| { + let mut s = TcpStream::connect(("127.0.0.1", port)).unwrap(); + write!(s, "GET /v1/jobs/{id}/events HTTP/1.1\r\nHost: x\r\nX-Forwarded-For: {me}\r\n\r\n").unwrap(); + s + }) + .collect(); + std::thread::sleep(Duration::from_millis(500)); + let mut s = TcpStream::connect(("127.0.0.1", port)).unwrap(); + s.set_read_timeout(Some(Duration::from_secs(60))).unwrap(); + let t = Instant::now(); + write!(s, "GET /v1/jobs/{id}?wait=25 HTTP/1.1\r\nHost: x\r\nX-Forwarded-For: {me}\r\nConnection: close\r\n\r\n").unwrap(); + let mut out = String::new(); + let _ = s.read_to_string(&mut out); + let took = t.elapsed(); + let (head, body) = out.split_once("\r\n\r\n").unwrap_or_default(); + let body: Value = serde_json::from_str(body).unwrap_or(Value::Null); + assert!(head.starts_with("HTTP/1.1 429") && head.to_ascii_lowercase().contains("retry-after: 2"), "a fifth hold: {head}"); + assert!(body["code"] == "J032" && (body["status"] == "running" || body["status"] == "queued") && took < Duration::from_secs(5), "{body} after {took:?}"); + // Another client's long poll is held. + let mut s = TcpStream::connect(("127.0.0.1", port)).unwrap(); + s.set_read_timeout(Some(Duration::from_secs(60))).unwrap(); + let t = Instant::now(); + write!(s, "GET /v1/jobs/{id}?wait=2 HTTP/1.1\r\nHost: x\r\nX-Forwarded-For: 198.51.100.94\r\nConnection: close\r\n\r\n").unwrap(); + let mut out = String::new(); + let _ = s.read_to_string(&mut out); + assert!(out.starts_with("HTTP/1.1 200") && t.elapsed() >= Duration::from_millis(1900), "another client's poll: {:?} {}", t.elapsed(), out.lines().next().unwrap_or("")); + drop(streams); + let _ = http(port, "DELETE", &format!("/v1/jobs/{id}"), None); + alive(port, &mut svc, "polls past the holds"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} + +/// What ended jobs keep unread stays under `WAI_QSVC_UNREAD_MIB`: a job is +/// taken only while every result not yet read, those of the jobs queued and +/// running at their bounds among them, and its own bound stay under the cap. +/// Thirty clients each send a job of about 0.8 MiB while one job holds the +/// only worker; where each was taken because nothing had ended, so their +/// results passed the 2 MiB cap six times over once they ended, those past +/// the cap are refused, `503`, and what they all hold unread stays under it. +#[test] +fn ended_jobs_unread_stay_under_the_unread_cap() { + let d = dir("unread-cap"); + let env = [("WAI_QSVC_TRUSTED_PROXIES", "127.0.0.1"), ("WAI_QSVC_UNREAD_MIB", "2"), ("WAI_QSVC_KEPT_MIB", "1"), ("WAI_QSVC_CLIENT_KEPT_MIB", "1"), ("WAI_QSVC_RESULT_MIB", "1"), ("WAI_QSVC_CLIENT_UNREAD_MIB", "4"), ("WAI_QSVC_MAX_QUEUE", "40")]; + let (mut svc, port) = start_with(&d, 1000, &env); + let (code, b) = json_as(port, "198.51.100.200", "/v1/jobs", &pump()); + assert_eq!(code, 202, "{b}"); + let blocker = b["id"].as_str().unwrap().to_string(); + let job = json!({ "qasm": "OPENQASM 3.0;\nqubit[15] q;\nbit[15] c;\nh q;\nc = measure q;\n", "shots": 30_000, "seed": 1 }); + let mut dry = job.clone(); + dry["dry_run"] = json!(true); + let (_, b) = json_as(port, "198.51.100.199", "/v1/jobs", &dry); + let bound = b["result_bytes"].as_u64().unwrap(); + assert!(b["accepted"] == true && bound > 512 << 10 && bound < 1 << 20, "{b}"); + let mut taken = Vec::new(); + for k in 0..30 { + let (code, b) = json_as(port, &format!("198.51.100.{}", k + 1), "/v1/jobs", &job); + match code { + 202 => taken.push(b["id"].as_str().unwrap().to_string()), + 503 => assert!(b["error"].as_str().is_some_and(|w| w.contains("not yet read") && w.contains("J032")), "{b}"), + _ => panic!("{code} {b}"), + } + } + let _ = http(port, "DELETE", &format!("/v1/jobs/{blocker}"), None); + // Each ended, none read until all have. + let t = Instant::now(); + loop { + let (_, v) = json_of(port, "GET", "/v1", None); + if v["load"]["queued"] == 0 && v["load"]["running"] == 0 { + break; + } + assert!(t.elapsed() < Duration::from_secs(120), "{}", v["load"]); + std::thread::sleep(Duration::from_millis(200)); + } + let held: u64 = taken.iter().map(|id| { + let (_, j) = json_of(port, "GET", &format!("/v1/jobs/{id}"), None); + assert_eq!(j["status"], "done", "{}", j["error"]); + http(port, "GET", &format!("/v1/jobs/{id}"), None).2.len() as u64 + }).sum(); + eprintln!("{} of 30 taken, bound {bound} each; held unread at once: {held} bytes of jobs' reports", taken.len()); + assert!(!taken.is_empty() && taken.len() < 30, "{} of 30 taken", taken.len()); + assert!(taken.len() as u64 * bound <= 2 << 20, "{} jobs of {bound} bytes taken under a 2 MiB cap", taken.len()); + alive(port, &mut svc, "the unread cap"); + svc.kill().unwrap(); + svc.wait().unwrap(); + let _ = std::fs::remove_dir_all(&d); +} diff --git a/wai/wai-quantum/schemas/verdict.v1.json b/wai/wai-quantum/schemas/verdict.v1.json index da023e48d..4a9f7e779 100644 --- a/wai/wai-quantum/schemas/verdict.v1.json +++ b/wai/wai-quantum/schemas/verdict.v1.json @@ -312,7 +312,12 @@ "integer", "array", "boolean" - ] + ], + "description": "The outcomes sampled that the stated distribution gives probability 0: the first 64, in order." + }, + "impossible_outcomes": { + "type": "integer", + "description": "How many outcomes sampled the stated distribution gives probability 0." }, "note": { "type": "string" diff --git a/wai/wai-quantum/src/engine/bound_tests.rs b/wai/wai-quantum/src/engine/bound_tests.rs new file mode 100644 index 000000000..ce0e76010 --- /dev/null +++ b/wai/wai-quantum/src/engine/bound_tests.rs @@ -0,0 +1,344 @@ +//! A job's result is never longer than the bound its plan gave before it ran +//! (`catalog::plan_bounded`), as a host writes it (RFC 8785, with its +//! provenance), for jobs of every kind drawn at random: sampled intents that +//! fail with many outcomes they rule out, distributions and references, +//! trajectories with their own count, verdicts over many notes and smells, +//! Clifford circuits of every shape on the stabilizer method, noise, observables, +//! amplitudes, compiles, repairs, mitigations, programs that measure in the +//! middle, device targets, pulse runs and intents carrying numbers RFC 8785 +//! writes long. And where the method knows its outcomes, the bound is close: +//! a GHZ state's two outcomes, and a long loop with nothing to remark on. + +use super::catalog::plan_bounded; +use super::run::run_with_provenance; +use serde_json::{Value, json}; + +const CAP: u64 = 8 << 30; + +/// The longest a host writes `v`: as the engine's RFC 8785 writer does (every +/// number as a double), or with each whole number exactly as read and every +/// other number as RFC 8785 does (a job service's served text). +fn served(v: &Value) -> u64 { + struct Es; + impl serde_json::ser::Formatter for Es { + fn write_f64(&mut self, w: &mut W, x: f64) -> std::io::Result<()> { + w.write_all(super::receipt::es_number(x).as_bytes()) + } + } + let mut out = Vec::new(); + let mut ser = serde_json::Serializer::with_formatter(&mut out, Es); + serde::Serialize::serialize(v, &mut ser).unwrap(); + (super::receipt::canonical(v).len() as u64).max(out.len() as u64) +} + +/// A small generator of its own (SplitMix64), so a failure names its seed. +struct Rng(u64); + +impl Rng { + fn next(&mut self) -> u64 { + self.0 = self.0.wrapping_add(0x9e37_79b9_7f4a_7c15); + let mut z = self.0; + z = (z ^ (z >> 30)).wrapping_mul(0xbf58_476d_1ce4_e5b9); + z = (z ^ (z >> 27)).wrapping_mul(0x94d0_49bb_1331_11eb); + z ^ (z >> 31) + } + /// One of `lo..=hi`. + fn int(&mut self, lo: u64, hi: u64) -> u64 { + lo + self.next() % (hi - lo + 1) + } + fn coin(&mut self) -> bool { + self.next() & 1 == 1 + } + fn pick<'a, T>(&mut self, xs: &'a [T]) -> &'a T { + &xs[(self.next() % xs.len() as u64) as usize] + } +} + +/// Numbers RFC 8785 writes longer than serde does, and some it does not. +fn long_numbers(rng: &mut Rng, n: u64) -> Value { + let pool = [json!(1e20), json!(1e-6), json!(1.5e-5), json!(123456.0), json!(-0.0), json!(1e21), json!(3), json!(0.1)]; + Value::Array((0..n).map(|_| rng.pick(&pool).clone()).collect()) +} + +/// The program `k` qubits under `h`, each fanned out by `cx` to the rest of +/// `w` qubits, every qubit measured into its bit: `2^k` outcomes `w` wide. +fn fanned(k: u64, w: u64) -> String { + let mut s = format!("OPENQASM 3.0;\ninclude \"stdgates.inc\";\nqubit[{w}] q;\nbit[{w}] c;\n"); + for i in 0..k { + s += &format!("h q[{i}];\n"); + } + for i in k..w { + s += &format!("cx q[{}], q[{i}];\n", i % k); + } + s + "c = measure q;\n" +} + +/// A sampled intent that the program's run does not fit: it states one +/// outcome (or a few), so every other outcome sampled is one it rules out. +fn sampled_intent(rng: &mut Rng, w: u64) -> Value { + let zeros = "0".repeat(w as usize); + let mut want = serde_json::Map::new(); + want.insert(zeros, json!(1.0)); + let mut i = json!({ "sampled": want, "test": rng.pick(&["chi_squared", "g_test"]), "alpha": 0.01, "shots": rng.int(200, 6000), "seed": rng.int(1, 9) }); + if rng.coin() { + i["name"] = json!(format!("intent {}", rng.int(0, 99))); + } + if rng.coin() { + let n = rng.int(1, 3000); + i["note"] = long_numbers(rng, n); + } + i +} + +/// A Clifford circuit drawn at random: `n` qubits, its gates of every +/// Clifford kind, some qubits measured, into bits that may repeat. +fn clifford(rng: &mut Rng, n: u64, gates: u64) -> String { + let bits = rng.int(1, n + 4); + let mut s = format!("OPENQASM 3.0;\ninclude \"stdgates.inc\";\nqubit[{n}] q;\nbit[{bits}] c;\n"); + for _ in 0..gates { + let a = rng.int(0, n - 1); + let b = (a + rng.int(1, n.max(2) - 1)) % n; + let g = *rng.pick(&["h", "s", "sdg", "x", "y", "z", "sx", "sxdg", "cx", "cz", "cy", "swap", "h", "cx"]); + if matches!(g, "cx" | "cz" | "cy" | "swap") && n > 1 { + s += &format!("{g} q[{a}], q[{b}];\n"); + } else if !matches!(g, "cx" | "cz" | "cy" | "swap") { + s += &format!("{g} q[{a}];\n"); + } + } + let measured = rng.int(1, n); + for _ in 0..measured { + s += &format!("c[{}] = measure q[{}];\n", rng.int(0, bits - 1), rng.int(0, n - 1)); + } + s +} + +/// A program with `pragmas` lines the reader notes, and gates that do +/// nothing, cancel or merge, among others, `gates` of them. +fn noted(rng: &mut Rng, pragmas: u64, gates: u64) -> String { + let n = rng.int(2, 6); + let mut s = format!("OPENQASM 3.0;\ninclude \"stdgates.inc\";\nqubit[{n}] q;\nbit[{n}] c;\n"); + for i in 0..pragmas { + s += &format!("pragma wai.unknown_{i} {}\n", "x".repeat(rng.int(0, 40) as usize)); + } + for _ in 0..gates { + let a = rng.int(0, n - 1); + s += &match rng.int(0, 5) { + 0 => format!("h q[{a}];\nh q[{a}];\n"), + 1 => format!("rz(0) q[{a}];\n"), + 2 => format!("rx(0.1) q[{a}];\nrx(0.2) q[{a}];\n"), + 3 => format!("cx q[{a}], q[{}];\ncx q[{a}], q[{}];\n", (a + 1) % n, (a + 1) % n), + 4 => format!("t q[{a}];\ntdg q[{a}];\n"), + _ => format!("h q[{a}];\n"), + }; + } + s + "c = measure q;\n" +} + +/// A job of the kind `kind`, drawn at random. +fn job(rng: &mut Rng, kind: u64) -> (String, Value) { + match kind { + // Sampled intents that fail, ruling out many wide outcomes. + 0 => { + let k = rng.int(3, 10); + let w = rng.int(k, 48); + let intents: Vec = (0..rng.int(1, 3)).map(|_| sampled_intent(rng, w)).collect(); + ("sampled intents that fail".into(), json!({ "qasm": fanned(k, w), "shots": rng.int(1, 200), "intent": intents, "seed": rng.int(1, 99) })) + } + // Trajectories, their own count apart from the shots. + 1 => { + let n = rng.int(2, 9); + let w = rng.int(n, 30); + let mut q = format!("OPENQASM 3.0;\ninclude \"stdgates.inc\";\nqubit[{n}] q;\nbit[{w}] c;\nh q;\n"); + for i in 0..n { + q += &format!("c[{}] = measure q[{i}];\n", rng.int(0, w - 1)); + } + let mut j = json!({ "qasm": q, "method": "trajectories", "shots": rng.int(1, 3000), "noise": { "depolarizing_1q": 0.01, "readout": 0.01 }, "options": { "trajectories": rng.int(1, 40_000) } }); + if rng.coin() { + j["verdict"] = json!(true); + } + ("trajectories".into(), j) + } + // A verdict over many notes and smells. + 2 => { + let (p, g) = (rng.int(0, 2500), rng.int(0, 3000)); + let mut j = json!({ "qasm": noted(rng, p, g), "shots": rng.int(1, 500), "verdict": true }); + if rng.coin() { + j["compile"] = json!({ "epsilon": 1e-3, "level": 2 }); + } + ("notes and smells, with a verdict".into(), j) + } + // A Clifford circuit, on the stabilizer method or as the planner picks. + 3 => { + let n = rng.int(1, 30); + let g = rng.int(0, 200); + let mut j = json!({ "qasm": clifford(rng, n, g), "shots": rng.int(1, 20_000), "seed": rng.int(1, 99) }); + if rng.coin() { + j["method"] = json!("stabilizer"); + } + if rng.int(0, 3) == 0 { + j["noise"] = json!({ "depolarizing_1q": 0.01, "readout": 0.02 }); + } + if rng.coin() { + j["verdict"] = json!(true); + } + ("a Clifford circuit".into(), j) + } + // Observables, amplitudes, a mitigation, a repair, measured in the middle. + 4 => { + let n = rng.int(2, 6); + let mut q = format!("OPENQASM 3.0;\ninclude \"stdgates.inc\";\nqubit[{n}] q;\nbit[{n}] c;\nh q;\nrx(0.3) q[0];\ncx q[0], q[1];\n"); + let mut j = json!({ "shots": rng.int(0, 4000), "seed": 3 }); + match rng.int(0, 4) { + 0 => j["observables"] = json!((0..rng.int(1, 40)).map(|i| format!("{}.25 Z{} X{}", i % 5, i % n, (i + 1) % n)).collect::>()), + 1 => j["amplitudes"] = json!((0..rng.int(1, 20)).map(|i| format!("{:0w$b}", i % (1 << n), w = n as usize)).collect::>()), + 2 => { + j["observables"] = json!(["Z0 Z1"]); + j["noise"] = json!({ "depolarizing_2q": 0.02, "depolarizing_1q": 0.004 }); + j["mitigate"] = json!({ "method": "zne", "shots": 2000 }); + } + 3 => { + q += "h q[0];\nh q[0];\n"; + j["repair"] = json!(true); + j["verdict"] = json!(true); + } + _ => q += "c[0] = measure q[0];\nreset q[0];\nh q[0];\n", + } + j["qasm"] = json!(q + "c = measure q;\n"); + // A seed the engine does not read, which the provenance names as sent. + match rng.int(0, 3) { + 0 => j["seed"] = json!("s".repeat(rng.int(1, 40_000) as usize)), + 1 => j["seed"] = long_numbers(rng, 3000), + _ => {} + } + if j["shots"] == 0 && j.get("observables").is_none() && j.get("amplitudes").is_none() { + j["shots"] = json!(100); + } + ("observables, amplitudes, mitigation, repair or a measurement in the middle".into(), j) + } + // A device target, its trajectories its own count. + 5 => { + let mut j = json!({ "qasm": "OPENQASM 3.0; qubit[3] q; bit[3] c; h q[0]; cx q[0], q[1]; cx q[1], q[2]; c = measure q;", "target": "synthetic-ring-8", "shots": rng.int(1, 3000), "seed": 2 }); + if rng.coin() { + j["method"] = json!("trajectories"); + j["options"] = json!({ "trajectories": rng.int(1, 30_000) }); + } + ("on a device target".into(), j) + } + // A pulse run with its bits named. + 6 => { + let n = rng.int(1, 300) as usize; + let regs: String = (0..n.div_ceil(64)).map(|r| format!("bit[{}] b{r};\n", 64.min(n - 64 * r))).collect(); + let q = format!("OPENQASM 3.0;\ndefcalgrammar \"openpulse\";\ncal {{\n extern port d0;\n extern port a0;\n frame q0 = newframe(d0, 5.0e9, 0);\n frame c0 = newframe(a0, 7.1e9, 0);\n}}\ndefcal x $0 {{ play(q0, gaussian(0.4, 64dt, 16dt)); }}\ndefcal measure $0 -> bit {{ return capture_v2(c0, 4dt); }}\n{regs}x $0;\nb{}[{}] = measure $0;\n", (n - 1) / 64, (n - 1) % 64); + ("a pulse run".into(), json!({ "method": "pulse", "shots": rng.int(1, 2000), "qasm": q, "verdict": rng.coin() })) + } + // A distribution, a reference and a state, as intents, with long numbers. + _ => { + let k = rng.int(1, 6); + let w = rng.int(k, 12); + let src = fanned(k, w); + let mut d = serde_json::Map::new(); + for i in 0..rng.int(1, 80) { + d.insert(format!("{:0w$b}", i % (1 << w.min(16)), w = w as usize), json!(0.0)); + } + let first = d.keys().next().cloned().unwrap(); + d.insert(first, json!(1.0)); + let mut intents = vec![json!({ "distribution": d, "tolerance": 1e-6, "note": long_numbers(rng, 200) }), json!({ "equivalent": src, "name": "itself" }), json!({ "prepares": fanned(k, w) })]; + intents.truncate(rng.int(1, 3) as usize); + ("a distribution, a reference and a state".into(), json!({ "qasm": src, "shots": rng.int(1, 3000), "intent": intents })) + } + } +} + +/// Every random job's result is within the bound its plan gave. The seed is +/// `WAI_BOUND_SEED` if set, so a failure can be read again. +#[test] +#[cfg(all(feature = "quantum_qasm", feature = "quantum_noise", feature = "quantum_receipt"))] +fn no_random_job_s_result_passes_its_bound() { + let seed = std::env::var("WAI_BOUND_SEED").ok().and_then(|s| s.parse().ok()).unwrap_or(676); + let mut rng = Rng(seed); + let (mut checked, mut refused) = (0, 0); + for i in 0..96u64 { + let (what, job) = job(&mut rng, i % 8); + let (p, bound) = match plan_bounded(&job, CAP) { + Ok((p, Some(b))) => (p, b), + Ok((_, None)) => panic!("job {i} (seed {seed}), {what}: no bound"), + Err(e) => { + eprintln!("job {i}, {what}: not planned: {e}"); + refused += 1; + continue; + } + }; + let result = match run_with_provenance(&job, &p, "a host 1.0") { + Ok(r) => r, + Err(e) => { + eprintln!("job {i}, {what}: did not run: {e}"); + refused += 1; + continue; + } + }; + let len = served(&result); + eprintln!("job {i}, {what}: {len} bytes, bound {bound}"); + assert!(len <= bound, "job {i} (seed {seed}), {what}: its result is {len} bytes, past its bound of {bound}\n{job}"); + checked += 1; + } + assert!(checked >= 80 && refused <= 16, "{checked} checked, {refused} not run"); +} + +/// The jobs of round 6's review, each past its bound before: a sampled +/// intent ruling out thousands of 64-bit outcomes, a trajectories run of one +/// shot and 30,000 trajectories, and a verdict over 3,000 notes. +#[test] +#[cfg(all(feature = "quantum_qasm", feature = "quantum_noise", feature = "quantum_receipt"))] +fn a_failing_intent_trajectories_and_notes_stay_within_their_bounds() { + let zeros = "0".repeat(64); + let jobs = [ + ("a sampled intent ruling out 4,095 outcomes", json!({ "qasm": fanned(12, 64), "shots": 1, "intent": [{ "sampled": { zeros.clone(): 1.0 }, "test": "chi_squared", "alpha": 0.01, "shots": 40_000 }, { "sampled": { zeros: 1.0 }, "test": "g_test", "alpha": 0.01, "shots": 40_000 }] })), + ("one shot, 30,000 trajectories", json!({ "qasm": fanned(12, 12).replace("cx", "// cx"), "method": "trajectories", "shots": 1, "noise": { "depolarizing_1q": 1e-6 }, "options": { "trajectories": 30_000 } })), + ("3,000 notes and a verdict", json!({ "qasm": format!("OPENQASM 3.0;\nqubit q;\nbit c;\n{}h q;\nc = measure q;\n", (0..3000).map(|i| format!("pragma s{i}\n")).collect::()), "shots": 10, "verdict": true })), + ]; + let mut past = Vec::new(); + for (what, job) in jobs { + let (p, bound) = plan_bounded(&job, CAP).unwrap(); + let bound = bound.unwrap(); + let result = run_with_provenance(&job, &p, "a host 1.0").unwrap(); + let len = served(&result); + eprintln!("{what}: {len} bytes, bound {bound}"); + if len > bound { + past.push(format!("{what}: its result is {len} bytes, past its bound of {bound}")); + } + } + assert!(past.is_empty(), "{past:#?}"); +} + +/// Where the method knows how many outcomes there can be, or the critique +/// has nothing to say, the bound is close to the result: a GHZ state of 24 +/// qubits at 30,000 shots on the stabilizer method lists two outcomes, and a +/// loop of 124,700 gates that cancel nowhere has no remark, so neither is +/// bounded as if it listed every shot or remarked on every gate. +#[test] +#[cfg(all(feature = "quantum_qasm", feature = "quantum_noise", feature = "quantum_receipt"))] +fn a_ghz_state_and_a_long_loop_are_bounded_by_what_they_can_list() { + let mut ghz = "OPENQASM 3.0;\ninclude \"stdgates.inc\";\nqubit[24] q;\nbit[24] c;\nh q[0];\n".to_string(); + for i in 1..24 { + ghz += &format!("cx q[{}], q[{i}];\n", i - 1); + } + ghz += "c = measure q;\n"; + let lp = "OPENQASM 3.0;\ninclude \"stdgates.inc\";\nqubit[8] q;\nbit[8] c;\nfor int s in [1:4300] {\n cx q[0], q[1]; rz(0.1) q[1]; cx q[0], q[1];\n cx q[1], q[2]; rz(0.2) q[2]; cx q[1], q[2];\n cx q[2], q[3]; rz(0.3) q[3]; cx q[2], q[3];\n cx q[3], q[4]; rz(0.4) q[4]; cx q[3], q[4];\n cx q[4], q[5]; rz(0.5) q[5]; cx q[4], q[5];\n cx q[5], q[6]; rz(0.6) q[6]; cx q[5], q[6];\n cx q[6], q[7]; rz(0.7) q[7]; cx q[6], q[7];\n rx(0.1) q[0]; rx(0.1) q[1]; rx(0.1) q[2]; rx(0.1) q[3]; rx(0.1) q[4]; rx(0.1) q[5]; rx(0.1) q[6]; rx(0.1) q[7];\n}\nc = measure q;\n"; + let mut loose = Vec::new(); + for (what, job, most) in [ + ("GHZ24, 30,000 shots, stabilizer", json!({ "qasm": ghz.clone(), "shots": 30_000, "method": "stabilizer", "seed": 5 }), 32u64 << 10), + ("GHZ24, 30,000 shots, planned", json!({ "qasm": ghz, "shots": 30_000, "seed": 5 }), 32 << 10), + ("a loop of 124,700 gates, with its verdict", json!({ "qasm": lp, "shots": 1000, "verdict": true }), 256 << 10), + ] { + let (p, bound) = plan_bounded(&job, CAP).unwrap(); + let bound = bound.unwrap(); + let result = run_with_provenance(&job, &p, "a host 1.0").unwrap(); + let len = served(&result); + eprintln!("{what}: {len} bytes, bound {bound}, on {}", p.method); + assert!(len <= bound, "{what}: its result is {len} bytes, past its bound of {bound}"); + if bound > most { + loose.push(format!("{what}: bounded at {bound} bytes, where its result is {len}")); + } + } + assert!(loose.is_empty(), "{loose:#?}"); +} diff --git a/wai/wai-quantum/src/engine/catalog.rs b/wai/wai-quantum/src/engine/catalog.rs index 69d051a5b..19882a81c 100644 --- a/wai/wai-quantum/src/engine/catalog.rs +++ b/wai/wai-quantum/src/engine/catalog.rs @@ -322,6 +322,12 @@ pub struct Request { /// With `"mitigate"`: the mitigation asked for, as given (read and /// checked by [`read`]; run after the job, its result under `mitigation`). pub mitigate: Option, + /// For a job that asks for a verdict: the most its critique takes as the + /// verdict lists it, bytes, from the program as read, before any compile + /// ([`super::critique::smells_bound`]; a program holding a gate whole, + /// which the critique reads in standard gates, at [`REMARKS_PER_GATE`] a + /// standard gate and a qubit). 0 for one that asks for none. + pub smells_bytes: u64, } impl Request { @@ -342,6 +348,7 @@ impl Request { compile_passes: self.compile_passes.clone(), repaired: self.repaired.clone(), mitigate: self.mitigate.clone(), + smells_bytes: self.smells_bytes, }) } } @@ -620,6 +627,14 @@ pub(crate) fn read_form(reg: &Registry, v: &Value, whole: bool) -> Result<(Reque if let Some(i) = v.get("intent").filter(|i| !i.is_null()) { super::verdict::fit(&super::verdict::intents(i)?, &program)?; } + // A verdict's critique reads the program as given, not as compiled. + let smells_bytes = if !super::verdict::asked(v) { + 0 + } else if program.has_whole() { + u64::try_from(program.standard_gates()).unwrap_or(u64::MAX).saturating_add(u64::from(program.n_qubits)).saturating_mul(REMARKS_PER_GATE) + } else { + super::critique::smells_bound(&program) + }; // "compile": {"epsilon", "level", "basis"}: the job runs the compiled // program, and its result says how far that is from the one given. let (program, compiled, compile_passes) = match v.get("compile") { @@ -709,6 +724,7 @@ pub(crate) fn read_form(reg: &Registry, v: &Value, whole: bool) -> Result<(Reque compile_passes, repaired, mitigate, + smells_bytes, }, uses)) } @@ -1251,14 +1267,70 @@ pub fn plan(v: &Value, job_bytes: u64) -> Result { /// [`plan`], with the most its result may take as JSON text, in bytes, /// beside what a host adds to it (a receipt, an energy figure), known before -/// it runs ([`result_bound`]); a method that reads its program itself bounds +/// it runs ([`result_bound`]); a method that reads its own program bounds /// its own ([`Method::plan_bounded`]: a pulse run's is /// [`super::pulse::plan_bounded`]), and `None` is one that gives no bound, /// which no built-in is. pub fn plan_bounded(v: &Value, job_bytes: u64) -> Result<(Plan, Option), String> { + plan_bounded_in(Registry::builtins(), v, job_bytes).map(|(p, b)| (p, b.map(|b| b.bytes()))) +} + +/// [`plan_bounded`], the bound given by part ([`Bound`]). +pub fn plan_bound(v: &Value, job_bytes: u64) -> Result<(Plan, Option), String> { plan_bounded_in(Registry::builtins(), v, job_bytes) } +/// The most a job's result takes as JSON text, bytes, part by part, known +/// before it runs ([`result_bound`]): what a host holds a result to before +/// it runs, and what a refusal past its cap names (the largest part, +/// [`Bound::largest`]). +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] +pub struct Bound { + /// Its counts (and a mitigation's), each listing up to the samples it + /// takes, the outcomes its program can give and 65,536. + pub counts: u64, + /// A verdict's: its critique's remarks, each intent's check, and the + /// reader's notes and the repair again. + pub verdict: u64, + /// The reader's warnings and notes on the program. + pub notes: u64, + /// Each item's request, which a set serves beside its result. + pub requests: u64, + /// The rest: expectation values, amplitudes, outputs, a compile's, + /// repair's or target's account, and the fields every result has. + pub rest: u64, +} + +impl Bound { + /// The whole bound, bytes. + pub fn bytes(&self) -> u64 { + self.counts.saturating_add(self.verdict).saturating_add(self.notes).saturating_add(self.requests).saturating_add(self.rest) + } + + /// Both bounds together, part by part: a set's items'. + pub fn plus(self, o: Bound) -> Bound { + Bound { + counts: self.counts.saturating_add(o.counts), + verdict: self.verdict.saturating_add(o.verdict), + notes: self.notes.saturating_add(o.notes), + requests: self.requests.saturating_add(o.requests), + rest: self.rest.saturating_add(o.rest), + } + } + + /// The largest part: its bytes, what it is, and what makes it smaller. + pub fn largest(&self) -> (u64, &'static str, &'static str) { + let parts = [ + (self.counts, "its counts, listing up to the shots (or a trajectories run's trajectories), the outcomes its measured bits can give and 65,536 outcomes an item", "send fewer points, fewer shots or fewer measured bits"), + (self.verdict, "its verdict: the critique's remark on each gate that does nothing, cancels or merges, each intent's check, and the reader's notes again", "ask for no verdict (leave out \"verdict\" and \"intent\", and send it to POST /v1/jobs, not an MCP run tool, which asks for one), or send fewer points"), + (self.notes, "the reader's warnings and notes on its program, one for each line they are on", "remove or fix the lines the reader notes (each `pragma` it does not use draws one), or send fewer points"), + (self.requests, "each item's request, which a set serves beside the item's result", "send fewer points, or keep large fields out of the set's request: each item serves the fields it shares"), + (self.rest, "its expectation values, amplitudes, outputs and its compile's, repair's or target's account", "ask for fewer observables, amplitudes or outputs, or send fewer points"), + ]; + parts.into_iter().fold(parts[0], |a, b| if b.0 > a.0 { b } else { a }) + } +} + /// What a result holds beside the parts [`result_bound`] counts one by one: /// its method, qubits and gates, its accuracy and its notes, its state's /// hash, its schema, its timing and its provenance, the plan's reason apart. @@ -1268,64 +1340,121 @@ const RESULT_REST: u64 = 16 << 10; const EXPECTATION_BYTES: u64 = 1 << 10; /// An amplitude's numbers and names, beside its bit string. const AMPLITUDE_BYTES: u64 = 1 << 9; -/// A verdict's own fields, beside its critique's remarks and its intents'. +/// A verdict's own fields, beside its critique's remarks, its intents', the +/// notes and the repair it repeats. const VERDICT_BYTES: u64 = 16 << 10; -/// One intent's check, beside the intent as given and the counts it may +/// One intent's check, beside the intent as given and the outcomes it may /// list ([`super::verdict::MAX_LISTED`]). const INTENT_BYTES: u64 = 16 << 10; -/// A critique's remarks on one gate of the program in standard gates, at -/// most: a gate draws one that it does nothing, and is one of a pair that -/// cancels or merges, each naming its lines and its qubits. -const REMARKS_PER_GATE: u64 = 512; +/// A critique's remarks on one gate of a program it reads in standard gates +/// where the program as read holds a gate whole, at most: a gate draws one +/// that it does nothing, and is one of a pair that cancels or merges, each +/// naming its lines and its qubits. +pub(crate) const REMARKS_PER_GATE: u64 = 512; /// A mitigation's own fields, beside a value for each observable and a /// count for each outcome. const MITIGATION_BYTES: u64 = 64 << 10; +/// An outcome as an intent's check lists it, beside its bit string: quotes, +/// a colon, a comma, and a count or a probability at its longest. +const LISTED_BYTES: u64 = 32; + +/// How many times an intent's check may list the outcomes it lists: a +/// sampled intent's counts, the outcomes of probability 0 it names (in its +/// fit, in the fit's note, and in its reason); a distribution's computed +/// outcomes and its largest gaps. +const LISTS_PER_INTENT: u64 = 4; + +/// The samples a job's counts are drawn from: its shots, or for the +/// trajectories method (and a device target's), its trajectories +/// ([`trajectory_count`]), one a sample, which the counts list. +pub(crate) fn samples_listed(r: &Request, p: &Plan) -> u64 { + if r.shots > 0 && p.method == "trajectories" { trajectory_count(r).max(r.shots) } else { r.shots } +} + +/// How many distinct outcomes a job's counts can list, at most: `2^b` for +/// the `b` classical bits its measurements write (the rest stay 0; every bit +/// of a program that measures in the middle), and for a noiseless run of the +/// stabilizer method, `2^d`, `d` its outcome dimension's bound: its measured +/// qubits, and the gates that branch (each `h`, `sx` and `sxdg` it runs as, +/// the rest permuting or phasing the basis states), each at most doubling +/// the outcomes a tableau can give. A GHZ state has 2, however wide. +pub(crate) fn outcomes_bound(r: &Request, p: &Plan) -> u64 { + let pow2 = |e: u64| if e >= 63 { u64::MAX } else { 1u64 << e }; + let prog = &r.program; + if prog.dynamic.is_some() { + return pow2(u64::from(prog.n_bits)); + } + let bits: std::collections::BTreeSet = prog.measured.iter().map(|&(_, b)| b).collect(); + let mut e = bits.len() as u64; + if p.method == "stabilizer" && r.noise.is_none() && p.target.is_none() { + use crate::quantum_tableau::Clifford; + let qubits: std::collections::BTreeSet = prog.measured.iter().map(|&(q, _)| q).collect(); + e = e.min(qubits.len() as u64); + if let Ok(gates) = prog.clifford() { + e = e.min(gates.iter().filter(|g| matches!(g, Clifford::H(_) | Clifford::SX(_) | Clifford::SXdg(_))).count() as u64); + } + } + pow2(e) +} /// The most the result of the job `v`, read as `r` and planned as `p`, takes -/// as JSON text, bytes: each part that grows with what the job asks bounded -/// by what it may list, from the job as read. Its counts (and a -/// mitigation's) by [`super::run::counts_bound`]; each expectation value by -/// its observable's text, escaped, and a fixed allowance; each amplitude by -/// its bit string and one; what its reading put in it (its diagnostics, its -/// outputs, its repair, its compile and its target) at its size as read; a -/// verdict by its fields, each intent as given and the counts its check may -/// list, and its critique's remarks at [`REMARKS_PER_GATE`] a gate; and the -/// rest at [`RESULT_REST`]. The counts are written exactly so; RFC 8785 -/// writes another number at most a few characters longer than serde does, so -/// a sixteenth more is allowed for the rest. -pub(crate) fn result_bound(v: &Value, r: &Request, p: &Plan) -> u64 { - let size = |x: &Value| serde_json::to_string(x).map_or(0, |s| s.len() as u64); +/// as JSON text, bytes, by part ([`Bound`]): each part that grows with what +/// the job asks bounded by what it may list, from the job as read. Its +/// counts (and a mitigation's) by [`super::run::counts_bound_among`], of +/// the samples it lists ([`samples_listed`]) among the outcomes it can give +/// ([`outcomes_bound`]); each expectation value by its observable's text, +/// escaped, and a fixed allowance; each amplitude by its bit string and one; +/// what its reading put in it (its diagnostics, its outputs, its repair, its +/// compile and its target) at its length as RFC 8785 writes it +/// ([`super::receipt::canonical_len`], every number in its own form); a +/// verdict by its fields, the diagnostics and the repair it repeats, its +/// critique's remarks ([`Request::smells_bytes`]), and each intent as given +/// with the outcomes its check lists, [`super::verdict::MAX_LISTED`] at a +/// time (and two intents' reasons again, in its statement); and the rest at +/// [`RESULT_REST`]. The counts are written exactly so; a sixteenth more is +/// allowed for the rest, for the numbers the engine writes. +pub(crate) fn result_bound(v: &Value, r: &Request, p: &Plan) -> Bound { + let size = super::receipt::canonical_len; // A string of the job's own, every character escaped at its longest. let text = |t: &str| 6 * t.len() as u64 + 2; let bits = r.program.n_bits; - let digits = r.shots.max(1).to_string().len() as u64; - let counts = super::run::counts_bound(bits, r.shots, None); - let mut b = RESULT_REST + 2 * size(&json!(p.why)); - b += r.observables.iter().map(|(t, _, _)| EXPECTATION_BYTES + text(t)).sum::(); - b += r.amplitudes.len() as u64 * (AMPLITUDE_BYTES + u64::from(r.program.n_qubits)); - b += r.diagnostics.iter().map(|d| size(&d.to_json()) + 1).sum::(); - b += r.program.extra.outputs.iter().map(|o| 64 + text(&o.name) + text(&o.ty) + match &o.value { + let samples = samples_listed(r, p); + let outcomes = outcomes_bound(r, p); + let mut counts = super::run::counts_bound_among(bits, samples, outcomes, None); + // The provenance names the job's shots and seed as they were sent, + // whatever they are. + let mut rest = RESULT_REST + 2 * size(&json!(p.why)) + ["shots", "seed"].iter().filter_map(|k| v.get(*k)).map(size).sum::(); + rest += r.observables.iter().map(|(t, _, _)| EXPECTATION_BYTES + text(t)).sum::(); + rest += r.amplitudes.len() as u64 * (AMPLITUDE_BYTES + u64::from(r.program.n_qubits)); + let notes = r.diagnostics.iter().map(|d| size(&d.to_json()) + 1).sum::(); + rest += r.program.extra.outputs.iter().map(|o| 64 + text(&o.name) + text(&o.ty) + match &o.value { qasm::OutputValue::Bits(bits) => 11 * bits.len() as u64, qasm::OutputValue::Value(_, x) => size(x), }).sum::(); let repaired = r.repaired.as_ref().map_or(0, size); - b += repaired + r.compiled.as_ref().map_or(0, size) + r.compile_passes.as_ref().map_or(0, size); - b += p.target.as_ref().map_or(0, size); + rest += repaired + r.compiled.as_ref().map_or(0, size) + r.compile_passes.as_ref().map_or(0, size); + rest += p.target.as_ref().map_or(0, size); if r.mitigate.is_some() { - b += MITIGATION_BYTES + r.observables.len() as u64 * EXPECTATION_BYTES + super::run::counts_bound(bits, r.shots, Some(24)); + rest += MITIGATION_BYTES + r.observables.len() as u64 * EXPECTATION_BYTES; + counts = counts.saturating_add(super::run::counts_bound_among(bits, samples, outcomes, Some(24))); } + let mut verdict = 0u64; if super::verdict::asked(v) { let intents: Vec<&Value> = match v.get("intent") { Some(Value::Array(a)) => a.iter().collect(), Some(i) if !i.is_null() => vec![i], _ => Vec::new(), }; - let listed = super::verdict::MAX_LISTED as u64 * (u64::from(bits) + 4 + digits); - b += VERDICT_BYTES + repaired + intents.iter().map(|i| INTENT_BYTES + 2 * size(i) + listed).sum::(); - let gates = u64::try_from(r.program.standard_gates()).unwrap_or(u64::MAX); - b = b.saturating_add(gates.saturating_mul(REMARKS_PER_GATE)).saturating_add(u64::from(r.program.n_qubits) * REMARKS_PER_GATE); + let listed = super::verdict::MAX_LISTED as u64 * (u64::from(bits) + LISTED_BYTES); + let each = |i: &Value| INTENT_BYTES + 2 * size(i) + LISTS_PER_INTENT * listed; + // The statement says the reason of the first that fails and of the + // first left unchecked again. + let said = intents.iter().map(|i| each(i)).max().unwrap_or(0); + verdict = VERDICT_BYTES + repaired + notes + r.smells_bytes; + verdict = verdict.saturating_add(intents.iter().map(|i| each(i)).sum::()).saturating_add(2 * said); } - b.saturating_add(b / 16).saturating_add(counts) + let rest = rest.saturating_add((rest + notes + verdict) / 16); + Bound { counts, verdict, notes, requests: 0, rest } } /// The method `v` names, when it reads its job's program itself (a pulse @@ -1339,10 +1468,15 @@ pub(crate) fn plan_in(reg: &Registry, v: &Value, job_bytes: u64) -> Result Result<(Plan, Option), String> { +/// [`plan_bound`] through `reg`. +pub(crate) fn plan_bounded_in(reg: &Registry, v: &Value, job_bytes: u64) -> Result<(Plan, Option), String> { if let Some(m) = reads_own(reg, v) { - return m.plan_bounded(&Job::new(reg, v, job_bytes, None, &[])); + // The built-in pulse run bounds its result by part; another method + // that reads its own program, as one sum. + if m.name() == "pulse" && reg.identity(super::registry::Kind::Method, "pulse").is_some_and(|i| i.is_builtin()) { + return super::pulse::plan_bound(v, job_bytes).map(|(p, b)| (p, Some(b))); + } + return m.plan_bounded(&Job::new(reg, v, job_bytes, None, &[])).map(|(p, b)| (p, b.map(|rest| Bound { rest, ..Bound::default() }))); } // A job that names a device target is compiled, placed and simulated as // the device would run it (engine::targets::job). diff --git a/wai/wai-quantum/src/engine/critique.rs b/wai/wai-quantum/src/engine/critique.rs index b18d693e6..d7e2fc8f5 100644 --- a/wai/wai-quantum/src/engine/critique.rs +++ b/wai/wai-quantum/src/engine/critique.rs @@ -313,6 +313,84 @@ fn qubit_list(qs: &[u32]) -> String { } } +/// A smell's place in the program's ops: an op that does nothing, or two in a +/// row that cancel or merge. +enum Site { + Nothing(usize), + Cancels(usize, usize), + Merges(usize, usize), +} + +/// Each smell of the gates of `p` as it is found, in order: the walk +/// [`smells`] lists and [`smells_bound`] bounds, so the two never differ. +fn sites(p: &Program, mut found: impl FnMut(Site)) { + let n = p.n_qubits as usize; + // The order the ops run in, with a barrier (None) at each measurement, + // reset or condition of a program that measures in the middle. + let order: Box>> = match &p.dynamic { + None => Box::new((0..p.ops.len()).map(Some)), + Some(steps) => Box::new(steps.iter().map(|s| if let Instr::Gate(k) = s { Some(*k) } else { None })), + }; + let mut last: Vec> = vec![None; n]; + let mut used = vec![false; p.ops.len()]; + for step in order { + let Some(j) = step else { + last.iter_mut().for_each(|l| *l = None); + continue; + }; + let b = &p.ops[j]; + if does_nothing(b) { + used[j] = true; + found(Site::Nothing(j)); + } + let prev: Vec> = b.qubits.iter().map(|&q| last[q as usize]).collect(); + if let Some(Some(i)) = prev.first().copied() { + let a = &p.ops[i]; + let adjacent = prev.iter().all(|x| *x == Some(i)) && a.qubits.len() == b.qubits.len() && !used[i] && !used[j]; + if adjacent && cancels(a, b) { + used[i] = true; + used[j] = true; + found(Site::Cancels(i, j)); + } else if adjacent && merges(a, b) { + used[i] = true; + used[j] = true; + found(Site::Merges(i, j)); + } + } + for &q in &b.qubits { + last[q as usize] = Some(j); + } + } +} + +/// What one smell takes as a verdict lists it (`{"kind", "message", +/// "lines"}`), bytes, at most, beside the names of its qubits: its kind, the +/// longest message's words and gate names, and three line numbers of ten +/// digits. +const SMELL_BYTES: u64 = 256; +/// A qubit's index in a smell's message, at most: ten digits and ", ". +const QUBIT_BYTES: u64 = 12; + +/// The most the smells of `src` take as a verdict lists them, bytes, where +/// `p` is `src` as read: each smell the walk of its gates finds +/// ([`SMELL_BYTES`] and its qubits' names), and the idle and unmeasured +/// qubits, at most every qubit each. A program's critique grows with its +/// smells, not with its gates: a loop of a hundred thousand gates that +/// cancel nowhere has none. +pub fn smells_bound(p: &Program) -> u64 { + let arity = |k: usize| p.ops[k].qubits.len() as u64; + let mut b = 0u64; + sites(p, |s| { + b += SMELL_BYTES + + QUBIT_BYTES + * match s { + Site::Nothing(j) => arity(j), + Site::Cancels(i, _) | Site::Merges(i, _) => arity(i), + }; + }); + b + 2 * (SMELL_BYTES + QUBIT_BYTES * u64::from(p.n_qubits)) +} + fn smells(src: &str, p: &Program, lines: &[usize]) -> Vec { let text: Vec<&str> = src.split('\n').collect(); let n = p.n_qubits as usize; @@ -329,22 +407,9 @@ fn smells(src: &str, p: &Program, lines: &[usize]) -> Vec { statement(text.get(l.wrapping_sub(1))?) }; let mut out = Vec::new(); - // The order the ops run in, with a barrier (None) at each measurement, - // reset or condition of a program that measures in the middle. - let order: Vec> = match &p.dynamic { - None => (0..p.ops.len()).map(Some).collect(), - Some(steps) => steps.iter().map(|s| if let Instr::Gate(k) = s { Some(*k) } else { None }).collect(), - }; - let mut last: Vec> = vec![None; n]; - let mut used = vec![false; p.ops.len()]; - for step in order { - let Some(j) = step else { - last.iter_mut().for_each(|l| *l = None); - continue; - }; - let b = &p.ops[j]; - if does_nothing(b) { - used[j] = true; + sites(p, |site| match site { + Site::Nothing(j) => { + let b = &p.ops[j]; out.push(Smell { kind: "does_nothing", message: format!("{} on {} (line {}) does nothing, up to a global phase", kind_name(b.kind), qubit_list(&b.qubits), lines[j]), @@ -354,43 +419,34 @@ fn smells(src: &str, p: &Program, lines: &[usize]) -> Vec { fix: alone(j).map(|_| vec![Edit::Delete(lines[j])]), }); } - let prev: Vec> = b.qubits.iter().map(|&q| last[q as usize]).collect(); - if let Some(Some(i)) = prev.first().copied() { + Site::Cancels(i, j) => { + let (a, b) = (&p.ops[i], &p.ops[j]); + let what = if a.kind == b.kind { format!("{} on {} twice in a row", kind_name(a.kind), qubit_list(&a.qubits)) } else { format!("{} then {} on {}", kind_name(a.kind), kind_name(b.kind), qubit_list(&a.qubits)) }; + let fix = (alone(i).is_some() && alone(j).is_some() && lines[i] != lines[j]).then(|| vec![Edit::Delete(lines[i]), Edit::Delete(lines[j])]); + out.push(Smell { kind: "cancels", message: format!("{what} (lines {} and {}): together they do nothing", lines[i], lines[j]), lines: vec![lines[i], lines[j]], ops: vec![i, j], qubits: a.qubits.clone(), fix }); + } + Site::Merges(i, j) => { let a = &p.ops[i]; - let adjacent = prev.iter().all(|x| *x == Some(i)) && a.qubits.len() == b.qubits.len() && !used[i] && !used[j]; - if adjacent && cancels(a, b) { - used[i] = true; - used[j] = true; - let what = if a.kind == b.kind { format!("{} on {} twice in a row", kind_name(a.kind), qubit_list(&a.qubits)) } else { format!("{} then {} on {}", kind_name(a.kind), kind_name(b.kind), qubit_list(&a.qubits)) }; - let fix = (alone(i).is_some() && alone(j).is_some() && lines[i] != lines[j]).then(|| vec![Edit::Delete(lines[i]), Edit::Delete(lines[j])]); - out.push(Smell { kind: "cancels", message: format!("{what} (lines {} and {}): together they do nothing", lines[i], lines[j]), lines: vec![lines[i], lines[j]], ops: vec![i, j], qubits: a.qubits.clone(), fix }); - } else if adjacent && merges(a, b) { - used[i] = true; - used[j] = true; - let fix = match (alone(i), alone(j)) { - (Some(sa), Some(sb)) if lines[i] != lines[j] && sa.name == sb.name => match (&sa.params, &sb.params) { - (Some(ea), Some(eb)) if !ea.is_empty() && !eb.is_empty() => Some(vec![ - Edit::Replace(lines[i], format!("{}{}({ea} + {eb}) {};{}", sa.indent, sa.name, sa.args, if sa.comment.is_empty() { String::new() } else { format!(" {}", sa.comment) })), - Edit::Delete(lines[j]), - ]), - _ => None, - }, + let fix = match (alone(i), alone(j)) { + (Some(sa), Some(sb)) if lines[i] != lines[j] && sa.name == sb.name => match (&sa.params, &sb.params) { + (Some(ea), Some(eb)) if !ea.is_empty() && !eb.is_empty() => Some(vec![ + Edit::Replace(lines[i], format!("{}{}({ea} + {eb}) {};{}", sa.indent, sa.name, sa.args, if sa.comment.is_empty() { String::new() } else { format!(" {}", sa.comment) })), + Edit::Delete(lines[j]), + ]), _ => None, - }; - out.push(Smell { - kind: "merges", - message: format!("{} on {} twice in a row (lines {} and {}): one {} with the angles added", kind_name(a.kind), qubit_list(&a.qubits), lines[i], lines[j], kind_name(a.kind)), - lines: vec![lines[i], lines[j]], - ops: vec![i, j], - qubits: a.qubits.clone(), - fix, - }); - } + }, + _ => None, + }; + out.push(Smell { + kind: "merges", + message: format!("{} on {} twice in a row (lines {} and {}): one {} with the angles added", kind_name(a.kind), qubit_list(&a.qubits), lines[i], lines[j], kind_name(a.kind)), + lines: vec![lines[i], lines[j]], + ops: vec![i, j], + qubits: a.qubits.clone(), + fix, + }); } - for &q in &b.qubits { - last[q as usize] = Some(j); - } - } + }); // Qubits nothing uses, and qubits acted on but never measured. let mut touched = vec![false; n]; for op in &p.ops { diff --git a/wai/wai-quantum/src/engine/mod.rs b/wai/wai-quantum/src/engine/mod.rs index 0338ed620..325367934 100644 --- a/wai/wai-quantum/src/engine/mod.rs +++ b/wai/wai-quantum/src/engine/mod.rs @@ -194,6 +194,8 @@ pub mod algorithms; #[cfg(feature = "quantum_receipt")] pub mod answer; pub mod catalog; +#[cfg(test)] +mod bound_tests; pub mod compile; #[cfg(feature = "quantum_receipt")] pub mod converge; diff --git a/wai/wai-quantum/src/engine/pulse.rs b/wai/wai-quantum/src/engine/pulse.rs index 00dd6e625..bfae29c8c 100644 --- a/wai/wai-quantum/src/engine/pulse.rs +++ b/wai/wai-quantum/src/engine/pulse.rs @@ -443,6 +443,11 @@ pub fn plan(v: &Value, job_bytes: u64) -> Result { /// bytes, known before it runs, from the same reading: as /// [`super::catalog::plan_bounded`] gives a circuit's. pub fn plan_bounded(v: &Value, job_bytes: u64) -> Result<(Plan, u64), String> { + plan_bound(v, job_bytes).map(|(p, b)| (p, b.bytes())) +} + +/// [`plan_bounded`], the bound given by part. +pub fn plan_bound(v: &Value, job_bytes: u64) -> Result<(Plan, super::catalog::Bound), String> { let (_, p) = prepare(v)?; let plan = plan_of(&p, job_bytes)?; let bound = result_bound(&p, &plan); @@ -473,8 +478,8 @@ const CAPTURE_BYTES: u64 = 192; /// in the result and in its verdict; the device as read and the target; and /// the rest at [`RESULT_REST`], a sixteenth more for the forms RFC 8785 /// writes a number in. -fn result_bound(p: &Prepared, plan: &Plan) -> u64 { - let size = |x: &Value| serde_json::to_string(x).map_or(0, |s| s.len() as u64); +fn result_bound(p: &Prepared, plan: &Plan) -> super::catalog::Bound { + let size = super::receipt::canonical_len; // A name of the program's own as JSON text: RFC 8785 escapes a string's // characters as serde does (`receipt::write_str`). let text = |t: &str| serde_json::to_string(t).map_or(6 * t.len() as u64 + 2, |s| s.len() as u64); @@ -494,19 +499,21 @@ fn result_bound(p: &Prepared, plan: &Plan) -> u64 { (0, 0) }; let entry = |w: u64| w + 4 + NUMBER; + // The populations, the distribution and the counts: what grows with the outcomes. + let mut counts = pow2(n).saturating_mul(entry(n)); + counts = counts.saturating_add(outcomes.saturating_mul(entry(width))); + counts = counts.saturating_add(super::run::counts_bound_among(u32::try_from(width).unwrap_or(u32::MAX), p.shots, outcomes, None)); let mut b = RESULT_REST + 2 * size(&json!(plan.why)); - b = b.saturating_add(pow2(n).saturating_mul(entry(n))); - b = b.saturating_add(outcomes.saturating_mul(entry(width))); - b = b.saturating_add(super::run::counts_bound_among(u32::try_from(width).unwrap_or(u32::MAX), p.shots, outcomes, None)); b = b.saturating_add(16 * n + s.bits.iter().map(|x| text(x) + 1).sum::() + 16 * caps.len() as u64); b = b.saturating_add(caps.iter().map(|i| CAPTURE_BYTES + text(&s.frames[i.frame].name) + match &i.op { Op::Capture { bit: Some(k), .. } => s.bits.get(*k).map_or(0, |x| text(x)), _ => 0, }).sum::()); b = b.saturating_add(n * (NUMBER + 32)); - b = b.saturating_add(2 * p.reading.diagnostics.iter().map(|d| size(&d.to_json()) + 1).sum::()); + let notes = 2 * p.reading.diagnostics.iter().map(|d| size(&d.to_json()) + 1).sum::(); b = b.saturating_add(size(&p.device_json) + p.target.as_ref().map_or(0, size)); - b.saturating_add(b / 16) + let rest = b.saturating_add(b.saturating_add(notes) / 16); + super::catalog::Bound { counts, notes, rest, ..Default::default() } } /// [`plan`] of the job read as `p`. diff --git a/wai/wai-quantum/src/engine/receipt.rs b/wai/wai-quantum/src/engine/receipt.rs index a699405c3..002a8992c 100644 --- a/wai/wai-quantum/src/engine/receipt.rs +++ b/wai/wai-quantum/src/engine/receipt.rs @@ -188,6 +188,54 @@ fn write(v: &Value, o: &mut Vec) { } } +/// The length of `v` as RFC 8785 text, bytes, written nowhere: its length as +/// [`canonical`] writes it, or as a host writes it with each whole number +/// exactly as read and every other number as [`canonical`] does (a job +/// service's served text), whichever is longer. A number's length is its own +/// form's, never serde's (`1e20` is 21 characters here, 4 there), so this is +/// what a result that copies a request takes as served. +pub fn canonical_len(v: &Value) -> u64 { + let mut seen: std::collections::HashMap = std::collections::HashMap::new(); + let mut number = |n: &serde_json::Number| -> u64 { + let x = n.as_f64().unwrap_or(f64::NAN); + let es = *seen.entry(x.to_bits()).or_insert_with(|| es_number(x).len() as u64); + // A whole number written as read takes its own digits. + let exact = if n.is_f64() { 0 } else { n.to_string().len() as u64 }; + es.max(exact) + }; + let mut total = 0u64; + let mut stack = vec![v]; + while let Some(v) = stack.pop() { + total += match v { + Value::Null => 4, + Value::Bool(b) => if *b { 4 } else { 5 }, + Value::Number(n) => number(n), + Value::String(s) => str_len(s), + Value::Array(a) => { + stack.extend(a.iter()); + 2 + a.len().saturating_sub(1) as u64 + } + Value::Object(m) => { + stack.extend(m.values()); + 2 + m.len().saturating_sub(1) as u64 + m.keys().map(|k| str_len(k) + 1).sum::() + } + }; + } + total +} + +/// A string's length as [`write_str`] writes it, bytes. +fn str_len(s: &str) -> u64 { + 2 + s + .chars() + .map(|c| match c { + '"' | '\\' | '\u{8}' | '\t' | '\n' | '\u{c}' | '\r' => 2, + c if (c as u32) < 0x20 => 6, + c => c.len_utf8() as u64, + }) + .sum::() +} + /// A string as RFC 8785 writes it: `"` and `\\` escaped, the five short /// control escapes, every other control character as `\u00xx`, and the rest /// as it is. diff --git a/wai/wai-quantum/src/engine/run.rs b/wai/wai-quantum/src/engine/run.rs index 5a50ad2d0..e6d5011b7 100644 --- a/wai/wai-quantum/src/engine/run.rs +++ b/wai/wai-quantum/src/engine/run.rs @@ -15,19 +15,13 @@ fn hex(b: &[u8]) -> String { const MAX_COUNTS_CHARS: usize = 16 << 20; /// The most a counts map ([`counts`]) of `shots` samples of `width`-bit -/// outcomes takes as JSON text, bytes: as many outcomes as it may list (no -/// more than the shots, `2^width`, [`MAX_OUTCOMES`] and what -/// [`MAX_COUNTS_CHARS`] of bit strings hold), each its bit string, two -/// quotes, a colon and a comma, and the map's own fields; with `value`, that -/// many bytes for each outcome's value, and without, its count's digits. `k` -/// counts that sum to `shots` have at most `k·(log10(shots/k) + 1)` digits -/// in all, the most where they are equal. -pub(crate) fn counts_bound(width: u32, shots: u64, value: Option) -> u64 { - counts_bound_among(width, shots, if width >= 63 { u64::MAX } else { 1u64 << width }, value) -} - -/// [`counts_bound`] where at most `distinct` outcomes can occur (a pulse -/// run's, which reads fewer qubits than its keys are wide). +/// outcomes, at most `distinct` of them possible, takes as JSON text, bytes: +/// as many outcomes as it may list (no more than the shots, `distinct`, +/// [`MAX_OUTCOMES`] and what [`MAX_COUNTS_CHARS`] of bit strings hold), each +/// its bit string, two quotes, a colon and a comma, and the map's own +/// fields; with `value`, that many bytes for each outcome's value, and +/// without, its count's digits. `k` counts that sum to `shots` have at most +/// `k·(log10(shots/k) + 1)` digits in all, the most where they are equal. pub(crate) fn counts_bound_among(width: u32, shots: u64, distinct: u64, value: Option) -> u64 { if shots == 0 || distinct == 0 { return 0; diff --git a/wai/wai-quantum/src/engine/sets.rs b/wai/wai-quantum/src/engine/sets.rs index 92381147c..739514304 100644 --- a/wai/wai-quantum/src/engine/sets.rs +++ b/wai/wai-quantum/src/engine/sets.rs @@ -216,9 +216,9 @@ pub fn plan(items: &[Item], job_bytes: u64) -> Result, String plan_bounded(items, job_bytes).map(|planned| planned.into_iter().map(|(p, _)| p).collect()) } -/// [`plan`], each item's plan with the most its result may take -/// ([`catalog::plan_bounded`]). -pub fn plan_bounded(items: &[Item], job_bytes: u64) -> Result)>, String> { +/// [`plan`], each item's plan with the most its result may take, by part +/// ([`catalog::plan_bound`]). +pub fn plan_bounded(items: &[Item], job_bytes: u64) -> Result)>, String> { items .iter() .enumerate() @@ -226,7 +226,7 @@ pub fn plan_bounded(items: &[Item], job_bytes: u64) -> Result f64 { gamma_q(k as f64 / 2.0, x / 2.0) } +/// The most outcomes of probability 0 an answer lists by name: past them it +/// gives how many there were ([`goodness_of_fit`]), so what it says stays +/// within a bound however many outcomes a run gave. +pub const MAX_IMPOSSIBLE_LISTED: usize = 64; + /// Whether `observed` counts are consistent with `expected` probabilities, /// by `test` (`chi_squared` or `g_test`) at significance `alpha`: /// `{"test", "statistic", "dof", "p_value", "alpha", "shots", "rejected", -/// "smallest_expected", "impossible", "note"}`. Rejected when the p-value is -/// below `alpha`, or when an outcome of probability 0 was sampled. +/// "smallest_expected", "impossible", "impossible_outcomes", "note"}`. +/// Rejected when the p-value is below `alpha`, or when an outcome of +/// probability 0 was sampled: `impossible` lists the first +/// [`MAX_IMPOSSIBLE_LISTED`] of those, in order, and `impossible_outcomes` +/// says how many there were. pub fn goodness_of_fit(test: &str, observed: &BTreeMap, expected: &BTreeMap, alpha: f64) -> Result { if !TESTS.contains(&test) { return Err(format!("{test}: the tests are {}", TESTS.join(" and "))); @@ -145,10 +153,15 @@ pub fn goodness_of_fit(test: &str, observed: &BTreeMap, expected: & String::new() }; let name = if test == "chi_squared" { "Pearson's chi-squared test" } else { "the G-test" }; - let note = if impossible.is_empty() { + let all = impossible.len(); + let mut impossible = impossible; + impossible.truncate(MAX_IMPOSSIBLE_LISTED); + let note = if all == 0 { format!("{name} of {shots} samples against the stated distribution, {dof} degrees of freedom{law}") } else { - format!("{} sampled, which the stated distribution gives probability 0", impossible.iter().map(|s| s.as_str()).collect::>().join(", ")) + let listed = impossible.iter().map(|s| s.as_str()).collect::>().join(", "); + let more = if all > impossible.len() { format!(" and {} more", all - impossible.len()) } else { String::new() }; + format!("{listed}{more} sampled, which the stated distribution gives probability 0") }; Ok(json!({ "test": test, @@ -160,6 +173,7 @@ pub fn goodness_of_fit(test: &str, observed: &BTreeMap, expected: & "rejected": rejected, "smallest_expected": if smallest.is_finite() { json!(smallest) } else { Value::Null }, "impossible": impossible, + "impossible_outcomes": all, "note": note, })) } diff --git a/wai/wai-quantum/src/engine/verdict.rs b/wai/wai-quantum/src/engine/verdict.rs index 44abefc91..5bc66e0c1 100644 --- a/wai/wai-quantum/src/engine/verdict.rs +++ b/wai/wai-quantum/src/engine/verdict.rs @@ -387,8 +387,10 @@ pub(crate) fn check_intent_for_in(reg: &super::registry::Registry, intent: &Valu let note = g["note"].as_str().unwrap_or("").to_string(); let on = format!("{shots} samples (seed {seed}, on {})", plan.method); let (outcome, strength, why) = if g["rejected"] == true { - let reason = if g["impossible"].as_array().is_some_and(|a| !a.is_empty()) { note.clone() } else { format!("p = {p:.3e} < α = {alpha}") }; - ("fails", "evidence", format!("{on} are not consistent with the intent's distribution: {reason}; {note}")) + // An outcome of probability 0 sampled is the reason, and the note says + // it; a p-value under α is the reason beside the note's test. + let why = if g["impossible_outcomes"].as_u64().is_some_and(|n| n > 0) { format!("{on} are not consistent with the intent's distribution: {note}") } else { format!("{on} are not consistent with the intent's distribution: p = {p:.3e} < α = {alpha}; {note}") }; + ("fails", "evidence", why) } else if proof_only { ("unchecked", "none", format!("{on} are consistent with the intent's distribution (p = {p:.4} ≥ α = {alpha}), but a statistical test is evidence and the intent requires a proof")) } else { -- 2.54.0 (Apple Git-157) From da91125fcd410ca9df6cba69a4ddfa8f2e5955e9 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Mon, 9 Nov 2026 13:21:12 -0500 Subject: [PATCH 17/18] =?UTF-8?q?wai:=20the=20compose=20fixture=20is=20'a?= =?UTF-8?q?=20test=20pattern=20of=20320=20=C3=97=20180',=20which=20the=20v?= =?UTF-8?q?anity=20check=20does=20not=20read=20as=20a=20count=20of=20tests?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 5df961d52 wrote 'a 320 × 180 test pattern' in browser-sink.md §14.6 and compose/README.md; scripts/check-no-vanity-metrics.sh reads '180 test' as a test count, which turned repository checks red on main (run 26220 at 1929b8436). Same words, reordered. Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 4c5b54a272e120171575273798324b5d07bdfd38) --- wai/extensions/browser-sink.md | 2 +- wai/wai-web/compose/README.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/wai/extensions/browser-sink.md b/wai/extensions/browser-sink.md index 673ee6223..92a6e7f5c 100644 --- a/wai/extensions/browser-sink.md +++ b/wai/extensions/browser-sink.md @@ -844,7 +844,7 @@ package of several components: the media classes `graphics` (7) and `text` (8) beside packaging's others, the rungs each medium takes, how each component is cut at the lead's piece starts, each piece's object and `target`, and the decoded-output digest each entry states. The reference package is written by -`wai-pack` under `compose/policy.toml`: the lead `v`, a 320 × 180 test pattern (the fixture +`wai-pack` under `compose/policy.toml`: the lead `v`, a test pattern of 320 × 180 (the fixture `compose-01`) with an integer base and a supplied AV1 floor for every piece; `g`, `vector-conformance`'s title card and credit roll; `t`, three captions; `s`, a score with a FLAC floor of each piece's mixdown, which this page does not decode (no route: it holds no diff --git a/wai/wai-web/compose/README.md b/wai/wai-web/compose/README.md index 97312aed0..a45fd8167 100644 --- a/wai/wai-web/compose/README.md +++ b/wai/wai-web/compose/README.md @@ -20,7 +20,7 @@ cut at the lead's piece starts, each piece one object: | component | capability | source | presented as | |---|---|---|---| -| `v` | `wai.video.int_motion` bases and a `wai.video.av1` floor for every piece | the fixture `compose-01`, a 320 × 180 test pattern at 25 frames a second | the picture, and the clock | +| `v` | `wai.video.int_motion` bases and a `wai.video.av1` floor for every piece | the fixture `compose-01`, a test pattern of 320 × 180 at 25 frames a second | the picture, and the clock | | `g` | `wai.graphics.vector` | `vector-conformance/vectors/motion/stroke-titles.wvg1`, a title card and a credit roll at 1280 × 720 | an overlay rasterised in WebAssembly | | `t` | `wai.text.timed` | `sources/t.vtt`, three captions | text, in the page's text layer | | `s` | `wai.audio.score`, and a `wai.audio.flac` floor of each piece's mixdown | `sources/s.wscr`, `score-conformance/vectors/scale.wscr`'s notes up and down | samples synthesised in the tab, on Web Audio | -- 2.54.0 (Apple Git-157) From b35ae093e69807441ad128ffaae2da99eaff9c92 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Mon, 9 Nov 2026 14:08:25 -0500 Subject: [PATCH 18/18] wai-quantum: the change's entry acknowledges Request::smells_bytes and the two reader constants of main's it replaces `wai breaking --against 1929b8436` named three changes: the public field a verdict's critique bound is carried in, and main's `qasm::MAX_NESTING` and `qasm::MAX_ELSE_IF` (15ae7d457, in no release), which this branch's reader replaces with `qasm::parse::MAX_NESTING` and flat `else if` ladders. Each is intended, and the entry says what a user does. Co-Authored-By: Claude Opus 5.5 --- wai/changes/2026-11-06-quantum-parse-depth.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/wai/changes/2026-11-06-quantum-parse-depth.md b/wai/changes/2026-11-06-quantum-parse-depth.md index 7843ac685..157feadb3 100644 --- a/wai/changes/2026-11-06-quantum-parse-depth.md +++ b/wai/changes/2026-11-06-quantum-parse-depth.md @@ -178,3 +178,12 @@ - **Breaking:** `api:rust:wai_quantum::qasm::ast::S::If`: an `if` and its `else if`s are one statement, `S::If { arms: [(cond, then), …], other }`, where it was `S::If { cond, then, other }` with the `else if`s nested in `other`; code that builds or destructures it reads each arm in turn. +- **Breaking:** `api:rust:wai_quantum::engine::catalog::Request::smells_bytes`: a request read for a verdict carries + the most its critique takes, so a result's bound counts the smells its program can draw rather than every gate; + code outside the crate that builds a `Request` by its fields sets it to 0 (or to `critique::smells_bound` of its + program, where it asks for a verdict). +- **Breaking:** `api:rust:wai_quantum::qasm::MAX_NESTING`: main's bound of 128 levels (a `u32`, from 15ae7d457, in no + release) is replaced by this reader's, `qasm::parse::MAX_NESTING` (64 levels in the text, a `usize`, and 256 through + calls); read that constant instead. +- **Breaking:** `api:rust:wai_quantum::qasm::MAX_ELSE_IF`: an `else if` ladder is read flat, at any length, so it has no + bound to name (main's 1024, from 15ae7d457, in no release); code that held a ladder to it holds nothing now. -- 2.54.0 (Apple Git-157)