open-standards/wai/wai-quantum-service/docs/self-host.md

Running the quantum job service yourself

The job service at https://wai.transaction.science/quantum-jobs is free and takes no account and no key. You can also run the same service on your own machine or in your own network, for a lab, a course, or work that should not leave your premises. This page says how: the binary or the image, its configuration, where it keeps jobs, its keys, and how to put it behind a reverse proxy.

Run it

From a checkout of the repository, with the toolchain rust-toolchain.toml names:

cd wai/wai-quantum-service
cargo build --locked --release
./target/release/wai-quantum-service serve          # 127.0.0.1:8241
bash smoke.sh http://127.0.0.1:8241                  # its promises, checked over HTTP

Or as an OCI image, built from the repository's wai/ directory (deploy/Containerfile):

docker build -f wai/wai-quantum-service/deploy/Containerfile -t wai-quantum-service wai/
docker run -d --name wai-quantum -p 127.0.0.1:8241:8241 \
  -v wai-quantum:/var/lib/wai-quantum-service \
  --read-only --tmpfs /tmp --cap-drop ALL --security-opt no-new-privileges \
  wai-quantum-service

podman takes the same commands. The image's build stage and run stage are each pinned by digest, the build uses the locked dependencies with paths remapped and the timestamp fixed, and the run stage holds the binary alone, run as an unprivileged user. bash wai/wai-quantum-service/deploy/check-image.sh builds the image, runs it, holds it to the smoke checks, restarts it to check that it keeps its jobs, and removes everything it made. It pushes nothing.

Each job runs in a child process of the same binary, at a lower scheduling priority than the host's other work, watched for its memory and its time. On macOS the child also runs in a sandbox with no network, no file writes and no other programs. Elsewhere, run the service in a container (as above) or under a service manager that confines it: it needs to listen on its port and to write its store, and nothing else.

Configure it

Everything is set by environment variables.

variable	default	meaning
WAI_QSVC_ADDR	127.0.0.1:8241	the address it listens on (0.0.0.0:8241 in the image)
WAI_QSVC_PUBLIC_BASE	the public service's	the URL it is reached at, for the links it hands out
WAI_QSVC_CONSOLE	the public console	the page a browser at / is sent to
WAI_QSVC_STORE	~/.local/share/wai-quantum-service/store	the job store's directory, or memory to keep jobs only while the process runs
WAI_QSVC_RETAIN_SECS	86400	how long a finished job is kept
WAI_QSVC_KEY	~/.config/wai-quantum-service/signing.key	the receipt key, created on first start
WAI_QSVC_API_KEYS	unset (no keys)	a file of API keys: set it and every request needs one
WAI_QSVC_TRUSTED_PROXIES	127.0.0.1,::1	peers whose X-Forwarded-For names the client
WAI_QSVC_ENERGY	1	0 meters no job's energy
WAI_QSVC_BUDGET_GB	96	the memory running jobs share
WAI_QSVC_JOB_GB	72	the memory one job may take
WAI_QSVC_WORKERS	4	jobs running at once
WAI_QSVC_MIN_FREE_PCT	15	the share of the host's memory a starting job must leave free
WAI_QSVC_JOB_SECS	900	the longest a job (or a whole set) may run
WAI_QSVC_CLIENT_ACTIVE	3	queued or running jobs one client may hold
WAI_QSVC_CLIENT_GB	the job size	the memory one client's jobs may hold together
WAI_QSVC_CLIENT_BURST	30	submissions one client may make in a window
WAI_QSVC_CLIENT_WINDOW_SECS	600	the window
WAI_QSVC_MAX_QUEUE	256	jobs waiting at once
WAI_QSVC_SANDBOX	1	0 runs the macOS worker unsandboxed, for debugging

Size the memory to the host: WAI_QSVC_BUDGET_GB below the host's memory, WAI_QSVC_JOB_GB at or below it. GET /v1 states every limit in force, and a dry run ("dry_run": true) states the ones a job meets.

The job store

Every job is written to the store before the request that admitted it is answered, and again at every change of its status. Programs, requests, results and receipts are kept content-addressed, each under the BLAKE3 digest of its bytes: a program's text, or the RFC 8785 canonical text of a request, a result without its receipt, or a receipt. A job's report names them under digests.

<store>/LOCK                    held while a process serves the store
<store>/client.key              the key client names are hashed with
<store>/objects/<ab>/<digest>   each object, named by its digest
<store>/jobs/<id>.json          each job's record

• Writes are whole. Every file is written under a temporary name, synced, renamed into place, and its directory synced, so a crash leaves the old file or the new one. A second process on the same store is refused.
• A restart loses nothing it answered. On start the service reloads the store. A finished job is served as it was, and its receipt still verifies. A job that was running comes back interrupted, with the reason; its request is kept, and sent again it runs to the same result. A queued job is queued again, in its order. A worker whose service has gone stops itself.
• Objects are checked. An object is checked against its digest when it is read, and a record that does not read is set aside, not served.
• Retention. A finished job is kept for WAI_QSVC_RETAIN_SECS, then dropped with every object no other job names.
• No addresses. A record names its client by a keyed hash of the client's address, so it can count the client's limits after a restart and says nothing else.
• Backups. Copy the store's directory while the service is stopped, or, while it runs, copy jobs/ first and objects/ after it: an object is never changed once written, and a record names only objects written before it.

Another kind of store is anything that implements the Store trait in src/store.rs: keep an object by its digest, read it back checked, keep and list the records, and sweep what no record names.

The receipt keys

Every result is sealed with the service's Ed25519 key, and the receipt names that key (key, in hex). The key set, every key the service has signed with, is published at GET /.well-known/wai-quantum-receipt-keys. It sits beside the key file as <key>.keys.json.

{ "format": "wai.quantum.receipt-keys/1", "algorithm": "ed25519",
  "keys": [
    { "key": "9f…", "did": "did:key:z6Mk…", "not_before_ms": null, "not_after_ms": 1793000000000,
      "successor": "4c…", "current": false,
      "transition": { "statement": { "format": "wai.quantum.key-transition/1", "retiring": "9f…", "successor": "4c…", "at_ms": 1793000000000 },
                      "retiring_signature": "…", "successor_signature": "…" } },
    { "key": "4c…", "did": "did:key:z6Mk…", "not_before_ms": 1793000000000, "current": true } ] }

To rotate the key, run wai-quantum-service rotate-key with the same WAI_QSVC_KEY, then restart the service. The rotation does four things:

• It retires the current key. Its entry stays in the set with its window closed (not_after_ms), so receipts it signed still verify and can be placed in time.
• It makes a new key, whose window opens at the same moment.
• It writes a transition statement naming both keys and the moment, signed by each under the domain wai:quantum-key-transition (version 1). Whoever trusted the old key can check that the new one came from its holder.
• It moves the old private key aside to <key>.retired-<ms>. The service never reads that file again, and you may destroy it.

not_before_ms is null for a key that signed before the set was kept. A key file replaced by hand is taken as current at the next start, and the old key is retired with no transition, which its entry says. The service refuses to start on a set whose recorded transition does not verify. A result itself carries no wall-clock time, so a receipt is placed in a key's window by the job's finished_ms or by your own record of when you received it.

API keys, for a private deployment

The public service takes no key, and nothing changes for it. A private deployment can require one. Make a key for each holder:

wai-quantum-service api-key lab-scheduler
# {"name": "lab-scheduler", "key": "3b9…", "line": "lab-scheduler 7f0…", …}

Hand key to its holder, and add line to the file WAI_QSVC_API_KEYS names. The file holds each key's BLAKE3 digest, never the key. With the variable set:

• every request but the open ones carries Authorization: Bearer <key>, or is refused with 401 and J033. The open ones are /, /v1, /v1/health, /v1/methods, /v1/openapi.json and the receipt keys.
• a client's limits are counted by its key's name, not its address, so one gateway or scheduler is one client wherever it connects from.

The command line sends a key with --api-key or WAI_QUANTUM_API_KEY.

Energy

Each finished job carries its energy inside the signed result: a counter reading with its class, its scope and its declared uncertainty, or {"unmetered": "<why>"}, never zero. GET /v1 says how this host meters (energy).

• macOS: the kernel's count of the worker process's own CPU energy (attribution: process).
• Linux: the processor packages' energy counters, read through the kernel's power-capping interface before the worker starts and after it exits. They count whole packages, so the figure is the packages' energy over the job's window, every process on the host included, with no attribution to the job (attribution: none). It names how many other jobs ran in the window (concurrent_jobs). The counter is used only when every package zone reads and its count is live; its declared uncertainty covers the two reads and the counter's update edges, measured at start.
• Permissions: many Linux hosts let only root, or a group such as energy, read the count. Add the service's user to that group. In a container the interface is usually hidden; a container that should meter needs it visible and readable, which widens what the container sees of the host. Without it, jobs say they are unmetered and why.

Behind a reverse proxy

Run the service on a loopback address and let the proxy take the public traffic.

• The client's address. The proxy sets X-Forwarded-For, and the service trusts it only from WAI_QSVC_TRUSTED_PROXIES. Have the proxy set the header from the connection itself, not pass on what the request carried, or a client could choose its own limits.
• A path prefix. Mount the service under a prefix (the public one is /quantum-jobs) by stripping the prefix before passing the request on. Set WAI_QSVC_PUBLIC_BASE to the prefixed URL, so the links the service hands out are right. A well-known path belongs at the origin's root: map /.well-known/wai-quantum-receipt-keys there to the service's own.
• Event streams. GET /v1/jobs/{id}/events is a server-sent event stream. The proxy must pass it through unbuffered (the service sends X-Accel-Buffering: no and a comment every 15 s), and allow it to stay open as long as WAI_QSVC_JOB_SECS. Polls with ?wait= are held up to 25 s.
• Bodies. A request body is at most 1 MiB, and POST /v1/run waits up to 25 s before answering 202.
• Rate-limit fields. Pass the RateLimit-Policy, RateLimit and Retry-After headers through.

A minimal site, in one common proxy's configuration format:

quantum.example.org {
    handle_path /quantum-jobs/* {
        reverse_proxy 127.0.0.1:8241 {
            header_up X-Forwarded-For {remote_host}
            flush_interval -1
        }
    }
}

Check it

bash wai/wai-quantum-service/smoke.sh https://quantum.example.org/quantum-jobs
wai-quantum submit prog.qasm --service https://quantum.example.org/quantum-jobs --dry-run

The command line's submit, task and verify work against any instance; verify checks a receipt offline against the key the receipt names, and the key set says when that key signed.