ad8fbf48ca
On macOS 27.0.1 an unprivileged reader sees the machine-wide IOReport CPU rail frozen, so every meter here has been an estimator on Apple Silicon. The kernel does keep a live per-process figure: `proc_pid_rusage(getpid(), RUSAGE_INFO_V6)` and its `ri_energy_nj`, the CPU energy it attributes to this process's threads, public libproc, no privilege. On this M5 Max it advances every few milliseconds under load. Its coverage is the trap. It counts this process's own threads and nothing else: not child processes (the kernel never folds a child's energy into its parent, not even after reaping it; `rusage_info` has a child CPU-time field and no child energy field), not other processes, not the GPU or the ANE. `PowerMeter` meters subprocess work in jcp-gateway (the upstream MCP server), joule-code-remediate (cargo) and sandbox-proof (the engine), and in-process work in joule-ui-serve's handlers. A probe added to `detect()` would read the gateway waiting, near zero, as the energy of the tool call. So the new `ProcessEnergyProbe` (`Cpu`, `HwShunt`, source "process-energy") is never part of `detect()`. It is reached only through a new constructor, `PowerMeter::detect_in_process()`, documented as only for work on this process's own threads: where the counter is live the meter reads it and nothing else (a machine-wide CPU probe beside it would count the same work twice); where it is not, and off macOS, it is `detect()`. The counter is gated by the shared rule, probed once per process and cached (`process_energy_status()`, a `CounterStatus`): it must advance in at least five intervals, at least four in five, over at least 300 ms, while the probe burns CPU, through the same `liveness::counter_run` and `gated` as RAPL and NVML. `Span::stop`'s held-CPU rule applies unchanged: a span over which the counter did not advance is estimated. The struct is the SDK's `rusage_info_v6` field for field, pinned by a test at the SDK's own sizeof and offsets (464 bytes, `ri_energy_nj` at 336). `run_attributed` (subprocess windows). The question was whether a child's own figure can be read reliably. It can, for the direct child: wait for the exit without reaping (`waitid(P_PID, pid, WEXITED | WNOWAIT)`), `proc_pid_rusage` the exited child, then reap with `wait4` as before. Measured here before writing it: the read succeeds on the zombie and is stable across repeated reads, then fails (ESRCH) once reaped; `sh -c 'awk ...'` (sh execs awk) read 1.647 J for 0.6 s of awk's CPU; `sh -c 'awk ...; true'` (awk a reaped child of sh) read 5.5 mJ, sh's own start-up, beside 0.63 s of reaped-descendant CPU time. So the new `Attribution::child_energy_uj` is the direct child's own energy, given only when the per-process counter is live, the child reaped no descendant that used CPU time, and it advanced. Grandchildren are not covered: cargo -> rustc gets `None`, as does any shell that forks its commands, and a descendant the child did not reap leaves no trace at all. The status is read before the window opens, so its first-use probe never lands inside it. The apportioned `attributed_uj` is unchanged. jcp-gateway, joule-code-remediate and sandbox-proof stay on `detect()`, and their docs now say why: their metered work is a subprocess. remediate's test helper sets the new field to `None` (cargo). Tests. Hardware, in their own process (`tests/process_energy.rs`, one at a time, each against an oracle that reads `ri_energy_nj` into a raw buffer at the SDK offset and shares no code with the probe): `the_process_energy_probe_is_live` (pinned Live on Apple Silicon; the oracle saw 39/39 intervals advance; the probe ran once), `an_in_process_burn_is_a_hardware_reading_of_its_energy` (200 ms of one thread's CPU, user-interactive QoS, via `detect_in_process`: one `Cpu` `HwShunt` record in 0.2..20 J and inside the oracle's reading around it), `an_in_process_sleep_reads_under_a_tenth_of_a_joule`, `detect_does_not_include_the_process_probe`, `work_in_a_child_process_is_not_attributed_to_the_in_process_probe`. Unit: `the_rusage_struct_is_the_sdks_layout`, `this_process_reads_itself_and_a_missing_pid_reads_nothing`, `a_childs_own_energy_is_a_reading_only_for_a_single_process_child`, `only_a_live_status_builds_the_process_probe`, `detect_never_reads_this_process_counter` (every status combination), `detect_in_process_reads_this_process_only_for_a_live_status` (and never calls the fallback when it reads), `a_single_process_childs_own_energy_is_read_before_it_is_reaped`, `a_child_whose_work_ran_in_a_descendant_has_no_reading`. Measured on this M5 Max (macOS 27.0.1, other builds loading it), verbatim: 200 ms in-process burn: 0.4361 J over 219.588375ms wall (oracle around it: 0.4361 J) 200 ms in-process sleep: 575 µJ child work: child 0.8730 J, this process 0.001304 J around the wait, span [... measured_uj: Some(1304) ... HwShunt] process energy: status=Live oracle advanced in 39/39 intervals, probe runs=1 child energy: status=Live child_energy_uj=Some(3128510) proc_cpu_secs=1.499 On the M3 Ultra (macOS 27.0.1, 10.10.10.2; lib and process_energy test binaries copied, run once with --nocapture, deleted), verbatim: 200 ms in-process burn: 1.3150 J over 200.032666ms wall (oracle around it: 1.3150 J) 200 ms in-process sleep: 1796 µJ process energy: status=Live oracle advanced in 40/40 intervals, probe runs=1 child work: child 2.3750 J, this process 0.014451 J around the wait, span [... measured_uj: Some(14451) ... HwShunt] child energy: status=Live child_energy_uj=Some(10145217) proc_cpu_secs=2.044 ioreport liveness: status=CpuRailFrozen oracle=CpuRailMoved(false) selected=false 105 lib tests and 5 process_energy tests passed. The per-process counter is live on both chips; IOReport's CPU rail stays frozen. sandbox-meter passes (105 lib, 5 process_energy, both once-per-process files); every sandbox-rs crate that depends on the meter passes (238 tests); jcp-gateway and jcp-action-receipt (33) and joule-code-remediate (13) pass. Clippy is clean for aarch64-apple-darwin and x86_64-unknown-linux-gnu, and x86_64-apple-darwin checks clean. Falsified, each restored from git and re-run green: the probe added to `detect()` fails `detect_never_reads_this_process_counter` and `detect_does_not_include_the_process_probe`; a constant counter, and the parent's pid in place of this one, fail `the_process_energy_probe_is_live` (status Frozen while the oracle saw it advance) and the 200 ms burn (it falls back to the 4.1 J estimate); `given` ignoring the status fails `only_a_live_status_builds_the_process_probe` and `detect_in_process_reads_this_process_only_for_a_live_status`; a child reading that ignores reaped descendants fails both child tests (sh's 5.5 mJ stood in for awk's work); one that ignores liveness fails the pure child test. Rebased onto7cf1f036(the RAPL figure sums the package zones by name), which builds `rapl` under test on every platform and adds its fake-tree tests. The one conflict was the module declarations in probe/mod.rs; both `process` and `rapl`'s new cfg stand. The counts above predate the rebase: on the M5 Max sandbox-meter now has 120 lib tests (114 before it), all passing, with process_energy (5) and both once-per-process tests; the sandbox-rs workspace, the sandbox-conformance binary, jcp-gateway, jcp-action-receipt and joule-code-remediate pass too. Clippy on sandbox-meter is clean for aarch64-apple-darwin and x86_64-unknown-linux-gnu. The M3 Ultra run was not repeated. Rebased again, ontoa16e36cf, whose sandbox-meter side builds `liveness` and `rapl` on every platform and adds `RaplPackages` and `counter_liveness`. The one conflict was again the module declarations in probe/mod.rs: `liveness` and `rapl` are unconditional, `process` stays macOS-only. On the M5 Max sandbox-meter now has 122 lib tests, all passing, with process_energy (5) and both once-per-process tests; the sandbox-rs workspace (321), the sandbox-conformance binary, jcp-gateway and jcp-action-receipt (33) and joule-code-remediate (13) pass. Clippy on sandbox-meter and sandbox-exec-microvm is clean for aarch64-apple-darwin and x86_64-unknown-linux-gnu. Co-Authored-By: Claude Opus 5.5 <[email protected]>
58 lines
13 KiB
Markdown
58 lines
13 KiB
Markdown
# Sandbox
|
||
|
||
A sandbox for the heterogeneous compute fabric. Isolate, capability-gate, meter, and attest — owned by no one.
|
||
|
||
Sandbox is the **isolated-execution substrate** of the Transaction Science open standards. Where [Smart Byte](../smart-byte/) carries value, [EOC](../eoc/) carries compute, and [Joule Code](../joule-code/) carries coding work, Sandbox carries the **place those operations run**: a bounded environment in which an untrusted unit of work executes, sees only what it was granted, and leaves behind a signed account of what it did and what it cost.
|
||
|
||
Every other sandbox in the field isolates a **CPU process** — a container, a microVM, a Linux process pinned to a core. The accelerator, where present, is an opaque passthrough that nobody meters and nobody attests. But agentic and AI work does not run on a CPU. It runs across a **fabric** — CPU, GPU, NPU, TPU, photonic, quantum. So in this standard the unit of isolation is not a process; it is a **heterogeneous unit of work**, and the accelerator is a first-class citizen of the isolation boundary: a capability you grant, a resource you meter in joules, and a line item you attest.
|
||
|
||
A Sandbox run carries exactly four guarantees: **isolation** (the unit runs at the lowest tier of an eight-rung ladder, S0–S7, that satisfies its needs, and cannot reach past it), **capability** (every resource the unit touches — each device class included — was explicitly granted; ungranted access is denied), **metering** (the run emits a signed energy receipt with per-device-class joule attribution and an honest provenance tag on every reading — and the reference meter reads vendor energy counters: Apple Silicon's energy model via IOReport, Intel/AMD RAPL via powercap, NVIDIA's total-energy counter via NVML. A reading is tagged `HwShunt` only while its counter is demonstrably live: each vendor counter is used only if every rail of it that the meter sums into one figure keeps advancing while the meter's probe burns CPU — in at least 5 sample intervals, at least 4 of every 5, over at least 300 ms — every IOReport CPU energy rail (the rail itself, one per die on a multi-die part), every RAPL package and every NVML GPU; and a span across which the CPU counter held is never a reading. A counter that is absent, or present but not live — frozen, or republished in lumps, as the IOReport CPU rails are on macOS 27 — yields no hardware-graded figure: the span degrades to the labeled `Estimator` (wall-clock × TDP), never a fabricated measurement, and `ioreport_status()`, `rapl_status()` and `nvml_status()` say why a host is estimated. For work on the meter's own process there is one more source, kept out of the default meter: on macOS, the CPU energy the kernel attributes to the process's own threads (`proc_pid_rusage`), read through `PowerMeter::detect_in_process()` under the same liveness rule — never for a subprocess, which it does not cover; a single-process child's own figure is read from the exited child instead. Because a live counter reports the whole SoC, the meter can **attribute** it to a single process tree by CPU-time apportionment — tagged `ModelBased`, since the reading is a measurement but the split is a model), and **attestation** (a signed statement of *what image ran, at what tier, under what grants, on what hardware, for what cost*, with a deterministic-replay journal hash where one was produced).
|
||
|
||
The substrate is intentionally **ownerless**: no protocol-level token, permissive licensing on spec text and reference code, and conformance vectors so any implementer can verify canonicity without a trust relationship with the authors. A conforming host runs a unit, enforces its grants, meters its energy, and seals its attestation **without contacting any single steward's servers as a condition of operation**. Transaction Science is one steward: it publishes the spec, ships a reference implementation, and operates commercial services around the substrate — not its proprietor.
|
||
|
||
## The doctrine
|
||
|
||
**Isolation per joule, across the whole fabric.** The durable layer of a sandbox is not fast container boot or a managed control plane — both are commoditized. It is the boundary itself, made honest: the lowest isolation tier that satisfies the work, every resource gated by an explicit capability, and a signed receipt that attributes energy to each device class it touched. "Run it safely" and "don't waste energy" are the same optimization: an over-isolated run burns joules it did not need, and an unmetered accelerator is exactly where the energy hides.
|
||
|
||
Three consequences, normative in [`SPEC.md`](SPEC.md):
|
||
|
||
1. **The ladder.** A host **MUST** run a unit at the lowest isolation tier of `S0:Bare → S1:Capability → S2:Sandbox → S3:Partition → S4:Unikernel → S5:MicroVM → S6:Confidential → S7:Sovereign` that satisfies the unit's capability set, and **MUST NOT** escalate beyond what the grants require. Escalation is the escape hatch, not the entry point — the same discipline the cascade applies to inference, applied to isolation.
|
||
2. **The grant.** Every resource a unit can touch — network, storage, key-value, and **each compute device class** (`Cpu | Simd | Gpu | Npu | Tpu | Lpu | Photonic | Quantum`) — **MUST** be explicitly granted. Access to an ungranted resource **MUST** be denied. Each compute grant carries a joule budget.
|
||
3. **The attested receipt.** Every completed run **MUST** emit an energy receipt accounting joules **per device class** — `measured + estimated`, integer microjoules — each reading tagged `HwShunt | ModelBased | Estimator`, sealed as a [JCR-1](../jouleclaw/) envelope; and **MUST** emit an attestation binding the unit, image, tier, grants, hardware, and receipt, signed Ed25519 over JCS-canonicalized JSON — the same attestation primitive the [ARL Sandbox (ARL-S)](../arl/) uses, so the two interoperate at the attestation layer.
|
||
|
||
## Contents
|
||
|
||
- [`SPEC.md`](SPEC.md) — the normative specification (RFC 2119): the unit of work, the S0–S7 isolation ladder, the capability-grant model, the per-device energy receipt, the attestation, and the conformance contract.
|
||
- [`spec/`](spec/) — sub-specifications: [`tiers.md`](spec/tiers.md) (the ladder), [`capabilities.md`](spec/capabilities.md) (the grant model), [`receipts.md`](spec/receipts.md) (per-device energy), [`attestation.md`](spec/attestation.md) (the signed claim + replay journal), [`conformance.md`](spec/conformance.md) (test obligations).
|
||
- [`conformance/`](conformance/) — signed wire vectors. An implementation is conformant if it round-trips these.
|
||
- [`proof/`](proof/) — the **proof matrix**: a ledger of signed proof cells, each a real run of the conformance pack + a sealed workload on a real target (substrate × OS × arch × cloud × hardware × tier). A green cell is a signature over a real run, not a claim.
|
||
- [`sandbox-rs/`](sandbox-rs/) — the Rust reference library: content addressing and tiers (`sandbox-schema`), the capability model and deny-by-default enforcer (`sandbox-capability`), the per-device receipt and JCR-1 sealing (`sandbox-meter`), the attestation and replay journal (`sandbox-attest`), the reference host — tier minimization plus sealed-run assembly (`sandbox-runtime`), a real **S2 (WASM) executor** on wasmtime with fuel→joule metering and a brokered, Enforcer-gated host-call surface that journals every nondeterministic input for replay (`sandbox-exec-wasm`), a real **S5 (microVM) executor** driving Firecracker — both an API-socket driver and a supervised-child lifecycle driver — with pluggable per-device power metering (`sandbox-exec-microvm`), the **S6 confidential-tier** TEE layer — SEV-SNP report model, policy + real ECDSA-P384/SHA-384 verification, and a `configfs-tsm` hardware provider with a software-grounded test provider (`sandbox-tee`), the conformance harness with the `gen_vectors` generator (`sandbox-conformance`), the **proof-matrix harness** that runs the pack + a sealed run on a target and emits a signed proof cell (`sandbox-proof`), the `sandbox` reference checker/runner (`sandbox-cli`), and the **JCP gate** (`sandbox-jcp`) — execution as a JCP consumer: a metered Grant bound to the unit's image *by content address* is decided by the JCP runtime before the executor is ever invoked, the declared energy ceiling is debited from the delegation chain pre-run, and the decision (allow and deny alike) seals as a JCR-1 receipt alongside the sandbox's own sealed run.
|
||
|
||
## Heterogeneity is normative, not an extension
|
||
|
||
The clause that distinguishes this standard from a process sandbox is in [`SPEC.md`](SPEC.md) §2.3 and §6.2: where a unit is granted an accelerator capability, a conforming host **MUST** attribute energy to that device class separately in the receipt, **MUST** record the device identity (class, model, driver) in the attestation, and at tiers S5 and above **MUST** report the measurement provenance honestly (`HwShunt` where the device exposes a live energy interface — NVML, RAPL, ROCm SMI, Level Zero — `ModelBased` or `Estimator` otherwise). A host that runs accelerated work but reports a single undifferentiated CPU energy figure is **not conformant**.
|
||
|
||
## Lineage
|
||
|
||
Sandbox draws its design from two Transaction Science runtimes and generalizes a third standard:
|
||
|
||
- The **isolation ladder** and the **capability-grant model** follow the layered, hardware-rooted isolation and the per-resource grant set proven in the invisible-infrastructure runtime — capabilities first, WASM and microVM tiers above, TEE and sovereign tiers at the top, accelerators as explicit device capabilities throughout.
|
||
- The **per-device energy receipt** and its **provenance discipline** follow the picojoule receipt and measurement-quality model of the JouleDB substrate, sealed in the family's shared JCR-1 envelope.
|
||
- The **attestation** generalizes the SUT / Harness / Supervisor separation and the signed-telemetry model of the [ARL Sandbox](../arl/) out of a measurement-only role into a standalone substrate that ARL-S, [JouleClaw](../jouleclaw/), [EOC](../eoc/), and [Joule Code](../joule-code/) can target instead of each re-specifying isolation.
|
||
|
||
## Status
|
||
|
||
v0.1.0 draft. Specification core published (the unit, the ladder, the grant model, the per-device receipt, the attestation); the reference library lands the schema, capability, meter, attest, runtime, and conformance crates with the `sandbox` CLI, plus two **real tier executors**: a wasmtime-backed S2 executor that runs untrusted WASM under fuel and memory caps, meters fuel→joules, and brokers every capability the guest exercises (kv, clock, randomness) through the deny-by-default `Enforcer` — journaling each nondeterministic input so the run's attestation carries a replay-journal root; and a Firecracker-driver S5 executor that boots a microVM (API-socket or supervised-child lifecycle) and meters CPU and an assigned accelerator **separately** through pluggable power meters (RAPL/NVML-class, with an honest `ModelBased` fallback). Energy receipts seal as **JCR-1** envelopes (canonical det-CBOR + COSE_Sign1, EdDSA), the signed-receipt format shared across the family; attestations sign Ed25519 over JCS-canonicalized JSON, interoperable with ARL-S. The conformance vectors in [`conformance/v1/`](conformance/v1/) are real, cryptographically computed values — including a committed pack of adversarial **negatives** a conformant host must reject — regenerated by `gen_vectors` and verified by the harness (in-code and file-backed).
|
||
|
||
Readiness is classified per component (Solid / Engineering / Research / Theory) in [`SPEC.md`](SPEC.md) §9 and respected when sequencing work. The S2 (WASM) executor runs and meters real guests; the S4–S5 (microVM) driver is real Firecracker integration (it boots a guest on Linux+KVM); and the **S6/S7 confidential and sovereign tiers** are implemented end-to-end — the host obtains a TEE report, verifies it (real ECDSA-P384/SHA-384 signature + measurement/TCB/debug/report-data policy), seals the evidence into the attestation, binds a locality, and links the run into a tamper-evident audit chain. The single honest caveat (SPEC §9): *generating* a genuine vendor-signed report requires the CPU's security processor — the `configfs-tsm` provider does so on real confidential hardware and reports `Unavailable` elsewhere, while a tagged software-grounded provider, which never claims a hardware root, covers wiring and tests.
|
||
|
||
## Related
|
||
|
||
- [ARL](../arl/) — the AI Readiness Level standard; its **ARL Sandbox (ARL-S)** is the measurement-specialized profile of this substrate. The attestation primitive (Ed25519 over JCS) is shared, so an ARL-S session attestation and a Sandbox run attestation verify under the same tooling.
|
||
- [JouleClaw](../jouleclaw/) — the energy-optimised AI runtime; a JouleClaw agent's tool execution runs **inside** a Sandbox, and the receipt provenance discipline (`HwShunt | ModelBased | Estimator`) and the JCR-1 envelope are JouleClaw's.
|
||
- [Joule Code](../joule-code/) — a coding operation that executes generated code (a test run, a build) runs that code inside a Sandbox; the joule receipt composes.
|
||
- [EOC](../eoc/) — the energy-optimised compute substrate; the discipline of stopping at the cheapest tier that closes the request is EOC's, applied here to isolation tiers.
|
||
|
||
## Licence
|
||
|
||
Reference code (everything under [`sandbox-rs/`](sandbox-rs/)) is **Apache-2.0** ([`LICENSE`](LICENSE)). Specification text (`SPEC.md`, `spec/`) is **CC-BY-4.0** ([`LICENSE-SPEC`](LICENSE-SPEC)). The wire format and the right to fork are public.
|