# `eoc-rs` — EOC reference implementation in Rust Apache-2.0 reference implementation of the **Energy-Optimized Compute** specification (CC-BY-4.0 spec lives in [`../spec/`](../spec/)). EOC is a four-stage memoizing cascade. Every query is tried at each stage in turn and the first that can answer wins. The four stages, cheapest to most expensive, are: 1. **Cache** (`eoc-cache`) — content-addressed LRU. Cache hits are nearly free. 2. **Key-value** (`eoc-kv`) — exact key match plus cosine-similarity embedding match. 3. **Graph** (`eoc-graph`) — triple-store retrieval. Reference impl is a small in-memory matcher; production deployments swap in a real graph backend driven by DCY (EOC-4). 4. **Neural** (`eoc-neural`) — last-resort inference. Reference impl ships an `EchoBackend`; production deployments wire in llama.cpp / Ollama / Anthropic API / etc. Joule cost is the unit of accounting. The `eoc-meter` crate reads cumulative micro-joules from whatever whole-work hardware counter is available and live (Linux RAPL, Apple Silicon IOReport; NVIDIA NVML behind the `cuda` feature is GPU telemetry, never the cascade's counter) and falls back to a `StubCounter` everywhere else — including WASM. ## What a resolve is charged A cost is `Measured` only when every micro-joule of it was read from a live counter over the work it is booked to; anything else is `Estimated`. The cascade reads its meter where the meter can resolve the work, and nowhere else. - **A resolve the neural stage answers is read.** Two reads of the meter bracket the stage, the first taken once the cache, the KV stage and the graph have missed. The span is combined with the cost the stage reported: measured when the stage estimated nothing, the larger of the two readings when the stage metered itself, an estimate when the stage estimated. The three misses are spent before the first read and are not in the figure; the meter's own closing cost is. On an M3 Ultra the misses together took 220 ns and 0.95 µJ, and a span over no work read 12 to 29 µJ, so the measured figure is not under the neural stage's energy. - **A resolve the cache, the KV stage or the graph answers reads no meter**, and is never measured. It is charged the stage's own figure plus what a hit on that tier costs on this host. The first time a tier answers, the cascade repeats that lookup inside one span of the meter until the span reads fifty times what a span over no work does (or 40 ms have gone), and keeps the reading per lookup, in nanojoules. Every later hit on the tier is charged that, in whole micro-joules with the remainder carried to the next hit, so the micro-joules charged over any number of hits sum to the calibrated total. `Cascade::calibration` returns the figure, the batch, the span and the empty span, or why a tier has none; `Cascade::calibrate` calibrates again; `Cascade::without_hit_calibration` switches a tier off. A hit is not bracketed because a read of a counter costs more than a hit. The in-process counter on macOS takes each read only once the reading thread is seen to have left its core, about 80 µs. With a span around every resolve, 10,000 cache hits took 1,647 ms on that host, 164 µs each, and each was sealed as a measured 13 to 28 µJ: the meter's own cost, the same as a span around a neural stage that does nothing. Without it they take 4.2 ms, 0.38 µs each, and are charged the 1.2 to 1.7 µJ a cache hit was calibrated at there (2.6 to 3.4 µJ for a KV or graph hit; the calibration takes 1 to 2.4 ms, once a tier), as an estimate: read by this counter on this host, but over a batch, and the mean of a batch is not a reading of any one hit. With no live counter there is no calibration and no span, and every resolve keeps the cost its stage reported. ## Workspace layout | Crate | Purpose | |----------------|--------------------------------------------------------------| | `eoc-core` | Types: `Query`, `Response`, `Stage`, `JouleCost`, `Receipt`. | | `eoc-cache` | Stage 1 — LRU cache + the `Stage` trait. | | `eoc-kv` | Stage 2 — KV + embedding-similarity lookup. | | `eoc-graph` | Stage 3 — in-memory triple store. | | `eoc-neural` | Stage 4 — neural inference trait + `EchoBackend`. | | `eoc-meter` | Hardware energy counters with stub fallback. | | `eoc-cascade` | Glue: walks the four stages, attributes joule cost. | | `eoc` | Reference CLI. | | `eoc-wasm` | `wasm-bindgen` wrapper — browser-runnable cascade. | ## Running ```sh cargo test --workspace cargo clippy --workspace --all-targets cargo run --bin eoc -- query "what is 2+2" cargo run --bin eoc -- meter ``` Example output, on a host with no live counter, where the cost is the stage's own figure: ``` $ cargo run --bin eoc -- query "what is 2+2" stage : kv payload: 4 cost : 50 µJ (estimated) hit : the kv stage's own figure; no calibration by stub (the counter did not advance) receipt: <64-char hex> ``` and on a Mac whose in-process counter is live (`--release`; the stage's 50 µJ plus the calibrated figure): ``` $ cargo run --release --bin eoc -- query "what is 2+2" stage : kv payload: 4 cost : 53 µJ (estimated) hit : a kv hit is charged 3357 nJ on this host, on top of the stage's own figure: process-energy read 6876 µJ over 2048 lookups in one span (24 µJ over none) receipt: <64-char hex> ``` `cargo run --release -p eoc-cascade --example hit_cost` prints what each tier's hit takes and is charged on the host it runs on. ## CUDA / NVML NVIDIA NVML support is feature-gated and **off by default** so the workspace builds without `libnvidia-ml`. NVML's total-energy counter reads the GPU alone, so `detect()` never hands it to the cascade, whose costs carry no field to say a figure covers part of the work: `NvmlCounter` is GPU telemetry, read beside the whole-work counter, and it is constructed only while its counter is live and counts no faster than the board can draw. To enable: ```sh cargo build -p eoc-meter --features cuda ``` ## WASM The cascade compiles to `wasm32-unknown-unknown`. The meter does not — RAPL, NVML, and `powermetrics` are all absent in the browser, so WASM builds use `StubCounter` and report estimated joule cost. ```sh rustup target add wasm32-unknown-unknown cargo build -p eoc-wasm --target wasm32-unknown-unknown --release wasm-bindgen target/wasm32-unknown-unknown/release/eoc_wasm.wasm \ --out-dir ./pkg --target web ``` Then from JavaScript: ```js import init, { WasmCascade } from "./pkg/eoc_wasm.js"; await init(); const cascade = new WasmCascade(); const r = await cascade.resolve("ping"); console.log(r); // { stage: "kv", payload: "pong", microjoules: 50, ... } ``` ## License Apache-2.0 for the Rust source in this workspace. The EOC specification text in [`../spec/`](../spec/) is CC-BY-4.0. See [`../LICENSE`](../LICENSE).