open-standards/eoc/eoc-rs

eoc-rs — EOC reference implementation in Rust

Apache-2.0 reference implementation of the Energy-Optimized Compute specification (CC-BY-4.0 spec lives in ../spec/).

EOC is a four-stage memoizing cascade. Every query is tried at each stage in turn and the first that can answer wins. The four stages, cheapest to most expensive, are:

1. Cache (eoc-cache) — content-addressed LRU. Cache hits are nearly free.
2. Key-value (eoc-kv) — exact key match plus cosine-similarity embedding match.
3. Graph (eoc-graph) — triple-store retrieval. Reference impl is a small in-memory matcher; production deployments swap in a real graph backend driven by DCY (EOC-4).
4. Neural (eoc-neural) — last-resort inference. Reference impl ships an EchoBackend; production deployments wire in llama.cpp / Ollama / Anthropic API / etc.

Joule cost is the unit of accounting. The eoc-meter crate reads cumulative micro-joules from whatever whole-work hardware counter is available and live (Linux RAPL, Apple Silicon IOReport; NVIDIA NVML behind the cuda feature is GPU telemetry, never the cascade's counter) and falls back to a StubCounter everywhere else — including WASM.

What a resolve is charged

A cost is Measured only when every micro-joule of it was read from a live counter over the work it is booked to; anything else is Estimated. The cascade reads its meter where the meter can resolve the work, and nowhere else.

• A resolve the neural stage answers is read. Two reads of the meter bracket the stage, the first taken once the cache, the KV stage and the graph have missed. The span is combined with the cost the stage reported: measured when the stage estimated nothing, the larger of the two readings when the stage metered itself, an estimate when the stage estimated. The three misses are spent before the first read and are not in the figure; the meter's own closing cost is. On an M3 Ultra the misses together took 220 ns and 0.95 µJ, and a span over no work read 12 to 29 µJ, so the measured figure is not under the neural stage's energy.
• A resolve the cache, the KV stage or the graph answers reads no meter, and is never measured. It is charged the stage's own figure plus what a hit on that tier costs on this host. The first time a tier answers, the cascade repeats that lookup inside one span of the meter until the span reads fifty times what a span over no work does (or 40 ms have gone), and keeps the reading per lookup, in nanojoules. Every later hit on the tier is charged that, in whole micro-joules with the remainder carried to the next hit, so the micro-joules charged over any number of hits sum to the calibrated total. Cascade::calibration returns the figure, the batch, the span and the empty span, or why a tier has none; Cascade::calibrate calibrates again; Cascade::without_hit_calibration switches a tier off.

A hit is not bracketed because a read of a counter costs more than a hit. The in-process counter on macOS takes each read only once the reading thread is seen to have left its core, about 80 µs. With a span around every resolve, 10,000 cache hits took 1,647 ms on that host, 164 µs each, and each was sealed as a measured 13 to 28 µJ: the meter's own cost, the same as a span around a neural stage that does nothing. Without it they take 4.2 ms, 0.38 µs each, and are charged the 1.2 to 1.7 µJ a cache hit was calibrated at there (2.6 to 3.4 µJ for a KV or graph hit; the calibration takes 1 to 2.4 ms, once a tier), as an estimate: read by this counter on this host, but over a batch, and the mean of a batch is not a reading of any one hit.

With no live counter there is no calibration and no span, and every resolve keeps the cost its stage reported.

Workspace layout

Crate	Purpose
eoc-core	Types: Query, Response, Stage, JouleCost, Receipt.
eoc-cache	Stage 1 — LRU cache + the Stage trait.
eoc-kv	Stage 2 — KV + embedding-similarity lookup.
eoc-graph	Stage 3 — in-memory triple store.
eoc-neural	Stage 4 — neural inference trait + EchoBackend.
eoc-meter	Hardware energy counters with stub fallback.
eoc-cascade	Glue: walks the four stages, attributes joule cost.
eoc	Reference CLI.
eoc-wasm	wasm-bindgen wrapper — browser-runnable cascade.

Running

cargo test --workspace
cargo clippy --workspace --all-targets
cargo run --bin eoc -- query "what is 2+2"
cargo run --bin eoc -- meter

Example output, on a host with no live counter, where the cost is the stage's own figure:

$ cargo run --bin eoc -- query "what is 2+2"
stage  : kv
payload: 4
cost   : 50 µJ (estimated)
hit    : the kv stage's own figure; no calibration by stub (the counter did not advance)
receipt: <64-char hex>

and on a Mac whose in-process counter is live (--release; the stage's 50 µJ plus the calibrated figure):

$ cargo run --release --bin eoc -- query "what is 2+2"
stage  : kv
payload: 4
cost   : 53 µJ (estimated)
hit    : a kv hit is charged 3357 nJ on this host, on top of the stage's own figure: process-energy read 6876 µJ over 2048 lookups in one span (24 µJ over none)
receipt: <64-char hex>

cargo run --release -p eoc-cascade --example hit_cost prints what each tier's hit takes and is charged on the host it runs on.

CUDA / NVML

NVIDIA NVML support is feature-gated and off by default so the workspace builds without libnvidia-ml. NVML's total-energy counter reads the GPU alone, so detect() never hands it to the cascade, whose costs carry no field to say a figure covers part of the work: NvmlCounter is GPU telemetry, read beside the whole-work counter, and it is constructed only while its counter is live and counts no faster than the board can draw. To enable:

cargo build -p eoc-meter --features cuda

WASM

The cascade compiles to wasm32-unknown-unknown. The meter does not — RAPL, NVML, and powermetrics are all absent in the browser, so WASM builds use StubCounter and report estimated joule cost.

rustup target add wasm32-unknown-unknown
cargo build -p eoc-wasm --target wasm32-unknown-unknown --release
wasm-bindgen target/wasm32-unknown-unknown/release/eoc_wasm.wasm \
    --out-dir ./pkg --target web

Then from JavaScript:

import init, { WasmCascade } from "./pkg/eoc_wasm.js";
await init();
const cascade = new WasmCascade();
const r = await cascade.resolve("ping");
console.log(r); // { stage: "kv", payload: "pong", microjoules: 50, ... }

License

Apache-2.0 for the Rust source in this workspace. The EOC specification text in ../spec/ is CC-BY-4.0. See ../LICENSE.