Watch

Transaction Science 76ce401fe9 eoc-meter: the GPU counter is never the cascade's counter, and one past its board is Implausible

Two defects of eoc's NVML path (feature cuda), both seen on jetson-hub's
RTX 4050 Laptop GPU (driver 595.84).

detect() ranked NVML first, so on a host with a live NVML counter every
consumer (eoc-cascade, eoc-local, eoc-streaming, eoc-agent) was handed the
GPU's counter and graded its advance Measured, with no coverage field to
say the CPU's share of the work went unread: on jetson-hub detect()
returned nvml over a live RAPL. By the family's rule a figure that does not
cover the work it is booked to is an estimate in a format without coverage.
detect() now has no NVML rung, and NvmlCounter is no longer a JouleCounter
(read_gpu_microjoules() reads it as GPU telemetry), so no consumer can grade
it Measured. JouleSource and the README say so.

And the counter does not count the board's energy while it is polled: at
the probe's 100 ms wait it ran at 81-136 W while the board drew 13-15 W
against a 65 W maximum limit. The probe now reads the board's draw with
each sample and its maximum power limit once (gpu_plausibility), and a
live counter past 1.5x the limit, past twice the mean draw plus 5 W, or
beside a draw reading past 1.5x the limit, is CounterStatus::Implausible
and NvmlCounter is not constructed.

Before: tests/nvml_hardware.rs failed on jetson-hub (status=Live against an
oracle verdict of Implausible, 119.4 W counter, 13-13.5 W draw;
detect()=nvml). After: status=Implausible, constructed=false,
detect()=rapl; 58 lib tests with cuda (57 without), the NVML and RAPL
hardware tests pass there with and without cuda.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

2026-10-01 10:18:29 -04:00

3.7 KiB

Raw Permalink Blame History

`eoc-rs` — EOC reference implementation in Rust

Apache-2.0 reference implementation of the Energy-Optimized Compute specification (CC-BY-4.0 spec lives in ../spec/).

EOC is a four-stage memoizing cascade. Every query is tried at each stage in turn and the first that can answer wins. The four stages, cheapest to most expensive, are:

Cache (eoc-cache) — content-addressed LRU. Cache hits are nearly free.
Key-value (eoc-kv) — exact key match plus cosine-similarity embedding match.
Graph (eoc-graph) — triple-store retrieval. Reference impl is a small in-memory matcher; production deployments swap in a real graph backend driven by DCY (EOC-4).
Neural (eoc-neural) — last-resort inference. Reference impl ships an EchoBackend; production deployments wire in llama.cpp / Ollama / Anthropic API / etc.

Joule cost is the unit of accounting. The eoc-meter crate reads cumulative micro-joules from whatever whole-work hardware counter is available and live (Linux RAPL, Apple Silicon IOReport; NVIDIA NVML behind the cuda feature is GPU telemetry, never the cascade's counter) and falls back to a StubCounter everywhere else — including WASM.

Workspace layout

Crate	Purpose
`eoc-core`	Types: `Query`, `Response`, `Stage`, `JouleCost`, `Receipt`.
`eoc-cache`	Stage 1 — LRU cache + the `Stage` trait.
`eoc-kv`	Stage 2 — KV + embedding-similarity lookup.
`eoc-graph`	Stage 3 — in-memory triple store.
`eoc-neural`	Stage 4 — neural inference trait + `EchoBackend`.
`eoc-meter`	Hardware energy counters with stub fallback.
`eoc-cascade`	Glue: walks the four stages, attributes joule cost.
`eoc`	Reference CLI.
`eoc-wasm`	`wasm-bindgen` wrapper — browser-runnable cascade.

Running

cargo test --workspace
cargo clippy --workspace --all-targets
cargo run --bin eoc -- query "what is 2+2"
cargo run --bin eoc -- meter

Example output:

$ cargo run --bin eoc -- query "what is 2+2"
stage  : kv
payload: 4
cost   : 50 µJ (estimated)
receipt: <64-char hex>

CUDA / NVML

NVIDIA NVML support is feature-gated and off by default so the workspace builds without libnvidia-ml. NVML's total-energy counter reads the GPU alone, so detect() never hands it to the cascade, whose costs carry no field to say a figure covers part of the work: NvmlCounter is GPU telemetry, read beside the whole-work counter, and it is constructed only while its counter is live and counts no faster than the board can draw. To enable:

cargo build -p eoc-meter --features cuda

WASM

The cascade compiles to wasm32-unknown-unknown. The meter does not — RAPL, NVML, and powermetrics are all absent in the browser, so WASM builds use StubCounter and report estimated joule cost.

rustup target add wasm32-unknown-unknown
cargo build -p eoc-wasm --target wasm32-unknown-unknown --release
wasm-bindgen target/wasm32-unknown-unknown/release/eoc_wasm.wasm \
    --out-dir ./pkg --target web

Then from JavaScript:

import init, { WasmCascade } from "./pkg/eoc_wasm.js";
await init();
const cascade = new WasmCascade();
const r = await cascade.resolve("ping");
console.log(r); // { stage: "kv", payload: "pong", microjoules: 50, ... }

License

Apache-2.0 for the Rust source in this workspace. The EOC specification text in ../spec/ is CC-BY-4.0. See ../LICENSE.

3.7 KiB Raw Permalink Blame History

eoc-rs — EOC reference implementation in Rust