| Filename | Latest commit message | Latest commit date |
|---|---|---|
f456270ca4
HwMeter passed the family meter's whole-host estimators straight through. Those accrue by the millisecond, so a span shorter than one — a served request, here — read 0 µJ: on a host with no live whole-request counter (this M5 Max), every request served with `server,hw-meter` or `server,hw-meter-apple` sealed 0 µJ [Estimator], against HwMeter's own rule that a span is never billed a zero. A backend that is a whole-host estimate is now billed through this crate's EstimatorMeter, which resolves a microjoule, and the bridge names itself `estimator` then. ProcessMeter (`server,metal`) sealed ~1.5–9.5 J [Estimator] per request for ~11 µs of engine work. The process CPU count does not advance over so short a serve, so ProcessSpanMeter falls back to the host meter, whose window opens before its first GPU-rail snapshot and closes after its last — 100 ms or more of the meter's own waiting at each end — and its load estimate billed that. A whole Estimator span (only that fallback gives one) is now billed this crate's estimate over the work's own window. The estimator's window for the rest of the host now opens after the span meter's starting readings, the same window as the CPU reading, instead of before its GPU snapshot wait. Measured here, one served request: hw-meter 0 → 103 µJ, hw-meter-apple 0 → 113 µJ, metal 9,480,000 → 168 µJ, all [Estimator]. Tests (each fails before): an_estimator_backend_never_bills_a_short_ span_nothing (50 spans of 50 µs over the family EstimationMeter); an_estimated_fallback_span_is_billed_the_works_own_window (energy_of over the fallback span seen here); serve_example asserts usage.energy.uj > 0 for every grade. The waits sit in jouleclaw-energy's ProcessSpanMeter (its fallback reads bracket the GPU snapshots); that is not changed here. Co-Authored-By: Claude Opus 5.5 <[email protected]> |
||
| .. | ||
| conformance/v1 | ||
| mesh-rs | ||
| tools/joules-per-decision | ||
| CONTRIBUTING.md | ||
| LICENSE | ||
| LICENSE-SPEC | ||
| README.md | ||
| SPEC.md | ||
Mesh
Energy-accounted inference serving & routing. One of the Transaction Science open standards.
Mesh treats every inference as a metered transaction. A request declares a joule ceiling and a fidelity floor; the mesh routes it to the cheapest-sufficient model on the cheapest-sufficient backend, serves it under an energy meter, and seals a receipt whose headline figure is microjoules per accepted output token — measured at the meter, not estimated from a price list.
The serving shelf reports dollars per token and tokens per second. Mesh reports the number that is physical, comparable across hardware, and impossible to inflate: the joule.
- Spec:
SPEC.md(CC-BY-4.0) - Reference code:
mesh-rs/(Apache-2.0) - Conformance vectors:
conformance/v1/
The shape
ServeRequest ──▶ authorise (JCP grant) ──▶ route (cheapest-sufficient)
│
▼
meter ◀── serve (engine) ──▶ tokens
│
▼
sealed JCR-1 receipt: µJ / accepted token
Four roles — caller, router, engine, steward — and four phases
per serve: authorise, route, meter, seal. See SPEC.md §2.
The reference workspace
mesh-rs/ is a pure-Rust reference implementation:
| Crate | What it is |
|---|---|
mesh-schema |
wire types — ServeRequest, Fidelity, two-part Energy, RouteDecision, ServeOutcome |
mesh-router |
energy-weighted cheapest-sufficient routing with a full candidate audit |
mesh-engine |
the ServingEngine interface + a deterministic, weights-free ReferenceEngine |
mesh-engine-gguf |
a real backend (behind gguf): wraps the family's pure-Rust GGUF/Metal inference substrate so the mesh routes and meters actual model tokens |
mesh-meter |
the meter window; an always-available estimator + an optional bridge to the family hardware meter (RAPL / NVML / Apple) behind hw-meter, and ProcessMeter: this process's own CPU and GPU energy for an engine that runs in process, measured when a span covers the whole request |
mesh-receipt |
the mesh/1 receipt, JCR-1 sealing (det-CBOR + COSE_Sign1), and a hash-chained log |
mesh-core |
the orchestration that ties authorise → route → meter → seal together |
mesh-conformance |
the deterministic reference scenario, its checks, and the gen-vectors bin |
Mesh rides on JCP (a serve request is authorised by a JCP grant;
receipts seal in JCR-1), dispatches to runtimes such as
JouleClaw, and routes across compute described by
EOC and isolated by Sandbox. It introduces no new
cryptography (see SPEC.md §9).
Run it
cd mesh-rs
# all crates green
cargo test --workspace
cargo clippy --workspace --all-targets
# end-to-end: mint a JCP grant, serve, print the sealed receipt
cargo run -p mesh-core --example demo
# on Linux/RAPL or a CUDA/NVML box, meter real silicon (HwShunt provenance, where
# the counter is live; a counter that is not live falls back to the estimator)
cargo run -p mesh-core --example demo --features hw-meter-rapl
# regenerate the published conformance vectors
cargo run -p mesh-conformance --bin gen-vectors -- ../conformance/v1
# serve a REAL model end-to-end through Mesh (Falcon-H1 0.5B), seal the receipt
cargo run --release -p mesh-engine-gguf --features gguf --bin serve_falcon -- \
/path/to/falcon-h1-0.5b-instruct 12 "The capital of France is"
The serve_falcon run demonstrates the whole standard on real model tokens:
a JCP grant authorises each request (an underfunded grant is refused before any
decode runs), the cheapest-sufficient backend is chosen, the decode is metered,
and a mesh/1 receipt is sealed and hash-chained across the session — e.g.
prompt "The capital of France is" → " Paris. It is known for its historical landmarks", prompt "Photosynthesis is the process by which" → " plants convert sunlight into chemical energy in the form of", with both seals verifying
and the chain's prev-links intact.
The demo prints the route (every backend weighed, with the reason each was kept or rejected), the honest energy provenance, and the wedge metric — then verifies the sealed envelope under the steward key.
Status
v0.1.0-draft.1. Reference workspace builds and tests green (mesh-schema,
mesh-router, mesh-engine, mesh-meter, mesh-receipt, mesh-core,
mesh-conformance); cargo clippy --all-targets warning-free; published vectors
in conformance/v1/. The reference engine is deterministic and
weights-free; production backends implement the same ServingEngine interface —
mesh-engine-gguf (behind gguf) is a real one over the family's pure-Rust
GGUF/Metal substrate, kept out of the default build so it needs no weights.