1fb670a512
All checks were successful
standards workspaces build + test / jouleclaw-rs (pull_request) Successful in 7m16s
standards workspaces build + test / aac-rs (pull_request) Successful in 6m34s
standards workspaces build + test / joule-code-rs (pull_request) Successful in 7m20s
standards workspaces build + test / joulecontract-rs (pull_request) Successful in 7m37s
standards workspaces build + test / map-rs (pull_request) Successful in 5m17s
standards workspaces build + test / mesh-rs (pull_request) Successful in 2m8s
standards workspaces build + test / openpay (pull_request) Successful in 13m7s
no-vanity-metrics / Scan committed Markdown for vanity metrics (pull_request) Successful in 5s
no-vanity-metrics / WAI public text names no comparison (pull_request) Successful in 5s
no-vanity-metrics / WAI public text names no company or product (pull_request) Successful in 5s
standards workspaces build + test / arl-rs (pull_request) Successful in 1m25s
standards workspaces build + test / eoc-rs (pull_request) Successful in 2m32s
standards workspaces build + test / jcp-rs (pull_request) Successful in 1m55s
standards workspaces build + test / joule-ui-rs (pull_request) Successful in 1m27s
standards workspaces build + test / joulehook-rs (pull_request) Successful in 1m27s
standards workspaces build + test / proof-rs (pull_request) Successful in 1m33s
standards workspaces build + test / sandbox-rs (pull_request) Successful in 3m4s
standards workspaces build + test / sense-rs (pull_request) Successful in 1m24s
standards workspaces build + test / smart-byte-rs (pull_request) Successful in 5m30s
workflow hygiene / every workflow supersedes its own queue (pull_request) Successful in 5s
standards workspaces build + test / jouledesk-rs (pull_request) Successful in 2m45s
standards workspaces build + test / macOS energy backends (pull_request) Successful in 4m51s
Mesh's headline figure, microjoules per accepted token, is to be measured
at the meter. Yet every real serve through mesh was estimated: the GGUF
serving binaries were hardwired to EstimatorMeter, and HwMeter bridges
the host meter, which on macOS 27 is the estimator (IOReport's CPU rails
are frozen).
- mesh-meter `ProcessMeter`, behind `hw-meter`, for an engine that runs
in this process. Each serve is metered by jouleclaw's ProcessSpanMeter:
this process's own CPU energy plus its own GPU energy.
- A span that covers the whole request is graded as its counters
grade it: HwShunt for a live reading. Whole means the process was
effectively the GPU's sole user and used no Neural Engine; a
process with no GPU time spent exactly 0 J on it, so CPU-only
engines qualify.
- Any other span is an estimate of the whole request, Estimator: the
reading plus EstimatorMeter for the rest of the host, as HwMeter
bills a counter that reads part of a request. `mesh/1` has no
coverage field.
- `Meter::open_span` and `Window`. The default is None, so existing
meters keep snapshot semantics. ProcessMeter grades over the whole
span, because its GPU grade depends on the process's share over it.
Session uses a span window where a meter keeps one.
- mesh-engine-gguf, with `metal`: serve_falcon and serve_openai_falcon
meter with ProcessMeter (the Metal liveness load comes from
jouleclaw-backend-apple). New `serve_gemma_metered` serves Gemma 4
through mesh with ProcessMeter.
- Fixes mesh-engine-gguf under `gguf`: it no longer compiled on main.
ServeOutcome and ServeRequest gained a `decision` field and three
initializers lacked it, unseen because CI never built the feature.
standards-build now runs `cargo test -p mesh-engine-gguf --features
gguf` on the mesh-rs leg.
Measured on an M5 Max:
- mesh-core tests/process_meter.rs: a ReferenceEngine serve sealed
HwShunt 1,051,668 µJ (4,108 µJ per accepted token), within the
1,090,946 µJ of this process's CPU energy the test read around the
serve.
- serve_gemma_metered, Gemma 4 E2B on CPU, 3 x 16 tokens: HwShunt
12.52 J, 12.16 J and 12.54 J (0.78, 0.76 and 0.78 J per accepted
token). Every envelope verifies.
Mutation caught: billing a whole span's figure plus the rest-of-host
estimate fails the unit test and the end-to-end bracket ("the receipt
read 55691805 µJ, more than this process's 966177 µJ").
Co-Authored-By: Claude Opus 5.5 <[email protected]>
101 lines
5 KiB
Markdown
101 lines
5 KiB
Markdown
# Mesh
|
|
|
|
**Energy-accounted inference serving & routing.** One of the
|
|
[Transaction Science open standards](../README.md).
|
|
|
|
Mesh treats every inference as a metered transaction. A request declares a
|
|
**joule ceiling** and a **fidelity floor**; the mesh routes it to the
|
|
cheapest-sufficient model on the cheapest-sufficient backend, serves it under an
|
|
energy meter, and seals a receipt whose headline figure is **microjoules per
|
|
accepted output token** — measured at the meter, not estimated from a price list.
|
|
|
|
The serving shelf reports dollars per token and tokens per second. Mesh reports
|
|
the number that is physical, comparable across hardware, and impossible to
|
|
inflate: the joule.
|
|
|
|
- **Spec:** [`SPEC.md`](SPEC.md) (CC-BY-4.0)
|
|
- **Reference code:** [`mesh-rs/`](mesh-rs/) (Apache-2.0)
|
|
- **Conformance vectors:** [`conformance/v1/`](conformance/v1/)
|
|
|
|
## The shape
|
|
|
|
```
|
|
ServeRequest ──▶ authorise (JCP grant) ──▶ route (cheapest-sufficient)
|
|
│
|
|
▼
|
|
meter ◀── serve (engine) ──▶ tokens
|
|
│
|
|
▼
|
|
sealed JCR-1 receipt: µJ / accepted token
|
|
```
|
|
|
|
Four roles — **caller**, **router**, **engine**, **steward** — and four phases
|
|
per serve: authorise, route, meter, seal. See [`SPEC.md`](SPEC.md) §2.
|
|
|
|
## The reference workspace
|
|
|
|
`mesh-rs/` is a pure-Rust reference implementation:
|
|
|
|
| Crate | What it is |
|
|
|---|---|
|
|
| `mesh-schema` | wire types — `ServeRequest`, `Fidelity`, two-part `Energy`, `RouteDecision`, `ServeOutcome` |
|
|
| `mesh-router` | energy-weighted cheapest-sufficient routing with a full candidate audit |
|
|
| `mesh-engine` | the `ServingEngine` interface + a deterministic, weights-free `ReferenceEngine` |
|
|
| `mesh-engine-gguf` | a **real** backend (behind `gguf`): wraps the family's pure-Rust GGUF/Metal inference substrate so the mesh routes and meters actual model tokens |
|
|
| `mesh-meter` | the meter window; an always-available estimator + an optional bridge to the family hardware meter (RAPL / NVML / Apple) behind `hw-meter`, and `ProcessMeter`: this process's own CPU and GPU energy for an engine that runs in process, measured when a span covers the whole request |
|
|
| `mesh-receipt` | the `mesh/1` receipt, JCR-1 sealing (det-CBOR + COSE_Sign1), and a hash-chained log |
|
|
| `mesh-core` | the orchestration that ties authorise → route → meter → seal together |
|
|
| `mesh-conformance` | the deterministic reference scenario, its checks, and the `gen-vectors` bin |
|
|
|
|
Mesh **rides on** [JCP](../jcp/) (a serve request is authorised by a JCP grant;
|
|
receipts seal in JCR-1), **dispatches to** runtimes such as
|
|
[JouleClaw](../jouleclaw/), and **routes across** compute described by
|
|
[EOC](../eoc/) and isolated by [Sandbox](../sandbox/). It introduces no new
|
|
cryptography (see [`SPEC.md`](SPEC.md) §9).
|
|
|
|
## Run it
|
|
|
|
```bash
|
|
cd mesh-rs
|
|
|
|
# all crates green
|
|
cargo test --workspace
|
|
cargo clippy --workspace --all-targets
|
|
|
|
# end-to-end: mint a JCP grant, serve, print the sealed receipt
|
|
cargo run -p mesh-core --example demo
|
|
|
|
# on Linux/RAPL or a CUDA/NVML box, meter real silicon (HwShunt provenance, where
|
|
# the counter is live; a counter that is not live falls back to the estimator)
|
|
cargo run -p mesh-core --example demo --features hw-meter-rapl
|
|
|
|
# regenerate the published conformance vectors
|
|
cargo run -p mesh-conformance --bin gen-vectors -- ../conformance/v1
|
|
|
|
# serve a REAL model end-to-end through Mesh (Falcon-H1 0.5B), seal the receipt
|
|
cargo run --release -p mesh-engine-gguf --features gguf --bin serve_falcon -- \
|
|
/path/to/falcon-h1-0.5b-instruct 12 "The capital of France is"
|
|
```
|
|
|
|
The `serve_falcon` run demonstrates the **whole standard** on real model tokens:
|
|
a JCP grant authorises each request (an underfunded grant is refused *before any
|
|
decode runs*), the cheapest-sufficient backend is chosen, the decode is metered,
|
|
and a `mesh/1` receipt is sealed and hash-chained across the session — e.g.
|
|
prompt *"The capital of France is"* → `" Paris. It is known for its historical
|
|
landmarks"`, prompt *"Photosynthesis is the process by which"* → `" plants
|
|
convert sunlight into chemical energy in the form of"`, with both seals verifying
|
|
and the chain's prev-links intact.
|
|
|
|
The demo prints the route (every backend weighed, with the reason each was kept
|
|
or rejected), the honest energy provenance, and the wedge metric — then verifies
|
|
the sealed envelope under the steward key.
|
|
|
|
## Status
|
|
|
|
v0.1.0-draft.1. Reference workspace builds and tests green (`mesh-schema`,
|
|
`mesh-router`, `mesh-engine`, `mesh-meter`, `mesh-receipt`, `mesh-core`,
|
|
`mesh-conformance`); `cargo clippy --all-targets` warning-free; published vectors
|
|
in [`conformance/v1/`](conformance/v1/). The reference engine is deterministic and
|
|
weights-free; production backends implement the same `ServingEngine` interface —
|
|
`mesh-engine-gguf` (behind `gguf`) is a real one over the family's pure-Rust
|
|
GGUF/Metal substrate, kept out of the default build so it needs no weights.
|