| Filename | Latest commit message | Latest commit date |
|---|---|---|
327295732b
`BackendSpec::estimate` was `uj_per_token × (prompt_tokens + max_tokens)`: one scalar, linear in token count, blind to how much context precedes each token. For a backend whose per-token cost genuinely does not depend on the prefix that is correct, and it stays expressible. For one that re-reads its context per token it is not — and the error has a **direction**. A linear model applied to a super-linear backend *under*-estimates, always, and the gap widens with context. At a gate, under-estimating means admitting work the ceiling cannot cover: the request passes, runs, and overruns — which is the one outcome a joule ceiling exists to prevent. A ceiling checked against a model that cannot represent the expensive case is not a ceiling, it is a formality that happens to pass. So "refuse a long-context route before it burns the budget" was a capability the docs implied and the code could not deliver, and the test that now pins it shows the same backend admitted under a flat profile and refused under a calibrated one, for the identical request. `CostProfile` carries the *shape* — a fixed base, a flat per-token term, and microjoules per token per 1024 tokens of context — and `CostProfile::linear` reproduces the old arithmetic exactly. **No profile ships as a default, deliberately.** The context term is a property of a particular backend on particular hardware and has to be calibrated by measurement there. The measured study that motivated this change used one model per attention condition, so architecture is confounded with model family and parameter count; lifting its slopes into a constant would inherit that confound and dress it as a fact. Same reason `BandingScale` ships no canonical scale: the type expresses the shape, the operator supplies the numbers. Details worth stating: - The context term is charged against `prompt_tokens`. Output tokens extend the prefix as they are generated, so this remains a lower bound on a growing-context backend — deliberately, because the alternative is to model that growth and then defend a second uncalibrated constant. - Every step saturates. An estimate that wrapped would come back *small*, and a small estimate is exactly what admits an unaffordable request. - `is_context_aware()` is inspectable, so a deployment gating long-context work can notice it is deciding with profiles that cannot represent the case. - The profile has a wire form, so a route decision can be re-derived from the receipt rather than trusted. Backward compatibility is proven, not asserted: `BackendSpec::new` is unchanged in signature and behaviour, and re-running `gen-vectors` leaves the published vectors byte-identical on disk. Found by the 2026-08-29 sweep, which ranked this second and correctly described it as falsifying TX's own cost model. The source paper is cite-only (arXiv non-exclusive licence, no code), and nothing from it is reproduced here. Verified: 38 mesh-rs tests pass (6 new), conformance passes, published vectors unchanged after regeneration, no clippy warnings in mesh-router. |
||
| .. | ||
| conformance/v1 | ||
| mesh-rs | ||
| CONTRIBUTING.md | ||
| LICENSE | ||
| LICENSE-SPEC | ||
| README.md | ||
| SPEC.md | ||
Mesh
Energy-accounted inference serving & routing. One of the Transaction Science open standards.
Mesh treats every inference as a metered transaction. A request declares a joule ceiling and a fidelity floor; the mesh routes it to the cheapest-sufficient model on the cheapest-sufficient backend, serves it under an energy meter, and seals a receipt whose headline figure is microjoules per accepted output token — measured at the meter, not estimated from a price list.
The serving shelf reports dollars per token and tokens per second. Mesh reports the number that is physical, comparable across hardware, and impossible to inflate: the joule.
- Spec:
SPEC.md(CC-BY-4.0) - Reference code:
mesh-rs/(Apache-2.0) - Conformance vectors:
conformance/v1/
The shape
ServeRequest ──▶ authorise (JCP grant) ──▶ route (cheapest-sufficient)
│
▼
meter ◀── serve (engine) ──▶ tokens
│
▼
sealed JCR-1 receipt: µJ / accepted token
Four roles — caller, router, engine, steward — and four phases
per serve: authorise, route, meter, seal. See SPEC.md §2.
The reference workspace
mesh-rs/ is a pure-Rust reference implementation:
| Crate | What it is |
|---|---|
mesh-schema |
wire types — ServeRequest, Fidelity, two-part Energy, RouteDecision, ServeOutcome |
mesh-router |
energy-weighted cheapest-sufficient routing with a full candidate audit |
mesh-engine |
the ServingEngine interface + a deterministic, weights-free ReferenceEngine |
mesh-engine-gguf |
a real backend (behind gguf): wraps the family's pure-Rust GGUF/Metal inference substrate so the mesh routes and meters actual model tokens |
mesh-meter |
the meter window; an always-available estimator + an optional bridge to the family hardware meter (RAPL / NVML / Apple) behind hw-meter |
mesh-receipt |
the mesh/1 receipt, JCR-1 sealing (det-CBOR + COSE_Sign1), and a hash-chained log |
mesh-core |
the orchestration that ties authorise → route → meter → seal together |
mesh-conformance |
the deterministic reference scenario, its checks, and the gen-vectors bin |
Mesh rides on JCP (a serve request is authorised by a JCP grant;
receipts seal in JCR-1), dispatches to runtimes such as
JouleClaw, and routes across compute described by
EOC and isolated by Sandbox. It introduces no new
cryptography (see SPEC.md §9).
Run it
cd mesh-rs
# all crates green
cargo test --workspace
cargo clippy --workspace --all-targets
# end-to-end: mint a JCP grant, serve, print the sealed receipt
cargo run -p mesh-core --example demo
# on Linux/RAPL or a CUDA/NVML box, meter real silicon (HwShunt provenance)
cargo run -p mesh-core --example demo --features hw-meter-rapl
# regenerate the published conformance vectors
cargo run -p mesh-conformance --bin gen-vectors -- ../conformance/v1
# serve a REAL model end-to-end through Mesh (Falcon-H1 0.5B), seal the receipt
cargo run --release -p mesh-engine-gguf --features gguf --bin serve_falcon -- \
/path/to/falcon-h1-0.5b-instruct 12 "The capital of France is"
The serve_falcon run demonstrates the whole standard on real model tokens:
a JCP grant authorises each request (an underfunded grant is refused before any
decode runs), the cheapest-sufficient backend is chosen, the decode is metered,
and a mesh/1 receipt is sealed and hash-chained across the session — e.g.
prompt "The capital of France is" → " Paris. It is known for its historical landmarks", prompt "Photosynthesis is the process by which" → " plants convert sunlight into chemical energy in the form of", with both seals verifying
and the chain's prev-links intact.
The demo prints the route (every backend weighed, with the reason each was kept or rejected), the honest energy provenance, and the wedge metric — then verifies the sealed envelope under the steward key.
Status
v0.1.0-draft.1. Reference workspace builds and tests green (mesh-schema,
mesh-router, mesh-engine, mesh-meter, mesh-receipt, mesh-core,
mesh-conformance); cargo clippy --all-targets warning-free; published vectors
in conformance/v1/. The reference engine is deterministic and
weights-free; production backends implement the same ServingEngine interface —
mesh-engine-gguf (behind gguf) is a real one over the family's pure-Rust
GGUF/Metal substrate, kept out of the default build so it needs no weights.