2f3b000153
Decision models — state plus typed questions in, a calibrated probability per
allowed answer out, no text generated — went from nonexistent to a category
between 2026-09-15 and 2026-10-01, converging on three primitives (`noul`,
`choice`, `score`) and one schema. Every quantity in Mesh is a token count, and
a decision emits no tokens, so the spec could neither price one, rate one, nor
report one. Three defects, each checked in the code rather than inferred:
- `MeshReceipt::uj_per_accepted_token`, documented as "the wedge metric", is
`checked_div` by `tokens_accepted`. A decision's is 0 — measured
`"output_tokens": 0` on every request — so the headline efficiency figure
returned `None` for the whole category. The receipt sealed with no number.
- `estimate()` charged `prompt_tokens + max_tokens`, so a caller that left
`max_tokens` at a generation's default was priced for an essay the backend
will never write, and refused at a ceiling that would have covered the work.
- Nothing could express the dimension a decision's cost actually follows: the
schema it scores. One question over 20 options is more work than one over 2
at identical token counts.
Added `DecisionShape` on the request and `DecisionOutcome` on the outcome
(questions, options scored, and how much of the input was schema rather than
state), `CostProfile::with_option_cost` + `estimate_decision_uj`, and
`uj_per_decision` / `uj_per_option_scored` on the receipt. SPEC §3.6 states the
three MUSTs; §4's estimate formula and §6's field table are updated.
Additive throughout, following the pattern `CostProfile::calibration` already
set here: the new fields are `Option`, skipped when absent, and a declared
option cost of zero reproduces the token-only model exactly. So an existing
request's `request_id` is unchanged, a generation's signed core gains no keys,
and **the sealed conformance vectors in `conformance/v1/reference` still verify
byte-identically** — which is the real test that this is compatible, and it
passes. 52 tests, 0 failures, 0 clippy findings.
Two of the four new tests are the defects stated as invariants: a decision
receipt now has an efficiency figure where the token metric has none, and a
decision is not priced for tokens it never emits. Both were falsified by
reverting the fix — removing the `estimate()` branch fails two router tests,
making `uj_per_decision` read `tokens_accepted` fails two receipt tests.
`tools/joules-per-decision/` is the evidence: a harness that meters a real
Apache-2.0 decision model (Laya, ModernBERT-large 421M, via MLX) per fabric on
Apple silicon, and the numbers in §3.6 come from it. On an M5 Max a
three-question decision costs **0.104 J** — 0.0966 J on the GPU rail plus
0.0075 J of process CPU energy — at an 11.85 ms p50, which is ~35 mJ per
answered question. Idle baselines bracket the run and correct it by 0.08%, so
the figure is measurement, not modelling. The energy counters are read
unprivileged; the CPU-rail/GPU-rail asymmetry on macOS is noted there and is
the subject of follow-up work on `jcp-energy`.
The harness also reproduced a live defect in the category it measures: the
published Laya checkpoint ships a fitted temperature of 0.1006 for choice
questions with 11+ options, which sharpens logits roughly tenfold and reports
near-certainty on what may be a coin flip. A third-party porter found it by
reproducing upstream numerically and clamps it with a warning; the publisher
ships it. A calibrated probability is a claim, and this category ships claims
with no receipt — which is the argument for §3.6 in one line.
Co-Authored-By: Claude Opus 5 <[email protected]>
350 lines
16 KiB
Markdown
350 lines
16 KiB
Markdown
# Mesh — Energy-Accounted Inference Serving & Routing
|
||
|
||
**Version:** 0.1.0-draft.1 · **Status:** Draft · **Profile:** `mesh/1`
|
||
**Specification text:** CC-BY-4.0 · **Reference code (`mesh-rs/`):** Apache-2.0
|
||
|
||
Mesh is the open standard for **serving and routing inference as a metered
|
||
transaction**. A request declares a joule ceiling and a fidelity floor; the mesh
|
||
routes it to the cheapest-sufficient model on the cheapest-sufficient backend,
|
||
serves it under an energy meter, and seals a receipt whose headline figure is
|
||
**microjoules per accepted output token** — measured at the meter, not estimated
|
||
from a price list.
|
||
|
||
Mesh is one of the [Transaction Science open standards](../README.md).
|
||
Transaction Science is the steward, not the proprietor: the wire format and the
|
||
right to fork are public.
|
||
|
||
---
|
||
|
||
## §1 Why Mesh exists
|
||
|
||
The serving shelf reports two numbers: dollars per token and tokens per second.
|
||
Neither tells an operator, an auditor, or a regulator what an inference **cost**
|
||
— in the one unit that is physical, comparable across hardware, and impossible to
|
||
inflate: the joule. Mesh makes that number first-class. Every served inference
|
||
produces a signed receipt carrying the route that was taken and *why*, the recipe
|
||
that served it, the honest provenance of the energy figure, and the joules per
|
||
accepted token.
|
||
|
||
Mesh does not replace the runtime that executes one inference, nor the substrate
|
||
that supplies the compute. It is the **front door** in front of many models and
|
||
many backends: the layer that decides where a request goes, meters what it spent,
|
||
and proves it (§9).
|
||
|
||
---
|
||
|
||
## §2 Roles and the serve lifecycle
|
||
|
||
A Mesh deployment has four roles:
|
||
|
||
- **Caller** — issues a `ServeRequest` and, optionally, the JCP grant that
|
||
authorises it.
|
||
- **Router** — selects a backend (§4).
|
||
- **Engine** — a backend that serves the request (§5).
|
||
- **Steward** — the endpoint that meters the span (§5/§6) and seals the receipt
|
||
(§6). The steward identity is the COSE `kid` and signs the receipt log.
|
||
|
||
One serve proceeds in four phases, in order:
|
||
|
||
1. **Authorise.** If the request rides on a JCP `SignedGrant`, the steward MUST
|
||
verify the issuer signature, confirm the capability permits the action
|
||
`infer` on the request's content id, and confirm the grant's joule budget is
|
||
≥ the request's joule ceiling. A failure at any step aborts the serve before
|
||
any work is done.
|
||
2. **Route.** The router selects the cheapest admissible backend (§4). If none is
|
||
admissible, the serve fails with the router's recorded reason.
|
||
3. **Meter.** The steward opens an energy window, calls the engine, and closes
|
||
the window. Energy MUST be metered **around** the engine; an engine MUST NOT
|
||
self-report energy.
|
||
4. **Seal.** The steward builds the receipt (§6), stamps it into its hash-chained
|
||
log, and seals it as a JCR-1 envelope.
|
||
|
||
---
|
||
|
||
## §3 The request and the energy record
|
||
|
||
### §3.1 `ServeRequest`
|
||
|
||
A request is content-addressed. Its identity `request_id = ContentId.of(request)`
|
||
is the resource a JCP capability authorises and the `request_id` on the receipt.
|
||
|
||
| Field | Type | Meaning |
|
||
|---|---|---|
|
||
| `prompt_hash` | ContentId (BLAKE3) | content address of the prompt bytes — never the prompt text |
|
||
| `prompt_tokens` | u32 | declared input token count (for the estimate) |
|
||
| `max_tokens` | u32 | output token ceiling |
|
||
| `fidelity_floor` | Fidelity | minimum acceptable fidelity (§3.2) |
|
||
| `joule_ceiling_uj` | u64 | hard energy budget, microjoules |
|
||
|
||
A conforming request MUST carry both a `fidelity_floor` and a `joule_ceiling_uj`.
|
||
The pair is the contract: *give me an answer at least this good, for at most this
|
||
much energy.*
|
||
|
||
A request MAY additionally carry a `decision` (§3.6), which says it is asking for
|
||
a bounded typed decision rather than generated text. Its absence is a generation,
|
||
and is omitted from the canonical form — so the `request_id` of every request
|
||
written before this field existed is unchanged.
|
||
|
||
### §3.2 Fidelity
|
||
|
||
Fidelity is an ordered scale `F0 … F4` — `F0` the coarsest sufficient answer
|
||
(cache / lawful / tiny model), `F4` a frontier-grade response. Ordering is
|
||
normative: a backend is admissible only when its level ≥ the request's floor
|
||
(§4).
|
||
|
||
### §3.3 Energy and provenance
|
||
|
||
Energy is a two-part integer-microjoule record with an honest provenance tag:
|
||
|
||
```
|
||
Energy { measured_uj: Option<u64>, estimated_uj: u64, provenance: Provenance }
|
||
Provenance ∈ { HwShunt, ModelBased, Estimator }
|
||
```
|
||
|
||
The coupling is normative and MUST be enforced before a receipt is sealed:
|
||
|
||
- `HwShunt` ⇒ `measured_uj` is present (a hardware reading exists).
|
||
- `Estimator` ⇒ `measured_uj` is absent (no measurement is implied).
|
||
- `ModelBased` ⇒ `measured_uj` MAY be absent.
|
||
|
||
A receipt that violates the coupling MUST NOT be sealable. This is the rule that
|
||
keeps "measured" honest: a deployment with no hardware meter reports `Estimator`
|
||
and says so, rather than dressing an estimate up as a measurement.
|
||
|
||
`HwShunt` covers a shunt and a vendor energy counter (RAPL, NVML, Apple
|
||
IOReport), but a counter counts only while it is live:
|
||
|
||
> A reading from a vendor energy counter (RAPL, NVML, Apple IOReport, …) MAY be
|
||
> graded as a hardware reading only while the counter is demonstrably live for
|
||
> the domain the figure claims: the implementation MUST have observed the
|
||
> counter covering that domain advance under load (for Apple IOReport: the CPU
|
||
> energy rail itself advancing in at least 5 sample intervals, in at least 4 of
|
||
> every 5, over at least 300 ms while the probing thread burns CPU). A counter
|
||
> that is present but not live — frozen, or republished in lumps — MUST NOT
|
||
> produce a hardware-graded figure; the implementation falls back to its
|
||
> estimator or reports the work unmetered, and SHOULD record why. A figure MUST
|
||
> NOT claim coverage of a domain its counter does not read; where the format
|
||
> has a coverage field, such a figure is Partial.
|
||
|
||
`mesh/1` has no coverage field. So a figure from a counter that does not cover
|
||
the whole request, such as a GPU counter on a host whose CPU has none, MUST NOT
|
||
be graded `HwShunt`.
|
||
|
||
### §3.6 Typed decisions — the other unit of work
|
||
|
||
A **decision model** takes state plus typed questions and returns a calibrated
|
||
probability per allowed answer, generating no text. The category converged
|
||
through 2026 on three primitives — a proposition (`noul`), a choice over 1–255
|
||
named options, and a score over an ordered rubric — and backends score the
|
||
schema **non-autoregressively** rather than emitting tokens.
|
||
|
||
Everything else in this spec measures work in tokens, and a decision's output
|
||
token count is zero. Measured on a 421M decision model: `output_tokens: 0`, with
|
||
132 input tokens of which **121 were the question schema and 11 the state under
|
||
judgement**. Three consequences follow, and a conforming implementation MUST
|
||
handle each.
|
||
|
||
A request asking for a decision carries:
|
||
|
||
| Field | Type | Meaning |
|
||
|---|---|---|
|
||
| `questions` | u32 | typed questions in the request |
|
||
| `options_scored` | u32 | options scored across all of them |
|
||
| `schema_tokens` | u32 | how many of `prompt_tokens` are schema, not state |
|
||
|
||
1. **It MUST NOT be priced for tokens it cannot emit.** The estimate for a
|
||
decision MUST exclude `max_tokens`. Including it refuses work at a ceiling
|
||
that would in fact have covered it.
|
||
2. **Cost MUST be allowed to follow the schema.** A profile MAY declare a cost
|
||
per scored option; one question over 20 options is more work than one over 2
|
||
at identical token counts, and no token count expresses the difference.
|
||
Declaring nothing reproduces the token-only model exactly, so an unmeasured
|
||
profile claims nothing.
|
||
3. **The efficiency figure MUST be defined.** `uj_per_accepted_token` (§6) is
|
||
undefined when no tokens were accepted, which is every request in this
|
||
category. A decision receipt MUST instead carry `uj_per_decision`, and
|
||
SHOULD carry `uj_per_option_scored`. The unit of work is the answered
|
||
question.
|
||
|
||
`schema_tokens` is reported because a fixed schema is a **cacheable input
|
||
prefix** — distinct from `cache_hit_tokens`, which counts output tokens served
|
||
from cache, of which a decision has none. In the measured request 92% of the
|
||
input was schema, so a reader that cannot see the split cannot tell a cold
|
||
decision from a warm one.
|
||
|
||
> **Why a decision receipt is worth more than a generation receipt.** A decision
|
||
> is a single forward pass with no sampling. Given the weight digest, revision
|
||
> and numeric precision, it is reproducible — so a receipt over one is verifiable
|
||
> **by re-execution**, which a receipt over sampled text can never be. A receipt
|
||
> for a decision SHOULD therefore bind the weight digest and the precision; the
|
||
> probabilities of a published decision model differ between `fp16` and `fp32`
|
||
> even where the selected answer agrees, so a receipt that omits precision does
|
||
> not reproduce.
|
||
|
||
---
|
||
|
||
## §4 Routing — cheapest-sufficient
|
||
|
||
For each registered backend the router computes an estimate
|
||
`est_total_uj = uj_per_token × (prompt_tokens + max_tokens)`, or for a typed
|
||
decision (§3.6) `uj_per_token × prompt_tokens + uj_per_option_scored ×
|
||
options_scored`, and marks it **admissible** iff:
|
||
|
||
```
|
||
backend.fidelity ≥ request.fidelity_floor AND est_total_uj ≤ request.joule_ceiling_uj
|
||
```
|
||
|
||
The router MUST choose the admissible backend with the **lowest** `est_total_uj`.
|
||
Ties are broken deterministically (registration order). When no backend is
|
||
admissible the serve fails, and the decision records why.
|
||
|
||
The router MUST record, for every backend it weighed, a `RouteCandidate` with its
|
||
fidelity, estimate, admissibility, and — when inadmissible — a human-readable
|
||
reason (below floor / over ceiling). The chosen route MUST carry a one-line
|
||
justification. This is what makes a route **auditable from the receipt alone**:
|
||
the cheapest sufficient answer is preferred and a frontier model is the expensive
|
||
last resort, and the comparison is denominated in joules.
|
||
|
||
A model is addressed as a **recipe** — a base plus optional merges/adapters — not
|
||
a fixed blob, so the receipt can record *which* composed model answered.
|
||
|
||
---
|
||
|
||
## §5 The serving engine
|
||
|
||
A backend implements one interface: advertise a fidelity level and a modeled
|
||
per-token energy cost, and serve a request producing tokens and a `ServeOutcome`:
|
||
|
||
```
|
||
ServeOutcome {
|
||
tokens_in, tokens_out, tokens_accepted,
|
||
cache_hit_tokens, spec_proposed, spec_accepted
|
||
}
|
||
```
|
||
|
||
The input token payload travels **with** the serve call, alongside the request —
|
||
it is never part of the request's sealed identity and never enters the receipt,
|
||
which records only the content-addressed `prompt_hash` (§3.1). A receipt is
|
||
therefore safe to publish without disclosing the prompt.
|
||
|
||
`tokens_accepted` is the denominator of the wedge metric. For greedy decode it
|
||
equals `tokens_out`; under speculative decoding it is the number of proposed
|
||
tokens the target model accepted. `cache_hit_tokens` records output served from a
|
||
prefix/result cache (energy saved). An engine MUST do real work proportional to
|
||
the tokens it serves so the surrounding meter reads a real span; it MUST NOT
|
||
self-report energy.
|
||
|
||
The reference engine (`mesh-engine::ReferenceEngine`) is deterministic and
|
||
weights-free, for conformance. Production backends — a quantized device-resident
|
||
decoder, a high-throughput batching server — implement the same interface and
|
||
drop in unchanged; `mesh-engine-gguf::GgufEngine` (behind the `gguf` feature) is
|
||
one such backend over a pure-Rust GGUF/Metal decoder, so the mesh routes and
|
||
meters real model tokens.
|
||
|
||
---
|
||
|
||
## §6 The receipt — `mesh/1`
|
||
|
||
A `MeshReceipt` is sealed with **JCR-1**: canonical det-CBOR (RFC 8949 §4.2.1)
|
||
inside a COSE_Sign1 envelope (RFC 9052, EdDSA/Ed25519, `alg = -8`), byte-compatible
|
||
with the rest of the family's receipts. The signed core forbids floats, booleans,
|
||
and nulls; booleans are encoded as `0|1` uints and absent options dropped.
|
||
|
||
### §6.1 Wire fields (`mesh/1`)
|
||
|
||
| Key | Meaning |
|
||
|---|---|
|
||
| `mesh_receipt` | `"mesh/1"` |
|
||
| `request_id` | hex content id of the request |
|
||
| `grant_id` | hex JCP grant id this inference debited (omitted if ungranted) |
|
||
| `backend`, `recipe` | the backend and composed model that served it |
|
||
| `route` | the router's one-line justification |
|
||
| `tokens_in/out/accepted`, `cache_hit_tokens`, `spec_proposed/accepted` | the outcome counters |
|
||
| `fidelity_floor` | the floor the request demanded (uint) |
|
||
| `fidelity_met` | `0|1` — did the served backend meet the floor |
|
||
| `estimated_uj` | always present |
|
||
| `measured_uj` | present **iff** provenance is `HwShunt` |
|
||
| `uj_per_accepted_token` | **the wedge metric** for a generation — `energy.uj() / tokens_accepted`; omitted when no tokens were accepted |
|
||
| `questions_answered`, `options_scored`, `schema_tokens` | present **iff** the request was a typed decision (§3.6) |
|
||
| `uj_per_decision` | **the wedge metric** for a decision — `energy.uj() / questions_answered` |
|
||
| `uj_per_option_scored` | the finer decision figure — `energy.uj() / options_scored` |
|
||
| `energy_provenance` | `HwShunt | ModelBased | Estimator` |
|
||
| `sequence`, `prev` | hash-chain position and prior envelope's content id |
|
||
| `nonce`, `steward` | per-receipt nonce and the sealing identity (COSE `kid`) |
|
||
|
||
### §6.2 The wedge metric
|
||
|
||
`uj_per_accepted_token = energy.uj() / tokens_accepted`, where `energy.uj()` is
|
||
the measured figure if present, else the estimate. It is `null`/absent when no
|
||
tokens were accepted. This is the number Mesh exists to make legible, and it is
|
||
sealed into every receipt.
|
||
|
||
### §6.3 Hash chain
|
||
|
||
A steward keeps a receipt log. Appending a receipt stamps its `sequence` (from 0)
|
||
and `prev` (the content id of the previous sealed envelope; absent at genesis),
|
||
seals it, and advances the head. The log is tamper-evident end to end: altering
|
||
any envelope breaks verification of that receipt and the chain link from the next.
|
||
|
||
---
|
||
|
||
## §7 Conformance
|
||
|
||
A conforming implementation MUST:
|
||
|
||
1. Reject a serve whose grant fails verification, does not permit `infer` on the
|
||
request, or carries a budget below the request's joule ceiling (§2.1).
|
||
2. Choose the cheapest admissible backend and record every candidate weighed
|
||
(§4).
|
||
3. Meter energy around the engine and tag provenance honestly, enforcing the
|
||
`Energy` coupling before sealing (§3.3).
|
||
4. Seal the receipt as a JCR-1 `mesh/1` envelope and compute
|
||
`uj_per_accepted_token` (§6).
|
||
5. Maintain a tamper-evident hash chain (§6.3).
|
||
|
||
The published vectors in [`../conformance/v1/`](../conformance/v1/) pin a
|
||
deterministic reference serve. `reference/signed_core.hex` MUST canonical-CBOR
|
||
encode identically across implementations, and `reference/envelope.hex` MUST be
|
||
reproducible from it with the steward key in `meta.json`. Regenerate with:
|
||
|
||
```text
|
||
cargo run -p mesh-conformance --bin gen-vectors -- ../conformance/v1
|
||
```
|
||
|
||
---
|
||
|
||
## §8 Security considerations
|
||
|
||
Mesh composes proven primitives: Ed25519 signatures, BLAKE3 content addressing,
|
||
RFC 9052 COSE_Sign1, and the JCP capability model. It introduces no new
|
||
cryptography. Two properties are correctness, not style:
|
||
|
||
- **Authorisation happens outside the model.** The grant check (§2.1) is enforced
|
||
by the steward before any inference runs; a prompt cannot grant itself budget
|
||
or capability.
|
||
- **Honesty is enforced at seal time.** A receipt cannot claim a measurement it
|
||
does not have (§3.3). An operator without a hardware meter produces honest
|
||
`Estimator` receipts.
|
||
|
||
Energy figures are only as honest as the worst meter in the span. A receipt's
|
||
provenance tag states which meter produced its figure; consumers SHOULD treat
|
||
`Estimator` figures as bounds, not measurements.
|
||
|
||
---
|
||
|
||
## §9 Relationship to the family
|
||
|
||
Mesh is the serving + routing fabric. It is expressed in primitives the family
|
||
already publishes, and it composes with the neighbouring standards rather than
|
||
duplicating them:
|
||
|
||
- It **rides on [JCP](../jcp/)**: a serve request is authorised by a JCP `Grant`
|
||
(capability ⊗ joule-budget), and the receipt seals in JCR-1 — the same envelope
|
||
every family receipt uses.
|
||
- It **dispatches to runtimes** such as [JouleClaw](../jouleclaw/): a backend that
|
||
executes one inference cheaply is one engine behind the mesh.
|
||
- It **routes across compute** described by [EOC](../eoc/) and isolated by
|
||
[Sandbox](../sandbox/): a backend may be a metered, attested execution target.
|
||
|
||
The boundary is deliberate: a runtime executes *one* inference; Mesh routes
|
||
across *many* models and *many* backends and proves what each one cost.
|