open-standards/mesh/tools/joules-per-decision

Joules per typed decision

A measurement harness, and what it found: Mesh cannot price, rate or report a typed decision, because every quantity in the schema is a token count and a decision emits no tokens.

Run it with uv:

uv venv --python 3.12 .venv
uv pip install --python .venv/bin/python laya-mlx
.venv/bin/python joules_per_decision.py

What a decision model is

A decision model takes application state plus typed questions and returns calibrated probabilities over a bounded answer set. It generates no text. The category appeared between 2026-09-15 (TypeSafe's Jev) and 2026-10-01 (Cloudflare's Clef), converging on three primitives:

primitive	returns
noul	P(true) for a proposition
choice	a probability per named option, 1–255 options
score	a probability per level of an ordered rubric

The schema is published as System One (OpenAPI 3.1, api.typesafe.ai), and Jev, Liquid d1, Kev, Clef and Laya all implement it. Clef states that it scores schema choices non-autoregressively rather than generating tokens.

Method

The harness measures laya-mlx — an independent Apache-2.0 MLX port of Convai's Laya (ModernBERT-large, 421M, FP16) — on Apple Silicon.

Energy comes from ri_energy_nj in proc_pid_rusage(getpid(), RUSAGE_INFO_V6), the same counter jcp-energy's rung 3 uses, read in-process so the reading is Coverage::Whole for this process's CPU work. The ABI constants (SIZE = 464, ENERGY_NJ = 336) are lifted from jcp-energy, which verified them with sizeof/offsetof against the macOS 26, 26.5 and 27 SDKs; they are not re-derived here. The harness applies jcp-energy's own liveness rule before trusting the counter — at least five advancing sample intervals and at least 80% of them, over at least 300 ms — and reports the result rather than assuming it.

Because the counter is per-process, the energy figures are unaffected by other load on the host. The latency figures are not, so they are upper bounds.

Results — M5 Max, macOS 27, 2026-10-04

device	questions	p50	p95	J/decision	coverage
CPU	1 (noul)	126.8 ms	636.2 ms	0.204	Whole (CPU), HwShunt
CPU	3 (choice+score+noul)	629.1 ms	1410.0 ms	0.486	Whole (CPU), HwShunt
GPU	1	21.1 ms	38.2 ms	0.0079	floor only
GPU	3	17.1 ms	22.8 ms	0.0082	floor only

Counter liveness: 25 advancing intervals of 25.

Three questions cost less wall time than one, because all three ride a single forward pass (batch_size defaults to 16) — the marginal cost of a question inside a batch is near zero, which no per-token model predicts.

The GPU rows are a floor, not the decision's energy. They are the host-side CPU energy only; the GPU's share is unmetered. jcp-energy has no whole-coverage rung for GPU-resident work on Apple Silicon and correctly refuses to pass off IOReport's machine-wide GPU rail as one task's energy, so no whole-task joule figure for the GPU path can be stated here. Every local inference on a Mac is GPU-resident, so this is the gap that matters most.

What the measurements show about this spec

Four findings, each checked against the code rather than inferred.

1. uj_per_accepted_token is undefined for the whole category. MeshReceipt::uj_per_accepted_token is documented as "the wedge metric" and is checked_div by outcome.tokens_accepted. A decision's tokens_accepted is 0 — measured "output_tokens": 0 on every request — so the metric returns None. The receipt seals, and the number that makes it worth sealing does not exist.

2. The unit of work is implicitly the accepted output token. Every field of ServeOutcome is a token count: tokens_in, tokens_out, tokens_accepted, cache_hit_tokens, spec_proposed, spec_accepted. A decision's unit is the answered question, and nothing in the outcome can carry it.

3. The cost model prices the wrong thing. CostProfile::estimate_uj is charged tokens = prompt_tokens + max_tokens. With max_tokens = 0 a decision collapses to base_uj plus its prompt, which ignores what the pass actually does: score N questions × K options. And the split inside the prompt is invisible — in the measured 3-question request, 121 of 132 input tokens were the question schema and 11 were the state. A fixed schema is a cacheable prefix (laya-mlx exposes cache_prompts), so 92% of a real decision request is re-priced on every call as though it were new input.

4. fidelity_met is a self-declaration sealed into a signed receipt. mesh-core computes fidelity_met = chosen_fidelity >= req.fidelity_floor, where chosen_fidelity is the level the backend advertises (SPEC §3.2: "ordered scale F0 … F4"). Energy in the same receipt carries a provenance tag — HwShunt | ModelBased | Estimator — with a normative coupling rule. Fidelity carries nothing. The receipt is honest about where its joules came from and silent about where its quality claim came from.

   Decision models are what makes this fixable rather than inherent: for a bounded question, calibration is measurable. Kev publishes Brier scores (27B: 0.225) alongside accuracy; Clef trains against a Brier loss. A fidelity rating can cite an evaluation instead of asserting a level.

Why a decision receipt is worth more than a generation receipt

A decision is a single forward pass with no sampling. Given the weight hash, revision, dtype and the fitted temperatures, it is reproducible — so a receipt over one is verifiable by re-execution, which a receipt over sampled text can never be. laya-mlx publishes pinned revisions and per-file checksums, and reports 63/63 validation questions matching upstream's selected answer in both FP32 and FP16, which is exactly the material a receipt would bind.

The precision matters and must be bound: probabilities differ across FP16 and FP32 even where the selected label agrees. A receipt that omits dtype does not reproduce.

The defect this category ships with today

Loading the published Laya checkpoint raises:

laya-mlx: this checkpoint ships temperatures outside [0.5, 5] which would
distort confidence; clamping choice:11+=0.1006. Treat confidence from the
affected buckets as uncalibrated.

Confirmed on the real weights: temperature_by_options_raw["choice:11+"] is 0.10058280825614929. A temperature of 0.1 sharpens logits roughly tenfold, so the shipped checkpoint reports near-certainty on what may be close to a coin flip, for every choice question with 11 or more options. The porter found it by reproducing upstream numerically and clamped it; the publisher ships it.

That is the argument for this work in one line: a calibrated probability is a claim, and this category ships claims with no receipt.

The attribution trade — two fabrics, one decision

two_fabrics.py measures a decision model on the CPU, where the kernel's per-process counter attributes energy to a principal, and compares it with the same class of work on the GPU, where the only counter is an SoC rail that counts the silicon and cannot tell one process from another.

Measured on an M5 Max, macOS 27, 2026-10-04. Both CPU rows are Attribution::Process, Coverage::Whole for CPU work, over spans far longer than the counter's declared 10 ms period:

model	params	runtime	fabric	J/decision	p50	attribution
Laya	421M	MLX	CPU	0.486	629 ms	Process, whole
GLiNER2.5-Decide	340M	torch	CPU	0.555	480 ms	Process, whole
Laya	421M	MLX	GPU	0.104	11.9 ms	Machine (GPU share)

With the fabric held constant the two models are close: GLiNER2.5-Decide spends 14% more energy per decision and returns it 24% faster — while scoring 60.1% against Laya's 46.6% on Fastino's 17-dataset suite, so on this evidence the better-scoring model is not the more expensive one in any meaningful sense.

Moving the same model to the GPU costs 4.7× fewer joules and is 53× faster — and the figure stops being attributable. IOReport's GPU Energy rail is a real reading of the whole GPU; it cannot say whose work it was. So the cheap path produces a number that cannot be bound to a principal, and the expensive path produces one that can.

That is the trade, and it is not a defect in either counter. It is why Attribution is a separate axis from Coverage: a receipt that wants to say this principal spent these joules has to choose the CPU path today, and a receipt that only needs this much energy was spent here can take the GPU.

The CPU figure is complete, not merely attributable. Over the GLiNER run the GPU rail, bracketed by idle baselines, moved by −0.0013 J per decision — negative, meaning the idle estimate slightly exceeded the real draw, so GPU involvement is below the measurement's own noise. The process-attributed CPU number is therefore the whole task, which is exactly what the GPU rows cannot claim.

⚠ Withdrawn: the GPU figure is not this decision's energy

gpu_share.py measures what the table above assumed away. accumulatedGPUTime per IOUserClient (I/O Registry, IOAccelerator subtree) gives each process's GPU time; the cumulative totals are useless because WindowServer has been running for days, so what matters is the delta over the measured span.

Measured on this host, over a 4.06 s span while an MLX matmul loop ran:

process	GPU time in span	share
com.apple.Safari	2.470 s	99.5%
WindowServer	0.007 s	0.3%
PhotosReliveWidget	0.005 s	0.2%
this process	0.000 s	0.0%

So a single unrelated application held essentially the whole GPU for the duration. A rail delta taken over that span is a reading of Safari, not of the decision. The host was at load average 132 on 18 CPUs at the time, running another session's VVC/VTM encodes — which is the normal state of this machine, not an unlucky moment.

Consequences for the table above:

• The two CPU rows stand unchanged. The kernel's per-process counter attributes to a principal, so other processes' load does not enter them. That is the whole value of Attribution::Process, demonstrated the hard way.
• The GPU row is withdrawn as a per-decision energy figure. 0.104 J is an upper bound on machine GPU energy over the span, and on this host the bound is dominated by other processes. Its latency (11.9 ms p50) is unaffected.
• The idle-bracket correction of 0.08% that I read as evidence of cleanliness proves nothing. A bracket subtracts steady background; it cannot subtract a contending process whose work overlaps the span and not the brackets.

The rule to apply, already shipped elsewhere in the fleet: a rail reading may be attributed to this process only when this process holds ≥99% of the GPU time advancing in the span; where it holds none, its GPU energy is exactly zero; and otherwise no GPU figure is issued and the span is CPU-only and partial. By that rule this measurement issues no GPU figure at all.

This is also why time-share apportionment cannot rescue it: apportioning a rail by GPU-time share was falsified elsewhere at 3–15× error, because a contending job can draw an order of magnitude more watts per GPU-second than a decode, and GPU time itself inflates ~5.4× under contention.

To get a sound GPU figure, re-measure on a host with no display server and no other GPU client — which on this fleet means a headless Linux box with NVML, not a Mac.