open-standards/mesh/tools/joules-per-decision/README.md

# Joules per typed decision

A measurement harness, and what it found: **Mesh cannot price, rate or report a
typed decision**, because every quantity in the schema is a token count and a
decision emits no tokens.

Run it with `uv`:

```sh
uv venv --python 3.12 .venv
uv pip install --python .venv/bin/python laya-mlx
.venv/bin/python joules_per_decision.py
```

## What a decision model is

A decision model takes application *state* plus *typed questions* and returns
calibrated probabilities over a bounded answer set. It generates no text. The
category appeared between 2026-09-15 (TypeSafe's Jev) and 2026-10-01
(Cloudflare's Clef), converging on three primitives:

| primitive | returns |
|---|---|
| `noul` | P(true) for a proposition |
| `choice` | a probability per named option, 1–255 options |
| `score` | a probability per level of an ordered rubric |

The schema is published as `System One` (OpenAPI 3.1, `api.typesafe.ai`), and
Jev, Liquid d1, Kev, Clef and Laya all implement it. Clef states that it scores
schema choices **non-autoregressively** rather than generating tokens.

## Method

The harness measures `laya-mlx` — an independent Apache-2.0 MLX port of Convai's
Laya (ModernBERT-large, 421M, FP16) — on Apple Silicon.

Energy comes from `ri_energy_nj` in `proc_pid_rusage(getpid(), RUSAGE_INFO_V6)`,
the same counter `jcp-energy`'s rung 3 uses, read in-process so the reading is
`Coverage::Whole` for this process's CPU work. The ABI constants (`SIZE = 464`,
`ENERGY_NJ = 336`) are lifted from `jcp-energy`, which verified them with
`sizeof`/`offsetof` against the macOS 26, 26.5 and 27 SDKs; they are not
re-derived here. The harness applies `jcp-energy`'s own liveness rule before
trusting the counter — at least five advancing sample intervals and at least
80% of them, over at least 300 ms — and reports the result rather than assuming
it.

Because the counter is per-process, the **energy** figures are unaffected by
other load on the host. The **latency** figures are not, so they are upper
bounds.

## Results — M5 Max, macOS 27, 2026-10-04

| device | questions | p50 | p95 | J/decision | coverage |
|---|---|---:|---:|---:|---|
| CPU | 1 (`noul`) | 126.8 ms | 636.2 ms | **0.204** | Whole (CPU), HwShunt |
| CPU | 3 (`choice`+`score`+`noul`) | 629.1 ms | 1410.0 ms | **0.486** | Whole (CPU), HwShunt |
| GPU | 1 | 21.1 ms | 38.2 ms | 0.0079 | **floor only** |
| GPU | 3 | 17.1 ms | 22.8 ms | 0.0082 | **floor only** |

Counter liveness: 25 advancing intervals of 25.

Three questions cost *less* wall time than one, because all three ride a single
forward pass (`batch_size` defaults to 16) — the marginal cost of a question
inside a batch is near zero, which no per-token model predicts.

The GPU rows are a **floor, not the decision's energy**. They are the host-side
CPU energy only; the GPU's share is unmetered. `jcp-energy` has no
whole-coverage rung for GPU-resident work on Apple Silicon and correctly refuses
to pass off IOReport's machine-wide GPU rail as one task's energy, so no
whole-task joule figure for the GPU path can be stated here. Every local
inference on a Mac is GPU-resident, so this is the gap that matters most.

## What the measurements show about this spec

Four findings, each checked against the code rather than inferred.

1. **`uj_per_accepted_token` is undefined for the whole category.**
   `MeshReceipt::uj_per_accepted_token` is documented as "the wedge metric" and
   is `checked_div` by `outcome.tokens_accepted`. A decision's
   `tokens_accepted` is 0 — measured `"output_tokens": 0` on every request — so
   the metric returns `None`. The receipt seals, and the number that makes it
   worth sealing does not exist.

2. **The unit of work is implicitly the accepted output token.** Every field of
   `ServeOutcome` is a token count: `tokens_in`, `tokens_out`,
   `tokens_accepted`, `cache_hit_tokens`, `spec_proposed`, `spec_accepted`. A
   decision's unit is the answered question, and nothing in the outcome can
   carry it.

3. **The cost model prices the wrong thing.** `CostProfile::estimate_uj` is
   charged `tokens = prompt_tokens + max_tokens`. With `max_tokens = 0` a
   decision collapses to `base_uj` plus its prompt, which ignores what the pass
   actually does: score N questions × K options. And the split inside the
   prompt is invisible — in the measured 3-question request, **121 of 132 input
   tokens were the question schema and 11 were the state**. A fixed schema is a
   cacheable prefix (`laya-mlx` exposes `cache_prompts`), so 92% of a real
   decision request is re-priced on every call as though it were new input.

4. **`fidelity_met` is a self-declaration sealed into a signed receipt.**
   `mesh-core` computes `fidelity_met = chosen_fidelity >= req.fidelity_floor`,
   where `chosen_fidelity` is the level the backend *advertises* (SPEC §3.2:
   "ordered scale F0 … F4"). `Energy` in the same receipt carries a provenance
   tag — `HwShunt | ModelBased | Estimator` — with a normative coupling rule.
   Fidelity carries nothing. The receipt is honest about where its joules came
   from and silent about where its quality claim came from.

   Decision models are what makes this fixable rather than inherent: for a
   bounded question, calibration is measurable. Kev publishes Brier scores
   (27B: 0.225) alongside accuracy; Clef trains against a Brier loss. A
   fidelity rating can cite an evaluation instead of asserting a level.

## Why a decision receipt is worth more than a generation receipt

A decision is a single forward pass with no sampling. Given the weight hash,
revision, dtype and the fitted temperatures, it is reproducible — so a receipt
over one is verifiable **by re-execution**, which a receipt over sampled text
can never be. `laya-mlx` publishes pinned revisions and per-file checksums, and
reports 63/63 validation questions matching upstream's selected answer in both
FP32 and FP16, which is exactly the material a receipt would bind.

The precision matters and must be bound: probabilities differ across FP16 and
FP32 even where the selected label agrees. A receipt that omits dtype does not
reproduce.

## The defect this category ships with today

Loading the published Laya checkpoint raises:

```
laya-mlx: this checkpoint ships temperatures outside [0.5, 5] which would
distort confidence; clamping choice:11+=0.1006. Treat confidence from the
affected buckets as uncalibrated.
```

Confirmed on the real weights: `temperature_by_options_raw["choice:11+"]` is
`0.10058280825614929`. A temperature of 0.1 sharpens logits roughly tenfold, so
the shipped checkpoint reports near-certainty on what may be close to a coin
flip, for every choice question with 11 or more options. The porter found it by
reproducing upstream numerically and clamped it; the publisher ships it.

That is the argument for this work in one line: **a calibrated probability is a
claim, and this category ships claims with no receipt.**

## The attribution trade — two fabrics, one decision

`two_fabrics.py` measures a decision model on the CPU, where the kernel's
per-process counter attributes energy to a **principal**, and compares it with
the same class of work on the GPU, where the only counter is an SoC rail that
counts the silicon and cannot tell one process from another.

Measured on an M5 Max, macOS 27, 2026-10-04. Both CPU rows are
`Attribution::Process`, `Coverage::Whole` for CPU work, over spans far longer
than the counter's declared 10 ms period:

| model | params | runtime | fabric | J/decision | p50 | attribution |
|---|---:|---|---|---:|---:|---|
| Laya | 421M | MLX | CPU | **0.486** | 629 ms | Process, whole |
| GLiNER2.5-Decide | 340M | torch | CPU | **0.555** | 480 ms | Process, whole |
| Laya | 421M | MLX | GPU | **0.104** | 11.9 ms | **Machine** (GPU share) |

With the fabric held constant the two models are close: GLiNER2.5-Decide spends
14% more energy per decision and returns it 24% faster — while scoring 60.1%
against Laya's 46.6% on Fastino's 17-dataset suite, so on this evidence the
better-scoring model is not the more expensive one in any meaningful sense.

Moving the *same* model to the GPU costs **4.7× fewer joules and is 53×
faster** — and the figure stops being attributable. IOReport's `GPU Energy`
rail is a real reading of the whole GPU; it cannot say whose work it was. So
the cheap path produces a number that cannot be bound to a principal, and the
expensive path produces one that can.

That is the trade, and it is not a defect in either counter. It is why
`Attribution` is a separate axis from `Coverage`: a receipt that wants to say
*this principal spent these joules* has to choose the CPU path today, and a
receipt that only needs *this much energy was spent here* can take the GPU.

The CPU figure is complete, not merely attributable. Over the GLiNER run the
GPU rail, bracketed by idle baselines, moved by **−0.0013 J per decision** —
negative, meaning the idle estimate slightly exceeded the real draw, so GPU
involvement is below the measurement's own noise. The process-attributed CPU
number is therefore the whole task, which is exactly what the GPU rows cannot
claim.

## ⚠ Withdrawn: the GPU figure is not this decision's energy

`gpu_share.py` measures what the table above assumed away. `accumulatedGPUTime`
per IOUserClient (I/O Registry, IOAccelerator subtree) gives each process's GPU
time; the cumulative totals are useless because WindowServer has been running
for days, so what matters is the **delta over the measured span**.

Measured on this host, over a 4.06 s span while an MLX matmul loop ran:

| process | GPU time in span | share |
|---|---:|---:|
| com.apple.Safari | 2.470 s | **99.5%** |
| WindowServer | 0.007 s | 0.3% |
| PhotosReliveWidget | 0.005 s | 0.2% |
| **this process** | **0.000 s** | **0.0%** |

So a single unrelated application held essentially the whole GPU for the
duration. A rail delta taken over that span is a reading of Safari, not of the
decision. The host was at load average 132 on 18 CPUs at the time, running
another session's VVC/VTM encodes — which is the normal state of this machine,
not an unlucky moment.

**Consequences for the table above:**

- The two **CPU** rows stand unchanged. The kernel's per-process counter
  attributes to a principal, so other processes' load does not enter them.
  That is the whole value of `Attribution::Process`, demonstrated the hard way.
- The **GPU** row is withdrawn as a per-decision energy figure. 0.104 J is an
  upper bound on *machine* GPU energy over the span, and on this host the bound
  is dominated by other processes. Its **latency** (11.9 ms p50) is unaffected.
- The idle-bracket correction of 0.08% that I read as evidence of cleanliness
  proves nothing. A bracket subtracts *steady* background; it cannot subtract a
  contending process whose work overlaps the span and not the brackets.

The rule to apply, already shipped elsewhere in the fleet: a rail reading may
be attributed to this process only when this process holds ≥99% of the GPU time
advancing in the span; where it holds none, its GPU energy is exactly zero; and
otherwise **no GPU figure is issued** and the span is CPU-only and partial. By
that rule this measurement issues no GPU figure at all.

This is also why time-share apportionment cannot rescue it: apportioning a rail
by GPU-time share was falsified elsewhere at 3–15× error, because a contending
job can draw an order of magnitude more watts per GPU-second than a decode, and
GPU time itself inflates ~5.4× under contention.

**To get a sound GPU figure**, re-measure on a host with no display server and
no other GPU client — which on this fleet means a headless Linux box with NVML,
not a Mac.