# Joules per typed decision A measurement harness, and what it found: **Mesh cannot price, rate or report a typed decision**, because every quantity in the schema is a token count and a decision emits no tokens. Run it with `uv`: ```sh uv venv --python 3.12 .venv uv pip install --python .venv/bin/python laya-mlx .venv/bin/python joules_per_decision.py ``` ## What a decision model is A decision model takes application *state* plus *typed questions* and returns calibrated probabilities over a bounded answer set. It generates no text. The category appeared between 2026-09-15 (TypeSafe's Jev) and 2026-10-01 (Cloudflare's Clef), converging on three primitives: | primitive | returns | |---|---| | `noul` | P(true) for a proposition | | `choice` | a probability per named option, 1–255 options | | `score` | a probability per level of an ordered rubric | The schema is published as `System One` (OpenAPI 3.1, `api.typesafe.ai`), and Jev, Liquid d1, Kev, Clef and Laya all implement it. Clef states that it scores schema choices **non-autoregressively** rather than generating tokens. ## Method The harness measures `laya-mlx` — an independent Apache-2.0 MLX port of Convai's Laya (ModernBERT-large, 421M, FP16) — on Apple Silicon. Energy comes from `ri_energy_nj` in `proc_pid_rusage(getpid(), RUSAGE_INFO_V6)`, the same counter `jcp-energy`'s rung 3 uses, read in-process so the reading is `Coverage::Whole` for this process's CPU work. The ABI constants (`SIZE = 464`, `ENERGY_NJ = 336`) are lifted from `jcp-energy`, which verified them with `sizeof`/`offsetof` against the macOS 26, 26.5 and 27 SDKs; they are not re-derived here. The harness applies `jcp-energy`'s own liveness rule before trusting the counter — at least five advancing sample intervals and at least 80% of them, over at least 300 ms — and reports the result rather than assuming it. Because the counter is per-process, the **energy** figures are unaffected by other load on the host. The **latency** figures are not, so they are upper bounds. ## Results — M5 Max, macOS 27, 2026-10-04 | device | questions | p50 | p95 | J/decision | coverage | |---|---|---:|---:|---:|---| | CPU | 1 (`noul`) | 126.8 ms | 636.2 ms | **0.204** | Whole (CPU), HwShunt | | CPU | 3 (`choice`+`score`+`noul`) | 629.1 ms | 1410.0 ms | **0.486** | Whole (CPU), HwShunt | | GPU | 1 | 21.1 ms | 38.2 ms | 0.0079 | **floor only** | | GPU | 3 | 17.1 ms | 22.8 ms | 0.0082 | **floor only** | Counter liveness: 25 advancing intervals of 25. Three questions cost *less* wall time than one, because all three ride a single forward pass (`batch_size` defaults to 16) — the marginal cost of a question inside a batch is near zero, which no per-token model predicts. The GPU rows are a **floor, not the decision's energy**. They are the host-side CPU energy only; the GPU's share is unmetered. `jcp-energy` has no whole-coverage rung for GPU-resident work on Apple Silicon and correctly refuses to pass off IOReport's machine-wide GPU rail as one task's energy, so no whole-task joule figure for the GPU path can be stated here. Every local inference on a Mac is GPU-resident, so this is the gap that matters most. ## What the measurements show about this spec Four findings, each checked against the code rather than inferred. 1. **`uj_per_accepted_token` is undefined for the whole category.** `MeshReceipt::uj_per_accepted_token` is documented as "the wedge metric" and is `checked_div` by `outcome.tokens_accepted`. A decision's `tokens_accepted` is 0 — measured `"output_tokens": 0` on every request — so the metric returns `None`. The receipt seals, and the number that makes it worth sealing does not exist. 2. **The unit of work is implicitly the accepted output token.** Every field of `ServeOutcome` is a token count: `tokens_in`, `tokens_out`, `tokens_accepted`, `cache_hit_tokens`, `spec_proposed`, `spec_accepted`. A decision's unit is the answered question, and nothing in the outcome can carry it. 3. **The cost model prices the wrong thing.** `CostProfile::estimate_uj` is charged `tokens = prompt_tokens + max_tokens`. With `max_tokens = 0` a decision collapses to `base_uj` plus its prompt, which ignores what the pass actually does: score N questions × K options. And the split inside the prompt is invisible — in the measured 3-question request, **121 of 132 input tokens were the question schema and 11 were the state**. A fixed schema is a cacheable prefix (`laya-mlx` exposes `cache_prompts`), so 92% of a real decision request is re-priced on every call as though it were new input. 4. **`fidelity_met` is a self-declaration sealed into a signed receipt.** `mesh-core` computes `fidelity_met = chosen_fidelity >= req.fidelity_floor`, where `chosen_fidelity` is the level the backend *advertises* (SPEC §3.2: "ordered scale F0 … F4"). `Energy` in the same receipt carries a provenance tag — `HwShunt | ModelBased | Estimator` — with a normative coupling rule. Fidelity carries nothing. The receipt is honest about where its joules came from and silent about where its quality claim came from. Decision models are what makes this fixable rather than inherent: for a bounded question, calibration is measurable. Kev publishes Brier scores (27B: 0.225) alongside accuracy; Clef trains against a Brier loss. A fidelity rating can cite an evaluation instead of asserting a level. ## Why a decision receipt is worth more than a generation receipt A decision is a single forward pass with no sampling. Given the weight hash, revision, dtype and the fitted temperatures, it is reproducible — so a receipt over one is verifiable **by re-execution**, which a receipt over sampled text can never be. `laya-mlx` publishes pinned revisions and per-file checksums, and reports 63/63 validation questions matching upstream's selected answer in both FP32 and FP16, which is exactly the material a receipt would bind. The precision matters and must be bound: probabilities differ across FP16 and FP32 even where the selected label agrees. A receipt that omits dtype does not reproduce. ## The defect this category ships with today Loading the published Laya checkpoint raises: ``` laya-mlx: this checkpoint ships temperatures outside [0.5, 5] which would distort confidence; clamping choice:11+=0.1006. Treat confidence from the affected buckets as uncalibrated. ``` Confirmed on the real weights: `temperature_by_options_raw["choice:11+"]` is `0.10058280825614929`. A temperature of 0.1 sharpens logits roughly tenfold, so the shipped checkpoint reports near-certainty on what may be close to a coin flip, for every choice question with 11 or more options. The porter found it by reproducing upstream numerically and clamped it; the publisher ships it. That is the argument for this work in one line: **a calibrated probability is a claim, and this category ships claims with no receipt.** ## The attribution trade — two fabrics, one decision `two_fabrics.py` measures a decision model on the CPU, where the kernel's per-process counter attributes energy to a **principal**, and compares it with the same class of work on the GPU, where the only counter is an SoC rail that counts the silicon and cannot tell one process from another. Measured on an M5 Max, macOS 27, 2026-10-04. Both CPU rows are `Attribution::Process`, `Coverage::Whole` for CPU work, over spans far longer than the counter's declared 10 ms period: | model | params | runtime | fabric | J/decision | p50 | attribution | |---|---:|---|---|---:|---:|---| | Laya | 421M | MLX | CPU | **0.486** | 629 ms | Process, whole | | GLiNER2.5-Decide | 340M | torch | CPU | **0.555** | 480 ms | Process, whole | | Laya | 421M | MLX | GPU | **0.104** | 11.9 ms | **Machine** (GPU share) | With the fabric held constant the two models are close: GLiNER2.5-Decide spends 14% more energy per decision and returns it 24% faster — while scoring 60.1% against Laya's 46.6% on Fastino's 17-dataset suite, so on this evidence the better-scoring model is not the more expensive one in any meaningful sense. Moving the *same* model to the GPU costs **4.7× fewer joules and is 53× faster** — and the figure stops being attributable. IOReport's `GPU Energy` rail is a real reading of the whole GPU; it cannot say whose work it was. So the cheap path produces a number that cannot be bound to a principal, and the expensive path produces one that can. That is the trade, and it is not a defect in either counter. It is why `Attribution` is a separate axis from `Coverage`: a receipt that wants to say *this principal spent these joules* has to choose the CPU path today, and a receipt that only needs *this much energy was spent here* can take the GPU. The CPU figure is complete, not merely attributable. Over the GLiNER run the GPU rail, bracketed by idle baselines, moved by **−0.0013 J per decision** — negative, meaning the idle estimate slightly exceeded the real draw, so GPU involvement is below the measurement's own noise. The process-attributed CPU number is therefore the whole task, which is exactly what the GPU rows cannot claim. ## ⚠ Withdrawn: the GPU figure is not this decision's energy `gpu_share.py` measures what the table above assumed away. `accumulatedGPUTime` per IOUserClient (I/O Registry, IOAccelerator subtree) gives each process's GPU time; the cumulative totals are useless because WindowServer has been running for days, so what matters is the **delta over the measured span**. Measured on this host, over a 4.06 s span while an MLX matmul loop ran: | process | GPU time in span | share | |---|---:|---:| | com.apple.Safari | 2.470 s | **99.5%** | | WindowServer | 0.007 s | 0.3% | | PhotosReliveWidget | 0.005 s | 0.2% | | **this process** | **0.000 s** | **0.0%** | So a single unrelated application held essentially the whole GPU for the duration. A rail delta taken over that span is a reading of Safari, not of the decision. The host was at load average 132 on 18 CPUs at the time, running another session's VVC/VTM encodes — which is the normal state of this machine, not an unlucky moment. **Consequences for the table above:** - The two **CPU** rows stand unchanged. The kernel's per-process counter attributes to a principal, so other processes' load does not enter them. That is the whole value of `Attribution::Process`, demonstrated the hard way. - The **GPU** row is withdrawn as a per-decision energy figure. 0.104 J is an upper bound on *machine* GPU energy over the span, and on this host the bound is dominated by other processes. Its **latency** (11.9 ms p50) is unaffected. - The idle-bracket correction of 0.08% that I read as evidence of cleanliness proves nothing. A bracket subtracts *steady* background; it cannot subtract a contending process whose work overlaps the span and not the brackets. The rule to apply, already shipped elsewhere in the fleet: a rail reading may be attributed to this process only when this process holds ≥99% of the GPU time advancing in the span; where it holds none, its GPU energy is exactly zero; and otherwise **no GPU figure is issued** and the span is CPU-only and partial. By that rule this measurement issues no GPU figure at all. This is also why time-share apportionment cannot rescue it: apportioning a rail by GPU-time share was falsified elsewhere at 3–15× error, because a contending job can draw an order of magnitude more watts per GPU-second than a decode, and GPU time itself inflates ~5.4× under contention. **To get a sound GPU figure**, re-measure on a host with no display server and no other GPU client — which on this fleet means a headless Linux box with NVML, not a Mac.