Transaction-Science/open-standards
Watch
1
0
Fork
You've already forked open-standards
 0
Code Issues Pull requests Projects Releases 15 Packages Wiki Activity Actions

Typed decisions, per-fabric energy, and two figures I had to withdraw #154

Merged
dcharlot merged 7 commits from feat/decision-models into main 2026-10-05 08:54:07 -04:00
Conversation 0 Commits 7 Files changed 20 +2283 -26
20 changed files with 2283 additions and 26 deletions
Download patch file Download diff file Expand all files Collapse all files

214
eoc/standards/sci-for-ai-energy-provenance-annex.md Normal file
View file

@ -0,0 +1,214 @@
# Proposed Annex: the provenance and validity of E
*Draft submission for filing to [github.com/Green-Software-Foundation/sci](https://github.com/Green-Software-Foundation/sci). Licensed CC-BY-4.0.*
*Status: working draft v0.1. Authors: Transaction Science. Target: SCI-for-AI, as an annex to the treatment of E. Companion to [`sci-for-ai-submission.md`](sci-for-ai-submission.md), which maps EOC receipts onto the four-component formula and assumes E is already trustworthy; this annex is about whether it is.*
---
## a. Summary
`SCI = (E × I + M) per R` is well specified in every term but one. `I` has a
location-based methodology. `M` has a lifecycle methodology. `R` is
deployment-defined and the specification is explicit that it must be *stated
and reproducible*. **`E` has no comparable requirement.** A figure may be
labelled "measured" without saying what measured it, over what span, on which
silicon, or on whose behalf — and in current practice it frequently is.
This annex proposes five requirements (E1–E5) that make an `E` figure
self-describing, and one refusal: **a figure taken over a span shorter than its
counter's update period is not a measurement of that span and MUST NOT be
reported as measured.** All five are machine-checkable, and a reference
implementation exists.
**What this annex does not propose.** No change to `R`. SCI-for-AI already
supports diverse functional units — tokens for language models, inferences for
classifiers, FLOPs for training — so non-generative inference, including the
typed-decision models that emerged in September and October 2026 and emit no
tokens at all, is already expressible as `R = one inference`. The gap is not in
the unit of work. It is in the energy.
## b. The problem, as evidenced
Three independent observations, none of them about a careless implementer:
1. **Published work that has a sensor reports only part of the machine.** A
survey of recent AI-energy measurement work finds the instrumented studies
are GPU-rail-only — no CPU package, no DRAM, no NIC, no storage, no
power-supply loss — and that none of them declares its sampling interval,
the device's update rate, or the integration method actually used.
2. **"Measured" is being applied to estimators.** A reference implementation in
this ecosystem stamps `measured` on the output of a lifecycle estimator
whose own documentation calls it an estimation. The alternative path uses
hard-coded kWh-per-token constants calibrated from a single figure at ±50%.
Neither is wrong as an estimate; both are wrong as a measurement.
3. **Per-call energy is being taken from provider SDKs**, which report a number
with no stated counter, period or boundary. The reporter cannot tell a
hardware reading from a model, and neither can the reader.
### Why this is not pedantry
The error is not symmetric. A figure used in a *budget* or a *ceiling* that
under-reports admits work the budget cannot cover, silently; one that
over-reports refuses work that would have fit, visibly. A figure used in a
*disclosure* that under-reports understates an obligation. In both cases the
unsafe direction is the one an uncharacterised counter takes.
And counters do take it, by large factors. A 588-run study of one widely used
cumulative-energy counter measured it **under-reporting integrated power by
95.4% at 0.5 ms sampling**, recovering to 1.6% only at 1 s, and recommended
against unvalidated use. Three weeks later a separate paper built its entire
per-token energy decomposition on that same counter, specifically *because*
power sampling cannot resolve short decode windows. Neither cites the other.
A third observation, on different hardware, has the same counter class reading
4–30× the board's actual draw when polled at 100 ms. Three mutually
inconsistent characterisations of one counter class — which means the validity
bound is itself sampling-interval-dependent and cannot be assumed.
We reproduced the shape on a fourth counter. Burning one thread's CPU and
sampling the macOS per-process energy counter 60 times per interval, the
fraction of intervals in which it advanced was **0% at 10 µs and 50 µs, 2% at
100 µs, 5% at 500 µs, 35% at 1 ms, 63% at 2 ms, 87% at 5 ms, and 100% at
10 ms**. Below 1 ms it does not lag smoothly — on the intervals where it moved
at all, the median step was ~20 mJ, two orders above the 1 ms step. It accrues
in lumps. A sampler reading it every 500 µs would see nothing most of the time
and a 20 mJ jump occasionally, and would report both as measurements.
## c. Proposed requirements
An `E` figure claiming to be **measured** MUST carry:
**E1 — Source.** The named counter or instrument: `rapl`, `nvml`, `rocm-smi`,
`level-zero`, `ioreport`, `proc-rusage`, `tegra-ina`, `qualcomm-qpt`,
`ascend-smi`, `hl-smi`, `cnmon`, `external-meter`, … Not the word "measured".
Two figures from different sources are not interchangeable: they have different
domains, different periods and different attribution.
**E2 — Period, and the span rule.** The counter's documented or measured update
period, and the span the figure covers. **If the span is shorter than the
period, the figure MUST be reported as modelled, not measured.** A source whose
period is *undeclared* can never support a measured claim at any span, because
the direction of its error is unknown. A per-operation figure for work shorter
than the period is sound only as the mean of many operations over a span longer
than it.
**E3 — Attribution boundary.** Whether the figure counts **one principal's**
work (a per-process or per-container counter) or **the whole machine** (an SoC
rail, a package counter, a board shunt). These are different claims, and the
distinction is independent of how much of the *task* is covered. A machine-wide
figure attributed to one workload MUST state the condition under which that is
valid — normally exclusive use — and MUST report the magnitude of its idle
correction, so a reader can see whether the correction is a rounding error or
the result.
**E4 — Fabric breakdown.** Where the work crosses silicon, `E` SHOULD be
reported per fabric (`cpu`, `gpu`, `npu`, `dram`, `package`, …) and MUST state
which fabrics are **not** covered. A total that silently omits a fabric the
work used is an understatement, and on accelerated inference the omitted
fabric is usually the dominant one.
**E5 — Why a fabric is uncovered.** Where a fabric is not measured, the reason
MUST be one of five, because they are not interchangeable:
| status | meaning |
|---|---|
| `frozen` | counter present, not advancing |
| `absent` | no such silicon on this host |
| `needs-privilege` | present, requires privilege the reporter lacks |
| `no-counter-exists` | present, and exposes **no energy counter at all** |
| `not-implemented` | a counter exists; this reporter has no reader for it |
Only `no-counter-exists` is a property of the hardware. The other four are
properties of the deployment or the tooling, and they are fixable.
This matters beyond accounting hygiene. The EU AI Act's GPAI Code of Practice
requires providers to document known or estimated energy consumption for
training *and* inference, while permitting an exemption where the provider
lacks "critical information from compute or hardware provider". That exemption
is a claim of `no-counter-exists`. E5 is what distinguishes it from the four
statuses that do not justify it — and an unprivileged on-die counter, which
needs no cooperation from the compute provider at all, narrows the set of
accelerators for which the claim can honestly be made.
## d. Worked example
A three-question typed decision served by a 421M-parameter decision model on an
Apple M5 Max, `R = one inference`:
```
E = 1.0413e-4 Wh (104 127 µJ)
R = 1 decision
source = { gpu: ioreport, cpu: proc-rusage }
period = 10 ms both; span = 2.606 s over 200 decisions → measured (E2 met)
fabrics = gpu 96 591 µJ attribution=machine, idle-corrected by 0.08%
cpu 7 536 µJ attribution=process, coverage=whole for CPU work
npu 0 µJ frozen (no ANE Energy aggregate on this part)
dram — not-implemented (rails exist; this reader omits them)
```
Two things the annex makes visible that a single number hides. First, the figure
is a **mean over 200 decisions**, because one decision takes 11.85 ms and a
10 ms-period counter cannot resolve it — under E2 a per-decision counter read
would have had to be reported as modelled. Second, 93% of the energy is on a
**machine-attributed** rail, so the figure is this workload's only under
exclusive use; the idle correction was 0.08%, which is what makes that credible
here and would not be on a shared host.
The same decision class run on the CPU instead costs **0.486–0.555 J** —
roughly five times more energy — and is entirely **principal-attributed** and
whole-coverage. Under E3 those are different claims, and a reporter choosing
the cheaper figure is also choosing the weaker one. Without E3 the two look
like the same kind of number, and the cheaper one wins every comparison.
## e. Conformance
All five requirements are machine-checkable, and we have an implementation
rather than only a proposal:
- A metering ladder that tags every figure with its source and refuses to
upgrade a model's output to a reading.
- A per-fabric survey that returns E1, E3, E4 and E5 for every
`(fabric, source)` pair on a host, probing presence so an unread accelerator
reports `not-implemented` rather than the false claim `absent`.
- The E2 refusal as a predicate over `(declared period, span)`, with the
counter characterisation above as its evidence.
- Measurement harnesses for both the counter characterisation and the
per-decision figures, so the numbers in §d are reproducible rather than
quoted.
We propose filing this as an annex under the extension URN
`urn:gsf:sci-ai:ext:energy-provenance:v1`, additive and imposing nothing on
deployments that report estimates — which remain perfectly conformant, provided
they say so.
## f. References
- **ISO/IEC 21031:2024** — Software Carbon Intensity (SCI) specification.
- **GSF SCI for AI** — ratified 2025; functional units for language models,
classifiers and training.
- **GSF, SCI for AI and EU AI Act environmental compliance** — the policy
framing this annex is intended to support.
- **EU AI Act GPAI Code of Practice**, Model Documentation Form — energy
consumption for training and inference; the compute/hardware-provider
exemption.
- Dauner et al., *Evaluating the Influence of Measurement Frequency on
[GPU energy counters]*, ACM SIGENERGY Energy Informatics Review 6(2),
July 2026 — the 588-run study; 95.4% under-read at 0.5 ms.
- *Characterization of Request and Token Energy Costs for LLM Inference
Workloads*, arXiv:2608.28044 — builds on the same counter for the opposite
reason.
- **CEN/CLC/TR 18145:2025**, *Green and Sustainable AI* — the European
guidance this annex is also offered to.
- **AFNOR SPEC 2314**, *Référentiel général pour l'IA frugale* — prescribes
estimator tooling for claims of frugality; E1/E2 are what would let such a
claim be checked.
- **Blauer Engel DE-UZ 215** — software energy criteria requiring measurement
on a defined test system; the German UBA's own reporting names energy per
classification as a KPI.
- **UNE 0086:2025** — energy, carbon, water and performance of AI systems.
---
*Working draft. The filing to `github.com/Green-Software-Foundation/sci` will
carry a JSON-Schema definition for the E-provenance block, the conformance
predicate for E2, and the reproducible harnesses behind §d.*

2
eoc/standards/sci-for-ai-submission.md
View file

@ -4,6 +4,8 @@
*Status: working draft v0.1. Authors: Transaction Science (steward of the EOC specification). Target SCI revision: SCI-for-AI v1.0 (ratified December 2025).*
*Companion: [`sci-for-ai-energy-provenance-annex.md`](sci-for-ai-energy-provenance-annex.md) proposes requirements on the **provenance and validity of E**. This document maps EOC receipt fields onto the formula and takes E as given; the annex is about whether a figure labelled "measured" earned the label.*
---
## a. Summary

1080
jcp/jcp-rs/crates/jcp-energy/src/lib.rs
View file

File diff suppressed because it is too large Load diff

52
jcp/tools/counter-resolution/README.md Normal file
View file

@ -0,0 +1,52 @@
# What a counter can actually resolve
Two harnesses behind the numbers `jcp-energy` declares. Both are the evidence
for a constant in the crate; neither is a test of the crate.
## `counter_resolution.py` — the per-process CPU counter's update period
`ri_energy_nj` is the only source in the survey that attributes energy to a
*principal* rather than to silicon, so whether it may grade a span decides
whether Apple silicon has any process-attributed reading at all.
Burns this thread's CPU for a fixed interval, samples the counter 60 times per
interval, and reports the fraction of intervals in which it advanced — the
crate's own liveness criterion is 80%.
Measured on an M5 Max, macOS 27:
| interval | advanced | median step | clears 80% |
|---:|---:|---:|---|
| 10 µs | 0% | — | no |
| 50 µs | 0% | — | no |
| 100 µs | 2% | 20.4 mJ | no |
| 500 µs | 5% | 21.1 mJ | no |
| 1 ms | 35% | 2.8 mJ | no |
| 2 ms | 63% | 4.9 mJ | no |
| **5 ms** | **87%** | 9.6 mJ | **yes** |
| 10 ms | 100% | 19.0 mJ | yes |
| 50 ms | 100% | 92.9 mJ | yes |
The crate declares **10 ms**, not the 5 ms where it first clears the criterion:
over-declaring a period refuses spans that would have been fine, while
under-declaring grades spans that were not, and at a ceiling only the second
error is dangerous.
Note the shape below 1 ms. The counter does not lag smoothly — on the few
intervals where it advanced at all, the median step was ~20 mJ, two orders
above the 1 ms step. It accrues in lumps, which is the same
batched-refresh pathology the crate already refuses on IOReport's CPU rails.
A sampler that read it every 500 µs would see nothing most of the time and a
20 mJ jump occasionally, and would report both as measurements.
This is also why a per-operation figure for an 11.85 ms decision is taken as
the mean of 200 decisions over a 2.6 s span rather than as 200 counter reads.
## `ioreport_rails.py` — which Apple rails exist and advance
Enumerates the IOReport `Energy Model` group unprivileged (364 rails on an
M5 Max) and prints each channel's name, unit and delta. What it establishes:
`GPU Energy` is live in **nJ**, every `CPU Energy` / `MCPU*` / `PACC*` /
`*_SRAM` rail is frozen for an unprivileged reader, and `ANE0` is the only ANE
channel — no `ANE Energy` aggregate, no sub-rails. Units differ *within one
sample*, so a summer that assumes one unit is wrong.

32
jcp/tools/counter-resolution/counter_resolution.py Normal file
View file

@ -0,0 +1,32 @@
"""At what interval does ri_energy_nj actually advance under continuous load?
The crate's own liveness criterion: a rail is live when it advanced in >=80% of
sampled intervals. Applied here to find the shortest interval that clears it,
which is the counter's declarable update period.
"""
import ctypes, os, time, statistics
_l = ctypes.CDLL("/usr/lib/libSystem.B.dylib"); _b = ctypes.create_string_buffer(464); _p = os.getpid()
def nj():
_l.proc_pid_rusage(ctypes.c_int(_p), ctypes.c_int(6), ctypes.byref(_b))
return int.from_bytes(_b.raw[336:344], "little")
def trial(interval_s, n=60):
"""Burn for `interval_s` per sample; report the fraction that advanced."""
adv = 0; deltas = []
prev = nj()
for _ in range(n):
t_end = time.perf_counter() + interval_s
x = 0
while time.perf_counter() < t_end:
x += 1 # burn this thread's CPU
cur = nj()
if cur > prev: adv += 1
deltas.append(cur - prev)
prev = cur
nz = [d for d in deltas if d > 0]
return adv / n, (statistics.median(nz)/1e3 if nz else 0.0)
print(f"{'interval':>10} {'advanced':>9} {'median ΔµJ':>11} live(>=80%)")
for us in (10, 50, 100, 250, 500, 1_000, 2_000, 5_000, 10_000, 20_000, 50_000):
frac, med = trial(us/1e6)
print(f"{us:>8}µs {frac*100:>8.0f}% {med:>11.1f} {'yes' if frac >= 0.80 else 'no'}")

48
jcp/tools/counter-resolution/gpu_rail_period.py Normal file
View file

@ -0,0 +1,48 @@
"""How long does the GPU Energy rail hold its value?
I declared 10 ms as its update period. PR #146 reports it "republishes in
steps and can hold 300-600 ms under load". Declaring too short a period is the
unsafe direction: it lets a span be graded that the counter never covered.
This measures the distribution of gaps between actual changes.
"""
import time, statistics, threading, sys
exec(open('rails.py').read().split('import time, subprocess, sys')[0])
def gpu_abs():
"""Absolute cumulative GPU nJ: delta against a fixed first sample."""
return None
def gaps(duration_s, label, load=None):
stop = threading.Event()
t = None
if load:
t = threading.Thread(target=load, args=(stop,), daemon=True); t.start()
time.sleep(0.3) # let the load spin up
prev_s = sample(); last_change = time.perf_counter(); g = []; steps = []
t_end = time.perf_counter() + duration_s
while time.perf_counter() < t_end:
time.sleep(0.002) # 2 ms sampling: finer than any claim
cur_s = sample()
d = IOR.IOReportCreateSamplesDelta(prev_s, cur_s, None)
tot = sum(v for _g, _sg, n, u, v in rails(d) if n == "GPU Energy")
if tot > 0:
now = time.perf_counter()
g.append((now - last_change) * 1e3); steps.append(tot / 1e6)
last_change = now
prev_s = cur_s # only advance the baseline on a change
if load:
stop.set(); t.join(timeout=2)
if not g:
print(f"{label:<16} no change in {duration_s}s"); return
g.sort()
print(f"{label:<16} n={len(g):<4} gap ms: p50={statistics.median(g):7.1f} "
f"p90={g[int(.9*len(g))-1]:7.1f} max={g[-1]:7.1f} median step={statistics.median(steps):8.2f} mJ")
def mlx_load(stop):
import mlx.core as mx
a = mx.random.normal((2048, 2048)); b = mx.random.normal((2048, 2048))
while not stop.is_set():
c = a @ b; mx.eval(c)
gaps(6.0, "idle")
gaps(6.0, "mlx matmul", mlx_load)

80
jcp/tools/counter-resolution/ioreport_rails.py Normal file
View file

@ -0,0 +1,80 @@
"""Enumerate IOReport Energy Model rails on Apple Silicon, unprivileged.
Decisive question: are the GPU and ANE energy rails readable without sudo on
macOS 27, and do they advance under a Metal workload?
"""
from ctypes import *
CF = CDLL("/System/Library/Frameworks/CoreFoundation.framework/CoreFoundation")
IOR = CDLL("/usr/lib/libIOReport.dylib")
CFStringRef = c_void_p; CFDictionaryRef = c_void_p; CFArrayRef = c_void_p
CF.CFStringCreateWithCString.restype = CFStringRef
CF.CFStringCreateWithCString.argtypes = [c_void_p, c_char_p, c_uint32]
CF.CFStringGetCStringPtr.restype = c_char_p
CF.CFStringGetCStringPtr.argtypes = [CFStringRef, c_uint32]
CF.CFStringGetCString.restype = c_bool
CF.CFStringGetCString.argtypes = [CFStringRef, c_char_p, c_long, c_uint32]
CF.CFDictionaryGetValue.restype = c_void_p
CF.CFDictionaryGetValue.argtypes = [CFDictionaryRef, c_void_p]
CF.CFArrayGetCount.restype = c_long
CF.CFArrayGetCount.argtypes = [CFArrayRef]
CF.CFArrayGetValueAtIndex.restype = c_void_p
CF.CFArrayGetValueAtIndex.argtypes = [CFArrayRef, c_long]
def cfstr(s): return CF.CFStringCreateWithCString(None, s.encode(), 0x08000100) # kCFStringEncodingUTF8
def pystr(ref):
if not ref: return None
p = CF.CFStringGetCStringPtr(ref, 0x08000100)
if p: return p.decode(errors="replace")
buf = create_string_buffer(512)
return buf.value.decode(errors="replace") if CF.CFStringGetCString(ref, buf, 512, 0x08000100) else None
IOR.IOReportCopyChannelsInGroup.restype = CFDictionaryRef
IOR.IOReportCopyChannelsInGroup.argtypes = [CFStringRef, CFStringRef, c_uint64, c_uint64, c_uint64]
IOR.IOReportCreateSubscription.restype = c_void_p
IOR.IOReportCreateSubscription.argtypes = [c_void_p, CFDictionaryRef, POINTER(CFDictionaryRef), c_uint64, c_void_p]
IOR.IOReportCreateSamples.restype = CFDictionaryRef
IOR.IOReportCreateSamples.argtypes = [c_void_p, CFDictionaryRef, c_void_p]
IOR.IOReportCreateSamplesDelta.restype = CFDictionaryRef
IOR.IOReportCreateSamplesDelta.argtypes = [CFDictionaryRef, CFDictionaryRef, c_void_p]
for fn, rt in (("IOReportChannelGetChannelName", CFStringRef), ("IOReportChannelGetGroup", CFStringRef),
("IOReportChannelGetSubGroup", CFStringRef), ("IOReportChannelGetUnitLabel", CFStringRef)):
getattr(IOR, fn).restype = rt; getattr(IOR, fn).argtypes = [CFDictionaryRef]
IOR.IOReportSimpleGetIntegerValue.restype = c_int64
IOR.IOReportSimpleGetIntegerValue.argtypes = [CFDictionaryRef, c_int32]
chans = IOR.IOReportCopyChannelsInGroup(cfstr("Energy Model"), None, 0, 0, 0)
if not chans: raise SystemExit("no Energy Model group")
subbed = CFDictionaryRef()
sub = IOR.IOReportCreateSubscription(None, chans, byref(subbed), 0, None)
if not sub: raise SystemExit("subscription refused (would mean privilege is required)")
def sample(): return IOR.IOReportCreateSamples(sub, subbed, None)
def rails(delta):
out = []
arr = CF.CFDictionaryGetValue(delta, cfstr("IOReportChannels"))
for i in range(CF.CFArrayGetCount(arr)):
ch = CF.CFArrayGetValueAtIndex(arr, i)
out.append((pystr(IOR.IOReportChannelGetGroup(ch)), pystr(IOR.IOReportChannelGetSubGroup(ch)),
pystr(IOR.IOReportChannelGetChannelName(ch)), pystr(IOR.IOReportChannelGetUnitLabel(ch)),
IOR.IOReportSimpleGetIntegerValue(ch, 0)))
return out
import time, subprocess, sys
s0 = sample()
# Load the GPU (and ANE if asked) while the rails are watched.
load = subprocess.Popen([sys.argv[1], sys.argv[2]]) if len(sys.argv) > 2 else None
time.sleep(2.0)
s1 = sample()
if load: load.wait()
d = IOR.IOReportCreateSamplesDelta(s0, s1, None)
rs = rails(d)
print(f"subscription: OK (unprivileged, uid={__import__('os').getuid()}) rails in Energy Model: {len(rs)}")
agg = {}
for g, sg, name, unit, val in rs:
key = (sg or "-", name or "-", unit or "-")
a = agg.setdefault(key, [0, 0]); a[0] += val; a[1] += 1
print(f"{'subgroup':<22}{'channel':<34}{'unit':<8}{'delta over 2s':>16}{'n':>5}")
for (sg, name, unit), (val, n) in sorted(agg.items(), key=lambda kv: -abs(kv[1][0]))[:28]:
print(f"{sg:<22}{name:<34}{unit:<8}{val:>16}{n:>5}")

66
mesh/SPEC.md
View file

@ -80,6 +80,11 @@ A conforming request MUST carry both a `fidelity_floor` and a `joule_ceiling_uj`
The pair is the contract: *give me an answer at least this good, for at most this
much energy.*
A request MAY additionally carry a `decision` (§3.6), which says it is asking for
a bounded typed decision rather than generated text. Its absence is a generation,
and is omitted from the canonical form — so the `request_id` of every request
written before this field existed is unchanged.
### §3.2 Fidelity
Fidelity is an ordered scale `F0 … F4` — `F0` the coarsest sufficient answer
@ -125,13 +130,65 @@ IOReport), but a counter counts only while it is live:
the whole request, such as a GPU counter on a host whose CPU has none, MUST NOT
be graded `HwShunt`.
### §3.6 Typed decisions — the other unit of work
A **decision model** takes state plus typed questions and returns a calibrated
probability per allowed answer, generating no text. The category converged
through 2026 on three primitives — a proposition (`noul`), a choice over 1–255
named options, and a score over an ordered rubric — and backends score the
schema **non-autoregressively** rather than emitting tokens.
Everything else in this spec measures work in tokens, and a decision's output
token count is zero. Measured on a 421M decision model: `output_tokens: 0`, with
132 input tokens of which **121 were the question schema and 11 the state under
judgement**. Three consequences follow, and a conforming implementation MUST
handle each.
A request asking for a decision carries:
| Field | Type | Meaning |
|---|---|---|
| `questions` | u32 | typed questions in the request |
| `options_scored` | u32 | options scored across all of them |
| `schema_tokens` | u32 | how many of `prompt_tokens` are schema, not state |
1. **It MUST NOT be priced for tokens it cannot emit.** The estimate for a
decision MUST exclude `max_tokens`. Including it refuses work at a ceiling
that would in fact have covered it.
2. **Cost MUST be allowed to follow the schema.** A profile MAY declare a cost
per scored option; one question over 20 options is more work than one over 2
at identical token counts, and no token count expresses the difference.
Declaring nothing reproduces the token-only model exactly, so an unmeasured
profile claims nothing.
3. **The efficiency figure MUST be defined.** `uj_per_accepted_token` (§6) is
undefined when no tokens were accepted, which is every request in this
category. A decision receipt MUST instead carry `uj_per_decision`, and
SHOULD carry `uj_per_option_scored`. The unit of work is the answered
question.
`schema_tokens` is reported because a fixed schema is a **cacheable input
prefix** — distinct from `cache_hit_tokens`, which counts output tokens served
from cache, of which a decision has none. In the measured request 92% of the
input was schema, so a reader that cannot see the split cannot tell a cold
decision from a warm one.
> **Why a decision receipt is worth more than a generation receipt.** A decision
> is a single forward pass with no sampling. Given the weight digest, revision
> and numeric precision, it is reproducible — so a receipt over one is verifiable
> **by re-execution**, which a receipt over sampled text can never be. A receipt
> for a decision SHOULD therefore bind the weight digest and the precision; the
> probabilities of a published decision model differ between `fp16` and `fp32`
> even where the selected answer agrees, so a receipt that omits precision does
> not reproduce.
---
## §4 Routing — cheapest-sufficient
For each registered backend the router computes an estimate
`est_total_uj = uj_per_token × (prompt_tokens + max_tokens)` and marks it
**admissible** iff:
`est_total_uj = uj_per_token × (prompt_tokens + max_tokens)`, or for a typed
decision (§3.6) `uj_per_token × prompt_tokens + uj_per_option_scored ×
options_scored`, and marks it **admissible** iff:
```
backend.fidelity ≥ request.fidelity_floor AND est_total_uj ≤ request.joule_ceiling_uj
@ -207,7 +264,10 @@ and nulls; booleans are encoded as `0|1` uints and absent options dropped.
| `fidelity_met` | `0|1` — did the served backend meet the floor |
| `estimated_uj` | always present |
| `measured_uj` | present **iff** provenance is `HwShunt` |
| `uj_per_accepted_token` | **the wedge metric** — `energy.uj() / tokens_accepted` |
| `uj_per_accepted_token` | **the wedge metric** for a generation — `energy.uj() / tokens_accepted`; omitted when no tokens were accepted |
| `questions_answered`, `options_scored`, `schema_tokens` | present **iff** the request was a typed decision (§3.6) |
| `uj_per_decision` | **the wedge metric** for a decision — `energy.uj() / questions_answered` |
| `uj_per_option_scored` | the finer decision figure — `energy.uj() / options_scored` |
| `energy_provenance` | `HwShunt | ModelBased | Estimator` |
| `sequence`, `prev` | hash-chain position and prior envelope's content id |
| `nonce`, `steward` | per-receipt nonce and the sealing identity (COSE `kid`) |

1
mesh/mesh-rs/crates/mesh-conformance/src/lib.rs
View file

@ -69,6 +69,7 @@ pub fn reference_scenario() -> ReferenceScenario {
max_tokens: MAX_TOKENS,
fidelity_floor: Fidelity::F1,
joule_ceiling_uj: 8_000_000,
decision: None,
};
// Empty prompt payload: the reference engine is synthetic (input-agnostic),

1
mesh/mesh-rs/crates/mesh-core/examples/demo.rs
View file

@ -65,6 +65,7 @@ fn main() {
max_tokens: 64,
fidelity_floor: Fidelity::F1,
joule_ceiling_uj: 8_000_000, // 8 J
decision: None, // a generation, not a typed decision
};
// --- the caller mints a JCP grant: capability(infer over this request) ⊗ budget ---

1
mesh/mesh-rs/crates/mesh-core/src/lib.rs
View file

@ -236,6 +236,7 @@ mod tests {
max_tokens,
fidelity_floor: floor,
joule_ceiling_uj: ceiling,
decision: None,
}
}

2
mesh/mesh-rs/crates/mesh-engine/src/lib.rs
View file

@ -173,6 +173,7 @@ impl ServingEngine for ReferenceEngine {
cache_hit_tokens: if cache_hit { n } else { 0 },
spec_proposed,
spec_accepted,
decision: None,
};
Served { tokens, outcome }
}
@ -192,6 +193,7 @@ mod tests {
max_tokens,
fidelity_floor: Fidelity::F0,
joule_ceiling_uj: u64::MAX,
decision: None,
}
}

6
mesh/mesh-rs/crates/mesh-openai/src/lib.rs
View file

@ -108,6 +108,9 @@ pub fn to_serve_request(
max_tokens,
fidelity_floor,
joule_ceiling_uj,
// The OpenAI-shaped surface describes a generation, never a typed
// decision: its request has no questions to score.
decision: None,
};
Ok((serve, prompt))
}
@ -353,6 +356,9 @@ pub fn to_serve_request_responses(
max_tokens,
fidelity_floor,
joule_ceiling_uj,
// The OpenAI-shaped surface describes a generation, never a typed
// decision: its request has no questions to score.
decision: None,
};
Ok((serve, prompt))
}

103
mesh/mesh-rs/crates/mesh-receipt/src/lib.rs
View file

@ -82,6 +82,26 @@ impl MeshReceipt {
self.energy.uj().checked_div(self.outcome.tokens_accepted as u64)
}
/// The wedge metric for a backend that emits no tokens: microjoules per
/// answered decision (SPEC §3.6). `None` when the request was not a
/// decision, or answered no questions.
///
/// A typed decision's `tokens_accepted` is zero, so
/// [`Self::uj_per_accepted_token`] is undefined for every request in that
/// category — the receipt would seal with no efficiency figure at all. The
/// unit of work here is the answered question, not the accepted token.
pub fn uj_per_decision(&self) -> Option<u64> {
let d = self.outcome.decision?;
self.energy.uj().checked_div(d.questions_answered as u64)
}
/// Microjoules per option scored — the finer figure, since one question
/// over 20 options is more work than one over 2.
pub fn uj_per_option_scored(&self) -> Option<u64> {
let d = self.outcome.decision?;
self.energy.uj().checked_div(d.options_scored as u64)
}
/// Project to the JCR-1 `mesh/1` wire JSON. Booleans are encoded as `0|1`
/// uints and absent options dropped, so the signed core stays within the
/// det-CBOR type set (no bools, no nulls). `steward` is the endpoint
@ -111,6 +131,19 @@ impl MeshReceipt {
if let Some(jpt) = self.uj_per_accepted_token() {
obj.insert("uj_per_accepted_token".into(), json!(jpt));
}
// A typed decision (§3.6). Emitted only when the request was one, so a
// generation's signed core is byte-identical to before.
if let Some(d) = self.outcome.decision {
obj.insert("questions_answered".into(), json!(d.questions_answered));
obj.insert("options_scored".into(), json!(d.options_scored));
obj.insert("schema_tokens".into(), json!(d.schema_tokens));
if let Some(jpd) = self.uj_per_decision() {
obj.insert("uj_per_decision".into(), json!(jpd));
}
if let Some(jpo) = self.uj_per_option_scored() {
obj.insert("uj_per_option_scored".into(), json!(jpo));
}
}
obj.insert(
"energy_provenance".into(),
json!(self.energy.provenance.jcr1_label()),
@ -223,7 +256,7 @@ fn hex_lower(bytes: &[u8]) -> String {
#[cfg(test)]
mod tests {
use super::*;
use mesh_schema::Provenance;
use mesh_schema::{DecisionOutcome, Provenance};
fn outcome(accepted: u32) -> ServeOutcome {
ServeOutcome {
@ -251,6 +284,74 @@ mod tests {
}
}
/// A typed decision accepts no output tokens, so the token metric is
/// undefined and the decision metric is the one that exists.
///
/// The figure is the **CPU-only, process-attributed** one: 486 000 µJ for a
/// three-question decision on an M5 Max, 162 mJ per question, from the
/// kernel's per-process counter over a span far longer than its measured
/// 10 ms period.
///
/// Deliberately not the GPU figure. A GPU reading here comes from an SoC
/// rail that counts the whole machine, and on this host another process
/// was measured holding **99.5% of GPU time** in a 4 s window — so a rail
/// delta over a decision's span is an upper bound on the machine, not that
/// decision's energy. The per-process CPU counter is unaffected by that
/// contention, which is why the sound example uses it.
#[test]
fn a_decision_has_an_efficiency_figure_where_the_token_metric_has_none() {
let mut o = outcome(0);
o.decision = Some(DecisionOutcome {
questions_answered: 3,
options_scored: 8, // choice:3 + score:3 + noul:2
schema_tokens: 121,
});
let mut r = receipt(Energy::measured(486_000), 0);
r.outcome = o;
// The defect: the documented "wedge metric" is undefined here.
assert_eq!(r.uj_per_accepted_token(), None);
// The fix: the unit of work is the answered question.
assert_eq!(r.uj_per_decision(), Some(162_000));
assert_eq!(r.uj_per_option_scored(), Some(60_750));
// It survives the wire, and the wire says what it measured.
let wire = r.to_jcr1_wire("did:example:mesh-endpoint");
assert_eq!(wire["uj_per_decision"], json!(162_000));
assert_eq!(wire["questions_answered"], json!(3));
assert_eq!(wire["schema_tokens"], json!(121));
assert!(
wire.get("uj_per_accepted_token").is_none(),
"an undefined metric must be absent, not zero"
);
// And it still seals.
let sk = SigningKey::from_bytes(&[9u8; 32]);
let steward = "did:example:mesh-endpoint";
let env = r.seal(&sk, steward).unwrap();
assert!(r
.verify_envelope(&env, steward, &sk.verifying_key().to_bytes())
.is_ok());
}
/// The addition must be invisible to a generation: no new keys, so every
/// receipt sealed before it verifies byte-identically.
#[test]
fn a_generation_receipt_gains_no_wire_keys() {
let r = receipt(Energy::measured(3_200_000), 64);
let wire = r.to_jcr1_wire("did:example:mesh-endpoint");
for k in [
"questions_answered",
"options_scored",
"schema_tokens",
"uj_per_decision",
"uj_per_option_scored",
] {
assert!(wire.get(k).is_none(), "{k} must not appear for a generation");
}
assert_eq!(r.uj_per_decision(), None);
}
#[test]
fn measured_receipt_seal_verify_and_metric() {
let sk = SigningKey::from_bytes(&[9u8; 32]);

129
mesh/mesh-rs/crates/mesh-router/src/lib.rs
View file

@ -53,6 +53,14 @@ pub struct CostProfile {
/// says this backend gets more expensive per token as the prefix grows,
/// which is what makes a long-context request refusable *before* it runs.
pub uj_per_token_per_kctx: u64,
/// Microjoules per option scored in a typed decision (§3.6), on top of the
/// per-token cost of reading the request.
///
/// Zero is the historical shape and the default: a backend that only
/// generates has no options to score, and every existing constructor and
/// sealed `wire()` string is unchanged. For a decision backend, zero means
/// the estimate ignores the only work the pass performs.
pub uj_per_option_scored: u64,
/// The operating point these constants were measured at, if the operator
/// declared one. See [`Calibration`].
///
@ -169,12 +177,53 @@ impl CostProfile {
/// Retained so existing callers are unchanged and the scalar case stays
/// honestly expressible — some backends really are flat.
pub fn linear(uj_per_token: u64) -> Self {
CostProfile { base_uj: 0, uj_per_token, uj_per_token_per_kctx: 0, calibration: None }
CostProfile {
base_uj: 0,
uj_per_token,
uj_per_token_per_kctx: 0,
uj_per_option_scored: 0,
calibration: None,
}
}
/// A profile whose per-token cost grows with the context length.
pub fn context_scaled(base_uj: u64, uj_per_token: u64, uj_per_token_per_kctx: u64) -> Self {
CostProfile { base_uj, uj_per_token, uj_per_token_per_kctx, calibration: None }
CostProfile {
base_uj,
uj_per_token,
uj_per_token_per_kctx,
uj_per_option_scored: 0,
calibration: None,
}
}
/// Declare what one scored option costs this backend, so a typed decision
/// can be priced on the work it does (§3.6).
///
/// Zero — the default — reproduces the token-only model exactly, which for
/// a decision means pricing a pass that generates nothing by the tokens it
/// does not generate.
pub fn with_option_cost(mut self, uj_per_option_scored: u64) -> Self {
self.uj_per_option_scored = uj_per_option_scored;
self
}
/// Estimated microjoules for a typed decision: the input it reads, plus the
/// schema it scores.
///
/// `max_tokens` has no part in it. A decision model emits no tokens — a
/// measured three-question request reported `output_tokens: 0` — so the
/// per-token term covers only what it reads, and the work of scoring
/// `options_scored` options is the term that would otherwise be invisible.
pub fn estimate_decision_uj(
&self,
prompt_tokens: u64,
context_tokens: u64,
options_scored: u64,
) -> u64 {
let read = self.estimate_uj(prompt_tokens, context_tokens);
let scored = self.uj_per_option_scored.saturating_mul(options_scored);
read.saturating_add(scored)
}
/// Whether this profile can represent a context-dependent cost at all.
@ -315,6 +364,16 @@ impl BackendSpec {
/// backend — deliberately, since the alternative is to model the growth and
/// then defend a second uncalibrated constant.
fn estimate(&self, req: &ServeRequest) -> u64 {
if let Some(d) = req.decision {
// A typed decision generates nothing, so `max_tokens` is not part
// of its cost; what it costs is reading the request and scoring
// the schema.
return self.cost.estimate_decision_uj(
req.prompt_tokens as u64,
req.prompt_tokens as u64,
d.options_scored as u64,
);
}
let tokens = req.prompt_tokens as u64 + req.max_tokens as u64;
self.cost.estimate_uj(tokens, req.prompt_tokens as u64)
}
@ -430,6 +489,7 @@ impl Router {
#[cfg(test)]
mod tests {
use super::*;
use mesh_schema::DecisionShape;
use mesh_schema::ContentId;
fn router() -> Router {
@ -448,9 +508,73 @@ mod tests {
max_tokens,
fidelity_floor: floor,
joule_ceiling_uj: ceiling,
decision: None,
}
}
fn req_decision(prompt_tokens: u32, max_tokens: u32, options_scored: u32) -> ServeRequest {
ServeRequest {
prompt_hash: ContentId::of_bytes(b"state"),
prompt_tokens,
max_tokens,
fidelity_floor: Fidelity::F0,
joule_ceiling_uj: u64::MAX,
decision: Some(DecisionShape { questions: 3, options_scored, schema_tokens: 121 }),
}
}
/// A typed decision must not be charged for tokens it cannot emit.
///
/// This is the defect the decision shape exists to close: a decision model
/// generates nothing (`output_tokens: 0` measured), so a caller that leaves
/// `max_tokens` at a generation's default was being priced for an essay the
/// backend will never write — and refused at a ceiling that would in fact
/// have covered the work.
#[test]
fn a_decision_is_not_priced_for_tokens_it_never_emits() {
let spec = BackendSpec::with_cost(
"decider",
"laya-421m",
Fidelity::F1,
CostProfile::linear(10).with_option_cost(1_000),
);
// 132 read tokens x 10 uJ + 8 options x 1000 uJ = 1320 + 8000.
let d = spec.estimate(&req_decision(132, 0, 8));
assert_eq!(d, 9_320);
// The same request with a generation's max_tokens costs the same: the
// ceiling is irrelevant to a pass that emits nothing.
assert_eq!(spec.estimate(&req_decision(132, 4_096, 8)), d);
// A generation with that max_tokens is priced for all of it - proving
// the branch is what changed, not the profile.
assert_eq!(
spec.estimate(&req_ctx(Fidelity::F1, u64::MAX, 132, 4_096)),
(132 + 4_096) * 10
);
}
/// Cost must follow the schema being scored, which is the dimension no
/// token count carries: one question over 20 options is more work than one
/// over 2, at identical token counts.
#[test]
fn a_decision_scales_with_the_options_it_scores() {
let spec = BackendSpec::with_cost(
"decider",
"laya-421m",
Fidelity::F1,
CostProfile::linear(10).with_option_cost(1_000),
);
let few = spec.estimate(&req_decision(132, 0, 2));
let many = spec.estimate(&req_decision(132, 0, 20));
assert!(many > few, "scoring 20 options must cost more than scoring 2");
assert_eq!(many - few, 18 * 1_000);
// Declaring no option cost reproduces the token-only model, so the
// change is opt-in and an unmeasured profile claims nothing.
let flat =
BackendSpec::with_cost("decider", "laya-421m", Fidelity::F1, CostProfile::linear(10));
assert_eq!(flat.estimate(&req_decision(132, 0, 2)), 1_320);
assert_eq!(flat.estimate(&req_decision(132, 0, 20)), 1_320);
}
/// The historical scalar model must be reproduced exactly, so existing
/// backends and their receipts do not shift under this change.
#[test]
@ -558,6 +682,7 @@ mod tests {
max_tokens,
fidelity_floor: floor,
joule_ceiling_uj: ceiling,
decision: None,
}
}

61
mesh/mesh-rs/crates/mesh-schema/src/lib.rs
View file

@ -199,6 +199,39 @@ pub struct ServeRequest {
pub fidelity_floor: Fidelity,
/// Hard energy budget for this request, in microjoules.
pub joule_ceiling_uj: u64,
/// The shape of a **typed decision**, when that is what is being asked for
/// rather than generated text (§3.6).
///
/// `None` is the historical shape — a generation, priced per token — and is
/// skipped on the wire, so an existing request's [`ServeRequest::id`] and
/// every sealed conformance vector are byte-identical to before.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub decision: Option<DecisionShape>,
}
/// What a backend is being asked to decide, in the terms that determine its
/// energy (§3.6).
///
/// A decision model takes state plus typed questions and returns a calibrated
/// probability per allowed answer, generating no text. Its work is therefore
/// not a token count: a non-autoregressive pass scores the *schema*, so cost
/// follows the questions asked and the options under each, and `max_tokens` is
/// zero. Pricing such a request per token prices the one thing it does not do.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct DecisionShape {
/// Typed questions in this request.
pub questions: u32,
/// Options scored across all of them — two 3-option `choice` questions and
/// one `noul` score 8. This is the dimension a token count cannot express.
pub options_scored: u32,
/// How many of `prompt_tokens` are the question schema rather than the
/// state under judgement.
///
/// A fixed schema is a cacheable prefix, and it dominates: in a measured
/// three-question request to a 421M decision model, 121 of 132 input
/// tokens were schema and 11 were state. A router that cannot see the
/// split re-prices 92% of the request on every call as though it were new.
pub schema_tokens: u32,
}
impl ServeRequest {
@ -252,4 +285,32 @@ pub struct ServeOutcome {
pub spec_proposed: u32,
/// Speculative tokens the target model accepted.
pub spec_accepted: u32,
/// What was decided, when the backend served a typed decision rather than
/// generating text (§3.6).
///
/// `None` is the historical shape, whose unit of work is the accepted
/// output token, and is skipped on the wire so existing receipts seal to
/// the same bytes.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub decision: Option<DecisionOutcome>,
}
/// Per-request decision counters — the inputs to the µJ/decision metric
/// (§3.6).
///
/// Every other field of [`ServeOutcome`] is a token count, and a decision's
/// `tokens_accepted` is zero, which leaves `uj_per_accepted_token` undefined.
/// The unit of work here is the answered question.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct DecisionOutcome {
/// Questions answered. The denominator of µJ/decision.
pub questions_answered: u32,
/// Options actually scored across them.
pub options_scored: u32,
/// Input tokens that were the question schema rather than the state.
///
/// Distinct from [`ServeOutcome::cache_hit_tokens`], which counts *output*
/// tokens served from cache. A decision has no output tokens; what it can
/// reuse is this input prefix.
pub schema_tokens: u32,
}

231
mesh/tools/joules-per-decision/README.md Normal file
View file

@ -0,0 +1,231 @@
# Joules per typed decision
A measurement harness, and what it found: **Mesh cannot price, rate or report a
typed decision**, because every quantity in the schema is a token count and a
decision emits no tokens.
Run it with `uv`:
```sh
uv venv --python 3.12 .venv
uv pip install --python .venv/bin/python laya-mlx
.venv/bin/python joules_per_decision.py
```
## What a decision model is
A decision model takes application *state* plus *typed questions* and returns
calibrated probabilities over a bounded answer set. It generates no text. The
category appeared between 2026-09-15 (TypeSafe's Jev) and 2026-10-01
(Cloudflare's Clef), converging on three primitives:
| primitive | returns |
|---|---|
| `noul` | P(true) for a proposition |
| `choice` | a probability per named option, 1–255 options |
| `score` | a probability per level of an ordered rubric |
The schema is published as `System One` (OpenAPI 3.1, `api.typesafe.ai`), and
Jev, Liquid d1, Kev, Clef and Laya all implement it. Clef states that it scores
schema choices **non-autoregressively** rather than generating tokens.
## Method
The harness measures `laya-mlx` — an independent Apache-2.0 MLX port of Convai's
Laya (ModernBERT-large, 421M, FP16) — on Apple Silicon.
Energy comes from `ri_energy_nj` in `proc_pid_rusage(getpid(), RUSAGE_INFO_V6)`,
the same counter `jcp-energy`'s rung 3 uses, read in-process so the reading is
`Coverage::Whole` for this process's CPU work. The ABI constants (`SIZE = 464`,
`ENERGY_NJ = 336`) are lifted from `jcp-energy`, which verified them with
`sizeof`/`offsetof` against the macOS 26, 26.5 and 27 SDKs; they are not
re-derived here. The harness applies `jcp-energy`'s own liveness rule before
trusting the counter — at least five advancing sample intervals and at least
80% of them, over at least 300 ms — and reports the result rather than assuming
it.
Because the counter is per-process, the **energy** figures are unaffected by
other load on the host. The **latency** figures are not, so they are upper
bounds.
## Results — M5 Max, macOS 27, 2026-10-04
| device | questions | p50 | p95 | J/decision | coverage |
|---|---|---:|---:|---:|---|
| CPU | 1 (`noul`) | 126.8 ms | 636.2 ms | **0.204** | Whole (CPU), HwShunt |
| CPU | 3 (`choice`+`score`+`noul`) | 629.1 ms | 1410.0 ms | **0.486** | Whole (CPU), HwShunt |
| GPU | 1 | 21.1 ms | 38.2 ms | 0.0079 | **floor only** |
| GPU | 3 | 17.1 ms | 22.8 ms | 0.0082 | **floor only** |
Counter liveness: 25 advancing intervals of 25.
Three questions cost *less* wall time than one, because all three ride a single
forward pass (`batch_size` defaults to 16) — the marginal cost of a question
inside a batch is near zero, which no per-token model predicts.
The GPU rows are a **floor, not the decision's energy**. They are the host-side
CPU energy only; the GPU's share is unmetered. `jcp-energy` has no
whole-coverage rung for GPU-resident work on Apple Silicon and correctly refuses
to pass off IOReport's machine-wide GPU rail as one task's energy, so no
whole-task joule figure for the GPU path can be stated here. Every local
inference on a Mac is GPU-resident, so this is the gap that matters most.
## What the measurements show about this spec
Four findings, each checked against the code rather than inferred.
1. **`uj_per_accepted_token` is undefined for the whole category.**
`MeshReceipt::uj_per_accepted_token` is documented as "the wedge metric" and
is `checked_div` by `outcome.tokens_accepted`. A decision's
`tokens_accepted` is 0 — measured `"output_tokens": 0` on every request — so
the metric returns `None`. The receipt seals, and the number that makes it
worth sealing does not exist.
2. **The unit of work is implicitly the accepted output token.** Every field of
`ServeOutcome` is a token count: `tokens_in`, `tokens_out`,
`tokens_accepted`, `cache_hit_tokens`, `spec_proposed`, `spec_accepted`. A
decision's unit is the answered question, and nothing in the outcome can
carry it.
3. **The cost model prices the wrong thing.** `CostProfile::estimate_uj` is
charged `tokens = prompt_tokens + max_tokens`. With `max_tokens = 0` a
decision collapses to `base_uj` plus its prompt, which ignores what the pass
actually does: score N questions × K options. And the split inside the
prompt is invisible — in the measured 3-question request, **121 of 132 input
tokens were the question schema and 11 were the state**. A fixed schema is a
cacheable prefix (`laya-mlx` exposes `cache_prompts`), so 92% of a real
decision request is re-priced on every call as though it were new input.
4. **`fidelity_met` is a self-declaration sealed into a signed receipt.**
`mesh-core` computes `fidelity_met = chosen_fidelity >= req.fidelity_floor`,
where `chosen_fidelity` is the level the backend *advertises* (SPEC §3.2:
"ordered scale F0 … F4"). `Energy` in the same receipt carries a provenance
tag — `HwShunt | ModelBased | Estimator` — with a normative coupling rule.
Fidelity carries nothing. The receipt is honest about where its joules came
from and silent about where its quality claim came from.
Decision models are what makes this fixable rather than inherent: for a
bounded question, calibration is measurable. Kev publishes Brier scores
(27B: 0.225) alongside accuracy; Clef trains against a Brier loss. A
fidelity rating can cite an evaluation instead of asserting a level.
## Why a decision receipt is worth more than a generation receipt
A decision is a single forward pass with no sampling. Given the weight hash,
revision, dtype and the fitted temperatures, it is reproducible — so a receipt
over one is verifiable **by re-execution**, which a receipt over sampled text
can never be. `laya-mlx` publishes pinned revisions and per-file checksums, and
reports 63/63 validation questions matching upstream's selected answer in both
FP32 and FP16, which is exactly the material a receipt would bind.
The precision matters and must be bound: probabilities differ across FP16 and
FP32 even where the selected label agrees. A receipt that omits dtype does not
reproduce.
## The defect this category ships with today
Loading the published Laya checkpoint raises:
```
laya-mlx: this checkpoint ships temperatures outside [0.5, 5] which would
distort confidence; clamping choice:11+=0.1006. Treat confidence from the
affected buckets as uncalibrated.
```
Confirmed on the real weights: `temperature_by_options_raw["choice:11+"]` is
`0.10058280825614929`. A temperature of 0.1 sharpens logits roughly tenfold, so
the shipped checkpoint reports near-certainty on what may be close to a coin
flip, for every choice question with 11 or more options. The porter found it by
reproducing upstream numerically and clamped it; the publisher ships it.
That is the argument for this work in one line: **a calibrated probability is a
claim, and this category ships claims with no receipt.**
## The attribution trade — two fabrics, one decision
`two_fabrics.py` measures a decision model on the CPU, where the kernel's
per-process counter attributes energy to a **principal**, and compares it with
the same class of work on the GPU, where the only counter is an SoC rail that
counts the silicon and cannot tell one process from another.
Measured on an M5 Max, macOS 27, 2026-10-04. Both CPU rows are
`Attribution::Process`, `Coverage::Whole` for CPU work, over spans far longer
than the counter's declared 10 ms period:
| model | params | runtime | fabric | J/decision | p50 | attribution |
|---|---:|---|---|---:|---:|---|
| Laya | 421M | MLX | CPU | **0.486** | 629 ms | Process, whole |
| GLiNER2.5-Decide | 340M | torch | CPU | **0.555** | 480 ms | Process, whole |
| Laya | 421M | MLX | GPU | **0.104** | 11.9 ms | **Machine** (GPU share) |
With the fabric held constant the two models are close: GLiNER2.5-Decide spends
14% more energy per decision and returns it 24% faster — while scoring 60.1%
against Laya's 46.6% on Fastino's 17-dataset suite, so on this evidence the
better-scoring model is not the more expensive one in any meaningful sense.
Moving the *same* model to the GPU costs **4.7× fewer joules and is 53×
faster** — and the figure stops being attributable. IOReport's `GPU Energy`
rail is a real reading of the whole GPU; it cannot say whose work it was. So
the cheap path produces a number that cannot be bound to a principal, and the
expensive path produces one that can.
That is the trade, and it is not a defect in either counter. It is why
`Attribution` is a separate axis from `Coverage`: a receipt that wants to say
*this principal spent these joules* has to choose the CPU path today, and a
receipt that only needs *this much energy was spent here* can take the GPU.
The CPU figure is complete, not merely attributable. Over the GLiNER run the
GPU rail, bracketed by idle baselines, moved by **−0.0013 J per decision** —
negative, meaning the idle estimate slightly exceeded the real draw, so GPU
involvement is below the measurement's own noise. The process-attributed CPU
number is therefore the whole task, which is exactly what the GPU rows cannot
claim.
## ⚠ Withdrawn: the GPU figure is not this decision's energy
`gpu_share.py` measures what the table above assumed away. `accumulatedGPUTime`
per IOUserClient (I/O Registry, IOAccelerator subtree) gives each process's GPU
time; the cumulative totals are useless because WindowServer has been running
for days, so what matters is the **delta over the measured span**.
Measured on this host, over a 4.06 s span while an MLX matmul loop ran:
| process | GPU time in span | share |
|---|---:|---:|
| com.apple.Safari | 2.470 s | **99.5%** |
| WindowServer | 0.007 s | 0.3% |
| PhotosReliveWidget | 0.005 s | 0.2% |
| **this process** | **0.000 s** | **0.0%** |
So a single unrelated application held essentially the whole GPU for the
duration. A rail delta taken over that span is a reading of Safari, not of the
decision. The host was at load average 132 on 18 CPUs at the time, running
another session's VVC/VTM encodes — which is the normal state of this machine,
not an unlucky moment.
**Consequences for the table above:**
- The two **CPU** rows stand unchanged. The kernel's per-process counter
attributes to a principal, so other processes' load does not enter them.
That is the whole value of `Attribution::Process`, demonstrated the hard way.
- The **GPU** row is withdrawn as a per-decision energy figure. 0.104 J is an
upper bound on *machine* GPU energy over the span, and on this host the bound
is dominated by other processes. Its **latency** (11.9 ms p50) is unaffected.
- The idle-bracket correction of 0.08% that I read as evidence of cleanliness
proves nothing. A bracket subtracts *steady* background; it cannot subtract a
contending process whose work overlaps the span and not the brackets.
The rule to apply, already shipped elsewhere in the fleet: a rail reading may
be attributed to this process only when this process holds ≥99% of the GPU time
advancing in the span; where it holds none, its GPU energy is exactly zero; and
otherwise **no GPU figure is issued** and the span is CPU-only and partial. By
that rule this measurement issues no GPU figure at all.
This is also why time-share apportionment cannot rescue it: apportioning a rail
by GPU-time share was falsified elsewhere at 3–15× error, because a contending
job can draw an order of magnitude more watts per GPU-second than a decode, and
GPU time itself inflates ~5.4× under contention.
**To get a sound GPU figure**, re-measure on a host with no display server and
no other GPU client — which on this fleet means a headless Linux box with NVML,
not a Mac.

49
mesh/tools/joules-per-decision/gpu_share.py Normal file
View file

@ -0,0 +1,49 @@
"""My process's share of GPU time over a span — the #146 admissibility rule.
accumulatedGPUTime is cumulative since client creation, so WindowServer always
dominates the totals. The question that decides whether a rail reading may be
attributed to this process is the DELTA over the measured span.
"""
import re, subprocess, time, os, sys, threading
def gpu_times():
out = subprocess.run(["ioreg","-r","-c","IOAccelerator","-l","-w0"],
capture_output=True, text=True, timeout=30).stdout
tot = {}
for b in out.split("IOUserClientCreator")[1:]:
m = re.search(r'pid (\d+), ([^"]+)', b)
if not m: continue
key = (int(m.group(1)), m.group(2).strip())
g = sum(int(x) for x in re.findall(r'"accumulatedGPUTime"\s*=\s*(\d+)', b[:4000]))
if g: tot[key] = tot.get(key, 0) + g
return tot
def load(stop):
import mlx.core as mx
a = mx.random.normal((2048,2048)); b = mx.random.normal((2048,2048))
while not stop.is_set():
mx.eval(a @ b)
me = os.getpid()
stop = threading.Event()
t = threading.Thread(target=load, args=(stop,), daemon=True); t.start()
time.sleep(0.5)
before = gpu_times(); t0 = time.perf_counter()
time.sleep(4.0)
after = gpu_times(); span = time.perf_counter() - t0
stop.set(); t.join(timeout=3)
delta = {}
for k, v in after.items():
d = v - before.get(k, 0)
if d > 0: delta[k] = d
s = sum(delta.values())
mine = sum(v for (pid, _n), v in delta.items() if pid == me)
print(f"span {span:.2f}s clients advancing: {len(delta)} total GPU time in span: {s/1e9:.3f}s")
print(f"this process (pid {me}): {mine/1e9:.3f}s = {100*mine/s if s else 0:.1f}% of GPU time in the span")
for (pid, n), v in sorted(delta.items(), key=lambda kv: -kv[1])[:6]:
tag = " <-- me" if pid == me else ""
print(f" pid {pid:>7} {n[:34]:<34} {v/1e9:>8.3f}s {100*v/s:>5.1f}%{tag}")
print()
print("#146 rule:", "HwShunt (>=99%)" if s and mine/s >= 0.99 else
("exactly 0 (no GPU time)" if mine == 0 else "NO GPU FIGURE — span is CPU only, Partial"))

80
mesh/tools/joules-per-decision/joules_per_decision.py Normal file
View file

@ -0,0 +1,80 @@
"""Joules per typed decision, metered with the same counter jcp-energy's rung 3 uses.
ABI constants are lifted from jcp-energy (sizeof/offsetof-verified against the
macOS 26, 26.5 and 27 SDKs), not re-derived here.
"""
import ctypes, os, time, statistics, json, warnings
FLAVOR, SIZE, ENERGY_NJ, PENERGY_NJ = 6, 464, 336, 344
_libc = ctypes.CDLL("/usr/lib/libSystem.B.dylib")
_buf = ctypes.create_string_buffer(SIZE)
_pid = os.getpid()
def read_nj():
rc = _libc.proc_pid_rusage(ctypes.c_int(_pid), ctypes.c_int(FLAVOR), ctypes.byref(_buf))
if rc != 0:
raise OSError(f"proc_pid_rusage failed: rc={rc}")
e = int.from_bytes(_buf.raw[ENERGY_NJ:ENERGY_NJ+8], "little")
p = int.from_bytes(_buf.raw[PENERGY_NJ:PENERGY_NJ+8], "little")
return e, p
def liveness(window_s=0.5, interval_s=0.02):
"""jcp-energy's rule: >=5 advancing intervals AND >=80% of them, over >=300ms."""
advanced = total = 0
prev, _ = read_nj(); t_end = time.perf_counter() + window_s
while time.perf_counter() < t_end:
t1 = time.perf_counter() + interval_s
x = 0
while time.perf_counter() < t1: # burn CPU
x += 1
cur, _ = read_nj(); total += 1
if cur > prev: advanced += 1
prev = cur
ok = advanced >= 5 and total > 0 and advanced / total >= 0.80
return ok, advanced, total
import laya_mlx as laya
with warnings.catch_warnings():
warnings.simplefilter("ignore")
agent = laya.load("aac6fef/laya-mlx", dtype="float16")
STATE = "I was billed twice. Please refund the duplicate today."
Q1 = {"refund": {"type":"noul","instructions":"Does the customer ask for money back?"}}
Q3 = {
"department": {"type":"choice","instructions":"Which team should handle this request?",
"criteria":{"billing":"invoices, payments, refunds","technical":"bugs and outages","sales":"new purchases"}},
"urgency": {"type":"score","instructions":"How urgent is this request?","criteria":["not urgent","soon","critical"]},
"refund": {"type":"noul","instructions":"Does the customer ask for money back?"}}
def bench(label, questions, n, device):
a = laya.load("aac6fef/laya-mlx", dtype="float16", device=device) if device else agent
for _ in range(3): a.predict(STATE, questions) # warm
e0, p0 = read_nj(); lat = []
t0 = time.perf_counter()
for _ in range(n):
t = time.perf_counter(); r = a.predict(STATE, questions); lat.append((time.perf_counter()-t)*1e3)
wall = time.perf_counter() - t0
e1, p1 = read_nj()
u = r["usage"]
return {
"label": label, "device": device or "default", "n": n,
"p50_ms": round(statistics.median(lat), 2),
"p95_ms": round(sorted(lat)[int(0.95*len(lat))-1], 2),
"wall_s": round(wall, 3),
"cpu_J_total": round((e1-e0)/1e9, 6),
"cpu_J_per_decision": round((e1-e0)/1e9/n, 6),
"pcore_J_per_decision": round((p1-p0)/1e9/n, 6),
"input_tokens": u["input_tokens"], "output_tokens": u["output_tokens"],
"state_tokens": u["state_tokens"],
}
ok, adv, tot = liveness()
print(json.dumps({"counter":"ri_energy_nj via proc_pid_rusage(RUSAGE_INFO_V6)",
"live": ok, "advancing_intervals": adv, "intervals": tot,
"rule": "jcp-energy rung 3: >=5 advancing AND >=80%, >=300ms"}, indent=2))
rows = []
for dev in ("cpu", "gpu"):
for lbl, qs, n in (("1 question (noul)", Q1, 30), ("3 questions (choice+score+noul)", Q3, 30)):
try: rows.append(bench(lbl, qs, n, dev))
except Exception as ex: rows.append({"label": lbl, "device": dev, "error": repr(ex)[:200]})
print(json.dumps(rows, indent=2))

71
mesh/tools/joules-per-decision/two_fabrics.py Normal file
View file

@ -0,0 +1,71 @@
"""One bounded decision, two fabrics, two attributions.
CPU: the kernel's per-process counter — Coverage::Whole for CPU work,
Attribution::Process, declared period 10 ms.
GPU: IOReport's `GPU Energy` rail — a real reading of the whole GPU that
cannot tell one process from another, so Attribution::Machine, bracketed by
idle baselines.
"""
import ctypes, os, time, json, statistics, warnings
warnings.filterwarnings("ignore")
exec(open('rails.py').read().split('import time, subprocess, sys')[0]) # IOReport
_l = ctypes.CDLL("/usr/lib/libSystem.B.dylib"); _b = ctypes.create_string_buffer(464); _pid = os.getpid()
def cpu_nj():
_l.proc_pid_rusage(ctypes.c_int(_pid), ctypes.c_int(6), ctypes.byref(_b))
return int.from_bytes(_b.raw[336:344], "little")
def gpu_nj(s0, s1):
d = IOR.IOReportCreateSamplesDelta(s0, s1, None); t = 0
for g, sg, name, unit, val in rails(d):
if name == "GPU Energy":
assert unit == "nJ"; t += val
return t
PERIOD_NS = 10_000_000 # the measured, declared period of both counters
STATE = ("Guest in room 1408 says the AC has been out since yesterday and they want to "
"move tonight or leave. They also asked for the incidentals hold to be released.")
SCHEMA = {
"intent": ["maintenance", "room_change", "checkout", "billing", "complaint", "amenity_request"],
"priority": ["low", "normal", "high", "urgent"],
"needs_human": ["yes", "no"],
}
N = 60
def idle(secs):
s0 = sample(); c0 = cpu_nj(); t = time.perf_counter(); time.sleep(secs)
s1 = sample(); c1 = cpu_nj(); dt = time.perf_counter() - t
return gpu_nj(s0, s1) / dt, (c1 - c0) / dt
# ─── CPU: GLiNER2.5-Decide, 340M, Apache-2.0 ────────────────────────────────
import torch
torch.set_num_threads(os.cpu_count() or 8)
from gliner2 import AutoExtractor
t0 = time.perf_counter()
m = AutoExtractor.from_pretrained("fastino/GLiNER2.5-Decide")
load_s = time.perf_counter() - t0
try: m.model.to("cpu")
except Exception: pass
for _ in range(3): m.classify_text(STATE, SCHEMA)
pre_g, pre_c = idle(1.0)
s0 = sample(); c0 = cpu_nj(); t0 = time.perf_counter(); lat = []
for _ in range(N):
t = time.perf_counter(); out = m.classify_text(STATE, SCHEMA); lat.append((time.perf_counter()-t)*1e3)
span = time.perf_counter() - t0
s1 = sample(); c1 = cpu_nj()
post_g, post_c = idle(1.0)
cpu_j = (c1 - c0) / 1e9
gpu_leak = (gpu_nj(s0, s1) - (pre_g + post_g) / 2 * span) / 1e9
gliner = {
"model": "fastino/GLiNER2.5-Decide (340M, Apache-2.0, DeBERTa-v3-large)",
"fabric": "cpu", "attribution": "Process", "coverage": "Whole (CPU work)",
"load_s": round(load_s, 2), "decisions": N, "span_s": round(span, 3),
"p50_ms": round(statistics.median(lat), 2), "p95_ms": round(sorted(lat)[int(0.95*N)-1], 2),
"cpu_J_per_decision": round(cpu_j / N, 6),
"gpu_J_per_decision_leak": round(gpu_leak / N, 6),
"span_gradeable": span * 1e9 >= PERIOD_NS,
"answer": out,
}
print(json.dumps(gliner, indent=2, default=str))