Typed decisions, per-fabric energy, and two figures I had to withdraw #154

Merged

dcharlot merged 7 commits from feat/decision-models into main 2026-10-05 08:54:07 -04:00

Conversation 0 Commits 7 Files changed 20 +2283 -26

2026-10-05

Transaction Science

mesh: withdraw the GPU per-decision figure — another process held 99.5% of the GPU ...

I reported 0.104 J per three-question decision as GPU 0.0966 J plus process CPU
0.0075 J, and put it in SPEC §3.6's worked example. The GPU half does not hold.

`gpu_share.py` measures the thing I had assumed away: per-process GPU time from
`accumulatedGPUTime` on each IOUserClient in the IOAccelerator subtree, as a
**delta over the span** rather than the cumulative totals (which are useless —
WindowServer has days of accrued time). Over a 4.06 s span with an MLX matmul
loop running, **com.apple.Safari held 2.470 s of 2.482 s — 99.5%** — while this
process registered 0.000 s. The host was at load average 132 on 18 CPUs,
running another session's VVC/VTM encodes, which is this machine's normal state
rather than an unlucky moment.

A rail delta over that span is a reading of Safari. The rule already shipped
elsewhere in the fleet is the right one: attribute a rail reading to this
process only when it holds ≥99% of the GPU time advancing in the span, report
exactly zero when it holds none, and otherwise **issue no GPU figure**, marking
the span CPU-only and partial. By that rule this measurement issues none.

The 0.08% idle-bracket correction I cited as evidence of cleanliness proves
nothing, and I should not have read it that way: a bracket subtracts *steady*
background, and cannot subtract a contending process whose work overlaps the
span but not the brackets.

What stands, and why it is the more interesting half: the **CPU** figures are
untouched — Laya 0.486 J and GLiNER2.5-Decide 0.555 J per three-question
decision — because the kernel's per-process counter attributes to a principal,
so neither load average 132 nor Safari's GPU work enters them. That is exactly
what `Attribution::Process` is for, and it took a contaminated GPU figure to
demonstrate it. SPEC §3.6's worked example now uses the CPU figure (486 000 µJ,
162 mJ per answered question) and says why it is not the GPU one. Latency
figures are unaffected.

Also corrects the direction of my earlier framing. I presented the GPU path as
"4.7× cheaper, and the price is that it cannot be bound to a principal". The
truth is stronger: on a shared host it cannot be bound to a principal *and*
the number itself is not the workload's. A sound GPU figure needs a host with
no display server and no other GPU client — on this fleet, a headless Linux box
with NVML, not a Mac.

`gpu_rail_period.py` is also added, the harness behind 92449670's correction of
the rail's declared period.

Co-Authored-By: Claude Opus 5 <[email protected]>

5208dc5936

• View at this point in history

2026-10-05 07:20:58 -04:00

Transaction Science

jcp-energy: the GPU rail's period was wrong by two orders, in the unsafe direction ...

I declared the Apple `GPU Energy` rail's update period as 10 ms yesterday
(ed00cab1) on the strength of "~10 ms refresh on the parts measured here". That
was an assumption dressed as an observation, and it is wrong the dangerous way:
too short a declared period lets `may_grade_span` grade a span the counter
never covered, and a span landing entirely inside one hold reads **zero** and
would have been graded as a measurement of zero.

Measured properly, sampling every 2 ms on an M5 Max, the gap between actual
changes in the rail:

| condition | p50 | p90 | max | median step |
|---|---:|---:|---:|---:|
| idle | 12.7 ms | 91.0 ms | **992.2 ms** | 1.06 mJ |
| sustained MLX matmul | 13.1 ms | 15.0 ms | 42.0 ms | 73.50 mJ |

The rail has no fixed period. It republishes when there is activity to report
and holds when there is not, so the tight figure is only available to a
measurer who can guarantee the fabric stays busy — and a measurer cannot.
**Declared 1 s**, the idle worst case, because that is the condition a short
span is most likely to fall inside.

This also reconciles a fleet note that put the hold at "300–600 ms under load",
measured from a GGUF decode. That decode is dispatch-bound with idle gaps
between tokens, so it was sampling the idle behaviour rather than the loaded
one; under genuinely sustained load the rail is tight (max 42 ms). The two
observations agree once the cause is named, and the cause is what decides which
number to declare.

The headline figure survives, for the reason it was taken that way: the Laya
GPU span was 2.606 s, above even the idle worst case. A single 11.85 ms
decision could never have been one counter read, which is pinned in the test
alongside the 42 ms and 992 ms refusals so the period cannot drift back.

**Also corrects an overclaim in ed00cab1's message**, which said this "stop[s]
metering the Neural Engine as free". The code change there was right — count
the `ANE Energy` aggregate where a part publishes one, else the per-engine
rails — but on this hardware it changes nothing, because `ANE0` is frozen for
an unprivileged reader. The survey already reported `npu/ioreport` as `Frozen`;
it now also records the only route that exists: `sudo powermetrics --samplers
ane_power`, machine-wide, `NeedsPrivilege`. ANE work runs in `aned`, so it is
not this process's CPU energy either, and the load-balancer clients carry no
statistics — a span that used the ANE is CPU+GPU only and Partial.

Co-Authored-By: Claude Opus 5 <[email protected]>

a625a7d70c

• View at this point in history

2026-10-05 07:20:58 -04:00

Transaction Science

eoc/standards: propose provenance requirements for E in SCI-for-AI ...

`SCI = (E × I + M) per R` is well specified in every term but one. `I` has a
location-based methodology, `M` a lifecycle one, and `R` must be stated and
reproducible. **`E` has no comparable requirement**: a figure can be labelled
"measured" without saying what measured it, over what span, on which silicon,
or on whose behalf.

Five proposed requirements, all machine-checkable: E1 name the counter (not the
word "measured"); E2 declare its update period and **refuse a measured claim
for a span shorter than that period**, with an undeclared period never
supporting one; E3 declare whether the figure counts one principal's work or
the whole machine, and if machine-wide, report the magnitude of the idle
correction; E4 break down per fabric and state which fabrics are uncovered;
E5 say *why* a fabric is uncovered, as one of frozen / absent /
needs-privilege / no-counter-exists / not-implemented.

E5 is the one with teeth outside accounting. The EU AI Act's GPAI Code of
Practice requires energy documentation for training and inference while
permitting an exemption where a provider lacks "critical information from
compute or hardware provider" — which is a claim of `no-counter-exists`. E5
distinguishes it from the four statuses that do not justify it, and an
unprivileged on-die counter needs no cooperation from the compute provider at
all, so it narrows the set of accelerators for which the claim is honest.

The evidence is three independent observations plus our own: instrumented
AI-energy studies are GPU-rail-only and none declares a sampling interval; a
reference implementation in this ecosystem stamps `measured` on a lifecycle
estimator's output; per-call energy is being read from provider SDKs with no
stated boundary. And counters do take the unsafe direction by large factors —
a 588-run study found one under-reporting integrated power by 95.4% at 0.5 ms
sampling, a second paper built its per-token decomposition on that same counter
for the opposite reason without citing the first, and a third observation has
the same counter class reading 4–30× the board's draw. We reproduced the shape
on a fourth counter and characterised it (0% advancing at 50 µs through 100% at
10 ms, with ~20 mJ batched steps below 1 ms), which is what let us declare a
period instead of assuming one.

**A correction this annex makes explicitly: no change to `R` is proposed.** I
had taken SCI-for-AI's "tokens are a natural functional unit" to mean the unit
was token-shaped, and therefore undefined for the zero-token decision models
that appeared in Sept–Oct 2026. That is wrong: SCI-for-AI supports diverse
functional units — tokens for language models, **inferences for classifiers**,
FLOPs for training — so `R = one inference` already covers them. The gap is in
the energy, not the unit of work. Our own Mesh defect was real and is fixed
(ed0c07b5); the standards-level version of it was not.

Complements `sci-for-ai-submission.md`, which maps EOC receipts onto the
formula and takes E as given. Cross-linked both ways. Proposed URN
`urn:gsf:sci-ai:ext:energy-provenance:v1`, additive, and imposing nothing on
deployments that report estimates — which stay conformant provided they say so.

Co-Authored-By: Claude Opus 5 <[email protected]>

1a774ed785

• View at this point in history

2026-10-05 07:20:58 -04:00

Transaction Science

mesh/tools: the attribution trade, measured with the fabric held constant ...

The cheapest fabric and the attributable one are not the same fabric, and this
measures the gap instead of asserting it.

On an M5 Max, the same three-question bounded decision:

  - Laya 421M on **CPU**: 0.486 J, 629 ms
  - GLiNER2.5-Decide 340M on **CPU**: 0.555 J, 480 ms
  - Laya 421M on **GPU**: 0.104 J, 11.9 ms

Holding the fabric constant, the two models are close — GLiNER2.5-Decide
spends 14% more per decision and answers 24% faster, while scoring 60.1% to
Laya's 46.6% on Fastino's 17-dataset suite. Worth recording because the Mesh
SPEC's §3.6 figures come from Laya, which is the **weakest** member of this
family on the only cross-model suite published for it; nothing here should
imply it is the best available.

Moving the same model to the GPU costs 4.7× fewer joules and is 53× faster, and
the figure stops being attributable: IOReport's `GPU Energy` rail is a real
reading of the whole GPU that cannot say whose work it was. So the cheap path
yields a number that cannot be bound to a principal and the expensive path
yields one that can — which is the concrete reason `Attribution` is a separate
axis from `Coverage` rather than a weaker grade of it.

The CPU figure is also *complete*, not just attributable: over the GLiNER run
the GPU rail, bracketed by idle baselines either side, moved by −0.0013 J per
decision. Negative — the idle estimate slightly exceeded the real draw — so GPU
involvement sits below the measurement's own noise and the process-attributed
CPU number is the whole task. That is the claim the GPU rows cannot make.

`two_fabrics.py` is the harness. It needs `gliner2`, `peft` and `torch`; the
model is Apache-2.0 and 340M, and it runs on CPU without a GPU at all, which
is what makes the attributable path available on any host rather than only on
one with a counter we can read per principal.

Co-Authored-By: Claude Opus 5 <[email protected]>

f78a796f3c

• View at this point in history

2026-10-05 07:20:58 -04:00

Transaction Science

jcp-energy: survey every accelerator, and refuse a span the counter cannot resolve ...

Heterogeneous compute is the premise, so the taxonomy had to stop being three
Apple rails. `Fabric` now covers `Cpu`, `Gpu`, `Npu`, `Fpga`, `Dsp`, `Dram` and
`Package` — `Ane` is gone, since Apple's Neural Engine, a Qualcomm or Intel
NPU, a TPU, Ascend, Gaudi and Cambricon are one class of silicon and the vendor
belongs in the source, not the fabric.

`FabricSource` names the counter: `ProcRusage`, `IoReport`, `Powermetrics`,
`Rapl`, `Nvml`, `RocmSmi`, `LevelZero`, `TegraIna`, `QualcommQpt`, `AscendSmi`,
`HabanaSmi`, `CambriconCnmon`, `ExternalMeter`. A receipt can now say *which*
counter produced a figure: two readings graded `HwShunt` from different sources
are not interchangeable, because they have different domains, different update
periods and different attribution.

`survey()` returns every `(fabric, source)` pair with this host's status, and
the **five negative statuses are the substance**:

  - `Frozen` — silicon and counter both present, counter not advancing.
  - `Absent` — no such silicon here.
  - `NeedsPrivilege` — present, needs root we do not have.
  - `NoCounterExists` — present, and exposes no energy counter at all, so any
    figure is modelled by anyone. A property of the hardware.
  - `NotImplemented` — a counter exists and we have no reader. Our gap, named.

Blurring those is what makes an energy survey worthless, and one of them is
load-bearing outside this crate: a disclosure regime that excuses a provider
who "lacks critical information from the compute or hardware provider" is
relying on `NoCounterExists`, and this distinguishes it from the three statuses
that are nobody's hardware's fault. Presence is probed (`/sys/class/kfd`,
`/dev/accel/accel0`, `/dev/davinci0`, `/dev/hl0`, `/dev/cambricon_dev0`,
`/sys/bus/i2c/drivers/ina3221`, …) so an unread accelerator reports
`NotImplemented` rather than the false claim `Absent`. Apple's rails are
probed live — two reads with a CPU burn between them — because which rails a
macOS version exposes to an unprivileged reader is a fact to observe on the
host, not to assume.

**And the refusal is now real.** `FabricStatus::may_grade_span` enforces what
`min_window_ns` never did: a span shorter than the counter's update period is
not a measurement of that span, and a counter whose period is **undeclared**
can never grade a span at any length, because an unknown error direction at a
ceiling must be read as possibly-under-reporting.

That rule immediately indicted our own best figure, which is how it earned its
place: the per-process CPU counter had an undeclared period, so the only
process-attributed reading on Apple silicon could not be graded. Rather than
weaken the rule, `tools/counter-resolution/` measures the counter. Burning this
thread's CPU and sampling 60 times per interval, the fraction of intervals in
which `ri_energy_nj` advanced: **0% at 10 µs and 50 µs, 2% at 100 µs, 5% at
500 µs, 35% at 1 ms, 63% at 2 ms, 87% at 5 ms, 100% at 10 ms**. The crate's
liveness criterion is 80%, so 5 ms is where it first clears — and **10 ms is
declared**, because over-declaring refuses work that would have fit while
under-declaring admits work the budget cannot cover.

Below 1 ms it does not lag, it **batches**: on the few intervals where it moved
at all the median step was ~20 mJ, two orders above the 1 ms step. A sampler
reading it every 500 µs would see nothing most of the time and a 20 mJ jump
occasionally, and would report both as measurements — the same pathology the
crate already refuses on IOReport's CPU rails, and the same shape as the
published 95.4% under-read at 0.5 ms sampling.

With the period declared, this morning's figure stands for the reason it was
taken that way: 2.606 s ≫ 10 ms, which is why it is the mean of 200 decisions
over one long span and not 200 counter reads.

98 tests pass, 0 clippy findings. Six new invariants: each pair named once, a
source with no backend is never `Live`, an undeclared period grades nothing at
any span, source names are stable and distinct, only `ProcRusage` may claim
per-process attribution, and the declared period refuses exactly the intervals
the measurement showed it cannot resolve.

Co-Authored-By: Claude Opus 5 <[email protected]>

421b7e9b71

• View at this point in history

2026-10-05 07:20:58 -04:00

Transaction Science

jcp-energy: read each compute fabric, and stop metering the Neural Engine as free ...

Two defects, both found by measuring a real GPU-resident inference on an
M5 Max and neither visible from the ladder's own tests.

**ANE energy was counted nowhere.** `counts_once` decides by rail name, and on
every Apple part seen here there is no `ANE Energy` aggregate to recognise — the
only ANE channel is a single `ANE0` in mJ, which the name rule rejects as a
summand of an aggregate that does not exist. A crate test pinned that state and
said in as many words that changing it should be "a decision, not an accident".
This is the decision: a Neural Engine workload metered as though it cost
nothing, and under-reporting at a ceiling is the direction this crate already
calls unsafe, because it admits work the budget cannot cover. `sum_energy_uj`
can see the whole sample, so it takes the aggregate where a part publishes one
and falls back to the per-engine `ANE<n>` rails where it does not — never both,
so nothing is counted twice. The name-level gate stays exactly as it was, which
is what keeps the two paths from colliding.

**A live counter was being discarded because another rail was frozen.** The
ladder admits one rung and reports one number. For an unprivileged reader on
macOS 27 the `CPU Energy` rails hold their value while `GPU Energy` advances
continuously in nanojoules, so the IOReport rung is declined whole — and with
it the only counter that reads the fabric every local inference on a Mac
actually runs on. The kernel's per-process counter that wins instead documents
that it excludes the GPU and the Neural Engine. So the platform's best figure
covered neither.

The fix separates two axes the ladder conflated. `Coverage` says whether a
figure covers the whole task; new `Attribution` says *whose* task. A GPU rail
reading is a real reading of the whole GPU that cannot tell one process's share
from another's — `Attribution::Machine`, not a lesser `Coverage`. The kernel's
CPU figure is `Attribution::Process`. `FabricMeter` composes the two counters
rather than choosing between them, because together they cover the fabrics the
work used, and `total_uj` returns the sum *and the fabrics it is a sum of*, so a
caller states its coverage instead of implying it.

`FabricEnergy` carries `span_ns` deliberately. A vendor energy counter has an
update period, and a figure taken over a shorter span is not a measurement of
that span: a published study of 588 runs found one such counter under-reporting
integrated power by 95.4% at 0.5 ms sampling, recovering to 1.6% only at 1 s.
That is why the figures below are the mean of 200 decisions over a 2.6 s span
and not 200 per-decision counter reads — an 11.85 ms span is unmeasurable by
this class of counter. Enforcing that bound at the grade boundary is the
follow-up; this commit makes the span visible so a caller can apply it.

Measured on an M5 Max, per three-question typed decision to a 421M decision
model: **GPU 96 591 µJ + process CPU 7 536 µJ = 104 127 µJ**, idle baselines
bracketing the run and correcting it by 0.08%. The harness is
`mesh/tools/joules-per-decision/` from ed0c07b5, and those are the numbers in
Mesh SPEC §3.6. The host test proves the composition live rather than asserting
it: 8M iterations of pure CPU work reported CPU 191 563 µJ as `Process` beside
GPU 7 234 156 µJ as `Machine` over the same 912 ms span — the GPU drawing 7.9 W
during a CPU-only test being exactly the reason `Machine` has to exist, and
exactly what a single-number ladder would have attributed to this process.

91 tests pass. The two clippy findings in this workspace are pre-existing in
`jcp-schema::flow` on main and are untouched.

Co-Authored-By: Claude Opus 5 <[email protected]>

f89b18964f

• View at this point in history

2026-10-05 07:20:58 -04:00

Transaction Science

mesh: a typed decision is a unit of work this spec could not measure ...

Decision models — state plus typed questions in, a calibrated probability per
allowed answer out, no text generated — went from nonexistent to a category
between 2026-09-15 and 2026-10-01, converging on three primitives (`noul`,
`choice`, `score`) and one schema. Every quantity in Mesh is a token count, and
a decision emits no tokens, so the spec could neither price one, rate one, nor
report one. Three defects, each checked in the code rather than inferred:

  - `MeshReceipt::uj_per_accepted_token`, documented as "the wedge metric", is
    `checked_div` by `tokens_accepted`. A decision's is 0 — measured
    `"output_tokens": 0` on every request — so the headline efficiency figure
    returned `None` for the whole category. The receipt sealed with no number.
  - `estimate()` charged `prompt_tokens + max_tokens`, so a caller that left
    `max_tokens` at a generation's default was priced for an essay the backend
    will never write, and refused at a ceiling that would have covered the work.
  - Nothing could express the dimension a decision's cost actually follows: the
    schema it scores. One question over 20 options is more work than one over 2
    at identical token counts.

Added `DecisionShape` on the request and `DecisionOutcome` on the outcome
(questions, options scored, and how much of the input was schema rather than
state), `CostProfile::with_option_cost` + `estimate_decision_uj`, and
`uj_per_decision` / `uj_per_option_scored` on the receipt. SPEC §3.6 states the
three MUSTs; §4's estimate formula and §6's field table are updated.

Additive throughout, following the pattern `CostProfile::calibration` already
set here: the new fields are `Option`, skipped when absent, and a declared
option cost of zero reproduces the token-only model exactly. So an existing
request's `request_id` is unchanged, a generation's signed core gains no keys,
and **the sealed conformance vectors in `conformance/v1/reference` still verify
byte-identically** — which is the real test that this is compatible, and it
passes. 52 tests, 0 failures, 0 clippy findings.

Two of the four new tests are the defects stated as invariants: a decision
receipt now has an efficiency figure where the token metric has none, and a
decision is not priced for tokens it never emits. Both were falsified by
reverting the fix — removing the `estimate()` branch fails two router tests,
making `uj_per_decision` read `tokens_accepted` fails two receipt tests.

`tools/joules-per-decision/` is the evidence: a harness that meters a real
Apache-2.0 decision model (Laya, ModernBERT-large 421M, via MLX) per fabric on
Apple silicon, and the numbers in §3.6 come from it. On an M5 Max a
three-question decision costs **0.104 J** — 0.0966 J on the GPU rail plus
0.0075 J of process CPU energy — at an 11.85 ms p50, which is ~35 mJ per
answered question. Idle baselines bracket the run and correct it by 0.08%, so
the figure is measurement, not modelling. The energy counters are read
unprivileged; the CPU-rail/GPU-rail asymmetry on macOS is noted there and is
the subject of follow-up work on `jcp-energy`.

The harness also reproduced a live defect in the category it measures: the
published Laya checkpoint ships a fitted temperature of 0.1006 for choice
questions with 11+ options, which sharpens logits roughly tenfold and reports
near-certainty on what may be a coin flip. A third-party porter found it by
reproducing upstream numerically and clamps it with a warning; the publisher
ships it. A calibrated probability is a claim, and this category ships claims
with no receipt — which is the argument for §3.6 in one line.

Co-Authored-By: Claude Opus 5 <[email protected]>

2f3b000153

• View at this point in history

2026-10-05 07:20:58 -04:00