Typed decisions, per-fabric energy, and two figures I had to withdraw #154
Loading…
Reference in a new issue
No description provided.
Delete branch "feat/decision-models"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Decision models — state plus typed questions in, a calibrated probability per allowed answer out, no text generated — went from nonexistent to a category between 2026-09-15 and 2026-10-01 (Jev, Kev, Clef, Laya, Liquid d1, GLiNER2.5-Decide, SemIf), converging on three primitives (
noul,choice,score) and one schema. Every quantity in Mesh is a token count, and a decision emits none, so the spec could neither price one, rate one, nor report one.Mesh §3.6 — a unit of work the spec could not measure
Three defects, each checked in the code rather than inferred:
MeshReceipt::uj_per_accepted_token, documented as "the wedge metric", ischecked_divbytokens_accepted. A decision's is 0 — measured"output_tokens": 0on every request — so the headline efficiency figure returnedNonefor the whole category. The receipt sealed with no number.estimate()chargedprompt_tokens + max_tokens, so a caller leavingmax_tokensat a generation's default was priced for an essay the backend will never write, and refused at a ceiling that would have covered the work.Added
DecisionShape/DecisionOutcome,CostProfile::with_option_cost+estimate_decision_uj, anduj_per_decision/uj_per_option_scored. Additive throughout, following the patternCostProfile::calibrationalready set here: new fields areOption, skipped when absent, and a declared option cost of zero reproduces the token-only model exactly. The sealed conformance vectors inconformance/v1/referencestill verify byte-identically, which is the real test that this is compatible.jcp-energy— per-fabric energy, and a refusal that has teethHeterogeneous compute is the premise, so
FabriccoversCpu,Gpu,Npu,Fpga,Dsp,Dram,Package, andFabricSourcenames 13 counters (ProcRusage,IoReport,Powermetrics,Rapl,Nvml,RocmSmi,LevelZero,TegraIna,QualcommQpt,AscendSmi,HabanaSmi,CambriconCnmon,ExternalMeter). A receipt can now say which counter produced a figure.Two defects fixed on the way. ANE energy was counted nowhere:
counts_oncedecides by rail name and looks for anANE Energyaggregate no Apple part here publishes, soANE0was rejected as a summand of a nonexistent parent. And a live counter was being discarded because a sibling rail was frozen: the ladder admits one rung, so it declined IOReport whole and fell back to a per-process counter that documents it excludes GPU and ANE.The fix splits two axes the ladder conflated:
Coverage(whole task?) and newAttribution(whose task?). A GPU rail is a real reading of the whole GPU that cannot attribute →Machine.survey()returns every(fabric, source)pair with a status, and the five negative statuses are the substance —Frozen,Absent,NeedsPrivilege,NoCounterExists,NotImplemented. OnlyNoCounterExistsis the hardware's fault, which is what makes an energy-disclosure exemption legitimate; presence is probed so an unread accelerator saysNotImplemented, never the falseAbsent.FabricStatus::may_grade_spanenforces whatmin_window_nsnever did: a span shorter than the counter's update period is not a measurement of it, and an undeclared period can never grade a span at all. That rule immediately indicted our own best figure, so the counter got characterised rather than the rule weakened —ri_energy_njadvanced in 0% of intervals at 50 µs, 35% at 1 ms, 87% at 5 ms, 100% at 10 ms, batching in ~20 mJ lumps below 1 ms. Declared 10 ms, not the 5 ms where it clears our 80% criterion, because over-declaring refuses work that would have fit while under-declaring admits work the budget cannot cover.⚠️ Two self-corrections in this branch
The GPU rail's declared period was wrong by two orders, in the unsafe direction. I first declared 10 ms. Measured properly, the gap between actual changes is p50 12.7 ms / p90 91 ms / max 992 ms at idle, and p50 13.1 ms / max 42 ms under sustained load — the rail republishes when there is activity and holds when there is not. Declared 1 s, the idle worst case, since a span inside a hold reads zero and would have been graded as a measurement of zero.
The GPU per-decision figure is withdrawn. I reported 0.104 J per three-question decision as GPU 0.0966 J + process CPU 0.0075 J. Measuring per-process GPU time as a delta over the span showed com.apple.Safari holding 99.5% of GPU time in a 4 s window while this process held 0.000 s, at host load average 132 on 18 CPUs. A rail delta over that span is a reading of Safari. The 0.08% idle-bracket correction I cited as evidence of cleanliness proves nothing — a bracket subtracts steady background, not a contender overlapping the span but not the brackets.
What stands is the better half: Laya 0.486 J and GLiNER2.5-Decide 0.555 J per three-question decision, process-attributed, immune to that contention. SPEC §3.6's worked example uses the CPU figure (486 000 µJ, 162 mJ per answered question) and says why it is not the GPU one. It took a contaminated GPU figure to demonstrate what
Attribution::Processis for.eoc/standards— proposed SCI-for-AI annex on the provenance of ESCI = (E × I + M) per Ris well specified in every term but one.Ihas a location-based methodology,Ma lifecycle one,Rmust be stated and reproducible —Ehas no comparable requirement, so a figure can be labelled "measured" with no counter, span, silicon or boundary named.Five machine-checkable requirements: name the counter; declare its period and refuse a measured claim for a shorter span; declare principal-vs-machine attribution with the idle-correction magnitude; break down per fabric; and say why a fabric is uncovered. The last is the lever — the EU AI Act GPAI exemption for lacking "critical information from compute or hardware provider" is a claim of
no-counter-exists, and an unprivileged on-die counter narrows where that is honest.No change to
Ris proposed, and that corrects me: I had read "tokens are a natural functional unit" as the unit being token-shaped. SCI-for-AI supports diverse units — tokens for language models, inferences for classifiers, FLOPs for training — soR = one inferencealready covers these models, as our own prior draft already recorded. The Mesh defect was real; the standards-level version of it was not.Verification
mesh: 52 tests, 0 clippy. jcp-energy: 99 tests. Rebased onto current
main; no file here was modified onmainsince the branch point. Measurement harnesses are in-tree (mesh/tools/joules-per-decision/,jcp/tools/counter-resolution/) so every number above is reproducible rather than quoted.🤖 Generated with Claude Code
Decision models — state plus typed questions in, a calibrated probability per allowed answer out, no text generated — went from nonexistent to a category between 2026-09-15 and 2026-10-01, converging on three primitives (`noul`, `choice`, `score`) and one schema. Every quantity in Mesh is a token count, and a decision emits no tokens, so the spec could neither price one, rate one, nor report one. Three defects, each checked in the code rather than inferred: - `MeshReceipt::uj_per_accepted_token`, documented as "the wedge metric", is `checked_div` by `tokens_accepted`. A decision's is 0 — measured `"output_tokens": 0` on every request — so the headline efficiency figure returned `None` for the whole category. The receipt sealed with no number. - `estimate()` charged `prompt_tokens + max_tokens`, so a caller that left `max_tokens` at a generation's default was priced for an essay the backend will never write, and refused at a ceiling that would have covered the work. - Nothing could express the dimension a decision's cost actually follows: the schema it scores. One question over 20 options is more work than one over 2 at identical token counts. Added `DecisionShape` on the request and `DecisionOutcome` on the outcome (questions, options scored, and how much of the input was schema rather than state), `CostProfile::with_option_cost` + `estimate_decision_uj`, and `uj_per_decision` / `uj_per_option_scored` on the receipt. SPEC §3.6 states the three MUSTs; §4's estimate formula and §6's field table are updated. Additive throughout, following the pattern `CostProfile::calibration` already set here: the new fields are `Option`, skipped when absent, and a declared option cost of zero reproduces the token-only model exactly. So an existing request's `request_id` is unchanged, a generation's signed core gains no keys, and **the sealed conformance vectors in `conformance/v1/reference` still verify byte-identically** — which is the real test that this is compatible, and it passes. 52 tests, 0 failures, 0 clippy findings. Two of the four new tests are the defects stated as invariants: a decision receipt now has an efficiency figure where the token metric has none, and a decision is not priced for tokens it never emits. Both were falsified by reverting the fix — removing the `estimate()` branch fails two router tests, making `uj_per_decision` read `tokens_accepted` fails two receipt tests. `tools/joules-per-decision/` is the evidence: a harness that meters a real Apache-2.0 decision model (Laya, ModernBERT-large 421M, via MLX) per fabric on Apple silicon, and the numbers in §3.6 come from it. On an M5 Max a three-question decision costs **0.104 J** — 0.0966 J on the GPU rail plus 0.0075 J of process CPU energy — at an 11.85 ms p50, which is ~35 mJ per answered question. Idle baselines bracket the run and correct it by 0.08%, so the figure is measurement, not modelling. The energy counters are read unprivileged; the CPU-rail/GPU-rail asymmetry on macOS is noted there and is the subject of follow-up work on `jcp-energy`. The harness also reproduced a live defect in the category it measures: the published Laya checkpoint ships a fitted temperature of 0.1006 for choice questions with 11+ options, which sharpens logits roughly tenfold and reports near-certainty on what may be a coin flip. A third-party porter found it by reproducing upstream numerically and clamps it with a warning; the publisher ships it. A calibrated probability is a claim, and this category ships claims with no receipt — which is the argument for §3.6 in one line. Co-Authored-By: Claude Opus 5 <[email protected]>Heterogeneous compute is the premise, so the taxonomy had to stop being three Apple rails. `Fabric` now covers `Cpu`, `Gpu`, `Npu`, `Fpga`, `Dsp`, `Dram` and `Package` — `Ane` is gone, since Apple's Neural Engine, a Qualcomm or Intel NPU, a TPU, Ascend, Gaudi and Cambricon are one class of silicon and the vendor belongs in the source, not the fabric. `FabricSource` names the counter: `ProcRusage`, `IoReport`, `Powermetrics`, `Rapl`, `Nvml`, `RocmSmi`, `LevelZero`, `TegraIna`, `QualcommQpt`, `AscendSmi`, `HabanaSmi`, `CambriconCnmon`, `ExternalMeter`. A receipt can now say *which* counter produced a figure: two readings graded `HwShunt` from different sources are not interchangeable, because they have different domains, different update periods and different attribution. `survey()` returns every `(fabric, source)` pair with this host's status, and the **five negative statuses are the substance**: - `Frozen` — silicon and counter both present, counter not advancing. - `Absent` — no such silicon here. - `NeedsPrivilege` — present, needs root we do not have. - `NoCounterExists` — present, and exposes no energy counter at all, so any figure is modelled by anyone. A property of the hardware. - `NotImplemented` — a counter exists and we have no reader. Our gap, named. Blurring those is what makes an energy survey worthless, and one of them is load-bearing outside this crate: a disclosure regime that excuses a provider who "lacks critical information from the compute or hardware provider" is relying on `NoCounterExists`, and this distinguishes it from the three statuses that are nobody's hardware's fault. Presence is probed (`/sys/class/kfd`, `/dev/accel/accel0`, `/dev/davinci0`, `/dev/hl0`, `/dev/cambricon_dev0`, `/sys/bus/i2c/drivers/ina3221`, …) so an unread accelerator reports `NotImplemented` rather than the false claim `Absent`. Apple's rails are probed live — two reads with a CPU burn between them — because which rails a macOS version exposes to an unprivileged reader is a fact to observe on the host, not to assume. **And the refusal is now real.** `FabricStatus::may_grade_span` enforces what `min_window_ns` never did: a span shorter than the counter's update period is not a measurement of that span, and a counter whose period is **undeclared** can never grade a span at any length, because an unknown error direction at a ceiling must be read as possibly-under-reporting. That rule immediately indicted our own best figure, which is how it earned its place: the per-process CPU counter had an undeclared period, so the only process-attributed reading on Apple silicon could not be graded. Rather than weaken the rule, `tools/counter-resolution/` measures the counter. Burning this thread's CPU and sampling 60 times per interval, the fraction of intervals in which `ri_energy_nj` advanced: **0% at 10 µs and 50 µs, 2% at 100 µs, 5% at 500 µs, 35% at 1 ms, 63% at 2 ms, 87% at 5 ms, 100% at 10 ms**. The crate's liveness criterion is 80%, so 5 ms is where it first clears — and **10 ms is declared**, because over-declaring refuses work that would have fit while under-declaring admits work the budget cannot cover. Below 1 ms it does not lag, it **batches**: on the few intervals where it moved at all the median step was ~20 mJ, two orders above the 1 ms step. A sampler reading it every 500 µs would see nothing most of the time and a 20 mJ jump occasionally, and would report both as measurements — the same pathology the crate already refuses on IOReport's CPU rails, and the same shape as the published 95.4% under-read at 0.5 ms sampling. With the period declared, this morning's figure stands for the reason it was taken that way: 2.606 s ≫ 10 ms, which is why it is the mean of 200 decisions over one long span and not 200 counter reads. 98 tests pass, 0 clippy findings. Six new invariants: each pair named once, a source with no backend is never `Live`, an undeclared period grades nothing at any span, source names are stable and distinct, only `ProcRusage` may claim per-process attribution, and the declared period refuses exactly the intervals the measurement showed it cannot resolve. Co-Authored-By: Claude Opus 5 <[email protected]>