meter model subprocesses where they ran: the L3 model enters the run's figure #211

Merged

dcharlot merged 12 commits from feat/meter-model-subprocesses into main 2026-10-10 22:25:18 -04:00

Conversation 0 Commits 12 Files changed 18 +2673 -291

2026-10-10

Transaction Science

ci: run the L3 and fabric-meter oracle tests on the macOS energy job ...

No job ran the oracle assertions this branch added. joule-code-rs's matrix
entry runs on linux-arm64, where joule-fix's L3 energy test has no tree
meter and takes its fallback branch; jouleclaw-loader-gguf's calibration
tests need the fabric-meter feature, which no job passed, and their oracle
needs macOS.

The macOS energy job now runs both, each step in the job's style (pipefail,
a tee'd log, a grep gate on a line the test prints only where its gated
assertion ran):
- joule-code-remediate `l3_`: 'l3 model process:' (the model's reading
  checked against the stand-in's own count). The stand-in itself stays
  #[ignore]: it is the subprocess the oracle test runs (filtered to it,
  --ignored), not a test of its own, and the oracle test is not ignored.
- jouleclaw-loader-gguf --features "rayon fabric-meter": the marginal_energy
  tests ('calibration row stamped on macOS:') and tests/process_calibration
  ('process calibration: CPU member'); this also builds the fabric-meter
  examples. joule-code-rs's target dir and Cargo.lock join the job's cache.

The l3-jouleclaw example workspace stays out of this job (a heavy separate
build on the one macOS runner); its README says its oracle tests run locally.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

68aa43bafd

• View at this point in history

2026-10-10 22:24:17 -04:00

Transaction Science

l3-jouleclaw: grade a fallback span as no reading; never read the llama engine whole ...

The docs said a host with no per-process counter gets the token-count
estimate off macOS, but there ProcessSpanMeter falls back to detect_meter()
(the 28.6 W x wall estimator in this build), which it marks Whole, and grade()
kept that figure. A fallback span is the machine's figure, not the engine's,
so grade() now treats it as no reading and uses the rest (host_counter, else
the token estimate): one way of reading the host whichever engine ran, and
the docs hold. That is simpler than documenting a second host figure that
only the in-process engine would get.

JOULE_CODE_L3_SELF_CONTAINED could never take effect: the llama engine is
read at its exit, which the tree meter always marks (TreeGap::Exited), so no
reading was ever whole. The setting and its plumbing are gone and the docs
say the reading is a lower bound, always graded as an estimate.

A missing llama binary had become LlmError::Upstream("... exited 127"): the
binary is exec'd by the gating sh, whose failed exec reads as the model's own
failure. The binary is now resolved first (a path as given, else PATH; a
regular executable file) and one that cannot run is Unavailable, as a failed
spawn was, with nothing spawned.

The README says the oracle tests run only locally on Apple Silicon (CI tests
this workspace on Linux, where they take the fallback), and lines this branch
added name the capability, not a third-party engine.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

cbcc27f1f7

• View at this point in history

2026-10-10 22:24:01 -04:00

Transaction Science

jouledesk-github: a model process is never read whole; a fallback carries no model reading ...

JOULEDESK_LLAMA_SELF_CONTAINED could never take effect. A reading was sealed
HwShunt only with no gap and that declaration, but run_model reads the model
at its exit, and the tree meter marks every member that exited in the span
(TreeGap::Exited: an exiting process's record lacks its last time on a
core). The model always exits, so no reading was ever whole. The setting,
the Llama struct that carried it and the whole-reading branch are gone; the
docs say plainly that the reading is a lower bound and the work is always
sealed as an estimate (the reading plus the host meter's figure). A reading
whose tree showed no gap says why it is still partial (NOT_READ_AT_EXIT).

When the model binary is missing, sh's exec fails (127), the answer falls
back to the deterministic extract, yet the work still carried the reading of
that sh, so the reply named deterministic-extract beside a "model process
... read on its own process tree" line. A fallback now carries no model
reading, as the field's docs said; a test runs a missing binary.

The lines this branch added that named a third-party engine now name the
capability (a model binary that offloads to the GPU, the binary's log).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

e325ab5d9a

• View at this point in history

2026-10-10 22:24:01 -04:00

Transaction Science

joule-code-remediate: seal the L3 window's device classes as separate records ...

The L3 window's machine-wide figure was Measurement::total_uj(), every
device class summed, and the tally sealed the run's whole figure as one
DeviceClass::Cpu record. Where the machine meter reads a GPU (an Apple GPU
rail, NVML on Linux), that GPU energy was sealed as CPU energy, which the
sandbox schema forbids (a host must not meter accelerator work as
undifferentiated Cpu work).

L3Energy now carries the machine figure per class (machine_by_class;
machine_uj is their sum), the tally keeps the non-CPU part of its estimate
per class, and meter_records() seals one Estimator record per class: the
CPU (cargo windows, the model's own reading, the machine's CPU) and each
other class the meter read. The records sum to the run's figure, so the
receipts' figure is unchanged. An attributed run is still one ModelBased
CPU record: every window attributed means no L3 window.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

49c88f4387

• View at this point in history

2026-10-10 22:24:01 -04:00

Transaction Science

joule-code-remediate: drop JOULE_FIX_L3_SELF_CONTAINED, which could never take effect ...

The L3 window was attributed only for a tree reading with no gap and the
operator's JOULE_FIX_L3_SELF_CONTAINED=1. But run_l3 reads the model at its
exit (waited for, not reaped), and the tree meter marks every member that
exited in the span (TreeGap::Exited): an exiting process's record lacks its
last time on a core. The root always exits, so no reading was ever whole and
the setting, the attributed field and the whole-reading branch were dead —
while the docs offered the setting as the way to a whole reading.

The setting and its plumbing are gone, and the docs now say plainly that a
model subprocess is never read whole: its reading is a lower bound on its CPU
energy, and the window is always an estimate, the reading plus the machine-
wide figure. A reading whose tree showed no gap (only possible if the wait
for the model's exit failed) says so (NOT_READ_AT_EXIT) rather than coming
back with nothing to explain why it is partial.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

8a2f1b1c0d

• View at this point in history

2026-10-10 22:24:01 -04:00

Transaction Science

jouleclaw-loader-gguf: fabric_calibrate replaces every row of the members it measures ...

fabric_calibrate merged its rows by keeping every existing row of a member
it did not write this run. A member refused this run (the GPU shared with
another process) or skipped at warm-up kept the row an earlier run took,
so the file came out mixing an old SoC-marginal row (no declared grade,
loaded Benchmarked) with this run's per-process Measured rows, and the
placer, which ranks on uj_per_gflop alone, compared the two as if they
were one kind of figure, while the run said the fabric now placed on
measured figures. A member skipped at warm-up also counted as neither
written nor refused, so the run exited 0.

marginal_energy::merge_calibration now drops every row of the members the
run owns (FABRIC_CALIBRATE_MEMBERS: cpu-neon, amx, metal-gpu) before adding
the rows it measured, so a refused or skipped member falls back to its
seeded figure; rows it does not own (local/metal-mpp, which mpp_energy
writes) stay as they were. A member skipped at warm-up is now refused
(exit 2) and loses any row an earlier shape gave it. mpp_energy already
writes its one row only when measured and nothing otherwise.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

86f079b9c3

• View at this point in history

2026-10-10 22:24:01 -04:00

Transaction Science

jouleclaw-loader-gguf: pin the process-figure refusal of a partial span of both counters ...

A span can read both of this process's counters and still be partial (work
declared to use the Neural Engine). process_figure refuses it by its
coverage, which no test drove: the CPU-only case is refused by its source
as well, so dropping the coverage check passed every test.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

d134936e7e

• View at this point in history

2026-10-10 22:24:01 -04:00

Transaction Science

jouleclaw-loader-gguf: calibrate the fabric on this process's own counters ...

fabric_calibrate and mpp_energy priced members only on the IOReport SoC
counter, and refused - writing nothing - wherever its CPU rail is frozen,
which is every macOS 27.0.1 host checked: the placer's calibration file could
not be produced at all.

Where IOReport declines they now do what tune_shape_live_process does: each
member's loop is one ProcessSpanMeter span, this process's CPU energy plus
its GPU energy, no idle to subtract. marginal_energy::process_figure is the
rule, pure and in one place: a figure only from a span metered whole on this
process's own counters (not the host-meter fallback), a CPU member's GPU
share an exact zero, a GPU member's GPU time spent (its energy the rail while
this process was the GPU's sole user); anything else is refused, saying why.
Rows are written at the span's grade through calibration_row, with a method
naming the new domain (process CPU + GPU, not the SoC marginal) and, for AMX,
that its Accelerate work is counted as the kernel attributes it to the
calling threads, unchecked against a rail. fabric_calibrate refuses member by
member and writes the rest (exit 2 when any was refused, 1 when none was
written); a member whose last shape was refused gets no row. Both examples
now merge into the file instead of clobbering it. The IOReport path is
unchanged where its rail is live.

Tested: the rule's every refusal, pure; and a live CPU member against an
oracle sharing no code (this process's ri_energy_nj read at the first op's
start and the last op's end, inside the span): equal, twice. On this host the
examples wrote the NEON and AMX rows (Measured) and refused the GPU members:
the GPU was shared.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

5f9639420c

• View at this point in history

2026-10-10 22:24:01 -04:00

Transaction Science

l3-jouleclaw: test the in-process meter through the path main runs ...

The in-process engine's span was opened inline in infer(), and the energy
test opened its own: the test exercised the meter but not the call main
makes. Both now go through in_process(), so the oracle test covers what the
binary does (SpanUses::CpuGpu included).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

03a1c1a59f

• View at this point in history

2026-10-10 22:24:01 -04:00

Transaction Science

l3-jouleclaw: meter each engine on its own process, graded by coverage ...

The run binary bracketed the inference with host_counter() reads: on macOS 27
the load-average estimator, so an in-process Metal decode and a llama.cpp
subprocess were both estimated from host load, although this host reads both
engines' own processes.

The in-process engine (jouleclaw-runtime, Metal kernels) is now metered by
ProcessSpanMeter(gpu_liveness_load) with SpanUses::CpuGpu: this process's CPU
plus its GPU (an exact zero with no GPU time, the rail when it owned the GPU).
The llama engine's subprocess runs behind a gate (sh waits, then execs
"$0" "$@") and is read on its own process tree by TreeMeter, waited for with
WNOWAIT before the reap (LlamaCompletionBackend::complete_metered). grade() is
the one place the receipt's energy is decided, mesh ProcessMeter's rule: a
reading that covers the whole inference keeps its grade; a partial one (GPU
shared, or a child that exited in its window, or undeclared) is an Estimator,
the reading plus the host counter's figure or the token estimate; no reading
leaves that rest as it was. The receipt has no coverage field, so a partial
reading is never sealed HwShunt.

Tested against oracles sharing no code: this process's own ri_energy_nj read
around a CPU burn (span 0.985 of it, GPU an exact zero, Whole, HwShunt), and
a stand-in llama (this test binary) reporting its own count from inside
(reading 0.998 of it, Estimator). The example workspace builds standalone.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

8ca94fd049

• View at this point in history

2026-10-10 22:24:01 -04:00

Transaction Science

jouledesk-github: read the model subprocess on its own process tree ...

Each mention runs one llama-completion subprocess, metered only by the host
meter: machine-wide, and on macOS 27 the load-average estimator. The model's
own process was never read, although this host counts each process's CPU
energy.

The model now runs behind a gate (sh waits for a line, then execs "$0" "$@",
so arguments are argv, never re-parsed): a TreeMeter rooted at that process
reads it at rest, the gate lets it go, its stdout is drained, it is waited
for with WNOWAIT, the span is stopped on its exited record, then it is
reaped. grade_work is the one place the figure is decided: HwShunt only for a
whole reading of a model the operator declared self-contained
(JOULEDESK_LLAMA_SELF_CONTAINED=1); a partial reading - every model that
exits in its window, and llama.cpp offloading to Metal - is sealed as an
Estimator, the reading plus the host meter's figure, because efficiency/1 has
no coverage field and its figure stands for the whole; a partial reading with
no host figure is no figure and is not sealed; no reading leaves the host
meter's figure as it was. The reply carries the reading and why it is partial
on its own line, on GitHub and Slack.

Tested against an oracle sharing no code: a stand-in model (this test binary
behind a script taking the model's arguments) reports its own ri_energy_nj
from inside; the reading was 0.998-0.999 of it, declared and undeclared.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

9546e45599

• View at this point in history

2026-10-10 22:24:01 -04:00

Transaction Science

joule-code-remediate: meter the L3 model's window into the run's figure ...

l3_pass ran the L3 command under no meter at all, so RunReport::total_uj,
uj_per_verified_fix and both receipts omitted the model's energy: the most
expensive tier of a run was the one the figure left out.

The command now runs under sh, which waits for a gate line and then execs a
plain argv, so the model is the very process a TreeMeter is rooted at and the
shell's start-up stays outside the window. The tree meter reads it at rest,
the machine-wide span opens, the gate and prompt are written, stdout is
drained, the child is waited for with WNOWAIT and the tree span stopped on its
exited record, then it is reaped. grade_l3 is the one place the window is
graded: attributed only for a whole reading of a command the operator
declared self-contained (JOULE_FIX_L3_SELF_CONTAINED, the gateway's rule);
any partial reading - every model that exits in its window has the Exited gap
- is an estimate, the reading plus the machine-wide figure (mesh's
ProcessMeter rule), never a hardware figure of the whole; no reading (Linux,
counter not live) is the machine-wide figure as a cargo window's. JCR-1
DeviceEnergy and JouleContract EnergyClass carry no coverage, so a partial
window grades the run down, never up. RunReport::l3 says what was read and
why it is partial.

Tested against an oracle sharing no code: a stand-in model (this test binary)
reports its own ri_energy_nj from inside; the window read 0.999 of it, twice.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

e2e4394eae

• View at this point in history

2026-10-10 22:24:01 -04:00