sandbox-meter + jcp-gateway: count, and flag, work services did on the tree's behalf #214

Merged

dcharlot merged 14 commits from feat/tree-delegated-energy into main 2026-10-11 05:08:35 -04:00

Conversation 0 Commits 14 Files changed 12 +1853 -129

2026-10-10

Transaction Science

sandbox-meter: the oracle's switch count needs no conversion ...

clippy flagged the i64::from calls in the process-energy test's switch
count as useless (ru_nvcsw is already a c_long, i64 here): two new
warnings, now gone. The count is summed as the c_long it is.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

e838070cdf

• View at this point in history

2026-10-10 23:05:45 -04:00

Transaction Science

sandbox-meter: test the wiring of the fold check and the stop's bill wait ...

The fold rule and the bill-hold rule were tested as pure functions, but
nothing failed if the probe's snapshot stopped calling the fold check, or
the tree meter's stop stopped calling the bill wait. The probe now holds
its thread-count reader (production: this thread's own energy count), so
a test gives it one that never moves and sees the read fail after every
fold sleep; and TreeMeter::stop is stop_with over its reads, so a test
sees a bill that landed after the stop walk reach the reading, and no
read or sleep when no bill moved. The integration test's assertion that a
delegating stop took at least one poll is dropped: a stop takes that long
anyway, so it could not fail; the stop times are still logged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

f5b8765a2a

• View at this point in history

2026-10-10 23:05:45 -04:00

Transaction Science

jcp: the in-process rung says it does not see work handed to a service ...

SPEC §5.1 rung 3 graded the per-process count `Whole` "for work that runs
on those threads", and detect_in_process's docs listed what the count
leaves out (children, GPU, Neural Engine) but not work the process's
threads hand to a system service: the service does it on its own
threads, and the kernel bills it to this process beside the count
(ri_billed_energy), never in it — so a span over work that calls a
service read as an HwShunt record with nothing said missing.

Rung 3 now says such a span is not Whole for that work, that the
reference meter does not read the bill on this rung and so does not
detect it, and that an implementation SHOULD meter such work on another
rung or grade it Partial; detect_in_process, jcp-gateway's scan_meter and
the sandbox README say the same, and that a pure computation over the
process's own memory (the flow scan, joule-meter's workload) is
unaffected. Detection is left out: the Measurement record has no coverage
field to carry it, and no caller of this meter delegates.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

9eeb7d5262

• View at this point in history

2026-10-10 23:05:44 -04:00

Transaction Science

sandbox-meter: the billed-count wait's docs link no private item ...

Three intra-doc links added with the stop's wait for the billed counts
pointed public docs (BILLED_HOLD_POLLS, TreeMeter::stop, the tree module)
at private items (`settled`, `hold_bills`), each a new cargo doc warning.
They are plain code spans now, as the module's other mentions of
`settled` are.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

0c826d2784

• View at this point in history

2026-10-10 23:05:44 -04:00

Transaction Science

sandbox-meter: cite a directory-lookup bill that was measured ...

The delegated-work docs gave "under 1 mJ for a directory-service lookup
that cost it 0.06 mJ of its own", a figure in no evidence file (the meter
sweep has no directory-service case). They now give what was measured on
this branch: 0.83-1.46 mJ billed for three getpwnam lookups against
0.57-1.17 mJ of the client's own per round trip, over 10 round trips of
the tree-energy test and a probe like it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

9b3186c338

• View at this point in history

2026-10-10 23:05:44 -04:00

Transaction Science

sandbox-meter: a delegated figure can be over as well as short; the stop waits for bills ...

A bill is credited when the servicing thread next changes voucher, so a
late credit for an earlier call's work lands in whichever span is open and
is added to that span's figure — tree.rs said so — yet the TreeGap, module,
gateway, SPEC and README docs called the bill a lower bound and every gap
but unsettled-at-start a shortfall. `delegated` is now described, beside
unsettled-at-start and serviced, as able to put the figure over the span's
energy (pre-span bills) as well as under it (unbilled work, the services'
GPU and Neural Engine energy, credits after the stop).

The stop also waits, when a counted member's billed counts moved over the
span, for them to hold: it re-reads every member's billed counts every
5 ms (BILLED_HOLD_POLL) until two reads agree, at most 20 times
(BILLED_HOLD_POLLS), and the reading takes the last read, so a call's own
credit landing just after its reply falls in its own span. It is separate
from settling (after the stop walk has settled; it reads only the billed
fields). Measured: it costs a delegating call one poll — stop median
9.6 ms against 2.7 ms without it, three directory lookups, 40 calls each —
and a pure-CPU call nothing. Its benefit was small on the services tried:
of 40 directory-scoped metadata searches, 5 had bills land after the reply
(one 0.40 J within 5 ms, the rest 0.03-0.44 mJ from 20 ms to 1.5 s), and
metered, 3 of 30 calls had a bill land after the stop without the wait and
2 of 30 with it. The docs say what it catches and what it cannot.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

d5d2de149a

• View at this point in history

2026-10-10 23:05:44 -04:00

Transaction Science

sandbox-meter: service a member did for others is a gap whenever it moved ...

The tree figure subtracts each member's serviced credit (work it did for
other processes, part of its own count) and raised a gap only when a
member's net went below zero; a positive net read Whole. But a servicing
thread is credited at its next voucher change, not as it works — a
daemon's serviced energy was 14-100% of its own energy over a window in
which its serviced CPU time was 93-99% — so a credit inside a span need
not be that span's service: one for work before the span subtracts energy
the span never held (short), and work in the span credited after the
stop stays in (over). A reading graded Whole could be off either way.

TreeGap::Serviced { serviced_nj, serviced_cpu } is now raised whenever a
counted member's serviced energy or CPU time moved over the span, however
the net came out; its docs, the TreeGap and module docs, SPEC §5.1 and
the sandbox README name it, beside unsettled-at-start, as a gap that can
put the figure OVER the span's energy as well as under. jcp-gateway names
it `serviced` (the compiler now insists every kind has a name), and its
docs say a `serviced` figure may be over. The unit test that asserted a
serviced member reads whole now asserts the gap.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

031c0153c1

• View at this point in history

2026-10-10 23:05:44 -04:00

Transaction Science

jcp-gateway: every TreeGap must have a wire name, checked by the compiler ...

gap_kind ended in a catch-all arm that named any kind it did not know
`other-gap`. That kept a new kind Partial, but it dropped the guarantee
that matters on the wire: a meter change adding a kind would compile and
ship a reason that says nothing of what was missing, and the test that
lists the kinds by hand could not notice. With no catch-all, a new
TreeGap variant fails to compile here until it is named. The test stays,
for distinctness.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

5d936dfe3c

• View at this point in history

2026-10-10 23:05:44 -04:00

Transaction Science

sandbox-meter: an in-process read waits until its thread is SEEN to fold ...

A 1 µs sleep before a self-read does not always take the reading thread
off its core: when the sleep's timer fires before the thread has blocked,
the thread goes on unswitched and the read misses everything it spent
since its last fold. The review saw 4 of ~44,000 such sleeps with no
context switch at load 85-118 (each read missing the whole burn), and the
in-process test fail 2 of 140 — once by the meter, once by its own oracle,
whose usleep(1) had the same flaw.

The probe now sleeps 50 µs, then 200 µs, 1 ms and 5 ms, until the reading
thread's own energy count (proc_pidinfo PROC_PIDTHREADCOUNTS) moved across
a sleep — it moves only when the kernel folds the thread — and only then
reads. A thread not seen to fold fails the read, so the span comes back
estimated instead of a short figure graded a reading. The process-wide
switch count (getrusage, pti_csw) was measured and rejected: with two
other threads switching it passed 2 of the 12 sleeps (of 25,000) after
which the reading thread had not folded. The probe is built only where
the thread count reads in the layout the code knows.

The test's oracle now folds its own, independent way: a sleep of 1 ms or
more and the process's switch count checked to have moved, failing loudly
if it never does. Docs (process, detect_in_process, SPEC §5.1 rung 3, the
sandbox README, jcp-gateway's scan_meter) no longer say the thread always
folds or that the figure is "never under"; they say the fold is checked
and what a failed check gives.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

b44170872e

• View at this point in history

2026-10-10 23:05:44 -04:00

Transaction Science

jcp: SPEC §5.1 rung 3 and the sandbox README say what delegated work a tree figure holds ...

The process-tree part of the per-process rung now says how a platform's
billing of delegated work enters a tree's figure: each member's own energy,
less its serviced energy, plus its billed energy, summed signed (service
between members counted once, through the payer; a negative member net a
gap, never wrapped); that the bill is a lower bound (no GPU/Neural Engine
energy in the service, no unbilled threads, no late credits, nothing billed
to another originator, nothing over a socket or off the host); that any
bill makes the figure Partial, named `delegated`; that billing takes no part
in settling; and that the operator's declaration does not outweigh a bill.

The in-process paragraph gains two measured facts the reference meter now
acts on: a process's read of itself does not fold the reading thread, so the
reading thread SHOULD leave its core just before each read (over the work's
energy, never under it, once it does); and a computation known to run on the
metering thread alone, touching no other process, is not made Partial by a
child the process happens to have (the gateway's flow scan, whose host
always has the upstream server as a child).

The README's meter paragraph says the same in brief, and that an exited
child's own figure is partial (an exit does not fold its last time on a
core).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

ea85fd8f84

• View at this point in history

2026-10-10 23:05:44 -04:00

Transaction Science

jcp-gateway: a delegating server is partial even when declared; the flow scan is metered in-process ...

Two changes to how a call's energy is graded and where the monitor's own
energy is read.

1. The tree meter now counts the CPU work a system service was billed to
   the upstream tree for, and lists it as TreeGap::Delegated. gap_kind
   names it `delegated` (and `serviced-beyond-spent` for the signed-net
   gap), so energy_partial_because carries it and the figure, which holds
   the bill, is Partial. That holds EVEN WHEN the operator declared the
   server self-contained: a bill is evidence against the declaration for
   that call, and the bill is a lower bound (no service GPU/ANE energy, no
   unbilled threads, no late credits). The declaration still gates Whole
   where no bill arrived, because billing misses socket/pipe servers,
   other originators and remote APIs. declare_upstream_self_contained and
   the binary's docs say so. gap_kind gains a wildcard (`other-gap`) so a
   gap kind added to the meter by another branch still reaches the wire
   and keeps the figure Partial instead of breaking the build.

2. Phase 0, the flow-label scan sealed as FlowFacts eval_uj/eval_prov, is
   a computation on this thread over this process's memory, so it is now
   metered with PowerMeter::detect_in_process() (scan_meter) instead of the
   machine meter, which stays the calls' fallback. On this fleet the
   machine meter only estimates (IOReport CPU rail frozen); the scan now
   reads HwShunt with its own figure (about 0.4 mJ per scan in a debug
   build). In a multithreaded host the span also counts the other threads'
   work: an over-count, never an under-count, now that the meter folds the
   reading thread before each read. A count that did not advance is still
   the estimator's figure, never a measured zero.

Tests: grading shapes with Delegated (declared and not) and
ServicedBeyondSpent; every gap kind named, none `other-gap`; the scan meter
is the process-energy probe where that count is live and every tainted
call's flow block is HwShunt > 0 (or Estimator where not live); and end to
end, a C server compiled at test time whose calls do directory-service
lookups: declared, its delegating calls are sealed HwShunt Partial with
["delegated"], cost >= the raw-buffer oracle's own - serviced + billed, and
its non-delegating calls before them are Whole and exact to the uJ.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

ab0ed7a1f4

• View at this point in history

2026-10-10 23:05:44 -04:00

Transaction Science

sandbox-meter: joule-meter meters its own threads' workload in-process ...

The demo's busy span runs on threads of its own process (one per core), so
the machine-wide meter read everything the host did beside it, and on this
fleet (IOReport CPU rail frozen) only estimated it. It now uses
detect_in_process(): where this process's CPU energy count is live, the
figure is the workload's own threads' CPU energy and the receipt seals
that; elsewhere it is detect(), as before, with the machine-wide wording.
The idle span uses the same meter, reading this process asleep (30 uJ over
0.5 s here) against the busy span's 4.66 J. Worker threads exit before the
stop read, which folds their energy (measured: a thread's exit in a live
process folds; the process's own exit does not).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

6ae2941f92

• View at this point in history

2026-10-10 23:05:44 -04:00

Transaction Science

sandbox-meter: fold the reading thread before an in-process read; an exited child's figure is partial ...

Two kernel facts the per-process figures relied on without checking, both
measured on this M5 Max (macOS 27.0.1):

1. A process's read of ITSELF does not fold the reading thread's energy
   (its CPU time is current, its energy is folded only when it leaves a
   core). A thread burning 1 ms between two self-reads read the same
   energy at both in 28-29 of 40, and up to 9.4 mJ (a quantum spent mostly
   before the first read) in others. So detect_in_process() read a short
   span as nothing (estimated) or as energy from before it (sealed
   HwShunt). ProcessEnergyProbe now sleeps 1 us before each read: the
   blocked thread leaves its core and is folded. Read so, the same burns
   gave 0.93-1.41 mJ in 80 of 80. Work on the thread that opens and closes
   a span is read whole; other threads add what they spent, so a span in a
   multithreaded process is over its work's energy, never under it. The
   liveness probe still reads raw.

2. A process's exit folds its CPU time into its record but not the energy
   of its threads' last time on a core. Paired 2 ms burns ended by exiting
   vs by blocking (equal CPU time) read 0-1.12 vs 0.81-1.20 mJ per CPU-ms,
   under 0.05 in 11 of 40 exits; 60 ms burns lose within the spread. A
   thread's exit in a live process does fold. tree.rs already grades an
   exited member a gap (TreeGap::Exited), which this confirms; but
   child_reading / Attribution::child_energy_uj called the same record a
   full HwShunt reading. Its docs now grade it Partial, a lower bound, and
   say how a sealer must label it. The value and the rule are unchanged
   (no in-repo consumer seals it; joule-code-remediate is untouched).

Test: a 1 ms in-process span, 20 times, is a Cpu HwShunt reading every time
and within 1 uJ above / a quarter below an oracle that folds and reads raw
around it (it read 98-99%). Removing the fold makes most spans estimates.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

df24b46775

• View at this point in history

2026-10-10 23:05:43 -04:00

Transaction Science

sandbox-meter: the tree meter counts work services did on its behalf, and flags it ...

The kernel's bank ledgers bill CPU work a system service does on a
process's behalf to that process (ri_billed_energy / ri_billed_system_time),
and credit the servicer (ri_serviced_*). Measured on this M5 Max: exactly 0
in every CPU-only control, 4-45x a client's own energy for on-device model
inference, ~200x for a metadata search, ~2.4x for speech. The tree meter
read none of it, so a server that delegated its work read a plausible own
share with nothing to show the rest.

Each member's count is now ri_energy_nj - ri_serviced_energy +
ri_billed_energy, its advances summed signed: work a member did for others
is theirs, work others did for it is the tree's, and service between two
members is counted once, through the payer. A member whose net goes below
zero (a credit for earlier service landing in the span) is counted as it is
and is a gap, TreeGap::ServicedBeyondSpent, never a wrapped u64; a sum not
above zero is no reading. Any billed energy or CPU time in the span is
TreeGap::Delegated { billed_nj, billed_cpu }: the bill is IN the figure and
a lower bound of the delegated work (no GPU or Neural Engine energy, no
unbilled threads, no late credits, nothing billed to another originator).

The billed counts take no part in settling or in confirming a walk: another
process's thread credits them, whatever the member is doing, up to a second
and more late, so waiting on them would stretch every span without making a
figure whole. The serviced counts stay in settling: the member's own thread
moves them. A CPU-only tree's figure is unchanged (its ledgers are 0).

Tests: the arithmetic (billed added, serviced subtracted, intra-tree service
counted once, a negative member net a gap and summed signed, a born member's
whole ledgers, a ledger that went back, billed CPU alone), settling with a
bill landing between two reads, the rusage layout at 216/224/264/272, and a
real server compiled at test time whose directory-service lookups are
billed to it: Delegated fires, its bill is at least the raw-buffer oracle's,
and the figure less the bill is the server's own advance to the nJ; its
pure-CPU control and the awk control read whole and exact.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

ba3654f0df

• View at this point in history

2026-10-10 23:05:43 -04:00