eoc-cascade: a hit reads no meter and is charged a calibrated estimate; the span opens at the neural stage #233

Merged

dcharlot merged 4 commits from feat/eoc-cascade-meter-at-its-resolution into main 2026-10-13 09:31:02 -04:00

Conversation 0 Commits 4 Files changed 10 +2217 -80

dcharlot commented 2026-10-13 02:27:32 -04:00

Owner

Copy link

Why

The eoc cascade metered every resolve with two reads of this process's energy counter. Since #229, each read first verifies that the reading thread has left its core. A resolve answered by the cache, the KV tier or the graph is about a microsecond and a microjoule of work, but the span around it read the meter's own closing wake-up instead.

• Before, on the M3 Ultra: 10⁴ cache-hit resolves took 1.64 s, a median of 162 µs each. Every one was tagged Measured at 13–28 µJ, which is the meter's own cost, not the hit's.

The measurement cost more than the work it measured, and the figure it sealed was the meter's.

What

The rule stands: nothing that was not read from a counter is Measured.

• A deterministic hit reads no meter.
  • The figure. It is charged a per-hit figure calibrated on this host and tagged Estimated, since nothing was read for this resolve.
  • Calibration. The first time a tier answers while the cascade holds a live counter, that same lookup repeats back to back on this thread inside one verified span. The batch grows until the span reads well over the meter's own cost, measured as an empty span, within a wall-time cap. Each tier is calibrated once, can be re-calibrated, and its calibration can be read.
  • Integer conservation. A per-tier nanojoule carry makes the µJ charged over any run of hits sum to what those hits accrued, within the one µJ still carried. A hit is never rounded up, and a sub-µJ hit never reads as a run of zeros.
  • No live counter. A hit stays the estimated zero it was.
• The meter span opens only when the walk reaches the neural stage. The opening read is taken after cache, KV and graph have all missed, and the closing read after the neural stage. The misses (about 1 µJ, 220 ns) are spent before the span and are not in it. The span carries the meter's own wake-up, so a measured neural figure is never under the neural stage's energy.
• Rule text. The last clause of f84f0b2f's rule, which made a cache hit on a live span measured, is replaced in the code docs, the crate docs, the README and eoc-meter's detect_in_process docs.
• The eoc CLI prints a hit's estimate and the calibration it came from.

Evidence (M3 Ultra)

After the change:

Tier	10⁴ resolves take	Median per resolve	Calibrated figure per hit	Measured?
cache	4.2 ms	0.38 µs	~1.56 µJ (1,564 nJ)	0 of 10,000
kv	7.2 ms	0.67 µs	2.7–3.3 µJ, plus the stage's own 50 µJ	0
graph	8.1 ms	0.79 µs	3.1–3.2 µJ, plus the stage's own 500 µJ	0

• Calibration cost: about 1–1.4 ms per tier, once. A batch of 640–1,064 lookups reads 90–210 times an empty span's 13–15 µJ.
• Neural resolves: still measured, at two verified reads (about 163 µs).

Tests:

• Injected counters:
  • a hit takes zero reads after calibration;
  • a neural resolve takes exactly two;
  • integer conservation holds for per-hit figures below and above 1 µJ, and across many threads;
  • with no live counter, hits are the estimated zero;
  • a hit is never Measured.
• Real counter, against an oracle: the calibrated per-hit figure × batch equals an independent folded read around the same batch, and a neural span equals an oracle around the neural stage. CPU burned before the walk reaches neural is shown not to be in the figure.
• Mutation checks:
  • hits charged Measured: 11 tests fail;
  • span opened before the walk: 9 fail;
  • carry dropped: 5 fail.
• Suites: the eoc-rs workspace passes, 739 tests (up from 723). The Linux and wasm32 checks are clean. Clippy reports nothing new in eoc and eoc-cascade.
• Vectors: no eoc conformance vector or pinned receipt depends on a hit's grade.

Two findings, now in the docs:

• In a debug build, a cache hit costs about 22 µJ, because unoptimized receipt minting is about as costly as the meter. Optimized, it is 1.2–1.7 µJ.
• A calibration that lands on an efficiency core reads roughly a quarter of a performance core's per-hit figure.

🤖 Generated with Claude Code

## Why The eoc cascade metered every resolve with two reads of this process's energy counter. Since #229, each read first verifies that the reading thread has left its core. A resolve answered by the cache, the KV tier or the graph is about a microsecond and a microjoule of work, but the span around it read the meter's own closing wake-up instead. - **Before, on the M3 Ultra:** 10⁴ cache-hit resolves took 1.64 s, a median of 162 µs each. Every one was tagged `Measured` at 13–28 µJ, which is the meter's own cost, not the hit's. The measurement cost more than the work it measured, and the figure it sealed was the meter's. ## What The rule stands: nothing that was not read from a counter is `Measured`. - **A deterministic hit reads no meter.** - **The figure.** It is charged a per-hit figure calibrated on this host and tagged `Estimated`, since nothing was read for this resolve. - **Calibration.** The first time a tier answers while the cascade holds a live counter, that same lookup repeats back to back on this thread inside one verified span. The batch grows until the span reads well over the meter's own cost, measured as an empty span, within a wall-time cap. Each tier is calibrated once, can be re-calibrated, and its calibration can be read. - **Integer conservation.** A per-tier nanojoule carry makes the µJ charged over any run of hits sum to what those hits accrued, within the one µJ still carried. A hit is never rounded up, and a sub-µJ hit never reads as a run of zeros. - **No live counter.** A hit stays the estimated zero it was. - **The meter span opens only when the walk reaches the neural stage.** The opening read is taken after cache, KV and graph have all missed, and the closing read after the neural stage. The misses (about 1 µJ, 220 ns) are spent before the span and are not in it. The span carries the meter's own wake-up, so a measured neural figure is never under the neural stage's energy. - **Rule text.** The last clause of f84f0b2f's rule, which made a cache hit on a live span measured, is replaced in the code docs, the crate docs, the README and eoc-meter's `detect_in_process` docs. - **The `eoc` CLI** prints a hit's estimate and the calibration it came from. ## Evidence (M3 Ultra) **After the change:** | Tier | 10⁴ resolves take | Median per resolve | Calibrated figure per hit | `Measured`? | |---|---|---|---|---| | cache | 4.2 ms | 0.38 µs | ~1.56 µJ (1,564 nJ) | 0 of 10,000 | | kv | 7.2 ms | 0.67 µs | 2.7–3.3 µJ, plus the stage's own 50 µJ | 0 | | graph | 8.1 ms | 0.79 µs | 3.1–3.2 µJ, plus the stage's own 500 µJ | 0 | - **Calibration cost:** about 1–1.4 ms per tier, once. A batch of 640–1,064 lookups reads 90–210 times an empty span's 13–15 µJ. - **Neural resolves:** still measured, at two verified reads (about 163 µs). **Tests:** - **Injected counters:** - a hit takes zero reads after calibration; - a neural resolve takes exactly two; - integer conservation holds for per-hit figures below and above 1 µJ, and across many threads; - with no live counter, hits are the estimated zero; - a hit is never `Measured`. - **Real counter, against an oracle:** the calibrated per-hit figure × batch equals an independent folded read around the same batch, and a neural span equals an oracle around the neural stage. CPU burned before the walk reaches neural is shown not to be in the figure. - **Mutation checks:** - hits charged `Measured`: 11 tests fail; - span opened before the walk: 9 fail; - carry dropped: 5 fail. - **Suites:** the eoc-rs workspace passes, 739 tests (up from 723). The Linux and wasm32 checks are clean. Clippy reports nothing new in eoc and eoc-cascade. - **Vectors:** no eoc conformance vector or pinned receipt depends on a hit's grade. **Two findings, now in the docs:** - In a debug build, a cache hit costs about 22 µJ, because unoptimized receipt minting is about as costly as the meter. Optimized, it is 1.2–1.7 µJ. - A calibration that lands on an efficiency core reads roughly a quarter of a performance core's per-hit figure. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dcharlot added 4 commits 2026-10-13 02:27:32 -04:00

eoc-cascade: a hit reads no meter and is charged a calibrated estimate; the span opens at the neural stage ... 2add533104

`Cascade::resolve` took one read of its meter before the walk and one after,
whatever answered. Since 0f6a9a3b a read of this process's own energy counter
first sees the reading thread leave its core, so it costs about 80 µs, and a
span holds its closing read's own wake-up. A cache, KV or graph hit is well
under a microsecond. So a resolve one of them answered paid two reads to
bracket a lookup three hundred times shorter than they are, and what it then
sealed as the hit's cost, tagged measured (f84f0b2f's last case: an estimated
zero on a live span is the span's reading), was the meter's own.

Measured on the build host (M3 Ultra, macOS 27.0.1, load average 11 to 19,
`examples/hit_cost`, release), parent ac93d9e5 against this commit, 10,000
resolves answered by the cache:

- before: 1,647 ms in all, 164 µs a resolve at the median; every one tagged
  measured, 13 to 28 µJ, 141,447 µJ in all. A neural stage that does nothing
  read the same: 164 µs and 15 to 20 µJ.
- after: 4.4 ms in all, 0.42 µs a resolve at the median; none tagged measured,
  16,650 µJ in all, which is the 1,665 nJ a hit the tier was calibrated at.

The rule is unchanged: nothing that was not read from a counter is measured.
What changes is where the meter is read.

- A resolve the neural stage answers is read as before, but the span opens
  once the three deterministic tiers have missed. The misses are spent before
  it and are not in the figure: 0.95 µJ and 223 ns for the three together,
  against the 13 to 24 µJ the span holds of the meter itself. So the measured
  figure is still not under the neural stage's energy. `cascade_cost` and
  `live_span` are as they were. A failed opening read is no longer followed
  by a closing one, which could not make a span.
- A resolve a deterministic tier answers reads no meter and is never
  measured. It is charged the cost its stage reported plus a figure
  calibrated on this host: the first time a tier answers, the cascade reads
  the counter over no work until it moves (the meter's own cost), then
  repeats that tier's lookup inside one span, growing the batch until the
  span reads fifty times that (and 200 µJ at least) or 40 ms have gone, and
  keeps the span's reading per lookup, in nanojoules. It is an estimate: read
  by this counter on this host, but over a batch, and a mean is not a reading
  of any one hit.
- Per-hit figures are around a microjoule and `JouleCost` is whole
  microjoules, so each tier carries the nanojoules not yet charged. The
  microjoules charged over any run of hits are the whole microjoules in what
  those hits accrued: never each hit rounded up, never a string of zeros.
- The stage's own figure is added, as `cascade_cost` adds an estimate to a
  span and for its reason: it may be energy the counter never saw. A cached
  response that carries a measured figure is charged as an estimate too: it
  replays what was read for an earlier resolve.
- No live counter, no calibration: the cascade given no meter (nothing is
  read to find the stub does not move), a failed read, a counter that holds
  for 2 ms of back-to-back reads or over every batch. The hit then keeps its
  stage's cost. The attempt is made once a tier; later hits read nothing.
- `Cascade::calibration` reads what a tier holds (per-hit nJ, batch, span µJ,
  empty-span µJ, time taken) or why it holds nothing; `Cascade::calibrate`
  calibrates the tier that answers a query now, again if asked, and keeps
  the carry; `without_hit_calibration` switches a tier off, for a KV backend
  whose `get` must not be repeated. The three stages here are safe to
  repeat: the cache's lookup only refreshes a recency the hit already
  refreshed, the KV and graph lookups only read.

Calibrated on that host: 1,356 to 1,665 nJ a cache hit, 2,669 to 3,250 nJ a
KV hit, 2,946 to 3,152 nJ a graph hit, each from 224 to 4,032 lookups in a
span of 0.7 to 5.9 mJ, 54 to 347 times the 12 to 29 µJ an empty span read,
in 0.85 to 2.4 ms.

Tests, on scripted counters and on a model host whose meter, clock and KV
backend are one count: a hit takes no read once its tier is calibrated (not
one in a thousand hits, each tier) and a neural resolve exactly two, the
first after the KV lookup; the microjoules charged over 10,000 hits equal
the calibrated nanojoules at every hit, for 139 nJ and 2,359 nJ a hit, and
across eight threads; a calibration's batch is the lookups between its last
two reads and its span their delta; a 1 ms lookup is repeated 39 times and a
100 ms one once; with no live counter every resolve keeps its stage's cost;
and no hit is measured, on any counter, the calibrating hit included.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

eoc-cascade: a hit's calibrated figure and a neural resolve's span, against an oracle on the real counter ... bd865deb9a

0d78becb's rules were tested on scripted counters. These two hold them
against this process's own CPU energy counter, each in a test binary of its
own with one test, as eoc-meter's are: the counter's domain is every thread
of the process, and the oracle's check is the process's context-switch count.
Gated the same way: macOS only, and the counter must be live.

The oracle (`tests/oracle`) shares no call and no number with the counter:
`proc_pid_rusage` declared there and read at the SDK's offset, taken once the
test's thread has left its core by the process's context-switch count rising
across a sleep. A `Watched` counter puts that read on either side of every
read the cascade takes, so a span lies between two windows of the oracle's:
the one inside it and the one around it and both reads. That a reading lies
between them holds for any count that only grows, so it is asserted every
time; what is allowed to miss a tenth of the time is how far apart the two
windows are, which is the two readers' own reads and wake-ups.

`hit_calibration_process_energy`:
- each tier calibrated 20 times through the watched counter: the span the
  calibration kept is its last two reads, it lies between the oracle's two
  windows around the same batch, the per-hit figure times the batch is that
  span, and the empty span it reports is the counter's first advance over no
  work. The inner window was 952/1000 of the outer at least, quiet and under
  40 busy loops, so the figure is pinned within a twentieth; a fifth is
  asked.
- on the bare counter, after a tier's first hit, 10,000 hits take no read of
  the meter (counted), every one is an estimate, and the microjoules charged
  are the calibrated nanojoules to the one microjoule still carried.
- what the change removes: a span around one hit, two reads about one
  lookup, reads more than the hit is charged, every time; in an optimized
  build 5.8 to 17 times as much at the median, and three times is asked. In an
  unoptimized build the lookup itself costs about what the meter does (22 µJ
  a cache hit against 1.4 optimized), so no multiple is asked there.

`neural_span_process_energy`, with a KV stage that burns 8 ms of the thread's
CPU on its way to missing:
- a neural stage that burns 2 ms: exactly two reads, the first after the KV
  lookup, and the measured figure between the oracle's window inside the span
  and its window around it, which opens after the burn. The inner window was
  785/1000 of the outer at least under load (992 quiet); two thirds is asked.
- a neural stage that only sleeps 2 ms, on the bare counter, nothing taking
  the thread off its core between the burn and the cascade's first read: the
  figure is under a quarter of the oracle's window over the whole resolve (at
  most 32/1000 of it). A span opened before the walk reads the burn.

No joule figure and no timing is asserted. A calibration that lands on an
efficiency core reads a fifth of what one on a performance core does (253 nJ
a cache hit against 1,400): the figure follows the core the batch ran on.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

eoc: a hit prints its estimate and the calibration it came from ... 3d941db705

With 0d78becb a prompt the cache, the KV store or the graph answers is no
longer bracketed by two reads of the meter: it is charged the stage's own
figure plus what a hit on that tier costs on this host, calibrated over a
batch the first time the tier answers. The CLI resolves one prompt a process,
so its hit is always the one that calibrates.

`eoc query` therefore prints, for a hit, an estimate (it printed
`measured(the meter's own cost)` for a cache hit on a live counter, and the
stage's estimate plus that cost for a KV or graph hit), and a `hit` line that
says where the figure came from: the per-hit nanojoules, the batch, what the
meter read over it and over no work; or why the tier has no calibration and
the cost is the stage's own figure. On the build host (M3 Ultra, release):

    $ eoc query ping
    stage  : kv
    payload: pong
    cost   : 53 µJ (estimated)
    hit    : a kv hit is charged 3357 nJ on this host, on top of the stage's
             own figure: process-energy read 6876 µJ over 2048 lookups in one
             span (24 µJ over none)

A prompt that falls through to the neural stage prints as before: the echo
backend's 50 J estimate plus what the counter read around the stage, an
estimate; measured only where the stage estimated nothing, which the
existing in-process test pins. `eoc meter`'s `query` line says how the meter
is used.

Tests: the CLI's own path for `ping`, a KV hit and then a cache hit, is an
estimate on every host; where the tier was calibrated the cost is the stage's
figure plus the whole microjoules of the calibrated one, and where it was
not, the stage's figure and a reason this host's meter can give (the stub
holds, the powermetrics stand-in fails its reads). It takes the lock the
burning tests take, since the meter's domain is the whole process. The `hit`
line is pinned on its own.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

eoc docs: what a resolve is charged, with the measured figures; the example prints the unmetered baseline ... 23eacd79a5

The README said only that joule cost is the unit of accounting. It now has
the rule and where the cascade reads its meter since 0d78becb: around the
neural stage, never around a hit, which is charged a per-hit figure
calibrated on the host. The example output shows the `hit` line, with and
without a live counter.

eoc-cascade's own docs carry the final figures from the build host (M3
Ultra, macOS 27.0.1, release, load average 5 to 14, three runs of
`examples/hit_cost`):

- 10,000 cache hits: 4.18 to 4.20 ms, 0.375 µs a resolve at the median
  (parent: 1,641 to 1,651 ms, 162 to 164 µs). A cascade given no meter at
  all: 3.55 to 3.60 ms, 0.333 µs. KV 0.667 µs, graph 0.75 to 0.79 µs.
- calibrated: 1,413 to 1,716 nJ a cache hit, 2,672 to 2,961 nJ a KV hit,
  3,007 to 3,163 nJ a graph hit, each from 416 to 3,328 lookups in a span
  51 to 351 times the 12 to 19 µJ an empty span read; 0.85 to 2.3 ms for a
  first hit, calibration included, 0.96 to 1.35 ms for one asked again.
- a neural stage that does nothing: 162.4 to 162.6 µs a resolve and 14.3 to
  14.8 µJ measured, the cascade's two reads; 0.79 to 0.83 µs with no meter.
  The three misses before the opening read: 0.95 µJ and 220 ns together.

`HitCalibration`'s docs say the figure is that of the core the batch ran on
and of the build, and that the calibration's own energy is charged to no
resolve. The module docs no longer link the private `cascade_cost`.

eoc-meter's crate docs and `detect_in_process` say how eoc-cascade uses a
counter now (it was the example of a caller that should bracket a batch, not
each operation). eoc-core's `JouleCost::zero` and eoc-cache said a lookup's
energy is below any counter's resolution; it is about a microjoule, readable
as the mean of a batch, and less than a read costs, which is why no counter
is read for a hit.

`examples/hit_cost` also runs the same stages with no meter, the baseline a
metered hit is now within 0.05 µs of.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot merged commit 54bdab08d7 into main 2026-10-13 09:31:02 -04:00

dcharlot referenced this pull request from a commit 2026-10-13 09:31:03 -04:00

Merge pull request 'eoc-cascade: a hit reads no meter and is charged a calibrated estimate; the span opens at the neural stage' (#233) from feat/eoc-cascade-meter-at-its-resolution into main

dcharlot referenced this pull request from a commit 2026-10-13 22:40:05 -04:00

Merge origin/main (at 1ec8f447, with #233, #234, #236, #238, #239, #240) into wai/d2c-decoders-on-ops

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Transaction-Science/open-standards!233

No description provided.