sandbox-meter: a member waiting on the kernel in the middle of its work is not settled #230

Merged

dcharlot merged 2 commits from fix/tree-energy-reap-under-load into main 2026-10-13 03:13:53 -04:00

Conversation 0 Commits 2 Files changed 5 +346 -50

2026-10-13

Transaction Science

jcp-gateway, proof-solvers: the tree tests' oracles ask each thread's wait state too ...

These two tests hold the tree meter to an oracle of their own, which took a
process for quiet with no thread on a core or queued. The meter now also
waits out a thread in a wait no signal ends, so an oracle without that rule
could read a member mid-work where the meter waits, and the two would
disagree for a reason that is the oracle's. Each oracle now asks every
thread's run state, read raw, as tree_energy's does.

Both pass on the build Mac: upstream_tree 7 of 7, z3_tree 5 of 5.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

c15615c785

• View at this point in history

2026-10-13 02:25:40 -04:00

Transaction Science

sandbox-meter: a member waiting on the kernel in the middle of its work is not settled ...

The tree meter took a member for settled when two reads of its counts
agreed with no thread on a core or queued between them. A thread in a wait
that no signal ends (the kernel's own waits: a block being written, a page
being brought in, a lock) passes that rule: it is off its core and folded.
But it runs again when the kernel is done, with nothing arriving, so the
member is in the middle of its work. A stop taken there read the span
short by the rest of that work, whole, with no gap.

Measured (M4 Max, macOS 27.0.1): a server whose reply is followed by
twenty rounds of a 4 KiB write, an fsync and a short burn read short in 24
of 60 round trips, at 16 to 92% of the oracle's advance, each reading
whole. With the rule below it read exact in 60 of 60, and in every one of
72 more under a load past 160.

The rule: once `pti_numrunning` is zero the meter asks each thread's own
run state (`PROC_PIDLISTTHREADS`, then `PROC_PIDTHREADINFO`), and a member
with a thread in `TH_STATE_UNINTERRUPTIBLE` is not settled; it is walked
again, and is `TreeGap::Unsettled` if it never comes to rest inside the
bound. A thread asleep, or blocked reading a pipe, is not in that state (0
of 20 looks each; a thread in an fsync loop, 18 of 60), and none of an
idle host's threads was at 5 looks, so a server waiting for its next
request settles as before. Cost: the start of a span on a tree of 61
processes took 1.73 to 1.75 ms against 1.68 to 1.72 ms.

JCP SPEC 5.1 states the requirement, and names the second cause beside
"still running" in the gap it already has.

Tests, two of which failed on a loaded host and are why this was found:

- `a_worker_reaped_while_its_tree_is_walked_is_never_missed` asserted no
  reading is whole. One round trip of 21 in a CI run read the worker alive
  and off every core, counted its record, and was whole: the worker was
  read, not missed. The test now asserts its name: a whole reading holds
  the worker as a member.
- `a_server_that_reaps_its_worker_is_not_whole` failed its equality with
  the oracle once at a load past 100 on a host short of memory. That is
  what a member read while it waited for a page looks like; the run's
  figures were not kept, so this is the likely cause and not a shown one.
- New: `a_reply_whose_tail_waits_on_the_disk_is_read_once_the_tail_is_done`,
  which fails against the earlier rule on an idle host.
- The oracle's own quiet rule asks the same of each thread, read raw.

Verified on the build Mac: sandbox-meter's tests idle and twelve full runs
under load, and the process-tree tests of jcp-gateway, proof-solvers,
sandbox-proof and joule-code-remediate.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

80e254933c

• View at this point in history

2026-10-13 01:03:29 -04:00