Timing under load: the GPU span test, co-watch re-entry, and the SRT link tests #614

Merged

dcharlot merged 22 commits from test/timing-under-load into main 2026-11-04 13:12:48 -05:00

Conversation 0 Commits 22 Files changed 23 +2371 -298

2026-11-04

Transaction Science

joule-contrib: the SRT link tests again require every recovered packet taken before its delivery time; a recovery is pinned to when its gap was found, and a held packet keeps its margin at the link's end ...

The loss test's last assertion had become `margin_min <= window`, which the
receiver's clamp and the verifier's rule already guarantee: the end-to-end test
checked no timing at all. It again asserts `margin_min > 0`, the only check
across the real link (the sender's NAK handling, the connection's loop, the
proxy) that every recovered packet was taken before its own delivery time. The
ordering defect fixed in 066d82edf delivered a retransmission taken after its
drop point as recovered with a margin of 0, with nothing dropped and every
payload in order: exactly CI's failure at the old line 148. That failure fits
the defect, a busy runner's lag, or both (with the starved timer fixed in
76264fae6 lengthening the first window's recoveries); the part was not printed,
so it cannot be told which, and the earlier commit's "the bound measured the
host, not the protocol" says more than the evidence does. The window stays a
second, which absorbs the receiving task's lag seen under load. The recovery
time stays unbounded there: it counts from when the gap was found, not from the
packet's timestamp (live-contribution §6.3).

What pins where a recovery counts from is now the receivers' unit tests: they
find each gap at 20 ms rather than at the clock's origin, so a recovery counted
from 0, or from the last tick, fails them. The loss test also asserts what does
not depend on the host: every recovered packet came back as a retransmission
(the proxy neither delays nor reorders), and every NAK names a sequence number.

The caller that receives takes its time base from the listener's CONCLUSION
reply, wiring no unit test reaches. Its lossy test now checks its window, its
account (nothing dropped, a margin above 0), and a new test holds one packet
for the caller's window as the listener's test does.

A recovered packet still held when a link ends, SRT or RIST, was recorded with
a margin of 0, which reads as one taken after its delivery time; it now keeps
the time it arrived before that. A unit test for each.

A receiver unit test feeds ACKACKs through the connection's loop on a clock of
its own and pins the periodic NAK after the RTT is measured: every
(RTT + 4 x RTTVar) / 2, 30 ms at a 60 ms round trip, and the 20 ms floor at
1 ms, until the drop point. The comment on the timer test now says the periodic
NAK counts from the link's start; the NAK sent at once does not restart it.

live-contribution: §5.3 and §6.8 said a recovered packet arrives on time, which
under §6.1 does not hold for SRT: one taken after its own delivery time, before
the first packet after it that has arrived is due, is delivered late, counted
recovered with a margin of 0, and the record does not count how many. They now
say so, and §6.8 that the record does not say when a packet delivered at the
first try was taken. §4.1 and §6.1 name the drop point as the delivery time of
the first packet after it that has arrived, since the next one can itself be
missing. The CHANGELOG entry says the same.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

d1b4bd0c01

• View at this point in history

2026-11-04 08:13:40 -05:00

Transaction Science

joule-contrib: an SRT receiver's timer runs every ACK period while datagrams keep coming ...

The connection's loop sleeps until the receiver's next wake, and the
receiver put that 10 ms after the moment it was asked: after each datagram.
While datagrams came closer together than 10 ms, the wake moved on with
each one, and the timer ran first when the first packet held fell due. Until
then, a link's whole first window, it sent no full ACK and no periodic NAK,
so a packet whose first NAK or retransmission was lost was asked for again
only as the window ran out. With the link test's window of a second, under
a load of 32 busy loops on the build host, 2 runs in 30 dropped 4 packets;
3 of them were sent in the stream's first 350 ms.

The timer now runs an ACK period after it last ran. A unit test runs the
connection's loop on a clock of its own, with data every 2 ms and a NAK
lost: the timer runs every 10 ms from the start, the first full ACK goes at
10 ms, and the NAK goes again 150 ms after the first. With the wake counted
from the moment asked, the timer first runs at 300 ms and the test fails.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

aff001ec92

• View at this point in history

2026-11-04 08:13:40 -05:00

Transaction Science

joule-contrib: a packet taken after its drop point is late, whatever the timer did; the SRT link tests stop timing the host ...

losses_each_way_are_recovered_within_the_window failed twice on the
thor-linux-arm64 runner (runs 16749 and 17926) at `part.recovery.max <
300_000 && part.margin_min > 0`, with nothing dropped and every payload
delivered whole and in order. Under induced load on the build host the
receiving task fell up to 384 ms behind its socket, and two cases met that
signature: a recovery of 349 ms with 50 ms to spare (a recovery counts from
when the packet was found missing, the window from its timestamp, which a
late time base puts later), and a margin of 0 (a retransmission that reached
the socket 152 ms early was taken 1 ms after its own delivery time, before
the first packet after it was due, and delivered as live-contribution §4.1
allows). No recovered packet was taken past its drop point in any of the
instrumented runs. The bound measured the host, not the protocol.

The test now gives the link a window of a second on both ends and asserts
what the protocol guarantees: nothing dropped, every lost first send found
and recovered, every payload whole and in order, the part's account, and
margin_min within the window. Its assertions print the part.
a_listener_that_sends_and_a_caller_that_receives, which dropped packets
under the same load with a 250 ms window, gets a second as well.

The reproductions also showed an order defect in both receivers: a datagram
was taken before what had fallen due was resolved, so a retransmission taken
after its drop point with no timer tick between was delivered as recovered
with a margin of 0, and with a tick between was dropped and counted late.
The SRT and RIST receivers now deliver and drop what is due before they take
each packet. Unit tests drive each receiver on a clock of their own: in time
(margin = due - arrival), after its own delivery time but before the drop
point (SRT: delivered, margin 0), at or after the drop point with and
without a tick (dropped and late), and a sweep in which a tick before an
arrival never changes the outcome. Without the reordering four of them fail.

live-contribution §6.1 says when a packet arrives and what becomes of one
taken between its delivery time and its drop point, §6.2 that its margin
counts as 0, and §6.3 that an SRT recovery is not bounded by the window;
the CHANGELOG records it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

208162b526

• View at this point in history

2026-11-04 08:13:40 -05:00

Transaction Science

jouleclaw-energy test: the rail check's 120 W gap cap, measured beside a heavy co-tenant ...

The cap on the rail's advance over the gap between the counter's window and the oracle's now
also runs on shared spans, and had been measured only beside the test's own burner child and an
empty-command-buffer churner. Measured on the build host beside three more burners of the test's
kernel (the test binary in its burner-child mode), so five clients burned at once and the GPU
drew about 49 W across each window: the power over the gap and republish allowance, the quantity
the cap bounds, was at most 19.2 W over three runs (15.4-19.2 W), against 12-17 W beside the child
alone; every run passed, every span shared or refused. That is well below 80% of the cap, so it
stays; the comment above the assertion records the measurement.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

f48b015d32

• View at this point in history

2026-11-04 08:13:11 -05:00

Transaction Science

joule-moq-web e2e: a cowatch control moves from its signed instant only when its page entered again ...

ff08b646c made the cowatch scenario read each control's instant as the room carried it, since a
page the room took for gone signs its controls again, no earlier than its new join plus the lead.
A carried instant other than the signed one was printed as moved, but never tied to that page
having entered again: a room that carried a control at the wrong instant, with no re-entry, would
have passed where the scenario had compared with the signed instant.

`control()` now reads the page's `rejoins` before it signs and after the control is carried, and
checks that the carried instant is the signed one unless that count rose. It reads `rejoins`
rather than `gone`: the page counts `gone` when its post is refused, before it enters again, so a
control signed while the page is already entering again is moved with `gone` unchanged since it
was signed; `rejoins` rises as the page signs again, after any control pushed before. Each of the
four controls adds its line, with both instants and the count.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

cd20a927b3

• View at this point in history

2026-11-04 08:13:11 -05:00

Transaction Science

jouleclaw-energy test: the docs say which spans the registry and the rail check cover ...

MEASURED_ENERGY.md and the test's module doc claimed more than the test does. The registry is
listed before, around and after each span of GPU work the test checks against the oracle or
through the span meter (steps 2, 4b, 4c and 6), not before every span: the idle spans of step 3
and step 5's spans are not bracketed that way. The rail is checked against the test's own read
on the two spans checked against the oracle (the 1.5 s burn and the contended burn) when their
rail was current at both ends, not on every such span. And a client whose time went back is
corroborated by a fall the registry shows inside the span, not merely by being listed. Both now
say so, and that a reading of this process's own GPU time the registry contradicts fails at once.
Step 2's comment on the rail check points at `agrees_with_oracle` rather than every span.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

65f1a0b1b8

• View at this point in history

2026-11-04 08:13:11 -05:00

Transaction Science

jouleclaw-energy test: the contending child outlives the spans it contends, and dies with the test ...

Step 4's burner child burned for at most 60 s, its wall time capped there whatever it was asked,
while steps 4b and 4c now each take spans again for up to 90 s (`BUDGET`) where the old test
allowed six quick tries. A long run of void spans could outlast the child; its exit then read as
a corroborated vanish, the view was no longer shared, and the test failed with messages that
look like a meter defect ("a shared GPU gave Ok(..)", "the child did not contend", "the child's
client is not in the start snapshot") instead of INCONCLUSIVE.

- The child burns for the seconds it is asked, of wall time, and step 4 asks for 2 × `BUDGET` +
  60 s, longer than 4a, 4b and 4c can take between them; step 5 still kills it.
- Right after 4b's and 4c's spans, the test fails `process-gpu INCONCLUSIVE: ... the contending
  child exited (<status>) while its spans were taken` if the child is gone.
- The child is held in `Burning`, which kills and reaps it on drop: std's `Child` does neither,
  and a failed check would otherwise leave a GPU burner on the shared host for the rest of its
  life. A child whose parent is gone (killed outright, dropping nothing) stops burning too.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

a8dc2ffdbb

• View at this point in history

2026-11-04 08:13:11 -05:00

Transaction Science

jouleclaw-energy test: a span meter's figure is held to the hidden-time gate too ...

`agrees_with_oracle` fails a figure of the counter's that is not a sole user's share of the time
the counter saw plus the time `classify` shows was hidden from it (a client made and destroyed
inside the span, a fall where the counter saw an advance). The span meter's spans of steps 4c and
6 had no such gate: a `SpanSource::Process` figure at 99% of what the counter saw passed there even
with GPU time hidden from both snapshots, so the test proved a figure wrong in one place and let
it through in another. `checkable_meter_span` now applies the same gate to the GPU counter's
figure over the meter's own snapshots, which the span is checked to hold. The meter is unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

621390d085

• View at this point in history

2026-11-04 08:13:11 -05:00

Transaction Science

jouleclaw-energy test: a figure whose rail the oracle did not check fails ...

`agrees_with_oracle` checks the rail against the test's own IOReport read only when both
snapshots read it current and their rails match; otherwise it printed "rail not checked" and went
on, and a figure there would have passed with no check of its rail. The meter gives none then
(`NotCurrent`, the rails differing, or one going back, pinned by its unit tests), but the hardware
test did not hold it to that. That branch now asserts that neither the span as taken nor its view
without the vanished clients is a non-zero figure.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

d69f2453d1

• View at this point in history

2026-11-04 08:13:11 -05:00

Transaction Science

jouleclaw-energy test: a span the registry vouched for is logged with its attempt and voids ...

`until_sorted` takes a span again while the registry cannot vouch for it, and a passing run said
nothing of how many were void first: an intermittent misread of another process's client, voided
and retried, left no trace in CI's logs. It now prints `process-gpu: <what>: vouched on attempt
<n>; voids: <kind ×count, ...>` (or `none`) for the span it returns, so a rising rate of void spans,
`OUT_OF_ORDER` above all, shows in the logs and can later be bounded. The INCONCLUSIVE failure
is unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

e9ac6a1787

• View at this point in history

2026-11-04 08:13:11 -05:00

Transaction Science

jouleclaw-energy test: the check that a refusal is the registry's has unit cases ...

`refusal_is_the_registrys` matches the counter's verdict on a span against the registry's
sorting of it: a refusal exactly when the registry vouches for a vanished client, naming the one
the meter's rule picks. On a quiet host no span has a vanished client, so its branches ran only
under churn, and an inverted condition in it would have gone unnoticed until then. It now returns
`Result<(), String>` (the spans' callers fail the test on `Err`, as before), and so does the
named-client check it uses, which step 5 still asserts through `assert_names_a_vanished_client`.

the_span_classifier_sorts_every_client drives it over built snapshots: with two vanished clients
the registry vouches for, a figure, `GpuShared`, a refusal naming the younger one and one naming a
client not at the start are each `Err`, and a refusal naming the oldest is `Ok`; with none, a
refusal is `Err` and a figure or `GpuShared` is `Ok`.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

d765a39173

• View at this point in history

2026-11-04 08:13:11 -05:00

Transaction Science

jouleclaw-energy test: a misread of this process's own GPU time fails at once, not void ...

`classify` voided a span when a counter reading of one of this process's own clients lay outside
the `ioreg` listings either side of it, and the span was taken again. The reason for voiding holds
for another process's client: a queue of it can close and the client accrue again within one gap,
which looks like a misread. It does not hold for this process's own: it runs no GPU work between
the counter's reads and those listings, so a correct read equals them, or one side of a fall of
its own they show. Voiding there meant `assert_eq!(self_ns, o_me)` could no longer see a wrong
own reading in a snapshot: any one lands outside the listings and was voided first, so an
intermittent misread passed on the next attempt.

A reading of this process's own client the registry does not explain, with no fall of it shown
across that reading, is now a contradiction, which fails the test at once, as the exact
own-time check had before the registry sorting. Voids stay for own resets the registry shows:
gone, gone back, fallen inside the span, made and destroyed there, and a reading between the two
sides of a fall of its own across it (two falls in one gap).

the_span_classifier_sorts_every_client: an own under-read at the start by 1 ns, an own over-read
at the end by 1 ns, and a 10% own under-read are each a contradiction naming the read; an own
reading between the sides of a fall of its own is void.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

06371d07c5

• View at this point in history

2026-11-04 08:13:11 -05:00

Transaction Science

joule-moq-web e2e: the cowatch re-entry check waits for the service's count to be the pages' ...

The check ff08b646c added read the pages' counts of being taken for gone either side of the
service's count of `not_present` refusals, and took the first reading in which the two page
readings agreed. The service counts a refusal before the page it answers can count it, so a
refusal landing between the first page reading and the service's could still be counted by the
service alone in a reading the check took: a page taken for gone at that moment would fail the
check although it entered again as designed. The readings are now taken again every 500 ms, for
at most 15 s, until the page readings agree and the service's count is theirs; the check takes
the last reading either way, so a real disagreement still fails with both counts printed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

47fd3bed99

• View at this point in history

2026-11-04 08:13:11 -05:00

Transaction Science

joule-moq-web e2e: the cowatch scenario accepts a page the room took for gone that entered again ...

The macOS browser e2e job failed its cowatch scenario five times with "cowatch: both play from
Ada's control: not within 60000 ms (last: false)" (jobs 67776, 70090, 73183, 76976, 77089). It
is the take-for-gone chain b01681570 records for watch-together, with the stall in Ada's own
entering: `begin()` posted nothing for longer than the room's 4 s silence limit on the loaded
host, the room took her for gone, and the old page, entering again, sent an event signed before
its new join, refused `time_reversed`, and stopped. This branch's page recovers (b01681570,
4cc9a3e1b), but the scenario still failed: its check that the service carried every event the
pages posted required the service to have refused nothing, and the service counts each
`not_present` a page taken for gone gets before it enters again. On the build host under load,
8 of 11 runs of the branch head failed on exactly that check, with {"not_present":1} or
{"not_present":2}, every page having entered again and played.

The check now takes the design at work for what it is. The service's refusals must all be
`not_present`, as many as the pages count themselves taken for gone (`gone`); each page must have
entered again as often (`rejoins`, at most three), had no other event refused (`post_refused`),
and not stopped. The pages' counts are read either side of the service's until the two readings
agree. That check and the closing one print each page's gone, rejoins, resigned and dropped.

Each control's instant is now read from the room's log, as the room carried it: a page that
entered again signs its controls again, no earlier than its new join plus the lead, so the
instant can be later than the one first signed. The waits for each control use it, and the
play's line reports it, with the instant first signed when it moved. Before, a seek moved that
way could pass its wait before it applied, and the position check after it then failed.

The scenario does not hold a page on purpose: the watch-together example does that every run,
and this scenario stays the room as people use it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dc2c331e91

• View at this point in history

2026-11-04 08:13:11 -05:00

Transaction Science

jouleclaw-energy test: a refused GPU span is checked against the registry, not retried ...

The macOS energy job's process_gpu test gave up when a GPU client vanished in six spans in a
row: its checks needed a span in which no client of the shared host vanished, and on a host
whose other sessions open and close GPU clients every second there may be none. Reproduced on
the build host beside a churner of short-lived GPU clients of our own: at one every 0.75 s, 4 of
4 runs gave up; at one every 1.5 s each run took 2-4 tries. Two other failures had the same
root: the oracle's sum over every client gave up when one went back inside its window (job
27254, the `.expect` on `oracle_times`), and fell below the counter's when a client came and went
inside the span (job 48857, the every-client assertion).

No span now needs to be free of churn. Each span of the counter's or the span meter's is
bracketed by `ioreg` listings of the registry, read independently of the counter: before it,
around its work and after it. `classify` sorts every client against them. A registry entry once
destroyed never comes back and its ID is not reused, so:

- a client two reads list is in every read between them, under one pid; anything else is a
  contradiction and fails the test: a client the end snapshot lacks while `ioreg` lists it after
  (step 5's own check, now on every span), one the start lacks while `ioreg` listed it before and
  after, one both snapshots hold that `ioreg` did not list around the work, one gone inside the
  work and back in the end snapshot;
- a client's time goes down only where a queue of it closed, which the listings show as a fall,
  so each counter reading lies between the listings either side of it, or on one side of a fall
  across it; a reading the registry does not explain voids the span;
- a client the counter saw vanish is corroborated if `ioreg` listed it under the same pid (gone)
  or showed its fall inside the span (gone back); one `ioreg` never listed voids the span;
- a client only `ioreg` listed inside the span, or a fall it showed where the counter saw an
  advance, is GPU time the counter's snapshots cannot hold, counted as hidden, a lower bound;
- this process's own clients must be clean, or the span is void.

A void span is taken again after 0.5, 1, 2, 4, 8, 8 ... s, and none the registry vouches for in
90 s fails as `process-gpu INCONCLUSIVE`, with each reason's count: not a verdict, and not a
pass. A refusal must be the registry's: a corroborated vanished client refuses the span, naming
the client the meter's rule picks (`expected_refusal`, factored out of step 5), and with none the
span is not refused. The checks then run on the view, the snapshots with the vanished clients
taken out of both, which can only raise this process's share.

`agrees_with_oracle` checks client by client. This process's GPU time equals `ioreg`'s exactly,
as before. The clients whose time went down nowhere advanced at most the oracle's advance plus
what the gap can hold (their lower bound is now the sorting's, per client). The meter's
every-client time is the sum over the view's clients. The rail is checked against the test's own
IOReport read on every span whose snapshots both read it current, shared ones included (it had
been on sole-user spans only), and a figure's rail is the snapshots' own advance. A figure must
be a sole user's share of the time the counter saw plus the time hidden from it. `oracle_times`
and its give-up are gone.

Steps 4c and 6 take the span meter's span through `finish_showing_gpu` (the previous commit) and
check that it is the GPU counter's figure, or the CPU alone with the counter's reason, over the
meter's own snapshots, where the test had picked a span out of six by the text of its reason. A
corroborated refusal is accepted there as the CPU alone, `HwShunt`, `Partial`, with CPU energy.
Step 5's span-meter span gives the counter's reason over its own snapshots. The idle spans of
steps 3 and 4a, and 4c's CPU-only span, also assert that `ioreg` saw no GPU time of this
process's inside them. After the liveness askers, the test waits, at most 3 s and never failing,
for this process's own clients to settle.

the_span_classifier_sorts_every_client drives `classify` over six built reads: clean; gone and
gone back, with the registry vouching and without; made and destroyed inside the span (48857's
shape); a fall inside the work that the counter saw as an advance (27254's); falls across each
snapshot inside the span and outside it; under- and over-reads; this process's own client not
clean; each contradiction and a pid mismatch; `expected_refusal`'s order; and the 99% gate with
hidden time.

standards-build.yml: the jouleclaw-energy step also greps `process-gpu: oracle-checked:`, which
step 2 prints once its burn was checked against the oracle. MEASURED_ENERGY.md says what the test
checks.

Verified on the build host, at this commit's jouleclaw files: clippy -D warnings on
jouleclaw-energy with `--features ioreport` and without; the macOS energy job's jouleclaw-energy
step as standards-build.yml runs it (under `bash -eo pipefail`, both greps), 5 of 5, three of
them beside a churner of short-lived GPU clients of our own every 0.75 s; and the process_gpu
test 16 of 16, 4 quiet and 4 beside a churner every 3, 1.5 and 0.75 s, with no void span, no
INCONCLUSIVE and no contradiction, every vanished client corroborated on the first attempt. A
read-only walk of the registry the way the counter walks it, every 5 ms for 60 s beside a churner
every 0.2 s (268 clients made and destroyed), found no walk lacking a client the walks either
side of it held and no child iterator invalid at the end of a walk (phantom_walks 0,
invalid_at_end 0, of 7266 walks): a contradiction is a fault in a read, not churn.

Falsified, each failing as named: a start or end walk dropping a client, or giving it another
pid, and a phantom client in both snapshots (contradictions); the counter skipping another
process's vanished client, or giving a figure across a gone one ("the registry vouches for ...
but the counter gave Ok"); refusing a client that did not vanish, or naming another than the
rule's; this process's time and every client's time each 1 ns off; the end snapshot misstating
when it read; the rail under-read by 10%, and a figure's rail not the snapshots' advance; the
sole-user gate at 1%; the span meter's figure, or its reason, not the counter's over its own
snapshots; GPU work in an idle span; the CI grep's line renamed; and the classifier's stretch
rule, fit rule, fall placement, the per-pid oracle, the refusal order and the hidden-time gate,
each broken in turn. Two persistent faults end INCONCLUSIVE, as designed: (c) a 10% under-read of
another process's client in every snapshot, "process-gpu INCONCLUSIVE: 1.5 s burn: no span in 92
s (12 spans) could be checked against the oracle: a counter reading out of the registry's order
×12; ...", and (d) a phantom client in every start snapshot, "... a vanished client the registry
never listed ×12; ...".

Co-Authored-By: Claude Opus 5.5 <[email protected]>

bd46749c27

• View at this point in history

2026-11-04 08:13:11 -05:00

Transaction Science

jouleclaw-energy: a span's GPU snapshots, for the hardware test that checks its refusals ...

`ProcessSpanMeter::finish_showing_gpu` is `finish`, and the span's two GPU snapshots when both
were taken; `finish` now calls it and drops them. It is `#[doc(hidden)]`, not part of the
metering API, as `probe_gpu_liveness_uncached` is not. tests/process_gpu.rs needs it to check
what the GPU counter made of the span meter's own snapshots against a registry read of its own;
it had picked a span out of six by the text of the reason. The GPU half is unchanged:
`gpu_figure` shares the two snapshots when both were taken, and is otherwise the start's failure,
then the end's, as `finish` had it.

span.rs's unit test a_spans_gpu_figure_comes_with_the_snapshots_it_was_shared_over: the share is
over the two snapshots and they come back with it; a failed start, a failed end, and both, are
each that failure, the start's first, with nothing shared and nothing handed back.

Verified on the build host: clippy -D warnings on jouleclaw-energy with `--features ioreport`
and without; span.rs's unit tests pass. Falsified: the end's failure ahead of the start's fails
the new test (`Err(NotAdvanced)` for a span whose start was `NotCurrent`), and so does handing
back the start snapshot twice.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

3bfe08eab8

• View at this point in history

2026-11-04 08:13:11 -05:00

Transaction Science

watch-together test: the example as a person runs it first, then a held page, held to the room's log ...

Review of b01681570, the example's test. It had replaced the example's own run with the hold:
Ada's play always went through take-for-gone and re-entry, never the path a present viewer's
control takes. And it passed defects the old test failed: it let either page re-enter any
number of times uncounted (only Ada's `rejoins >= 1` was read, Ben's never), a `not_present`
refusal went nowhere, and "the room refused no page's event" held while the test had Ada's
events refused. Moving app.js's statements back after `begin()` passed (Ada was held either
way; a Ben slow past 4 s just re-entered), as did a page that stopped stating, or a room whose
silence limit shrank. And it checked the play against the instant Ada first signed it for, not
the one the room carried after re-signing moved it.

Now:

1. As before, both pages enter; Ada plays and Ben pauses, each while present; both apply each
   control at its instant within a quarter second of the room and pause at one position.
2. Ben's page is held until the room's log carries his `leave`; Ada seeks back to 2 s
   meanwhile; Ben's play is signed as the hold ends, before his page can enter again. The
   test reads the room's log until Ben's play is carried after his second `join`, takes the
   instant the room carried (at least that join's `t` plus the lead, and no earlier than first
   asked), and both pages must play from the seek at it within the bound, then pause at one
   position again when Ada pauses 3 s later.
3. The room's log, read once and every record verified (`roomTest.watchLog`): exactly one
   `leave`, Ben's, after his hold began; Ada joined once, Ben twice, the second after the
   `leave`; every slot wholly within each stretch of each page's presence holds a statement of
   it (co-watch §7.3), but Ben's hold; and each page's first statement comes before the instant
   it began to follow the log, which it does only once it has its relay session. That last is
   ordering, not timing: with the statements started after `begin()` it cannot hold.
4. Each page's own counts: Ada never taken for gone and never entering again, Ben taken for
   gone once (`stats.gone`), entering again once with his play signed again; no other event of
   either refused, and no page stopped.

A wait still ends at once when a page stops and says each page's counts.

Verified on the build host, the room client and the archive built as wai-examples.yml builds
them (load 62-73): watch-together through run-examples.sh passed 5 of 5, every check held.
Falsified, each built and run the same way: statements started after `begin()` again fails the
first-statement check; a page that stops stating once it has entered again fails the slot and
`leave` checks; a room silence limit of 400 ms fails the `leave` count and the pages' counts.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

b4f33410cb

• View at this point in history

2026-11-04 08:13:11 -05:00

Transaction Science

joule-moq-web: a co-watch page taken for gone again at once enters again, and keeps its controls' order ...

Review of b01681570, the page's side.

- A page taken for gone again right after it entered again (a second stall, or a service that
  signed a queued `leave` after its `join`) stopped for good: one re-entry was allowed between
  carried posts. It now enters again up to three times with nothing of it carried between, then
  stops. A join that throws (the service answers 503 when its clock cannot sign) no longer
  leaves the page marked as entered with a stale outbox, which stopped it at the next post: the
  page logs it and the next post tries again, as after any other failed post. Each post refused
  `not_present` is counted (`stats.gone`), apart from `rejoins`.
- `resign()` moved a control signed while the page was gone to the earliest instant the lead
  allows even past another viewer's control carried meanwhile; controls apply in the order of
  their instants (co-watch §5.4), so the earlier press overrode the later one. Such a control is
  now dropped (logged, counted in `dropped`) when another viewer's control the page holds lies
  at or after its instant as pressed and at or before the instant it would move to; otherwise it
  is signed again as before.
- Signing again a position whose event the room refused is not equivocation: the room holds no
  event it refuses. Co-watch §10 now says so, and `resign()`'s doc.
- A presentation statement whose signing straddled a re-entry was carried with an `at` from
  before the new join; it is dropped and the page states anew at once.
- `run()` starts at the join (b01681570); if `begin()` then fails, the page now stops its
  co-watch log instead of stating a presentation it does not keep as a viewer that applies no
  control.
- The snapshot gives the page's principal, its outbox, and `followed_at`, the room clock when it
  began to follow the log; `roomTest.watchLog()` reads the room's log with each record verified,
  and `watchHoldUntilGone` says when the hold began.

cowatch.test.mjs adds, against its watch.rs-like service: a page taken for gone again right
after it entered again enters once more and its pause is carried; a join that fails once is
tried again and the page goes on; a page taken for gone at every entry stops after three, its
refusal counted; and a control pressed while gone is dropped where it would move past another
viewer's, signed again where it keeps its place.

Verified on the build host (load 60-75): cowatch.test.mjs, 12 tests, passed 5 of 5.
Falsified: allowing one re-entry, rethrowing a failed join, and removing the drop each fail the
test written for it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

37ae9fa074

• View at this point in history

2026-11-04 08:13:11 -05:00

Transaction Science

joule-moq-room: a watch viewer taken for gone has its pending leave signed before its next join ...

Review of b01681570. Under the state lock, `ended()` marks a silent viewer `leaving`;
`run_watch` then signs the window's due skew statements, each through the clock's lock with its
rate wait and relay write, and only after them the viewer's `leave`. A post of the viewer's in
that window is refused `not_present`, and its page enters again: `watch_join` saw it not present
and asked the clock for a `join`, which could take the clock's lock before the queued `leave`.
The `leave` was then signed after the new `join` and ended the presence it began, the page's
post signed again was refused `not_present` once more, and the page stopped for good: the
"never" this branch set out to remove. The clock's chain also held a `join` of a viewer already
present, which live-rooms §7.2 refuses (`already_present`), so the room no longer verified.

The order is now decided under the clock's lock, from the room's state read under it:

- `Viewer::leaving` holds the reason of a `leave` asked of the clock and not yet signed;
- `Service::watch_leave` signs that `leave` only if it is still pending: a viewer that entered
  again in between is not ended by it. `run_watch` signs each `leave` through it;
- `Service::watch_enter` (`POST /watch/join`) signs a pending `leave` first, then the `join`;
  a viewer present and not leaving is answered `again`, as before.

`emit_items_after` is split into taking the lock and `sign_items`, which both use with the
lock held. The README and http.rs's table say what a join does now.

tests/room_service.rs: a_watch_viewer_taken_for_gone_enters_again_after_its_leave runs the
service with the clock but not its watch loop, so the test decides when the `leave` is signed.
The room takes the viewer for gone; its post is refused `not_present`; it enters again before
the `leave` is signed; the room's own `leave` then signs nothing, and the refused position,
signed again, is carried. Then the other order: the `leave` signed (once), then the `join`
alone. The clock's lifecycle of the viewer is join, leave, join, leave, join, the viewer's
chain is whole, and the room verifies (co-watch §11, which runs live-rooms §9).

Verified on the build host (load 55-65): `cargo test --locked --all-targets` in
joule-moq-room passed; room_service's two watch tests and watch.rs's unit test passed 5 of 5.
Falsified: signing the `join` without the pending `leave` first fails the new test, and so does
signing the room's `leave` without checking it is still pending.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

398919b71f

• View at this point in history

2026-11-04 08:13:11 -05:00

Transaction Science

jouleclaw-energy: a refusal names the oldest vanished client, by registry ID, as its doc says ...

Review of 822bb2ac5. `GpuClientVanished`'s doc said that, of several vanished clients, the
meter names this process's own first, else the first in the start snapshot, "which the I/O
Registry lists oldest first". The meter did not make that so: it walked the snapshot in the
order the registry read gave it, which is attach order only within one accelerator's children,
and the new unit test pinned that order (reversed, it expected client 20 named over the older
10). Nothing tested "oldest".

`spent` now checks the start's clients in the order of their registry entry IDs, which the
kernel gives entries in the order it makes them and never reuses while the host is up: the
oldest vanished client is named, wherever the snapshot lists it. `GpuClient::id`'s doc says so.
The unit test expects client 10 for the snapshot as built, reversed and shuffled, and this
process's own oldest vanished client (31 of 31 and 32) wherever they sit. Step 5 of
`a_gpu_span_is_a_sole_users_rail_or_an_exact_zero_or_no_figure` picks the expected client the
same way.

Step 5 also reads the registry once with `ioreg`, independently of the counter, after the
counter's end snapshot, and asserts that no client the end snapshot lacks is still listed: an
entry once destroyed never comes back and its ID is not reused, so such a client would be one
the counter's read left out, not one that closed. When the refusal names another process's
client, it prints every client that vanished as (pid, id, start ns, end ns or gone), so a
failure log tells host churn from a fault in the read.

Verified on the build host (load 55-65): clippy -D warnings on jouleclaw-energy with
`--features ioreport` and without; the CI step (`cargo test -p jouleclaw-energy --features
ioreport -- --nocapture`, the liveness line printed) passed 5 of 5. Falsified: without the sort
by ID, the unit test fails on the reversed snapshot.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dfbb87d0be

• View at this point in history

2026-11-04 08:13:11 -05:00

Transaction Science

joule-moq-web: a co-watch page the room took for gone enters again with its chain whole ...

watch-together failed on the macOS examples job five times (jobs 65458, 71887,
72455, 76030, 76114), each with "both play on Ada's control: not within 60000 ms
(last: false)". It was not slow. Each of those pages' consoles shows Ada joining
the watch room, a 409, a second join, another 409, and "the room refused an
event: time_reversed"; from then on her page applied nothing.

The room service ends a viewer that posts nothing for 4 s (SILENT_US), and the
page posted nothing between its co-watch join and the end of `begin()`: the
compositor, the audio worklet and the relay session. On a loaded host that took
4.4-8.2 s in every failure against 1.2-3.7 s in every passing run. Ada's first
post then got `not_present`; `flush()` entered again, and the clock signed her
new `join` at a later t than the event she had signed before it, which `flush()`
then sent again: refused as `time_reversed` (watch.rs `post`), the page stopped,
and nothing clears a stop. When that event was her play control, the room never
carried it at all. So any stall past 4 s, at any point, stopped a page for good.

Two changes in the page:

- cowatch.js: re-entering works. The page keeps where the room's copy of its
  chain ends (each outbox entry has its seq and digest), and after the new join
  signs every control the room had not carried again from there, at or after
  the join: at its instant if the policy's lead still allows it, else at the
  earliest it does. A presentation statement signed before then describes an
  instant the page was not present for and is dropped; the page states anew
  within half a slot. The page counts its re-entries and what it signed again
  and dropped.
- app.js: the statements start at the join (co-watch 7.3: a present player
  states at least once a slot), not after `begin()`. That removes the usual way
  into the silence; it cannot prevent one, since a thread held by a loaded
  machine runs no timer. The log is still followed after `begin()`, which reads
  the grants a control's author is checked against.

The example's test now makes that silence happen instead of hoping the host
does: `roomTest.watchHoldUntilGone` holds Ada's page's thread from the moment it
enters until the room's log carries her `leave` (read with blocking requests),
while Ben's page enters, then signs her play in the same task, before her page
can enter again. Both pages must still apply it at its instant within 250 ms,
and Ben's pause, at one position; her page must have entered again with the
play signed again; and at the end no page's event was refused and none stopped,
so each viewer's chain was carried whole. A wait now ends at once when a page
stops, and says each page's controls, playing, error and stop instead of "last:
false". cowatch.test.mjs adds the same path against a service that applies
watch.rs's rules: a pause signed while the page was gone is carried after its
second join, the stale statement is not, and the chain goes on.

Verified on the build host, the room client and the archive built as the job
builds them: cowatch.test.mjs 8 of 8; watch-together through run-examples.sh 5
of 5 at load 130-150 and 3 of 3 beside 96 busy loops (load 240-260), the
setting that failed it 1 in 3 before. In one loaded run the room had already
taken Ada for gone before the hold began, as in CI, and her page entered again
and went on. Falsified: without the re-signing, the unit test fails and the
example fails in 9 s with Ada stopped on time_reversed and no control carried.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

707ffe2cb1

• View at this point in history

2026-11-04 08:13:11 -05:00

Transaction Science

jouleclaw-energy test: across a child's exit, check the client a refusal names, not its pid ...

Step 5 of `a_gpu_span_is_a_sole_users_rail_or_an_exact_zero_or_no_figure`
burns this process's GPU across a second child's exit and asserted

    matches!(refused, Err(GpuClientVanished { pid, .. }) if pid == second_pid)

It failed on the macOS energy job four times since 2026-10-11 (jobs 21067,
36264, 75996, 76205), each time naming another process's pid. The meter was
right each time; the assertion assumed the child's client is the only one that
can vanish in that ~0.3 s span, and the build host is shared: Playwright's
headless browsers, other test processes and mediaanalysisd open, close and reset
GPU clients all the time (the same run's 1.5 s burn had already retried twice
for two other pids).

`apportion` checks this process's own clients first, then the others, each
pass walking the start snapshot in order and naming the first client that is
gone or went back. The registry lists clients oldest first, so any older client
of another process's that vanished in the same span is named before the child's.
Every such client is a true witness: its time over the span is lost, so
`all_ns`, and the 99% sole-user test, cannot be decided.

The step now proves what it is for, whatever else the host does:

- the refusal is `GpuClientVanished`, never a figure;
- the client it names was at the start under that pid and is gone or went back
  at the end (the check `until_no_client_vanished` already made, now one helper);
- it is exactly the one the rule picks: this process's own vanished client
  first, else the first vanished in the start snapshot;
- every client of the second child's is gone from the end snapshot;
- with the host's other vanished clients taken out of the real start snapshot,
  the child's exit alone still refuses the span, naming the child; so the check
  cannot pass only because unrelated churn refused it.

When another client was named it says which. The step is not retried: there is
nothing to wait out, and a retry would hide a regression in the rule.

The tie-break gets a unit test of its own over built snapshots (two other
processes' clients vanish: the first at the start is named, in either order,
gone or gone back alike; this process's own vanished client is named even when
it comes last), and `GpuClientVanished`'s doc states it. The meter is unchanged.

Verified on the build host (load 95-160): clippy -D warnings on
jouleclaw-energy with `--features ioreport` and without; the CI step
(`cargo test -p jouleclaw-energy --features ioreport`, the liveness line
printed) passed; process_gpu passed 5 of 5 on the host as it is and 6 of 6
beside a churner opening an idle GPU client for 1.5 s every 2.5 s, which failed
the old assertion 2 of 6. In one of those six the refusal named a churner's
client, as in CI, and every check above held. Falsified: dropping a vanished
other-process client instead of refusing fails step 5 (it gave GpuShared) and
two unit tests; walking the start snapshot last-first fails the new unit test
alone.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

06c914ce16

• View at this point in history

2026-11-04 08:13:10 -05:00