jouleclaw-graph: a node in flight survives the process, and an interrupted one is held rather than silently run again #333

Merged

dcharlot merged 1 commit from fix/jouleclaw-graph-dispatch-before-run into main 2026-10-19 06:08:23 -04:00

Conversation 0 Commits 1 Files changed 3 +400 -2

2026-10-19

Transaction Science

jouleclaw-graph: a node in flight survives the process, and an interrupted one is held rather than silently run again ...

A JouleClaw run journal recorded a node only when it completed
(`NodeRecorded`). Nothing recorded that a node had begun. A process that
died in the middle of a node therefore left no trace of it, and a
restarted run found the node ready again. The next dispatch repeated
whatever it had done outside the run (a message sent, a payment made, a
record written) with no sign it might already have happened, and the
interrupted attempt's energy was lost. JCP §4.11 requires the decision
to be recorded before the action runs and forbids silently retrying one
whose outcome is unknown. jcp-orchestrate, mesh and jcp/1 already do
this; the run journal did not.

A comparison with durable agent harnesses prompted this. The common shape
commits a tool call's intent before running it, reruns calls declared
safe, and tells the model an unsafe one may have partly run. Published
crash tests of that default show its weakness: a model retried an
interrupted write it could not see and left a duplicate. Here an
interrupted `never` node is not left to a model: it is held until someone
settles it.

- **Rule and events.** `Replay { Safe, Never }`, with `Never` the
  default. New `RunEvent`s: `NodeDispatched { attempt, replay }`,
  written before work begins; `NodeInterrupted { attempt, observed_uj }`;
  and `RerunApproved { approved_by }`. The rule travels with the
  dispatch, not on `Node`, so every graph's content address and every
  persisted journal are unchanged.
- **`Run::dispatch`** records the dispatch and returns the attempt. A
  node in flight is not ready, so it is not dispatched twice.
- **`Run::recover`** marks every node still in flight after a restart as
  interrupted, keeping the energy observed for the attempt as a lower
  bound. An interrupted `Safe` node is ready again as a new attempt. A
  `Never` node is held (`held_nodes`) until it is recorded, approved for
  a rerun by a named party (`approve_rerun`), or the run fails.
- **Fail-safe.** A consumer that omits `recover` leaves the node in
  flight and out of `ready_nodes`, so it stalls rather than repeats.
  `record` without a prior `dispatch` works as before, so existing
  consumers are unaffected.
- **jouleclaw-stuck** counts dispatch, interruption and approval
  timestamps as activity, so a run whose last event is a long node's
  dispatch is not reported abandoned.
- **Spec.** JouleClaw SPEC §17.3 states it.

Checked on the build host:
- jouleclaw-graph: 47 tests pass, six of them new. jouleclaw-stuck: 17,
  one new.
- jouleclaw-gate, graph-mcp, mission, artifact, reflection and promote
  pass, and the JouleClaw workspace checks. Clippy is clean.
- Mutations each fail tests: ready_nodes ignoring in-flight and held
  nodes, dispatch ignoring a held node, every interrupted node rerunnable,
  interrupted energy dropped, and stuck ignoring dispatch.

Not in this change: jouleclaw-mission dispatches features rather than
graph nodes and has no resume of its own. Wiring its workers through
`dispatch` / `recover` is the next step.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

35b7bd3972

• View at this point in history

2026-10-19 02:56:06 -04:00