joule-moq-relay: a MOQT draft-22 relay over QUIC, WebTransport and a WebSocket fallback #293

Merged
dcharlot merged 19 commits from joule-moq/relay into main 2026-10-18 12:37:19 -04:00

2026-10-18

Transaction Science
joule-moq-relay: clippy's notes on the new test code
Some checks failed
wai formal verification (integer core) / kani (rans64-t2) (pull_request) Has been cancelled
wai formal verification (integer core) / kani (stage-long) (pull_request) Has been cancelled
wai formal verification (integer core) / kani (integer-core) (pull_request) Has been cancelled
WAI energy receipts / does this change touch it (pull_request) Has been cancelled
WAI energy receipts / energy receipts · linux (pull_request) Has been cancelled
WAI energy receipts / energy receipts · linux-arm64 (pull_request) Has been cancelled
WAI energy receipts / energy receipts · macos (pull_request) Has been cancelled
WAI conformance / does this change touch it (pull_request) Has been cancelled
WAI conformance / cargo-class corpora · linux (pull_request) Has been cancelled
WAI conformance / cargo-class corpora · linux-arm64 (pull_request) Has been cancelled
WAI conformance / cargo-class corpora · macos (pull_request) Has been cancelled
WAI conformance / quantum core and corpus under wasm32-wasip2 (wasmtime) (pull_request) Has been cancelled
WAI conformance / v2 vocabulary · manifests validate as stated (pull_request) Has been cancelled
WAI conformance / independent verifiers · claims, JWP, key discovery and integer payloads (Python) (pull_request) Has been cancelled
WAI conformance / delivery telemetry · CMCD and CMSD against an independent library (pull_request) Has been cancelled
WAI conformance / cross-arch byte-equality · linux (pull_request) Has been cancelled
WAI conformance / cross-arch byte-equality · linux-arm64 (pull_request) Has been cancelled
WAI conformance / cross-arch byte-equality · macos (pull_request) Has been cancelled
WAI conformance / codec wrappers · fuzz smoke, text decode limits, C entry points, provenance, the semantic path (pull_request) Has been cancelled
WAI conformance / C2PA manifest emission · signed, read back, validated (pull_request) Has been cancelled
wai cross-architecture byte-equality / does this change touch it (pull_request) Has been cancelled
wai cross-architecture byte-equality / linux-aarch64 (pull_request) Has been cancelled
wai cross-architecture byte-equality / linux-x86_64 (pull_request) Has been cancelled
wai cross-architecture byte-equality / macos-arm64 (reference) (pull_request) Has been cancelled
joule-code conformance gate / Build + round-trip C1-C4 vectors (pull_request) Successful in 2m3s
standards workspaces build + test / which workspaces this change touches (pull_request) Successful in 17s
joule-code conformance gate / joule-moq-relay, every target (pull_request) Successful in 1m31s
repository checks / lockfiles, tests in CI, workflow hygiene, spec and section references, public text (pull_request) Successful in 2m29s
standards workspaces build + test / macOS energy backends (pull_request) Successful in 8m20s
standards workspaces build + test / joule-code-rs (joule-code/joule-code-rs, joule-code-rs, linux-arm64, true) (pull_request) Successful in 2m50s
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-18 07:40:14 -04:00
Transaction Science
joule-moq-relay: the slow subscriber starts reading after the load has arrived
With its streams read concurrently, the test's drops rested on the stall
alone; under a load average near 200 a slow publisher was still sending when
the stall ended, and three runs of 144 dropped no layer object at all. The
subscriber now reads nothing until the publisher has sent everything and the
stall has passed after that, so every layer object waiting then is past its
deadline. 144 of 144 runs passed at a load average of 204 to 244.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-18 07:36:51 -04:00
Transaction Science
joule-moq-relay: the slow subscriber reads its streams concurrently, and says why
CI failed slow_subscriber once (39 of 40 base objects). Reproduced on the
build host under load: 3 failures in 64 runs, then 8 in 144. In every one the
relay's own tally for the base track was 40 forwarded, none dropped, none left
at close; no base stream arrived reset; the subscriber's one-at-a-time reader
had given up waiting on a stream's header with 22 to 88 accepted streams
unread behind it; and once those were read concurrently every missing base
object arrived. The relay lost nothing: the reader waited on a stream whose
bytes waited for connection credit the unread streams behind it held.

- The test now reads its streams concurrently, as a subscriber does, until
  the base track and every receipt are whole and a second passes with
  nothing new, or the overall deadline passes. Its assertions are unchanged.
  A failure report says where any missing base object was.
- The client keeps a stream next_data timed out on, rather than dropping it,
  records each such wait, and can hand out streams unread (next_stream).
- Connection::diagnostics gives a QUIC session's flow-control counters;
  Relay::session_diagnostics each session's.
- README: a subscriber that does not read stalls its own high-priority track
  through connection flow control; what the relay does about it (§5.2), and
  that it does not yet end such a subscription with TOO_FAR_BEHIND.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-18 07:32:54 -04:00
Transaction Science
joule-code CHANGELOG: joule-moq-relay 0.1.0
Some checks failed
joule-code conformance gate / Build + round-trip C1-C4 vectors (pull_request) Successful in 2m11s
standards workspaces build + test / which workspaces this change touches (pull_request) Successful in 18s
joule-code conformance gate / joule-moq-relay, every target (pull_request) Successful in 3m31s
repository checks / lockfiles, tests in CI, workflow hygiene, spec and section references, public text (pull_request) Successful in 3m57s
standards workspaces build + test / macOS energy backends (pull_request) Successful in 5m37s
standards workspaces build + test / joule-code-rs (joule-code/joule-code-rs, joule-code-rs, linux-arm64, true) (pull_request) Failing after 2m37s
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-18 00:42:30 -04:00

2026-10-17

Transaction Science
joule-moq-relay: the cache's test-only import where its tests are
Some checks failed
joule-code conformance gate / Build + round-trip C1-C4 vectors (pull_request) Successful in 2m3s
joule-code conformance gate / joule-moq-relay, every target (pull_request) Successful in 1m23s
standards workspaces build + test / which workspaces this change touches (pull_request) Successful in 22s
repository checks / lockfiles, tests in CI, workflow hygiene, spec and section references, public text (pull_request) Successful in 1m49s
standards workspaces build + test / macOS energy backends (pull_request) Failing after 3s
standards workspaces build + test / joule-code-rs (joule-code/joule-code-rs, joule-code-rs, linux-arm64, true) (pull_request) Successful in 3m53s
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 12:15:20 -04:00
Transaction Science
joule-moq-relay: the cache covers the run of whole groups, not where its first object fell
The cache latched coverage on the first object it kept. When a group's
streams finished after a later group's, as they do on a loaded host, the
latch sat on the later group, and a FETCH from the earlier one went upstream
for objects the relay held: a late joiner on the arm64 runner was refused
when the publisher did not answer.

Coverage is now worked out from what is known of each group. A group is
whole when its start arrived (Object ID 0, or a Prior Object ID Gap), none of
its subgroup streams is open or was reset, and its end is known (its streams
ended with a FIN, or an End of Group or End of Track status, or a datagram
with END_OF_GROUP). A fetch is covered when every group from its start to the
Largest Object's is whole or known absent (a Prior Group ID Gap), that last
group has its start, and nothing between was evicted. A group never seen is
a gap the cache does not answer across.

The relay counts each upstream subgroup stream from its first kept object and
closes the count at its FIN (clean) or on any other end (the group is then
not whole). The late-joiner test now waits on the cache's coverage rather
than on objects_in, which counts arrivals before the cache; a new test sends
group 1 whole before group 0 and FETCHes from group 0, served from the cache.
It failed against the latch before this change.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 12:12:35 -04:00
Transaction Science
Merge remote-tracking branch 'origin/main' into joule-moq/relay
Some checks failed
joule-code conformance gate / Build + round-trip C1-C4 vectors (pull_request) Successful in 2m1s
joule-code conformance gate / joule-moq-relay, every target (pull_request) Successful in 1m33s
standards workspaces build + test / which workspaces this change touches (pull_request) Successful in 19s
repository checks / lockfiles, tests in CI, workflow hygiene, spec and section references, public text (pull_request) Successful in 2m4s
standards workspaces build + test / macOS energy backends (pull_request) Successful in 10m3s
standards workspaces build + test / joule-code-rs (joule-code/joule-code-rs, joule-code-rs, linux-arm64, true) (pull_request) Failing after 4m35s
2026-10-17 09:24:15 -04:00
Transaction Science
joule-moq-relay: the slow subscriber's last streams drained until quiet
Read one stream at a time until a second passes with none, bounded by the
number of streams, rather than for a fixed window a loaded runner could
outlast.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 05:00:12 -04:00
Transaction Science
joule-moq-relay: README, its CI job on linux-arm64, and joule-moq's pointers to it
The README states the draft the relay speaks and how each transport selects
it, the WebTransport library and why, how a browser connects (a certificate
hash in development; the WebSocket fallback), the WebSocket binding in full
as a non-normative binding of webtrans-http2 capsules, what the relay does
section by section, its scheduling and timeouts, receipts and the door, the
session summary and how its energy is labelled, and what is not yet built.

CI: a job of its own on linux-arm64 runs every target of the crate; the x86
workspace step excludes it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 04:56:38 -04:00
Transaction Science
joule-moq-relay: clippy's notes
Let-chains where it collapsed nested ifs, values() where it iterated a map
for its values; three allows where the shape is deliberate.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 04:52:11 -04:00
Transaction Science
joule-moq-relay: Request ID rules and requests bounded by the stream limit
A Request ID of the server's parity, or one used twice, closes the session
with INVALID_REQUEST_ID; with a limit of two requests a third waits for
WT_MAX_STREAMS credit, which returns when a request is cancelled.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 04:49:29 -04:00
Transaction Science
joule-moq-relay: the slow subscriber asks for a timeout on its receipts tracks
It does what moq-streaming-format 4.1 says a subscriber should not, and the
relay still applies no delivery timeout to a receipts track: every receipt
reaches it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 04:47:03 -04:00
Transaction Science
joule-moq-relay: GOAWAY on shutdown, tokens by alias, and fan-out order under load
- goaway.rs: every session gets GOAWAY with the New Session URI and Timeout;
  a new request after it is refused with GOING_AWAY; the sessions that leave
  close with NO_ERROR, and the one that stays is closed with GOAWAY_TIMEOUT.
- auth_tokens.rs: a token registered in SETUP authorizes a publisher; one
  registered in a request is used by alias in the next; no token or a wrong
  one is UNAUTHORIZED; an alias never registered closes the session with
  UNKNOWN_AUTH_TOKEN_ALIAS.
- priority_fan_out.rs: two publishers, four tracks, three subscribers with
  their own subscriber priorities and group orders, the subscribers' pipes
  held shut while the load arrives; each receives every object in section
  5.1.2's order.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 04:45:24 -04:00
Transaction Science
joule-moq-relay: a late joiner served from the cache, and a FETCH forwarded upstream
A late joiner over the WebSocket fallback reads TRACK_STATUS's Largest Object,
FETCHes a whole track and its receipts track from the cache and verifies
every object, FETCHes a range in descending group order, is refused a range
past the Largest Object with INVALID_RANGE, and joins at a past group by a
SUBSCRIBE whose fill arrives on a fetch stream named by its Request ID. The
publisher is asked for nothing. A FETCH for a track the relay holds nothing
of goes to the namespace's publisher, and its fetch stream is piped through.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 04:42:55 -04:00
Transaction Science
joule-moq-relay: a slow subscriber loses late layers and keeps every receipt
The scheduler now bounds opening a stream, not only the write, by the
object's deadline; the relay's QUIC send window is configurable, so what
cannot be sent waits in the scheduler where delivery timeouts apply; the
test client can read lazily, holding flow control like a slow reader.

The test: a base track at priority 0 and a layer at 200 with a 100 ms
OBJECT_DELIVERY_TIMEOUT. A slow subscriber reads nothing for 1.5 s. Its base
track arrives whole; some layer objects pass their deadline and their
streams are reset with DELIVERY_TIMEOUT; every receipt of both tracks
arrives and each dropped object verifies by inclusion; the relay's signed
summary counts each layer object as forwarded or dropped, exactly.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 04:41:03 -04:00
Transaction Science
joule-moq-relay: two publishers and three subscribers over QUIC, WebTransport and WebSocket
One scenario run over each transport on loopback: a publisher reached by the
namespace route and one by PUBLISH, three subscribers of both, one of them
discovering the first namespace by SUBSCRIBE_NAMESPACE. Every object reaches
every subscriber and verifies against its publisher's signed receipts, which
the relay checks at the door; every session's summary verifies under the
relay's key and labels the relay's energy unmetered.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 04:37:09 -04:00
Transaction Science
joule-moq-relay: a MOQT draft-22 relay over native QUIC, WebTransport and a WebSocket fallback
A new crate in the joule-code workspace, library and binary:

- transports behind one interface: native QUIC (ALPN moqt-22) and WebTransport
  over HTTP/3 (ALPN h3, WT-Available-Protocols) on one quinn endpoint, and a
  WebSocket binding whose messages are draft-ietf-webtrans-http2-15 capsules;
- the relay: SETUP, PUBLISH_NAMESPACE, SUBSCRIBE_NAMESPACE, PUBLISH, SUBSCRIBE
  from an upstream it holds or makes, RENDEZVOUS_TIMEOUT, FETCH and fills from
  the cache or upstream, TRACK_STATUS, REQUEST_UPDATE, GOAWAY, Request ID rules,
  MAX_REQUEST_UPDATES, the authorization-token alias cache;
- a per-session scheduler in section 5.1.2's order with delivery timeouts at
  this hop; receipts tracks carry none;
- receipts checked at the door per namespace, and signed session summaries.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 04:33:00 -04:00