joule-moq-relay: an object is held once, and the socket keeps what arrives #583

Merged

dcharlot merged 2 commits from relay/fanout-load into main 2026-11-02 18:36:39 -05:00

Conversation 0 Commits 2 Files changed 14 +529 -28

dcharlot commented 2026-11-02 13:29:35 -05:00

Owner

Copy link

Under load the relay collapsed with cores to spare. A new load test, examples/fanout_load.rs, found two reasons, and this fixes the cheap part of both.

One socket with a small buffer. The relay's UDP socket kept the kernel's default, about 212 KB. Each fan-out's acknowledgements arrive together, so at 2,000 viewers the kernel dropped about a million packets in 20 seconds. A dropped acknowledgement reads as loss, which means a retransmission and a smaller congestion window. limits.quic_socket_buffer_bytes now asks for 4 MiB, on the sockets the relay listens on and dials from.

A payload copied per viewer, under the relay's one lock. Each subscriber got its own copy during the fan-out and more on the way to the wire. An object is now held once, from when it's read until the last QUIC stream has it acknowledged: the cache, every session's queue and every QUIC send buffer share one allocation.

Measured

Unmodified main against this branch, the same harness, 8 KB objects at 30/s (about 2 Mbit/s a viewer), relay pinned to 8 cores, two rounds:

	main	this branch
Peak memory, 2,000 viewers, 256 KB keyframes	1,229–4,131 MB	375–378 MB
Delivered, 2,000 viewers, keyframes	99.63–99.95%	100%
Delivered, 4,000 viewers	90.4–94.7%	100%
Peak memory, 4,000 viewers	1,697–2,064 MB	654–748 MB
Packets dropped at the relay, 2,000 viewers	928k–1.04M	0–35k

Latency improved in most runs, but the host was busy (load average 20–32 of 32 threads), so I'm not quoting it as a result. The clearest pair is p99 at 2,000 viewers, 88 → 26 ms in the first round. In the second round, a noisier one, it was 974 → 815 ms.

An earlier run on a quieter host used the same buffer and a shared payload, without the QUIC zero-copy added here. It took the longest stall from one 256 KB keyframe at 2,000 viewers from 52–58 ms to 7–8 ms.

What this doesn't fix

• Keyframes still overflow the buffer (140–144k drops), and 4,000 viewers still drop 412–665k. One socket is the limit, and per-core sockets are the next change.
• The fan-out still holds one relay-wide lock, only for less time, so a big room can still delay a small one.
• Only native QUIC gets the zero-copy. A WebTransport or WebSocket session, or one bridged to an earlier draft, still has its own copy in its transport. A SendStream that keeps no buffers gets an object's head and payload as one write, so the bridges see what they saw before.
• The buffer needs the host's permission. Linux silently caps it at net.core.rmem_max, often 212992, so the relay logs a warning, and a deployment has to raise that limit to benefit.
• All of this is loopback on one machine. No real network has been involved, and the harness's own client sockets drop some packets on keyframes (30–46k in both columns), which inflates the keyframe tail in both.

What ran

cargo test -p joule-moq-relay: 126 passed, covering all transports. -p joule-moq --lib: 128 passed. cargo check --workspace --all-targets for joule-code-rs. New tests:

• a shared payload is the object's own bytes, with no copy;
• an object's head plus its payload is its whole encoding;
• parts reach a copying transport as one write;
• a socket gets the buffer asked for, and an absurd size is capped rather than wrapping around, which an earlier version of the code did.

🤖 Generated with Claude Code

Under load the relay collapsed with cores to spare. A new load test, `examples/fanout_load.rs`, found two reasons, and this fixes the cheap part of both. **One socket with a small buffer.** The relay's UDP socket kept the kernel's default, about 212 KB. Each fan-out's acknowledgements arrive together, so at 2,000 viewers the kernel dropped about a million packets in 20 seconds. A dropped acknowledgement reads as loss, which means a retransmission and a smaller congestion window. `limits.quic_socket_buffer_bytes` now asks for 4 MiB, on the sockets the relay listens on and dials from. **A payload copied per viewer, under the relay's one lock.** Each subscriber got its own copy during the fan-out and more on the way to the wire. An object is now held once, from when it's read until the last QUIC stream has it acknowledged: the cache, every session's queue and every QUIC send buffer share one allocation. ## Measured Unmodified main against this branch, the same harness, 8 KB objects at 30/s (about 2 Mbit/s a viewer), relay pinned to 8 cores, two rounds: | | main | this branch | |---|---|---| | Peak memory, 2,000 viewers, 256 KB keyframes | 1,229–4,131 MB | 375–378 MB | | Delivered, 2,000 viewers, keyframes | 99.63–99.95% | 100% | | Delivered, 4,000 viewers | 90.4–94.7% | 100% | | Peak memory, 4,000 viewers | 1,697–2,064 MB | 654–748 MB | | Packets dropped at the relay, 2,000 viewers | 928k–1.04M | 0–35k | Latency improved in most runs, but the host was busy (load average 20–32 of 32 threads), so I'm not quoting it as a result. The clearest pair is p99 at 2,000 viewers, 88 → 26 ms in the first round. In the second round, a noisier one, it was 974 → 815 ms. An earlier run on a quieter host used the same buffer and a shared payload, without the QUIC zero-copy added here. It took the longest stall from one 256 KB keyframe at 2,000 viewers from 52–58 ms to 7–8 ms. ## What this doesn't fix - **Keyframes still overflow the buffer** (140–144k drops), and 4,000 viewers still drop 412–665k. One socket is the limit, and per-core sockets are the next change. - **The fan-out still holds one relay-wide lock,** only for less time, so a big room can still delay a small one. - **Only native QUIC gets the zero-copy.** A WebTransport or WebSocket session, or one bridged to an earlier draft, still has its own copy in its transport. A `SendStream` that keeps no buffers gets an object's head and payload as one write, so the bridges see what they saw before. - **The buffer needs the host's permission.** Linux silently caps it at `net.core.rmem_max`, often 212992, so the relay logs a warning, and a deployment has to raise that limit to benefit. - **All of this is loopback on one machine.** No real network has been involved, and the harness's own client sockets drop some packets on keyframes (30–46k in both columns), which inflates the keyframe tail in both. ## What ran `cargo test -p joule-moq-relay`: 126 passed, covering all transports. `-p joule-moq --lib`: 128 passed. `cargo check --workspace --all-targets` for joule-code-rs. New tests: - a shared payload is the object's own bytes, with no copy; - an object's head plus its payload is its whole encoding; - parts reach a copying transport as one write; - a socket gets the buffer asked for, and an absurd size is capped rather than wrapping around, which an earlier version of the code did. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dcharlot added 2 commits 2026-11-02 13:29:35 -05:00

joule-moq-relay: a fan-out load test: one publisher, N subscribers, a bystander, the relay's own memory and CPU ... b6db529c6f

Co-Authored-By: Claude Opus 5.5 <[email protected]>

joule-moq-relay: an object is held once, and the socket keeps what arrives ... 88ff316805

Under load the relay collapsed with cores to spare, for two reasons a load
test showed.

Its one UDP socket kept the kernel's default buffer, about 212 KB. Each
fan-out's acknowledgements arrive together, so at 2,000 viewers of 2 Mbit/s the
kernel dropped about a million packets in 20 seconds. A dropped acknowledgement
reads as loss: a retransmission and a smaller congestion window. The buffer is
now a setting, 4 MiB by default, for the sockets the relay listens on and dials
from. The system caps it, so the relay warns when it gets less.

Each subscriber also got its own copy of every payload, while the fan-out held
the relay's one lock, and two more copies on the way to the wire. An object is
now held once, from when it is read until the last QUIC stream has it
acknowledged. The cache, every session's queue and every QUIC send buffer share
one allocation, and a keyframe stops being N keyframes in memory. A transport
that keeps no buffers still copies, once, where it writes, and a bridge to an
earlier draft still gets an object's head and payload as one write.

`examples/fanout_load.rs` is the load test. A bystander on its own track measures
what the audience costs everyone else on the relay.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot merged commit 893036c935 into main 2026-11-02 18:36:39 -05:00

dcharlot referenced this pull request from a commit 2026-11-02 18:36:40 -05:00

Merge pull request 'joule-moq-relay: an object is held once, and the socket keeps what arrives' (#583) from relay/fanout-load into main

dcharlot referenced this pull request 2026-11-02 19:16:34 -05:00

joule-moq-relay: a fan-out holds up no other track, and each core has its own socket #587

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Transaction-Science/open-standards!583

No description provided.