joule-moq-relay: a fan-out holds up no other track, and each core has its own socket #587

Merged

dcharlot merged 1 commit from relay/fanout-shards into main 2026-11-03 00:48:02 -05:00

Conversation 0 Commits 1 Files changed 15 +397 -59

dcharlot commented 2026-11-02 19:16:34 -05:00

Owner

Copy link

Step 2 of the relay's fan-out work, after #583. Two limits remained once an object was held once: one track's fan-out held up every other track, and one UDP socket took every packet of every viewer.

What changes

• A fan-out holds up no other track. Subscribers are a copy-on-write snapshot (Arc<Vec<Arc<DownSub>>>, written with Arc::make_mut). fan_out_object, end_subgroup and fan_out_datagram take the snapshot under the track's lock, drop the lock, and push to each subscriber outside it. A subscription that has left is marked gone under its own parameters' lock, so nothing is pushed to it after its PUBLISH_DONE. Unit test: a_subscription_that_left_passes_nothing.
• Each core has its own socket (limits.quic_sockets; 0 = one per core up to 16, Linux only, otherwise 1). The sockets share a port with SO_REUSEPORT. A cBPF program steers each datagram by the first byte of its destination connection ID (long header offset 6, short header offset 1). ShardedCid issues 8-byte IDs whose first byte maps back to the socket that issued them, so a viewer whose address changes stays on its session. tests/quic_sockets.rs: 32 viewers on 4 sockets all rebind mid-session and all keep receiving.
• examples/fanout_load.rs gains --relay-sockets. Its own client sockets now get 4 MiB buffers: without them, the harness dropped packets and looked like a relay collapse.

Measured (Linux, 32 threads, 8 relay cores, shared host at load 16–24, three rounds each):

	one socket	eight sockets
4,000 viewers (8 Gbit/s), p99	207–349 ms	25–36 ms
p99.9	388–605 ms	28–53 ms
relay drops	126k–384k	0
bystander track, median	0.15–38 ms	0.10–0.15 ms

• 8,000 viewers (16 Gbit/s) on eight sockets: 100% delivered in two of three rounds (90.8% in the third). p99 was 309–1,004 ms in the rounds that delivered everything, on 5.7–7 of 8 cores. That is about this machine's limit.
• 2,000 viewers with 256 KB keyframes: eight sockets drop nothing; one socket drops 107k.

Falsified: the steering program's offsets, its modulus and the CID mapping were each broken in turn. 5 of 5 were caught by tests/quic_sockets.rs, and the relay's 130 tests pass.

For deployment: a Linux relay needs net.core.rmem_max of at least 4 MiB, or the 4 MiB socket buffer is clamped to the kernel's limit. Only the operator can set that.

🤖 Generated with Claude Code

Step 2 of the relay's fan-out work, after #583. Two limits remained once an object was held once: one track's fan-out held up every other track, and one UDP socket took every packet of every viewer. **What changes** - **A fan-out holds up no other track.** Subscribers are a copy-on-write snapshot (`Arc<Vec<Arc<DownSub>>>`, written with `Arc::make_mut`). `fan_out_object`, `end_subgroup` and `fan_out_datagram` take the snapshot under the track's lock, drop the lock, and push to each subscriber outside it. A subscription that has left is marked `gone` under its own parameters' lock, so nothing is pushed to it after its PUBLISH_DONE. Unit test: `a_subscription_that_left_passes_nothing`. - **Each core has its own socket** (`limits.quic_sockets`; 0 = one per core up to 16, Linux only, otherwise 1). The sockets share a port with SO_REUSEPORT. A cBPF program steers each datagram by the first byte of its destination connection ID (long header offset 6, short header offset 1). `ShardedCid` issues 8-byte IDs whose first byte maps back to the socket that issued them, so a viewer whose address changes stays on its session. `tests/quic_sockets.rs`: 32 viewers on 4 sockets all rebind mid-session and all keep receiving. - `examples/fanout_load.rs` gains `--relay-sockets`. Its own client sockets now get 4 MiB buffers: without them, the harness dropped packets and looked like a relay collapse. **Measured** (Linux, 32 threads, 8 relay cores, shared host at load 16–24, three rounds each): | | one socket | eight sockets | |---|---|---| | 4,000 viewers (8 Gbit/s), p99 | 207–349 ms | **25–36 ms** | | p99.9 | 388–605 ms | **28–53 ms** | | relay drops | 126k–384k | **0** | | bystander track, median | 0.15–38 ms | **0.10–0.15 ms** | - **8,000 viewers (16 Gbit/s) on eight sockets:** 100% delivered in two of three rounds (90.8% in the third). p99 was 309–1,004 ms in the rounds that delivered everything, on 5.7–7 of 8 cores. That is about this machine's limit. - **2,000 viewers with 256 KB keyframes:** eight sockets drop nothing; one socket drops 107k. **Falsified:** the steering program's offsets, its modulus and the CID mapping were each broken in turn. 5 of 5 were caught by `tests/quic_sockets.rs`, and the relay's 130 tests pass. **For deployment:** a Linux relay needs `net.core.rmem_max` of at least 4 MiB, or the 4 MiB socket buffer is clamped to the kernel's limit. Only the operator can set that. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dcharlot added 1 commit 2026-11-02 19:17:03 -05:00

joule-moq-relay: a fan-out holds up no other track, and each core has its own socket ... dcfb9068a1

A big room's fan-out held the relay's one lock for as long as it took to reach
every subscriber, so every other room waited. A track's subscribers are now copy
on write: a fan-out takes a snapshot under the lock and delivers outside it, so
the time under the lock no longer grows with the audience.

With a snapshot, a fan-out could still push to a subscription that has just
left, which would put a stream after its cancel or past its PUBLISH_DONE count.
So a subscription is marked gone under its own lock, and a fan-out holds that
lock across each push: a push comes before the mark, or sees it and isn't made.

One UDP socket and one endpoint driver carried every peer's packets. The
listener now binds one socket per core on its one port, each its own endpoint.
Steering by address, which SO_REUSEPORT does by default, would end a session
whenever a viewer's address changed: its packets would reach a socket holding no
such connection. So a small kernel filter steers by the first byte of the
connection ID, and each socket's endpoint issues IDs that come back to it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot merged commit 9bbd6df851 into main 2026-11-03 00:48:02 -05:00

dcharlot referenced this pull request from a commit 2026-11-03 00:48:03 -05:00

Merge pull request 'joule-moq-relay: a fan-out holds up no other track, and each core has its own socket' (#587) from relay/fanout-shards into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Transaction-Science/open-standards!587

No description provided.