open-standards/joule-code/joule-code-rs/crates/joule-moq-relay/tests/quic_sockets.rs
Transaction Science dbfa8b5aa1
Some checks failed
joule-moq-room room service / joule-moq-room · does this change touch it (push) Successful in 22s
joule-moq-agent AI participants / joule-moq-agent · does this change touch it (push) Successful in 29s
WAI Studio / does this change touch it (push) Successful in 24s
wai-fleet service / does this change touch it (push) Successful in 43s
joule-moq-web browser client / joule-moq-web · does this change touch it (push) Successful in 53s
joule-moq-http live HTTP egress / joule-moq-http · does this change touch it (push) Successful in 59s
joule-moq-remote host agent / joule-moq-remote · does this change touch it (push) Successful in 58s
wai-app and Create / wai-app · does this change touch it (push) Successful in 54s
WAI breaking changes and the conformance runner / WAI breaking · does this change touch WAI (push) Successful in 28s
WAI examples / WAI examples · does this change touch them (push) Successful in 17s
wai-app and Create / wai-app · tests, clippy, the line language's verifier (linux) (push) Successful in 44s
WAI Studio / the catalogue's actions, the unit tests and the core · linux (push) Successful in 1m42s
wai-fleet service / service · linux (push) Successful in 4m31s
joule-code conformance gate / Build + round-trip C1-C4 vectors (push) Successful in 6m3s
WAI Studio / Studio in WebKit · linux (push) Successful in 31m13s
WAI breaking changes and the conformance runner / WAI breaking · wire, record and API against main · linux (push) Successful in 1m0s
WAI breaking changes and the conformance runner / WAI breaking and conform · their tests, and the independent verifiers through wai conform · linux (push) Successful in 13m56s
WAI examples / WAI examples · portable · linux (push) Successful in 22m38s
joule-code conformance gate / joule-moq-relay, every target (push) Failing after 8m42s
joule-moq-agent AI participants / joule-moq-agent, every target (push) Successful in 6m19s
joule-moq-http live HTTP egress / joule-moq-http, every target (push) Successful in 4m40s
joule-moq-remote host agent / joule-moq-remote, every target (linux-arm64) (push) Successful in 5m7s
repository checks / lockfiles, advisories, tests in CI, workflow hygiene, spec and section references, public text (push) Successful in 15m18s
joule-moq-room room service / joule-moq-room, every target (push) Successful in 13m30s
joule-code conformance gate / joule-moq-ingest, every target (push) Failing after 33m33s
wai-fleet service / service · macos (push) Successful in 42s
joule-code conformance gate / joule-moq-relay, its own energy on a counter (macos) (push) Successful in 1m20s
joule-moq-remote host agent / joule-moq-remote, every target (macos) (push) Successful in 1m44s
WAI Studio / Studio in Chromium and Firefox · macos (push) Successful in 11m15s
joule-moq-web browser client / joule-moq-web · browser end-to-end (macos) (push) Failing after 16m32s
wai-app and Create / wai-app · Create, browser end-to-end (macos) (push) Successful in 15m30s
WAI examples / WAI examples · android-app · linux-arm64 (push) Successful in 13m59s
WAI examples / WAI examples · the archive, the readiness check, the browser and Apple · macos (push) Successful in 16m47s
wai release binaries build on Windows / the seven release binaries · windows (push) Failing after 14m44s
joule-moq-relay: off Linux the QUIC listener sends through several sockets on its one port, and a connection's datagrams are sent with its state let go
On a 28-core macOS host the relay's records stopped at a fan-out of about 500, 25 meetings of four
and datagrams to 1000, its relay at 4.3 to 4.6 of eight workers' cores whatever the audience: a
sampled profile put the workers half their time in the send call of the listener's one socket, and
most of their lock waits on the transport's per-connection lock, held across that send.

Off Linux the listener's limits.quic_sockets (one per core up to 16) are now one endpoint's socket,
transport::quic::SpreadSocket: bound to the one port, each peer's datagrams sent through one of
them by its address (another where that one would block), and every one read, since such a system
hands a datagram to one socket of its own choosing. Linux keeps one endpoint a socket, steered by
connection ID. The vendored quinn's connection driver makes a connection's datagrams under its lock
and hands them to the socket once the lock is let go, from a queue of its own, in order
(vendor/quinn-proto/VENDORED.md; D48's licence). macOS has no public call that sends several
datagrams (D72), so batching is not a lever there.

Tests the old behaviour fails, each planted and caught on macOS (exit 101): quinn's own
connection.rs (tests/quic_send_path.rs: a datagram held in the send call, the connection's
statistics and a stream write waited); every peer through the first socket and only the first
socket read (transport::quic's spread group test); one socket off Linux (tests/quic_sockets.rs).
relay_bench takes --relay-sockets.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 11:03:11 -05:00

117 lines
5.8 KiB
Rust

//! The QUIC listener's UDP sockets: the sessions spread across them, and a viewer whose address
//! changes keeps its connection, because the kernel steers a packet by its connection ID and not by
//! its address (Linux). On the other Unix systems the sockets are one endpoint's, and what it sends
//! goes out through all of them, each viewer's through one; a viewer that moves is still read, since
//! every socket is. On a system with one socket, this holds trivially.
mod common;
use common::*;
use joule_moq::draft22::data::{SubgroupHeader, SubgroupId, SubgroupObject};
use joule_moq::draft22::{Parameters, UniEvent};
use joule_moq_relay::client::ClientOptions;
use std::time::Duration;
use tokio::time::timeout;
const SOCKETS: usize = 4;
const VIEWERS: usize = 32;
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn a_viewer_whose_address_changes_keeps_its_connection() {
timeout(Duration::from_secs(120), run()).await.expect("the test finishes in time");
}
async fn next_payload(c: &joule_moq_relay::client::Client) -> Vec<u8> {
let mut r = timeout(STEP, c.next_data(STEP)).await.expect("in time").expect("a stream");
loop {
match timeout(STEP, r.next()).await.expect("in time").expect("a read") {
Some(UniEvent::SubgroupObject(o)) => return o.payload,
Some(_) => continue,
None => panic!("a stream with no object"),
}
}
}
async fn run() {
let server = relay(|c| c.limits.quic_sockets = SOCKETS).await;
let (publisher, _pub_ep) = connect(&server, Via::Quic, ClientOptions::default()).await;
let (space, track) = (ns(&["moved"]), name("video"));
let _published = publisher.publish(space.clone(), track.clone(), 1, Parameters::new(), Default::default()).await.expect("PUBLISH");
let mut viewers = Vec::with_capacity(VIEWERS);
for _ in 0..VIEWERS {
let (c, ep) = connect(&server, Via::Quic, ClientOptions::default()).await;
let held = c.subscribe(space.clone(), track.clone(), Parameters::new()).await.expect("SUBSCRIBE");
viewers.push((c, ep.expect("a QUIC endpoint"), held));
}
let per_socket = server.quic_connections_per_socket();
assert_eq!(per_socket.iter().sum::<usize>(), VIEWERS + 1, "{per_socket:?}");
if cfg!(target_os = "linux") {
assert_eq!(per_socket.len(), SOCKETS);
assert!(per_socket.iter().filter(|n| **n > 0).count() > 1, "the sessions spread across the sockets: {per_socket:?}");
}
let send = |g: u64| {
let h = SubgroupHeader { track_alias: 1, group_id: g, subgroup_id: SubgroupId::Zero, publisher_priority: Some(128), properties: false, end_of_group: true, first_object: true };
(h, [SubgroupObject::normal(0, g.to_le_bytes().to_vec())])
};
let (h, o) = send(0);
timeout(STEP, publisher.send_subgroup(h, &o)).await.expect("in time").expect("a send");
for (c, _, _) in &viewers {
assert_eq!(next_payload(c).await, 0u64.to_le_bytes());
}
// Off Linux, one endpoint's sends went out through more than one of its sockets: before, every
// datagram to every viewer went through one.
if cfg!(unix) && !cfg!(target_os = "linux") {
let sends = server.quic_sends_per_socket();
assert_eq!(sends.len(), SOCKETS, "{sends:?}");
assert!(sends.iter().filter(|n| **n > 0).count() > 1, "the sends spread across the sockets: {sends:?}");
}
// Every viewer moves to a new address. Steered by address, most of their packets would reach
// sockets that hold no such connection, and their sessions would end.
for (_, ep, _) in &viewers {
ep.rebind(std::net::UdpSocket::bind(loopback()).expect("a new socket")).expect("rebinding");
}
for g in 1..=3 {
let (h, o) = send(g);
timeout(STEP, publisher.send_subgroup(h, &o)).await.expect("in time").expect("a send");
for (c, _, _) in &viewers {
assert_eq!(next_payload(c).await, g.to_le_bytes(), "group {g}, after the move");
}
}
assert_eq!(server.quic_connections_per_socket().iter().sum::<usize>(), VIEWERS + 1, "no session ended");
}
/// Two relays never share a UDP port. A second relay asked for the port a first one holds is
/// refused, as on a system with one socket a listener; before, its sockets joined the first
/// relay's group, the kernel steered each relay's packets to the other's sockets, and a client
/// of one met the other's certificate ("invalid peer certificate: BadSignature", both being
/// self-signed under one name) or no answer. The same happened by chance between two relays
/// bound to port 0, in one test binary or two (`transport::quic`'s unit tests hold that case).
#[cfg(unix)]
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn a_second_relay_is_refused_the_port_a_first_one_holds() {
use joule_moq_relay::transport::quic;
let first = relay(|c| c.limits.quic_sockets = SOCKETS).await;
let mut cfg = joule_moq_relay::RelayConfig::loopback();
cfg.limits.quic_sockets = SOCKETS;
cfg.quic_listen = first.quic_addr;
match joule_moq_relay::RelayServer::start(cfg).await {
Err(e) => assert!(e.contains("binding"), "{e}"),
Ok(second) => {
let ep = quic::client_endpoint(loopback(), vec![second.identity.chain[0].clone()], vec![joule_moq::draft22::ALPN.to_vec()], None).unwrap();
let met = match timeout(Duration::from_secs(5), quic::connect(&ep, second.quic_addr.unwrap(), "localhost")).await {
Err(_) => "no answer in five seconds".to_string(),
Ok(Err(e)) => e.to_string(),
Ok(Ok(_)) => "a session".to_string(),
};
panic!("two relays hold {}: a client of the second met: {met}", first.quic_addr.unwrap());
}
}
// The first relay is untouched by the refusal.
let (c, _ep) = connect(&first, Via::Quic, ClientOptions::default()).await;
c.close(0, "done");
first.shutdown().await;
}