Some checks failed
joule-moq-room room service / joule-moq-room · does this change touch it (push) Successful in 22s
joule-moq-agent AI participants / joule-moq-agent · does this change touch it (push) Successful in 29s
WAI Studio / does this change touch it (push) Successful in 24s
wai-fleet service / does this change touch it (push) Successful in 43s
joule-moq-web browser client / joule-moq-web · does this change touch it (push) Successful in 53s
joule-moq-http live HTTP egress / joule-moq-http · does this change touch it (push) Successful in 59s
joule-moq-remote host agent / joule-moq-remote · does this change touch it (push) Successful in 58s
wai-app and Create / wai-app · does this change touch it (push) Successful in 54s
WAI breaking changes and the conformance runner / WAI breaking · does this change touch WAI (push) Successful in 28s
WAI examples / WAI examples · does this change touch them (push) Successful in 17s
wai-app and Create / wai-app · tests, clippy, the line language's verifier (linux) (push) Successful in 44s
WAI Studio / the catalogue's actions, the unit tests and the core · linux (push) Successful in 1m42s
wai-fleet service / service · linux (push) Successful in 4m31s
joule-code conformance gate / Build + round-trip C1-C4 vectors (push) Successful in 6m3s
WAI Studio / Studio in WebKit · linux (push) Successful in 31m13s
WAI breaking changes and the conformance runner / WAI breaking · wire, record and API against main · linux (push) Successful in 1m0s
WAI breaking changes and the conformance runner / WAI breaking and conform · their tests, and the independent verifiers through wai conform · linux (push) Successful in 13m56s
WAI examples / WAI examples · portable · linux (push) Successful in 22m38s
joule-code conformance gate / joule-moq-relay, every target (push) Failing after 8m42s
joule-moq-agent AI participants / joule-moq-agent, every target (push) Successful in 6m19s
joule-moq-http live HTTP egress / joule-moq-http, every target (push) Successful in 4m40s
joule-moq-remote host agent / joule-moq-remote, every target (linux-arm64) (push) Successful in 5m7s
repository checks / lockfiles, advisories, tests in CI, workflow hygiene, spec and section references, public text (push) Successful in 15m18s
joule-moq-room room service / joule-moq-room, every target (push) Successful in 13m30s
joule-code conformance gate / joule-moq-ingest, every target (push) Failing after 33m33s
wai-fleet service / service · macos (push) Successful in 42s
joule-code conformance gate / joule-moq-relay, its own energy on a counter (macos) (push) Successful in 1m20s
joule-moq-remote host agent / joule-moq-remote, every target (macos) (push) Successful in 1m44s
WAI Studio / Studio in Chromium and Firefox · macos (push) Successful in 11m15s
joule-moq-web browser client / joule-moq-web · browser end-to-end (macos) (push) Failing after 16m32s
wai-app and Create / wai-app · Create, browser end-to-end (macos) (push) Successful in 15m30s
WAI examples / WAI examples · android-app · linux-arm64 (push) Successful in 13m59s
WAI examples / WAI examples · the archive, the readiness check, the browser and Apple · macos (push) Successful in 16m47s
wai release binaries build on Windows / the seven release binaries · windows (push) Failing after 14m44s
On a 28-core macOS host the relay's records stopped at a fan-out of about 500, 25 meetings of four and datagrams to 1000, its relay at 4.3 to 4.6 of eight workers' cores whatever the audience: a sampled profile put the workers half their time in the send call of the listener's one socket, and most of their lock waits on the transport's per-connection lock, held across that send. Off Linux the listener's limits.quic_sockets (one per core up to 16) are now one endpoint's socket, transport::quic::SpreadSocket: bound to the one port, each peer's datagrams sent through one of them by its address (another where that one would block), and every one read, since such a system hands a datagram to one socket of its own choosing. Linux keeps one endpoint a socket, steered by connection ID. The vendored quinn's connection driver makes a connection's datagrams under its lock and hands them to the socket once the lock is let go, from a queue of its own, in order (vendor/quinn-proto/VENDORED.md; D48's licence). macOS has no public call that sends several datagrams (D72), so batching is not a lever there. Tests the old behaviour fails, each planted and caught on macOS (exit 101): quinn's own connection.rs (tests/quic_send_path.rs: a datagram held in the send call, the connection's statistics and a stream write waited); every peer through the first socket and only the first socket read (transport::quic's spread group test); one socket off Linux (tests/quic_sockets.rs). relay_bench takes --relay-sockets. Co-Authored-By: Claude Opus 5.5 <[email protected]>
117 lines
5.8 KiB
Rust
117 lines
5.8 KiB
Rust
//! The QUIC listener's UDP sockets: the sessions spread across them, and a viewer whose address
|
|
//! changes keeps its connection, because the kernel steers a packet by its connection ID and not by
|
|
//! its address (Linux). On the other Unix systems the sockets are one endpoint's, and what it sends
|
|
//! goes out through all of them, each viewer's through one; a viewer that moves is still read, since
|
|
//! every socket is. On a system with one socket, this holds trivially.
|
|
|
|
mod common;
|
|
|
|
use common::*;
|
|
use joule_moq::draft22::data::{SubgroupHeader, SubgroupId, SubgroupObject};
|
|
use joule_moq::draft22::{Parameters, UniEvent};
|
|
use joule_moq_relay::client::ClientOptions;
|
|
use std::time::Duration;
|
|
use tokio::time::timeout;
|
|
|
|
const SOCKETS: usize = 4;
|
|
const VIEWERS: usize = 32;
|
|
|
|
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
|
async fn a_viewer_whose_address_changes_keeps_its_connection() {
|
|
timeout(Duration::from_secs(120), run()).await.expect("the test finishes in time");
|
|
}
|
|
|
|
async fn next_payload(c: &joule_moq_relay::client::Client) -> Vec<u8> {
|
|
let mut r = timeout(STEP, c.next_data(STEP)).await.expect("in time").expect("a stream");
|
|
loop {
|
|
match timeout(STEP, r.next()).await.expect("in time").expect("a read") {
|
|
Some(UniEvent::SubgroupObject(o)) => return o.payload,
|
|
Some(_) => continue,
|
|
None => panic!("a stream with no object"),
|
|
}
|
|
}
|
|
}
|
|
|
|
async fn run() {
|
|
let server = relay(|c| c.limits.quic_sockets = SOCKETS).await;
|
|
let (publisher, _pub_ep) = connect(&server, Via::Quic, ClientOptions::default()).await;
|
|
let (space, track) = (ns(&["moved"]), name("video"));
|
|
let _published = publisher.publish(space.clone(), track.clone(), 1, Parameters::new(), Default::default()).await.expect("PUBLISH");
|
|
|
|
let mut viewers = Vec::with_capacity(VIEWERS);
|
|
for _ in 0..VIEWERS {
|
|
let (c, ep) = connect(&server, Via::Quic, ClientOptions::default()).await;
|
|
let held = c.subscribe(space.clone(), track.clone(), Parameters::new()).await.expect("SUBSCRIBE");
|
|
viewers.push((c, ep.expect("a QUIC endpoint"), held));
|
|
}
|
|
|
|
let per_socket = server.quic_connections_per_socket();
|
|
assert_eq!(per_socket.iter().sum::<usize>(), VIEWERS + 1, "{per_socket:?}");
|
|
if cfg!(target_os = "linux") {
|
|
assert_eq!(per_socket.len(), SOCKETS);
|
|
assert!(per_socket.iter().filter(|n| **n > 0).count() > 1, "the sessions spread across the sockets: {per_socket:?}");
|
|
}
|
|
|
|
let send = |g: u64| {
|
|
let h = SubgroupHeader { track_alias: 1, group_id: g, subgroup_id: SubgroupId::Zero, publisher_priority: Some(128), properties: false, end_of_group: true, first_object: true };
|
|
(h, [SubgroupObject::normal(0, g.to_le_bytes().to_vec())])
|
|
};
|
|
let (h, o) = send(0);
|
|
timeout(STEP, publisher.send_subgroup(h, &o)).await.expect("in time").expect("a send");
|
|
for (c, _, _) in &viewers {
|
|
assert_eq!(next_payload(c).await, 0u64.to_le_bytes());
|
|
}
|
|
// Off Linux, one endpoint's sends went out through more than one of its sockets: before, every
|
|
// datagram to every viewer went through one.
|
|
if cfg!(unix) && !cfg!(target_os = "linux") {
|
|
let sends = server.quic_sends_per_socket();
|
|
assert_eq!(sends.len(), SOCKETS, "{sends:?}");
|
|
assert!(sends.iter().filter(|n| **n > 0).count() > 1, "the sends spread across the sockets: {sends:?}");
|
|
}
|
|
|
|
// Every viewer moves to a new address. Steered by address, most of their packets would reach
|
|
// sockets that hold no such connection, and their sessions would end.
|
|
for (_, ep, _) in &viewers {
|
|
ep.rebind(std::net::UdpSocket::bind(loopback()).expect("a new socket")).expect("rebinding");
|
|
}
|
|
for g in 1..=3 {
|
|
let (h, o) = send(g);
|
|
timeout(STEP, publisher.send_subgroup(h, &o)).await.expect("in time").expect("a send");
|
|
for (c, _, _) in &viewers {
|
|
assert_eq!(next_payload(c).await, g.to_le_bytes(), "group {g}, after the move");
|
|
}
|
|
}
|
|
assert_eq!(server.quic_connections_per_socket().iter().sum::<usize>(), VIEWERS + 1, "no session ended");
|
|
}
|
|
|
|
/// Two relays never share a UDP port. A second relay asked for the port a first one holds is
|
|
/// refused, as on a system with one socket a listener; before, its sockets joined the first
|
|
/// relay's group, the kernel steered each relay's packets to the other's sockets, and a client
|
|
/// of one met the other's certificate ("invalid peer certificate: BadSignature", both being
|
|
/// self-signed under one name) or no answer. The same happened by chance between two relays
|
|
/// bound to port 0, in one test binary or two (`transport::quic`'s unit tests hold that case).
|
|
#[cfg(unix)]
|
|
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
|
async fn a_second_relay_is_refused_the_port_a_first_one_holds() {
|
|
use joule_moq_relay::transport::quic;
|
|
let first = relay(|c| c.limits.quic_sockets = SOCKETS).await;
|
|
let mut cfg = joule_moq_relay::RelayConfig::loopback();
|
|
cfg.limits.quic_sockets = SOCKETS;
|
|
cfg.quic_listen = first.quic_addr;
|
|
match joule_moq_relay::RelayServer::start(cfg).await {
|
|
Err(e) => assert!(e.contains("binding"), "{e}"),
|
|
Ok(second) => {
|
|
let ep = quic::client_endpoint(loopback(), vec![second.identity.chain[0].clone()], vec![joule_moq::draft22::ALPN.to_vec()], None).unwrap();
|
|
let met = match timeout(Duration::from_secs(5), quic::connect(&ep, second.quic_addr.unwrap(), "localhost")).await {
|
|
Err(_) => "no answer in five seconds".to_string(),
|
|
Ok(Err(e)) => e.to_string(),
|
|
Ok(Ok(_)) => "a session".to_string(),
|
|
};
|
|
panic!("two relays hold {}: a client of the second met: {met}", first.quic_addr.unwrap());
|
|
}
|
|
}
|
|
// The first relay is untouched by the refusal.
|
|
let (c, _ep) = connect(&first, Via::Quic, ClientOptions::default()).await;
|
|
c.close(0, "done");
|
|
first.shutdown().await;
|
|
}
|