| Filename | Latest commit message | Latest commit date |
|---|---|---|
Two waits |
||
| .. | ||
| deploy | ||
| examples | ||
| src | ||
| tests | ||
| Cargo.toml | ||
| README.md | ||
| relay.example.toml | ||
joule-moq-relay
A relay for MOQT, draft-ietf-moq-transport-22, which also speaks -18 and -16 to
the sessions that choose them. Publishers send it tracks, subscribers
ask it for them, and it fans each object out to every subscription, in the order the
draft's priorities give, dropping what passes a subscriber's delivery timeout at this hop.
Whatever it drops, the object's receipts still reach the subscriber, on the track's
receipts track.
It is a library (Relay, RelayServer, client::Client) and a binary, wai-relay (the
crate keeps its name; the binary is named for WAI). The wire codec is joule-moq's.
RelayServer::shutdown sends GOAWAY (below) and returns once every session's task has ended,
its summary signed, and every port the relay listened on is closed: each UDP socket of its
endpoint (quinn closes one when its endpoint's driver ends), the WebSocket listener and the
control listener. A relay can start again on the same ports at once, in the same process.
RelayServer::start refused part way (a port another socket holds, a file in the control
socket's place) is shut down so before its error returns: every port it bound is free, and the
next start can bind them at once (Control::start likewise closes what it began).
The drafts it speaks
draft-ietf-moq-transport-23 and -22, and -18 and -16 for a peer that offers no
later one. Drafts do not interoperate, so a session speaks one from its first byte. The
draft is chosen before any MOQT byte, the relay preferring -23, then -22, -18 and -16:
- by ALPN on native QUIC:
moqt-23,moqt-22,moqt-18,moqt-16(§6.2.2); - by
WT-Available-Protocolson WebTransport, which the relay answers with theWT-Protocolit chose (§6.2.1;draft-ietf-webtrans-http3-16§3.3). A client that sends noWT-Available-Protocolsis taken to speak -22; one that offers only drafts the relay does not speak is refused; - by the subprotocol
moqt-23ormoqt-22on the WebSocket fallback (below), which carries those two.
The relay is written once, against -22. A session at -23, -18 or -16 runs behind
transport::bridge, which translates each message and object at the session's edge with
joule-moq's draft23::bridge, draft18::bridge and draft16::bridge: at -23 and -18
stream by stream (the drafts share their shape; at -23 every control message is reframed
with a varint Length and every fetch stream re-encoded, REQUEST_OK and FETCH_OK mapped by
the FETCH's range, and a refusal of the -23 codec closes the session with the code -23 gives
it); at -16 by giving each request on -16's one control stream a
stream of its own, its Request IDs renumbered into -16's sequence, MAX_REQUEST_ID granted
ahead, a cancelled stream becoming UNSUBSCRIBE, FETCH_CANCEL, PUBLISH_NAMESPACE_DONE or
PUBLISH_NAMESPACE_CANCEL. What a session's draft cannot say is refused (a request, with
REQUEST_ERROR) or left out, by name, and counted in the session's notes. Two rules differ
at the edge: at -18 a requester's FIN on PUBLISH_NAMESPACE, SUBSCRIBE_NAMESPACE or
SUBSCRIBE_TRACKS cancels it (-18 §6.1, §6.2), where at -22 a FIN is not a cancellation
(§6.4.2.2); and at -16 a new Request ID may arrive ahead of one still in flight on another
stream (SUBSCRIBE_NAMESPACE rides its own), which is admitted, where -16 §9.1 asks for the
next in sequence. Two rules the relay keeps by draft: a -23 session may hold namespace
subscriptions whose prefixes overlap (-23 §4.2), where an earlier draft's second is refused
PREFIX_OVERLAP; and a -23 requester that sends GOAWAY on its own request stream has its
session closed with PROTOCOL_VIOLATION (-23 §9.2). The session summary's draft names the
draft the session spoke on the wire. Each message in and out is logged at debug level. joule-moq's
README tables every difference. Receipts ride objects' Immutable Properties and receipts
tracks, which are the same on every draft, so they cross the bridge unchanged.
The relay reads the client's SETUP before it sends its own, at every draft
(draft-ietf-moq-privacy-pass-auth-04 §3.5.1; WAI live-rooms §10.5.2), so its SETUP says
what came of the client's. A client sends its SETUP as the session opens; one that waits for
the relay's first is sent nothing and is closed CONTROL_MESSAGE_TIMEOUT ten seconds on.
The relay's SETUP declares MOQT_IMPLEMENTATION, MAX_REQUEST_UPDATES,
MAX_AUTH_TOKEN_CACHE_SIZE, joule-moq's receipt extension (Setup Option 0x3A46, its
value the extension version; on for the session when the client declares it too), its relay
id (Cascading), GRANT_CHALLENGE (0x3A47): 32 bytes, fresh in every SETUP, which a
client's next SETUP here proves its room grants over; and, on a session it refuses,
GRANT_REFUSED (0x3A53) with the reason (below, Tokens and room grants).
What a -23 session is held to that a -22 one is not. A -23 session's subgroup streams have -22's bytes, and the bridge passes them as they are, so the relay's reader applies the object rules -23 changed where it reads them: a property the codec supports outside its scope (a Track-only one on an object) or more than once across the two property lists makes the track malformed, and a grease value in the Mandatory Track Property range is no Mandatory Track Property and passes. A -22 session keeps -22's reading of each: the first two pass, and the grease value makes the track malformed. The rules hold what a session sends: an object a -23 publisher sent with such a grease value is forwarded as it is, and a -22 subscriber's reader holds it malformed, as -22 reads it; an object a -22 publisher sent with a Track-only property is forwarded to a -23 subscriber as it is. A grant that expires ends its subscriptions and not the session at -23 (A room's authority), and relays peer at -23 as at -22 (Cascading).
LARGEST_OBJECT (-23 §7.5, #1978). The relay's is the largest of the LARGEST_OBJECT values its upstream publisher stated (in SUBSCRIBE_OK, in PUBLISH, and in PUBLISH_STATE_NOTIFY, which it did not take before) and the Locations of the objects it received on its upstream subscriptions, at every draft. §7.5 lets a relay compute it only from an upstream subscription that is unpaused and unfiltered or wide enough: every upstream subscription the relay holds is unpaused and unfiltered, and a track it holds none to is asked upstream. Two readings stand as they were. An object counts once the relay has it whole and its door has passed it, where -23 §3.1.4 has the Largest Object updated at an object's first byte: a subscription that joins while an object is arriving is delivered that object whole. And a SUBSCRIBE_OK states the live track's, never the DVR store's of an earlier publication of the track, which TRACK_STATUS and FETCH state where nothing is live.
Transports
| Path | Listener | Selected by |
|---|---|---|
| Native QUIC | UDP quic_listen |
ALPN moqt-23, moqt-22, moqt-18 or moqt-16 |
| WebTransport over HTTP/3 | the same UDP port | ALPN h3, extended CONNECT to path, WT-Available-Protocols |
| WebSocket fallback | TCP websocket_listen |
upgrade at path with subprotocol moqt-23 or moqt-22 |
Native QUIC and WebTransport share one quinn endpoint: the relay reads the ALPN of each connection and either runs MOQT on it directly or hands it to the WebTransport layer.
The WebTransport implementation is wtransport 0.7.2 (MIT OR Apache-2.0), vendored
(below). It was chosen because it runs on the same quinn 0.11 as joule-moq, so one UDP
endpoint serves both ALPNs; it gives the server the CONNECT request's headers and lets it answer with its
own, which is what WT-Available-Protocols and WT-Protocol need; and its client checks
a certificate by hash under the rules a browser applies to serverCertificateHashes
(ECDSA P-256, a validity of fourteen days at most), so the relay's tests connect the way a
page does. Stream bytes are read and written on the quinn streams underneath; application
error codes on WebTransport streams are mapped into the HTTP/3 error space as
draft-ietf-webtrans-http3-16 §4.4 gives.
Closing a WebTransport session. A MOQT session over WebTransport is closed by closing
the WebTransport session (draft-ietf-moq-transport-22 §6.6). The relay sends a
WT_CLOSE_SESSION capsule (0x2843) on the session's CONNECT stream, then a FIN
(draft-ietf-webtrans-http3-16 §6): the MOQT session termination code (§12.2) is the
capsule's 32-bit application error code as it stands, not mapped into the HTTP/3 space as
stream codes are, and the reason, at most 1,024 bytes of UTF-8, is its message. The QUIC
connection is closed with H3_NO_ERROR once the peer has closed its side of the CONNECT
stream, or two seconds after the capsule, so the connection's close cannot overtake it.
Nothing else of the session overtakes it either: a stream the relay lets go once the session
is closing (the client's control stream of a refused session, say) is held until the QUIC
connection has closed, rather than ended at once with a FIN or STOP_SENDING that would reach
the page before the capsule; a page whose stream was stopped just ahead of the close was
seen to lose its renderer. A page reads the code and reason from WebTransport.closed
(closeCode, reason), as it reads them from the WebSocket binding's WT_CLOSE_SESSION. A session the client closes is
read the same way: the relay's closed() gives the client's code and reason, and the
session's summary records the code as the peer's. A QUIC connection that closes with the
session still open gives no MOQT code; its code is an HTTP/3 error code.
The release of wtransport reads WT_CLOSE_SESSION but cannot send it, so the joule-code
workspace vendors 0.7.2 with a session close added (vendor/wtransport, patched in with
[patch.crates-io]; vendor/wtransport/VENDORED.md lists what changed and why).
How a browser connects
WebTransport, in development. Start the relay with --dev (or with no certificate
files in the configuration). It makes a certificate a browser accepts by hash and logs its
SHA-256 three ways, including a ready serverCertificateHashes entry:
const transport = new WebTransport("https://127.0.0.1:4443/moq", {
serverCertificateHashes: [{ algorithm: "sha-256", value: new Uint8Array([/* printed */]) }],
protocols: ["moqt-22"],
});
await transport.ready; // transport.protocol === "moqt-22"
The certificate is valid fourteen days from the relay's start; restart the relay to make a
new one. In production give it a certificate chain and key from files, and a page connects
without serverCertificateHashes.
WebSocket, where there is no UDP path. A site behind a front that carries HTTP over TCP only reaches no UDP endpoint, so neither QUIC path is open to its pages. Such a page uses the fallback:
const ws = new WebSocket("wss://relay.example/moq", "moqt-22");
ws.binaryType = "arraybuffer"; // each message is one capsule, below
The WebSocket listener speaks plain HTTP; TLS is the front's.
The WebSocket binding (non-normative)
This binding is this crate's, not a standard's. It carries what a WebTransport session over
HTTP/2 carries (draft-ietf-webtrans-http2-15), on a WebSocket instead of an extended
CONNECT stream, so that a client written for one is most of the way to the other.
-
Opening. The client opens a WebSocket (RFC 6455) to the relay's
path, offering the subprotocolmoqt-23ormoqt-22(or both) inSec-WebSocket-Protocol. The relay answers with the one it prefers of those offered, -23 first, and the session speaks that draft; it refuses the upgrade without either (400) or at another path (404). -
Messages. Every message is a binary message holding exactly one capsule:
Type (i) Length (i) Value, with QUIC variable-length integers (RFC 9000 §16), not MOQT's. A text message carries nothing and is ignored. A binary message that is not one whole capsule closes the session. -
Capsules. Those of
draft-ietf-webtrans-http2-15§6, with their type codes:Capsule Type Value Use here WT_STREAM0x190B4D3C,0x190B4D3Bwith FINStream ID, data stream data; a stream opens with its first WT_STREAMWT_RESET_STREAM0x190B4D39Stream ID, code, Reliable Size reset a stream (Reliable Size 0) WT_STOP_SENDING0x190B4D3AStream ID, code ask the peer to stop WT_MAX_STREAMS0x190B4D3Fbidi,0x190B4D40uniMaximum Streams stream-count credit WT_STREAMS_BLOCKED0x190B4D43,0x190B4D44Maximum Streams sent when out of credit; informational DATAGRAM0x00payload one MOQT datagram, reliable and ordered WT_CLOSE_SESSION0x2843code (32 bits), message (UTF-8, at most 1,024 bytes) close with a MOQT session termination code WT_DRAIN_SESSION0x78AEempty read and ignored; MOQT's GOAWAY is the drain others read and ignored, as unknown capsules are -
Streams. Stream IDs are numbered as QUIC numbers them: bit 0 the initiator (0 the client), bit 1 the direction (0 bidirectional), and the
nth stream of a kind is4n + kind. MOQT's control streams, request streams and data streams ride them byte for byte as they would ride QUIC streams: the client's control stream is its first unidirectional stream (ID 2), beginning with SETUP; a request is a bidirectional stream; a subgroup, fetch or padding stream is a unidirectional one. -
Stream credit. Each end first sends its
WT_MAX_STREAMSfor both kinds; until the peer's arrive, an end opens nothing. Credit is cumulative, as in QUIC. A stream's credit comes back once both of its halves are done, and is granted in batches. Opening past the limit closes the session with code0x045D4487. Because MOQT -22 bounds requests in flight by stream limits alone (§6.4.2), the relay's bidirectional limit (max_requests) is its bound on requests in flight on this binding, as MAX_STREAMS is on QUIC. -
Data flow control. None at the capsule level (
WT_MAX_DATA,WT_MAX_STREAM_DATAand theirBLOCKEDcapsules are read and ignored): TCP's backpressure bounds a sender, and a receiver that holds more thanwebsocket_buffer_bytesunread closes the session with0x045D4487. -
Closing.
WT_CLOSE_SESSIONwith a MOQT session termination code (§12.2), then the WebSocket's close. A WebSocket that closes without one is a session closed with code 0. -
Keeping alive. Each end the relay runs, accepting or dialling, sends a WebSocket ping (RFC 6455 §5.5.2) every 30 seconds, and the peer's WebSocket answers each with a pong. A front between the two that closes a connection it sees idle (a tunnel, a proxy) then keeps a session that has nothing to carry, such as a relay's link to its upstream while no one watches.
The first capsules a relay with the default limits sends are its credit:
99 0b 4d 3f 02 41 00 WT_MAX_STREAMS (bidirectional), 256
99 0b 4d 40 02 44 00 WT_MAX_STREAMS (unidirectional), 1024
and a session it closes at the end of a GOAWAY's timeout ends with:
68 43 12 00 00 00 10 47 4f 41 57 41 59 20 74 69 6d 65 6f 75 74
WT_CLOSE_SESSION, GOAWAY_TIMEOUT (0x10), "GOAWAY timeout"
What the relay does
| -22 | Here |
|---|---|
| §6.3 session initialization | Each end's control stream opens with SETUP; data streams that come first wait for it. A client's SETUP is held to §9.1's rules for a client (no PATH or AUTHORITY over WebTransport; no DELETE or USE_ALIAS token). The relay's control stream carries only its SETUP and a GOAWAY; the end of the peer's closes the session with PROTOCOL_VIOLATION, once nothing of the peer's publishing is left (each track it published ended, every data stream of its read) or after three seconds, so a publisher that closes its control stream as it leaves loses no object it sent. |
| §6.4.2 requests | A request stream must begin with one of the seven requests. A Request ID of the wrong parity or seen before closes the session with INVALID_REQUEST_ID. MAX_REQUEST_UPDATES is enforced per stream; requests in flight are bounded by the stream limit (max_requests). A request stream whose first message has not come is held until it comes or the session ends, with no timer, as a data stream without its header is: QUIC opens every stream below the highest a packet has named, so a request whose first packet was lost stands open and empty until QUIC sends it again; it holds no bytes and one of max_requests (tests/request_wait.rs). |
| §8.9, §9.1.3, §9.1.4 tokens | A receiver's alias cache per session: REGISTER, DELETE, USE_ALIAS, USE_VALUE; AUTH_TOKEN_CACHE_OVERFLOW, DUPLICATE_AUTH_TOKEN_ALIAS, UNKNOWN_AUTH_TOKEN_ALIAS; a registration in SETUP past the cache reads as USE_VALUE. Grants name a namespace prefix, rights to publish and subscribe, the SHA-256 of a token of type 0, and whether it is a room grant honoured only beside a proof of possession (Tokens and room grants). |
| §4.1, §4.2 namespaces | PUBLISH_NAMESPACE is accepted (REQUEST_OK) and stands until it is cancelled or its session ends. SUBSCRIBE_NAMESPACE answers with NAMESPACE for each published namespace under the prefix, then NAMESPACE and NAMESPACE_DONE as they change, each carrying the suffix after the prefix; an overlapping prefix in one session is PREFIX_OVERLAP. |
| §7.6, §9.8 PUBLISH | Accepted with REQUEST_OK; its Track Alias is the publisher's for its data. A second track under one alias in one session closes it with DUPLICATE_TRACK_ALIAS. PUBLISH_DONE ends the track's downstream subscriptions with its status, once the relay has read as many of the publisher's data streams as its Stream Count says it opened (they and the request stream are not ordered with each other, §9.9), or after three seconds, or at once when the count is 0 or unknown or the publisher's session is gone. A publisher's session that ends ends its tracks from their own request streams: with the status of a PUBLISH_DONE already read, else GOING_AWAY. A track that ended, by its publisher's PUBLISH_DONE whatever the status or by its session's close, and that a session then publishes is a new instance: what the relay held of the ended one (its cache, its Largest Location) goes, unless the DVR store keeps the track, where an object published again has to equal the one stored. No subscription spans the two: each was ended with PUBLISH_DONE when the last publisher went. A subgroup is taken from one upstream session only while a session publishes the track: once none does, none is owned, so a later publisher's objects are never counted duplicates of a session that has gone. |
| §7.4, §7.6, §9.6 SUBSCRIBE | SUBSCRIBE_OK only once an upstream subscription is Established (§7.4): one the relay holds, or a SUBSCRIBE it sends to every publisher of a matching namespace. The relay's own upstream SUBSCRIBE asks for every object at priority 128 with no timeout, copying no subscriber's priority, group order or timeout (§5.1.3, §9.20.3, §9.20.4). RENDEZVOUS_TIMEOUT holds a SUBSCRIBE for a publisher, up to max_rendezvous_ms; without it a track with no publisher is DOES_NOT_EXIST. Location Filters set where the subscription starts and ends. A publisher of a namespace that arrives later is sent a SUBSCRIBE for every subscription the relay holds under it. When the last subscriber leaves, the relay cancels its own upstream SUBSCRIBE. |
| §9.5 REQUEST_UPDATE | Subscriber priority (re-applied to what is queued), FORWARD, the Location Filter and both delivery timeouts. |
| §3.2, §3.4, §7.1, §9.11 FETCH and fills | A fetch the cache covers is served from it, groups in the requested order (coverage, below); otherwise from the cache and the DVR store together where they cover it (The DVR store, below); otherwise it goes to a publisher of the track or its namespace and its fetch stream is piped through, past the Fill Timeout a Timed-Out range. Where no upstream answers and the relay holds the track, it answers with what it holds, the rest of the range ended as Unknown (A FETCH no upstream answers, below). A range that starts after the Largest Object is INVALID_RANGE; one that starts below the live edge needs the seek power (Tokens and room grants). A SUBSCRIBE with FILL_PARAMETERS gets its fill range the same way on a fill fetch stream named by its Request ID; a fill it cannot serve is a fetch stream reset after its header (§3.4.1). |
| §9.13 TRACK_STATUS | Answered from what the relay holds: the Largest Object and the track's properties, from the DVR store when the track's publisher is gone (or before a restart). |
| §7.3 several publishers | A track may have several upstreams; each (group, subgroup) is taken from the first that sends it, and the rest are counted as deduplicated. When the stream a subgroup is taken from ends without a FIN (its session closed, or it reset the stream), the subscribers' streams of that subgroup are reset with its code, and the subgroup is released: what another publisher of the track sends of it past the last object forwarded goes to them on a new stream of the same Group and Subgroup (§2.2 allows one subgroup on two streams where one was reset), and what it sends of objects already forwarded is counted deduplicated. Objects that other publisher sent while the subgroup was still the first one's were dropped as duplicates and are not sent again: its new stream begins at the next object it sends. A track's last publisher going still ends it (tests/publisher_handover.rs). |
| §5.1 priorities, §5.2 delivery timeouts | Below. |
| §6.6.1, §9.2 GOAWAY | On shutdown every session gets GOAWAY with goaway_uri and the timeout; new requests are refused with GOING_AWAY; a session still open at the timeout is closed with GOAWAY_TIMEOUT. |
§6.5 .session |
With probes.enabled, the relay answers every request under .session itself (Probe tracks): nothing of it reaches a publisher, an upstream link, the cache or another session. Without it no session-level track is defined, and a request for one is DOES_NOT_EXIST. |
What the cache covers
The cache answers a fetch alone only where it holds every object of the range, so that a gap in its answer means an object that does not exist (§3.2.2). It keeps what it knows of each group's arrival. A group is whole when its start arrived (Object ID 0, or a Prior Object ID Gap saying the objects before do not exist, §10.9), none of its subgroup streams is open or was reset, and its end is known: its streams ended with a FIN, or an End of Group or End of Track status, or a datagram with END_OF_GROUP, ended it. A fetch from a group is covered when every group from it up to the group of the Largest Object is whole or known not to exist (a Prior Group ID Gap, §10.8), that last group has its start, and nothing in between was evicted. Coverage is worked out from the groups held each time it is asked, so a group whose streams finish after a later group's still counts once it is whole; a group ID never seen is a gap the cache does not answer across. The relay cannot tell a subgroup it has not yet seen from one that does not exist, so a group whose subgroups have all closed counts as whole.
A fetch that is not covered only because groups are still arriving waits for them: while the track's publisher is still sending and nothing from the fetch's start was evicted, the relay checks again every 20 ms, for up to the fetch's fill budget or one second, whichever is less, before it asks upstream. A track of one group per object, each on its own stream (a page's audio is a group every 20 ms), always has a stream or two in flight, and a publisher by PUBLISH need not answer a FETCH at all.
A FETCH no upstream answers
MOQT -23 §3.2.3: "If an upstream FETCH fails, the relay indicates the range of Unknown
Objects [...] and continues serving other known Objects." A FETCH's default range starts at
{0, 0}, and a relay cannot tell that a track whose first object is {5, 1} has nothing before
it: those Locations are Unknown to it (§2.1.2), so it asks upstream. Where every upstream it
may ask fails to answer (no answer within upstream_timeout_ms, TIMEOUT, INTERNAL_ERROR,
GOING_AWAY, EXCESSIVE_LOAD, or a link that is down), or it has none to ask, and it holds
the track, the relay answers FETCH_OK itself, ending at the range's end or the Largest Object:
| Of the range | On the fetch stream |
|---|---|
| A group the cache holds whole, or the DVR store holds | Its objects (a gap among them says an object does not exist, as in a fetch the cache covers). |
| A group known not to exist (a Prior Group ID Gap) | Nothing: an unmarked gap. |
| A group held from an object after its first, every stream of it ended with a FIN | End of Unknown Range through the object before the first one held, then its objects. |
| A group the relay holds nothing of, one a stream of which was reset, one still arriving | End of Unknown Range through the group's end; a run of such groups is one range. |
A group with a reset stream is not served in part: the relay cannot say which of its objects
are missing, and the relay's fetch entries are -22's, in which an End of Range cannot follow
an object of its own group. An upstream that answers with a refusal about the track
(DOES_NOT_EXIST, UNAUTHORIZED, INVALID_RANGE) is still passed on, and a track the relay
holds nothing of is still refused TIMEOUT no_upstream or DOES_NOT_EXIST: it has no End
Location to answer with. wai_relay_fetches_served_total{from="held"} counts these answers,
beside no_upstream (tests/fetch_unanswered.rs). A fill is unchanged: one the relay cannot
serve is a fetch stream reset after its header.
A media group and its receipts group leave the cache together, since a group served without its receipts could not be verified. Receipts are cached just before their media, so the cache's budget reaches them first. Evicting either moves the other track's mark to that group as well, even when the other's group has not arrived yet, so a late member of the pair is never served from the cache alone.
Priorities and delivery timeouts
Each downstream session has a scheduler. Every data stream it will send (a subgroup, a
fetch or fill, a datagram) is a lane whose items go out in order and one at a time, so no
stream's objects are reordered (§7.4). Among the lanes with something to send it takes the
item §5.1.2 puts first: the lower subscriber priority number; then the lower publisher
priority number; then, within a subscription, the group its Group Order puts first (and in
one group a fill before its subscription); then the lower Subgroup ID or, for a datagram,
Object ID, a datagram before a subgroup object. At most write_concurrency writes are in
flight, and a QUIC session's send window is quic_send_window_bytes: what waits, waits in
the scheduler, where it can still be ordered and dropped (§5.2: "minimize the amount of
data buffered at the underlying transport layer"). Each stream is also given a transport
priority from its subscriber and publisher priorities.
An object's deadline is its OBJECT_DELIVERY_TIMEOUT from when it reached the relay: the smaller of the publisher's (the first object of its subgroup, else the track's) and the subscriber's, where both are set. It is checked before the object goes to the transport and bounds both opening its stream and writing it. Past it the stream is reset with DELIVERY_TIMEOUT and nothing more of that subgroup goes to that subscriber; a datagram is dropped. SUBGROUP_DELIVERY_TIMEOUT runs from the subgroup's FIN until all its data is acknowledged, and resets the stream if it is not. On the WebSocket binding a FIN counts as acknowledged once written, as webtrans-http2 §5.2 makes the transitions an acknowledgement would make at once.
A subscriber that does not read
QUIC's connection-level flow control is shared by every stream of a session (RFC 9000 §4.1), and bytes a subscriber has received but not read hold it. So yes: a subscriber that stops reading stalls its own high-priority track along with everything else, until it reads again. And a subscriber that reads its streams one at a time, in the order they arrive, can stall itself while still reading: the stream it waits on may have its bytes held back for credit that the unread streams behind it are holding. A subscriber reads its streams concurrently.
The relay cannot release credit the subscriber holds, and does not try to by dropping what has no deadline. What it does:
- An object with no delivery timeout (a base track, a receipts track) is not dropped while its subscription stands. It waits, in the scheduler or the transport, and is delivered once the subscriber reads, up to the bound below.
- What can still wait, waits in the relay rather than in the transport (§5.2: "minimize the
amount of data buffered at the underlying transport layer"). Writes in flight are
bounded by
write_concurrencyand the transport's unacknowledged bytes byquic_send_window_bytes. So an object whose deadline passes while it waits is dropped before it takes any credit. - An object that passes its deadline is reset. A reset stream's final size settles its account (RFC 9000 §4.5), and a receiver can credit its unread bytes back to the connection at once, as quinn does. The credit a dropped layer held then comes back without the subscriber reading it.
- When credit frees, the highest-priority bytes go first: each stream carries its subscriber and publisher priority as its transport priority.
How far behind. MOQT lets a publisher end a subscription whose subscriber falls too far
behind, with PUBLISH_DONE TOO_FAR_BEHIND (§5.2), and the relay does, at a stated bound:
limits.max_queued_per_subscription (8192; 0 is no bound) items waiting for one
subscription in its session's scheduler. An item is an object, a datagram or the end of a
stream, queued as it arrived from upstream and not yet handed to the transport. What the
relay reads for the subscription at its own pace is not counted, because it reads no more
until the queue drains: a fetch, a fill, and the earlier Groups of a Rewind. A push that
finds the bound reached ends that subscription and no other:
- everything queued for it is dropped and counted
(
wai_relay_objects_dropped_too_far_behind_total); - each of its streams is reset
TOO_FAR_BEHIND(0x5), and a write that was waiting on the subscriber's credit is abandoned and its stream reset the same way, so no write is left holding the subscription's last word; - PUBLISH_DONE
TOO_FAR_BEHINDis sent on its request stream, reasonqueue_bound, with the streams opened for it, and the stream ends. A subscriber whose connection credit is all held by whole streams it has not read reads the PUBLISH_DONE when it reads them: the reset streams gave their credit back, the whole ones did not. Until it reads, the relay can send it nothing at all, and what bounds that is the stalled-session time below; - the subscription is counted (
wai_relay_subscriptions_too_far_behind_total), and the relay's upstream subscription goes with it when it was the last.
A session that grants no credit. A subscriber that stops reading still acknowledges, so
the QUIC idle timeout (idle_timeout_ms) never ends its session. limits.stalled_session_ms
(60000; 0 is never) does: once, for that long, the relay has had a write waiting on the
session (an object, a control message such as a PUBLISH_DONE, or a stream to open), the peer
has granted no flow-control credit and the relay has sent it no stream data, the session is
closed CONTROL_MESSAGE_TIMEOUT with the reason stalled_session, and counted
(wai_relay_sessions_stalled_total). A subscriber that reads, however slowly, grants credit
and is not closed; to one that is quiet the relay has nothing waiting; one that reads some
of its tracks is sent data and keeps its session, and the track it does not read is held to
the queue bound above. A subscriber that waits on its streams in the order they arrive (the
first paragraph of this section) holds the credit the stream it waits on needs: it has stalled
itself, the relay cannot tell it from one that reads nothing, and it is closed the same way. The counts are the QUIC transport's, on native QUIC and WebTransport.
Not bounded: the WebSocket binding keeps no such count, and a session there that stops
reading holds what is queued for it until its TCP connection ends
(tests/stalled_session.rs).
The session stands, with its other subscriptions, and the subscriber may subscribe again: it
is served from the live edge, or from a Rewind or a fill it asks for. A subscription to a
receipts track is held to the same bound: the receipts of what was dropped are fetched, as
any past group's are. The bound counts items and not bytes: an object's payload is held once
for every subscriber, so what one more queue costs is its bookkeeping, and the bytes are the
cache's to bound (tests/too_far_behind.rs).
Receipts
A media track's receipts ride its receipts track (WAI extensions/moq-streaming-format.md
§4.1): group g of it is one object holding the group receipt and the object receipts of
media group g. The relay recognises a receipts track by its name: the media track's name
followed by receipts_track_suffix (_receipts unless configured, the name WAI
extensions/live-rooms.md §6.1 gives a room's receipts tracks: 0_receipts for layer 0).
Each object of it is the group's receipts in the binary form of WAI
extensions/jwp-receipts.md §2.7.3 (WJR1): the group receipt as its canonical bytes, its
link hash computed over them as received, then the object receipts. The door reads that
form, and refuses a receipts object that does not read (§2.7.4) as it refuses one that does
not verify.
- The receipts of dropped objects are forwarded. A receipts track has no delivery
timeout, and the relay applies none to it, even one a subscriber asks for. So when an
object of a media track passes its deadline, its receipt still reaches every subscriber of
that track's receipts track, which can account for the object by inclusion (WAI
extensions/jwp-receipts.md§4.1, §4.4). A receipt that rides on the object itself, as a property, is lost with it. - Elsewhere receipts are opaque. Outside the namespaces a door rule names, the relay forwards receipts tracks as it forwards any track and reads nothing in them (§7.8).
- At the door. In a namespace a
[[receipts.door]]rule names, the relay reads the receipts it forwards, and takes on jwp-receipts §4.1's duties: each group receipt's signature is verified, under the rule's trusted keys or any key, before the first object of its group is fanned out, and a group that does not verify is dropped with a structured error in the log; each media object must match the receipt of its position (its group, its Object ID, its content hash, its Merkle proof to the signed root), and an object that carries its receipt as a property must carry that same one. An object waits for its group's receipts up tohold_ms, and its stream is not read meanwhile; a publisher whose receipts follow its group's last object can finish the group only while the whole group fits the stream's flow-control window,limits.quic_stream_window_bytes(16 MiB by default). What bounds the bytes one session leaves unread is not that window but the session's,limits.quic_connection_window_bytes, over all its streams together (64 MiB by default, the budgetwebsocket_buffer_bytesgives a WebSocket session, and never below the stream window): a peer that opens many streams and finishes none of their groups stalls there, however many streams it opens. Both windows hold as well for the links the relay dials to its upstreams, over native QUIC or WebTransport (tests/upstream_window.rs). Receipt bytes are forwarded unaltered and nothing is re-signed. A refused object ends its downstream streams rather than leaving a hole in them.
Load
examples/fanout_load.rs measures the fan-out: one publisher, N subscribers of one track over
native QUIC, and a bystander on a track of its own, whose lateness is what the audience costs
everyone else on the relay. It reports the relay's resident memory and CPU, how many objects
arrived, how late, and the packets the kernel dropped at each end. A fan-out delivers from a
snapshot of its track's subscribers, outside the relay's lock, so a large audience holds up no
other track. On Linux the QUIC listener binds limits.quic_sockets UDP sockets on its one port
(one per core up to 16 by default), each its own endpoint, and the kernel steers each packet by its
connection ID, so a viewer whose address changes keeps its connection. The first of them binds
alone and is opened to the others once it holds the port, so a listener's port is its own: a
second relay asked for it is refused, and one bound to port 0 is never given a port another
relay holds (tests/quic_sockets.rs). A deployment sets
net.core.rmem_max to at least limits.quic_socket_buffer_bytes (4 MiB by default): the relay
asks for that, Linux quietly gives no more than its limit, and the relay logs a warning. An object
is held once from when it is read until the last subscriber's QUIC stream has it acknowledged: the
cache, each session's queue and each QUIC send buffer share one allocation. A WebTransport or
WebSocket session, and a session bridged to an earlier draft, has its own copy in its transport.
The harness and its record
src/bench.rs is a load harness to the method of draft-evens-moq-bench-00: a configuration
profile of tracks (interval, objects a group, the first object's size and the others', duration,
start delay), the draft's START, DATA and COMPLETION messages as object payloads, the start
synchronized by START, and the draft's per-track metrics (objects and groups sent and received,
lost objects, durations, receive and publisher variance, average and expected bits per second),
driven by this crate's client. A track is a stream track, each object on a subgroup stream of
its own, or with "reliable": false a datagram track, each object one object datagram (at most
1100 bytes of data, refused before the run otherwise). The draft does not say how a datagram
track carries START and COMPLETION: each is a datagram as DATA is, and COMPLETION is sent three
times, as START is repeated, so one dropped datagram fails no subscriber; a DATA datagram that
does not arrive is a lost object and nothing is sent again. A run is one publisher and N
subscribers of every track (the draft's Scenarios 1 and 2) or, with
"meeting": {"meetings": M, "participants": P}, its Scenario 3: each of a meeting's P sessions
publishes every track, under a namespace field for its meeting and one for itself, and
subscribes to every track of the P - 1 others, and a track's result is over all its publishers
(publishers) and their subscribers.
A run's outcome is a measurement record: the profile, the figures, the host (a label the
operator gives, the operating system, the architecture, the processor count) and the date,
signed with a key the operator holds (examples/relay_bench.rs writes one; SignedRecord
reads one only under the key expected). It is a regression guard and nothing else: guard
holds a later run to an earlier record of the same host, profile and transport within stated
tolerances (by default no more objects lost, 20 ms more average receive variance, 250 ms more
on the longest gap, 10 percent less on the bit rate), names each figure outside them, and
answers not_comparable for records of two hosts or two profiles. No figure of a record is a
property of the relay: it is what one host measured on one day (tests/bench_harness.rs).
Denial of service
draft-englishm-moq-relay-dos-02 (2 November 2026, read against draft-ietf-moq-transport-23)
lists what a relay deployment has to bound. Each consideration, and what this relay does about
it; tests/dos_limits.rs and the tests named trip each limit that is the relay's own.
| The draft | Here | Past the limit |
|---|---|---|
| §3.2 sessions cheap to open in quantity | limits.max_sessions sessions in all. A session refused at SETUP is held open up to setup_refusal_wait_ms for its client to read the refusal, outside that count: at most max_held_refusals (256) are held at once, and max_held_refusals_per_address (8) from one address. |
A session past max_sessions is closed INTERNAL_ERROR. Past a bound on held refusals the oldest held (of the address, or of all) is closed UNAUTHORIZED at once (tests/dos_limits.rs). |
| §3.1.3 GOAWAY naming a victim | The relay sends GOAWAY only on its own shutdown, with the one configured goaway_uri; it follows no peer's New Session URI except to move an upstream link it dialed (Cascading). |
|
| §4.2 rapid request cycles | limits.max_requests_per_s (1000) requests a session a second, a second's worth at once; requests in flight by the stream limit, max_requests (tests/request_ids.rs). |
REQUEST_ERROR EXCESSIVE_LOAD, reason request_rate, with the Retry Interval after which a request is served; the session stands. |
| §4.2.1 update coalescing | MAX_REQUEST_UPDATES (max_request_updates, 16) outstanding a request stream. An update is applied as it is read: a priority change re-keys what is queued once, so updates are not coalesced, and their rate is bounded by the stream's credit alone. |
TOO_MANY_REQUEST_UPDATES closes the session. |
| §4.3 namespace advertisement flooding | limits.max_namespaces_per_session (256), PUBLISH_NAMESPACE and SUBSCRIBE_NAMESPACE together. SUBSCRIBE_TRACKS is not implemented and is refused. |
REQUEST_ERROR EXCESSIVE_LOAD, reason session_limit. |
| §5.1 slow subscribers | What waits, waits in the session's scheduler: writes in flight bounded by write_concurrency, the transport's unacknowledged bytes by quic_send_window_bytes; an object past its delivery timeout resets its stream (tests/slow_subscriber.rs). |
DELIVERY_TIMEOUT on the stream. Objects with no delivery timeout wait up to limits.max_queued_per_subscription (8192) items a subscription; past it that subscription's queue is dropped, its streams are reset TOO_FAR_BEHIND and it is ended PUBLISH_DONE TOO_FAR_BEHIND, the session standing (tests/too_far_behind.rs). A session the relay has a write waiting on, whose peer grants no flow-control credit and is sent no stream data for limits.stalled_session_ms (60000), is closed CONTROL_MESSAGE_TIMEOUT, reason stalled_session: this is what bounds a subscriber whose whole connection credit is held by streams it does not read, which can be sent no PUBLISH_DONE (tests/stalled_session.rs). Not on the WebSocket binding. |
| §5.2 join-time buffering | A fill or FETCH is read from the cache and the store as its stream drains, about a mebibyte at a time, under fill_timeout_ms. |
A fill fetch stream reset; a Timed-Out range. |
| §5.3 unknown Track Aliases | A data stream whose alias is not yet known is held unknown_alias_wait_ms, unread past its header, so the peer's stream window bounds its bytes and max_uni_streams their number; 64 data streams at most before SETUP. A data stream whose header has not come is held until it comes or the session ends, with no timer: QUIC opens every stream below the highest a packet has named, so one whose first packet was lost stands open and empty until QUIC sends that packet again, which on a path that drops packets is not the peer's delay; it holds no bytes and one of max_uni_streams (tests/late_stream_header.rs). A datagram whose alias is not yet known is held the same time, 64 of them a session at once (tests/first_objects.rs). What reaches a track while a subscription that found it with no upstream is still being attached is held for that subscription, 1024 deliveries a track, and let go when the last such subscription is attached or refused. |
The stream is stopped; past 64 before SETUP the session is closed PROTOCOL_VIOLATION. A datagram past the 64, or still unknown after the wait, is dropped (-23 §3.1.3.1). Past 1024 held deliveries they are let go, and a subscription being attached starts where it is attached. |
| §5.4 flow control across sessions | quic_stream_window_bytes and quic_connection_window_bytes a session (tests/connection_window.rs, tests/upstream_window.rs), websocket_buffer_bytes on the WebSocket binding, max_object_bytes an object, max_ingress_kbps a session. What all sessions hold together is bounded by its parts: payloads by the cache (cache.max_bytes; a queue shares the cache's allocation), queued items by max_queued_per_subscription × max_subscriptions_per_session × max_sessions, unread bytes by the windows × max_sessions. Not bounded: no single budget is held across sessions, because spending one would mean choosing which session's objects to drop, and the relay drops only by a rule a subscriber can read (a deadline, its own queue's bound); past max_resident_mb the binary exits for its supervisor. |
A WebSocket session past its buffer is closed; a session past its ingress rate is closed INTERNAL_ERROR. |
| Monitoring (Monitoring; not in the draft) | A served metrics or log subscription that falls 16 captures or 1024 log objects behind is ended. monitoring.log_messages_per_second (100) messages a second on the log tracks; qlog_max_events (1000000) events a session's qlog; feedback_tracks_per_session (16) Feedback Tracks a session, feedback_report_bytes (65536) a report, feedback_objects (100000) objects a track's tally; the latest 1024 observations in memory. |
PUBLISH_DONE TOO_FAR_BEHIND. A message past the second's allowance is dropped and counted, as is a qlog event past the bound. REQUEST_ERROR EXCESSIVE_LOAD, reason feedback_tracks_over; a report past its bytes is refused report_too_large; a tally past its objects ends its Feedback Track, feedback_objects_over. |
| §6.1 resource isolation | limits.max_subscriptions_per_session (1024) and max_tracks_per_session (1024); a scheduler a session, so one subscriber's queue delays no other's; a fan-out delivers outside the relay's lock. |
REQUEST_ERROR EXCESSIVE_LOAD, reason session_limit; nothing the session holds is ended. |
| §6.2 upstream load protection | Downstream subscriptions to a track share one upstream subscription; a FETCH the cache or the store covers goes nowhere; a request crosses at most max_relay_hops relays. Not bounded: FETCHes the relay cannot answer are forwarded one for one, because a FETCH's range, order and authorization are its requester's own and two are not one request; what holds them is each requesting session's request rate (max_requests_per_s) and stream limit, upstream_timeout_ms, and the upstream's own limits. |
relay_hops_exceeded. |
| §6.3 paused subscriptions | A subscription with FORWARD 0 counts as any other against max_subscriptions_per_session. The relay's upstream subscription stays unpaused while any downstream subscription stands, and is cancelled when the last leaves. |
As §6.1. |
| §7.1 cost tracking | The signed session summary counts each session's requests, objects and bytes; GET /metrics gives wai_relay_requests_refused_total{why="load"} with the other refusals. |
|
| §7.2 rate limiting | Limits are per session. Not bounded: none is per authenticated identity or per address but the held refusals, so an identity that opens many sessions is held only by max_sessions. The reason: the relay knows a session by its grant, and a grant's ceilings are the room authority's to set (hold_joule_ceilings, a grant's expiry, its revocation), while an address stands for many viewers behind one gateway; a count per address is the operator's front's to keep. |
|
| §8 load-bearing mechanisms | Stream limits (max_requests, max_uni_streams), the idle timeout, MAX_AUTH_TOKEN_CACHE_SIZE, MAX_FILTER_RANGES 0, the Full Track Name limits of the codec, max_rendezvous_ms, fill_timeout_ms, and nothing forwarded or cached of an object that closed a session or made its track malformed. MAX_CONTROL_MESSAGE_SIZE is not declared: the relay accepts the default. NEW_GROUP_REQUEST is not forwarded upstream. |
A hop link from another relay (Cascading) carries many sessions' requests and is not held to the per-session counts or the request rate. The limits state no capacity: what a host carries is measured on that host (Load).
Tokens and room grants
[[auth.grants]] rules name a token by the SHA-256 of its value (Token Type 0), a
namespace prefix, and the rights it gives there. With auth.required, a request is served
only where a token of its own or of the session's SETUP matches a rule.
A rule with proof = true names a room grant: the token's value is a WRG1 record (WAI
live-rooms §4.1). Every participant of a room holds every grant of it, to verify the room,
so the record proves nothing about who presents it, and the relay honours it only beside a
proof of possession of the grant's principal's key (live-rooms §10.5.2):
- every SETUP the relay sends carries a
GRANT_CHALLENGE, fresh, which the relay keeps (src/challenge.rs): good for the proofs of one later SETUP, at this relay, forlimits.grant_challenge_ms(60 s); - the client's SETUP, which the relay reads before it sends its own, carries the grant as
an AUTHORIZATION TOKEN and
GRANT_PROOF(0x3A49): aWRP1record naming the grant's digest and the principal's key, over a challenge an earlier SETUP of this relay gave it and the contextSETUP //<authority><path>of the URI it addressed, signed by the principal; - the relay checks it (live-rooms §10.5.4: the grant named, the key the grant's principal
— read from the record at its fixed place, since the rule names these exact bytes — the
signature under WAI's strict Ed25519 rule, a challenge the relay issued and still holds,
and the target it received: a WebTransport CONNECT's
:authorityand:path, a WebSocket upgrade'sHostand path, native QUIC's AUTHORITY and PATH) before it sends its SETUP or serves any request, and consumes the challenge once every proof of the SETUP has passed; - a SETUP that presents such a grant without a proof that passes is refused: the relay's
SETUP carries
GRANT_REFUSED(0x3A53) with the reason's code (pop_missing,pop_wrong_challenge,pop_wrong_key, …) beside the challenge, nothing of the session is served, and the relay closes itUNAUTHORIZED(§12.2) with the reason once the client has left orlimits.setup_refusal_wait_ms(2 s) has passed, since a close carries nothing sent before it; a grant first presented in a request, never proved in SETUP, gives that request nothing (REQUEST_ERRORUNAUTHORIZED).
So a client that holds no challenge of the relay presents its grant unproved, is refused
pop_missing and given one, and proves on a second session (Client::connect_proving);
one that holds the challenge of the relay's last SETUP to it, within its time, proves on its
first. The relay keeps the challenges it consumed for as long again, so a proof presented
again is pop_replayed, and one over a challenge it never issued, or one too old, is
pop_wrong_challenge; it holds at most 131072, the oldest dropped first. A challenge is this
relay's own: relays behind one name that share nothing each refuse a proof over another's
and give their own. Every refusal as not authorised at SETUP goes the same way (a revoked
grant, a grant past its joule ceiling, a relay id declared unproved). A rule without proof
is a bearer token, for a key the relay's operator gives a party directly (a room's clock, an
ingest gateway's link).
A room's authority. A room whose grants are issued as visitors arrive (a room service
such as joule-moq-room) cannot be listed rule by rule. [[auth.authorities]] names the
authority's key instead (and, optionally, the room ids it is honoured for): a token of type
0 that is a WRG1 record whose signature verifies strictly under that key (live-rooms §4.2;
joule-moq's GrantHead) is a room grant, honoured only beside a proof as above and only
within its window (nbf to exp), and mapped onto rights as live-rooms §10.2 maps it:
PUBLISH under [R, P, m] for each media kind m its publish bits allow, of [R, P] (its
catalog) when it may publish any, under [R, P, events] when its policy permits an
event kind, and under [R, P, world] when its attribute world is act (its world track,
WAI live-worlds §5.2); SUBSCRIBE, FETCH and the rest under [R] when it may subscribe. The relay's
rights are per namespace, so §10.2's track matches (layers up to max_layer) are not
narrowed. When such a grant expires, what it gave ends. At -23, whose PUBLISH_DONE has the
status EXPIRED_AUTH_TOKEN (0x7), each subscription the session holds in the grant's room
ends with it (Relay::grant_expired), its namespace subscriptions there end, its threshold
there falls to 0, and the session goes on: a later request by the expired grant is refused,
and another grant of the room it proved, still good, keeps what that one gives. A -23 session
that publishes by the expired grant is closed EXPIRED_AUTH_TOKEN, since no PUBLISH_DONE of
the relay's ends a publication. At -22, -18 and -16, whose PUBLISH_DONE has no such status,
the session ends, EXPIRED_AUTH_TOKEN, when the earliest such grant it proved expires, as
before.
Revocation (live-rooms §5). [[auth.revocations]] holds an authority's revocation list,
as its key log's head lists it in revoked_modules: grant digests (§4.2) and viewer tags
(SHA-256 over wai:live-viewer, 0x01 and the grant's 16-byte viewer), 64 hexadecimal digits
each. A grant whose digest or whose viewer's tag its authority's list holds is no grant: a
SETUP that presents it is closed UNAUTHORIZED, its reason live-rooms §5: grant_revoked,
before its proof is read. The relay reads no key log itself; the operator gives it the lists,
in the configuration, the grants file or on the operator's surface (below), where a revocation
also ends the sessions that presented the grant, in their SETUP or in a request, proved or not.
It refuses every use: a rule that names the grant gives it nothing, a request that presents it is
refused (REQUEST_ERROR UNAUTHORIZED, its reason live-rooms §5: grant_revoked), on a relay
that requires no authorization too, its session standing under what else it holds, and a list
holds when its authority is dropped. So that a revocation reaches a session by its requests, the
relay keeps every token of theirs a rule names, and 64 KiB of records no rule names
(relay::PRESENTED_BYTES); a request past that is refused UNAUTHORIZED. A list only grows
(§5: "a revocation stays"): a reload, or a set that does not repeat it, leaves it on the list,
and control.revocations_path carries it to the next start.
An authority's grant whose funds ceiling and currency break live-rooms §4.3 step 5's rule
(funds_ceiling is 2^64 − 1 exactly when currency is three zero bytes, and otherwise three
capitals) is no grant either: a SETUP that presents it is closed UNAUTHORIZED, its reason
live-rooms §4.3: grant_invalid.
Seeking back (live-rooms §10.6). The live edge of a track is the group of its Largest
Object. A FETCH whose range starts below it, or a SUBSCRIBE whose fill does, seeks back, and
with auth.required it is served only to a session holding the seek power: an
authority's grant whose powers hold bit 4 (§3.3), proved as above, or a [[auth.grants]]
rule with subscribe and seek. Any other is refused REQUEST_ERROR UNAUTHORIZED with
the reason phrase seek_not_granted. A FETCH that starts at the live edge, as a page joining
the group in progress does, needs nothing more. The relay's seek_refused counter counts the
refusals.
Age ratings (WAI age-assurance §3.6, §5.5, §5.7). A rating claim reaches the relay as the
property WAI_RATING (0x3A4D, its value the claim's text): a Track Property of a PUBLISH or
of an upstream SUBSCRIBE_OK, or an Object Property of an object it forwards. From the instant it
reads one, the relay holds it where the extension lets it honour it: a claim about a room signed
by an authority it trusts for that room; a claim about a track (from a group on, or a span of
groups) signed by such an authority or by the track's owner, the principal its namespace names,
and carried on that track, or carried anywhere in the room by the authority; an object's claim
it passes over. It keeps at most 64 claims a room or track and 65 536 in all, forgetting the
claims of rooms it no longer carries first. A session's threshold in a room is the greatest
age attribute (§5.4) of the room's grants its SETUP proved; 0 otherwise. A group's age is the
greatest age of the claims that cover it, and the relay serves the group to a session only when
the threshold reaches it: a live object of a group above it is not forwarded; when a newly held
claim raises a group above a subscriber's threshold, the streams of the groups it covers are
ended for that subscriber (CANCELLED) and nothing more of them is sent; a SUBSCRIBE that
forwards objects, a FETCH, or a fill whose first group is above it is refused, REQUEST_ERROR
UNAUTHORIZED with the reason phrase below_threshold; and one whose range reaches such a group
later is answered up to the group before it. The counters age_refused, age_withheld and
ratings_held count the refusals, the objects not forwarded, and the claims held.
The room's own service is not held to the rule (§5.5): a session whose SETUP carries a
GRANT_PROOF naming no grant (32 bytes of zero) by a room key, over a challenge of the relay's, is served
every object of that room. The relay checks the proof as a grant's proof (live-rooms §10.5.4,
the zero digest for the grant and the proof's key for the principal) and closes the session
UNAUTHORIZED for one that does not pass. A key it holds as a room key is an authority's of
[[auth.authorities]], for the rooms that authority is honoured for, or one [floor] room_keys
or [dvr] room_keys names, for every room; a proof that passes by any other key gives nothing.
The client presents such proofs with ClientOptions::room_keys; the room service's clock does,
so it follows and logs a rated room's tracks.
Patterns. A field * in a rule's prefix, or a door rule's, matches any one field: a
room's clock token can be given ["*", "<clock>", "events"], its events track in every
room. A door rule's trust_field names the namespace field holding the one key it accepts
there: prefix = ["*", "*", "video"] with trust_field = 1 holds each publisher's video in
a room to its own key (live-rooms §6.1), whoever joins; ["*", "*", "world"] does the same
for each actor's world track (WAI live-worlds §4.3).
Limits per session. limits.max_session_ms closes a session at that age (NO_ERROR,
the limit as its reason); limits.max_ingress_kbps closes one that sends more object bytes
than that rate allows over ingress_burst_ms (INTERNAL_ERROR). A session whose SETUP
presents a token whose rule says unlimited = true is exempt from both: a deployment's own
long-lived sessions, such as a room service's clock. limits.max_resident_mb makes the
binary exit with status 70 past that resident memory, for its supervisor to start it again.
Grants that change while the relay runs
The grants in force ([auth] and [[metering.grants]]) change while the relay runs, and a
change ends no session but those that presented a grant it withdraws:
-
The operator's surface (
[control],src/control.rs): a Unix socket (socket), made mode 0600, on which the relay serves a peer of its own user or root; or a TCP address on loopback (listen), on which it serves a peer on loopback. A peer of another user, or one off loopback, is answered with a refusal, and alistenaddress off loopback stops the relay at start. On a host with no Unix sockets,listenis the surface, andsocketstops the relay at start. A request is one JSON object a line, its answer one a line; a line that does not read as a request is answered and ends the connection:opfields showthe grants in force, and the grants this hop holds past their joule ceiling setauth(required,grants,authorities,revocations?),meteredthese grants in place of those in force. It states what it replaces: one that leaves out any of them but revocations, which only add, does not read and changes nothinggrantgrantadd an [[auth.grants]]rulewithdrawtoken_sha256withdraw every rule and metered grant naming the token, ending the sessions that presented it authority/drop_authorityauthority/keyhonour an authority's grants, or stop revokeauthority,revokedadd to an authority's revocation list metergrantadd a [[metering.grants]]entryreloadread the grants file again flushclose the window open now (below), wait flush_grace_ms, seal it at its own cut; answers the window, its span, its sessions and itscutwithcut_sha256closeclose the window open now and answer its own cut, sealing nothing: the first of a flush at one cut across relays (below)sealwindow,cutwait flush_grace_msand seal the closed window at the cut stated, answered asflush; refused, with nothing sealed, for a window that is not the next to seal or a cut below the relay's own (cut_too_early)windowwindow,room?the sealed window's counts by session, signed by the relay ( statement), bound to its records (below); withroom, that room's sessions and tracksrecordroom,t0_unix_us,window?the room's per-hop record over a sealed window, or over the sessions that ended summariesafter?,limit?the signed summaries not yet acknowledged above sequence number after, oldest first, at mostlimit(256; 4096 at most), each with itsseq; andheld,spilled(The session summary)ackthroughfree every summary through through, which asummarieshanded out; refused past what was handed outsessionsroom?the sessions open now, by room, signed by the relay ( statement): each with its id, the draft it speaks, its transport, when it opened and whether the room's objects are sealed (below); withroom, that room'sfilesdir,room?write the DVR store's window as MoQ files into dir(The DVR store, As files): an absolute path that does not exist or is an empty directory; answersdir,tracksandfiles, the.moqfiles written; withroom, that room's tracksA connection that opens with an HTTP/1 request line is answered as HTTP, once, after the same check of its peer:
GET /metrics(andHEAD, without the text) answers the relay's counters in the text exposition format (wai/wai-metrics), at the listener itself. A request carrying a field a front sets (Forwarded,Via,X-Forwarded-*,X-Real-IPand the like) is answered404, as is any other path, and another method405. The families, each written from the start:wai_relay_sessions,wai_relay_sessions_opened_total,wai_relay_tracks,wai_relay_subscriptions(the fan-out: downstream subscriptions across every track),wai_relay_queued,wai_relay_objects_in_total,wai_relay_fetches_served_total{from},wai_relay_door_refused_total,wai_relay_door_groups_dropped_total,wai_relay_requests_refused_total{why},wai_relay_age_withheld_total,wai_relay_subscriptions_too_far_behind_total,wai_relay_objects_dropped_too_far_behind_total,wai_relay_sessions_stalled_total,wai_relay_probes_total{direction},wai_relay_probes_refused_total,wai_relay_probes_ended_for_shape_total,wai_relay_probe_bytes_total{direction},wai_relay_upstream_total{outcome},wai_relay_ratings_held,wai_relay_going_away,wai_relay_energy_unmetered(1 when no meter samples the relay's own energy, so every record it signs carries the unmetered marker),wai_relay_summaries_held,wai_relay_summaries_spilled,wai_relay_summaries_spill_failed_total,wai_relay_summaries_unreadable,wai_relay_summaries_refusing(1 while new sessions are refused for summaries not acknowledged),wai_relay_checkpoints_written_total,wai_relay_checkpoints_failed_total,wai_relay_window_resumed(1 while the window open now is one the relay started again in),wai_relay_control_requests_total{op}andwai_relay_control_refused_total{why}. Every label value is fixed in the code: no room, track, principal, token or address is one. -
One request from a shell:
wai-relay --control <socket> '<request>'sends one request to the surface of a relay already running on the host and prints its answer on one line.<socket>is the path of itscontrol.socket, or the loopback address of itscontrol.listen. It starts no relay and changes nothing about the surface: the socket is the same one, mode 0600, and the relay checks this peer as it checks any, so the program runs as the relay's own user or root. It exits 0 where the answer is"ok": true, 1 where it is a refusal (the answer is printed all the same), 2 for a request it would not send (not one JSON object on one line, or a TCP address off loopback), and 3 where the surface could not be reached or did not answer within five minutes. An operator on an edge's own host so closes and seals that edge's window there (wai-relay --control /run/wai-relay/control.sock '{"op":"close"}', thensealwith the cut the relays that carry the room joined), and no edge is stopped with a GOAWAY at a window's end. -
The grants file (
control.grants_path):[[grants]],[[authorities]],[[revocations]]and[[metered]]in the forms of[[auth.grants]],[[auth.authorities]],[[auth.revocations]]and[[metering.grants]]. The relay reads it at start, its entries beside the configuration's, and again on SIGHUP or areload: the rules, authorities and metered grants in force are then the configuration's and the file's, in place of every change made since. Whether authorization is required stays as it stands, and every revocation stays. A file that does not read, or a change that does not check, changes nothing. -
The revocations file (
control.revocations_path): the relay appends each revocation it takes while it runs to this file, in[[revocations]]form, before it is in force, and reads the file at start; a file that does not read stops the start. Where the file cannot be written the revocation is put in force all the same, and the answer says it will not outlive this run. With none named, a revocation given while the relay runs lasts until it stops. -
The library:
Relay::set_grants,add_grant,withdraw_token,add_authority,remove_authority,revoke,add_metered_grantandreload_grants, andauth_configandmetered_grantsfor the grants in force (Relay::configis the configuration the relay started with).
A change is checked whole and put in force at once: a new session and a new request are held
to it, and so is every session open then. One that presented a grant the change no longer
honours (in its SETUP, or a rule's token in a request) or revokes, one whose SETUP token no
metered grant maps any longer, or one that holds a subscription or a publication its tokens no
longer give it (a rule moved to another prefix, a right removed), is closed UNAUTHORIZED, its
reason beginning grant_revoked, and every other session goes on, in every room, with no
GOAWAY. A session that loses a right it does not use goes on. A request admitted before a change
and placed after it, as a subscription that waited for its upstream is, is held to the grants in
force as it takes its place. A session that goes on runs, from the change, under the grant its
SETUP token maps to then. The limits a session was given at SETUP stand, and the floor and the
DVR store hold the room keys they started with.
An authority's grants metered with no configuration. A session that proved an authority's
grant and whose token no metered grant names runs under that grant, read from the grant itself:
its digest, its joules_ceiling, funds_ceiling and currency (live-rooms §4.1,
joule-moq's GrantHead), which its summary and the room's ledger name.
A session's grant in each room. A session that holds grants for several rooms runs in each
under that room's. An authority's grant states its room, and so does a token that is a room
grant's record and that a metered grant maps, with no authority configured: the metered grant
names exactly those bytes, so it vouches for the room they state (a relay that lists each grant
by a rule with proof and a metered grant). A token that is no record states no room: a session
runs under it in each room none of its grants names, and of two such tokens under the first. A
room's ledger therefore counts a session against that room's grant, or against a grant of no
room, and never against another room's.
Ceilings during the run (live-metering §4.6). A meter holds a use to its ceilings through the
room's own records, and this surface gives it what it needs while the sessions run: flush and
record state each grant's bytes and time in the room's ledger over a window, and revoke or
withdraw end a use. A funds ceiling is held by the meter that prices those records: the relay
holds no tariff and prices nothing, so it states each grant's funds_ceiling and currency in
the summary and the room's ledger, beside the bytes and time they are priced on, refuses a grant
whose two break live-rooms §4.3, and ends a use when the meter withdraws or revokes its grant. With
control.hold_joule_ceilings the relay also holds the joule ceiling itself, on its own reading:
at each seal it shares its figure for a room it served alone among the room's grants by the bytes
each moved, and a grant whose shares here pass its ceiling has its sessions closed UNAUTHORIZED,
reason joule_ceiling_passed, and its next SETUP refused the same. A hop's share is part of what
§4.6 sums over every hop, so a grant past its ceiling on it is past it. Nothing seals by itself but
what control.flush_interval_ms asks for, so a relay told to hold ceilings without that interval
does not start. A window with no figure is shared nothing, the shares count from the relay's
start and are held in memory, so a restart counts them from zero, and show lists the grants
held. It ends the hop's service to the grant; the room's records say when the principal leaves.
The open sessions, stated (sessions, src/sessions.rs, Relay::sessions_statement). A
supervisor that keeps a clock on a relay's sessions reads them from the surface: by room, each
session that holds something of the room now (a track it publishes or is subscribed to, a
namespace it published or subscribed to under the room's id), with its id (the one its summary
and its parts of a window carry), the draft it speaks (moqt-23), its transport, when it opened,
its peer relay where it is a hop link, and sealed: whether the room is one whose objects are
sealed (WAI confidentiality, an end-to-end encrypted room). The relay opens no object and knows
such a room as its DVR store does, by the first object whose properties carry the secure-objects
Key ID; every session of the room reads sealed from then on, since each holds the room's keys
for the epoch at its own end and a session moved elsewhere must come by them again. A session
that holds nothing of any room is listed under other. The statement is signed with the summary
key under a domain of its own (joule-moq-relay/sessions/1); an operator holds it to the key of
the relay it asked.
The open window kept on disk (control.checkpoint_path, src/checkpoint.rs). A relay
stopped by SIGTERM ends every session first, and loses nothing. One killed outright ends none:
without a checkpoint, the counts of the sessions open then are lost back to the last seal. With
checkpoint_path the relay writes its open window's counts there (one part for each session
that counted since the last seal, an open session's time cut at the instant of the write) every
checkpoint_interval_ms (1000), at each seal, and once more at a clean stop; each write is made
beside the file, synced, and renamed over it. The bound: a hard kill loses, of the sessions
open then, what they counted after the last checkpoint was taken: at most
checkpoint_interval_ms and the time one write takes. A write is of every part of the open
window, so it grows with the sessions open and the time since the last seal. Beside it,
<path>.sealed holds the last window sealed, written at the seal before the checkpoint that
follows. Both are signed with the summary key, so a checkpoint needs summary.key_path, and the
relay does not start without it.
At start the relay reads both. A checkpoint that reads whole, is signed by this relay's key and
is of a window it has not sealed gives the window back: the same index, the same start, and the
killed sessions' parts, which stand in it beside the new sessions'; the last sealed window is
kept and answered as before; session ids go on from where they were. The next window statement
of that window carries resumed: when the relay started again, when the checkpoint was taken,
and unmetered, the span from the checkpoint to the restart that no count holds. That span is
never stated as zero: the window's parts are a count of what was counted, the room's record over
it carries the unmetered marker for its energy, and a hop's shares of a joule ceiling count
nothing for it. A checkpoint that cannot be trusted is refused, the reason stated first in
resumed[].checkpoint_refused, none of its counts read, and the whole span since the last seal
stated unmetered:
| reason | the checkpoint |
|---|---|
checkpoint_truncated |
does not read whole |
checkpoint_other_relay |
names another relay's key |
checkpoint_bad_signature |
names this relay's key, and its signature does not verify |
checkpoint_stale |
is of a window this relay has sealed since: its counts are in that window, and read again would be billed twice |
sealed_unreadable |
cannot be placed, because <path>.sealed is there and does not read or is another relay's |
checkpoint_missing |
is not there, and a sealed window is |
A relay started again after a clean stop reads its window back with nothing uncounted, and
resumed says so by carrying no unmetered span. What a checkpoint does not keep: the sealed
windows before the last (an operator reads each window's statement before the next seal), the
DVR store's figures, and a hop's shares toward a joule ceiling, which count from the start.
Cascading: one room across relays
A relay can carry a room's tracks to and from other relays, so a room spans relays in
different places: the home relay behind an HTTP tunnel that carries WebSocket alone, say, and
an edge relay with public UDP. draft-ietf-moq-transport-22 leaves how relays interconnect out
of scope (§7) and says namespace publication "does not protect against loops" (§4.1); what the
relay does is below, and WAI live-rooms.md §10.7 is what a room needs of it. The module is
cascade.
Links. Each [[upstream]] entry is a relay this one dials, as a client:
[[upstream]]
name = "home"
url = "wss://home.example/moq" # moqt://host:4433/moq (QUIC), https://host:4433/moq (WebTransport)
transport = "websocket" # "quic", "webtransport" or "websocket"
prefixes = [["*"]] # the namespaces routed there; a field "*" matches any one field
native_roots = true # or ca_path = "<PEM file>", or cert_sha256 = "<hex>" (pinned)
token_sha256 = "<hex>" # the token presented in SETUP, named by its SHA-256 ...
token_path = "/etc/joule-moq-relay/home.token" # ... and read from this file
relay_key = "<hex>" # optional: the relay id the upstream must declare
A pinned certificate is trusted by the SHA-256 of its DER alone, whatever its validity or names,
its handshake signature still verified with its key. The token's file holds its value (trailing
white space removed), and the relay does not start when it does not hash to token_sha256. The
session is a MOQT session at -23, or at -22 where the upstream speaks no later (the link
offers both and the upstream chooses), in which each end publishes and subscribes; it is redialed
after it ends, waiting limits.upstream_reconnect_ms (1 s) and doubling to 30 s. Either relay
can dial the other: an edge relay dials the home relay's wss:// endpoint through its tunnel,
or the home relay dials an edge's UDP port over QUIC or WebTransport.
Relay ids. Every relay's SETUP declares the Setup Option RELAY_ID (0x3A4F): its id, the
public key it signs its session summaries and per-hop records with. A relay that dials proves
it: its SETUP carries a GRANT_PROOF naming no grant, by that key, over a GRANT_CHALLENGE
of the other's and the target it addressed, which the other checks as it checks a room key's
proof (Tokens and room grants). The other reads the dialer's SETUP before it sends its own,
so a link's first SETUP holds no challenge: it is refused (relay_id_unproved), its answer
carries one, and the link dials again at once and proves over it; each later dial proves over
the challenge the last SETUP gave, and dials twice only where that one is too old. A declared
id without its proof is so refused UNAUTHORIZED, and a relay's session to itself closed
PROTOCOL_VIOLATION (relay_loop). Every other client ignores the option (§9.1). A session
with both ids is a hop link, at -23 or at -22: RELAY_PATH has the same form at both
(joule-moq's README), read by the -23 codec in the messages -22's reads it in and carried by
the bridge both ways; on a session that negotiated none it is an unknown parameter, which
closes a -23 session INVALID_PARAMETER and a -22 one PROTOCOL_VIOLATION. At -18 and -16
there is no form for it: a relay id declared there is left unread and the session is an
ordinary one. Relays that peer are upgraded together: one that sends its SETUP first and one
that waits for the other's open no link.
Routes. A SUBSCRIBE, FETCH or TRACK_STATUS for a track that no session of this relay
publishes (by PUBLISH, or under a namespace published by PUBLISH_NAMESPACE) is forwarded on the
first link, in the configuration's order, that routes the track's namespace and whose session
is open and not going away. It is forwarded once: the subscription it makes is the relay's own
upstream subscription of the track (§7.4), cached and fanned out to every local subscriber as a
publisher's is, and a RENDEZVOUS_TIMEOUT goes upstream with the time left of it. A link that
refuses (TIMEOUT, INTERNAL_ERROR, GOING_AWAY, EXCESSIVE_LOAD) or does not answer is
passed over for the next; when every link that routes the track is down, the request waits for
one within its rendezvous, and is then refused REQUEST_ERROR TIMEOUT with the reason phrase
no_upstream. A FETCH of a track the relay knows nothing of asks the upstream for its live edge
first (TRACK_STATUS), so the range it forwards is the request's own and the seek power is held
here (below).
Announcements. Every namespace published to this relay under a link's prefixes, by PUBLISH_NAMESPACE or as the namespace of a track a publisher sent by PUBLISH, is published on the link (PUBLISH_NAMESPACE), so the upstream's subscribers are sent here (§7.6), and is withdrawn when it goes. A SUBSCRIBE_NAMESPACE whose prefix overlaps a link's is forwarded on it, as each link's session opens, and the upstream's NAMESPACE and NAMESPACE_DONE, of the namespaces the link routes, are merged with this relay's own: one NAMESPACE when a namespace is first seen from either, one NAMESPACE_DONE when the last source of it goes.
No loops. On a hop link the Message Parameter RELAY_PATH (0x3A51) is negotiated (-22
§9.20: "negotiated via Setup Options"): every request one relay sends another carries the
relays it has passed through, the sender last. A relay never forwards a request back on the
session it came from, nor to a relay on its path, nor past limits.max_relay_hops (8); a
namespace is published on no link to a relay its path holds. A request that arrives naming the
relay that receives it is refused relay_loop, one past the bound relay_hops_exceeded
(UNINTERESTED for a PUBLISH_NAMESPACE, DOES_NOT_EXIST otherwise), and one that could only be
sent toward a relay on its path is refused DOES_NOT_EXIST relay_loop at once. RELAY_PATH on
a session that did not negotiate it, or whose last relay is not the peer, closes the session
PROTOCOL_VIOLATION. A relay-id path rather than a hop limit alone: the path stops a request
at the first relay it would repeat and names the relay that closed the loop, and its length is
the hop limit.
GOAWAY. An upstream's GOAWAY ends nothing downstream: the relay sends that session no new request (§9.2), opens a session to the GOAWAY's New Session URI (one of the link's own scheme; else the link's URL), subscribes there to every track it held through the old session (§7.6.1: what arrives on both is deduplicated, §7.3), publishes its namespaces there, and closes the old session. It does not pass the GOAWAY on to its own sessions. A GOAWAY from a relay that dialed this one stops new requests to it. The relay's own GOAWAY on shutdown names no URI on a session it dialed (a client's, §9.2).
What each relay holds to its own rules. The door checks what a peer forwards as it checks
a publisher's: a tampered object one relay forwarded is refused at the next relay's door. A
link runs under its token's rule: the home relay names the edge's token in [[auth.grants]]
with publish, subscribe and seek over the routed prefixes, and unlimited; the edge
grants its upstream the right to subscribe to what it routes there. Each relay holds its own
sessions to their grants (rights, the seek power, a rated track's threshold); a hop link proves
no room grant, so its threshold is 0 and a rated room is not carried across one. The floor and
audience inputs are decided at the relay the performer or the author publishes to: an object or
an input that arrived over a hop link is forwarded as that relay decided, and its events are read
for their schedules, handovers, windows and commitments alone (WAI co-performance.md §8). The
counters forwarded_upstream, no_upstream and relay_loops count the requests forwarded and
Established upstream, those no upstream answered, and those refused for their path; each link
keeps its dials, sessions, GOAWAYs, moved subscriptions and announcements (Link::stats).
The session summary
When a session ends the relay signs a summary of it with its own Ed25519 key (from
summary.key_path, or made at start and its public half logged), keeps it until its operator
acknowledges it (below), and appends it as one JSON line to summary.out_path if set. It
holds, for the session:
- as a subscriber: objects and their bytes forwarded, every byte written to its data streams, objects dropped for their delivery timeout and how many of those were on a track whose receipts track the session also held, subgroups reset after their FIN, objects served by fetch or fill, objects still queued at the end, and the same per track, with each track's fetched bytes, objects still queued at the end and objects of Object ID above 0;
- the spans of its layer class in each WAI room it subscribed to: the highest layer of the room's media tracks it held, or 8 when it held none of them but other tracks of the room;
- as a publisher: objects and bytes received, objects refused at the door, objects deduplicated, and the same per track;
relay_energy:{"acquisition": "unmetered", "joules_micro": 0}. A session's own share of the relay's energy is not read: the counter covers the whole relay process, in which sessions run together (a room's record carries the relay's figure where it can, below). The 0 is the unmetered marker of WAIextensions/energy-measurement.md§2, never a measurement of zero joules, and must not be read or shown as one;publisher_signed: where the door verified an object's receipt, the publisher's signed figure for producing it, summed under the acquisition class its leaf was signed with. These are the publisher's statements about the objects, not the relay's energy;room_grant, when the token the session presented in SETUP is one a metered grant names, or an authority's grant it proved: the room grant's digest and its ceilings;room_grants, by room, the grant it ran under there: an authority's grant it proved for that room, or a metered grant whose token is a record of that room;peer, when the session is a hop link: the other relay's id, hex.
The summary is relay-scoped telemetry (jwp-receipts §4.2), not a receipt of the publisher's.
Its format is joule-moq-relay/session-summary/2; the signature covers
joule-moq-relay:session-summary\x01 followed by the summary's JSON bytes, and
SignedSessionSummary::verify checks it.
Drained, and bounded (src/outbox.rs). The relay holds each summary, numbered from 1 in each
run, until its operator acknowledges it: the operator reads them in order (summaries on the
surface, Relay::drain_summaries), keeps them where it bills from, and acknowledges through the
last it kept (ack, Relay::ack_summaries); the relay frees every one up to there.
Relay::summaries is those not yet acknowledged. An acknowledgement past the last summary
handed out is refused and frees nothing. At most summary.held_max (65536) are held in memory.
Past it:
- with
summary.spill_path, each further summary is appended to that file whole, synced, and held there: the drain reads it back in order with the rest. How much of the file is acknowledged is recorded beside it (<spill_path>.acked), and the file is emptied once all of it is. A relay that starts and finds summaries there not acknowledged hands them over first, and none it acknowledged, so a restart loses no spilled summary and bills none twice; a line that does not read (one torn as the process died) is kept aside in<spill_path>.unreadableand counted. - with none, or where a write to it fails, the summary is held in memory all the same, and the
relay admits no new session until the operator acknowledges enough to bring what it holds in
memory back to the bound: a SETUP is refused, closed
INTERNAL_ERROR, reasonsummaries_unacknowledged. The sessions open go on, and each one's summary is kept.
So no summary is ever dropped, and memory holds at most the bound and the sessions open when it
was reached. Refusing new windows instead would bound nothing (each session that ends still adds
its summary) and would stop the billing of the sessions still open; refusing new sessions serves
no one whose use could not be billed. What is held in memory does not outlive the process:
summary.out_path keeps every summary on disk as it is signed. The relay's first flush puts in
window 0 the ended sessions whose summaries are still held.
The per-hop session record
Relay::hop_record(room, t0_unix_us) signs WAI's per-hop session record, WMH1
(wai/extensions/live-metering.md §3), of the room whose id is room, over the sessions
that ended, on the room clock of the room's t0. A track is the room's when its
namespace's first field is the room id in hexadecimal (WAI live-rooms §6.1); it is a media
track when its namespace ends audio, video or screen and its name is a layer 0 to
7, an events track when its namespace ends events and its name is 0, and another
track of the room otherwise. The record holds:
- per media track and direction, and for the events tracks and the other tracks together: the objects and bytes received and sent, the objects dropped for their delivery timeout, refused at the door, deduplicated and still queued at the end, and those of the dropped whose receipts the relay forwarded;
- its subscriber sessions, their microseconds of each layer class, and the events it delivered to them;
- the root of its ledger, one entry for each room grant its sessions ran under (or none, for sessions with no grant): the sessions, the bytes in and out and the subscriber time;
- its energy over the span (below).
Its energy. The relay reads a counter the host keeps, every 100 ms while it runs, and
keeps four hours of samples (src/energy.rs):
- on macOS, the relay process's own CPU energy,
ri_energy_njfromproc_pid_rusage(getpid(), RUSAGE_INFO_V6), unprivileged: scope task, every thread of the relay and nothing else (not other processes, the GPU, the network interface, or work a system service does for the relay); - on Linux, the CPU packages' RAPL counters through powercap, scope package: the whole
package, every process on it. It is read only with
[metering] dedicated_host = true, the operator's statement that the host runs the relay alone, since it is otherwise not the relay's energy; - elsewhere, or with
[metering] energy = "off", none.
A record carries the counter's advance over its span as OnChipCounter, filtering state
Unknown, with a declared uncertainty (the interpolation at each end of the span, the
counter's lag at each end, for a package counter the noise a filtered counter's vendor
states for the window, and the rounding to whole microjoules) over the span as its window,
when three things hold: the counter was live across the span (WAI energy-measurement §3.1:
it advanced in two consecutive intervals, in at least five, and in four of every five, over
at least 300 ms); the relay served that room alone in the span (every session that
overlapped it has ended and carried tracks of that room and of no other); and the figure is
at least one microjoule. Otherwise the record carries the unmetered marker, and the relay
logs why. A figure of the task scope covers the relay's CPU and nothing past it; a figure of
the package scope covers the whole package of a host the operator stated was the relay's
alone.
As a session closes, the relay ends its subscriptions one at a time, each when its request
stream fails; the moments between are classed by what the session still holds, so a
session whose media subscriptions end before its receipts or events ones has a span of
class 8, typically microseconds, at its end. HopRecord::to_bytes is the record and
HopRecord::ledger_bytes the ledger beside it. A hop link (Cascading) is counted in the
track entries, its objects in at one end and out at the other, and is neither a subscriber
session nor in the ledger (live-metering §3.10); every other session of the relay that
subscribes is a subscriber session. Nothing in the record is per object or per viewer, so its size depends on the
room's published tracks and not on its audience. tests/room_metering.rs checks that the
record recomputes from the summaries.
Windows, with sessions open (live-metering §3.8). Relay::flush closes the window open now
at that instant and opens the next, with no GOAWAY and no session ended, and from a relay's first
flush each session's tally counts by window beside its whole. Until then there is one window,
and its counts are the sessions' own, so a relay never flushed holds no second copy of them; the
first flush puts everything counted so far in window 0. control.flush_interval_ms has the relay
flush and seal by itself; without it a window closes only when asked. An object is counted in
the window that was open when its group's first object reached the relay, so each track is cut
at a group boundary: the
rest of a group that began before the flush, arriving after it, is counted in the window it
began in. Time on a room's tracks is cut at the flush's instant. Relay::seal takes every
session's counts in a closed window, those of the sessions still open among them, a grace
after the flush (flush_and_seal, and the surface's flush, wait control.flush_grace_ms);
an object of a sealed window's group counted after that goes to the window open then, and an
object served by FETCH or a fill is counted in the window open when it is sent
(live-metering §3.8). What becomes of an object counted (its refusal at the door, its
deduplication) stands in the window that counted it, sealed or not, so an entry's drops stay
among its objects. Each count goes to the whole and to one window, so a session's windows sum
to its summary. A relay with a DVR store states in a window's record the objects and bytes it
has written since the window before, as far as the window's in entries count (§3.6, §3.9):
what a window cannot state waits for the next with objects in.
Relay::window_record(room, t0, window), and the surface's record, sign the room's record
over a sealed window: its span is the window's, so records over successive windows do not
overlap, and its energy is the relay's own over the window where it served that room alone
in it, read from its own meter. The relay keeps its latest 64 sealed windows, and an ended
session's counts in a window only until that window is sealed.
A window's cut (src/window.rs). A window ends, track by track, at a group: its cut
names, for each track (rendered), the first group of the window after it, and the window holds
the groups below it that the window before did not. Relay::flush fixes the relay's own cut,
for each track the group after the largest that began here before the flush's instant
(Relay::cut), and Relay::seal seals at it, so an object is counted in the window that was
open when its group's first object arrived, as above. A group first seen while a closed window
waits for its seal is counted under a provisional window of its own, and the seal places it:
in the closed window when it is below the cut, else in the next, every count of it moving with
it, so a session's windows still sum to its summary.
One cut across relays (live-metering §3.8, §3.10). Relays that carry a room together, a core
and its edges joined by hop links, close a window at one cut, and none sends a GOAWAY: the
operator sends each a close (Relay::flush), joins the cuts they answer, track by track the
greatest (Cut::join), and sends each a seal with the join (Relay::seal_after,
Relay::seal_at). Each relay's own cut is above every group that began there before its
instant, so no relay has counted a group at or above the join in the window it closes, and a
group that began between two relays' instants is counted in the same window at both. A relay
refuses a cut below its own for a track (cut_too_early), with nothing sealed: it counted a
group at or above it in the window already. Then each link chains window by window: what one
end counted out over it, the other counted in (window::link_chain, from the two relays'
statements below). Two relays that each cut at their own instant count a group that began
between the instants in different windows, and their records of the link state, in one window
or the next, more in at one end than the other sent (§11's contradiction). An edge so rides
through a window's end: its viewers are sent nothing, and its link lives across windows. Until a
closed window is sealed, every group first seen waits for its cut, so the operator seals what it
closes; a relay whose windows close at one cut has no control.flush_interval_ms, whose own flush
would seal the closed window at the relay's own cut first.
A window's counts by session (Relay::window_statement, the surface's window). What a
room's record over a window is made from, which the record is not: each session's counts in
the window (who published which media under which grant, who subscribed to which layer from
when to when), the cut, and the DVR store's writes the window states, format
joule-moq-relay/window/1, signed with the relay's summary key, the key its records name as
their hop, over joule-moq-relay:window\x01 and the statement's JSON bytes. An operator that
closes a room's window from it checks it against the room's record over the window
(SignedWindow::check_record): both verify under the one key, and the record recomputes from
the parts (metering::counts_over), but for the relay's own energy, which its meter states,
and the DVR store's byte-seconds and drops; parts that do not give the signed record are
refused (parts_mismatch). So a relay in a process of its own hands its operator what the room's
lifecycle is written from, bound to the record, and its windows close with nothing lost.
The DVR store
With [dvr] configured the relay keeps, on disk, a window of the tracks under each
[[dvr.window]] prefix with their receipts, and serves it to FETCH below the live edge (WAI
extensions/live-recording.md §8). Its module is dvr.
What it keeps. Every track a window's prefix matches (the rule with the longest prefix keeps it): media tracks, their receipts tracks, events tracks, catalogs, takes. A group is stored once the cache holds it whole (What the cache covers), with every field a FETCH carries. For a track the store keeps, the cache does not evict a group while a stream of it is still arriving, past its budget by that group if it must, and it takes a group whose streams finish after a later group was evicted (coverage is unchanged: nothing at or below an evicted group is covered), so the store gets every group whatever order its streams finish in.
What it never keeps. A room whose objects are sealed (WAI confidentiality.md: an
end-to-end encrypted room) is never recorded, and a relay keeps nothing of its tracks beyond
delivery (§9; live-recording §8.2). The store knows such a room by its first object whose
properties carry the secure-objects Key ID (SECURE_OBJECT_KEY_ID, read without a key), or
whose properties do not read. That object's group is never written. From then on the store
writes nothing of the room and deletes what it held of it: the room's events and receipts
tracks are in the clear and can be whole before its first sealed group. It also drops the
room's decisions from decisions.wdl, while still honouring them for what the cache serves.
The room is recorded in sealed.wdl (room(32) check(32), check the SHA-256 of room), so
a restart keeps refusing it; rooms_sealed counts them.
A room in which a minor publishes (WAI age-assurance.md §8.4) is never kept either. The relay
knows one by a minor's publishing grant: a session whose SETUP proved a grant of a configured
authority whose publish is not 0 and whose attribute minor is true. From then on the store
treats the room as a sealed one: it writes nothing of it, deletes what it held and its
decisions, and records it in sealed.wdl, which so holds every room never kept;
rooms_sealed counts these too. A minor's grant that publishes nothing changes nothing.
The window. Each [[dvr.window]] has a max_age_s, from when a group's first object
reached the relay, of 1 to 86 400 (24 hours, the start-over buffer a live-to-VOD harvest clips
from), and a max_bytes of payload over its tracks together. Past either the oldest group goes
first, whole, and group g of a media track goes with group g of its receipts track. A group
that becomes whole after the window has moved past its number is not stored (counted as
groups_late).
On disk. Under path, one directory per track, tracks/<the first 16 bytes of the SHA-256 of its rendered name, hex>/, holding TRACK (namespace, name, properties, and a SHA-256
check) and the track's segments, s<n, 20 digits>.wds: files each group's record is
appended to as it is stored, a new one begun every ten seconds or 16 MiB. A segment is a
sequence of frames, len(u32) state(u8) record, the state 1 for a record that stands and 0 for
one a later record replaced (a group written again: an object erased, or a subgroup that came
late), whose bytes are then overwritten with zeros. A record is
header "WDG1" 1 track(32) group(u64) arrived_us(u64) count(u32) erased(u32) check(32)
object id(u64) subgroup(u64, 2^64-1 for a datagram) priority(u8) status(u8)
props_len(u32) payload_len(u32) digest(32) check(32) props payload
erased id(u64) digest(32) why(u8: 1 redacted, 2 did not verify)
Integers are little-endian. track is the SHA-256 of the rendered track; the header's check
is SHA-256 over the header before it; an object's digest is the content hash its receipt
names it by (jwp-receipts §2.1), and its check SHA-256 over its fixed fields before check,
its properties and its payload. A segment is deleted once no group's standing record is in it,
so on disk a window holds up to a segment of each track more than its budget. decisions.wdl
beside tracks/ holds the moderation decisions the relay honoured, each a length, the WEV1
record and its SHA-256.
Durability. The writer syncs every segment it wrote, and the directories it made files in, at least once a second and before it answers a flush or stops; a shutdown flushes. A crash loses at most the last second's groups. (One sync per group could not keep up with a room's audio, a group every 20 ms per publisher.) On start the relay reads every segment again:
- a frame a crash cut short, at a segment's end, is cut off (
torn_writes); - a record whose header does not check is dropped;
- every object is checked: its
checkanddigest; on a media track whose receipts track is stored, against group g's receipts (the group receipt's signature under the door's trust for the namespace, or any key that verifies where no door rule covers it; the object's receipt at its position; its Merkle proof to the signed root; its content hash); on an events track, every record under its principal. One that does not verify is dropped, counted (objects_dropped, and theWMH1record'sdvr_dropped), and its group written again with it among the erased. It is never served.
What is below the first group the store holds of a track after a restart is unknown to it. Each object's digest is checked again as it is read.
Answering a FETCH. A FETCH the cache covers is answered from it, as before. Otherwise, where every group of the range is held whole by the cache or the store, known not to exist, or evicted from the store, the relay answers from the cache and the store together, groups in the requested order, reading the store's groups as the fetch stream drains (about a mebibyte at a time). Otherwise it asks upstream. A FETCH of the receipts track brings each group's receipts the same way, so a viewer checks a recorded object exactly as a live one.
What is no longer there. Evicted groups are marked with an End of Unknown Range
(0x10C, MOQT §11.4.1.2) through the last of them: a range partly evicted gets what remains
and the marker, one wholly evicted the marker alone. Not an unmarked gap and not an End of
Non-Existent Range: an evicted object existed, and an unmarked gap would say it did not (MOQT
§3.2.2). An object of a group that cannot be served (dropped, withheld) is marked by one End
of Unknown Range from the group's start through the last such object, and the group's objects
after it follow. (An End of Range is a Location, MOQT -22 §11.4.1.2, so one could mark a single
object inside a group; joule-moq read it as moving to a later group until 11 November 2026,
which allowed only this form, and the form is kept.)
Moderation. The relay reads the decision events (WAI live-events §4.11) of the events
tracks it forwards in a kept room. It honours one whose record verifies, whose principal owns
the track ([R, P, events]), and whose principal is a room key (dvr.room_keys, or a key of
[[auth.authorities]]) or proved a grant with the moderate power on its publishing session.
The last honoured decision on a target stands. Deny and redact withhold the target from every
FETCH and fill, from the store and from the cache, for every session but a moderator's; redact
also erases it from its group's file; allow restores what deny withheld. A media object is
named by its receipt's digest, an event by its record's digest: a withheld event is left out of
the events-track object that carries it, the other records byte for byte, and an object left
with none is withheld whole. Live subscriptions are not filtered here (Not yet implemented).
Metering. A relay with a store signs WMH1 version 2 (live-metering §3.9): the objects and
bytes its sessions were served from the store (dvr_objects, dvr_bytes in each summary's
tracks), and from the store's own series, kept a point a second, the objects and bytes it wrote
for the room, the bytes it held integrated over the span in byte-seconds, and the objects it
dropped. Dvr::counts gives the store's counters, Dvr::figures a room's figures over a span.
Configuration.
[dvr]
path = "/var/lib/joule-moq-relay/dvr"
room_keys = [] # a room's authority and clock, hex; the authorities' keys count too
[[dvr.window]]
prefix = ["*"] # a field "*" matches any one field
max_age_s = 900
max_bytes = 4294967296
The store must be kept where the room's region pin allows (live-rooms §3.7).
As files. Dvr::write_files writes the window of the tracks named (or of every track the
store holds) as .moq and .dat files (draft-jennings-moq-file-05; WAI live-recording §8.7,
joule-moq's file module): each object's Location, forwarding preference, priority,
properties and payload byte for byte, with the instant its group reached the relay. An object
withheld or erased is left out, as is one with a status other than Normal. A reader that holds
the files and the publisher's key verifies a track against its receipts track with nothing
else (tests/dvr_files.rs). The operator asks for it with files on the control surface
(Grants that change while the relay runs): wai-relay --control <socket> '{"op":"files","dir":"/takes/room-1","room":"<room>"}' writes that room's tracks, or without
room every track the store holds, and answers the .moq files written. dir is an absolute
path that does not exist or is an empty directory: a request overwrites nothing, and is refused
(dir_not_empty, dir_not_absolute, no_store, no_tracks) before anything is written.
The Rewind filter
draft-duke-moq-subscribe-rewind-02: a subscriber joins a few Groups back with a
subscription's delivery (WAI extensions/live-recording.md §8.8). The relay's SETUP declares
MAX_REWIND (limits.max_rewind_groups, 8) at -23 and -22; a filter that asks more closes the
session PROTOCOL_VIOLATION. The code points are the numbers the draft states, 0x16 three
times, requests and not assignments, in joule_moq::rewind alone.
- The Groups before the current one come from the DVR store, each subgroup on a stream of its own, ended; the count in REWIND_GROUPS stops at the first Group the store does not hold or the serving rule refuses. No store for the track, or no seek power: 0.
- The current Group comes from the cache from its first object, queued as the subscription joins, so an object that arrives afterwards follows on its subgroup's stream; the stream ends when the subgroup ends upstream.
- An object the store will not serve (its bytes no longer hold its receipt's hash, withheld,
taken down) is a hole: nothing of its Group at or after it is sent and the Group's streams
are reset
UNKNOWN_OBJECT_STATUS. - Not rewound, and so answered as a Next Group filter: a subscription with a delivery timeout in force, a FORWARD 0 subscription, a current Group the cache does not hold from its start. Objects delivered as datagrams are not sent again. A rewound object keeps the HopTime it arrived with.
- A fill beside a Rewind ends before the Rewind's first Group. The relay sends no Rewind upstream. -18 and -16 carry none of it: toward them the option and the parameter are left out and the filter is the Next Group filter.
tests/dvr.rs rewinds a track and its receipts and verifies every object, with the planted
cases the text names.
Probe tracks
draft-gurel-moq-probing-00 (read whole at -00, the revision in the IETF archive on 11
November 2026) lets an endpoint measure the hop between itself and its relay with a track of
padding whose name states its shape. joule_moq::probe holds the names, their resolution,
the limits and the refusals; with probes.enabled the relay serves them (src/probes.rs).
It is off by default: a probe is traffic the relay generates or accepts on request.
A request whose namespace begins .session is answered by the relay before any lookup of a
publisher or a link. It holds no track state for one: nothing is cached, nothing is
forwarded, and no other session can ask for it.
| The request | What the relay does |
|---|---|
SUBSCRIBE (.session, .probe) catalog |
The probe catalog (§9.2): one object, the configured presets with their selectionParams and probeName, and no limit. The subscription stands until its subscriber leaves. |
SUBSCRIBE (.session, .probe) and a Probe Name, or the empty namespace and .probe/ before the name |
A downlink probe (§9), from Group 0, Object 0 whatever the filter: each group on its own stream with Subgroup ID 0, publisher priority 255, a LOC Timestamp (0x10) on every object, a payload of zeros. A paced probe releases object i at i / f. A greedy one sends each object once the one before is with the transport, never faster than max_rate_bps, and ends at max_duration_ms where it names no d. What is still queued a second after the probe's time is dropped. Then PUBLISH_DONE TRACK_ENDED. A subscriber ends a probe by cancelling its request. |
PUBLISH (.session, .sink) and a Dynamic Probe Name, FORWARD 0 |
An uplink probe (§10): REQUEST_OK, then at once a REQUEST_UPDATE that sets FORWARD 1 (PAUSED 0 at -23), which is how the transport drafts spoken say the draft's "PUBLISH_OK with Forward State 1": their answer to a PUBLISH carries no forward state, and a subscriber resumes a paused subscription by an update. Then each object is measured as it arrives and discarded. A probe that sends more than its b bytes, has not ended a second after its d (or max_duration_ms), or sends faster than its rate (a greedy one, max_rate_bps) by uplink_rate_margin_percent, counted over the time since it was accepted plus one second, is ended: its streams are stopped and its request cancelled. The session stands. |
| Any other request in those two namespaces | REQUEST_ERROR NOT_SUPPORTED, reason probe_request_not_supported. |
Any other request under .session |
DOES_NOT_EXIST (a PUBLISH or PUBLISH_NAMESPACE: UNINTERESTED), reason session_track_unknown. |
A refusal carries the code the draft's Table 6 gives for the direction, the refusal's own
reason (probe_duration_over_limit, probe_session_gap, ...; joule_moq::probe lists
them), and, for a limit that can be met later, the Retry Interval after which it is expected
to be. Nothing is clamped. The limits are Table 5's, each direction held to them separately
([probes] in the configuration): 5000 ms, 10 000 000 bytes, 50 Mbit/s and 1 MiB an object a
probe; one probe at a time a session, a second between two, 50 000 000 bytes a minute. The
two the table leaves to the operator default to 16 probes at once and 200 Mbit/s across all
sessions, and max_objects_per_second (1000), which the table does not name, keeps a probe
of one-byte objects from being millions of them.
Readings where the draft leaves room, beside those joule_moq::probe states: a session the
relay admitted may use probes, since where the relay requires authorization the session's
SETUP met it; an uplink probe's measurement is logged and not reported, the draft having no
way to report one yet (§10.4); the bytes of an uplink probe refused for its Forward State are
stopped at their streams and not counted against the session's probe bytes; and uplink
support is not advertised (§10.3 is open). Probe bytes are not in a session's signed summary,
which counts objects of tracks: /metrics has them (wai_relay_probes_total{direction},
wai_relay_probes_refused_total, wai_relay_probes_ended_for_shape_total,
wai_relay_probe_bytes_total{direction}). A probe's own objects wait in the session's
scheduler behind everything of a higher priority, as the draft intends, and are not held to
max_queued_per_subscription: the probe's own byte limit bounds them.
The same path serves any track of the relay's own: it answers the SUBSCRIBE under an alias it
mints and queues the objects in the subscriber's scheduler (probes::Served).
Monitoring
Four drafts carry what an operator watches a deployment by (WAI moq-streaming-format §4.8,
delivery-telemetry §11, decision D14): draft-jennings-moq-metrics-03,
draft-jennings-moq-log-04, draft-pardue-moq-qlog-moq-events-08 and
draft-liu-moq-feedback-01. joule_moq's metrics_track, log_track, qlog and
feedback hold the formats and every refusal; with [monitoring] the relay uses them
(src/monitoring.rs). Each part is off by default.
The relay is a resource. Its ResourceID is the first eight bytes of SHA-256 over
wai moq resource id, a zero byte and the public half of its summary key
(joule_moq::metrics_track::resource_id): a key its summaries and per-hop records already
carry, and never an address. The two kinds of track are served as probe tracks are
(probes::Served): to the one subscription that asked, from the group that begins after
it was accepted, nothing cached, nothing forwarded upstream, no FETCH and no TRACK_STATUS
(NOT_SUPPORTED). A session subscribes with a grant that covers the namespace, as to any
track, and nobody else publishes there (UNAUTHORIZED).
| Part | What the relay does |
|---|---|
metrics_track |
Under (metrics.moq.arpa, v1, the ResourceID as 16 hexadecimal digits): the catalog track and the data track 6 (informational, priority 224). Every metrics_interval_ms (1000, at least 100) the relay captures the text GET /metrics answers at that instant (control::exposition, the same function) and publishes it as one group: the Group ID is the capture time in milliseconds. A reader that holds a group and the text refuses the first number in which they differ (metrics_track::equal_to_exposition). The capture counts itself first, so it is in its own text (wai_relay_monitoring_captures_total). A new series is a new catalog version, sent on the catalog track before the data group; a catalog subscriber is sent the version in force first. A capture the track's rules refuse (a label value outside letters, digits and _, a family that is no counter or gauge) publishes nothing and is counted by why. |
log_track |
Under (logs.moq.arpa, v1, the ResourceID): a track a severity, 0 to 7, one group a second in which the relay logged at it, ended when a later message comes or its second has passed. The relay logs session_opened and session_closed at 6, with the session's number, its transport and its close code, and feedback_refused at 4 with the reason; an embedder logs with Relay::monitor().log. HOSTNAME and PROCID are left empty: nothing names the machine. A message with an attribute whose value is a network address, alone or with a port, is refused address_in_attribute, counted and never published. It is not the relay's tracing output: only what is logged through Monitor::log is on a track. |
qlog_dir (feature qlog) |
One file a session, <session number>.sqlog: JSON Text Sequences, the header record, then an event a record. Logged: the client's SETUP, every control message of every request stream both ways, and every subgroup stream's header and objects both ways, lengths and never bytes. Each time is taken under the lock its record is written under, on one monotonic clock, so a log's times ascend. stream_id is the order in which the relay took the stream up on the session, from 0, not the QUIC stream ID: the transports are behind one interface that does not state it, and the WebSocket binding has none. Messages are logged as the relay reads and writes them, which at -23, -18 and -16 is after the bridge to -22's forms. Not logged: the relay's own SETUP and GOAWAY, fetch streams, datagrams. A relay built without the feature holds no qlog code and does not start with qlog_dir set. |
feedback |
The relay's SETUP declares MOQT_FEEDBACK with SUBSCRIBE_FEEDBACK and OPTIONAL_METRICS (the placeholder 0x3A80). A PUBLISH whose namespace ends (mqf, hop-feedback, a source id) is the relay's to consume: accepted (REQUEST_OK, and a REQUEST_UPDATE that resumes it where it began with FORWARD 0) from a session that declared PUBLISH_FEEDBACK and holds a subscription here to the media track the name states, with the MQF_FEEDBACK_TYPE property agreeing with the name. Each object is one report, mapped by feedback::Mapper and never forwarded, cached or fanned out. What is written onward is the stall-start and stall-end observations of delivery-telemetry §2, in a session id the relay draws fresh for the Feedback Track: held in memory (Relay::monitor().observations(), the latest 1024) and appended to feedback_out as JSON lines where it is set. The tally of the track's objects stays in the relay and is held against what it sent that session on the media track (more_than_sent ends the Feedback Track). Withheld fields are counted by reason. A report with an Optional Metric that has no mapping is refused whole, metric_unmapped: nothing is written from it. The feedback-source-id is never read. An origin-feedback track is not the relay's to consume and is a track like any other. |
Refusals of a Feedback Track's PUBLISH: not_negotiated, type_missing, type_unsupported,
type_mismatch, source_id_empty (NOT_SUPPORTED), feedback_not_subscribed
(UNAUTHORIZED), feedback_tracks_over (EXCESSIVE_LOAD). /metrics has
wai_relay_monitoring_captures_total, …_captures_refused_total{why},
…_log_messages_total, …_log_dropped_total, …_log_refused_total{why},
…_qlog_events_total, …_qlog_dropped_total, wai_relay_feedback_reports_total,
wai_relay_feedback_observations_total, wai_relay_feedback_refused_total{why} and
wai_relay_feedback_withheld_total{why}. Nothing here is evidence of delivery: a figure the
relay states of itself is its operator's to read, and a report is unsigned.
Playout timing
draft-nandakumar-moq-tempo-00 has a relay overwrite an object's HopTime with its own clock
as it forwards, for subscribers that said they read it. The draft's four code points are
unassigned; joule-moq's tempo module holds the placeholders this relay uses (properties
0x3A71 PlayoutDelay, 0x3A73 CaptureTimestamp, 0x3A75 HopTime; the Sync Support parameter
0x3A77), and [tempo] sets the three property types where a deployment must meet another
choice.
- Sync Support. The parameter is no parameter of MOQT, and an unknown message parameter
cannot be skipped, so it is negotiated: the relay's SETUP declares the Setup Option
0x3A79(empty), and a SUBSCRIBE at -23 or -22 may then carry Sync Support (length 0). -18 and -16 have no form for it: toward such a peer it is left out, and a subscriber at those drafts is stamped only withstamp = "always". - HopTime. For a subscription that carried Sync Support (
stamp = "asked", the default) or for every subscription ("always"), an object that carries exactly one readable HopTime, outside Immutable Properties, is forwarded with the relay's clock in its place, a short timestamp (the draft's §4.1) staying short. It is rewritten once an object, as the object is handed to its subscribers' schedulers, so the time it then waits for a subscriber's credit is part of the hop that subscriber measures. Nothing else of the object changes, and the relay adds no HopTime to an object that has none. What the cache and the DVR store keep, and so what a FETCH or a fill brings, carries the HopTime the object arrived with; datagrams are forwarded as they came. - The clock is the host's, which the draft requires to be synchronized; the relay does not
check it. With
stamp = "off"the relay declares nothing and a SUBSCRIBE carrying the parameter closes its session (INVALID_PARAMETER at -23, PROTOCOL_VIOLATION at -22).
So a sink behind a relay has the draft's second and third scenarios (a relay's clock on every
object) and not only the first (tests/tempo_hop.rs; joule-moq-room's tests/tempo_relay.rs
holds stamped objects to the room through the relay).
Configuration
wai-relay --config relay.toml, or wai-relay --dev for loopback with a
development certificate. relay.example.toml shows every field: the listen addresses and
path, the certificate files (or the names for a development certificate), the cache's byte
budget, the limits (sessions, requests and streams in flight, MAX_REQUEST_UPDATES, the token
cache, the rendezvous, upstream and fill timeouts, write concurrency, the QUIC send window,
the GOAWAY timeout, the session, ingress and memory limits, the held refusals and the per-session counts and request rate, the relay-path bound and the
upstream redial wait), the receipts suffix and door rules, the summary's key and file, the
grants, the room authorities and their revocation lists, the DVR store and its windows, the
upstream links, the operator's surface with its grants file and its flush interval, and playout timing. The configuration holds no secret: keys are files, and tokens are named by their
SHA-256. JOULE_MOQ_RELAY_LOG sets the log level.
The image
deploy/Containerfile builds wai-relay as an OCI image from the repository's root: the binary
built with the pinned toolchain from the locked dependencies, on a slim base pinned by digest, run
as the unprivileged user wai (uid 10001), with a configuration (deploy/relay.toml) that serves
native QUIC and WebTransport on UDP 4443 and the WebSocket fallback on TCP 4480 on every
interface, with a development certificate. A deployment mounts its own configuration over
/etc/wai-relay/relay.toml. deploy/check-image.sh builds the image and holds it to its checks:
its user, wai-relay --version, and a WebSocket opening on /moq answered for moqt-22 with the
relay's first capsule, from the image run with a read-only root and no capabilities. A release
pushes it to the forge's container registry as
git.transaction.science/transaction-science/wai-relay:<version>.
Not yet implemented
- SUBSCRIBE_TRACKS (§3.6): refused with NOT_SUPPORTED.
- Range Filters (§3.3.2): the relay declares MAX_FILTER_RANGES 0, so a request that carries one is INVALID_FILTER, and it evaluates no Track or Object Property filter.
- WebTransport over HTTP/2 (
draft-ietf-webtrans-http2-15), the path a browser takes when HTTP/3 is unavailable. The WebSocket binding carries the same capsules and covers the same case for now. - Parts of
draft-ietf-webtrans-http3-16the library does not speak: it negotiates with the settings deployed browsers send rather than -16'sSETTINGS_WT_ENABLED, has no per-session flow-control capsules, and closes a session by closing its QUIC connection with the code rather than withWT_CLOSE_SESSION; and quinn 0.11 has no RESET_STREAM_AT, so a reset WebTransport stream is not guaranteed to deliver its stream header first. - 0-RTT is not accepted.
- Migration as a client: the relay moves an upstream link's subscriptions after its GOAWAY (Cascading), but follows no REDIRECT, and a publisher's session that receives the relay's GOAWAY is not moved by the relay.
- Cascading: the per-hop record names no peer hop (a link is counted in the track entries, live-metering §3.10), so the chain is read by track, not by link; a rated room is not carried across a hop link (its threshold is 0); a FETCH piped through from upstream is checked against its receipts by the subscriber, not at this relay's door, as on any upstream fetch; routes are the configured prefixes, and a namespace an upstream names in NAMESPACE routes nothing by itself.
- Pausing upstream when every downstream subscriber is paused (§7.2), and forwarding NEW_GROUP_REQUEST upstream. EXPIRES is neither issued nor acted on.
- A cancellation the relay cannot see: a subscriber that sends FIN on its direction and later only STOP_SENDING is noticed when the relay next writes on the request stream, or when the session ends.
- Receipts: the receipts track of a media track is found by the configured suffix, not
by reading the catalog's
wai_receipts_track; the relay signs nodelivery-gapclaim; datagrams at the door are checked against their group's receipts, not a property. - Energy: a room's record carries the relay's own energy only where the relay served that room alone across the record's span; dividing the counter among rooms or sessions that ran together is not done here, and a session's summary always carries the unmetered marker. No figure is read on a host whose counter is neither the macOS task counter nor the RAPL package counter of a host stated to run the relay alone.
- The DVR store: decisions withhold from FETCH and fills, not yet from live subscriptions
(a reviewed item's
heldstate, live-moderation §6.3, needs the review policy); objects are checked against their receipts at a restart, and by the door as they arrive where a door rule covers them, not again as they are stored; no HLS or DASH presentation of the window is written (live-recording §8.6 is a mapping). - Re-validation (
moqt-reval, live-rooms §10.2): a grant is checked at SETUP and on each request, its expiry ends the session, and a revocation the operator gives ends its sessions (Grants that change while the relay runs); the relay reads no key log itself (§5).
Tests
All on loopback, every wait bounded:
| Test | What it shows |
|---|---|
relay_quic, relay_webtransport, relay_websocket |
Two publishers (one by PUBLISH_NAMESPACE and the relay's SUBSCRIBE, one by PUBLISH) and three subscribers over each transport; every object verifies against its publisher's receipts, which the door also checks; SUBSCRIBE_NAMESPACE; every summary verifies and labels the relay's energy unmetered. |
slow_subscriber |
A base track at priority 0 and a layer at 200 with a 100 ms delivery timeout. A subscriber that reads nothing for 1.5 s, then reads all its streams concurrently, loses layer objects to the timeout (their streams reset with DELIVERY_TIMEOUT), keeps the whole base track, and receives every receipt of both tracks although it asked for a timeout on its receipts tracks; each dropped object verifies by inclusion; the summary counts each layer object as forwarded or dropped. |
too_far_behind |
A track with no delivery timeout, a relay bound of 64 items a subscription, and a subscriber that reads nothing: its subscription is ended PUBLISH_DONE TOO_FAR_BEHIND (reason queue_bound) with the streams opened for it, what it had queued dropped and counted, and when it reads at last its streams are whole (sent before) or reset TOO_FAR_BEHIND, the one a write was stopped on among them. A subscriber that reads loses nothing; the slow session's other subscription stands and is served; the slow subscriber subscribes again and is served. With no bound, or with the write in flight left to wait, the test fails. |
first_objects |
Datagrams and a subgroup's stream sent with nothing awaited after the publisher's SUBSCRIBE_OK, twelve rounds each: every object reaches the subscriber once and in order, the subgroup on one stream that begins with its first object and ends with a FIN, and a Next Object filter starts at the publisher's first object. datagram_resubscribe: a datagram track subscribed to again is served, a repeat forwarded once. |
fetch_unanswered |
A track by PUBLISH that starts at {5, 1} and skips group 6, its publisher answering no FETCH: a FETCH from {0, 0} is answered FETCH_OK with the groups before, the object before the first held and group 6 each ended as Unknown and every object held served, ascending and descending; from {5, 1} the same without the head; past the Largest Object INVALID_RANGE; a track the relay holds nothing of DOES_NOT_EXIST. |
monitoring |
The metrics track read off the relay and held, number for number, to what GET /metrics answers before the next capture, with a counter one above and a series too many each refused; the log track with the relay's own messages, a second a group, and a message with an address in an attribute refused and never published; with the feature qlog, each session's log held to the draft's events and its order, with no payload byte in it, and an object moved before its header found (without the feature, a relay asked for a qlog does not start); a hop Feedback Track consumed, its stalls written onward in the relay's own session id, a report with an unmapped metric refused whole, a tally above what was sent ending the track, and three PUBLISHes refused each for its reason. |
stalled_session |
Three sessions on a track of 24 KiB groups, two with a 64 KiB connection window: the one that reads nothing is closed by the relay CONTROL_MESSAGE_TIMEOUT, reason stalled_session, no sooner than stalled_session_ms, and counted once; the one that reads each stream 200 ms late, its streams beside one another, has every group and stands; twice the time with nothing sent closes no session. |
late_stream_header |
Two data streams of a publisher, the second written whole and the first, open at the relay from that instant with nothing in it, written twelve seconds later: both objects reach the subscriber, the late one whole, and the publisher's stream was not stopped. |
request_wait |
A SUBSCRIBE of a track nobody publishes, which the relay answers after the RENDEZVOUS_TIMEOUT it carries (1.5 s): a client session whose ClientOptions::request_wait is 300 ms gives up no response in time with the answer still to come, and one that waits longer has the relay's REQUEST_ERROR. The default is ten seconds; a session over a path that drops packets states a longer one. And two request streams of a client, the second written and answered and the first, open at the relay from that instant with nothing in it, written twelve seconds later: it is answered as any. |
publisher_handover |
Two publishers of one track, the subgroup taken from the first: when the first one's session closes, and when it resets its stream and stays, the subscriber's stream is reset (SESSION_CLOSED, or the publisher's code) and its subscription stands; the second publisher's objects already forwarded are not sent again, its next ones arrive on a new stream of the same Group and Subgroup that does not claim the subgroup's first object and ends with its FIN, and a later group is its from the start; its summary counts three duplicates of six objects. Without the release the subscriber is sent nothing more of the subgroup. |
probes |
With probes on: the catalog reads back as the configured presets; a paced probe by its name, by the .probe/ form and by a preset, and a greedy one, each object for object what its name resolves to (Locations, sizes, zeros, a Timestamp and no other property, Subgroup 0 at priority 255, a stream a group, released on the schedule), ended PUBLISH_DONE TRACK_ENDED, with no track left at the relay. Planted, each with its code and reason: an unknown preset, a probe with no bound, one past the duration and one past the rate limit (DOES_NOT_EXIST); a FETCH and a SUBSCRIBE in the uplink namespace (NOT_SUPPORTED); another track under .session; a second probe while one runs, one from another session past the relay-wide limit and one too soon after (EXCESSIVE_LOAD with the time to wait, after which it is served). An uplink probe without FORWARD 0 and one naming a preset are refused UNINTERESTED; an accepted one is answered REQUEST_OK and resumed with a REQUEST_UPDATE setting FORWARD 1, its bytes counted and discarded, and no other session can ask for it; one that sends more than its bytes is ended and its session stands. A relay with probes off answers the catalog DOES_NOT_EXIST. |
late_joiner |
TRACK_STATUS; FETCH of a track and its receipts from the cache; a range in descending order; INVALID_RANGE; a fill on SUBSCRIBE; a group that reaches the relay after a later one, the FETCH from it still served from the cache; a FETCH forwarded upstream when the cache holds nothing. |
goaway |
GOAWAY with its URI and timeout to every session; GOING_AWAY for a new request; GOAWAY_TIMEOUT for the session that stays. |
shutdown_ports |
From a task on a runtime of one worker, the relay shut down and started again on the same ports many times, a session open over QUIC, WebTransport and WebSocket each time: when the shutdown returns its UDP, WebSocket and control ports bind at once, no session is open and every summary is in. |
start_ports |
From a task on a runtime of one worker, a start on fixed ports refused part way (the WebSocket port held, the control port held, a file where the control socket goes): when it returns, every other port binds at once, and a start with nothing held comes up on all of them; the control surface alone, refused at its socket, frees the TCP port it had bound. |
auth_tokens |
Tokens in SETUP, by value and by alias; UNAUTHORIZED; UNKNOWN_AUTH_TOKEN_ALIAS. |
grant_proof |
The relay sends nothing before the client's SETUP. Over each transport a client that holds no challenge is refused pop_missing and given one, proves over it on its second session and has the grant's rights, and proves at once on a third over the challenge the admitting SETUP gave; nine challenges, none twice. Planted, each refused with its reason in GRANT_REFUSED: the grant as a bare token, a proof by another key, one for another relay, one over a consumed challenge (pop_replayed), one over a challenge never issued, one over another relay's, one past its time. A client that does not know GRANT_REFUSED is closed UNAUTHORIZED by the relay. A grant presented only in a request is refused it. |
room_authority |
An authority's grants, each proved, give the rights live-rooms §10.2 maps them onto; the door holds each publisher's video to its key; the relay's limits end a session (its session limit, a -22 session's grant expiring, its ingress limit). At -23 a grant that expires ends each subscription it gave with PUBLISH_DONE EXPIRED_AUTH_TOKEN (0x7) and leaves the session open, nothing more forwarded and a new subscription by the expired grant refused; beside it a -22 session under a grant of the same expiry, and a -23 session that publishes by one, are closed EXPIRED_AUTH_TOKEN. |
request_ids |
INVALID_REQUEST_ID for the wrong parity and for a repeat; requests in flight bounded by the stream limit. |
relay_energy |
On a host with a counter (macOS's task counter; on Linux, the package counter with dedicated_host): a room's record, after a publisher sends for over a second, carries OnChipCounter with its evidence, reads back, and verifies; with a second room's session in the span it carries the unmetered marker; with energy = "off" it carries the marker. A record over a window of sessions still open carries the relay's figure for the window, a session that counted nothing in it sharing nothing, and with a second room's objects in the window the marker. With hold_joule_ceilings and a flush_interval_ms, a grant whose ceiling its share passes has its session closed at the relay's own seal, with nobody flushing it, and its next SETUP refused, and a grant with no ceiling goes on; without it, nobody ends. Where the host has no counter the test says SKIPPED, and with JOULE_MOQ_RELAY_ENERGY_REQUIRE=1 it fails instead. The figure's arithmetic, its liveness rule and its uncertainty are src/energy.rs's unit tests, which run on every host. |
dvr |
A long publish crosses the window: the oldest groups go, whole, each media group with its receipts group, within the byte budget; a window by age empties as its groups pass it. FETCH inside the window (ascending and descending), across it (an End of Unknown Range through the last evicted group, then every object, from the store and the cache together) and before it (the marker alone); every object verifies against the fetched receipts. A restart keeps the store, drops and counts a flipped byte and bytes changed with their checks made again (only the receipt finds those), removes a cut-short write, and never serves what it dropped. A room key's redaction erases its target and withholds it, a deny withholds until an allow, a redacted event leaves the object that carries it, a decision by no moderator changes nothing, and the decisions hold across a restart. Without the seek power a FETCH and a fill below the live edge are refused seek_not_granted, and a FETCH at the live edge is served; with it, the window. The record of the room is WMH1 version 2 with the store's figures. A room whose objects are sealed is never kept: its first sealed group is not written, its events and receipts already held and its decisions are deleted, nothing more of it is written, and so after a restart, while a room in the clear beside it keeps its window. A room in which a session proves a minor's publishing grant is never kept from then on, likewise, while a minor's grant that publishes nothing changes nothing. A window past 24 hours is refused. |
age_serving |
A room rated 13 by its authority, the claim on the publisher's PUBLISH, and a stranger's rating ignored: a session at no threshold is refused the subscription below_threshold; the performer rates groups 5 to 7 of its track 18 on group 5's object, after group 4 went out, and from then the viewer at 16 is forwarded none of them while the viewer at 18 is forwarded all; a FETCH across the span is answered up to group 4 and one from inside it is refused; a SUBSCRIBE at a live edge rated 18 is refused. |
age_room_service |
A room rated 18 by its authority, with the floor held ([floor] room_keys the clock): sessions proving the clock's key and the authority's (a GRANT_PROOF naming no grant) are served every group, verified; the clock's token with no proof, and a proof by a key that is no room key, are refused the subscription below_threshold; a proof naming the clock's key under another key's signature, and the clock's proof over another session's challenge, close the session UNAUTHORIZED. |
age_rising |
A rating that rises while a subscriber's objects wait at the relay: a viewer at 18 that reads nothing, with small QUIC windows and the relay a small send window, holds groups 0 to 9 queued when the performer rates its whole track 21 on group 10's object; once the relay holds the claim the viewer is given no more of what waited (those streams ended, CANCELLED) and nothing of group 10, while a viewer at 21 is given every group. |
priority_fan_out |
Two publishers, four tracks, three subscribers with their own subscriber priorities and group orders, their transports held shut while the load arrives: each receives every object in §5.1.2's order. |
drafts |
The two-publisher scenario with every session at -18 (over QUIC and over WebTransport), at -16, and with publishers at -16 and -18 and subscribers at -22, -18 and -16: every object verifies against its publisher's receipts at every subscriber. FETCH from the cache at -18 and -16 (the drafts' exclusive End Locations, -16's absolute fetch IDs). A SUBSCRIBE with a Range Filter, refused at an -18 client by name. An -18 session and a -16 session written from those drafts' own messages, not through this crate's client: SETUP each way, a request answered, a Joining FETCH refused with NOT_SUPPORTED, an unknown Fetch Type closing the session with PROTOCOL_VIOLATION; a -16 Request ID ahead of one in flight on another stream admitted and one seen before closing the session with INVALID_REQUEST_ID. An -18 publisher's FIN on PUBLISH_NAMESPACE withdrawing the namespace, a -22 FIN not; an -18 publisher that closes its control stream before its last object and PUBLISH_DONE losing neither. |
draft23 |
Sessions at -23 over QUIC, WebTransport and the WebSocket binding, beside -22 ones: two publishers and three subscribers at -23, and at -23 and -22 mixed both ways, every object verified; a FETCH from the cache at -23, its REQUEST_OK read as FETCH_OK and a range whole; a relative filter in FETCH refused INVALID_RANGE; a tampered object at -23 refused at the door alone; a session written byte by byte from -23's messages (overlapping namespace subscriptions accepted); and planted faults, each closing a session with -23's code: FETCH_OK (0x18, UNEXPECTED_MESSAGE), FORWARD (INVALID_PARAMETER), a 16-bit Length and a missing parameter count (MALFORMED_MESSAGE), a requester's GOAWAY on its own request stream and an empty Track Name (PROTOCOL_VIOLATION); a -23 session's summary names moqt-23. A -23 session's objects held to -23's rules beside the same objects from a -22 session: a grease value in the Mandatory range forwarded at -23 and malformed at -22, a Track-only property on an object and a property repeated across the two lists malformed at -23 and forwarded at -22, a malformed object ending its stream alone. The relay's LARGEST_OBJECT by -23 §7.5, a vector of six steps (what the publisher states in PUBLISH and PUBLISH_STATE_NOTIFY and what it sends) and the value TRACK_STATUS_OK and SUBSCRIBE_OK answer after each, at -23 and at -22. |
cascade |
An edge that dials a home relay over native QUIC, WebTransport (both trusting a pinned certificate) and the WebSocket binding: a publisher on each, a subscriber of the other's track on each, every object verified at the subscriber against its publisher's receipts and by both doors; each namespace published both ways and discovered by SUBSCRIBE_NAMESPACE from either relay; each relay's summary of the link naming the other. Three relays in a chain, a room's track each way: a namespace's path; a per-hop record at each relay, each signed by its own key with its own unmetered figure, the in and out entries of each track chaining hop to hop, only the end viewers subscriber sessions, the middle relay's ledger empty. A home relay with no door forwarding a tampered object that the edge's door refuses. A loop of three relays: an announcement never returns to its origin, and a SUBSCRIBE for a track nobody publishes refused relay_loop at once. A link to an upstream that is down passed over for the next, and with none up TIMEOUT no_upstream, with and without a rendezvous. An upstream's GOAWAY naming another relay: the edge moves its subscriptions there, its subscriber sees no GOAWAY and loses no group. The home relay's DVR window fetched from the edge with the seek power, and refused seek_not_granted at the edge without it, by FETCH and by fill. A member's votes, sent out of order on the edge, decided once there in chain order, a watcher on the home relay given exactly what one on the edge is; and an object a relay with no floor forwarded not held to the floor again over the link. A relay's id proved on its second session, over the challenge the first was given, and a relay id declared with no proof refused relay_id_unproved. At -23 as at -22: a path naming the relay refused relay_loop, one past the bound relay_hops_exceeded, the bridge dropping nothing of it, and a path that is not the peer's, or from a session that negotiated none, closing the session INVALID_PARAMETER; at -18 a path has no form and is refused by name. |
metrics |
GET /metrics on the control listener read strictly as the text format, every family there from the start: a publisher of two tracks and a subscriber to both over the WebSocket binding are two sessions open and opened, two tracks held and two subscriptions, the objects counted in; once they leave none is open and the opened count stands. A JSON-lines request on the same listener is answered as before and counted by its op. HEAD is answered without the text, another method 405, another path and a request through a front (each field a front sets) 404. No label value is a key, a digest, an address or an account. On the socket (src/control.rs), a peer stated as another user is refused before the counters are answered, and the refusal is counted. |
live_control |
Two rooms, a publisher and a subscriber in each under a token of its own: the first room's token withdrawn on the control socket closes its two sessions UNAUTHORIZED, grant_revoked, while the second room delivers every later group with no GOAWAY; a token granted at run time is served and the withdrawn one refused; a metered grant added at run time names the grant of sessions already open. A metered grant withdrawn, or left out of a set, ends the sessions under it with no rule naming its token, and a set that maps a token to another grant moves its session there. A session subscribed by a token its requests carry ends when that token is withdrawn, and goes on through a change that leaves it alone. A change that moves a rule to another prefix, or takes a right, ends the sessions holding what it no longer covers, and none that loses a right it does not use. A control listen address off loopback stops the relay at start; the socket is mode 0600; a peer of another user or off loopback is refused (a peer stated as another user on the socket on every host, and with root a peer that dropped to another user, said SKIPPED otherwise), and a connection from a non-loopback address of the host to the loopback listener is answered with a refusal (said SKIPPED where the host has no such address); an HTTP request on the listener ends its connection, its body unread. A set that leaves out required or metered changes nothing, and a reload leaves required as it stands. An authority's grant revoked by its digest or its viewer's tag is refused at SETUP before its proof; one revoked while its session runs ends that session alone; an authority's grant is metered from its own ceilings; one whose funds ceiling and currency disagree is refused grant_invalid. A revocation ends a session that proved nothing on a relay requiring no authorization, one a rule admits as well and one whose request presented the grant, with a rule naming it or on a relay requiring no authorization; a request presenting a revoked grant is refused on both, its session standing and left alone by a later change; and the list holds once its authority is dropped. A session's requests are refused past the bound on the records they may present. An authority added at run time is honoured, and one dropped ends the sessions that proved its grants alone. A session holding grants for two rooms is in each room's ledger under that room's grant, and under none in a third room's, over a window and over the sessions ended; on a relay holding no authority, by rules with proof and metered grants, a token that is no record names the third room. A revocation outlives a reload, a set that does not repeat it and a restart. A flush inside a group: the group's later subgroup, after the cut, counted in the window it began in and the next group in the next; the record signed on the socket over the first window, with every session open, read by this crate and by wai/live-conformance/live_metering_verify.py (said SKIPPED where the host has no Python 3.8; under CI, or with JOULE_MOQ_RELAY_VERIFY_REQUIRE, it fails instead); each session's windows summing to its summary, and the windows' records to the whole's. A relay never flushed holds no second copy of its sessions' counts, and its first flush puts them in window 0, an ended session's whole among them. An interval seals with nobody asking, and a relay told to hold a ceiling without one does not start. With a DVR store, groups cut by a flush and sealed before or after the store wrote them: each window's record reads, its stored figures within its in entries. SIGHUP reads the grants file again: the dropped token's session ends, the added one is served. |
window_cuts |
A window closed from the operator's surface, each check held against a planted fault. Its counts by session: the surface's window statement of a sealed window, with both sessions open, recomputes the room's record over it, and names the publisher's media and the viewer's layer and time; a part's count raised and signed by the relay's own key, a session's part left out and signed so, a part changed after it was signed, and the next window's parts set against this record are each refused. One cut across relays: a core and two edges, a viewer on each edge; the edges close before a group begins and the core after it, and sealed at the join of their cuts the group is in the closing window at all three, the core's and the first edge's statements of their link chaining exactly in that window and the next; a cut below the core's own is refused with nothing sealed; the second edge, sealed at its own cut as an edge cutting at its own instant would, is caught short in the first window and with more in than was sent in the next. Ride-through: the first edge's viewer is sent no GOAWAY across both window ends, stays open, and has every group once, verified, and no session anywhere ends; the second edge, restarted at a window's end as edges were, sends its viewer GOAWAY, which the same check catches. Summaries: past a bound of 2 with no spill file every summary is held and a new SETUP refused summaries_unacknowledged; an acknowledgement past what was handed out frees nothing; an acknowledged summary is freed and not handed out again, and sessions are admitted once under the bound; with a spill file the summaries past a bound of 1 go there and come back in order, and a restart hands over the one not acknowledged and none that was, the file emptied once all are. |
control_sessions |
The edge's link to the core runs at -23 and opens on its second dial (a challenge on the first), and the core states it among its sessions. sessions: a publisher at -22, a viewer at -23 and a session that holds nothing, stated by room with each one's draft, transport and opening time, signed by the relay's key; the room reads sealed from its first sealed object for every session of it, a room in the clear beside it does not; a sealed room stated in the clear after signing and a session left out after signing are refused by the signature, a statement under another key is caught by the key it names, and a session that left is no longer listed. wai-relay --control: an edge's window closed and sealed with the program on its own socket, the core's on its own, at the joined cut, their link chaining exactly and the edge's viewer sent no GOAWAY; a seal of a window that is not the next exits 1 with the refusal printed and nothing sealed, an unknown op exits 1, two requests in one exits 2 with nothing sent, a socket nobody serves exits 3; the socket keeps mode 0600. |
checkpoint |
A hard kill: the wai-relay program itself, bound to port 0 and its port read from its log, under a guard that kills and reaps it. Window 0 sealed, two more groups, then SIGKILL; started again it answers window 0 as it stated it, and window 1, under the same index, holds the killed sessions' counts beside the new ones', marked resumed with the checkpoint's instant and the span to the restart unmetered; the room's record recomputes from the parts with the unmetered marker, and over both windows each of seven groups is counted once. Planted: the checkpoint from before window 1's seal put back, refused checkpoint_stale, its two groups not counted again; a truncated checkpoint, one of another relay's key and one changed after signing, each refused with its reason in the next window's statement and no count read. A clean stop writes every count and the restart states no unmetered span. A checkpoint with no summary key stops the start. src/checkpoint.rs's and src/relay.rs's unit tests hold the same refusals and that a resumed window's record is unmetered where the same window held whole carries a figure. |
publish_done_streams |
A PUBLISH_DONE that arrives before the data stream it counts, at -22, -18 (QUIC and WebTransport) and -16: the object still reaches the subscriber, then PUBLISH_DONE; a count no stream meets ends the track after the wait; 0 and the unknown count at once. A close right after PUBLISH_DONE ends the track with the publisher's status, not GOING_AWAY, and without the wait. A track published again after TRACK_ENDED is a new instance, and so is one published again after its publisher's session closed with no PUBLISH_DONE: the subscriber it had is told GOING_AWAY, and a new one is sent the second publisher's objects under the Group IDs the first used. |
dos_limits |
The relay's own limits against draft-englishm-moq-relay-dos-02. Ten SETUPs the relay refuses, from one address, against a bound of three held an address: the seven oldest closed UNAUTHORIZED at once, the three newest held, none a session; against a bound of four in all, the six oldest; planted, the bounds removed, and all ten are held. A third subscription, a third published track and a second namespace of one session each refused EXCESSIVE_LOAD (session_limit), what the session holds standing and a slot coming back when its request ends; planted, the limit removed, and each is served. A request past a session's rate refused EXCESSIVE_LOAD (request_rate) with the Retry Interval after which one is served. |
tempo_hop |
HopTime rewritten with the relay's clock for subscribers at -23 and -22 that carried Sync Support, not for one that did not; a short timestamp kept short; no HopTime added; Immutable Properties and payload as sent; stamp = "always" for a subscriber at -18; stamp = "off" declaring nothing and closing a session that sends the parameter; another configured type leaving the default's property alone. |
dvr_files |
A room's video and receipts published through the relay, kept by the store, written as .moq and .dat files and read back: every object verifies against its group's receipt from the files alone, and the files agree with a FETCH of the window byte for byte. Planted: a data file cut short, a .moq file cut short and a data file outside the directory, each refused with its reason; a payload changed in the data file, which the receipt refuses. The operator's request: wai-relay --control with files writes one of two recorded rooms into a directory the relay makes, then every track into another, each read back and verified from the files alone; a directory that holds files (left as it was), a relative path, a room the store holds nothing of and a relay with no store are each refused with their reason and nothing written. |
bench_harness |
The load harness run small on loopback. Scenario 2: one publisher, three subscribers, a video-like stream track and an audio-like datagram track; every stream object received by every subscriber between START and COMPLETION, at most one datagram object in twenty missing, no subscriber without its START and COMPLETION; the record signed and read back under its key. Scenario 1: the datagram track alone, and one whose object does not fit a datagram refused. Scenario 3: two meetings of three participants, each publishing both tracks and taking the two others', the result over six publishers. Planted: the record changed after signing (refused), the run claimed for another host or another meeting shape (not_comparable), a run that lost objects (named by the guard). |
Licence
Apache-2.0.