open-standards/joule-code/joule-code-rs/crates/joule-moq-relay

joule-moq-relay

A relay for MOQT, draft-ietf-moq-transport-22. Publishers send it tracks, subscribers ask it for them, and it fans each object out to every subscription, in the order the draft's priorities give, dropping what passes a subscriber's delivery timeout at this hop. Whatever it drops, the object's receipts still reach the subscriber, on the track's receipts track.

It is a library (Relay, RelayServer, client::Client) and a binary, joule-moq-relay. The wire codec is joule-moq's.

The draft it speaks

draft-ietf-moq-transport-22, and no other. Drafts do not interoperate, so a session speaks one from its first byte. The draft is chosen before any MOQT byte:

• by the ALPN moqt-22 on native QUIC (§6.2.2);
• by WT-Available-Protocols: "moqt-22" on WebTransport, which the relay answers with WT-Protocol: "moqt-22" (§6.2.1; draft-ietf-webtrans-http3-16 §3.3). A client that sends no WT-Available-Protocols is taken to speak -22; one that offers other protocols only is refused;
• by the subprotocol moqt-22 on the WebSocket fallback (below).

In its SETUP the relay declares MOQT_IMPLEMENTATION, MAX_REQUEST_UPDATES and MAX_AUTH_TOKEN_CACHE_SIZE. Where the client declares joule-moq's receipt extension (Setup Option 0x3A46, its value the extension version), the relay declares it back.

Transports

Path	Listener	Selected by
Native QUIC	UDP quic_listen	ALPN moqt-22
WebTransport over HTTP/3	the same UDP port	ALPN h3, extended CONNECT to path, WT-Available-Protocols
WebSocket fallback	TCP websocket_listen	upgrade at path with subprotocol moqt-22

Native QUIC and WebTransport share one quinn endpoint: the relay reads the ALPN of each connection and either runs MOQT on it directly or hands it to the WebTransport layer.

The WebTransport implementation is wtransport 0.7 (MIT OR Apache-2.0). It was chosen because it runs on the same quinn 0.11 as joule-moq, so one UDP endpoint serves both ALPNs; it gives the server the CONNECT request's headers and lets it answer with its own, which is what WT-Available-Protocols and WT-Protocol need; and its client checks a certificate by hash under the rules a browser applies to serverCertificateHashes (ECDSA P-256, a validity of fourteen days at most), so the relay's tests connect the way a page does. Stream bytes are read and written on the quinn streams underneath; application error codes on WebTransport streams are mapped into the HTTP/3 error space as draft-ietf-webtrans-http3-16 §4.4 gives.

How a browser connects

WebTransport, in development. Start the relay with --dev (or with no certificate files in the configuration). It makes a certificate a browser accepts by hash and logs its SHA-256 three ways, including a ready serverCertificateHashes entry:

const transport = new WebTransport("https://127.0.0.1:4443/moq", {
  serverCertificateHashes: [{ algorithm: "sha-256", value: new Uint8Array([/* printed */]) }],
  protocols: ["moqt-22"],
});
await transport.ready;          // transport.protocol === "moqt-22"

The certificate is valid fourteen days from the relay's start; restart the relay to make a new one. In production give it a certificate chain and key from files, and a page connects without serverCertificateHashes.

WebSocket, where there is no UDP path. A site behind a front that carries HTTP over TCP only reaches no UDP endpoint, so neither QUIC path is open to its pages. Such a page uses the fallback:

const ws = new WebSocket("wss://relay.example/moq", "moqt-22");
ws.binaryType = "arraybuffer";  // each message is one capsule, below

The WebSocket listener speaks plain HTTP; TLS is the front's.

The WebSocket binding (non-normative)

This binding is this crate's, not a standard's. It carries what a WebTransport session over HTTP/2 carries (draft-ietf-webtrans-http2-15), on a WebSocket instead of an extended CONNECT stream, so that a client written for one is most of the way to the other.

1. Opening. The client opens a WebSocket (RFC 6455) to the relay's path, offering the subprotocol moqt-22 in Sec-WebSocket-Protocol. The relay answers with that subprotocol, or refuses the upgrade (400 without it, 404 at another path).

2. Messages. Every message is a binary message holding exactly one capsule: Type (i) Length (i) Value, with QUIC variable-length integers (RFC 9000 §16), not MOQT's. A text message carries nothing and is ignored. A binary message that is not one whole capsule closes the session.

3. Capsules. Those of draft-ietf-webtrans-http2-15 §6, with their type codes:

   Capsule	Type	Value	Use here
   WT_STREAM	0x190B4D3C, 0x190B4D3B with FIN	Stream ID, data	stream data; a stream opens with its first WT_STREAM
   WT_RESET_STREAM	0x190B4D39	Stream ID, code, Reliable Size	reset a stream (Reliable Size 0)
   WT_STOP_SENDING	0x190B4D3A	Stream ID, code	ask the peer to stop
   WT_MAX_STREAMS	0x190B4D3F bidi, 0x190B4D40 uni	Maximum Streams	stream-count credit
   WT_STREAMS_BLOCKED	0x190B4D43, 0x190B4D44	Maximum Streams	sent when out of credit; informational
   DATAGRAM	0x00	payload	one MOQT datagram, reliable and ordered
   WT_CLOSE_SESSION	0x2843	code (32 bits), message (UTF-8, at most 1,024 bytes)	close with a MOQT session termination code
   WT_DRAIN_SESSION	0x78AE	empty	read and ignored; MOQT's GOAWAY is the drain
   others			read and ignored, as unknown capsules are

4. Streams. Stream IDs are numbered as QUIC numbers them: bit 0 the initiator (0 the client), bit 1 the direction (0 bidirectional), and the nth stream of a kind is 4n + kind. MOQT's control streams, request streams and data streams ride them byte for byte as they would ride QUIC streams: the client's control stream is its first unidirectional stream (ID 2), beginning with SETUP; a request is a bidirectional stream; a subgroup, fetch or padding stream is a unidirectional one.

5. Stream credit. Each end first sends its WT_MAX_STREAMS for both kinds; until the peer's arrive, an end opens nothing. Credit is cumulative, as in QUIC. A stream's credit comes back once both of its halves are done, and is granted in batches. Opening past the limit closes the session with code 0x045D4487. Because MOQT -22 bounds requests in flight by stream limits alone (§6.4.2), the relay's bidirectional limit (max_requests) is its bound on requests in flight on this binding, as MAX_STREAMS is on QUIC.

6. Data flow control. None at the capsule level (WT_MAX_DATA, WT_MAX_STREAM_DATA and their BLOCKED capsules are read and ignored): TCP's backpressure bounds a sender, and a receiver that holds more than websocket_buffer_bytes unread closes the session with 0x045D4487.

7. Closing. WT_CLOSE_SESSION with a MOQT session termination code (§12.2), then the WebSocket's close. A WebSocket that closes without one is a session closed with code 0.

The first capsules a relay with the default limits sends are its credit:

99 0b 4d 3f 02 41 00     WT_MAX_STREAMS (bidirectional), 256
99 0b 4d 40 02 44 00     WT_MAX_STREAMS (unidirectional), 1024

and a session it closes at the end of a GOAWAY's timeout ends with:

68 43 12 00 00 00 10 47 4f 41 57 41 59 20 74 69 6d 65 6f 75 74
                         WT_CLOSE_SESSION, GOAWAY_TIMEOUT (0x10), "GOAWAY timeout"

What the relay does

-22	Here
§6.3 session initialization	Each end's control stream opens with SETUP; data streams that come first wait for it. A client's SETUP is held to §9.1's rules for a client (no PATH or AUTHORITY over WebTransport; no DELETE or USE_ALIAS token). The relay's control stream carries only its SETUP and a GOAWAY; the end of the peer's closes the session with PROTOCOL_VIOLATION.
§6.4.2 requests	A request stream must begin with one of the seven requests. A Request ID of the wrong parity or seen before closes the session with INVALID_REQUEST_ID. MAX_REQUEST_UPDATES is enforced per stream; requests in flight are bounded by the stream limit (max_requests).
§8.9, §9.1.3, §9.1.4 tokens	A receiver's alias cache per session: REGISTER, DELETE, USE_ALIAS, USE_VALUE; AUTH_TOKEN_CACHE_OVERFLOW, DUPLICATE_AUTH_TOKEN_ALIAS, UNKNOWN_AUTH_TOKEN_ALIAS; a registration in SETUP past the cache reads as USE_VALUE. Grants name a namespace prefix, rights to publish and subscribe, and the SHA-256 of a token of type 0.
§4.1, §4.2 namespaces	PUBLISH_NAMESPACE is accepted (REQUEST_OK) and stands until it is cancelled or its session ends. SUBSCRIBE_NAMESPACE answers with NAMESPACE for each published namespace under the prefix, then NAMESPACE and NAMESPACE_DONE as they change, each carrying the suffix after the prefix; an overlapping prefix in one session is PREFIX_OVERLAP.
§7.6, §9.8 PUBLISH	Accepted with REQUEST_OK; its Track Alias is the publisher's for its data. A second track under one alias in one session closes it with DUPLICATE_TRACK_ALIAS. PUBLISH_DONE ends the track's downstream subscriptions.
§7.4, §7.6, §9.6 SUBSCRIBE	SUBSCRIBE_OK only once an upstream subscription is Established (§7.4): one the relay holds, or a SUBSCRIBE it sends to every publisher of a matching namespace. The relay's own upstream SUBSCRIBE asks for every object at priority 128 with no timeout, copying no subscriber's priority, group order or timeout (§5.1.3, §9.20.3, §9.20.4). RENDEZVOUS_TIMEOUT holds a SUBSCRIBE for a publisher, up to max_rendezvous_ms; without it a track with no publisher is DOES_NOT_EXIST. Location Filters set where the subscription starts and ends. A publisher of a namespace that arrives later is sent a SUBSCRIBE for every subscription the relay holds under it. When the last subscriber leaves, the relay cancels its own upstream SUBSCRIBE.
§9.5 REQUEST_UPDATE	Subscriber priority (re-applied to what is queued), FORWARD, the Location Filter and both delivery timeouts.
§3.2, §3.4, §7.1, §9.11 FETCH and fills	A fetch the cache covers is served from it, groups in the requested order (coverage, below); otherwise it goes to a publisher of the track or its namespace and its fetch stream is piped through, past the Fill Timeout a Timed-Out range. A range that starts after the Largest Object is INVALID_RANGE. A SUBSCRIBE with FILL_PARAMETERS gets its fill range the same way on a fill fetch stream named by its Request ID; a fill it cannot serve is a fetch stream reset after its header (§3.4.1).
§9.13 TRACK_STATUS	Answered from what the relay holds: the Largest Object and the track's properties.
§7.3 several publishers	A track may have several upstreams; each (group, subgroup) is taken from the first that sends it, and the rest are counted as deduplicated.
§5.1 priorities, §5.2 delivery timeouts	Below.
§6.6.1, §9.2 GOAWAY	On shutdown every session gets GOAWAY with goaway_uri and the timeout; new requests are refused with GOING_AWAY; a session still open at the timeout is closed with GOAWAY_TIMEOUT.
§6.5 .session	No session-level track is defined: a request for one is DOES_NOT_EXIST.

What the cache covers

The cache answers a fetch alone only where it holds every object of the range, so that a gap in its answer means an object that does not exist (§3.2.2). It keeps what it knows of each group's arrival. A group is whole when its start arrived (Object ID 0, or a Prior Object ID Gap saying the objects before do not exist, §10.9), none of its subgroup streams is open or was reset, and its end is known: its streams ended with a FIN, or an End of Group or End of Track status, or a datagram with END_OF_GROUP, ended it. A fetch from a group is covered when every group from it up to the group of the Largest Object is whole or known not to exist (a Prior Group ID Gap, §10.8), that last group has its start, and nothing in between was evicted. Coverage is worked out from the groups held each time it is asked, so a group whose streams finish after a later group's still counts once it is whole; a group ID never seen is a gap the cache does not answer across. The relay cannot tell a subgroup it has not yet seen from one that does not exist, so a group whose subgroups have all closed counts as whole.

Priorities and delivery timeouts

Each downstream session has a scheduler. Every data stream it will send (a subgroup, a fetch or fill, a datagram) is a lane whose items go out in order and one at a time, so no stream's objects are reordered (§7.4). Among the lanes with something to send it takes the item §5.1.2 puts first: the lower subscriber priority number; then the lower publisher priority number; then, within a subscription, the group its Group Order puts first (and in one group a fill before its subscription); then the lower Subgroup ID or, for a datagram, Object ID, a datagram before a subgroup object. At most write_concurrency writes are in flight, and a QUIC session's send window is quic_send_window_bytes: what waits, waits in the scheduler, where it can still be ordered and dropped (§5.2: "minimize the amount of data buffered at the underlying transport layer"). Each stream is also given a transport priority from its subscriber and publisher priorities.

An object's deadline is its OBJECT_DELIVERY_TIMEOUT from when it reached the relay: the smaller of the publisher's (the first object of its subgroup, else the track's) and the subscriber's, where both are set. It is checked before the object goes to the transport and bounds both opening its stream and writing it. Past it the stream is reset with DELIVERY_TIMEOUT and nothing more of that subgroup goes to that subscriber; a datagram is dropped. SUBGROUP_DELIVERY_TIMEOUT runs from the subgroup's FIN until all its data is acknowledged, and resets the stream if it is not. On the WebSocket binding a FIN counts as acknowledged once written, as webtrans-http2 §5.2 makes the transitions an acknowledgement would make at once.

A subscriber that does not read

QUIC's connection-level flow control is shared by every stream of a session (RFC 9000 §4.1), and bytes a subscriber has received but not read hold it. So yes: a subscriber that stops reading stalls its own high-priority track along with everything else, until it reads again. And a subscriber that reads its streams one at a time, in the order they arrive, can stall itself while still reading: the stream it waits on may have its bytes held back for credit that the unread streams behind it are holding. A subscriber reads its streams concurrently.

The relay cannot release credit the subscriber holds, and does not try to by dropping what has no deadline. What it does:

• An object with no delivery timeout (a base track, a receipts track) is never dropped. It waits, in the scheduler or the transport, and is delivered once the subscriber reads.
• What can still wait, waits in the relay rather than in the transport (§5.2: "minimize the amount of data buffered at the underlying transport layer"). Writes in flight are bounded by write_concurrency and the transport's unacknowledged bytes by quic_send_window_bytes. So an object whose deadline passes while it waits is dropped before it takes any credit.
• An object that passes its deadline is reset. A reset stream's final size settles its account (RFC 9000 §4.5), and a receiver can credit its unread bytes back to the connection at once, as quinn does. The credit a dropped layer held then comes back without the subscriber reading it.
• When credit frees, the highest-priority bytes go first: each stream carries its subscriber and publisher priority as its transport priority.

MOQT lets a publisher end a subscription whose subscriber falls too far behind, with PUBLISH_DONE TOO_FAR_BEHIND (§5.2); this relay does not yet.

Receipts

A media track's receipts ride its receipts track (WAI extensions/moq-streaming-format.md §4.1): group g of it is one object holding the group receipt and the object receipts of media group g. The relay recognises a receipts track by its name: the media track's name followed by receipts_track_suffix (/receipts unless configured).

• The receipts of dropped objects are forwarded. A receipts track has no delivery timeout, and the relay applies none to it, even one a subscriber asks for. So when an object of a media track passes its deadline, its receipt still reaches every subscriber of that track's receipts track, which can account for the object by inclusion (WAI extensions/jwp-receipts.md §4.1, §4.4). A receipt that rides on the object itself, as a property, is lost with it.
• Elsewhere receipts are opaque. Outside the namespaces a door rule names, the relay forwards receipts tracks as it forwards any track and reads nothing in them (§7.8).
• At the door. In a namespace a [[receipts.door]] rule names, the relay reads the receipts it forwards, and takes on jwp-receipts §4.1's duties: each group receipt's signature is verified, under the rule's trusted keys or any key, before the first object of its group is fanned out, and a group that does not verify is dropped with a structured error in the log; each media object must match the receipt of its position (its group, its Object ID, its content hash, its Merkle proof to the signed root), and an object that carries its receipt as a property must carry that same one. An object waits for its group's receipts up to hold_ms. Receipt bytes are forwarded unaltered and nothing is re-signed. A refused object ends its downstream streams rather than leaving a hole in them.

The session summary

When a session ends the relay signs a summary of it with its own Ed25519 key (from summary.key_path, or made at start and its public half logged), keeps it (Relay::summaries), and appends it as one JSON line to summary.out_path if set. It holds, for the session:

• as a subscriber: objects and their bytes forwarded, every byte written to its data streams, objects dropped for their delivery timeout and how many of those were on a track whose receipts track the session also held, subgroups reset after their FIN, objects served by fetch or fill, objects still queued at the end, and the same per track;
• as a publisher: objects and bytes received, objects refused at the door, objects deduplicated;
• relay_energy: {"acquisition": "unmetered", "joules_micro": 0}. No meter backs this relay. The 0 is the unmetered marker of WAI extensions/energy-measurement.md §2, never a measurement of zero joules, and must not be read or shown as one;
• publisher_signed: where the door verified an object's receipt, the publisher's signed figure for producing it, summed under the acquisition class its leaf was signed with. These are the publisher's statements about the objects, not the relay's energy.

The summary is relay-scoped telemetry (jwp-receipts §4.2), not a receipt of the publisher's. Its format is joule-moq-relay/session-summary/1; the signature covers joule-moq-relay:session-summary\x01 followed by the summary's JSON bytes, and SignedSessionSummary::verify checks it.

Configuration

joule-moq-relay --config relay.toml, or joule-moq-relay --dev for loopback with a development certificate. relay.example.toml shows every field: the listen addresses and path, the certificate files (or the names for a development certificate), the cache's byte budget, the limits (sessions, requests and streams in flight, MAX_REQUEST_UPDATES, the token cache, the rendezvous, upstream and fill timeouts, write concurrency, the QUIC send window, the GOAWAY timeout), the receipts suffix and door rules, the summary's key and file, and the grants. The configuration holds no secret: keys are files, and tokens are named by their SHA-256. JOULE_MOQ_RELAY_LOG sets the log level.

Not yet implemented

• SUBSCRIBE_TRACKS (§3.6): refused with NOT_SUPPORTED.
• Range Filters (§3.3.2): the relay declares MAX_FILTER_RANGES 0, so a request that carries one is INVALID_FILTER, and it evaluates no Track or Object Property filter.
• WebTransport over HTTP/2 (draft-ietf-webtrans-http2-15), the path a browser takes when HTTP/3 is unavailable. The WebSocket binding carries the same capsules and covers the same case for now.
• Parts of draft-ietf-webtrans-http3-16 the library does not speak: it negotiates with the settings deployed browsers send rather than -16's SETTINGS_WT_ENABLED, has no per-session flow-control capsules, and closes a session by closing its QUIC connection with the code rather than with WT_CLOSE_SESSION; and quinn 0.11 has no RESET_STREAM_AT, so a reset WebTransport stream is not guaranteed to deliver its stream header first.
• 0-RTT is not accepted.
• Migration as a client: the relay sends GOAWAY to its sessions, but does not itself move its upstream subscriptions to a new relay, or follow a REDIRECT.
• Pausing upstream when every downstream subscriber is paused (§7.2), and forwarding NEW_GROUP_REQUEST upstream. EXPIRES is neither issued nor acted on.
• TOO_FAR_BEHIND: a subscription whose subscriber does not read is held, not ended (§5.2 allows ending it with PUBLISH_DONE TOO_FAR_BEHIND).
• A cancellation the relay cannot see: a subscriber that sends FIN on its direction and later only STOP_SENDING is noticed when the relay next writes on the request stream, or when the session ends.
• Receipts: the receipts track of a media track is found by the configured suffix, not by reading the catalog's wai_receipts_track; the relay signs no delivery-gap claim; datagrams at the door are checked against their group's receipts, not a property.
• Energy: no meter backs the relay, so its own figure is always the unmetered marker. Attributing a metered host's energy to sessions is not done here.
• The browser client and a page that uses it.

Tests

All on loopback, every wait bounded:

Test	What it shows
relay_quic, relay_webtransport, relay_websocket	Two publishers (one by PUBLISH_NAMESPACE and the relay's SUBSCRIBE, one by PUBLISH) and three subscribers over each transport; every object verifies against its publisher's receipts, which the door also checks; SUBSCRIBE_NAMESPACE; every summary verifies and labels the relay's energy unmetered.
slow_subscriber	A base track at priority 0 and a layer at 200 with a 100 ms delivery timeout. A subscriber that reads nothing for 1.5 s, then reads all its streams concurrently, loses layer objects to the timeout (their streams reset with DELIVERY_TIMEOUT), keeps the whole base track, and receives every receipt of both tracks although it asked for a timeout on its receipts tracks; each dropped object verifies by inclusion; the summary counts each layer object as forwarded or dropped.
late_joiner	TRACK_STATUS; FETCH of a track and its receipts from the cache; a range in descending order; INVALID_RANGE; a fill on SUBSCRIBE; a group that reaches the relay after a later one, the FETCH from it still served from the cache; a FETCH forwarded upstream when the cache holds nothing.
goaway	GOAWAY with its URI and timeout to every session; GOING_AWAY for a new request; GOAWAY_TIMEOUT for the session that stays.
auth_tokens	Tokens in SETUP, by value and by alias; UNAUTHORIZED; UNKNOWN_AUTH_TOKEN_ALIAS.
request_ids	INVALID_REQUEST_ID for the wrong parity and for a repeat; requests in flight bounded by the stream limit.
priority_fan_out	Two publishers, four tracks, three subscribers with their own subscriber priorities and group orders, their transports held shut while the load arrives: each receives every object in §5.1.2's order.

Licence

Apache-2.0.