29594e8ea9
#479 (Enhanced RTMP, SCTE 104) and #483 (SRT, RIST) both extend joule-moq-ingest; both ingests are kept whole, on one set of types: - frame.rs: one Codec with both sides' codecs (HEVC, VVC, AAC, FLAC; ALL is nine), #479's WEBRTC constant and Option-returning to_str0m, #483's webrtc() and aac_header; one Header whose audio rate is #479's `rate` (#483 called it `samplerate`; its uses are renamed); unframe keeps #483's whole-ADTS check for AAC; the module text names both ingests' codecs. - package.rs: one packager. HEVC and VVC take an Enhanced RTMP track's hint, and from a transport stream (no hint) the header or a default, as #483 did; AAC's first header is its ADTS header where there is no hint (#483) and the hint's otherwise (#479). - catalog.rs: #479's (name per entry; the rate as above); contrib.rs's catalog entry now names its layer, as #479 made every entry do. - config.rs, gateway.rs (routes, the Gateway's fields and methods, the server's listeners, srt_task and shutdown), rtc.rs, lib.rs, gateway.example.toml, the README and live-ingest's status, header and test account: both sides. - tests/common: watch_kinds -> watch_layers (#479) -> watch_tracks (#483), one subscriber; Watched.layer is the track name where it is a number. tests/ertmp.rs imports common::rtmp::adts by name, since common now has #483's adts too. - joule-code-conformance.yml's paths, the live-conformance README; CHANGELOG, #483's entry on top. cargo check --locked --all-targets of joule-moq-ingest (with rist), joule-contrib (with rist), joule-moq-relay and joule-moq passes. Co-Authored-By: Claude Opus 5.5 <[email protected]>
343 lines
18 KiB
YAML
343 lines
18 KiB
YAML
name: joule-code conformance gate
|
||
|
||
# Enforces the Joule Code conformance contract on every change: the reference
|
||
# library must build and pass its tests, and the published C1–C4 wire vectors
|
||
# must round-trip (SPEC §8). "Conformant = round-trips the vectors" — checked,
|
||
# not asserted.
|
||
|
||
# One run per branch at a time. A push supersedes the run before it.
|
||
#
|
||
# There is one `macos` runner and it is the dev Mac, so every macOS job in this
|
||
# repo queues on it. With no concurrency group, a session pushing four versions
|
||
# of a branch queued four complete sets of runs and every one of them had to
|
||
# drain: on 2026-10-05 a PR's 21 contexts settled none in 70 minutes behind a
|
||
# backlog of 25 runs, and the queue grew faster than it cleared. Only the newest
|
||
# commit's verdict is wanted, so the older runs are cancelled.
|
||
#
|
||
# Verified on this forge (Forgejo 16.0.5) in `ci-test` before being added here:
|
||
# the key parses, and a newer push moves an older run to `cancelled` even while
|
||
# it is still queued, which is the case that matters.
|
||
#
|
||
# `cancel-in-progress: true` also applies to `main`. That is deliberate and
|
||
# safe here: main advances by merge commits whose tree contains the commits
|
||
# before them, so the newer run covers the older one's changes.
|
||
concurrency:
|
||
group: ${{ github.workflow }}-${{ github.ref }}
|
||
cancel-in-progress: true
|
||
|
||
on:
|
||
pull_request:
|
||
paths:
|
||
- 'joule-code/**'
|
||
# joule-moq's receipt tests pin leaves, roots and signatures copied from WAI's
|
||
# jwp-conformance corpus, and run every leaf, group and object vector in it:
|
||
# a change there must re-run them.
|
||
- 'wai/jwp-conformance/**'
|
||
# joule-moq's tests/staged_moq_trace.rs runs the MoQ trace of
|
||
# wai/staged-conformance/moq/ over the envelopes of one of its cases.
|
||
- 'wai/staged-conformance/**'
|
||
# joule-moq-ingest's tests/room_corpus.rs holds its reading of room descriptors
|
||
# and grants to the live-rooms corpus.
|
||
- 'wai/live-conformance/rooms/**'
|
||
# joule-moq-ingest's tests/conformance_corpus.rs regenerates the Enhanced RTMP and the
|
||
# cue-injection corpora byte for byte, and holds these texts' reason tables to the codes
|
||
# it reports.
|
||
- 'wai/live-conformance/ertmp/**'
|
||
- 'wai/live-conformance/injection/**'
|
||
- 'wai/extensions/live-ingest.md'
|
||
- 'wai/extensions/cue-injection.md'
|
||
# joule-moq-ingest's tests/link_corpus.rs and its generator's --check read the
|
||
# contribution link records' corpus.
|
||
- 'wai/live-conformance/links/**'
|
||
# joule-moq's tests/secure_objects_vectors.rs and joule-moq-mls's tests/corpus.rs
|
||
# read the end-to-end encrypted rooms corpus, and the latter that extension's §13.
|
||
- 'wai/live-conformance/e2ee/**'
|
||
- 'wai/extensions/confidentiality.md'
|
||
# joule-moq's tests/rating_vectors.rs reads every rating claim and serving request of
|
||
# the age-assurance corpus, and the relay's ratings are held to the same reading.
|
||
- 'wai/live-conformance/age/**'
|
||
# joule-moq's tests/loc_vectors.rs regenerates and checks the LOC carriage corpus, from
|
||
# access units of the H.265 vector it reads, and requires loc-carriage §6's table.
|
||
- 'wai/live-conformance/loc/**'
|
||
- 'wai/format-probe-conformance/ext/hevc/**'
|
||
- 'wai/extensions/loc-carriage.md'
|
||
- '.github/workflows/joule-code-conformance.yml'
|
||
push:
|
||
branches:
|
||
- main
|
||
paths:
|
||
- 'joule-code/**'
|
||
# joule-moq's receipt tests pin leaves, roots and signatures copied from WAI's
|
||
# jwp-conformance corpus, and run every leaf, group and object vector in it:
|
||
# a change there must re-run them.
|
||
- 'wai/jwp-conformance/**'
|
||
# joule-moq's tests/staged_moq_trace.rs runs the MoQ trace of
|
||
# wai/staged-conformance/moq/ over the envelopes of one of its cases.
|
||
- 'wai/staged-conformance/**'
|
||
# joule-moq-ingest's tests/room_corpus.rs holds its reading of room descriptors
|
||
# and grants to the live-rooms corpus.
|
||
- 'wai/live-conformance/rooms/**'
|
||
# joule-moq-ingest's tests/conformance_corpus.rs regenerates the Enhanced RTMP and the
|
||
# cue-injection corpora byte for byte, and holds these texts' reason tables to the codes
|
||
# it reports.
|
||
- 'wai/live-conformance/ertmp/**'
|
||
- 'wai/live-conformance/injection/**'
|
||
- 'wai/extensions/live-ingest.md'
|
||
- 'wai/extensions/cue-injection.md'
|
||
# joule-moq-ingest's tests/link_corpus.rs and its generator's --check read the
|
||
# contribution link records' corpus.
|
||
- 'wai/live-conformance/links/**'
|
||
# joule-moq's tests/secure_objects_vectors.rs and joule-moq-mls's tests/corpus.rs
|
||
# read the end-to-end encrypted rooms corpus, and the latter that extension's §13.
|
||
- 'wai/live-conformance/e2ee/**'
|
||
- 'wai/extensions/confidentiality.md'
|
||
# joule-moq's tests/rating_vectors.rs reads every rating claim and serving request of
|
||
# the age-assurance corpus, and the relay's ratings are held to the same reading.
|
||
- 'wai/live-conformance/age/**'
|
||
# joule-moq's tests/loc_vectors.rs regenerates and checks the LOC carriage corpus, from
|
||
# access units of the H.265 vector it reads, and requires loc-carriage §6's table.
|
||
- 'wai/live-conformance/loc/**'
|
||
- 'wai/format-probe-conformance/ext/hevc/**'
|
||
- 'wai/extensions/loc-carriage.md'
|
||
- '.github/workflows/joule-code-conformance.yml'
|
||
|
||
jobs:
|
||
conformance:
|
||
name: Build + round-trip C1-C4 vectors
|
||
runs-on: linux
|
||
steps:
|
||
- name: Checkout
|
||
uses: actions/checkout@v4
|
||
|
||
# Bootstrap rustup: the docker runner image ships no Rust, and
|
||
# dtolnay/rust-toolchain cannot help — it calls `rustup`, which is absent.
|
||
- name: Install Rust 1.98.0
|
||
run: |
|
||
export PATH="$HOME/.cargo/bin:$PATH"
|
||
command -v rustup >/dev/null 2>&1 || \
|
||
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.98.0
|
||
rustup toolchain install 1.98.0 --profile minimal
|
||
rustup default 1.98.0
|
||
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
|
||
|
||
- name: Cache cargo
|
||
uses: actions/cache@v4
|
||
with:
|
||
path: |
|
||
~/.cargo/registry
|
||
~/.cargo/git
|
||
joule-code/joule-code-rs/target
|
||
key: joule-code-${{ runner.os }}-${{ hashFiles('joule-code/joule-code-rs/Cargo.lock') }}
|
||
|
||
# joule-code-remediate's formal gate shells out to `z3`, and the runner
|
||
# image has no solver. Its tests assert in both branches. With a solver
|
||
# installed here, JOULE_FIX_EXPECT_Z3=1 makes its absence a failure, so
|
||
# this gate runs the real-solver branch rather than the fail-closed one.
|
||
- name: Install z3
|
||
shell: bash
|
||
run: |
|
||
SUDO=""; [ "$(id -u)" -ne 0 ] && SUDO="sudo"
|
||
$SUDO apt-get update -qq
|
||
$SUDO apt-get install -y -qq z3
|
||
z3 --version
|
||
|
||
# joule-moq-relay, joule-moq-ingest and joule-contrib run in jobs of their own
|
||
# below, on linux-arm64: they need nothing x86-specific, and this runner is the one
|
||
# there is least of.
|
||
- name: Reference library tests
|
||
working-directory: joule-code/joule-code-rs
|
||
env:
|
||
JOULE_FIX_EXPECT_Z3: '1'
|
||
run: cargo test --workspace --exclude joule-moq-relay --exclude joule-moq-ingest --exclude joule-contrib
|
||
|
||
# joule-moq's QUIC transport and its example compile only with `quic`. Named
|
||
# here so they stay built and tested whatever the workspace's other members
|
||
# happen to enable; this also runs tests/wai_jwp_corpus.rs, every leaf, group
|
||
# and object vector of wai/jwp-conformance through the crate.
|
||
- name: joule-moq with the QUIC transport, every target
|
||
working-directory: joule-code/joule-code-rs
|
||
run: cargo test -p joule-moq --features quic --all-targets
|
||
|
||
# End-to-end encrypted rooms (WAI extensions/confidentiality.md). joule-moq's sealed
|
||
# objects, named so they stay built and tested whatever the workspace's other members
|
||
# enable: RFC 9605's and draft-ietf-moq-secure-objects-01's published vectors, and
|
||
# the receiving checks. Then joule-moq-mls: real MLS groups (a room of three each
|
||
# sending, a removal, a late joiner, a change of committer, and every misbehaviour
|
||
# refused), and the corpus of wai/live-conformance/e2ee regenerating byte for byte
|
||
# with every vector's outcome, and members behind a log that orders their messages
|
||
# (confidentiality §15.2). The relay's own job runs tests/sealed_objects.rs.
|
||
- name: Sealed objects and the room's MLS binding
|
||
working-directory: joule-code/joule-code-rs
|
||
run: |
|
||
set -e
|
||
cargo test -p joule-moq --features secure-objects --lib secure_objects
|
||
cargo test -p joule-moq --features secure-objects --test secure_objects_vectors
|
||
cargo test -p joule-moq-mls
|
||
cargo run -q -p joule-moq-mls --bin joule_moq_e2ee_vectors verify ../../wai/live-conformance/e2ee
|
||
# The browser member's surface (joule-moq-web, feature e2ee), on the host: its
|
||
# tests run only with the feature.
|
||
cargo test -p joule-moq-web --features e2ee --lib
|
||
|
||
# joule-moq's draft-ietf-moq-transport-22 vectors, read by a second decoder
|
||
# written from the draft's text rather than from the crate. The step above
|
||
# (tests/draft22_vectors.rs) holds the crate to the same corpus and fails if
|
||
# the committed corpus is not what its generator makes. This one must read
|
||
# every good vector as stated and refuse every negative one as stated; it
|
||
# fails on an empty corpus.
|
||
- name: Ensure python3 is available
|
||
shell: bash
|
||
run: |
|
||
if ! command -v python3 >/dev/null 2>&1; then
|
||
SUDO=""; [ "$(id -u)" -ne 0 ] && SUDO="sudo"
|
||
$SUDO apt-get update -qq
|
||
$SUDO apt-get install -y -qq python3
|
||
fi
|
||
python3 --version
|
||
|
||
- name: joule-moq draft-22 vectors, independent decoder
|
||
run: python3 -I joule-code/joule-code-rs/crates/joule-moq/tests/vectors/draft-22/moqt22_check.py
|
||
|
||
# The same for draft-ietf-moq-transport-18 and -16, each corpus read by a decoder
|
||
# written from that draft's text (tests/draft18_vectors.rs and
|
||
# tests/draft16_vectors.rs hold the crate to them in the step above). The
|
||
# interoperability runs against other implementations' relays reach the public
|
||
# internet and are not run here (joule-moq-interop's scripts/public.py).
|
||
- name: joule-moq draft-18 vectors, independent decoder
|
||
run: python3 -I joule-code/joule-code-rs/crates/joule-moq/tests/vectors/draft-18/moqt18_check.py
|
||
|
||
- name: joule-moq draft-16 vectors, independent decoder
|
||
run: python3 -I joule-code/joule-code-rs/crates/joule-moq/tests/vectors/draft-16/moqt16_check.py
|
||
|
||
- name: Round-trip the v1 conformance pack (C1-C4)
|
||
working-directory: joule-code/joule-code-rs
|
||
run: |
|
||
set -e
|
||
out="$(cargo run -q -p joule-code-conformance -- verify ../conformance/v1)"
|
||
echo "$out"
|
||
# The pack must pass with zero failures. The summary line reads
|
||
# "N passed, F failed, S skipped"; anchor on it, so that "10 failed" can never
|
||
# read as "0 failed", and require at least one pass, so an empty pack fails.
|
||
echo "$out" | grep -Eq '^[1-9][0-9]* passed, 0 failed, ' || { echo "::error::conformance pack reported failures"; exit 1; }
|
||
|
||
- name: L3 JouleClaw binding example builds + tests
|
||
working-directory: joule-code/examples/l3-jouleclaw
|
||
run: cargo test
|
||
|
||
# The MOQT relay, at draft-22 and, through its bridge, -18 and -16 (tests/drafts.rs,
|
||
# tests/publish_done_streams.rs). Every test is on loopback: two publishers and three
|
||
# subscribers over native QUIC, WebTransport over HTTP/3 and the WebSocket fallback; a
|
||
# room's sealed objects, forwarded byte for byte and checked at the door on their
|
||
# receipts over the sealed bytes, opened by members and by no one else;
|
||
# a slow subscriber whose late layers time out while every receipt still arrives; a
|
||
# late joiner served by FETCH and a fill from the cache, and a FETCH forwarded
|
||
# upstream; GOAWAY on shutdown; tokens by alias; Request IDs and the stream limit;
|
||
# fan-out order under load. `--all-targets` runs the unit tests and the binary's
|
||
# build as well.
|
||
moq-relay:
|
||
name: joule-moq-relay, every target
|
||
runs-on: linux-arm64
|
||
timeout-minutes: 45
|
||
steps:
|
||
- name: Checkout
|
||
uses: actions/checkout@v4
|
||
|
||
- name: Install Rust 1.98.0
|
||
shell: bash
|
||
run: |
|
||
export PATH="$HOME/.cargo/bin:$PATH"
|
||
command -v rustup >/dev/null 2>&1 || \
|
||
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.98.0
|
||
rustup toolchain install 1.98.0 --profile minimal
|
||
rustup default 1.98.0
|
||
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
|
||
|
||
- name: joule-moq-relay, every target
|
||
working-directory: joule-code/joule-code-rs
|
||
shell: bash
|
||
run: cargo test --locked -p joule-moq-relay --all-targets
|
||
|
||
# The relay's WebTransport layer, wtransport 0.7.2 vendored with a session close
|
||
# (vendor/wtransport/VENDORED.md): its unit tests, the WT_CLOSE_SESSION capsule's bytes
|
||
# against the layout of draft-ietf-webtrans-http3-16 §6 among them. It is not a member
|
||
# of the workspace, so `--all-targets` above does not reach it.
|
||
- name: The vendored wtransport's unit tests
|
||
working-directory: joule-code/joule-code-rs
|
||
shell: bash
|
||
run: cargo test --locked -p wtransport --lib
|
||
|
||
# The relay's own energy on a counter (WAI live-metering §8): tests/relay_energy.rs on
|
||
# macOS, where the relay reads its process's per-task CPU energy unprivileged. The test
|
||
# is required here (JOULE_MOQ_RELAY_ENERGY_REQUIRE): a host with no counter fails it
|
||
# rather than skipping. On linux-arm64 above the same test prints SKIPPED, which is not a
|
||
# pass of the measurement: that VM has no counter the relay reads.
|
||
moq-relay-energy:
|
||
name: joule-moq-relay, its own energy on a counter (macos)
|
||
runs-on: macos
|
||
timeout-minutes: 45
|
||
steps:
|
||
- name: Checkout
|
||
uses: actions/checkout@v4
|
||
|
||
- name: The room's record carries the relay's measured energy, and the marker where it is not the room's
|
||
working-directory: joule-code/joule-code-rs
|
||
shell: bash
|
||
env:
|
||
JOULE_MOQ_RELAY_ENERGY_REQUIRE: '1'
|
||
run: |
|
||
set -euo pipefail
|
||
export PATH="$HOME/.cargo/bin:$PATH"
|
||
cargo test --locked -p joule-moq-relay --test relay_energy -- --nocapture --test-threads=1 2>&1 | tee "$RUNNER_TEMP/relay-energy.log"
|
||
grep -q "test result: ok. 3 passed; 0 failed" "$RUNNER_TEMP/relay-energy.log"
|
||
grep -q "measured: .* scope task" "$RUNNER_TEMP/relay-energy.log"
|
||
! grep -q "SKIPPED" "$RUNNER_TEMP/relay-energy.log"
|
||
|
||
# The ingest gateway. Every test is on loopback, against a relay that checks every
|
||
# receipt at its door: a WHIP publisher (str0m as the client) of H.264 and Opus whose
|
||
# frames a relay subscriber receives whole with every receipt verified, and DELETE ending
|
||
# the tracks; a WHEP player that gets the same frames from the group it joined; VP8, VP9
|
||
# and AV1 through WHIP; the refusals of RFC 9725 and RFC 6750; trickle ICE and an ICE
|
||
# restart with the media carried across it; the room reader against the live-rooms
|
||
# corpus; an Enhanced RTMP publisher of the tests' own over RTMP and RTMPS (every payload
|
||
# framing, multitrack layers, the stream key's refusals, the reconnect request); an SCTE
|
||
# 104 automation system of the tests' own driving the injector, the chain read at the
|
||
# relay; and the Enhanced RTMP and cue-injection corpora regenerated byte for byte.
|
||
# `--all-targets` runs the unit tests and builds the binary. tests/ertmp_ffmpeg.rs, which
|
||
# runs a local encoder as the test client, is ignored here: this runner has no encoder,
|
||
# and the run lists it as ignored, not passed.
|
||
moq-ingest:
|
||
name: joule-moq-ingest, every target
|
||
runs-on: linux-arm64
|
||
timeout-minutes: 45
|
||
steps:
|
||
- name: Checkout
|
||
uses: actions/checkout@v4
|
||
|
||
- name: Install Rust 1.98.0
|
||
shell: bash
|
||
run: |
|
||
export PATH="$HOME/.cargo/bin:$PATH"
|
||
command -v rustup >/dev/null 2>&1 || \
|
||
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.98.0
|
||
rustup toolchain install 1.98.0 --profile minimal
|
||
rustup default 1.98.0
|
||
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
|
||
|
||
- name: joule-moq-ingest, every target
|
||
working-directory: joule-code/joule-code-rs
|
||
shell: bash
|
||
run: cargo test --locked -p joule-moq-ingest --all-targets
|
||
|
||
# SRT and RIST contribution (WAI extensions/live-contribution.md). joule-contrib's
|
||
# transports on loopback through lossy paths, each with and without the feature
|
||
# `rist` (off by default, its §10); the gateway's SRT and RIST sessions into a relay
|
||
# that checks every receipt; the link record against its corpus, which the generator
|
||
# regenerates and compares byte for byte. The tests against the reference tools are
|
||
# ignored here: they run where the tools are installed.
|
||
- name: joule-contrib and joule-moq-ingest with RIST, every target
|
||
working-directory: joule-code/joule-code-rs
|
||
shell: bash
|
||
run: |
|
||
cargo test --locked -p joule-contrib --all-targets
|
||
cargo test --locked -p joule-contrib --all-targets --features rist
|
||
cargo test --locked -p joule-moq-ingest --all-targets --features rist
|
||
cargo run --locked -q -p joule-moq-ingest --bin joule-moq-ingest-link-vectors -- --check ../../wai/live-conformance/links
|