Transaction-Science/open-standards
Watch
1
0
Fork
You've already forked open-standards
 0
Code Issues Pull requests Projects Releases 17 Packages Wiki Activity Actions
open-standards/.github/workflows/wai-conformance.yml
Transaction Science ac10a748f5 SPEC §5: a sink that presents video decodes AV1, the video floor, with its decoder suite 
The mandatory floor (PNG, FLAC, zstd) has no video capability, so a bare
envelope carrying video had no codec every video sink holds, while packaging
§7 already gives every packaged video piece an AV1 floor rendition. A sink
that presents the video media class now MUST decode wai.video.av1. The floor
is scoped to the class: the mandatory floor holds every sink, audio players,
text readers and devices that show no picture among them, and none of those
can use a video decoder.

- SPEC §1, §5 (Video floor) and §7: the rule, what presenting a class means,
  and the conformance consequence.
- SPEC §7 Digests: RGB8 is defined in place, and a standard codec whose
  standard fixes its samples in another layout and no conversion to RGB8 (AV1,
  AVIF) is digested in that layout, I420 for 8-bit 4:2:0 AV1, as the
  grain-free AV1 corpus already did.
- Registry: wai.video.av1 joins the set video-floor (codecs::video_floor); the
  schema's set enumeration gains the value.
- floor-conformance/: the decoder suite, a case for each mandatory-floor
  capability and the video-floor case (a grain-free AV1 stream, each I420
  frame's digest and the clip's), with the class rule as sink rows.
  tests/floor_conformance.rs runs it in the floor build, and through libdav1d
  in the codec-wrappers job; floor_verify.py, the second implementation,
  decodes PNG and FLAC itself, checks the class rule against the registry and
  catches every planted fault.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-10 13:42:23 -05:00

1560 lines
97 KiB
YAML
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

name: WAI conformance
# WAI's load-bearing claim is bit-exact reconstruction on every machine,
# no tolerance parameter. This workflow enforces it mechanically: every
# cargo-class corpus — and the integer-exact neural decoders, which make the
# same promise (the registry's NeuralIntegerExact class: i64-only, no float
# ONNX) — is re-derived on every OS/arch leg of the matrix below (x86-64 and
# arm64) and must reproduce its frozen BLAKE3 hashes; and the integer
# byte-equality goldens (rANS / int_synth / JPEG-AI mANS, generated once on
# a reference machine) must decode identically everywhere. All of it is pure
# Rust (no C codec chain, no ONNX Runtime), so the matrix needs no system
# dependencies.
# A pull request's newer push supersedes its older run; a push to main does not.
#
# On a pull request only the newest commit's verdict is wanted. With no
# concurrency group, a session pushing four versions of a branch queued four
# complete sets of runs and every one of them had to drain: on 2026-10-05 a PR's
# 21 contexts settled none in 70 minutes behind a backlog of 25 runs, and the
# queue grew faster than it cleared. So a pull request's older runs are
# cancelled.
#
# Verified on this forge (Forgejo 16.0.5) in `ci-test` before being added here:
# the key parses, and a newer push moves an older run to `cancelled` even while
# it is still queued, which is the case that matters.
#
# A push to main takes a group of its own, its commit, so no later push cancels
# its run. Work lands as direct commits to main, several an hour, and Forgejo
# hands runners the oldest queued job first, so a run the next push cancelled
# was replaced at the back of the queue: while the linux-arm64 queue was hours
# deep, a workflow touched more often than that never reached the front. In the
# day to 2026-11-10 01:11Z, 578 of the 810 runs main's pushes started were
# cancelled; joule-moq-room's last run on main to finish ended at 13:58Z on
# 11-09, and 31 of the 32 after it were cancelled. `cancel-in-progress: false`
# would not help: Forgejo then holds each run of a group until the one before it
# has finished, and keeps every one waiting. Forgejo cancels a job left waiting
# 24 hours.
concurrency:
group: ${{ github.workflow }}-${{ github.event_name == 'push' && github.sha || github.ref }}
cancel-in-progress: true
on:
pull_request:
paths:
- 'wai/wai-rs/**'
# `wai create` holds the examples' files (include_bytes!), and tests/cli_dx.rs holds them
# to wai/examples/ byte for byte.
- 'wai/examples/**'
# The project file's reader (dx/project.rs, in the binary's tests) holds wai.json's schema to it.
- 'wai/schema/**'
# `wai doctor` holds this file's port table (include_str!), and tests/cli_doctor.rs holds it to the table.
- 'wai/quickstart/README.md'
# wai-rs path-depends on wai-quantum, and the quantum corpora below are
# verified through it: a change there changes what this workflow checks.
- 'wai/wai-quantum/**'
# The integer decoders that decode through the integer operator set depend on it.
- 'wai/wai-ops/**'
# `wai create --from app.oui`, `wai app` and `wai mcp --toolset app` (the feature `app`)
# link Create's library and the joule-ui crates it builds on.
- 'wai/wai-app/**'
- 'joule-ui/joule-ui-rs/crates/joule-ui-core/**'
- 'joule-ui/joule-ui-rs/crates/joule-ui-lang/**'
- 'joule-ui/joule-ui-rs/crates/joule-ui-interaction/**'
- 'joule-ui/joule-ui-rs/crates/joule-ui-catalogue/**'
- 'wai/*-conformance/**'
- 'wai/byte-exact-conformance/**'
# The registry export and the two library tests that read files outside
# wai-rs (the codec glossary, SPEC.md) must fire this workflow too: a
# hand edit of an exported file, of SPEC §5 or of the glossary is checked
# here. vocab/v1's and v2's pin terms are checked by a library test.
- 'wai/registry/**'
- 'wai/vocab/**'
- 'wai/glossary/**'
- 'wai/SPEC.md'
- 'wai/REGISTRY.md'
# A library test checks SPEC §5's pinned integer payloads against this
# extension's sections, so an edit of it is checked here too.
- 'wai/extensions/integer-payloads.md'
# A library test (staged::tests::staged_delivery_text_lists_the_codes) checks
# that this extension's §4.2 and A.6 list the codes the sink reports.
- 'wai/extensions/staged-delivery.md'
# A library test (codecs::registry::tests::wire_identifiers_are_in_their_documents)
# checks every wire identifier against the section of these texts it names, and
# staged::moq::tests::moq_streaming_format_lists_the_codes checks §4.1's codes.
- 'wai/extensions/http-delivery.md'
- 'wai/extensions/moq-streaming-format.md'
- 'wai/extensions/jwp-receipts.md'
# The quantum referee requires this extension's Appendix B to state its phase tables.
- 'wai/extensions/quantum-sim.md'
# The ad-presentation step runs the browser client's reference producer against the
# corpus, and the stitched-presentation step runs its stitched producer.
- 'joule-code/joule-code-rs/crates/joule-moq-web/web/ads/**'
# The tool that trained the face corpus's generator and wrote its inputs and its report.
- 'wai/tools/wai_avatar_face_train.py'
# browser_sink_verify.py reads capability-constraints' §3.1 register, and
# tests/stream_termination.rs requires its §4.2 to name the match's bits.
- 'wai/extensions/capability-constraints.md'
# The browser sink's functions for a page, held to the same corpus natively.
- 'wai/wai-live-web/**'
# The co-watch step runs the browser client's co-watch module against the corpus.
- 'joule-code/joule-code-rs/crates/joule-moq-web/web/room/cowatch*'
# The remote desktop step runs the browser viewer's module against the corpus.
- 'joule-code/joule-code-rs/crates/joule-moq-web/web/room/remote*'
# tests/telemetry_aggregate.rs requires §3.4's table to class the keys the reference
# classes, and telemetry_aggregate_verify.py reads that table and §4.1 and §4.2's shards.
- 'wai/extensions/delivery-telemetry.md'
# splat_exact_verify.py decodes each payload with this tool's whm1/1 lowering and
# interpreter.
- 'wai/tools/wai_ops_check.py'
# tests/format_probe.rs requires projection.md §4 and §5 to state its codes, and the
# registry's wire identifiers test reads projection.md, loc-carriage.md and
# delivery-telemetry.md (whose §6.6 cmsd_read_verify.py reads).
- 'wai/extensions/projection.md'
- 'wai/extensions/loc-carriage.md'
- 'wai/extensions/delivery-telemetry.md'
# The determinism-cost record, its verifier, and the two texts whose quoted figures
# the verifier checks against the record.
- 'wai/research/determinism-cost.md'
- 'wai/research/determinism-cost/**'
- 'wai/tools/wai_determinism_cost_verify.py'
- 'wai/extensions/determinism-tiers.md'
# The catalogue-prior record, its verifier, and the page whose quoted figures the
# verifier checks against the readings.
- 'wai/research/catalogue-prior.md'
- 'wai/research/catalogue-prior/**'
- 'wai/tools/wai_catalogue_prior_verify.py'
# The learned arm's harness, whose self-test reads the record (verifiers/catalogue-harness.sh).
- 'wai/tools/wai_int_hyper_catalogue.py'
- 'wai/wai-web/**'
# The dcvc-drift record and its readings, fixtures and verifier, and the text-dictionary
# record and its corpus, rule, dictionaries and script: each step checks its page's
# quoted figures against the committed files (verifiers/dcvc-drift.sh and
# verifiers/text-dictionary.sh name the same paths).
- 'wai/research/dcvc-drift.md'
- 'wai/research/dcvc-drift/**'
- 'wai/research/text-dictionary.md'
- 'wai/research/text-dictionary/**'
# The WAISM profile's two checkers, its corpus and their differential.
- 'wai/wai-module-check/**'
# The integer level's module of record, which both checkers hold, its plant and its corpus.
- 'wai/wai-int-module/**'
# The float level's module of record, which both checkers hold there, and its corpus.
- 'wai/wai-det-jpegai/**'
# The audio provenance check below reads the corpora above with this tool.
- 'wai/tools/wai_audio_provenance_check.py'
# The trained int_synth vector's picture check below runs this tool.
- 'wai/tools/wai_int_synth_real.py'
# The trained int_hyper set's record check (verifiers/int-hyper-set.sh) runs this tool.
- 'wai/tools/wai_int_hyper_train.py'
- '.github/workflows/wai-conformance.yml'
# The steps of the independent verifiers, the corpora of cargo-classes, and the script
# that derives this workflow's lists from them (scripts/wai-conformance-lists.py).
- '.github/wai-conformance/**'
- 'scripts/wai-conformance-lists.py'
# Each verifier step names the extension texts its tests and verifier read, in its own
# file (`# paths:`), so a new step adds no line here. The forge cannot read those files,
# so it starts this workflow on any extension text, and the gate below (the `relevant`
# job) runs it only for the paths the files name and the ones listed above.
- 'wai/extensions/**'
# No global `shell:` default. The Windows runner is a host executor whose
# `bash` is the WSL stub (a different OS), so shell-specific steps below pick
# their shell per-OS: bash on unix, pwsh on Windows. Bare `cargo …` steps take
# each runner's native default: `sh -e` in the linux containers, `bash -e` on
# the macOS host, and pwsh on Windows. pwsh fails a step on its last command's
# status alone, so a bare step that runs on Windows and holds two commands
# chains them with `&&`, and a pwsh step holding more than one sets
# $PSNativeCommandUseErrorActionPreference, which stops it at the first command
# that fails. The cargo-class matrix has a Windows leg; the byte-exact matrix
# leaves Windows to wai-byte-equality.yml (see the comment there).
schedule:
- cron: '20 5 * * *'
jobs:
# Does this pull request's own diff touch the paths below? This forge matches a
# workflow's `paths:` against the push, so a merge of main into a branch fires the
# workflow whatever the pull request changes; every job here waits on this answer.
# A push to main, a schedule, or anything the script cannot determine answers
# `true` (scripts/pr-touches-paths.py). The paths are the list above, with
# `wai/extensions/**` replaced by the paths the verifier steps' files name (--wide).
relevant:
name: does this change touch it
runs-on: gate
outputs:
run: ${{ steps.gate.outputs.run }}
steps:
- uses: actions/checkout@v5
with:
fetch-depth: 0
- id: gate
run: |
set -euo pipefail
r=$(python3 scripts/pr-touches-paths.py .github/workflows/wai-conformance.yml \
"${{ github.event.pull_request.base.sha }}" \
"${{ github.event.pull_request.head.sha }}" \
--paths-from .github/wai-conformance/verifiers --wide 'wai/extensions/**')
echo "this change touches wai-conformance: $r"
echo "run=$r" >> "$GITHUB_OUTPUT"
cargo-classes:
needs: relevant
if: needs.relevant.outputs.run == 'true'
name: cargo-class corpora · ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
# Self-hosted fleet labels. Both architectures: x86_64 by linux, arm64
# by linux-arm64 + macos — what matters for the Fx integer floor is the
# arch, not the OS. No Intel-Mac runner exists in the fleet, and it
# would add no arch coverage anyway.
# windows: x86_64 under another OS, compiler and C runtime (cbio-windows,
# a host executor). Out from 2026-09-30, while its runner was offline, and
# back since it is. Not a required status: a job on an offline runner
# stays pending and never settles, so if the runner goes offline again
# the leg comes out of this list until it is back.
os: [linux, linux-arm64, macos, windows]
# No debuginfo: nothing in CI reads DWARF, and it is most of a debug target's size. Three of
# these jobs at once on the linux-arm64 VM's 196 GB data disk ran out of space mid-link
# ("No space left on device", #542 and #550). Panic messages keep their file:line, which
# comes from the panic location, not from DWARF (standards-build.yml trims jouleclaw-rs the
# same way: ~6 GB without, against the full-debuginfo target that filled its disk).
env:
CARGO_PROFILE_DEV_DEBUG: '0'
steps:
- uses: actions/checkout@v5
# unix: the docker runners (linux / linux-arm64) ship no Rust, so rustup
# is bootstrapped; the macOS `host` runner has it but launchd's PATH omits
# ~/.cargo/bin. dtolnay/rust-toolchain is unusable — it calls a `rustup`
# the docker image lacks.
- name: Install Rust 1.99.1 (unix)
if: matrix.os != 'windows'
shell: bash
run: |
export PATH="$HOME/.cargo/bin:$PATH"
command -v rustup >/dev/null 2>&1 || \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.99.1
rustup toolchain install 1.99.1 --profile minimal
rustup default 1.99.1
rustup component add --toolchain 1.99.1 clippy
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
# Windows: a host executor with rustup already installed. Only ensure the
# pinned toolchain is present + default. pwsh, never bash (that is WSL here).
- name: Install Rust 1.99.1 (Windows)
if: matrix.os == 'windows'
shell: pwsh
run: |
$PSNativeCommandUseErrorActionPreference = $true
rustup toolchain install 1.99.1 --profile minimal
rustup default 1.99.1
# scripts/wai-conformance-lists.py reads the integer kernels' tests and the corpora below.
- name: python3 for the derived lists (unix)
if: matrix.os != 'windows'
shell: bash
run: |
command -v python3 >/dev/null 2>&1 && exit 0
SUDO=""; [ "$(id -u)" -ne 0 ] && SUDO="sudo"
$SUDO apt-get update -qq
$SUDO apt-get install -y -qq python3
# On the Windows host `python3` is the WindowsApps alias, which runs no Python ("Python was
# not found; run without arguments to install from the Microsoft Store"); its Python is
# `python`. The steps below run `python3`, and so do tests that spawn it (wai mcp's
# conformance_run among them), where Rust starts only a `.exe`: so a virtual environment of
# the host's Python, with its launcher also named python3.exe, goes first on the PATH. No
# packages: the scripts and verifiers these steps run use the standard library alone.
- name: python3 for the derived lists (Windows)
if: matrix.os == 'windows'
shell: pwsh
run: |
$PSNativeCommandUseErrorActionPreference = $true
$v = Join-Path $env:RUNNER_TEMP 'python3-venv'
python -m venv --clear --without-pip $v
$bin = Join-Path $v 'Scripts'
Copy-Item -Force (Join-Path $bin 'python.exe') (Join-Path $bin 'python3.exe')
& (Join-Path $bin 'python3.exe') -I -c "import sys; print('python3 is', sys.version.split()[0], 'at', sys.executable)"
$bin | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
- name: Build the cargo classes (pure Rust, no C codec chain)
working-directory: wai/wai-rs
# neural_int joins the pure-Rust set: the integer-synthesis decoder is
# i64-only (no ONNX, no float), so it carries the same bit-exact promise
# and is checkable on this matrix with no ONNX Runtime present. receipt_claims
# is in the set because the reference meters (wai_meter, wai_world_meter) write a
# measurement report by default and require it: without it they would drop out
# of this build on every leg of this matrix, without a word. staged and the integer
# media features build the staged-delivery core, its corpus tool and `wai stage`;
# stage_measure builds the staged-measurement core and wai_stage_measure; telemetry
# the delivery-telemetry mapping and wai_telemetry; http the HTTP delivery binding,
# its corpus tool and `wai http`.
run: cargo build --no-default-features --features world,audio_scene,haptic,splat4d,score,provenance,neural_int,film,receipt_claims,staged,video_int,audio_int,splat_int,stage_measure,telemetry,http
# video_package runs the package receipt's dedup tests, which content_dedup feeds.
# The dev profile keeps overflow checks on, so an arithmetic overflow in the HTTP
# binding (http_binding::tests, among them a seeded test of every parser of untrusted
# input) fails here rather than wrapping.
- name: Library unit tests (the fixed-point determinism floors)
working-directory: wai/wai-rs
run: cargo test --lib --no-default-features --features world,audio_scene,haptic,splat4d,score,provenance,neural_int,film,oer,receipt_claims,key_discovery,staged,video_int,audio_int,splat_int,stage_measure,telemetry,video_package,http,format_probe
# The JPEG AI exact decoder reproduces across platforms only if no platform math
# library is on its path. Two guards, each failing the step: clippy's
# disallowed-methods (wai-rs/clippy.toml), which each module on the decode path
# denies, rejects a listed float method however it is spelled; and
# tests/jpegai_decode_symbols.rs lists the linked decoder's symbols with `nm` and
# rejects any C math-library function it imports, whatever made the call, and any
# it defines other than the round-to-integral ones (see its doc). `-A clippy::all`
# keeps the output to that guard. Unix legs only: `nm` is not on the Windows
# runner, and for clippy the unix legs already cover both architectures.
- name: JPEG AI exact decode calls no platform math library (unix)
if: matrix.os != 'windows'
shell: bash
working-directory: wai/wai-rs
run: |
set -euo pipefail
cargo clippy --lib --no-default-features --features neural_int -- -A clippy::all
cargo test --no-default-features --features world,audio_scene,haptic,splat4d,score,provenance,neural_int,film --test jpegai_decode_symbols
# Each extension container refuses a count its bytes cannot hold, without reserving
# it first (found by the coverage-guided fuzzer, wai-rs/fuzz/, target `containers`),
# and a declared duration past the reader limits it shares with SPEC §2.
- name: Extension containers reserve no more than their bytes hold, and are held to their duration limits
working-directory: wai/wai-rs
run: cargo test --no-default-features --features world,audio_scene,haptic,splat4d,score,film --test container_reservations --test extension_duration_limits
# The integer decoders run fast kernels (src/int_kernels.rs) that must give
# the same integer as the scalar reference kernels for every output element.
# The feature list above leaves out audio_int, video_int, world_int and
# splat_int, so without these two steps int_audio's kernel-equivalence tests
# and tests/int_kernels_vectors.rs (every committed integer-decoder vector
# decoded under the reference and under each fast policy, against its frozen
# hash) would run nowhere. `--lib` also carries the other kernels' equivalence
# modules; tests/int_kernels_geometry.rs runs edge geometry (strides up to
# usize::MAX, padding far beyond the kernel) through the public decoders
# under every policy. Both profiles, because the overflow-parity tests assert
# different things in each: with overflow checks on (dev) that the fast path panics
# where the reference panics, with them off (release) that it returns the
# same integers. All pure Rust, so every leg runs them.
# tests/int_synth_ops.rs holds wai.neural.int_synth to its lowering through the
# integer operator set (extensions/integer-operators.md §13.1): every committed set
# lowers to its committed graph and loads with its certificate's verdict, decodes
# through it to every committed golden, and refuses whatever the direct decode
# refuses. tests/int_ops_differential.rs (feature legacy_int: the direct decode,
# test-only) decodes random sets and payloads both ways: the same output or both
# refused, except on the planned breaks of integer-operators §15.3, where the decode
# through the operator set refuses. wai-ops-differential.yml runs it for an hour in
# each profile, nightly. tests/int_splat_ops.rs holds wai.splat.int_codec to its
# lowering whm1/1 the same way (integer-operators §13.2): its committed set lowers,
# from its bytes and from the model this crate parses, to its committed graph, and
# loads with its certificate's verdict. tests/int_audio_ops.rs does the same for
# wai.audio.int_codec and its lowering wam1/1 (§13.3), and tests/int_hyper_ops.rs
# for wai.neural.int_hyper and the keyframe of wai.video.int_hyper (whm1/1: h_s,
# then g_s with the RGB8 step), and tests/int_mlicpp_ops.rs for wai.neural.int_mlicpp
# (wmb1/1: h_s, entropy, then g_s with the RGB8 step), and tests/int_world_ops.rs for
# wai.world.model's integer renderer (world/1, informative: T with its action planes,
# then g_s with the RGB8 step).
# tests/int_mlicpp.rs decodes the committed wai.neural.int_mlicpp vector against
# its pinned hashes (image, every scale bucket, every symbol) under every kernel
# policy and thread count, and checks that damaged payloads are refused rather than
# overflowing — which is why it runs in both profiles too.
# tests/int_mlicpp_preflight.rs (its own binary: it installs a counting global
# allocator) checks that a header the model cannot fit is refused before the
# decode allocates anything sized from it.
# tests/int_audio.rs and tests/int_splat.rs decode the committed wai.audio.int_codec and
# wai.splat.int_codec vectors against the hashes the Python integer reference computed at
# export, and check each vector's metadata. `--lib` runs int_payload's unit tests: each
# binary payload (WIV1, WIH1, WIA1, WIS1) against the JSON it is converted from, under
# every kernel policy, and decode_int for every integer capability in this build, which
# has no `neural`. tests/int_payload.rs refuses every fixture of int-payload-conformance
# with its code, every strict prefix of every payload, and runs a byte-change sweep in
# which nothing may panic; tests/int_payload_preflight.rs (its own binary, a counting
# allocator) checks that a header past the caps is refused before anything sized from it
# is allocated; tests/int_payload_cost.rs (its own binary, a live-bytes allocator) checks
# that int_payload::declared_cost bounds what each decode holds, with the store of loaded
# sets off, then on (a decode that loads its set, and one that finds it held), that the
# store counts at least the bytes it holds, and, in the release step only, decodes a
# 2^30-byte clip a 3 KB payload codes; tests/int_set_cache.rs checks that store under
# the decodes: a second decode through a set is a hit with the same bytes on every
# committed vector, sets one byte apart never share an entry, a refused set stays
# refused, refusals keep their codes, and the least recently used set goes first;
# tests/int_world_memory.rs (its own binary, the same allocator) checks that the world
# renderer, with its set loaded, holds no more than the direct renderer it replaced.
# tests/int_video_enc_memory.rs
# (its own binary, the same allocator) checks that int_video_enc::working_bytes bounds
# what the encoder holds. tests/staged_resources.rs (its
# own binary, a counting allocator) checks that the staged sink's work and memory per
# layer grow linearly in the layer's units, that a layer past the chain budget is
# refused before any stream is decoded, that what no unit holds is dropped, and that a
# WIR1 parse holds no more than its declared working bytes. tests/stage_measure.rs counts every
# committed staged envelope as exactly its length, refuses JSON, base64 and another capability's
# format, and evaluates a measured title read from a map of files to its golden, refusing a
# changed byte of a layer, the base, the source or a parameter-set file (extensions/
# staged-measurement.md). Cargo.toml declares them, and the other
# feature-gated targets named here, with required-features, so if this step ever loses a
# feature, cargo stops with an error instead of compiling an empty binary that reports ok.
# Each committed vector's decoded output is also checked against the SHA-256 in
# its sha256.json sidecar (SPEC §7 *Digests*), and
# int_kernels_vectors::every_decoded_output_states_a_sha256 requires a sidecar,
# or a named exclusion, for every corpus directory. tests/generate.rs checks the
# wai.generate.spectral vectors, which `generate` enables: the 96x96 vector, sizes.json
# and instr-v0.json, the instruction object of form v0 (SPEC §5), whose two larger
# listed sizes run only in the release step (seconds each in a debug build).
# tests/cli_generate_score.rs runs `wai generate` and `wai score build|render`, which
# the features `generate` and `score` build into the `wai` binary: every pin it prints
# against those vectors and score-conformance/ (pcm_sha256 among them), every object
# instr-v0.json lists as accepted (one at each bound) made and wrapped, every object
# it lists as refused refused for the reason it states, and each refusal writing
# nothing; tests/generate.rs holds the same lists to a second statement of form v0.
# tests/manifest_parse_memory.rs (its own binary, a counting allocator) checks that
# reading a manifest allocates within the bound SPEC §2's reader limits give, at the
# length limit and at the value limit, and tests/manifest_reader_equivalence.rs holds
# the readers to the derived reader's result on every committed envelope. Both run in
# the release step only: the first writes manifests of 2^27 bytes, which takes a debug
# build most of a minute.
# tests/int_video.rs, int_video_halfpel.rs and int_video_keyframe.rs hold the integer
# video codec to its goldens (whole clips, half-pel motion, the keyframe through the
# integer hyperprior); int_world.rs and int_world2.rs the integer world renderers;
# jpegai_bitstream_calibration.rs the JPEG AI bitstream reader against the committed
# codestream. No step named any of them until scripts/check-tests-in-ci.py asked.
# jpegai_ops.rs holds the JPEG AI scale path (integer-operators §13.6.1) to its committed
# lowered graphs and certificate here; its cases on the jpegai/v1 bundle skip here and
# run in wai-jpegai-e2e.yml, which fetches the bundle and requires them.
# jpegai_nets_ops.rs does the same for its other integer networks (§13.6.2): the
# committed synthetic networks' lowered graphs and certificate, and every kernel
# policy on them, here; its jpegai/v1 cases there.
# int_video_ops.rs and wir1_ops.rs hold WIV1 and WIR1, which decode through the
# operator set's stage operators with no graph (integer-operators §13.5), to their
# stage forms, goldens and refusals; `i32_forms` builds E9's 32-bit form, which they
# and int_ops_differential.rs hold to the exact one. tests/int_ops_conformance.rs
# decodes every graph vector of ops-conformance/ through the fast kernels and holds
# each to the integers it states. Each of these runs every kernel set the leg's
# processor runs (int_kernels::equivalence_policies: the wide kernels, and NEON on
# arm64, AVX2 with or without VNNI on x86_64); int-kernels-amd below runs AVX-512.
# The tests of both steps are the files of tests/ whose line `// wai-conformance:` names
# int-kernels (and, in the release step only, int-kernels-release), and the features are
# their required-features (Cargo.toml) with `i32_forms`: scripts/wai-conformance-lists.py
# reads both and prints the command before it runs it, so a new test tags itself and edits
# no line here (its `dry-run` prints the lists).
- name: Fast integer kernels ≡ reference kernels (overflow checks on)
working-directory: wai/wai-rs
run: python3 ../../scripts/wai-conformance-lists.py test-tagged int-kernels --extra-features i32_forms -- --no-default-features --lib
- name: Fast integer kernels ≡ reference kernels (release, overflow checks off)
working-directory: wai/wai-rs
run: python3 ../../scripts/wai-conformance-lists.py test-tagged int-kernels int-kernels-release --extra-features i32_forms -- --release --no-default-features --lib
# The integer decode path builds under each of its features alone, so a sink can take
# one medium without the others, and for wasm32 (a browser sink); `int_payload` arms a
# medium only under its feature, and none needs `neural`. So does the staged-measurement
# core (`stage_measure`), which pulls no ONNX and reads no file on wasm32. The HTTP
# binding (`http`) performs no I/O, so it builds alone and for wasm32 too.
- name: Each integer feature alone, and wasm32 (unix)
if: matrix.os != 'windows'
shell: bash
working-directory: wai/wai-rs
run: |
set -euo pipefail
for f in neural_int video_int audio_int splat_int staged stage_measure telemetry http format_probe identify; do
cargo check --lib --no-default-features --features "$f"
done
# The decode bench prints its machine line from bench_env; no other step builds it.
cargo build --no-default-features --features neural_int,video_int,audio_int,world_int,splat_int --bin wai_decode_bench
rustup target add --toolchain 1.99.1 wasm32-unknown-unknown
cargo check --lib --target wasm32-unknown-unknown --no-default-features --features video_int,audio_int,splat_int,staged,http
# The staged-measurement core is wasm32-clean: a browser computes the same figures.
# Built, not only checked, so codegen and linking of the library run for wasm32.
cargo build --lib --target wasm32-unknown-unknown --no-default-features --features stage_measure
# With the claim classes too: a browser sink seals its reconstruct-session claim
# (jwp-receipts §2.4.12), and wai-web's measurement binding signs with them.
cargo build --lib --target wasm32-unknown-unknown --no-default-features --features receipt_claims,stage_measure
# So is delivery telemetry: a browser sink writes the same reports.
cargo build --lib --target wasm32-unknown-unknown --no-default-features --features telemetry
# And the header probes: a browser reads a payload's parameters the same way.
cargo build --lib --target wasm32-unknown-unknown --no-default-features --features format_probe
# And identification: a browser says what a dropped file is the same way.
cargo build --lib --target wasm32-unknown-unknown --no-default-features --features identify
# carriage: every registered capability round-trips byte-exact. cli_wrap_intent and
# cli_registry run the `wai` binary, which builds without the codecs feature: the
# intent rule of `wai wrap`, its refusal of candidate capabilities (SPEC §5), its media
# class, and `wai registry export --check` on the tree and on a drifted copy.
# tests/manifest_builders.rs: the manifest types are #[non_exhaustive], and the
# builders write the JSON a struct literal wrote; members the types keep only
# as `extra` (and `security`) survive a round trip.
# corpus_provenance: every corpus README states where its content comes from.
- name: Carriage conformance, the wai CLI (wrap, registry export) and corpus provenance
working-directory: wai/wai-rs
run: cargo test --no-default-features --test carriage --test manifest_builders --test cli_wrap_intent --test cli_registry --test corpus_provenance
# tests/floor.rs: the mandatory floor (SPEC 5: PNG, FLAC, zstd) in a build with the
# feature floor alone, no system codec library: decoded by the library and by `wai decode`,
# PNG and zstd written by `wai encode`, a codec outside the floor refused.
# tests/floor_conformance.rs: the decoder suite (wai/floor-conformance/): each floor case
# decoded to its stated digest, the video-floor case's framing, and the class rule (a
# sink is held to the video case exactly when it presents video); the codec-wrappers job
# decodes the video case through libdav1d. tests/cli_dx.rs
# (feature quickstart): `wai hello` and `wai verify` as a person runs them, a changed byte
# of the object or of the group's receipt refused; `wai create`, every template held to
# wai/examples/ file for file and byte for byte. The binary's own tests: `wai dev`'s origin
# (a service's prefix stripped, the relay's upgrade carried both ways, nothing served from
# outside its roots; WAI Studio at /_wai/, its API to this origin alone) and the project
# file's reader. tests/cli_studio.rs: `wai dev` against a stand-in relay and room service
# serves Studio at /_wai/, every socket the stack listens on is on loopback, and the
# local wai runs an operation for the page. tests/cli_doctor.rs: `wai doctor` on a made-up
# machine: a wai earlier on PATH, the relay's and the room service's versions, a port held,
# a key directory too open, and what reads as not checked.
- name: The floor build, and wai hello, create, dev and verify
working-directory: wai/wai-rs
run: |
cargo test --no-default-features --features floor --test floor --test floor_conformance &&
cargo test --no-default-features --features quickstart --bin wai --test floor --test cli_dx --test cli_fleet --test cli_studio --test cli_doctor
# tests/cli_http.rs runs `wai http publish`, `codecs`, `origin` and `bench` as a user
# does: objects named by their content hash with their head proofs, a sequence's union
# and floor, the origin's headers, ranges and If-Range on loopback, and a fetch by
# the byte-range procedure whose record states energy as not measured; and `hls`,
# `mpd`, `seal-playlist` and `check-playlist`, whose receipts need video_manifest;
# and `receipts`, an object's receipt index checked over HTTP with its signers
# confirmed under a key log, which needs key_discovery: the log anchored over TLS
# (cli_tls; the binary's own tests run its TLS client against servers under a test
# CA, tests/fixtures/tls/), never over plaintext, and a plaintext test anchor
# labelled as one. The second run builds without cli_tls: the CLI then refuses to
# anchor and connects to nothing.
# Cargo.toml declares it with required-features = ["http"]. tests/dash_read_bounds.rs
# holds Appendix I's reader to the document's size under a counting allocator, and an
# element's attributes to linear CPU time; a binary of its own, one test.
- name: The wai http commands
working-directory: wai/wai-rs
run: |
cargo test --no-default-features --features http,video_manifest,key_discovery,cli_tls --bin wai --test cli_http --test dash_read_bounds &&
cargo test --no-default-features --features http,video_manifest,key_discovery --test cli_http -- receipts_never_anchors_over_plaintext
# The wai commands for coding agents. tests/cli_mcp.rs starts `wai mcp` (the feature `mcp`)
# and drives it with an MCP client over its standard input and output: the handshake, the
# tool list (every tool an operation of the catalogue named by it, `core` by default and
# `dev` when asked, an unknown toolset refused) with each tool's schema and safety hints,
# and every tool. receipt_verify agrees vector by vector with jwp-conformance (group and
# object receipts), claim-conformance (each claim with the context the tool takes),
# live-conformance/rooms and live-conformance/audit, and a receipt changed by one character
# is refused; envelope_wrap writes only inside the project directory and never through a
# link; conformance_run runs live_rooms_verify.py; --read-only offers no tool that writes,
# serves or asks for a grant; the dev tools refuse any service off this machine before
# connecting, and any target but local (the room itself, joined, published to and verified
# through the tools, is joule-moq-room's tests/mcp_dev_room.rs). `--bin wai` runs the
# binary's unit tests: the tool table held to the operation catalogue both ways (every tool
# an operation the catalogue says wai mcp serves, in its toolset, every such operation a
# tool, a tool of MCP alone taking the catalogue's input), the loopback guard and
# setup-agent's merges and diff; `--lib catalogue::` the catalogue's rules (and
# wai-catalogue.yml the rest). tests/cli_setup_agent.rs runs `wai setup-agent` in scratch
# projects as an agent does (no terminal, --json, the JSON envelope out): a fresh project, a
# second run that changes nothing, a CLAUDE.md made only where one is, a file that exists
# changed only with --yes (exit 4 and the proposed
# changes otherwise), a dry run, --toolset registered, a JSON file with comments left
# alone, nothing written through a link out of the project, and the table's exit statuses.
# Cargo.toml declares cli_mcp with required-features = ["mcp"].
- name: The wai commands for coding agents (mcp, setup-agent)
working-directory: wai/wai-rs
run: |
cargo test --no-default-features --features mcp --bin wai --test cli_mcp --test cli_setup_agent &&
cargo test --no-default-features --lib catalogue::
# Apps made with Create, through `wai` built with the feature `app` (wai/wai-app linked
# in). tests/cli_app.rs: `wai create --from app.oui` writes an ordinary project, its
# wai.json laid out as a template's and naming the program and the room `wai dev` runs,
# its runtime under vendor/ and LICENSE; `wai app check` passes on it and refuses it with
# its room not its app's, another audience, no room named, a runtime file changed or a
# refused line, and a watch room until its file is there; `wai create --from` refuses a
# program with errors or no room, and a directory with anything in it; `wai mcp --toolset
# app`: the tools app_guide, app_check, app_stream and app_export, a program streamed
# through the bridge to a page with the pairing code and refused to one without, an export
# inside the project directory alone, and read-only with no bridge. `--bin wai` runs the
# binary's unit tests with the app toolset in its table, and the dev front's show mode.
- name: Apps made with Create (wai create --from, wai app, wai mcp --toolset app)
working-directory: wai/wai-rs
run: cargo test --no-default-features --features app --bin wai --test cli_app
# tests/staged_moq.rs runs the MoQ trace of staged-conformance/moq/ (moq-streaming-format
# §4.1) with signed receipts: a chain for each track, a receipt held for its parent and
# admitted when it arrives, the relay's per-hop delivery timeout, dropped objects checked
# by inclusion and named by the relay's delivery-gap claims, and each base's stage claim,
# which seals and verifies; the subscriber of two tracks holds exactly their chains.
# Cargo.toml declares it with required-features = ["staged", "video_int", "receipt_claims"].
# The interactive-world tests, which no step named: world_replay.rs (the replay
# class, under `world`), world_model.rs (the signed action trace and per-frame
# provenance, under `world_model`) and int_world_shared.rs (two sinks converging on
# one integer world, under both). `world_model` is enabled by no other step.
- name: World replay, the world model and the shared integer world
working-directory: wai/wai-rs
run: cargo test --no-default-features --features world_model,world_int --test world_replay --test world_model --test int_world_shared
- name: Staged tracks over MoQ, with signed receipts
working-directory: wai/wai-rs
run: cargo test --no-default-features --features staged,video_int,receipt_claims --test staged_moq
# The machine-readable registry (wai/registry/, wai/vocab/v2.*, the table in
# wai/REGISTRY.md) is generated from wai-rs/src/codecs/registry.rs. Regenerate it in
# memory and fail on any difference; the message says how to regenerate. One leg: the
# export is the same text on every platform.
- name: The registry export is up to date (linux)
if: matrix.os == 'linux'
working-directory: wai/wai-rs
run: cargo run -q --no-default-features --bin wai -- registry export --check
# The C ABI's envelope surface builds with no codec library (feature `ffi`):
# unpack returns each manifest as the envelope carries it, the WAI2 table carries
# every component's identity and selectability, wai_envelope_select follows
# SPEC §7.1 and never picks a candidate, and the packers refuse a candidate
# (SPEC §5). Cargo.toml declares the target with required-features.
- name: C ABI envelope surface (no codec library)
working-directory: wai/wai-rs
run: cargo test --no-default-features --features ffi --test ffi_envelope
# The browser crate wraps the same envelope types (wai::container), so role,
# companion_of, component identity and the pin reach JS, and its pick is SPEC
# §7.1's. Its tests target the pure-Rust half (`core`), which runs natively; one
# leg suffices, the logic is architecture-independent. The JS surface itself runs
# under wasm32 in wai-web-wasm.yml.
- name: wai-web envelope surface (native tests)
if: matrix.os == 'linux'
working-directory: wai/wai-web
run: cargo test --locked
# jwp-receipts §7.1, §7.8: no `did:web`/`did:key` literal in a tracked .rs, .json or
# .js file under wai/ names a real web origin — every key the tree signs with has a
# published seed. Ungated: it reads the tree through `git ls-files` and classifies
# with `signer_id`, which builds with no feature.
- name: Every signer id in the tree is a label
working-directory: wai/wai-rs
run: cargo test --no-default-features --test signer_ids_are_labels
# jwp-receipts §7: `wai-keys` builds a key log from operator-supplied seeds, appends
# revocations and rotations, signs statuses and module grants, verifies a log as
# §7.3 anchors one, and checks a DID document against it. The test drives the binary.
- name: wai-keys — build, check and publish an origin key log
working-directory: wai/wai-rs
run: cargo test --no-default-features --features key_discovery --test wai_keys_cli
# key_log has no I/O, clock or threads, so a browser sink could run it before it
# instantiates a module (§7.9); no browser sink does yet. Nor do the HTTP binding's
# receipts and keys (http-delivery §8.1–§8.5), which a browser client checks what it
# fetched with. This step checks that both build for wasm32 on the unix legs (the
# target is added to the pinned toolchain here; the Windows host runner is not set up
# for it).
- name: key_log and the HTTP receipts build for wasm32 (unix)
if: matrix.os != 'windows'
shell: bash
working-directory: wai/wai-rs
run: |
set -euo pipefail
rustup target add --toolchain 1.99.1 wasm32-unknown-unknown
cargo check --lib --target wasm32-unknown-unknown --no-default-features --features key_discovery,http
# Every cargo-class corpus, verified by its own tool (`<tool> verify <corpus>`), each in a
# log group; the first that fails stops the step. The corpora are the files of
# .github/wai-conformance/corpora/, one a corpus naming its tool and directory, so a new
# corpus adds a file and edits no line here. The features are those the tools require
# (Cargo.toml) with provenance and video_manifest. scripts/wai-conformance-lists.py runs
# them and prints each command; `corpora` lists them, and `dry-run` prints the features.
# One step for every leg: the script is the wrapper each shell family needed (on Windows
# it needs python3 on the PATH).
# Each tool runs on the main thread each platform gives a program, as a user's build of it
# does: 1 MiB of stack on Windows, 8 MiB on Linux. A tool that outgrows 1 MiB fails here.
# wai_pq_vectors did (STATUS_STACK_OVERFLOW, 2026-11-10): its hybrid keys held their
# expanded ML-DSA keys by value, and its debug build needed 3.4 MiB, most of it one frame
# of the corpus's key logs. They are on the heap now (wai-rs/src/pq_sig.rs, HybridKey).
- name: Every cargo-class corpus reproduces its frozen hashes
working-directory: wai/wai-rs
run: python3 ../../scripts/wai-conformance-lists.py corpora --verify --extra-features provenance,video_manifest
# Every committed audio vector must come from a seeded generator. This check reads
# each one to PCM (each audio base's stage 0 in the staged corpus, each measured
# source over one, each spatial-audio source, every audio file under wai/) and
# fails if any correlates above 0.5, at any lag, with a recording in the reference
# set the host has: its system sound directory, or the directories named in
# WAI_AUDIO_REFERENCE_DIRS. A generated signal stays near 0.2; a lossy coding of a
# recording is near 1. A host with no reference set, or no converter to read it,
# prints why and skips: the linux legs skip, the macOS leg checks.
- name: No committed audio vector is a coding of a reference recording (unix)
if: matrix.os != 'windows'
shell: bash
working-directory: wai
run: |
if command -v python3 >/dev/null 2>&1; then
python3 tools/wai_audio_provenance_check.py
else
echo "audio provenance: SKIPPED — no python3 on this host"
fi
# neural-conformance's trained vector codes the picture its tool generates: the tool
# regenerates that picture from the seed in the vector's meta.json, with the Python
# standard library alone, and requires the vector's reference decode, whose SHA-256
# is the corpus's digest for it, to reconstruct the picture at the PSNR meta.json
# records, and the vector's licence record to state the seed and streams meta.json
# records. One leg: the generator is integer arithmetic.
- name: The trained int_synth vector codes the picture its tool generates (macOS)
if: matrix.os == 'macos'
shell: bash
working-directory: wai
run: python3 -I tools/wai_int_synth_real.py --check neural-conformance/vectors/photo
# The same check on a copy of the corpus whose licence record gives the training
# pictures as streams 1 to 64 instead of meta.json's 1 to 512: it must fail.
- name: The trained int_synth vector's check fails on a licence record that misstates its training pictures (macOS)
if: matrix.os == 'macos'
shell: bash
working-directory: wai
run: |
set -uo pipefail
t=$(mktemp -d)
trap 'rm -rf "$t"' EXIT
mkdir -p "$t/vectors"
cp -R neural-conformance/vectors/photo "$t/vectors/"
cp neural-conformance/sha256.json neural-conformance/licences.json "$t/"
python3 -I - "$t/licences.json" <<'P'
import json, sys
d = json.load(open(sys.argv[1]))
tr = d["vectors"]["photo"]["training"]
assert tr["input"].endswith("streams 1 to 512"), tr["input"]
tr["input"] = tr["input"][: -len("512")] + "64"
json.dump(d, open(sys.argv[1], "w"))
P
s=0
python3 -I tools/wai_int_synth_real.py --check "$t/vectors/photo" > "$t/out" 2>&1 || s=$?
cat "$t/out"
test "$s" = 1
grep -qF "streams 1 to 64'; meta.json says" "$t/out"
# OER's second implementation is portable C, not Rust. Rebuild it from source and
# require its output to match the pinned corpus byte for byte: the kernel is
# deterministic, and wai_oer_vectors above has already verified the pinned copy,
# so equality carries that verification to what the source produces today.
# The harness then requires the kernel to refuse a grade its meter cannot support
# (oer-receipts §2: a model's figure graded measured, a grade outside the four) and
# exits 4 if it does not (settle/README.md).
# `cc` with the Makefile's flags, so the step needs no `make` on the runner.
- name: OER C kernel reproduces its pinned receipts (unix)
if: matrix.os != 'windows'
shell: bash
working-directory: wai/oer-conformance/settle
run: |
set -euo pipefail
cc -O2 -std=c99 -Wall -Wextra -Wno-unused-parameter -Wno-sign-compare \
-o "$RUNNER_TEMP/settle_host" host_main.c settle.c sha256.c tweetnacl.c -lm
"$RUNNER_TEMP/settle_host" "$RUNNER_TEMP/firmware_vectors.json" 400
cmp "$RUNNER_TEMP/firmware_vectors.json" ../firmware_vectors.json
# wai-quantum and the quantum corpus under wasm32-wasip2, run by wasmtime. On wasm32
# usize is 32 bits, so arithmetic that cannot overflow on a 64-bit host can there: a
# section's offset plus its length, a capacity reserved from a count a file claims, a
# shift by a qubit count. The builds are debug builds, so overflow checks are on and
# such an overflow panics instead of wrapping. Three steps:
# 1. tests/bit_identity.rs, which pins one result of each f64 layer, so the wasm32
# build must give the bits every native leg gives;
# 2. the library's unit tests in the modules that hold the circuit core, its
# container, its JSON reader and its pins (quantum, ijson, receipt_pins,
# quantum_fabric, quantum_source, quantum_channel, quantum_qudit, quantum_receipt),
# each required to have run, and each test of the fabric's groundwork for
# accelerator members required by name (the WebGPU member itself has no wasm32-
# wasip2 build; wai-quantum-webgpu.yml checks that the feature is inert there);
# 3. wai_quantum_vectors verify over all four trees of quantum-conformance/ (both
# capabilities, the reject corpus and the accept corpus), the same check the matrix
# above runs natively.
# One leg: wasm32 is the same machine on every host. Linux, so it does not queue on the
# single macOS runner.
# The fast integer kernels on the AVX-512 host (labelled linux-amd), whose processor runs
# every x86_64 kernel set (wide, AVX2, AVX2 with VNNI, AVX-512, AVX-512 with VNNI), none
# of whose AVX-512 kernels the other legs' processors run. The same tests as the kernel
# steps of cargo-classes, in both profiles; the runner declares its sets, and a set the
# tests did not find fails the job instead of passing as a skip.
int-kernels-amd:
name: fast integer kernels, every x86_64 kernel set (AVX-512 silicon)
needs: relevant
if: needs.relevant.outputs.run == 'true'
runs-on: linux-amd
timeout-minutes: 90
env:
CARGO_PROFILE_DEV_DEBUG: '0'
CARGO_BUILD_JOBS: '12'
steps:
- name: Fetch the source
shell: bash
run: |
set -euo pipefail
want="${{ github.event.pull_request.head.sha || github.sha }}"
rm -rf src && mkdir src && cd src
git init -q
# Fetch the commit itself, not the ref. The host names its own route to the
# forge (WAI_FORGE_URL).
git fetch -q --depth 1 "${WAI_FORGE_URL:-http://100.64.224.0:3400}/Transaction-Science/open-standards.git" "$want"
git checkout -q FETCH_HEAD
got="$(git rev-parse HEAD)"
[ "$got" = "$want" ] || { echo "::error::fetched $got, not $want"; exit 1; }
- name: Fast integer kernels ≡ reference kernels, every kernel set, both profiles
shell: bash
working-directory: src/wai/wai-rs
run: |
set -euo pipefail
export PATH="$HOME/.cargo/bin:$PATH"
export CARGO_TARGET_DIR="$HOME/hw-target/wai-rs-int-kernels"
# The sets this host's processor runs, declared here (or by the runner); the
# library test the_kernel_sets_found_are_the_declared_ones fails on any other.
export WAI_INT_ISAS_EXPECT="${WAI_INT_AMD_ISAS_EXPECT:-wide,avx2,avx2-vnni,avx512,avx512-vnni}"
# The features of cargo-classes' kernel steps, and the test files tagged
# `// wai-conformance: int-kernels-amd` (scripts/wai-conformance-lists.py).
F=$(python3 ../../scripts/wai-conformance-lists.py features int-kernels --extra-features i32_forms)
T=$(python3 ../../scripts/wai-conformance-lists.py tests int-kernels-amd)
echo "features $F"
echo "tests $T"
for p in "" --release; do
nice -n 10 cargo test --locked $p --no-default-features --features $F --lib $T
done
quantum-wasip2:
needs: relevant
if: needs.relevant.outputs.run == 'true'
name: quantum core and corpus under wasm32-wasip2 (wasmtime)
runs-on: linux
# The fabric's planner test runs fabric-v1 under every planner configuration in a
# debug wasm build, several minutes on its own; the timeout leaves it room rather
# than the test being skipped on wasm32.
timeout-minutes: 90
steps:
- uses: actions/checkout@v5
- name: Install Rust 1.99.1 with the wasm32-wasip2 target
shell: bash
run: |
export PATH="$HOME/.cargo/bin:$PATH"
command -v rustup >/dev/null 2>&1 || \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.99.1
rustup toolchain install 1.99.1 --profile minimal
rustup default 1.99.1
rustup target add --toolchain 1.99.1 wasm32-wasip2
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
# The release binary, pinned by version and SHA-256, rather than a build from source.
# Every WAI workflow that runs wasmtime pins this one version, and every run passes
# -W nan-canonicalization=y -W relaxed-simd-deterministic=y: every NaN a float
# instruction makes is the canonical one, and each relaxed SIMD instruction has one
# behaviour on every host. A stock wasmtime leaves both off. The fixed relaxed
# behaviour is the engine's own: on 50.0.1 relaxed_madd is fused (measured on arm64 and
# x86_64), where WebAssembly's deterministic profile prescribes the unfused result.
# The repository checks (scripts/check-wasmtime-pin.py) fail a workflow that pins another
# release or runs wasmtime without both settings.
- name: Install wasmtime 50.0.1
shell: bash
run: |
set -euo pipefail
[ "$(uname -m)" = x86_64 ] || { echo "::error::the pinned wasmtime archive is x86_64; this host is $(uname -m)"; exit 1; }
if ! command -v xz >/dev/null 2>&1; then
SUDO=""; [ "$(id -u)" -ne 0 ] && SUDO="sudo"
$SUDO apt-get update -qq
$SUDO apt-get install -y -qq xz-utils
fi
v=50.0.1
f="wasmtime-v$v-x86_64-linux.tar.xz"
curl --proto '=https' --tlsv1.2 -sSfL -o "$RUNNER_TEMP/$f" \
"https://github.com/bytecodealliance/wasmtime/releases/download/v$v/$f"
echo "685466455dd42a14774a86c36c8f2e45cbb9bd6d92a850139315df60a75498a5 $RUNNER_TEMP/$f" | sha256sum -c -
tar -xJf "$RUNNER_TEMP/$f" -C "$RUNNER_TEMP"
echo "$RUNNER_TEMP/wasmtime-v$v-x86_64-linux" >> "$GITHUB_PATH"
- name: tests/bit_identity.rs under wasmtime
working-directory: wai/wai-quantum
env:
CARGO_TARGET_WASM32_WASIP2_RUNNER: wasmtime run -W nan-canonicalization=y -W relaxed-simd-deterministic=y
run: cargo test --locked --target wasm32-wasip2 --features full --test bit_identity
# The filters are module paths, so a module that is renamed or loses its tests would
# leave its filter matching nothing and the run passing; the loop fails the step
# unless every module had a test that passed.
- name: The circuit core's unit tests under wasmtime
shell: bash
working-directory: wai/wai-quantum
env:
CARGO_TARGET_WASM32_WASIP2_RUNNER: wasmtime run -W nan-canonicalization=y -W relaxed-simd-deterministic=y
run: |
set -euo pipefail
mods="quantum ijson receipt_pins quantum_fabric quantum_source quantum_channel quantum_qudit quantum_receipt engine::synth engine::compile engine::estimate"
filters=()
for m in $mods; do filters+=("$m::"); done
cargo test --locked --target wasm32-wasip2 --features full --lib -- "${filters[@]}" 2>&1 \
| tee "$RUNNER_TEMP/wasip2-lib.log"
for m in $mods; do
grep -q "^test $m::.* \.\.\. ok$" "$RUNNER_TEMP/wasip2-lib.log" \
|| { echo "::error::no test of $m ran"; exit 1; }
done
# The fabric's groundwork for accelerator members (certification one case at
# a time, the fabric-v1 corpus and its pins, the 32-bit-lane words and their
# range guard, run meters, metered calibration, placement for a run and the
# fabric's fallback, the program a member runs in every planner configuration,
# the WGSL kernels' host layouts, member ids and mutants), each by name: a module
# filter is satisfied by any one test of the module, so a test dropped or
# compiled out on wasm32 would otherwise go unnoticed.
for t in quantum_fabric::tests::certification_is_certify \
quantum_fabric::tests::fxmul_wide_is_the_exact_product \
quantum_fabric::tests::apply_words_is_apply \
quantum_fabric::tests::malformed_words_are_refused \
quantum_fabric::tests::z_on_the_most_negative_lane_is_flagged \
quantum_fabric::tests::simulate_checked_is_simulate \
quantum_fabric::tests::a_rounded_product_is_never_negated \
quantum_fabric::tests::the_guard_horizon_is_pinned \
quantum_fabric::tests::valid_circuits_stay_in_lanes \
quantum_fabric::tests::fabric_corpus_starts_with_random_v1 \
quantum_fabric::tests::the_fabric_v1_corpus_of_each_gateset_is_pinned \
quantum_fabric::tests::fabric_v1_covers_every_dispatch_shape \
quantum_fabric::tests::gate_words_are_the_table \
quantum_fabric::tests::a_meter_reading_is_labelled_or_dropped \
quantum_fabric::tests::a_placement_figure_is_a_measurement_or_nothing \
quantum_fabric::tests::figures_go_on_both_members_or_neither \
quantum_fabric::tests::place_for_sends_small_runs_to_the_cpu_and_large_ones_to_the_gpu \
quantum_fabric::tests::the_fabric_falls_back_and_names_the_member_that_produced_the_result \
quantum_fabric::tests::a_metered_fabric_run_seals_its_reading \
quantum_fabric::tests::a_calibration_gives_the_cpu_reference_its_figure \
quantum_fabric::plan::tests::execute_is_the_native_simulator \
quantum_fabric::plan::tests::every_configuration_is_the_native_simulator \
quantum_fabric::plan::tests::long_circuits_tile_as_they_run_gate_by_gate \
quantum_fabric::plan::tests::tiles_are_exact_on_any_words \
quantum_fabric::plan::tests::tiles_cover_every_path \
quantum_fabric::plan::tests::the_v1_tail_phases_are_not_skipped \
quantum_fabric::plan::tests::malformed_programs_are_refused \
quantum_fabric::wgsl::tests::implementation_id_is_pinned \
quantum_fabric::wgsl::tests::gate_record_layout \
quantum_fabric::wgsl::tests::grids_cover_every_item_once \
quantum_fabric::wgsl::tests::the_index_map_is_a_bijection \
quantum_fabric::wgsl::tests::member_ids_are_pinned \
quantum_fabric::wgsl::tests::the_cpu_reference_is_placed_first_when_nothing_is_metered \
quantum_fabric::wgsl::tests::mutants_apply_once \
quantum_fabric::wgsl::tests::mutant_witnesses_exist \
quantum_fabric::wgsl::tests::placement_breaks_an_implementation_tie_by_name \
quantum_fabric::wgsl::tests::tile_implementation_id_is_pinned \
quantum_fabric::wgsl::tests::tile_records_layout \
quantum_fabric::wgsl::tests::tile_grids_cover_every_tile_once \
quantum_fabric::wgsl::tests::tile_mutants_apply_once \
quantum_fabric::wgsl::tests::tile_mutant_witnesses_exist; do
grep -q "^test $t \.\.\. ok$" "$RUNNER_TEMP/wasip2-lib.log" \
|| { echo "::error::$t did not run"; exit 1; }
done
# The wai package is built as one codegen unit here. Its lib target is also a cdylib,
# and the wasm linker in Rust 1.98 (LLD 22) can crash with SIGSEGV linking that cdylib:
# the linker keeps the flags of the first object that names the function table, so
# when that object names it without NO_STRIP the table is no garbage-collection root.
# Nothing the cdylib keeps calls through the table, but it does keep function
# addresses, and writing their element segment then dereferences a table that was
# never created. Which object comes first follows the hashed names of the codegen
# units, so an unrelated change to the crate, or a different checkout path, can bring
# the crash on or off. As one unit, wai's object comes first and takes function
# addresses, so it names the table with NO_STRIP. LLD 23 keeps NO_STRIP from any
# object. The units change how the crate is split for code generation, not what the
# program computes.
- name: wai_quantum_vectors verify under wasmtime
shell: bash
working-directory: wai/wai-rs
run: |
set -euo pipefail
cargo build --locked --target wasm32-wasip2 --no-default-features --features quantum --bin wai_quantum_vectors \
--config 'profile.dev.package.wai.codegen-units=1'
wasmtime run -W nan-canonicalization=y -W relaxed-simd-deterministic=y --dir ../quantum-conformance::/corpus \
target/wasm32-wasip2/debug/wai_quantum_vectors.wasm verify /corpus
# The face generator under wasm32-wasip2, run by wasmtime (extensions/avatars.md §7.8):
# wai_avatar_face_vectors verify, built for wasm32, regenerates every file of
# live-conformance/avatar-face from its seeds and inputs, every frame through the integer
# operator set's interpreter compiled to wasm32, and requires the committed bytes, which a
# native build wrote: the frames are the same on wasm32 as on the CPU. One leg: wasm32 is
# the same machine on every host. Linux, so it does not queue on the single macOS runner.
avatar-face-wasip2:
needs: relevant
if: needs.relevant.outputs.run == 'true'
name: faces from keypoints under wasm32-wasip2 (wasmtime)
runs-on: linux
timeout-minutes: 60
steps:
- uses: actions/checkout@v5
- name: Install Rust 1.99.1 with the wasm32-wasip2 target
shell: bash
run: |
export PATH="$HOME/.cargo/bin:$PATH"
command -v rustup >/dev/null 2>&1 || \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.99.1
rustup toolchain install 1.99.1 --profile minimal
rustup default 1.99.1
rustup target add --toolchain 1.99.1 wasm32-wasip2
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
# The release binary, pinned by version and SHA-256, rather than a build from source.
# Every WAI workflow that runs wasmtime pins this one version, and every run passes
# -W nan-canonicalization=y -W relaxed-simd-deterministic=y: every NaN a float
# instruction makes is the canonical one, and each relaxed SIMD instruction has one
# behaviour on every host. A stock wasmtime leaves both off. The fixed relaxed
# behaviour is the engine's own: on 50.0.1 relaxed_madd is fused (measured on arm64 and
# x86_64), where WebAssembly's deterministic profile prescribes the unfused result.
# The repository checks (scripts/check-wasmtime-pin.py) fail a workflow that pins another
# release or runs wasmtime without both settings.
- name: Install wasmtime 50.0.1
shell: bash
run: |
set -euo pipefail
[ "$(uname -m)" = x86_64 ] || { echo "::error::the pinned wasmtime archive is x86_64; this host is $(uname -m)"; exit 1; }
if ! command -v xz >/dev/null 2>&1; then
SUDO=""; [ "$(id -u)" -ne 0 ] && SUDO="sudo"
$SUDO apt-get update -qq
$SUDO apt-get install -y -qq xz-utils
fi
v=50.0.1
f="wasmtime-v$v-x86_64-linux.tar.xz"
curl --proto '=https' --tlsv1.2 -sSfL -o "$RUNNER_TEMP/$f" \
"https://github.com/bytecodealliance/wasmtime/releases/download/v$v/$f"
echo "685466455dd42a14774a86c36c8f2e45cbb9bd6d92a850139315df60a75498a5 $RUNNER_TEMP/$f" | sha256sum -c -
tar -xJf "$RUNNER_TEMP/$f" -C "$RUNNER_TEMP"
echo "$RUNNER_TEMP/wasmtime-v$v-x86_64-linux" >> "$GITHUB_PATH"
# One codegen unit for the wai package: see the quantum job's last step for the LLD 22
# cdylib crash this avoids.
- name: wai_avatar_face_vectors verify under wasmtime
shell: bash
working-directory: wai/wai-rs
run: |
set -euo pipefail
cargo build --locked --release --target wasm32-wasip2 --no-default-features --features avatar_face --bin wai_avatar_face_vectors \
--config 'profile.release.package.wai.codegen-units=1'
t="${CARGO_TARGET_DIR:-target}"
wasmtime run -W nan-canonicalization=y -W relaxed-simd-deterministic=y --dir ../live-conformance/avatar-face::/corpus \
"$t/wasm32-wasip2/release/wai_avatar_face_vectors.wasm" verify /corpus | tee "$RUNNER_TEMP/face-wasm.log"
[ "$(grep -c '^PASS ' "$RUNNER_TEMP/face-wasm.log")" = 3 ] || { echo "::error::not every check passed under wasm32"; exit 1; }
# The generated v2 vocabulary (wai/vocab/v2.*) is checked as a consumer uses it:
# sample manifests, typed and untyped, WAI1 and WAI2, are expanded through the
# JSON-LD context exactly as written on the wire and validated against the SHACL
# shapes, each with the result it must give (a candidate in any place is
# rejected; a media value no revision lists is accepted). One leg: RDF tooling,
# not arithmetic. The tool versions are pinned.
vocab-shapes:
needs: relevant
if: needs.relevant.outputs.run == 'true'
name: v2 vocabulary · manifests validate as stated
runs-on: linux-arm64
steps:
- uses: actions/checkout@v5
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install the RDF tools
shell: bash
run: python -m pip install rdflib==7.6.0 PyLD==3.3.0 pyshacl==0.40.1
- name: Sample manifests against vocab/v2.shapes.ttl
shell: bash
run: python wai/vocab/check_shapes.py
# The corpora above are generated and verified by wai-rs, one implementation. These are
# the second implementations, written from the extension text in Python and sharing no
# code with wai-rs: claim_verify.py (every claim vector, the canonical fixture, and the
# bracket arithmetic of energy-measurement §6.2 recomputed with Python's integers),
# differential.py (both verifiers' verdicts on generated inputs the corpus does not hold,
# under a fixed seed, so a run is reproducible), jwp_verify.py (every JWP vector), and
# signer_verify.py (every key-discovery vector and the canonical fixture, with its
# verdicts required to equal wai_signer_vectors' line for line), and
# int_payload_verify.py (the integer payloads and their refusal fixtures), and
# staged_verify.py (the staged-delivery corpus), and http_verify.py (the HTTP delivery
# corpus), and quantum-conformance/tools/referee.py (both quantum-sim capabilities), and
# one verifier for each corpus under live-conformance/, which live-conformance/README.md
# lists beside its directory, extension and generator.
# Each exits non-zero on any failure or any difference, and the step fails with it.
# One platform: what they check is the reading of the text, not the architecture.
#
# Each step is a file of .github/wai-conformance/verifiers/: its shell body, run in wai/
# with bash -eo pipefail as a workflow step runs, and a header that names its features,
# the binaries it runs, the paths it reads and its place in the order. Every list this job
# used to keep by hand is derived from those files when it runs (scripts/wai-conformance-
# lists.py), so a new verifier adds one file and edits none of this job's lists (a step
# whose `# paths:` falls under none of the trigger's paths still needs a trigger line,
# which the script's `check` names). The
# script's `dry-run` prints every list and step this job runs; `run --only <file name>`
# runs one step.
independent-verifiers:
needs: relevant
if: needs.relevant.outputs.run == 'true'
name: independent verifiers · claims, JWP, key discovery and integer payloads (Python)
runs-on: linux-arm64
env:
# No debuginfo, for the reason given at cargo-classes: this job ran out of the VM's disk
# mid-link beside two others (#550).
CARGO_PROFILE_DEV_DEBUG: '0'
steps:
- uses: actions/checkout@v5
- name: Install Rust 1.99.1
shell: bash
run: |
export PATH="$HOME/.cargo/bin:$PATH"
command -v rustup >/dev/null 2>&1 || \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.99.1
rustup toolchain install 1.99.1 --profile minimal
rustup default 1.99.1
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
- uses: actions/setup-python@v5
with:
python-version: '3.11'
# The ad-presentation step runs the browser client's reference producer under Node, whose
# WebCrypto signs with Ed25519.
- uses: actions/setup-node@v4
with:
node-version: '22'
# The verifiers check Ed25519's equation with the `cryptography` package and make the
# rest of jwp-receipts §3.1 step 3 themselves.
- name: Install the Ed25519 library
shell: bash
run: python -m pip install cryptography
# Every wai-rs step below runs with one of three feature lists, so cargo builds each target
# once and every later `cargo test` finds it fresh. A step selects its tests exactly as
# before (`--lib <module>::` or `--test <name>`, with its filters); only the feature list
# is shared. Each feature still compiles alone in the `features-alone` job. The lists are
# the steps' files' (the first step below prints them):
#
# WAI_FEATURES the union of every step's `features:` but the hybrid signatures'.
# WAI_PQ_FEATURES the steps of `build: pq`, the hybrid signatures. `pq_signatures`
# changes what a build without it refuses (pq-signatures §6.3: such
# a build reads a hybrid member as an unknown one), so it stays out
# of the union and every other step keeps testing a build without it.
# WAI_RELEASE_FEATURES the steps' `release-features:`, for the tests they run optimised.
#
# Builds of the library that differ in features, or in the dependencies' features,
# write the same file, so each switch between two of them rebuilds it. `cargo build`
# and `cargo test` are two such builds (the dev-dependencies turn on wai-ops' `oracle`
# and serde_json's `preserve_order`), and so are two feature lists. Hence the order of
# the build step's script: the corpus tools the steps run as programs first (every
# `wai-bins/union/<tool>` and `wai-bins/pq/<tool>` a step names, and its `bins:` and
# `pq-bins:`), as `cargo build` builds them, copied to $RUNNER_TEMP/wai-bins/union and
# $RUNNER_TEMP/wai-bins/pq (the steps run those copies); then the union's test build,
# last, of every `--test` a step runs with "$WAI_FEATURES", which every step but the
# hybrid signatures' runs; the release build, of every `--test` run with
# "$WAI_RELEASE_FEATURES", has a directory of its own. The hybrid signatures step builds
# its tests itself, the one other build of the test profile.
#
# The claims step's differentials drive wai_claim_vectors' and wai_jwp_vectors'
# `verdicts` modes, and compare wai_signer_vectors' key-discovery verdicts with
# signer_verify.py's. Both builds of the three are copied and both are held to the
# Python verifiers: the union's, which has key_discovery and no pq_signatures, and the
# hybrid signatures', which reads the hybrid members the other refuses.
- name: The steps' feature lists, from their files
shell: bash
run: python3 scripts/wai-conformance-lists.py env | tee -a "$GITHUB_ENV"
- name: Build wai-rs once for every step below, and the corpus verifiers it ships
working-directory: wai/wai-rs
shell: bash
run: |
set -euo pipefail
python3 ../../scripts/wai-conformance-lists.py prebuild > "$RUNNER_TEMP/wai-prebuild.sh"
cat "$RUNNER_TEMP/wai-prebuild.sh"
bash --noprofile --norc -eo pipefail "$RUNNER_TEMP/wai-prebuild.sh"
# The corpus union, one command: every corpus's independent verifier, its second
# implementation, written from the extension's text and sharing no code with wai-rs, run
# by `wai conform --independent` in wai/ with the command line its README gives. The list
# is the table `wai conform` holds (`wai conform --list` prints it), which build.rs reads
# from the `# independent:` lines of the step files below, so a corpus's verifier joins it
# in its own step's file; the table's tests require every *_verify.py of the tree to be in
# it or to say where it runs instead, and no step to run one of them by hand. What each
# verifier checks is in its corpus's README and its own header; those of the claim, JWP,
# key-discovery, HTTP, ledger, route and post-quantum corpora, and the ones that read
# them, check Ed25519's equation with the `cryptography` package installed above (without
# it their cases fail, never skip). Each run is a signed result record, which `wai verify`
# checks; the JUnit report goes with the records, and a failed verifier's output is in the
# log. `wai` is built with the feature quickstart into a target directory of its own, so
# the union's builds above stay as they are. The steps below are each corpus's Rust side.
# `set -e` decides the step: the union's exit status is the verdict. Every change runs
# scripts/wai-conformance-lists.py `check`, which reads this step's text and holds it to
# that: no `if:` or `continue-on-error:`, `set -e` never turned off, the `wai` it built
# and no other, no flag that narrows the union, stops it before it runs or changes what
# runs a verifier (--suite, --dry-run, --list, --timeout, --python), nothing after it on
# its line, no `||`, condition, trap or exit, `wai verify` after it, and both lines at the
# step's top level, in no function, block, quoted string or here-document.
- name: Every corpus's independent verifier, through wai conform --independent
working-directory: wai
shell: bash
run: |
set -euo pipefail
(cd wai-rs && cargo build --locked --no-default-features --features quickstart --bin wai --target-dir "$RUNNER_TEMP/wai-cli-target")
w="$RUNNER_TEMP/wai-cli-target/debug/wai"
d="$RUNNER_TEMP/independent"
mkdir -p "$d"
"$w" conform --independent --root . --out "$d/records" --format junit > "$d/junit.xml"
"$w" verify "$d/records"
# An upload this forge cannot take must not decide the job: the verdict is the step above.
- name: Upload the independent verifiers' records and report
if: always()
continue-on-error: true
uses: actions/upload-artifact@v3
with:
name: independent-verifiers
path: ${{ runner.temp }}/independent
# `needs=` on an `# independent:` line is what a failed verifier is told it lacks. Each night
# the union runs again on an interpreter with no packages: an entry that declares one must
# fail there, and one that declares none must pass. A second run of the union on every
# change would double this job, so it stays on the schedule.
- name: Each verifier's needs= is what a run without packages shows
if: github.event_name == 'schedule'
shell: bash
run: python3 scripts/wai-conformance-lists.py needs "$RUNNER_TEMP/wai-cli-target/debug/wai"
# Every step, in its order, each in a group of the log; the first that fails stops the
# job, as a failed step did, and is named in an error line with its file. A file that
# declares verifiers and runs nothing itself (no body) is no step: the union ran them.
- name: Every verifier step (.github/wai-conformance/verifiers/)
shell: bash
run: python3 scripts/wai-conformance-lists.py run
# Each feature list a step of the independent verifiers built alone, before those steps
# shared one build of the union, still compiles alone: a feature that compiles only beside
# another passes there and would fail a consumer that asks for it alone. A check, with no
# code generation, of every target the list enables (`--tests`: the library, as itself and
# as its unit tests see it, its binaries and its integration tests), for each list in turn;
# the tests themselves run in that job, under the union.
features-alone:
needs: relevant
if: needs.relevant.outputs.run == 'true'
name: each feature of the independent verifiers' steps compiles alone
# A check, so any architecture serves; linux-arm64 has the free slots.
runs-on: linux-arm64
steps:
- uses: actions/checkout@v5
- name: Install Rust 1.99.1
shell: bash
run: |
export PATH="$HOME/.cargo/bin:$PATH"
command -v rustup >/dev/null 2>&1 || \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.99.1
rustup toolchain install 1.99.1 --profile minimal
rustup default 1.99.1
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
command -v python3 >/dev/null 2>&1 || { SUDO=""; [ "$(id -u)" -ne 0 ] && SUDO="sudo"; $SUDO apt-get update -qq; $SUDO apt-get install -y -qq python3; }
- name: wai-rs, each feature list alone
working-directory: wai/wai-rs
shell: bash
run: |
set -euo pipefail
# The verifier steps' feature lists (`features:`, or `alone:` where a file gives one).
lists=$(python3 ../../scripts/wai-conformance-lists.py alone)
[ -n "$lists" ] || { echo "::error::no feature list was derived"; exit 1; }
for f in $lists; do
echo "::group::$f"
cargo check --no-default-features --features "$f" --tests
echo "::endgroup::"
done
# The browser sink step runs wai-live-web's tests with `int_video` only: they are a
# superset of the default build's, and the feature gates nothing off.
- name: wai-live-web without int_video
working-directory: wai/wai-live-web
run: cargo check --locked --tests
# Delivery telemetry's second implementation is an independent open-source CMCD and CMSD
# library at locked versions (telemetry-conformance/cml/package-lock.json). The
# differential builds each observation's report from the extension's tables, encodes it
# with the library, requires the bytes of expected/cmcd.json and expected/cmsd.json, reads
# the expected strings back as dictionary, query and header shards, and validates every
# report with no error. One platform: string encoding, not arithmetic.
telemetry-differential:
needs: relevant
if: needs.relevant.outputs.run == 'true'
name: delivery telemetry · CMCD and CMSD against an independent library
runs-on: linux-arm64
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v4
with:
node-version: '22'
- name: npm ci, then the differential
working-directory: wai/telemetry-conformance/cml
shell: bash
run: |
set -euo pipefail
npm ci --no-audit --no-fund
node differential.mjs ..
byte-exact:
needs: relevant
if: needs.relevant.outputs.run == 'true'
name: cross-arch byte-equality · ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
# Self-hosted fleet labels. Both architectures: x86_64 by linux, arm64
# by linux-arm64 + macos — what matters for the Fx integer floor is the
# arch, not the OS. No Intel-Mac runner exists in the fleet, and it
# would add no arch coverage anyway.
# Windows x86_64 runs this suite in wai-byte-equality.yml, whose
# windows-x86_64 leg has pwsh forms of the disk steps; this job keeps
# the unix legs, whose disk steps below are bash.
os: [linux, linux-arm64, macos]
env:
# The floor of the disk check below, in GiB, as in wai-byte-equality.yml, whose
# comment gives the leg's writes and the readings behind it.
DISK_FLOOR_GIB: '4'
steps:
# The two disk steps of wai-byte-equality.yml's legs, on the same three labels:
# a full runner disk fails here as infrastructure, before the checkout, and a
# failure later in the leg is named infrastructure when the disk filled meanwhile.
# wai-byte-equality.yml's comment says what they measure and why.
- name: "infrastructure: this runner's disk has room for the leg"
id: disk
if: matrix.os != 'windows'
shell: bash
run: |
set -euo pipefail
short=0; seen=""
for d in "${GITHUB_WORKSPACE:-$PWD}" "${RUNNER_TEMP:-/tmp}" "${TMPDIR:-/tmp}" "$HOME"; do
p=$d; while [ ! -e "$p" ]; do p=$(dirname "$p"); done
set -- $(df -Pk "$p" | awk 'NR == 2 {print $4, $6}')
echo "$d: $(($1 / 1048576)) GiB free on $2"
if [ "$1" -lt $((DISK_FLOOR_GIB * 1048576)) ]; then
echo "::error::infrastructure: ${{ matrix.os }}: $d is on $2, which has $(($1 / 1024)) MiB free, below the floor of $DISK_FLOOR_GIB GiB. The runner host's disk is full: no fixture was decoded, so this is not a byte-equality failure."
short=1; continue
fi
case " $seen " in *" $2 "*) continue ;; esac
seen="$seen $2"; f="$p/.wai-disk-probe.$$"
if ! err=$(dd if=/dev/zero of="$f" bs=1048576 count=32 conv=fsync 2>&1); then
case "$err" in
*"No space left"*|*"Input/output error"*|*"Read-only file system"*|*"quota"*)
echo "::error::infrastructure: ${{ matrix.os }}: a 32 MiB write under $p, on $2, failed: $(printf '%s' "$err" | head -n 1). The runner host's disk cannot take the leg's writes: no fixture was decoded, so this is not a byte-equality failure."
short=1 ;;
*) echo "::warning::a 32 MiB write under $p failed for another reason: $(printf '%s' "$err" | head -n 1)" ;;
esac
fi
rm -f "$f"
done
exit $short
- uses: actions/checkout@v5
- name: Install Rust 1.99.1 (unix)
if: matrix.os != 'windows'
shell: bash
run: |
export PATH="$HOME/.cargo/bin:$PATH"
command -v rustup >/dev/null 2>&1 || \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.99.1
rustup toolchain install 1.99.1 --profile minimal
rustup default 1.99.1
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
- name: Install Rust 1.99.1 (Windows)
if: matrix.os == 'windows'
shell: pwsh
run: |
rustup toolchain install 1.99.1 --profile minimal
rustup default 1.99.1
- name: Integer rANS + int_synth + JPEG-AI mANS goldens (one machine → every machine)
working-directory: wai/byte-exact-conformance
run: cargo test --release
- name: "infrastructure: was the disk full when the leg failed?"
if: failure() && steps.disk.outcome == 'success'
shell: bash
run: |
set -euo pipefail
short=0; seen=""
for d in "${GITHUB_WORKSPACE:-$PWD}" "${RUNNER_TEMP:-/tmp}" "${TMPDIR:-/tmp}" "$HOME"; do
p=$d; while [ ! -e "$p" ]; do p=$(dirname "$p"); done
set -- $(df -Pk "$p" | awk 'NR == 2 {print $4, $6}')
echo "$d: $(($1 / 1048576)) GiB free on $2"
if [ "$1" -lt $((DISK_FLOOR_GIB * 1048576)) ]; then
echo "::error::infrastructure: ${{ matrix.os }}: $d is on $2, which has $(($1 / 1024)) MiB free, below the floor of $DISK_FLOOR_GIB GiB. The disk filled while the leg ran: read the failure above as the runner host's disk, not as a byte-equality failure."
short=1; continue
fi
case " $seen " in *" $2 "*) continue ;; esac
seen="$seen $2"; f="$p/.wai-disk-probe.$$"
if ! err=$(dd if=/dev/zero of="$f" bs=1048576 count=32 conv=fsync 2>&1); then
case "$err" in
*"No space left"*|*"Input/output error"*|*"Read-only file system"*|*"quota"*)
echo "::error::infrastructure: ${{ matrix.os }}: a 32 MiB write under $p, on $2, failed: $(printf '%s' "$err" | head -n 1). The disk filled while the leg ran: read the failure above as the runner host's disk, not as a byte-equality failure."
short=1 ;;
*) echo "::warning::a 32 MiB write under $p failed for another reason: $(printf '%s' "$err" | head -n 1)" ;;
esac
fi
rm -f "$f"
done
[ "$short" = 1 ] || echo "Every disk this leg writes to has room: the failure above is not the disk."
exit $short
# The codec wrappers are behind `codecs`, which binds C and C++ libraries that no
# container image of this fleet carries, so no other job compiles them. A feature no
# workflow enables has no CI coverage at all: without this job the wrappers' tests,
# and the limits on what a text payload may decode to (SPEC §2), pass by not running.
#
# One platform, the `macos` runner, whose host has the libraries. What the job proves
# is how the wrappers treat hostile bytes and where their limits fall, which is the
# same code on every host. A missing library fails the build step by name.
codec-wrappers:
needs: relevant
if: needs.relevant.outputs.run == 'true'
name: codec wrappers · fuzz smoke, text decode limits, C entry points, provenance, the semantic path
runs-on: macos
steps:
- uses: actions/checkout@v5
- name: Install Rust 1.99.1
shell: bash
run: |
export PATH="$HOME/.cargo/bin:$PATH"
rustup toolchain install 1.99.1 --profile minimal
rustup default 1.99.1
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
# Every one of these test files is `#![cfg(feature = "codecs")]`: built without
# the feature each reports "0 passed" and exits 0. So the step names the feature,
# and then requires that each of the five binaries ran at least one test.
- name: Fuzz smoke, the text and media decode limits and the C entry points
working-directory: wai/wai-rs
shell: bash
run: |
set -o pipefail
cargo test --locked --features codecs \
--test fuzz_smoke --test text_decode_bounds --test text_decode_memory \
--test media_decode_bounds --test ffi_robustness \
2>&1 | tee "$RUNNER_TEMP/codec-wrappers.log"
ran=$(grep -cE '^test result: ok\. [1-9][0-9]* passed' "$RUNNER_TEMP/codec-wrappers.log" || true)
if [ "$ran" -ne 5 ]; then
echo "::error::expected five test binaries that each ran tests; $ran did"
exit 1
fi
# The decoder suite's video-floor case (SPEC §5, wai/floor-conformance/): libdav1d decodes
# its grain-free wai.video.av1 envelope to the stated digest of every I420 frame and of
# the clip. The test is behind av1_base, so the step requires that it ran and passed.
- name: video floor · the decoder suite's AV1 case through libdav1d
working-directory: wai/wai-rs
shell: bash
run: |
set -o pipefail
cargo test --locked --no-default-features --features floor,av1_base --test floor_conformance 2>&1 | tee "$RUNNER_TEMP/video-floor.log"
grep -q '^test the_video_floor_case_decodes_to_its_stated_frames \.\.\. ok$' "$RUNNER_TEMP/video-floor.log" \
|| { echo "::error::the libdav1d decode of the video-floor case did not run and pass"; exit 1; }
# A grain-free AV1 base refined by wai.video.int_refine (platform-decode §6): libdav1d's
# decode of the corpus's stream is base.i420 byte for byte, the stream with film grain is
# refused before it is decoded, every other file of the corpus regenerates, and the
# cases replay over the decode. Both binaries must run their tests.
- name: grain-free AV1 · the reference decode through libdav1d
working-directory: wai/wai-rs
shell: bash
run: |
set -o pipefail
cargo run --locked --release --no-default-features --features av1_refine_vectors --bin wai_av1_refine_vectors -- verify ../platform-decode-conformance/grain-free-av1
cargo test --locked --no-default-features --features av1_base --test av1_refine 2>&1 | tee "$RUNNER_TEMP/av1-refine.log"
grep -q '^test the_reference_decode_gives_the_committed_frames \.\.\. ok$' "$RUNNER_TEMP/av1-refine.log" \
|| { echo "::error::the libdav1d decode test did not run and pass"; exit 1; }
# The platform's media framework encodes each codec through its hardware engine, the stream
# holds to platform-encode §4.3, its signed record verifies with the stream and the source
# held, and the platform's decoder gives one picture per frame. On this runner the test
# must run and pass: a host without the engine fails it rather than skipping it, so the
# step requires all ten tests of the binary, the encode among them, to have run.
- name: platform-encode · the media framework's hardware encoder, end to end
working-directory: wai/wai-rs
shell: bash
run: |
set -o pipefail
cargo test --locked --no-default-features --features platform_encode --test platform_encode \
2>&1 | tee "$RUNNER_TEMP/platform-encode.log"
grep -q '^test the_platform_encoder_makes_a_stream_that_holds_and_decodes \.\.\. ok$' "$RUNNER_TEMP/platform-encode.log" \
|| { echo "::error::the platform encoder test did not run and pass"; exit 1; }
grep -qE '^test result: ok\. 10 passed' "$RUNNER_TEMP/platform-encode.log" \
|| { echo "::error::expected the ten tests of tests/platform_encode.rs to pass"; exit 1; }
# wai_prov needs `codecs`, so no other job builds it. Its --carry path writes the
# provenance asset and receipt that wai-web's reach vectors commit: carried onto
# its own receipt, a run must write both again byte for byte. A receipt changed
# by one byte must be refused, which shows the comparison is not vacuous.
- name: wai_prov carries the committed provenance receipt, byte for byte
working-directory: wai/wai-rs
shell: bash
run: |
set -euo pipefail
cargo build --locked --features provenance,codecs,receipt_claims --bin wai_prov
bin="${CARGO_TARGET_DIR:-target}/debug/wai_prov"
v=../wai-web/reach/test/vectors/provenance
"$bin" "$RUNNER_TEMP/prov" --carry "$v/captured.receipt.json"
cmp "$RUNNER_TEMP/prov/captured.jpeg.wai" "$v/captured.jpeg.wai"
cmp "$RUNNER_TEMP/prov/captured.receipt.json" "$v/captured.receipt.json"
sed 's/"encode.jpeg"/"encode.jpeh"/' "$v/captured.receipt.json" > "$RUNNER_TEMP/changed.receipt.json"
if cmp -s "$RUNNER_TEMP/changed.receipt.json" "$v/captured.receipt.json"; then
echo "::error::the changed receipt is not changed"; exit 1
fi
if "$bin" "$RUNNER_TEMP/prov-changed" --carry "$RUNNER_TEMP/changed.receipt.json"; then
echo "::error::a changed receipt was carried"; exit 1
fi
# tests/semantic_path.rs needs `semantic`, which builds on `codecs` and binds the
# ONNX runtime, so this is the one job that can run it.
- name: The semantic path
working-directory: wai/wai-rs
shell: bash
run: |
set -o pipefail
cargo test --locked --features codecs,semantic --test semantic_path 2>&1 | tee "$RUNNER_TEMP/semantic-path.log"
ran=$(grep -cE '^test result: ok\. [1-9][0-9]* passed' "$RUNNER_TEMP/semantic-path.log" || true)
if [ "$ran" -ne 1 ]; then
echo "::error::the semantic path's test binary ran no test"
exit 1
fi
# The C2PA emitter is gated behind an off-by-default feature, so the four-platform
# job above — which builds a fixed feature list — never compiles it. A feature no
# workflow enables has no CI coverage at all: the suite passes, and the tests in it
# simply do not run.
#
# One platform, deliberately. Everything the matrix above exists to prove is about
# byte-identical reconstruction across architectures; a signed manifest makes no
# such claim and would not become more true for being emitted on four machines. What
# this job proves is that the manifest is still well-formed, still carries the three
# assertions, and still validates — which is architecture-independent.
c2pa-emitter:
needs: relevant
if: needs.relevant.outputs.run == 'true'
name: C2PA manifest emission · signed, read back, validated
runs-on: linux-arm64
steps:
- uses: actions/checkout@v4
- name: Install Rust 1.99.1
shell: bash
run: |
export PATH="$HOME/.cargo/bin:$PATH"
command -v rustup >/dev/null 2>&1 || \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.99.1
rustup toolchain install 1.99.1 --profile minimal
rustup default 1.99.1
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
# `rust_native_crypto` rather than the SDK's default openssl backend, so this
# job needs no system libraries — the same property the matrix above depends on.
# Asserted, not assumed: an openssl-linked build would still pass the tests here
# while quietly making the crate undistributable to a bare runner.
- name: The emitter pulls in no system crypto
working-directory: wai/wai-rs
shell: bash
run: |
for dep in openssl openssl-sys; do
if cargo tree --no-default-features --features c2pa_emit -i "$dep" >/dev/null 2>&1; then
echo "::error::$dep reached the c2pa_emit dependency graph"
exit 1
fi
done
echo "no system crypto in the graph"
- name: Emit, sign, read back, validate
working-directory: wai/wai-rs
run: cargo test --lib --no-default-features --features c2pa_emit c2pa_emit
# A room-session receipt (extensions/live-metering.md §7) in a real signed manifest:
# energy_kwh only where one class covers the session whole, the companion always.
- name: Emit a room-session receipt, sign, read back, validate
working-directory: wai/wai-rs
run: cargo test --lib --no-default-features --features c2pa_emit,live_metering c2pa_emit::tests::a_session
# The feature must stay optional. If `provenance` ever pulls c2pa in, every
# consumer that wanted only the binding types inherits a large dependency tree
# they did not ask for, and the "off by default" claim in the extension is false.
- name: The feature stays optional
working-directory: wai/wai-rs
shell: bash
run: |
if cargo tree --no-default-features --features provenance -i c2pa >/dev/null 2>&1; then
echo "::error::c2pa is reachable from the provenance feature; it must stay opt-in"
exit 1
fi
cargo build --no-default-features --features provenance
echo "provenance builds without c2pa"
Reference in a new issue View git blame Copy permalink