5896a597e9
The job's install step named Rust 1.98.0, two releases behind the pin in rust-toolchain.toml (1.99.1 since #606). It now bootstraps rustup with no default toolchain and runs `rustup toolchain install` with no name, which installs the toolchain rust-toolchain.toml names, as the wai-quantum member jobs do. A later change of the pin needs no edit to this job. The step then logs the active toolchain. What the job tests is unchanged. Main's run 19418 (job 81788) shows the old step installed 1.98.0 and set it as the default, and cargo then auto-installed 1.99.1 from the toolchain file's override and compiled the test with it. The 1.98.0 download was wasted and the step's name was wrong. The new step installs the same six components up front. Co-Authored-By: Claude Opus 5.5 <[email protected]>
321 lines
18 KiB
YAML
321 lines
18 KiB
YAML
name: WAI neural sink gates
|
|
|
|
# The neural sink refuses some envelopes before it touches a model: a pin that
|
|
# the parameter sets it holds do not satisfy, with SPEC §3.1's reason
|
|
# (ModelRegistry::resolve_verified, decode_envelope, decode_multi), and a `replicate` that
|
|
# SPEC §7 does not back (decode_envelope_strict): over a capability whose registry
|
|
# class cannot back it, or with a prior that is float or absent, or that declares a
|
|
# form not established to reconstruct the same tensors at every reader. Those
|
|
# refusals live behind the
|
|
# `neural` feature, which wai-conformance does not build, and the tests that
|
|
# exercise them need no model, so wai-neural-e2e's reference filter never selects
|
|
# them. This job runs exactly those tests and fails if any of them is not run:
|
|
# a filter that matches nothing reports "0 passed" and exits 0.
|
|
#
|
|
# It also runs the `neural` dispatch tests that need no model download, over the
|
|
# committed int_mlicpp vector and the int_synth corpus: the form-less and set-form
|
|
# pins through the strict gate, the int_synth pin over the spec it reads, the
|
|
# several-file capabilities refusing a form-less pin, a pinned decode reading only
|
|
# the bytes it verified, an unpinned decode at a sink with a key log checked over
|
|
# every identity of what it read once and reading only that (or, holding the
|
|
# capability's sets to §7.9, the part §7.9 authorised), an encrypted envelope inert at the
|
|
# sink (SPEC §6.4 step 2), a WAI2 envelope whose pins pick the held model, the strict
|
|
# rule read for the capability decoded through (a sound fallback behind an unsound
|
|
# primary), SPEC §4 step 6's order of reasons after a set_incomplete decode, and the
|
|
# replicate_* reason the strict WAI2 policy reports for an entry it leaves out,
|
|
# and a candidate held at the sink leaving the strict path's reason unchanged.
|
|
# With `key_discovery` as well, it runs SPEC §3.1's check 5 at the neural sink: a
|
|
# registry given a key log refuses the committed int_mlicpp set once the log revokes
|
|
# its digest, and when the sink holds no fresh status, and decodes it while a grant
|
|
# holds it.
|
|
#
|
|
# The dispatch reports the decode path that ran (determinism-tiers §5). The tests
|
|
# with one path run on committed inputs in the first job. A second job fetches the
|
|
# pinned jpegai/v1 bundle and runs wai.image.jpegai's exact decoder through the
|
|
# dispatch, with WAI_REQUIRE_SOTA_VECTORS=1, so a missing input is a failure. The
|
|
# float decode needs the JPEG-AI reference-software assets, which are not in the
|
|
# repository, so its test runs locally only, as tests/jpegai_live.rs does
|
|
# (wai-jpegai-e2e.yml).
|
|
|
|
# One run per branch at a time. A push supersedes the run before it.
|
|
#
|
|
# There is one `macos` runner and it is the dev Mac, so every macOS job in this
|
|
# repo queues on it. With no concurrency group, a session pushing four versions
|
|
# of a branch queued four complete sets of runs and every one of them had to
|
|
# drain: on 2026-10-05 a PR's 21 contexts settled none in 70 minutes behind a
|
|
# backlog of 25 runs, and the queue grew faster than it cleared. Only the newest
|
|
# commit's verdict is wanted, so the older runs are cancelled.
|
|
#
|
|
# Verified on this forge (Forgejo 16.0.5) in `ci-test` before being added here:
|
|
# the key parses, and a newer push moves an older run to `cancelled` even while
|
|
# it is still queued, which is the case that matters.
|
|
#
|
|
# `cancel-in-progress: true` also applies to `main`. That is deliberate and
|
|
# safe here: main advances by merge commits whose tree contains the commits
|
|
# before them, so the newer run covers the older one's changes.
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
on:
|
|
pull_request:
|
|
paths:
|
|
- 'wai/wai-rs/src/neural/**'
|
|
- 'wai/wai-rs/src/container.rs'
|
|
- 'wai/wai-rs/src/codecs/mod.rs'
|
|
- 'wai/wai-rs/src/codecs/prior_form.rs'
|
|
- 'wai/wai-rs/src/pin.rs'
|
|
- 'wai/wai-rs/src/key_log.rs'
|
|
- 'wai/wai-rs/src/key_log/**'
|
|
- 'wai/wai-rs/src/signer_id.rs'
|
|
- 'wai/wai-rs/src/claim_receipt.rs'
|
|
- 'wai/wai-rs/src/claim_text.rs'
|
|
- 'wai/wai-rs/src/ffi.rs'
|
|
- 'wai/wai-rs/src/ffi_neural.rs'
|
|
- 'wai/wai-rs/tests/ffi_neural_pin.rs'
|
|
- 'wai/wai-rs/tests/neural_cost_limit.rs'
|
|
- 'wai/wai-rs/tests/jpegai_decode_path.rs'
|
|
# The exact JPEG-AI decoder the second job runs through the dispatch, and the
|
|
# fetch of its pinned inputs.
|
|
- 'wai/wai-rs/src/jpegai_*.rs'
|
|
- 'wai/wai-rs/src/float_repro.rs'
|
|
- 'wai/wai-rs/src/int_kernels.rs'
|
|
- 'wai/tools/wai_jpegai_fetch_e2e_inputs.sh'
|
|
- 'wai/tools/jpegai-e2e-v1.sha256'
|
|
- 'wai/wai-rs/tests/vectors/jpegai/**'
|
|
- 'wai/wai-rs/tests/support/**'
|
|
- 'wai/wai-rs/src/session_recorder.rs'
|
|
- 'wai/wai-rs/src/reconstruction_binding.rs'
|
|
- 'wai/wai-rs/src/claim_receipt/**'
|
|
- 'wai/wai-rs/src/int_synth.rs'
|
|
# The integer operator set the integer decoders load their sets through.
|
|
- 'wai/wai-rs/src/int_ops.rs'
|
|
- 'wai/wai-ops/**'
|
|
- 'wai/wai-rs/src/ort_session.rs'
|
|
- 'wai/neural-conformance/vectors/**'
|
|
- 'wai/pin-conformance/**'
|
|
- 'wai/wai-rs/tests/neural_native_decode.rs'
|
|
- 'wai/wai-rs/src/int_mlicpp.rs'
|
|
- 'wai/wai-rs/tests/vectors/int_mlicpp/**'
|
|
# The integer capabilities the neural sink delegates to int_payload::decode_int,
|
|
# and the vectors its tests decode through it.
|
|
- 'wai/wai-rs/src/int_payload/**'
|
|
- 'wai/wai-rs/src/int_video.rs'
|
|
- 'wai/wai-rs/src/int_hyper_synth.rs'
|
|
- 'wai/wai-rs/src/int_audio.rs'
|
|
- 'wai/wai-rs/src/int_splat.rs'
|
|
- 'wai/wai-rs/tests/vectors/int_hyper/**'
|
|
- 'wai/wai-rs/tests/vectors/int_video/**'
|
|
- 'wai/wai-rs/tests/vectors/int_audio/**'
|
|
- 'wai/wai-rs/tests/vectors/int_splat/**'
|
|
- 'wai/wai-rs/Cargo.lock'
|
|
- '.github/workflows/wai-neural-gates.yml'
|
|
schedule:
|
|
- cron: '25 5 * * *'
|
|
|
|
jobs:
|
|
# Does this pull request's own diff touch the paths below? This forge matches a
|
|
# workflow's `paths:` against the push, so a merge of main into a branch fires the
|
|
# workflow whatever the pull request changes; every job here waits on this answer.
|
|
# A push to main, a schedule, or anything the script cannot determine answers
|
|
# `true` (scripts/pr-touches-paths.py).
|
|
relevant:
|
|
name: does this change touch it
|
|
runs-on: linux
|
|
outputs:
|
|
run: ${{ steps.gate.outputs.run }}
|
|
steps:
|
|
- uses: actions/checkout@v5
|
|
with:
|
|
fetch-depth: 0
|
|
- id: gate
|
|
run: |
|
|
set -euo pipefail
|
|
r=$(python3 scripts/pr-touches-paths.py .github/workflows/wai-neural-gates.yml \
|
|
"${{ github.event.pull_request.base.sha }}" \
|
|
"${{ github.event.pull_request.head.sha }}")
|
|
echo "this change touches wai-neural-gates: $r"
|
|
echo "run=$r" >> "$GITHUB_OUTPUT"
|
|
|
|
neural-gates:
|
|
needs: relevant
|
|
if: needs.relevant.outputs.run == 'true'
|
|
name: model-free refusals of the neural sink
|
|
runs-on: linux
|
|
# The ONNX Runtime binary the `ort` crate downloads needs glibc 2.38 and a
|
|
# GCC 13 libstdc++ at link time (`__isoc23_strtol`, `_M_replace_cold`). The
|
|
# `linux` label's default image is Ubuntu 22.04 (glibc 2.35, GCC 11), where
|
|
# the `neural` test binary fails to link, so this job runs in the 24.04 image.
|
|
container:
|
|
image: catthehacker/ubuntu:act-24.04
|
|
steps:
|
|
- uses: actions/checkout@v5
|
|
# Bootstrap rustup: the docker runner image ships no Rust.
|
|
- name: Install Rust 1.99.1
|
|
run: |
|
|
export PATH="$HOME/.cargo/bin:$PATH"
|
|
command -v rustup >/dev/null 2>&1 || \
|
|
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.99.1
|
|
rustup toolchain install 1.99.1 --profile minimal
|
|
rustup default 1.99.1
|
|
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
|
|
- name: Prior-hash refusal, the strict replicate gate, and the int_mlicpp dispatch
|
|
working-directory: wai/wai-rs
|
|
shell: bash
|
|
# --no-default-features: `neural` needs only `ort`, not the C codec chain.
|
|
# --exact: each name selects one test. The count check turns a renamed or
|
|
# removed test into a failure instead of a silent "0 passed".
|
|
run: |
|
|
set -euo pipefail
|
|
cargo test --no-default-features --features neural --test neural_native_decode -- \
|
|
--exact resolve_checked_verifies_prior_hash strict_decode_gates_unsound_replicate \
|
|
int_mlicpp_decodes_through_the_strict_dispatch int_mlicpp_set_pin_through_the_strict_dispatch \
|
|
int_synth_pin_covers_the_spec_it_reads several_file_capabilities_need_the_set_form \
|
|
pinned_decode_reads_only_verified_bytes wai2_decode_multi_picks_the_verified_rendition \
|
|
strict_gate_reads_the_capability_it_decodes_through set_incomplete_at_step_3_outranks_the_fallback_refusal \
|
|
decode_multi_strict_reports_replicate_reasons strict_gate_ignores_a_held_capability_it_cannot_dispatch \
|
|
int_hyper_decodes_through_the_neural_dispatch int_synth_check_5_covers_the_spec_a_model_directory_holds \
|
|
an_integer_capability_without_a_decoder_moves_on_to_the_fallback \
|
|
a_listing_that_does_not_match_the_files_read_authorises_nothing \
|
|
a_pinned_set_revoked_under_its_file_digest_is_refused_through_a_listing_pin \
|
|
a_granted_listing_authorises_only_the_files_it_names \
|
|
an_encrypted_envelope_is_inert_at_the_reference_sink \
|
|
a_revoke_only_key_log_checks_and_reads_the_set_once \
|
|
2>&1 | tee "$RUNNER_TEMP/neural-gates.log"
|
|
grep -q "test result: ok. 20 passed; 0 failed" "$RUNNER_TEMP/neural-gates.log"
|
|
# The integer video, audio and splat capabilities through the neural dispatch, which
|
|
# delegates them to int_payload::decode_int: each decodes to Decoded::Int with its
|
|
# committed SHA-256, and wai.video.int_motion is advertised with ModelRegistry::builtin.
|
|
- name: The neural sink decodes the integer video, audio and splat payloads
|
|
working-directory: wai/wai-rs
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
cargo test --no-default-features --features neural,video_int,audio_int,splat_int --test neural_native_decode -- \
|
|
--exact integer_media_decode_through_the_neural_dispatch \
|
|
2>&1 | tee "$RUNNER_TEMP/neural-int.log"
|
|
grep -q "test result: ok. 1 passed; 0 failed" "$RUNNER_TEMP/neural-int.log"
|
|
# The registry's cost limit refuses an integer payload over it before any symbol is
|
|
# decoded (cost_over_limit) and moves on to the next entry; the C ABI refuses an
|
|
# integer video it has no output kind for before decoding it; and the declared
|
|
# cost over a registry's set, and through the C ABI given the set's path, is the
|
|
# library's. Its own binary: a live-bytes allocator measures each refusal.
|
|
- name: The neural sink applies the declared cost before decoding
|
|
working-directory: wai/wai-rs
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
cargo test --no-default-features --features neural,ffi,video_int,audio_int,splat_int --test neural_cost_limit \
|
|
2>&1 | tee "$RUNNER_TEMP/neural-cost.log"
|
|
grep -q "test result: ok. 3 passed; 0 failed" "$RUNNER_TEMP/neural-cost.log"
|
|
# The dispatch reports the capability it decoded through and the path that ran
|
|
# (extensions/determinism-tiers.md §5): a capability with one path says so, through
|
|
# its own capability and through its fallback, and a reconstruction binding over a
|
|
# decode-equivalence capability is checked under its descriptor. Committed inputs
|
|
# only. The exact JPEG-AI decoder runs in the next job; the float decode runs locally
|
|
# only (see the header).
|
|
- name: The neural dispatch reports the decode path that ran
|
|
working-directory: wai/wai-rs
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
cargo test --no-default-features --features neural,receipt_claims,video_int --test jpegai_decode_path -- \
|
|
--exact a_capability_with_one_path_reports_it_and_the_capability_it_decoded_through \
|
|
a_binding_over_a_decode_equivalence_capability_is_checked_under_its_descriptor \
|
|
2>&1 | tee "$RUNNER_TEMP/neural-path.log"
|
|
grep -q "test result: ok. 2 passed; 0 failed" "$RUNNER_TEMP/neural-path.log"
|
|
- name: A key log refuses a revoked pinned set at the neural sink (SPEC §3.1 check 5)
|
|
working-directory: wai/wai-rs
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
cargo test --no-default-features --features neural,key_discovery --test neural_native_decode -- \
|
|
--exact int_mlicpp_revoked_set_is_refused_through_the_key_log \
|
|
2>&1 | tee "$RUNNER_TEMP/neural-key-log.log"
|
|
grep -q "test result: ok. 1 passed; 0 failed" "$RUNNER_TEMP/neural-key-log.log"
|
|
# Library unit tests that exist only under `neural` and need no model: the
|
|
# bmshj2018 CDF table cache stays bounded and evicts least recently used;
|
|
# the ONNX reader finds a tensor kept in another file, wherever it sits; and a
|
|
# pinned decode refuses such a model with set_incomplete before the runtime
|
|
# sees it, while the same graph with its tensor inside loads.
|
|
- name: Model-free neural unit tests
|
|
working-directory: wai/wai-rs
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
cargo test --no-default-features --features neural --lib -- --exact neural::image::tests::cdf_cache_is_bounded \
|
|
neural::onnx_external::tests::a_tensor_kept_inside_the_model_is_not_external \
|
|
neural::onnx_external::tests::an_external_initializer_is_found_with_its_location \
|
|
neural::onnx_external::tests::an_external_tensor_in_a_subgraph_is_found \
|
|
neural::onnx_external::tests::external_data_without_a_location_still_marks_the_tensor \
|
|
neural::onnx_external::tests::bytes_that_are_not_a_protobuf_message_are_an_error \
|
|
neural::params::tests::a_pinned_model_with_external_tensors_is_set_incomplete \
|
|
2>&1 | tee "$RUNNER_TEMP/neural-lib.log"
|
|
grep -q "test result: ok. 7 passed; 0 failed" "$RUNNER_TEMP/neural-lib.log"
|
|
- name: The neural C ABI reports pin and replicate refusals and decodes WAI2
|
|
working-directory: wai/wai-rs
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
cargo test --no-default-features --features neural,ffi --test ffi_neural_pin -- \
|
|
--exact tampered_model_is_wai_err_prior strict_replicate_is_wai_err_unsound_replicate \
|
|
wai2_decode_ex_picks_the_held_pin failed_decode_ex_clears_out \
|
|
decode_ex_reports_the_path_and_the_fallback \
|
|
2>&1 | tee "$RUNNER_TEMP/neural-ffi.log"
|
|
grep -q "test result: ok. 5 passed; 0 failed" "$RUNNER_TEMP/neural-ffi.log"
|
|
|
|
# wai.image.jpegai's exact decoder through the dispatch: the result says
|
|
# DecodePath::Exact, a session claim recorded from it states `variant` `exact` and
|
|
# verifies, and the binding check holds over its pixels, differs over a changed one,
|
|
# and reports a binding of another decoderId as another reconstruction
|
|
# (tests/jpegai_decode_path.rs). Its inputs are the pinned jpegai/v1 bundle, fetched
|
|
# and checked against wai/tools/jpegai-e2e-v1.sha256 as wai-jpegai-e2e.yml does; an
|
|
# outage of the models root fails the job. The `neural` feature links the prebuilt
|
|
# ONNX Runtime, so this job runs in the 24.04 image, as the job above does.
|
|
jpegai-decode-path:
|
|
needs: relevant
|
|
if: needs.relevant.outputs.run == 'true'
|
|
name: the exact JPEG-AI decoder reports its path (pinned inputs, required)
|
|
runs-on: linux
|
|
timeout-minutes: 60
|
|
container:
|
|
image: catthehacker/ubuntu:act-24.04
|
|
env:
|
|
WAI_REQUIRE_SOTA_VECTORS: '1'
|
|
steps:
|
|
- uses: actions/checkout@v5
|
|
# Bootstrap rustup: the docker runner image ships no Rust. The toolchain is the
|
|
# repository's pin: `rustup toolchain install` with no name installs the one
|
|
# rust-toolchain.toml names, as the wai-quantum member jobs do, so a change of the
|
|
# pin needs no edit here. The last line logs the toolchain cargo will use.
|
|
- name: Install Rust (the pin in rust-toolchain.toml)
|
|
run: |
|
|
export PATH="$HOME/.cargo/bin:$PATH"
|
|
command -v rustup >/dev/null 2>&1 || \
|
|
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain none
|
|
rustup toolchain install
|
|
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
|
|
rustup show active-toolchain
|
|
- name: Fetch the JPEG-AI inputs and check each against the pinned manifest
|
|
working-directory: wai
|
|
shell: bash
|
|
run: tools/wai_jpegai_fetch_e2e_inputs.sh "$RUNNER_TEMP/jpegai_e2e"
|
|
- name: The exact decoder reports its path, in the result, the claim and the binding check (missing input = failure)
|
|
working-directory: wai/wai-rs
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
[ "${WAI_REQUIRE_SOTA_VECTORS:-}" = 1 ] \
|
|
|| { echo "::error::WAI_REQUIRE_SOTA_VECTORS must be 1 in this job"; exit 1; }
|
|
export WAI_JPEGAI_WEIGHTS_DIR="$RUNNER_TEMP/jpegai_e2e"
|
|
export WAI_JPEGAI_NORM="$RUNNER_TEMP/jpegai_e2e/norm.json"
|
|
cargo test --release --no-default-features --features neural,receipt_claims,video_int --test jpegai_decode_path -- \
|
|
--exact the_jpegai_exact_decoder_reports_itself --show-output 2>&1 | tee "$RUNNER_TEMP/jpegai-path.log"
|
|
grep -q "test result: ok. 1 passed; 0 failed" "$RUNNER_TEMP/jpegai-path.log"
|
|
grep -q "^test the_jpegai_exact_decoder_reports_itself \.\.\. ok$" "$RUNNER_TEMP/jpegai-path.log"
|
|
# A skip also prints `test X ... ok`; every skip writes a banner with
|
|
# "SKIPPED" and "NOT VERIFIED" (tests/support/require_vectors.rs).
|
|
if grep -nE 'SKIPPED|NOT VERIFIED' "$RUNNER_TEMP/jpegai-path.log"; then
|
|
echo "::error::the exact JPEG-AI decode skipped instead of checking its input"
|
|
exit 1
|
|
fi
|