b52398f4f1
The jobs that hold the CI-regenerated int_hyper set and the jpegai/v1 bundle now also run tools/wai_ops_check.py certify on the same files, with --check against the certificate committed by digest; the checker joins both path filters. wai-ops.yml's checker step already runs the Python certifier on every committed set and both falsification sets; its name and the header say so. The ops-conformance README describes the certifier half, the --external option and the certify exit statuses. Co-Authored-By: Claude Opus 5.5 <[email protected]>
268 lines
12 KiB
YAML
268 lines
12 KiB
YAML
name: wai-ops integer operators
|
|
|
|
# The integer operator set (wai/extensions/integer-operators.md), checked on every
|
|
# fabric leg the corpus has today. Every leg recomputes every vector file of
|
|
# wai/ops-conformance/ from its inputs and requires the committed bytes:
|
|
#
|
|
# - Linux x86_64, Linux arm64 and macOS arm64, natively, debug and release: the
|
|
# unit tests (the bound rules against hand-computed values among them), the
|
|
# goldens (every committed parameter set lowered, loaded through load step 8 and
|
|
# decoded through the interpreter to its committed output digest), the step-8
|
|
# and verdict tests and the doc tests; `wai-verify vectors verify`, which also
|
|
# re-lowers every set, recomputes its bounds certificate from the committed
|
|
# lowered graphs and regenerates the whole corpus, so every certificate is
|
|
# byte-identical on every leg; and `wai-verify bounds` on the falsification sets
|
|
# of integer-operators §14.3, which must exit 1, and on committed sets, which
|
|
# must exit 0 under their profile;
|
|
# - WASI (wasm32-wasip2) under wasmtime, a 32-bit target: the same tests, and the
|
|
# same `vectors verify`;
|
|
# - a browser's engine (wasm32-unknown-unknown) under Node: the recomputation of
|
|
# every vector file;
|
|
# - the independent checker, tools/wai_ops_check.py: Python standard library only,
|
|
# sharing no code with wai-ops, rewrites every vector file and every lowered graph,
|
|
# and, as a second certifier, every certificate, the falsification sets' included
|
|
# (each must fail the profile it claims).
|
|
#
|
|
# There is no Windows leg: the Windows runner has been offline since 2026-08-28.
|
|
|
|
# One run per branch at a time. A push supersedes the run before it.
|
|
#
|
|
# There is one `macos` runner and it is the dev Mac, so every macOS job in this repo
|
|
# queues on it; only the newest commit's verdict is wanted, so older runs are
|
|
# cancelled. `cancel-in-progress: true` also applies to `main`, which advances by
|
|
# merge commits whose tree contains the commits before them.
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
on:
|
|
pull_request:
|
|
paths:
|
|
- 'wai/wai-ops/**'
|
|
# The certifier, the lowerings and the corpus tool, and the Kani host they share.
|
|
- 'wai/wai-verify/**'
|
|
- 'wai/ops-conformance/**'
|
|
- 'wai/tools/wai_ops_*.py'
|
|
# The text the checker is held to.
|
|
- 'wai/extensions/integer-operators.md'
|
|
# Every committed parameter set the corpus lowers, and the digests the goldens
|
|
# compare against (neural-conformance/sha256.json among them).
|
|
- 'wai/neural-conformance/**'
|
|
- 'wai/wai-rs/tests/vectors/int_hyper/**'
|
|
- 'wai/wai-rs/tests/vectors/int_audio/**'
|
|
- 'wai/wai-rs/tests/vectors/int_splat/**'
|
|
- 'wai/wai-rs/tests/vectors/int_mlicpp/**'
|
|
- 'wai/wai-rs/tests/vectors/int_world/**'
|
|
- 'wai/wai-rs/tests/vectors/int_world2/**'
|
|
- '.github/workflows/wai-ops.yml'
|
|
push:
|
|
branches:
|
|
- main
|
|
paths:
|
|
- 'wai/wai-ops/**'
|
|
# The certifier, the lowerings and the corpus tool, and the Kani host they share.
|
|
- 'wai/wai-verify/**'
|
|
- 'wai/ops-conformance/**'
|
|
- 'wai/tools/wai_ops_*.py'
|
|
- 'wai/extensions/integer-operators.md'
|
|
- 'wai/neural-conformance/**'
|
|
- 'wai/wai-rs/tests/vectors/int_hyper/**'
|
|
- 'wai/wai-rs/tests/vectors/int_audio/**'
|
|
- 'wai/wai-rs/tests/vectors/int_splat/**'
|
|
- 'wai/wai-rs/tests/vectors/int_mlicpp/**'
|
|
- 'wai/wai-rs/tests/vectors/int_world/**'
|
|
- 'wai/wai-rs/tests/vectors/int_world2/**'
|
|
- '.github/workflows/wai-ops.yml'
|
|
|
|
jobs:
|
|
native:
|
|
name: ${{ matrix.label }}
|
|
runs-on: ${{ matrix.os }}
|
|
timeout-minutes: 40
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
- { os: linux, label: linux-x86_64 }
|
|
- { os: linux-arm64, label: linux-aarch64 }
|
|
- { os: macos, label: macos-arm64 }
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
# The docker images ship no Rust; the macOS host has it, but launchd's PATH omits
|
|
# ~/.cargo/bin.
|
|
- name: Install Rust 1.98.0
|
|
shell: bash
|
|
run: |
|
|
export PATH="$HOME/.cargo/bin:$PATH"
|
|
command -v rustup >/dev/null 2>&1 || \
|
|
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.98.0
|
|
rustup toolchain install 1.98.0 --profile minimal
|
|
rustup default 1.98.0
|
|
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
|
|
|
|
- name: Unit, goldens, step-8, verdict and doc tests, debug
|
|
working-directory: wai/wai-ops
|
|
shell: bash
|
|
run: cargo test --locked
|
|
|
|
- name: Unit, goldens, step-8 and verdict tests, release
|
|
working-directory: wai/wai-ops
|
|
shell: bash
|
|
run: cargo test --locked --release
|
|
|
|
- name: Build wai-verify, and its own tests (the witness through the interpreter; --check's path)
|
|
working-directory: wai/wai-verify
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
cargo build --locked --release --features cli --bin wai-verify
|
|
cargo test --locked --release --features cli --bin wai-verify
|
|
|
|
- name: Every vector file recomputed, every set re-lowered and certified, the corpus regenerated
|
|
working-directory: wai
|
|
shell: bash
|
|
run: wai-verify/target/release/wai-verify vectors verify .
|
|
|
|
# The exit status of `wai-verify bounds` (0: the verdict meets the profile, 1: it
|
|
# fails it, 3: a difference from the committed certificate), on the sets whose
|
|
# verdict integer-operators §14.3 and the design's predictions fix.
|
|
- name: The falsification sets fail their profile; committed sets meet theirs
|
|
working-directory: wai
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
V=wai-verify/target/release/wai-verify
|
|
C=ops-conformance/certificates/wai.neural.int_synth
|
|
T="${RUNNER_TEMP:-/tmp}"
|
|
bounds() {
|
|
local want=$1; shift
|
|
set +e; "$V" bounds "$@" --out "$T/cert.json"; local got=$?; set -e
|
|
[ "$got" = "$want" ] || { echo "::error::wai-verify bounds $* exited $got, not $want"; exit 1; }
|
|
}
|
|
# One unit past i64 under acc64, and a claim of acc32 one weight past it.
|
|
bounds 1 ops-conformance/sets/overflow_acc64/int_synth.json --capability wai.neural.int_synth --profile acc64 \
|
|
--check $C/b35460f39f31a559e97e66b8dde4597a309cbc44199c40097f935dd8e6e7305a.json
|
|
bounds 1 ops-conformance/sets/claim_acc32/int_synth.json --capability wai.neural.int_synth --profile acc32 \
|
|
--check $C/76916c8f85da7663c09fba8cb091d628ed38328b680563285b8056cec82fd1ac.json
|
|
# facade meets acc32; photo meets acc64 and fails acc32 (at node 6, convT.acc).
|
|
bounds 0 neural-conformance/vectors/facade/int_synth.json --capability wai.neural.int_synth --profile acc32 \
|
|
--check $C/c6a362696d61605931ef4da98ae630362dcd73ea6be7864db6e61d6c5abbcd24.json
|
|
bounds 0 neural-conformance/vectors/photo/int_synth.json --capability wai.neural.int_synth --profile acc64 \
|
|
--check $C/50868e72f5f1599569c413ed1f67ab8610fa0b3c17919162cbf370f49275125c.json
|
|
# With --witness, a node-local graph for each failing node: photo's node 6, whose
|
|
# input attains convT.acc's upper end. With --advise, the clamps that would let
|
|
# the overflow set meet acc64.
|
|
bounds 1 neural-conformance/vectors/photo/int_synth.json --capability wai.neural.int_synth --profile acc32 \
|
|
--witness "$T/witness"
|
|
test -s "$T/witness/synth-6.json"
|
|
grep -q '"acc":"26172239136"' "$T/witness/synth-6.json"
|
|
bounds 1 ops-conformance/sets/overflow_acc64/int_synth.json --capability wai.neural.int_synth --advise acc64
|
|
# A certificate that is not the committed one is exit 3.
|
|
bounds 3 neural-conformance/vectors/photo/int_synth.json --capability wai.neural.int_synth \
|
|
--check $C/c6a362696d61605931ef4da98ae630362dcd73ea6be7864db6e61d6c5abbcd24.json
|
|
|
|
wasip2:
|
|
name: wasm32-wasip2 (wasmtime)
|
|
runs-on: linux
|
|
timeout-minutes: 30
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Install Rust 1.98.0 with the wasm32-wasip2 target
|
|
shell: bash
|
|
run: |
|
|
export PATH="$HOME/.cargo/bin:$PATH"
|
|
command -v rustup >/dev/null 2>&1 || \
|
|
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.98.0
|
|
rustup toolchain install 1.98.0 --profile minimal
|
|
rustup default 1.98.0
|
|
rustup target add wasm32-wasip2
|
|
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
|
|
|
|
- name: Install wasmtime 38.0.3
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
v=38.0.3
|
|
curl --proto '=https' --tlsv1.2 -sSfL -o "$RUNNER_TEMP/wasmtime.tar.xz" \
|
|
"https://github.com/bytecodealliance/wasmtime/releases/download/v$v/wasmtime-v$v-x86_64-linux.tar.xz"
|
|
tar -C "$RUNNER_TEMP" -xf "$RUNNER_TEMP/wasmtime.tar.xz"
|
|
echo "$RUNNER_TEMP/wasmtime-v$v-x86_64-linux" >> "$GITHUB_PATH"
|
|
|
|
# The tests read committed files through CARGO_MANIFEST_DIR/.., so the wai
|
|
# directory is preopened at its own absolute path.
|
|
- name: Unit, goldens, step-8 and verdict tests, under WASI
|
|
working-directory: wai/wai-ops
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
W=$(cd .. && pwd)
|
|
CARGO_TARGET_WASM32_WASIP2_RUNNER="wasmtime run --dir $W::$W" \
|
|
cargo test --locked --release --target wasm32-wasip2
|
|
|
|
- name: Build wai-verify for wasm32-wasip2
|
|
working-directory: wai/wai-verify
|
|
shell: bash
|
|
run: cargo build --locked --release --features cli --target wasm32-wasip2 --bin wai-verify
|
|
|
|
- name: Every vector file recomputed, every set re-lowered and certified, the corpus regenerated, under WASI
|
|
working-directory: wai
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
wasmtime --version
|
|
wasmtime run --dir .::/wai wai-verify/target/wasm32-wasip2/release/wai-verify.wasm vectors verify /wai
|
|
|
|
node:
|
|
name: wasm32 under Node
|
|
runs-on: linux
|
|
timeout-minutes: 30
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Install Rust 1.98.0 with the wasm32 target
|
|
shell: bash
|
|
run: |
|
|
export PATH="$HOME/.cargo/bin:$PATH"
|
|
command -v rustup >/dev/null 2>&1 || \
|
|
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.98.0
|
|
rustup toolchain install 1.98.0 --profile minimal
|
|
rustup default 1.98.0
|
|
rustup target add wasm32-unknown-unknown
|
|
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
|
|
|
|
- uses: actions/setup-node@v4
|
|
with:
|
|
node-version: '22'
|
|
|
|
- name: Build the recomputation for wasm32-unknown-unknown
|
|
working-directory: wai/wai-ops/node
|
|
shell: bash
|
|
run: cargo build --locked --release --target wasm32-unknown-unknown
|
|
|
|
- name: Every vector file recomputed under Node
|
|
working-directory: wai/wai-ops/node
|
|
shell: bash
|
|
run: node verify.mjs target/wasm32-unknown-unknown/release/wai_ops_node.wasm ../../ops-conformance
|
|
|
|
checker:
|
|
name: independent checker (Python)
|
|
runs-on: linux
|
|
timeout-minutes: 20
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- uses: actions/setup-python@v5
|
|
with:
|
|
python-version: '3.11'
|
|
|
|
# Standard library only, so it needs nothing installed.
|
|
- name: wai_ops_check.py rewrites every vector file, every lowered graph and every certificate
|
|
working-directory: wai
|
|
shell: bash
|
|
run: python3 -I tools/wai_ops_check.py ops-conformance
|
|
|
|
- name: Every workflow declares a concurrency group
|
|
shell: bash
|
|
run: bash scripts/check-workflow-concurrency.sh
|