98f2755d7f
The metal job runs the arithmetic probe, the corpus at both widths and the mutants on the host's GPU, every capability that runs a wai-ops/1 graph with its graphs there (the jpegai/v1 inputs fetched and required), and the corpus in a headless browser's WebGPU. WAI_OPS_GPU_EXPECT declares the adapter and WAI_OPS_GPU_REQUIRE makes its absence a failure; each step then requires its report lines with their counts. The checks job, on a runner with no GPU, requires that the device tests skip without WAI_OPS_GPU_REQUIRE and fail with it, and builds the browser module, wai-rs's GPU target and the bench tool. The arith test now appends its result line to the report as the others do. Co-Authored-By: Claude Opus 5.5 <[email protected]>
220 lines
12 KiB
YAML
220 lines
12 KiB
YAML
name: wai-ops-gpu integer operators on GPUs
|
|
|
|
# wai-ops/1-i32 on GPUs (wai/wai-ops-gpu): the integer operator set's graph operators
|
|
# as WGSL kernels through wgpu, exact at acc32 (one 32-bit word per value) and at acc64
|
|
# (64- and 128-bit arithmetic built from 32-bit words), and the integer decoders with
|
|
# their graphs on the device (wai-rs `ops_gpu`).
|
|
#
|
|
# - metal (the `macos` runner, whose host has a GPU): the arithmetic probe on a
|
|
# million cases; every graph vector of wai/ops-conformance that the subset covers,
|
|
# at both widths, against the committed results and the interpreter; the shader
|
|
# mutants, every one of which the corpus must catch; every committed vector and
|
|
# refusal fixture of every capability that runs a wai-ops/1 graph, the jpegai/v1
|
|
# inputs included, against the decode on the CPU; and the corpus again in a
|
|
# headless browser's WebGPU, the kernels compiled by the browser's own WGSL
|
|
# compiler. WAI_OPS_GPU_EXPECT declares the one adapter the host must offer, and
|
|
# WAI_OPS_GPU_REQUIRE makes an absent adapter a failure, never a skip; each step
|
|
# then requires the report lines with their counts, so a filtered or empty run
|
|
# fails too.
|
|
# - checks (`linux`, a container with no GPU): the crate and its tests build, the
|
|
# device tests print SKIPPED and pass, and the same tests under
|
|
# WAI_OPS_GPU_REQUIRE=1 fail: an absent adapter is not a pass. The browser module
|
|
# builds for wasm32-unknown-unknown; the bench tool builds.
|
|
#
|
|
# The equality on other adapters (a discrete and an integrated GPU on Vulkan, an
|
|
# integrated GPU on Vulkan elsewhere, software Vulkan) was run by hand and is recorded in
|
|
# wai/extensions/determinism-tiers.md §6; this workflow runs no job on those hosts.
|
|
|
|
# One run per branch at a time; a push supersedes the run before it.
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
on:
|
|
pull_request:
|
|
paths:
|
|
- 'wai/wai-ops-gpu/**'
|
|
- 'wai/wai-ops/**'
|
|
- 'wai/ops-conformance/**'
|
|
# The decoders whose graphs run on the device, and what they decode.
|
|
- 'wai/wai-rs/Cargo.toml'
|
|
- 'wai/wai-rs/Cargo.lock'
|
|
- 'wai/wai-rs/src/int_*.rs'
|
|
- 'wai/wai-rs/src/int_payload/**'
|
|
- 'wai/wai-rs/src/jpegai_*.rs'
|
|
- 'wai/wai-rs/tests/ops_gpu.rs'
|
|
- 'wai/wai-rs/tests/vectors/**'
|
|
- 'wai/int-payload-conformance/**'
|
|
- 'wai/neural-conformance/**'
|
|
- 'wai/tools/ops-gpu-bench/**'
|
|
- 'wai/tools/wai_jpegai_fetch_e2e_inputs.sh'
|
|
- 'wai/tools/jpegai-e2e-v1.sha256'
|
|
- 'rust-toolchain.toml'
|
|
- '.github/workflows/wai-ops-gpu.yml'
|
|
schedule:
|
|
- cron: '17 6 * * *'
|
|
|
|
jobs:
|
|
# Does this pull request's own diff touch the paths below? This forge matches a
|
|
# workflow's `paths:` against the push, so a merge of main into a branch fires the
|
|
# workflow whatever the pull request changes; every job here waits on this answer.
|
|
# A push to main, a schedule, or anything the script cannot determine answers
|
|
# `true` (scripts/pr-touches-paths.py).
|
|
relevant:
|
|
name: does this change touch it
|
|
runs-on: linux-arm64
|
|
outputs:
|
|
run: ${{ steps.gate.outputs.run }}
|
|
steps:
|
|
- uses: actions/checkout@v5
|
|
with:
|
|
fetch-depth: 0
|
|
- id: gate
|
|
run: |
|
|
set -euo pipefail
|
|
r=$(python3 scripts/pr-touches-paths.py .github/workflows/wai-ops-gpu.yml \
|
|
"${{ github.event.pull_request.base.sha }}" \
|
|
"${{ github.event.pull_request.head.sha }}")
|
|
echo "this change touches wai-ops-gpu: $r"
|
|
echo "run=$r" >> "$GITHUB_OUTPUT"
|
|
|
|
checks:
|
|
needs: relevant
|
|
if: needs.relevant.outputs.run == 'true'
|
|
name: builds · no adapter is not a pass (linux)
|
|
runs-on: linux
|
|
timeout-minutes: 60
|
|
steps:
|
|
- uses: actions/checkout@v5
|
|
- name: Install Rust 1.98.0 with the wasm32-unknown-unknown target
|
|
shell: bash
|
|
run: |
|
|
export PATH="$HOME/.cargo/bin:$PATH"
|
|
command -v rustup >/dev/null 2>&1 || \
|
|
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.98.0
|
|
rustup toolchain install 1.98.0 --profile minimal
|
|
rustup default 1.98.0
|
|
rustup target add --toolchain 1.98.0 wasm32-unknown-unknown
|
|
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
|
|
# This container offers no adapter. The device tests must say so and pass; under
|
|
# WAI_OPS_GPU_REQUIRE=1 the same tests must fail. Were an adapter ever offered
|
|
# here, its results must then be equal instead.
|
|
- name: The crate's tests, and an absent adapter under WAI_OPS_GPU_REQUIRE
|
|
shell: bash
|
|
working-directory: wai/wai-ops-gpu
|
|
run: |
|
|
set -euo pipefail
|
|
log="$RUNNER_TEMP/ops-gpu.log"
|
|
rep="$RUNNER_TEMP/ops-gpu.report"
|
|
: > "$rep"
|
|
WAI_OPS_GPU_REPORT="$rep" cargo test --locked --release -- --nocapture --test-threads=1 2>&1 | tee "$log"
|
|
if grep -q '^ops-gpu corpus: key=' "$rep"; then
|
|
echo "this runner offered an adapter"
|
|
if grep -E '^ops-gpu corpus: key=' "$rep" | grep -vq ' mismatches=0 errors=0 '; then
|
|
echo "::error::an adapter here gave a result that is not the interpreter's"; exit 1
|
|
fi
|
|
else
|
|
grep -q '^ops-gpu corpus: SKIPPED' "$rep" || { echo "::error::the corpus test printed neither a result nor SKIPPED"; exit 1; }
|
|
if WAI_OPS_GPU_REQUIRE=1 cargo test --locked --release --test corpus --test arith --test mutants > "$RUNNER_TEMP/require.log" 2>&1; then
|
|
echo "::error::with no adapter, the device tests passed under WAI_OPS_GPU_REQUIRE=1"; exit 1
|
|
fi
|
|
grep -q 'WAI_OPS_GPU_REQUIRE is set' "$RUNNER_TEMP/require.log" \
|
|
|| { echo "::error::the tests failed under WAI_OPS_GPU_REQUIRE=1, but not for the absent adapter"; cat "$RUNNER_TEMP/require.log"; exit 1; }
|
|
echo "no adapter: SKIPPED without WAI_OPS_GPU_REQUIRE, a failure with it"
|
|
fi
|
|
- name: The browser module builds for wasm32-unknown-unknown
|
|
working-directory: wai/wai-ops-gpu/browser
|
|
run: cargo build --locked --release --target wasm32-unknown-unknown
|
|
- name: wai-rs builds with ops_gpu, and its GPU test target
|
|
working-directory: wai/wai-rs
|
|
run: cargo test --locked --no-default-features --features ops_gpu,video_int,audio_int,splat_int,world_int --test ops_gpu --no-run
|
|
- name: The bench tool builds
|
|
working-directory: wai/tools/ops-gpu-bench
|
|
run: cargo build --locked --release
|
|
|
|
metal:
|
|
needs: relevant
|
|
if: needs.relevant.outputs.run == 'true'
|
|
name: Metal GPU and browser WebGPU (macos)
|
|
runs-on: macos
|
|
timeout-minutes: 90
|
|
env:
|
|
WAI_OPS_GPU_EXPECT: gpu-metal-0
|
|
WAI_OPS_GPU_REQUIRE: '1'
|
|
steps:
|
|
- uses: actions/checkout@v5
|
|
# launchd's PATH omits cargo on the macOS host runner.
|
|
- name: Put cargo on PATH
|
|
shell: bash
|
|
run: echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
|
|
- name: The probe, the corpus at both widths, and the mutants, on the GPU
|
|
shell: bash
|
|
working-directory: wai/wai-ops-gpu
|
|
run: |
|
|
set -euo pipefail
|
|
log="$RUNNER_TEMP/ops-gpu.log"
|
|
export WAI_OPS_GPU_REPORT="$RUNNER_TEMP/ops-gpu.report"
|
|
: > "$WAI_OPS_GPU_REPORT"
|
|
nice -n 10 cargo test --locked --release -- --nocapture --test-threads=1 2>&1 | tee "$log"
|
|
k=gpu-metal-0
|
|
# Six tests in four binaries: the library's one, arith's two, corpus's two and
|
|
# mutants' one, none filtered, ignored or failed.
|
|
ran=$(awk '/^test result: ok\. [0-9]+ passed; 0 failed; 0 ignored; 0 measured; 0 filtered out/ { n += $4 } END { print n + 0 }' "$log")
|
|
[ "$ran" = 6 ] || { echo "::error::$ran tests ran and passed, not 6"; exit 1; }
|
|
! grep -q '^test result: FAILED' "$log" || { echo "::error::a test failed"; exit 1; }
|
|
grep -qxF "ops-gpu adapters: as declared ($k)" "$WAI_OPS_GPU_REPORT" || { echo "::error::the adapters found are not $k"; exit 1; }
|
|
grep -qE "^ops-gpu arith: key=$k .* cases=[0-9]+ words=[0-9]+ equal$" "$WAI_OPS_GPU_REPORT" || { echo "::error::no probe result for $k"; exit 1; }
|
|
grep -qE "^ops-gpu corpus: key=$k .* acc32 vectors=[1-9][0-9]* decodes=[1-9][0-9]*; acc64 vectors=[1-9][0-9]* decodes=[1-9][0-9]*; .* mismatches=0 errors=0 " "$WAI_OPS_GPU_REPORT" \
|
|
|| { echo "::error::no equal corpus run for $k"; exit 1; }
|
|
grep -qxF "ops-gpu mutants: key=$k killed 18 of 18" "$WAI_OPS_GPU_REPORT" || { echo "::error::a mutant survived on $k, or the mutant test did not run"; exit 1; }
|
|
- name: The jpegai/v1 inputs
|
|
shell: bash
|
|
run: bash wai/tools/wai_jpegai_fetch_e2e_inputs.sh "$RUNNER_TEMP/jpegai"
|
|
- name: Every capability that runs a wai-ops/1 graph, with its graphs on the GPU
|
|
shell: bash
|
|
working-directory: wai/wai-rs
|
|
run: |
|
|
set -euo pipefail
|
|
log="$RUNNER_TEMP/ops-gpu-rs.log"
|
|
rep="$RUNNER_TEMP/ops-gpu-rs.report"
|
|
: > "$rep"
|
|
J="$RUNNER_TEMP/jpegai"
|
|
WAI_OPS_GPU_REPORT="$rep" WAI_JPEGAI_CODESTREAM="$J/test256.bin" WAI_JPEGAI_WEIGHTS_DIR="$J" WAI_JPEGAI_NORM="$J/norm.json" WAI_REQUIRE_SOTA_VECTORS=1 \
|
|
nice -n 10 cargo test --locked --release --no-default-features --features ops_gpu,video_int,audio_int,splat_int,world_int \
|
|
--test ops_gpu -- --nocapture 2>&1 | tee "$log"
|
|
grep -q '^test result: ok\. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out' "$log" || { echo "::error::the capability test did not run and pass"; exit 1; }
|
|
for cap in wai.neural.int_hyper wai.video.int_hyper wai.audio.int_codec wai.splat.int_codec wai.neural.int_synth \
|
|
wai.neural.int_mlicpp wai.world.model wai.image.jpegai; do
|
|
grep -qE "^ops-gpu capability: key=gpu-metal-0 .* cap=$cap cases=[1-9][0-9]* .* differ=0 errors=0$" "$rep" \
|
|
|| { echo "::error::$cap: no equal run on the GPU"; exit 1; }
|
|
done
|
|
- name: wasm-bindgen CLI 0.2.127, and the wasm32-unknown-unknown target
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
rustup target add wasm32-unknown-unknown
|
|
cargo install --locked wasm-bindgen-cli --version 0.2.127
|
|
# Playwright and its headless browser go into the job's temporary directory, not
|
|
# the host's caches. `chromium` is Playwright's name for its engine.
|
|
- name: Playwright 1.60.0 and its headless browser
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
pw="$RUNNER_TEMP/playwright"
|
|
mkdir -p "$pw"
|
|
npm install --prefix "$pw" --no-save --no-audit --no-fund [email protected]
|
|
PLAYWRIGHT_BROWSERS_PATH="$pw/browsers" "$pw/node_modules/.bin/playwright" install --only-shell chromium
|
|
echo "PLAYWRIGHT_HOST=$pw/" >> "$GITHUB_ENV"
|
|
echo "PLAYWRIGHT_BROWSERS_PATH=$pw/browsers" >> "$GITHUB_ENV"
|
|
- name: The corpus in a browser's WebGPU
|
|
shell: bash
|
|
working-directory: wai/wai-ops-gpu/browser
|
|
run: |
|
|
set -euo pipefail
|
|
nice -n 10 cargo build --locked --release --target wasm32-unknown-unknown
|
|
t="${CARGO_TARGET_DIR:-target}"
|
|
wasm-bindgen --target web --out-dir pkg --out-name wai_ops_gpu_browser "$t/wasm32-unknown-unknown/release/wai_ops_gpu_browser.wasm"
|
|
log="$RUNNER_TEMP/browser.log"
|
|
nice -n 10 node run.mjs 2>&1 | tee "$log"
|
|
grep -qE '^browser webgpu: key=gpu-webgpu-[0-9a-f]+ .* acc32 vectors=[1-9][0-9]* decodes=[1-9][0-9]*; acc64 vectors=[1-9][0-9]* decodes=[1-9][0-9]*; .* mismatches=0 errors=0$' "$log" \
|
|
|| { echo "::error::no equal corpus run on a GPU adapter in the browser"; exit 1; }
|