d7425f89a2
- ops-npu-bench: time and energy of the corpus and of an int_synth graph on the device and on the CPU, every device result compared with the interpreter's first; the package's energy counter (a vendor counter) and the device runtime's own estimate (a model), each named as what it is. - wai-ops-npu.yml: on the linux-amd runner, the probes, the corpus, the twelve kernel edits and the width test on the device, with the device declared and required; on linux-arm64, the crate builds and an absent device is SKIPPED without WAI_OPS_NPU_REQUIRE and a failure with it. - The corpus test requires the vectors it runs to be exactly those npu.jsonl lists in the subset. Co-Authored-By: Claude Opus 5.5 <[email protected]>
182 lines
9.9 KiB
YAML
182 lines
9.9 KiB
YAML
name: wai-ops-npu integer operators on neural processors
|
|
|
|
# wai-ops/1-npu on neural processors (wai/wai-ops-npu): the integer operator set's
|
|
# accelerator subset (extensions/integer-operators.md §9.2, §9.5) with its arithmetic
|
|
# on one compute core of a tile array. The crate builds the core's program with the
|
|
# array's open toolchain (Apache-2.0 with the LLVM exception, pinned below) and its
|
|
# host side with the system's C++ compiler against the device runtime, both at run
|
|
# time, into a cache keyed by their sources and the toolchain.
|
|
#
|
|
# - npu-hw (`linux-amd`, framework-128gb, its neural processor): the probes (the
|
|
# unit's rounding modes at every shift, its saturation, its accumulator lanes,
|
|
# and the kernel's requantisation on 786,432 cases); every graph vector of
|
|
# wai/ops-conformance whose graph is in the subset, against the committed results
|
|
# and the interpreter; the twelve kernel edits, each its own program, every one of
|
|
# which the corpus must catch; and the graphs the certificate keeps out of the
|
|
# subset by acc32 alone, which must not all agree. WAI_OPS_NPU_EXPECT declares the
|
|
# one device the host must have, and WAI_OPS_NPU_REQUIRE makes an absent device a
|
|
# failure, never a skip; the step then requires the report lines with their
|
|
# counts, so a filtered or empty run fails too.
|
|
# - checks (`linux-arm64`, no such device): the crate and its tests build, its unit
|
|
# tests pass, the device tests print SKIPPED and pass, and the same tests under
|
|
# WAI_OPS_NPU_REQUIRE=1 fail: an absent device is not a pass.
|
|
#
|
|
# Time and energy (`ops-npu-bench`) are measured by hand and recorded in
|
|
# integer-operators §9.5; this workflow does not run the bench.
|
|
|
|
# One run per branch at a time; a push supersedes the run before it.
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
on:
|
|
pull_request:
|
|
paths:
|
|
- 'wai/wai-ops-npu/**'
|
|
- 'wai/wai-ops/**'
|
|
- 'wai/ops-conformance/**'
|
|
- 'rust-toolchain.toml'
|
|
- '.github/workflows/wai-ops-npu.yml'
|
|
schedule:
|
|
- cron: '41 7 * * *'
|
|
|
|
env:
|
|
# The tile array's toolchain: the compiler and its core back end, the build the
|
|
# compiler's release pins (its utils/peano-requirements.txt). The same environment
|
|
# as the wai-quantum npu-cores member's, at the same path.
|
|
TOOLCHAIN_VERSION: 1.4.3
|
|
CORE_COMPILER: llvm-aie==22.0.0.2026090701+3e93bf7b
|
|
|
|
jobs:
|
|
# Does this pull request's own diff touch the paths below? This forge matches a
|
|
# workflow's `paths:` against the push, so a merge of main into a branch fires the
|
|
# workflow whatever the pull request changes; every job here waits on this answer.
|
|
# A push to main, a schedule, or anything the script cannot determine answers
|
|
# `true` (scripts/pr-touches-paths.py).
|
|
relevant:
|
|
name: does this change touch it
|
|
runs-on: linux-arm64
|
|
outputs:
|
|
run: ${{ steps.gate.outputs.run }}
|
|
steps:
|
|
- uses: actions/checkout@v5
|
|
with:
|
|
fetch-depth: 0
|
|
- id: gate
|
|
run: |
|
|
set -euo pipefail
|
|
r=$(python3 scripts/pr-touches-paths.py .github/workflows/wai-ops-npu.yml \
|
|
"${{ github.event.pull_request.base.sha }}" \
|
|
"${{ github.event.pull_request.head.sha }}")
|
|
echo "this change touches wai-ops-npu: $r"
|
|
echo "run=$r" >> "$GITHUB_OUTPUT"
|
|
|
|
npu-hw:
|
|
name: neural processor (framework-128gb silicon)
|
|
needs: relevant
|
|
if: needs.relevant.outputs.run == 'true'
|
|
runs-on: linux-amd
|
|
timeout-minutes: 90
|
|
env:
|
|
CARGO_PROFILE_DEV_DEBUG: '0'
|
|
CARGO_BUILD_JOBS: '12'
|
|
steps:
|
|
- name: Fetch the source
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
want="${{ github.event.pull_request.head.sha || github.sha }}"
|
|
rm -rf src && mkdir src && cd src
|
|
git init -q
|
|
# Fetch the commit itself, not the ref. The host names its own route to the
|
|
# forge (WAI_FORGE_URL).
|
|
git fetch -q --depth 1 "${WAI_FORGE_URL:-http://100.64.224.0:3400}/Transaction-Science/open-standards.git" "$want"
|
|
git checkout -q FETCH_HEAD
|
|
got="$(git rev-parse HEAD)"
|
|
[ "$got" = "$want" ] || { echo "::error::fetched $got, not $want"; exit 1; }
|
|
- name: The toolchain (cached)
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
venv="$HOME/aie-$TOOLCHAIN_VERSION"
|
|
if ! "$venv/bin/python" -I -c 'import aie.iron' 2>/dev/null; then
|
|
rm -rf "$venv"
|
|
python3 -m venv "$venv"
|
|
tag="$(python3 -c 'import sys; print(f"cp{sys.version_info.major}{sys.version_info.minor}")')"
|
|
"$venv/bin/pip" install -q "https://github.com/Xilinx/mlir-aie/releases/download/v$TOOLCHAIN_VERSION/mlir_aie-$TOOLCHAIN_VERSION-$tag-$tag-manylinux_2_35_x86_64.whl" # allow-name: the toolchain's release URL
|
|
"$venv/bin/pip" install -q -f https://github.com/Xilinx/llvm-aie/releases/expanded_assets/nightly "$CORE_COMPILER" # allow-name: the core compiler's release index
|
|
"$venv/bin/pip" install -q "numpy>=2.5.2,<3.0" rich ml_dtypes cloudpickle aiofiles
|
|
"$venv/bin/pip" install -q -f https://llvm.github.io/eudsl "eudsl-python-extras==0.1.0.20260801.905+68a0d7a" --config-settings=EUDSL_PYTHON_EXTRAS_HOST_PACKAGE_PREFIX=aie
|
|
fi
|
|
echo "WAI_OPS_NPU_PYTHON=$venv/bin/python" >> "$GITHUB_ENV"
|
|
- name: The probes, the corpus, the kernel edits and the width, on the device
|
|
shell: bash
|
|
working-directory: src/wai/wai-ops-npu
|
|
run: |
|
|
set -euo pipefail
|
|
export PATH="$HOME/.cargo/bin:$PATH"
|
|
export WAI_OPS_NPU_CACHE="$HOME/ops-npu-cache"
|
|
# The device this host has, declared here (or by the runner).
|
|
k="${WAI_OPS_NPU_HOST_EXPECT:-npu-1022-17f0}"
|
|
log="$RUNNER_TEMP/ops-npu.log"
|
|
export WAI_OPS_NPU_REPORT="$RUNNER_TEMP/ops-npu.report"
|
|
: > "$WAI_OPS_NPU_REPORT"
|
|
WAI_OPS_NPU_EXPECT="$k" WAI_OPS_NPU_REQUIRE=1 CARGO_TARGET_DIR="$HOME/hw-target/wai-ops-npu" \
|
|
nice -n 10 cargo test --locked --release -- --nocapture --test-threads=1 2>&1 | tee "$log"
|
|
# Fourteen tests in five binaries: the library's eight, corpus's one, probe's
|
|
# three, mutants' one and width's one, none filtered, ignored or failed.
|
|
ran=$(awk '/^test result: ok\. [0-9]+ passed; 0 failed; 0 ignored; 0 measured; 0 filtered out/ { n += $4 } END { print n + 0 }' "$log")
|
|
[ "$ran" = 14 ] || { echo "::error::$ran tests ran and passed, not 14"; exit 1; }
|
|
! grep -q '^test result: FAILED' "$log" || { echo "::error::a test failed"; exit 1; }
|
|
rep="$WAI_OPS_NPU_REPORT"
|
|
grep -qxF "ops-npu devices: as declared ($k)" "$rep" || { echo "::error::the device found is not $k"; exit 1; }
|
|
grep -qE "^ops-npu requantisation: key=$k [0-9]+ cases over shifts 0\.\.63 .*, 0 differ$" "$rep" || { echo "::error::the requantisation probe did not agree"; exit 1; }
|
|
grep -qE "^ops-npu corpus: key=$k .* vectors=[1-9][0-9]* decodes=[1-9][0-9]* .* declined=0 mismatches=0 errors=0 " "$rep" \
|
|
|| { echo "::error::no equal corpus run on $k"; exit 1; }
|
|
grep -qxF "ops-npu mutants: key=$k killed 12 of 12" "$rep" || { echo "::error::a kernel edit survived on $k, or the mutant test did not run"; exit 1; }
|
|
grep -qE "^ops-npu width: key=$k .* on the device [1-9][0-9]* differ " "$rep" || { echo "::error::the width test did not run"; exit 1; }
|
|
cat "$rep"
|
|
|
|
checks:
|
|
name: builds · no device is not a pass (linux-arm64)
|
|
needs: relevant
|
|
if: needs.relevant.outputs.run == 'true'
|
|
runs-on: linux-arm64
|
|
timeout-minutes: 60
|
|
steps:
|
|
- uses: actions/checkout@v5
|
|
- name: Install Rust 1.98.0
|
|
shell: bash
|
|
run: |
|
|
export PATH="$HOME/.cargo/bin:$PATH"
|
|
command -v rustup >/dev/null 2>&1 || \
|
|
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.98.0
|
|
rustup toolchain install 1.98.0 --profile minimal
|
|
rustup default 1.98.0
|
|
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
|
|
# This host has no such device. The device tests must say so and pass; under
|
|
# WAI_OPS_NPU_REQUIRE=1 the same tests must fail.
|
|
- name: The crate's tests, and an absent device under WAI_OPS_NPU_REQUIRE
|
|
shell: bash
|
|
working-directory: wai/wai-ops-npu
|
|
env:
|
|
WAI_OPS_NPU_EXPECT: none
|
|
run: |
|
|
set -euo pipefail
|
|
log="$RUNNER_TEMP/ops-npu.log"
|
|
rep="$RUNNER_TEMP/ops-npu.report"
|
|
: > "$rep"
|
|
WAI_OPS_NPU_REPORT="$rep" cargo test --locked --release -- --nocapture --test-threads=1 2>&1 | tee "$log"
|
|
ran=$(awk '/^test result: ok\. [0-9]+ passed; 0 failed; 0 ignored; 0 measured; 0 filtered out/ { n += $4 } END { print n + 0 }' "$log")
|
|
[ "$ran" = 14 ] || { echo "::error::$ran tests ran and passed, not 14"; exit 1; }
|
|
grep -qxF 'ops-npu devices: as declared (none)' "$rep" || { echo "::error::the absent case was not established"; exit 1; }
|
|
for t in corpus mutants width 'requantisation probe' 'rounding probe' 'lane probe'; do
|
|
grep -qF "ops-npu $t: SKIPPED" "$rep" || { echo "::error::the $t did not say SKIPPED"; exit 1; }
|
|
done
|
|
grep -qE '^ops-npu subset: vectors=[1-9][0-9]* .* in_subset=[1-9][0-9]* ' "$rep" || { echo "::error::the corpus was not read"; exit 1; }
|
|
if WAI_OPS_NPU_REQUIRE=1 cargo test --locked --release --test corpus --test probe --test mutants --test width > "$RUNNER_TEMP/require.log" 2>&1; then
|
|
echo "::error::with no device, the device tests passed under WAI_OPS_NPU_REQUIRE=1"; exit 1
|
|
fi
|
|
grep -q 'WAI_OPS_NPU_REQUIRE is set' "$RUNNER_TEMP/require.log" \
|
|
|| { echo "::error::the tests failed under WAI_OPS_NPU_REQUIRE=1, but not for the absent device"; cat "$RUNNER_TEMP/require.log"; exit 1; }
|
|
echo "no device: SKIPPED without WAI_OPS_NPU_REQUIRE, a failure with it"
|