6517401f01
Every path-gated workflow begins with one small job -- `does this change touch
it`, or `which workspaces this change touches` for the standards build -- that
all of its real jobs declare in `needs`. All 55 of those gate jobs ran on
`linux-arm64`.
That is the one label the fleet cannot spare. Sampled while 267 runs were
queued, every waiting run had the same shape:
[waiting] linux-arm64 does this change touch it
[blocked] linux member · dataflow device, simulated
[blocked] macos member · dataflow device absent
237 runs, each holding a gate in the longest queue, and every real job blocked
behind it. Not only `linux` jobs: `macos`, `linux-hw`, `linux-arm64-hw` and
`macos-m3-ultra` jobs were blocked too. The measured effect was `linux-arm64`
263 deep against `linux` **zero**, with jetson-hub's 20 cores taking no job for
a full hour -- starved not for want of work but because the work could not be
unblocked. A gate waiting on the resource it is rationing is head-of-line
blocking, and it makes every other runner in the fleet idle on its behalf.
The gates move to `linux`. Nothing else changes: the diff is 55 `runs-on` lines
and no others, and every touched file still parses.
Why `linux` and not elsewhere. The gate is not free -- it compares
`BASE...HEAD`, which needs the merge base, so `fetch-depth: 0` is a full clone
rather than a ten-second script. `macos` is served by hosts on a Mac Studio
sitting at load 185; `linux` is a separate 20-core box that was measurably
empty. This moves a serialisation point to the least contended label; it does
not remove it, and the gate is still a single point of failure for its
workflow. The structural fix is an early-exit first step inside each job, which
costs one checkout per leg and is a larger change.
Follow-up, deliberately not bundled: jetson-hub runs at capacity 2 with 15 GB of
RAM, so 55 clones still serialise two at a time. Raising capacity helps gates
and hurts the heavy emulated-architecture jobs that share the label, so it wants
measuring on its own.
Co-Authored-By: Claude Opus 5 <[email protected]>
392 lines
17 KiB
YAML
392 lines
17 KiB
YAML
name: wai-vector vector graphics
|
|
|
|
# The vector graphics capability (wai/extensions/graphics-vector.md), checked on every
|
|
# fabric leg the corpus has today. Every leg recomputes every expected file of
|
|
# wai/vector-conformance/ from its vector and the committed sets, and requires the
|
|
# committed bytes:
|
|
#
|
|
# - Linux x86_64, Linux arm64 and macOS arm64, natively, debug and release: the unit
|
|
# tests, the refusal-order, canonical, clock, track, geometry, cost, held-set and
|
|
# raster tests, the corpus tests (every frame of the rendering groups recomputed
|
|
# to its SHA-256, and every text record of the a11y group recomputed byte for
|
|
# byte), the in-house sets' tests (the default stroke set's SHA-256 is Annex B's,
|
|
# Annex A's table is the expansion's, and both sets re-derive from their WSF1
|
|
# sources), the text layer's tests (the projection and the transcript of section
|
|
# 5.4 against JSON written out by hand, and the worked example read from the
|
|
# extension), and `wai-vector vectors verify`, which also regenerates the whole
|
|
# corpus and converts every payload's authoring form back to its bytes; and the
|
|
# sink's build, without the feature `author`, so with no stroker;
|
|
# - WASI (wasm32-wasip2) under wasmtime, a 32-bit target: the same tests and the same
|
|
# `vectors verify`;
|
|
# - a browser's engine (wasm32-unknown-unknown) under Node: the recomputation of
|
|
# every expected file and every authoring form, in a sink's build (an a11y
|
|
# payload's expected file states the SHA-256 of its text record, so the projection
|
|
# and the transcript are recomputed here too), and the default stroke set's
|
|
# SHA-256 against Annex B;
|
|
# - three browser engines (chromium, webkit and firefox, as Playwright 1.60.0 names
|
|
# the engines it drives), in that version's container image, pinned by digest. The
|
|
# package a site serves (the sink's exports alone, built for wasm32v1-none and read
|
|
# back as WebAssembly 1.0 by wasm/check-mvp.mjs) loads every payload to its code,
|
|
# states each listed size's cost, and renders the frames through the page's hook:
|
|
# each is hashed by the module and by the platform, and the sampled ones are sent
|
|
# to Node and hashed there. For each a11y payload the sink's transcript and its
|
|
# projection at every frame of every listed rate, written out as the text
|
|
# record's lines, must be the committed record byte for byte. The recomputation
|
|
# module then recomputes every expected file and authoring form in each engine.
|
|
# A missing engine fails the job;
|
|
# - the fuzz leg: the corpus-seeded fuzz of the WOS1 and WVG1 parsers at depth, each
|
|
# accepted payload of small cost rendered and each accepted payload's text layer
|
|
# held (its transcript is JSON, and its projections list the blocks the frame
|
|
# procedure draws), the peak-allocation test and the duplicate-detection
|
|
# comparison count;
|
|
# - the independent checker, tools/wai_vector_check.py: Python standard library only,
|
|
# written from the extension text alone and sharing no code with wai-vector, which
|
|
# recomputes every expected file byte for byte (every frame by the closed-form
|
|
# crossings in big integers), rewrites every payload and set to its own bytes, and
|
|
# recomputes every digest of sha256.json.
|
|
#
|
|
# There is no Windows leg: the Windows runner has been offline since 2026-08-28.
|
|
|
|
# One run per branch at a time. A push supersedes the run before it.
|
|
#
|
|
# There is one `macos` runner and it is the dev Mac, so every macOS job in this repo
|
|
# queues on it; only the newest commit's verdict is wanted, so older runs are
|
|
# cancelled. `cancel-in-progress: true` also applies to `main`, which advances by
|
|
# merge commits whose tree contains the commits before them.
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
on:
|
|
pull_request:
|
|
paths:
|
|
- 'wai/wai-vector/**'
|
|
# SHA-256 is wai-ops's, included by #[path].
|
|
- 'wai/wai-ops/src/sha256.rs'
|
|
- 'wai/vector-conformance/**'
|
|
- 'wai/tools/wai_vector_*.py'
|
|
# The text the crate and the corpus are held to.
|
|
- 'wai/extensions/graphics-vector.md'
|
|
# The licence text the in-house sets carry in their notes.
|
|
- 'LICENSE'
|
|
- '.github/workflows/wai-vector.yml'
|
|
push:
|
|
branches:
|
|
- main
|
|
paths:
|
|
- 'wai/wai-vector/**'
|
|
- 'wai/wai-ops/src/sha256.rs'
|
|
- 'wai/vector-conformance/**'
|
|
- 'wai/tools/wai_vector_*.py'
|
|
- 'wai/extensions/graphics-vector.md'
|
|
- 'LICENSE'
|
|
- '.github/workflows/wai-vector.yml'
|
|
|
|
jobs:
|
|
# Does this pull request's own diff touch the paths below? This forge matches a
|
|
# workflow's `paths:` against the push, so a merge of main into a branch fires the
|
|
# workflow whatever the pull request changes; every job here waits on this answer.
|
|
# A push to main, a schedule, or anything the script cannot determine answers
|
|
# `true` (scripts/pr-touches-paths.py).
|
|
relevant:
|
|
name: does this change touch it
|
|
runs-on: linux
|
|
outputs:
|
|
run: ${{ steps.gate.outputs.run }}
|
|
steps:
|
|
- uses: actions/checkout@v5
|
|
with:
|
|
fetch-depth: 0
|
|
- id: gate
|
|
run: |
|
|
set -euo pipefail
|
|
r=$(python3 scripts/pr-touches-paths.py .github/workflows/wai-vector.yml \
|
|
"${{ github.event.pull_request.base.sha }}" \
|
|
"${{ github.event.pull_request.head.sha }}")
|
|
echo "this change touches wai-vector: $r"
|
|
echo "run=$r" >> "$GITHUB_OUTPUT"
|
|
|
|
native:
|
|
needs: relevant
|
|
if: needs.relevant.outputs.run == 'true'
|
|
name: ${{ matrix.label }}
|
|
runs-on: ${{ matrix.os }}
|
|
timeout-minutes: 40
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
- { os: linux, label: linux-x86_64 }
|
|
- { os: linux-arm64, label: linux-aarch64 }
|
|
- { os: macos, label: macos-arm64 }
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
# The docker images ship no Rust; the macOS host has it, but launchd's PATH omits
|
|
# ~/.cargo/bin.
|
|
- name: Install Rust 1.98.0
|
|
shell: bash
|
|
run: |
|
|
export PATH="$HOME/.cargo/bin:$PATH"
|
|
command -v rustup >/dev/null 2>&1 || \
|
|
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.98.0
|
|
rustup toolchain install 1.98.0 --profile minimal
|
|
rustup default 1.98.0
|
|
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
|
|
|
|
# Debug builds check every arithmetic operation for overflow, so a bound the
|
|
# text states that the code exceeds panics here.
|
|
- name: Every test, debug
|
|
working-directory: wai/wai-vector
|
|
shell: bash
|
|
run: cargo test --locked
|
|
|
|
- name: Every test, release
|
|
working-directory: wai/wai-vector
|
|
shell: bash
|
|
run: cargo test --locked --release
|
|
|
|
- name: The corpus regenerated, every expected file recomputed
|
|
working-directory: wai/wai-vector
|
|
shell: bash
|
|
run: cargo run --locked --release --features std --bin wai-vector -- vectors verify ..
|
|
|
|
# A sink has no stroker (graphics-vector.md §3.14): the library builds without
|
|
# the authoring code.
|
|
- name: The sink's build, without the feature author
|
|
working-directory: wai/wai-vector
|
|
shell: bash
|
|
run: cargo build --locked --no-default-features --lib
|
|
|
|
wasip2:
|
|
needs: relevant
|
|
if: needs.relevant.outputs.run == 'true'
|
|
name: wasm32-wasip2 (wasmtime)
|
|
runs-on: linux
|
|
timeout-minutes: 30
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Install Rust 1.98.0 with the wasm32-wasip2 target
|
|
shell: bash
|
|
run: |
|
|
export PATH="$HOME/.cargo/bin:$PATH"
|
|
command -v rustup >/dev/null 2>&1 || \
|
|
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.98.0
|
|
rustup toolchain install 1.98.0 --profile minimal
|
|
rustup default 1.98.0
|
|
rustup target add wasm32-wasip2
|
|
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
|
|
|
|
- name: Install wasmtime 38.0.3
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
v=38.0.3
|
|
curl --proto '=https' --tlsv1.2 -sSfL -o "$RUNNER_TEMP/wasmtime.tar.xz" \
|
|
"https://github.com/bytecodealliance/wasmtime/releases/download/v$v/wasmtime-v$v-x86_64-linux.tar.xz"
|
|
tar -C "$RUNNER_TEMP" -xf "$RUNNER_TEMP/wasmtime.tar.xz"
|
|
echo "$RUNNER_TEMP/wasmtime-v$v-x86_64-linux" >> "$GITHUB_PATH"
|
|
|
|
# The tests read committed files through CARGO_MANIFEST_DIR/.., so the wai
|
|
# directory is preopened at its own absolute path.
|
|
- name: Every test, under WASI
|
|
working-directory: wai/wai-vector
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
W=$(cd .. && pwd)
|
|
CARGO_TARGET_WASM32_WASIP2_RUNNER="wasmtime run --dir $W::$W" \
|
|
cargo test --locked --release --target wasm32-wasip2
|
|
|
|
- name: Build the tool for wasm32-wasip2
|
|
working-directory: wai/wai-vector
|
|
shell: bash
|
|
run: cargo build --locked --release --features std --target wasm32-wasip2 --bin wai-vector
|
|
|
|
- name: The corpus regenerated, every expected file recomputed, under WASI
|
|
working-directory: wai
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
wasmtime --version
|
|
wasmtime run --dir .::/wai wai-vector/target/wasm32-wasip2/release/wai-vector.wasm vectors verify /wai
|
|
|
|
node:
|
|
needs: relevant
|
|
if: needs.relevant.outputs.run == 'true'
|
|
name: wasm32 under Node
|
|
runs-on: linux
|
|
timeout-minutes: 30
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Install Rust 1.98.0 with the wasm32 target
|
|
shell: bash
|
|
run: |
|
|
export PATH="$HOME/.cargo/bin:$PATH"
|
|
command -v rustup >/dev/null 2>&1 || \
|
|
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.98.0
|
|
rustup toolchain install 1.98.0 --profile minimal
|
|
rustup default 1.98.0
|
|
rustup target add wasm32-unknown-unknown
|
|
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
|
|
|
|
- uses: actions/setup-node@v4
|
|
with:
|
|
node-version: '22'
|
|
|
|
- name: Build the recomputation for wasm32-unknown-unknown
|
|
working-directory: wai/wai-vector/wasm
|
|
shell: bash
|
|
run: cargo build --locked --release --target wasm32-unknown-unknown
|
|
|
|
- name: Every expected file and authoring form recomputed under Node
|
|
working-directory: wai/wai-vector/wasm
|
|
shell: bash
|
|
run: node verify.mjs target/wasm32-unknown-unknown/release/wai_vector_wasm.wasm ../../vector-conformance
|
|
|
|
browser:
|
|
needs: relevant
|
|
if: needs.relevant.outputs.run == 'true'
|
|
name: wasm32 in three browser engines
|
|
runs-on: linux
|
|
timeout-minutes: 45
|
|
# Playwright 1.60.0's own image: Node and the three engines that version drives,
|
|
# already installed. Pinned by the digest of its linux/amd64 manifest (the linux
|
|
# runner is x86_64), so the engines' builds cannot move under the tag.
|
|
container:
|
|
image: mcr.microsoft.com/playwright:v1.60.0-noble@sha256:83192064c7510f7ee73dd63dc5f22a5e01a92c81a2e6a9c715d9e3fe55471fd9
|
|
env:
|
|
# The image runs as root, and an engine that finds HOME owned by another user
|
|
# refuses to start.
|
|
HOME: /root
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
# The image ships no Rust. wasm32v1-none is the package's target: its
|
|
# precompiled core and alloc are WebAssembly 1.0, which those of
|
|
# wasm32-unknown-unknown are not (wasm/build-pkg.sh says why that decides it).
|
|
- name: Install Rust 1.98.0 with the wasm32v1-none and wasm32-unknown-unknown targets
|
|
shell: bash
|
|
run: |
|
|
export PATH="$HOME/.cargo/bin:$PATH"
|
|
command -v rustup >/dev/null 2>&1 || \
|
|
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.98.0
|
|
rustup toolchain install 1.98.0 --profile minimal
|
|
rustup default 1.98.0
|
|
rustup target add wasm32v1-none wasm32-unknown-unknown
|
|
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
|
|
|
|
# The sink's exports alone, built and then read back: every section, type,
|
|
# segment and instruction WebAssembly 1.0's, no imports, no 64-bit integer at an
|
|
# exported function, no floating point, and the exported functions the script
|
|
# lists. The build fails otherwise.
|
|
- name: The package, built for wasm32v1-none and read back as WebAssembly 1.0
|
|
shell: bash
|
|
run: bash wai/wai-vector/wasm/build-pkg.sh
|
|
|
|
- name: Build the recomputation for wasm32-unknown-unknown
|
|
working-directory: wai/wai-vector/wasm
|
|
shell: bash
|
|
run: cargo build --locked --release --target wasm32-unknown-unknown
|
|
|
|
# The check must be able to fail. The recomputation module is the default
|
|
# target's build, whose precompiled core and alloc use bulk memory: the reader
|
|
# must refuse it, and for that reason.
|
|
- name: The reader refuses a module that is not WebAssembly 1.0
|
|
working-directory: wai/wai-vector/wasm
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
if node check-mvp.mjs target/wasm32-unknown-unknown/release/wai_vector_wasm.wasm > "$RUNNER_TEMP/check-default.log" 2>&1; then
|
|
cat "$RUNNER_TEMP/check-default.log"
|
|
echo "::error::check-mvp.mjs passed the default target's build"
|
|
exit 1
|
|
fi
|
|
cat "$RUNNER_TEMP/check-default.log"
|
|
grep -q 'NOT 1.0: bulk-memory (memory.copy)' "$RUNNER_TEMP/check-default.log" \
|
|
|| { echo "::error::check-mvp.mjs did not report bulk memory in the default target's build"; exit 1; }
|
|
|
|
# The image's engines are the ones this Playwright drives; the package comes
|
|
# from the lockfile, with its integrity hash.
|
|
- name: Playwright 1.60.0, from the lockfile
|
|
working-directory: wai/wai-vector/browser
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
node --version
|
|
npm ci --no-audit --no-fund
|
|
|
|
# A missing engine is a failure: run.mjs passes only when all three ran.
|
|
- name: Every payload, frame, text record and vector in chromium, webkit and firefox
|
|
working-directory: wai/wai-vector/browser
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
node run.mjs --engines chromium,webkit,firefox \
|
|
--pkg ../wasm/dist/pkg-vector \
|
|
--recompute ../wasm/target/wasm32-unknown-unknown/release/wai_vector_wasm.wasm \
|
|
../../vector-conformance
|
|
|
|
fuzz:
|
|
needs: relevant
|
|
if: needs.relevant.outputs.run == 'true'
|
|
name: fuzz, peak allocation, comparison count
|
|
runs-on: linux
|
|
timeout-minutes: 30
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Install Rust 1.98.0
|
|
shell: bash
|
|
run: |
|
|
export PATH="$HOME/.cargo/bin:$PATH"
|
|
command -v rustup >/dev/null 2>&1 || \
|
|
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.98.0
|
|
rustup toolchain install 1.98.0 --profile minimal
|
|
rustup default 1.98.0
|
|
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
|
|
|
|
# Every corpus vector and every file under fuzz/seeds/ as a seed, 2,000
|
|
# mutations each: no input panics, and every accepted input writes back to
|
|
# its own bytes.
|
|
- name: Corpus-seeded fuzz at depth
|
|
working-directory: wai/wai-vector
|
|
shell: bash
|
|
env:
|
|
WAI_VECTOR_FUZZ_ITERS: '2000'
|
|
run: cargo test --locked --release --test fuzz_smoke -- --nocapture
|
|
|
|
- name: Peak allocation and the duplicate-detection comparison count
|
|
working-directory: wai/wai-vector
|
|
shell: bash
|
|
run: cargo test --locked --release --test peak_alloc --test canonical -- --nocapture
|
|
|
|
- name: Every workflow declares a concurrency group
|
|
shell: bash
|
|
run: bash scripts/check-workflow-concurrency.sh
|
|
|
|
checker:
|
|
needs: relevant
|
|
if: needs.relevant.outputs.run == 'true'
|
|
name: independent checker (Python)
|
|
runs-on: linux
|
|
timeout-minutes: 30
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- uses: actions/setup-python@v5
|
|
with:
|
|
python-version: '3.11'
|
|
|
|
# Standard library only, so it needs nothing installed. Every frame of every
|
|
# rendering vector is rendered, the clock vector's 3840 x 2160 frames included.
|
|
- name: wai_vector_check.py recomputes every expected file of the corpus
|
|
working-directory: wai
|
|
shell: bash
|
|
run: python3 -I tools/wai_vector_check.py corpus vector-conformance --quiet
|
|
|
|
- name: Every workflow declares a concurrency group
|
|
shell: bash
|
|
run: bash scripts/check-workflow-concurrency.sh
|