d3017c90e6
The check read the workflow's line that runs the quickstart and nothing of how
the scripts reach their own exit status, so a release whose quickstart failure
was swallowed inside the script passed it. Six such faults, planted, each gave
no finding: clean-machine.sh's last line `[ -z "$failed" ]` removed, or given
`|| true`; step() no longer setting `failed`; `says_usage wai-pack 2 || true`;
clean-machine.ps1's `if ($Failed) { exit 1 }` removed; and the macos step's
cross-built test inverted from `!=` to `=`, which skips the host's own archive
with exit 0.
Now: clean-machine.sh ends with `[ -z "$failed" ]`, its step() sets `failed`
in the branch where the command failed, nothing else sets `failed` but its
declaration, and it sets no trap. clean-machine.ps1 ends with
`if ($Failed) { exit 1 }` and `exit 0`, its Step sets `$script:Failed` when
its block throws or its program exits non-zero, and nothing else sets it. Each
contents step fails when a binary it runs does: in the script, every such line
but the last ends in `|| return 1` and the last has nothing after it; in the
twin, a throw follows each run. A line that ends the step before it runs the
script is allowed in the macos job alone, inside the `then` of
`if [ "${{ matrix.triple }}" != <the host's triple> ]`, and nowhere else.
The selftest plants each: 17 new cases, each of which passed the old check and
fails this one (49 cases in all, every one held). The header and the
repo-checks comment now say what the check reads instead of that the run can
"neither be skipped nor fail".
Changelog: none
Co-Authored-By: Claude Opus 5.5 <[email protected]>
275 lines
15 KiB
YAML
275 lines
15 KiB
YAML
# Repository checks: the script checks every change runs, in one job.
|
|
#
|
|
# Each of these is a script that reads the tree for a few seconds. They were six
|
|
# workflows and nine jobs (lockfiles, tests-in-ci, workflow-hygiene, wai-section-refs,
|
|
# spec-refs and no-vanity-metrics' four), and each job paid for a container and a
|
|
# checkout to run them. On 2026-10-14 the light checks were about half of the 119 jobs
|
|
# waiting for the linux-arm64 runner, queued behind the long standards-build legs, and
|
|
# spec-refs held a place on the one x86_64 runner. One job runs them all now, one step
|
|
# a check: every step runs even after an earlier one fails (`!cancelled()`), so a red
|
|
# job names each failing check in its own step, and the job fails if any did.
|
|
#
|
|
# The two path filters the old workflows had (public text on Markdown and wai/ sources,
|
|
# section references on the WAI documents) are gone: the checks take seconds, and a
|
|
# filter that is wrong skips a check silently.
|
|
name: repository checks
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
pull_request:
|
|
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
jobs:
|
|
checks:
|
|
name: lockfiles, advisories, tests in CI, workflow hygiene, spec and section references, public text
|
|
runs-on: linux-arm64
|
|
timeout-minutes: 30
|
|
steps:
|
|
# The whole history: the changelog check diffs a pull request against its base and
|
|
# reads, from the base's first-parent history, when the check reached main
|
|
# (scripts/check-changelog-fragment.py), and it fails rather than pass unchecked when
|
|
# it cannot.
|
|
- uses: actions/checkout@v5
|
|
with:
|
|
fetch-depth: 0
|
|
|
|
- uses: actions/setup-python@v5
|
|
with:
|
|
python-version: '3.11'
|
|
|
|
# check-lockfiles.sh runs `cargo` against every lockfile.
|
|
- name: Install Rust 1.99.1
|
|
shell: bash
|
|
run: |
|
|
export PATH="$HOME/.cargo/bin:$PATH"
|
|
command -v rustup >/dev/null 2>&1 || \
|
|
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.99.1
|
|
rustup toolchain install 1.99.1 --profile minimal
|
|
rustup default 1.99.1
|
|
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
|
|
|
|
# ── Every Cargo.lock is current ──────────────────────────────────────────
|
|
- name: "lockfiles: the check fails closed (self-test)"
|
|
if: ${{ !cancelled() }}
|
|
run: ./scripts/check-lockfiles-selftest.sh
|
|
- name: "lockfiles: every Cargo.lock is current"
|
|
if: ${{ !cancelled() }}
|
|
run: ./scripts/check-lockfiles.sh
|
|
|
|
# ── No workspace carries an advisory nobody has read ─────────────────────
|
|
# cargo-audit reads the lockfile and the advisory database; it builds
|
|
# nothing. The runner is persistent, so this installs once.
|
|
- name: Install cargo-audit
|
|
if: ${{ !cancelled() }}
|
|
shell: bash
|
|
run: command -v cargo-audit >/dev/null || cargo install cargo-audit --locked
|
|
- name: "advisories: the check fails closed (self-test)"
|
|
if: ${{ !cancelled() }}
|
|
run: ./scripts/check-advisories-selftest.sh
|
|
- name: "advisories: every advisory is fixed or allowed with a reason"
|
|
if: ${{ !cancelled() }}
|
|
run: python3 -I scripts/check-advisories.py
|
|
|
|
# ── Every test file is run by a job or listed with its reason ────────────
|
|
- name: "tests in CI: the check fails closed (self-test)"
|
|
if: ${{ !cancelled() }}
|
|
run: ./scripts/check-tests-in-ci-selftest.sh
|
|
- name: "tests in CI: every test file is run or listed, and none can run empty"
|
|
if: ${{ !cancelled() }}
|
|
run: python3 -I scripts/check-tests-in-ci.py
|
|
|
|
# ── Every workflow supersedes its own queue; the path gate fails open ───
|
|
- name: "workflow hygiene: the check fails closed (self-test)"
|
|
if: ${{ !cancelled() }}
|
|
run: ./scripts/check-workflow-concurrency-selftest.sh
|
|
- name: "workflow hygiene: every workflow is grouped per ref"
|
|
if: ${{ !cancelled() }}
|
|
run: ./scripts/check-workflow-concurrency.sh
|
|
- name: "workflow hygiene: the path gate fails open (self-test)"
|
|
if: ${{ !cancelled() }}
|
|
run: ./scripts/pr-touches-paths-selftest.sh
|
|
|
|
# ── One pinned wasmtime, run with both determinism settings ─────────────
|
|
- name: "workflow hygiene: the wasmtime pin check fails closed (self-test)"
|
|
if: ${{ !cancelled() }}
|
|
run: bash scripts/check-wasmtime-pin-selftest.sh
|
|
- name: "workflow hygiene: every wasmtime is the pinned release, run with both determinism settings"
|
|
if: ${{ !cancelled() }}
|
|
run: python3 -I scripts/check-wasmtime-pin.py
|
|
# ── The release runs the quickstart from each archive it attaches ───────
|
|
# wai-release.yml runs only on a tag, so a change that stops it running the
|
|
# quickstart, or lets it fail, would first be seen at a release. This reads the
|
|
# workflow and the two scripts on every change, for the cost of a few lines of text:
|
|
# it refuses the ways of skipping the run or passing a failed one that its header
|
|
# lists, each planted by the self-test, and reads the text, not what it does.
|
|
- name: "release quickstart: the check fails closed (self-test)"
|
|
if: ${{ !cancelled() }}
|
|
run: bash scripts/check-wai-release-quickstart-selftest.sh
|
|
- name: "release quickstart: each archive runs it before it is attached, and the twin keeps its steps"
|
|
if: ${{ !cancelled() }}
|
|
run: python3 -I scripts/check-wai-release-quickstart.py
|
|
|
|
# ── Code cites no section its SPEC lacks; WAI's cited sections exist ────
|
|
- name: "spec references: the check can fail (self-test)"
|
|
if: ${{ !cancelled() }}
|
|
run: bash scripts/check-spec-refs-selftest.sh
|
|
- name: "spec references: every SPEC citation resolves"
|
|
if: ${{ !cancelled() }}
|
|
run: python3 scripts/check-spec-refs.py
|
|
- name: "section references: every section WAI's documents cite exists"
|
|
if: ${{ !cancelled() }}
|
|
run: python scripts/check-wai-section-refs.py
|
|
|
|
# ── JCP's published replay log reads the same to a second reader ────────
|
|
- name: "jcp replay log: a second reader restores the published ledger and refuses every planted fault"
|
|
if: ${{ !cancelled() }}
|
|
run: python3 -I jcp/conformance/verify_replay.py --selftest
|
|
|
|
# ── Every wai.quantum string is registered, listed or a schema's name ────
|
|
# The registry export (wai/registry/capabilities.json, generated from
|
|
# wai/wai-rs/src/codecs/registry.rs) and the quantum crates' schema files are the
|
|
# lists; this reads them and every text file, and names the file and line of any
|
|
# string neither accounts for (wai/REGISTRY.md).
|
|
- name: "quantum names: the check fails closed (self-test)"
|
|
if: ${{ !cancelled() }}
|
|
run: bash scripts/check-wai-quantum-names-selftest.sh
|
|
- name: "quantum names: every wai.quantum string is registered, listed or a schema's name"
|
|
if: ${{ !cancelled() }}
|
|
run: python3 -I scripts/check-wai-quantum-names.py
|
|
|
|
# ── The lists wai-conformance.yml derives hold the hand lists they replaced ─
|
|
# Its feature lists, test lists, verifier steps, corpora and gate paths are read from
|
|
# the files that own them (.github/wai-conformance/, the tags of wai/wai-rs/tests/).
|
|
- name: "workflow lists: the derivation fails closed (self-test)"
|
|
if: ${{ !cancelled() }}
|
|
run: bash scripts/wai-conformance-lists-selftest.sh
|
|
- name: "workflow lists: every derived list holds every entry of the hand list it replaced"
|
|
if: ${{ !cancelled() }}
|
|
run: python3 -I scripts/wai-conformance-lists.py selftest
|
|
|
|
# ── A change to WAI carries its changelog entry, in a file of its own ──────
|
|
# wai/changes/README.md states the form; the entries go into wai/CHANGELOG.md at a
|
|
# release (scripts/changelog-assemble.py). A pull request opened before the check
|
|
# reached main passes as it is; the check reads when it was opened from the event
|
|
# payload, and when the check reached main from the base's first-parent history.
|
|
- name: "changelog: the check and the assembler fail closed (self-test)"
|
|
if: ${{ !cancelled() }}
|
|
run: bash scripts/changelog-fragments-selftest.sh
|
|
- name: "changelog: every entry file is one entry in the changelog's form"
|
|
if: ${{ !cancelled() }}
|
|
run: python3 -I scripts/changelog-assemble.py --check
|
|
- name: "changelog: a change to wai/ adds its entry (pull request)"
|
|
if: ${{ !cancelled() && github.event_name == 'pull_request' }}
|
|
run: |
|
|
git config --global --add safe.directory "$GITHUB_WORKSPACE"
|
|
python3 -I scripts/check-changelog-fragment.py \
|
|
"${{ github.event.pull_request.base.sha }}" "${{ github.event.pull_request.head.sha }}"
|
|
|
|
# ── A rule committed before the runs it judges stays as it first reached main ──
|
|
# A research record's rule (wai/research/<record>/*-rule*.json) is worth what it is
|
|
# because it was fixed before the data, and a rule changed together with every copy of
|
|
# its digest passes every check that reads the tree alone. This reads the first-parent
|
|
# history from the start, so a cancelled run hides nothing: the next run finds the same
|
|
# change. The step below runs scripts/check-frozen-rules.sh as it first reached the
|
|
# history and as it stands, and that script runs scripts/check-frozen-rules.py the same
|
|
# way, so an edit of either script hides no change to a rule. The step itself is read
|
|
# from the commit under test: a commit that edits it silences this check from then on,
|
|
# until that edit is undone, and the diff in the history records it. A pull request is
|
|
# also compared with its base. The self-test runs the step's text from its marker on.
|
|
- name: "frozen rules: the check fails closed (self-test)"
|
|
if: ${{ !cancelled() }}
|
|
run: bash scripts/check-frozen-rules-selftest.sh
|
|
- name: "frozen rules: every rule file is as it first reached main"
|
|
if: ${{ !cancelled() }}
|
|
run: |
|
|
set -euo pipefail
|
|
git config --global --add safe.directory "$GITHUB_WORKSPACE"
|
|
if [ "${{ github.event_name }}" = pull_request ]; then
|
|
set -- "${{ github.event.pull_request.head.sha }}" "${{ github.event.pull_request.base.sha }}"
|
|
else
|
|
set -- "${{ github.sha }}"
|
|
fi
|
|
# frozen rules: the runner as it first reached the history, then as it stands.
|
|
runner=scripts/check-frozen-rules.sh
|
|
first_add() { git log --first-parent -m --diff-filter=A --format=%H "$1" -- "$runner" | tail -n 1; }
|
|
first=$(first_add "$1" || true)
|
|
if [ -z "$first" ] && [ $# -eq 2 ]; then first=$(first_add "$2" || true); fi
|
|
if [ -z "$first" ]; then
|
|
echo "::error::frozen rules: no commit of the first-parent history of $* adds $runner"
|
|
exit 1
|
|
fi
|
|
copy=$(mktemp)
|
|
git show "$first:$runner" > "$copy"
|
|
rc=0
|
|
echo "frozen rules: the runner as it first reached this history, in $first"
|
|
bash "$copy" "$@" || rc=1
|
|
echo "frozen rules: the runner as this checkout holds it"
|
|
bash "$runner" "$@" || rc=1
|
|
rm -f "$copy"
|
|
exit $rc
|
|
|
|
# ── Public text: no vanity metrics, no comparison, no company names ─────
|
|
- name: "public text: no vanity metrics in committed Markdown"
|
|
if: ${{ !cancelled() }}
|
|
run: bash scripts/check-no-vanity-metrics.sh
|
|
- name: "public text: the two public-text checks fail closed (self-test)"
|
|
if: ${{ !cancelled() }}
|
|
run: bash scripts/test-public-text-checks.sh
|
|
- name: "public text: WAI's public text names no comparison"
|
|
if: ${{ !cancelled() }}
|
|
run: bash scripts/check-comparative-prose.sh
|
|
# The maintainers' product names are public, held in the script itself: this run
|
|
# reads no secret, so it checks every change, a pull request included.
|
|
- name: "public text: WAI's public text names no product of its maintainers"
|
|
if: ${{ !cancelled() }}
|
|
run: bash scripts/check-public-names.sh --maintainer-products-only
|
|
# NOTICE states WAI's licence from the repository's root, outside the check's default
|
|
# scope, so it is given by name. The rest of its wording is reviewed by hand.
|
|
- name: "public text: the root NOTICE names no product of WAI's maintainers"
|
|
if: ${{ !cancelled() }}
|
|
run: bash scripts/check-public-names.sh --maintainer-products-only NOTICE
|
|
# The pattern list is a secret, never given to pull-request code: the check runs
|
|
# on the push to main.
|
|
- name: "public text: WAI's public text names no company or product (push to main)"
|
|
if: ${{ !cancelled() && github.event_name != 'pull_request' }}
|
|
env:
|
|
WAI_NAME_DENYLIST: ${{ secrets.WAI_NAME_DENYLIST }}
|
|
run: |
|
|
if [ -z "$WAI_NAME_DENYLIST" ]; then
|
|
echo "::error::WAI_NAME_DENYLIST is not set: the public-name check cannot run"
|
|
exit 1
|
|
fi
|
|
bash scripts/check-public-names.sh
|
|
bash scripts/check-public-names.sh NOTICE
|
|
|
|
# ── Every vector of a corpus that keeps licence records has one ──────────
|
|
- name: "corpus licences: the check fails closed (self-test)"
|
|
if: ${{ !cancelled() }}
|
|
run: bash scripts/check-wai-corpus-licences-selftest.sh
|
|
- name: "corpus licences: every vector has a record and a licence that permits redistribution"
|
|
if: ${{ !cancelled() }}
|
|
run: python3 -I scripts/check-wai-corpus-licences.py
|
|
|
|
# ── No library source file is unreachable from its crate root ───────────
|
|
# An orphan `.rs` (one no `mod`, `#[path]` or `include!` names) is never compiled,
|
|
# so nothing else in this repository can see it. Both directions, before trusting
|
|
# the result: a planted orphan must fail the check, and removing it must pass.
|
|
- name: "orphan sources: the check fails closed (self-test)"
|
|
if: ${{ !cancelled() }}
|
|
run: |
|
|
set -euo pipefail
|
|
probe=jcp/jcp-rs/crates/jcp-efficiency/src/_ci_orphan_probe.rs
|
|
printf 'pub fn never_compiled() -> u64 { 0 }\n' > "$probe"
|
|
if python3 scripts/check-orphan-sources.py >/dev/null 2>&1; then
|
|
rm -f "$probe"
|
|
echo "::error::the orphan check passed with a planted orphan present"
|
|
exit 1
|
|
fi
|
|
rm -f "$probe"
|
|
- name: "orphan sources: no new orphan source files"
|
|
if: ${{ !cancelled() }}
|
|
run: python3 scripts/check-orphan-sources.py
|