open-standards/scripts/check-spec-refs-selftest.sh
David Charlot 4b05fd80e1
All checks were successful
standards workspaces build + test / jouleclaw-rs (pull_request) Successful in 2m15s
standards workspaces build + test / joule-code-rs (pull_request) Successful in 1m22s
standards workspaces build + test / aac-rs (pull_request) Successful in 2m31s
standards workspaces build + test / joulecontract-rs (pull_request) Successful in 54s
standards workspaces build + test / map-rs (pull_request) Successful in 55s
standards workspaces build + test / mesh-rs (pull_request) Successful in 55s
standards workspaces build + test / jouledesk-rs (pull_request) Successful in 56s
standards workspaces build + test / openpay (pull_request) Successful in 3m4s
standards workspaces build + test / macOS energy backends (pull_request) Successful in 4m36s
lockfiles / every Cargo.lock is current (pull_request) Successful in 40s
spec citations resolve / Code cites no section its SPEC lacks (pull_request) Successful in 4s
standards workspaces build + test / arl-rs (pull_request) Successful in 1m10s
standards workspaces build + test / eoc-rs (pull_request) Successful in 2m0s
standards workspaces build + test / joule-ui-rs (pull_request) Successful in 1m12s
standards workspaces build + test / jcp-rs (pull_request) Successful in 1m31s
standards workspaces build + test / joulehook-rs (pull_request) Successful in 1m11s
standards workspaces build + test / proof-rs (pull_request) Successful in 1m19s
standards workspaces build + test / sense-rs (pull_request) Successful in 1m12s
standards workspaces build + test / sandbox-rs (pull_request) Successful in 2m22s
workflow hygiene / every workflow supersedes its own queue (pull_request) Successful in 4s
standards workspaces build + test / smart-byte-rs (pull_request) Successful in 3m46s
ci: a citation to a section that does not exist is now a failure
The commit before this fixed five of them. They were written long before
anyone read them, and the one in #190 — `jcp-runtime` citing `SPEC §4.10` in
six places, with no §4.10 — survived a gate-enforcing, tested mechanism for
months. Reading the tree once finds these; reading it every time is a check.

`scripts/check-spec-refs.py` resolves every `SPEC §N` in a standard's `*.rs`
against the numbered headings of the `SPEC.md` beside it. 1454 citations
across 15 standards resolve today.

Three decisions worth stating, because each is a way the check could have been
written to find nothing:

  - Parents are not synthesised. A cited `§4` needs its own `§4` heading; a
    `§4.10` heading does not satisfy it. Otherwise a standard citing a section
    it never wrote is excused by a child it happens to have.
  - A citation resolves against the SPEC.md of its own directory. Every
    cross-standard citation in the tree today is a self-reference that names
    its standard for the reader ("JouleClaw SPEC §1.3", inside jouleclaw/), so
    this is right for all of them; the header says what to do instead if that
    ever stops being true, rather than loosening the check.
  - Sections of a `spec/*.md` subfile are not sections of `SPEC.md`.

The self-test asserts the check can fail, in the four ways that matter: an
unresolvable citation fails, both heading spellings (`## 4.` and `## §4`) are
accepted, a child heading does not satisfy its parent, and one standard's
section does not excuse another's citation. A regex that quietly stops matching
would otherwise find nothing and exit 0 — the failure mode this repo has
already met in gated and skipped tests.

The workflow carries no `paths:` filter, deliberately. The check is a second of
Python with no toolchain, and a correct filter would have to name every
standard's `**/*.rs` and every `SPEC.md`; `standards-build.yml` records that
`**`-prefixed globs do not match on this forge and that a gate which quietly
fails to fire is worse than none, since it reports nothing and that reads as
green.

This is `check-wai-section-refs.py`'s complement, not a duplicate: that one
checks WAI markdown against WAI markdown, this one checks every standard's
code against its spec.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-10-13 03:07:23 -04:00

70 lines
2.5 KiB
Shell
Executable file

#!/usr/bin/env bash
#
# check-spec-refs-selftest.sh — check-spec-refs.py can fail.
#
# A check that cannot fail is decoration. This builds scratch standards under a
# temporary root and asserts that the check
# 1. passes when every cited section is a heading;
# 2. fails when a cited section is not a heading (the case it exists for);
# 3. accepts both heading spellings, `## 4.` and `## §4`;
# 4. does NOT synthesise parents: a `## 4.10` heading alone must not satisfy a
# `SPEC §4` citation, which is how a standard citing a section it never
# wrote would otherwise pass;
# 5. resolves a citation against the SPEC.md of the standard the file sits in,
# so one standard's section cannot excuse another's citation;
# 6. ignores `*.rs` under a `target/` directory, which is build output.
# Nothing here reads the real tree, and nothing needs a network.
set -uo pipefail
here=$(cd "$(dirname "$0")" && pwd)
check=$here/check-spec-refs.py
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
fails=0
# mk <root> <standard> <spec-headings> <rs-relative-path> <rs-body>
mk() {
mkdir -p "$1/$2/$(dirname "$4")"
printf '%s\n' "$3" > "$1/$2/SPEC.md"
printf '%s\n' "$5" > "$1/$2/$4"
}
want() { # want <expect-status> <root> <label>
python3 "$check" "$2" > "$tmp/out" 2>&1
got=$?
if [ "$got" != "$1" ]; then
echo "FAIL: $3 — expected exit $1, got $got"
sed 's/^/ /' "$tmp/out"
fails=$((fails + 1))
else
echo "ok: $3"
fi
}
r=$tmp/a; mk "$r" alpha '## 4. Flow
### 4.1 Detail' src/lib.rs '/// Enforces SPEC §4.1 and SPEC §4.'
want 0 "$r" "passes when every citation resolves"
r=$tmp/b; mk "$r" alpha '## 4. Flow' src/lib.rs '/// Enforces SPEC §4.10.'
want 1 "$r" "fails on a citation with no heading"
r=$tmp/c; mk "$r" alpha '## §4. Flow
### §4.1 Detail' src/lib.rs '/// Enforces SPEC §4.1.'
want 0 "$r" "accepts the section-sign heading spelling"
r=$tmp/d; mk "$r" alpha '### 4.10 Child only' src/lib.rs '/// Enforces SPEC §4.'
want 1 "$r" "does not let a child heading satisfy its parent"
r=$tmp/e
mk "$r" alpha '## 7. Only in alpha' src/lib.rs '/// nothing cited here'
mk "$r" beta '## 1. Only in beta' src/lib.rs '/// Enforces SPEC §7.'
want 1 "$r" "does not resolve beta's citation against alpha's SPEC"
r=$tmp/f; mk "$r" alpha '## 4. Flow' target/debug/build.rs '/// Enforces SPEC §9.'
want 0 "$r" "ignores .rs under target/"
if [ "$fails" != 0 ]; then
echo "check-spec-refs-selftest: $fails case(s) failed."
exit 1
fi
echo "check-spec-refs-selftest: all cases pass."