417847daab
The change of wai/conform-output, on current main, without its wai doctor: main's doctor, its tests, its catalogue entry, its MCP tool and its changelog entry stand as they are, and this change's copies of them are gone, with the lines dev.rs held for them. wai/README.md, which had no line for main's doctor, gains one in its first commands and `toolchain_doctor` in the core tools. - wai conform --format text|json|junit|html, reports made from the signed records. - --independent beside a kind holds the corpus's independent verifier to the same cases; alone, it runs every corpus's verifier, a record each, from one table. - wai-rs/build.rs reads that table from the `# independent:` lines of the step files (.github/wai-conformance/verifiers/), which take the verifiers' command lines out of the steps' bodies. scripts/wai-conformance-lists.py reads and checks the same lines. - wai-conformance.yml runs the union as wai conform --independent --format junit. - wai mcp's conformance tools read the table. - wai-release.yml runs the quickstart from each archive as a new user runs it (clean-machine.sh, its PowerShell twin clean-machine.ps1), held by check-wai-release-quickstart.py. clean-machine.ps1 keeps clean-machine.sh's contents step and its check of WAI Studio. Each leg keeps main's archive_doctor.py check; the step's own wai hello, verify and dev --check are clean-machine.sh's now. - check-wai-ops-doors.py reads VERBS and NEAR_SYNONYMS from catalogue.rs and keeps no list, and the verb-list test reads what that check reads. - The catalogue's export and Studio's actions hold the new operations. Co-Authored-By: Claude Opus 5.5 <[email protected]>
31 lines
2.2 KiB
Shell
31 lines
2.2 KiB
Shell
# step: robustness evidence · wai-rs and robustness_verify.py
|
|
# order: 320
|
|
# features: robustness
|
|
# release-features: robustness
|
|
# paths: wai/extensions/robustness-evidence.md
|
|
# independent: robustness live python3 -I live-conformance/robustness_verify.py live-conformance/robustness
|
|
#
|
|
# Robustness evidence for soft bindings (extensions/robustness-evidence.md): the
|
|
# reference's unit tests (gamma is the rounded power, rates are exact and reduced, every
|
|
# attack of synthesis §7.5.7's battery is a transform that gives its bytes, parameters
|
|
# outside their ranges are refused), then tests/robustness.rs in release, since
|
|
# regenerating the corpus runs the harness over the measurement sets of synthesis §7.5.8
|
|
# (every file of live-conformance/robustness/ regenerates byte for byte and no other file
|
|
# is there; every case gets what it states; §12 lists exactly the reference's codes and
|
|
# the corpus refuses a case for each; §4 registers exactly the reference's transforms;
|
|
# the tent64 records' tables are the synthesis corpus's battery.json; the tent64 claim is
|
|
# synthesis §7.5.8's list; jwp-receipts §3.1 lists the span reasons); and, in the step of
|
|
# every independent verifier above, the second
|
|
# implementation, robustness_verify.py with _wai_mark.py, written from the extension's
|
|
# text and sharing no code with wai-rs, which reruns the conformance record's battery end
|
|
# to end and requires the same table, checks the tent64 records against battery.json and
|
|
# the claim against synthesis §7.5.8, mutates copies of the corpus and weakens its own
|
|
# verifier in turn; standard library only, run isolated (-I).
|
|
#
|
|
# The paths this step brings to the workflow's trigger, and why:
|
|
# wai/extensions/robustness-evidence.md: tests/robustness.rs requires this extension's §12 to list
|
|
# the codes the reference reports and its §4 to register the reference's transforms,
|
|
# robustness_verify.py reads both, and the registry's wire identifiers test reads its sections.
|
|
set -euo pipefail
|
|
(cd wai-rs && cargo test --no-default-features --features "$WAI_FEATURES" --lib robustness::)
|
|
(cd wai-rs && cargo test --release --no-default-features --features "$WAI_RELEASE_FEATURES" --test robustness)
|