open-standards/.github/workflows/conformance.yml
Transaction Science becbfa8466
Some checks failed
standards workspaces build + test / jouleclaw-rs (pull_request) Successful in 2m52s
standards workspaces build + test / aac-rs (pull_request) Successful in 1m10s
standards workspaces build + test / joule-code-rs (pull_request) Successful in 45s
standards workspaces build + test / joulecontract-rs (pull_request) Successful in 42s
smart-byte conformance gate / Validate conformance JSON files (pull_request) Successful in 4s
smart-byte conformance gate / Second implementation reproduces every vector (pull_request) Successful in 3s
smart-byte conformance gate / Spec changes require vector-pack updates (pull_request) Successful in 4s
smart-byte conformance gate / VERSIONS.md and README.md must exist (pull_request) Successful in 3s
no-vanity-metrics / Scan committed Markdown for vanity metrics (pull_request) Successful in 3s
standards workspaces build + test / map-rs (pull_request) Successful in 42s
standards workspaces build + test / mesh-rs (pull_request) Successful in 40s
standards workspaces build + test / eoc-rs (pull_request) Successful in 1m48s
standards workspaces build + test / jcp-rs (pull_request) Successful in 1m32s
standards workspaces build + test / proof-rs (pull_request) Successful in 1m11s
standards workspaces build + test / openpay (pull_request) Successful in 3m41s
standards workspaces build + test / sandbox-rs (pull_request) Successful in 2m8s
standards workspaces build + test / smart-byte-rs (pull_request) Successful in 2m35s
standards workspaces build + test / arl-rs (pull_request) Has been cancelled
standards workspaces build + test / joule-ui-rs (pull_request) Has been cancelled
standards workspaces build + test / jouledesk-rs (pull_request) Has been cancelled
standards workspaces build + test / joulehook-rs (pull_request) Has been cancelled
standards workspaces build + test / sense-rs (pull_request) Has been cancelled
standards workspaces build + test / macOS energy backend (jcp-energy) (pull_request) Has been cancelled
smart-byte: generate the conformance pack from the reference implementation
The v1 pack was filler. Its SAIDs were runs of one repeated character, every
`canonical_cbor_hex` was the same 7-byte stub, and its signatures were RFC 8032
test signatures over the empty message. All five canonicalisation encodings
were wrong; two were odd-length hex that could not be decoded at all. The gate
only ran `jq` over it, and the spec said implementers MUST pass it.

Now every value is computed, and checked twice:

- New crate `smart-byte-conformance`. It is the `gen-conformance-vectors` /
  `verify-conformance-vectors` pair that regeneration.md always promised and
  that never existed. The generator computes every expected value with
  smart-byte-core and smart-byte-lockstep, and fails if a negative vector is
  refused for any reason other than the one it names. Its `cargo test` (run by
  standards-build on every push) checks:
    * every published vector verifies;
    * the published files equal the generator's output byte for byte;
    * a defect planted in each file is caught on the vector it was planted in;
    * identity spec §2.4 prints the worked example's real values.
- `conformance/crosscheck/verify_v1.py`: a second implementation, in Python,
  stdlib only, with BLAKE3 and Ed25519 written from their specs and checked
  against known answers first. It reproduces all 47 vectors, and the
  conformance gate now runs it.

The pack now tests what the spec says:

- canonicalisation: each §2.3 rule, accepted and refused (19 vectors).
- SAIDs: the suite §2.5 names, which the pack never contained — the
  empty-map case, the one-field case, the worked example, five
  envelope-shaped objects, and two adversarial non-canonical encodings (one
  aimed at re-encoding verifiers, one at verifiers that hash bytes as
  received).
- signatures: over real SAIDs, plus a non-canonical S.
- lockstep: §13.3 outcomes and §14.1 dissent entries.
- gossip: content-addressed intake.

Vectors that contradicted the spec are corrected:

- canon-004 encoded floats, which §2.3 forbids;
- lockstep-003 logged dissent on a frame that never committed;
- gossip-002 settled a "collision" by arrival order;
- gossip-003 asserted backfill limits no spec defines.

regeneration.md records each change and why the draft pack was regenerated in
place, not by supersession.

Spec: §2.4 printed an invented digest and SAID, and miscounted the encoding as
62 bytes (it is 61); it now prints the real values. §2.5 names the vectors. §6
pointed at modules that did not exist, and now says where each part lives —
including that the key-event log does not derive §2 SAIDs yet.

Replaces this branch's placeholder pins in smart-byte-core/tests/pack.rs, which
were written to fail once the pack was regenerated, with the assertions they
asked for.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 20:30:04 -04:00

182 lines
6.4 KiB
YAML

name: smart-byte conformance gate
on:
pull_request:
paths:
- 'smart-byte/spec/**'
- 'smart-byte/conformance/**'
- '.github/workflows/conformance.yml'
push:
branches:
- main
paths:
- 'smart-byte/spec/**'
- 'smart-byte/conformance/**'
- '.github/workflows/conformance.yml'
jobs:
validate-json:
name: Validate conformance JSON files
runs-on: linux
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Ensure jq is available
run: jq --version
- name: Validate every JSON file under smart-byte/conformance/v*
run: |
set -euo pipefail
found=0
fail=0
while IFS= read -r -d '' f; do
found=$((found+1))
if ! jq empty "$f" >/dev/null 2>&1; then
echo "::error file=$f::invalid JSON"
fail=$((fail+1))
else
echo "ok: $f"
fi
done < <(find smart-byte/conformance -type f -name '*.json' -print0)
if [ "$found" -eq 0 ]; then
echo "::error::no JSON files found under smart-byte/conformance/v*"
exit 1
fi
if [ "$fail" -ne 0 ]; then
echo "::error::$fail JSON file(s) failed validation"
exit 1
fi
echo "validated $found JSON file(s)"
- name: Validate vector schema (top-level array, name field present)
run: |
set -euo pipefail
fail=0
while IFS= read -r -d '' f; do
if ! jq -e 'type == "array"' "$f" >/dev/null; then
echo "::error file=$f::top-level value must be a JSON array"
fail=$((fail+1))
continue
fi
if ! jq -e 'all(.[]; has("name") and has("description"))' "$f" >/dev/null; then
echo "::error file=$f::every vector must have name and description fields"
fail=$((fail+1))
fi
done < <(find smart-byte/conformance -type f -name '*_vectors.json' -print0)
if [ "$fail" -ne 0 ]; then
echo "::error::$fail vector file(s) failed schema validation"
exit 1
fi
- name: Validate vector names are unique within each file
run: |
set -euo pipefail
fail=0
while IFS= read -r -d '' f; do
dupes=$(jq -r '[.[].name] | group_by(.) | map(select(length>1)) | .[][0] // empty' "$f")
if [ -n "$dupes" ]; then
echo "::error file=$f::duplicate vector name(s): $dupes"
fail=$((fail+1))
fi
done < <(find smart-byte/conformance -type f -name '*_vectors.json' -print0)
if [ "$fail" -ne 0 ]; then
exit 1
fi
# The pack's values come from the Rust reference implementation, and
# standards-build checks them against it under `cargo test`. This job checks
# them against a second implementation, written separately in another
# language, so a mistake shared by the generator and its own verifier still
# fails here. Standard library only: nothing to install beyond python3.
crosscheck:
name: Second implementation reproduces every vector
runs-on: linux
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Ensure python3 is available
shell: bash
run: |
if ! command -v python3 >/dev/null 2>&1; then
SUDO=""; [ "$(id -u)" -ne 0 ] && SUDO="sudo"
$SUDO apt-get update -qq
$SUDO apt-get install -y -qq python3
fi
python3 --version
- name: Cross-check v1
run: python3 smart-byte/conformance/crosscheck/verify_v1.py smart-byte/conformance/v1
spec-vector-coupling:
name: Spec changes require vector-pack updates
runs-on: linux
if: github.event_name == 'pull_request'
steps:
- name: Checkout (with full history for diff)
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Determine the PR base
id: base
run: |
base="${{ github.event.pull_request.base.sha }}"
if [ -z "$base" ]; then
base="origin/main"
fi
echo "base=$base" >> "$GITHUB_OUTPUT"
- name: Enforce coupling between spec and conformance changes
run: |
set -euo pipefail
base="${{ steps.base.outputs.base }}"
head="${{ github.event.pull_request.head.sha }}"
if [ -z "$head" ]; then
head="HEAD"
fi
changed=$(git diff --name-only "$base" "$head")
echo "Files changed in this PR:"
echo "$changed"
spec_touched=0
conformance_touched=0
if echo "$changed" | grep -qE '^smart-byte/spec/'; then
spec_touched=1
fi
if echo "$changed" | grep -qE '^smart-byte/conformance/'; then
conformance_touched=1
fi
if [ "$spec_touched" -eq 1 ] && [ "$conformance_touched" -eq 0 ]; then
echo "::error::This PR modifies smart-byte/spec/ without modifying smart-byte/conformance/."
echo "::error::Per the conformance social contract, every spec change must be accompanied by a vector-pack update."
echo "::error::If the spec change is editorial and the wire format is unchanged, record the editorial reason in smart-byte/conformance/v<N>/regeneration.md and include that file in the PR."
exit 1
fi
echo "coupling check passed (spec_touched=$spec_touched, conformance_touched=$conformance_touched)"
versions-present:
name: VERSIONS.md and README.md must exist
runs-on: linux
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Required pack metadata files
run: |
set -euo pipefail
missing=0
for f in smart-byte/conformance/README.md smart-byte/conformance/VERSIONS.md; do
if [ ! -f "$f" ]; then
echo "::error file=$f::missing required pack metadata file"
missing=$((missing+1))
fi
done
for d in smart-byte/conformance/v*/ ; do
if [ ! -f "$d/regeneration.md" ]; then
echo "::error file=$d/regeneration.md::missing regeneration.md for $d"
missing=$((missing+1))
fi
done
if [ "$missing" -ne 0 ]; then
exit 1
fi