Transaction-Science/open-standards
Watch
1
0
Fork
You've already forked open-standards
 0
Code Issues Pull requests Projects Releases 17 Packages Wiki Activity Actions 30
open-standards/.github/wai-conformance
History
Exact
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Transaction Science 78a739203e The catalogue-prior record holds its rule for a learned prior and an episodic series, pinned by SHA-256 
research/catalogue-prior/decisive-rule.json is the rule that judges a run on a
learned prior tuned on a catalogue (H1c) or on an episodic series (H1b),
committed before any such run. Its SHA-256 is
8e265cfa2a3cb3de8c02a2c79a65bba59b52f2c134f4baf39d8bf163c0c38bc2.

It keeps the record's rule and answers the three faults §2.1 lists: K1 is
diagnostic, the bar for a video claim is plain per-title AV1, and K4 kills H1c
for an arm whose catalogue-tuned set beats the same codec's generic set by less
than 5 % BD-rate on PSNR on the median held-out title, before the set's bytes
are charged.

Its `base` field names what is new in this file and not carried over:
- GO judges H1b on the openings arm, against per-episode encodes, where the
  record's rule named no comparator for H1b and only K3 decided it.
- Each clause names the arms it judges. GO judges wai.neural.int_hyper's
  catalogue-tuned set against AV1 all-intra, the catalogue-tuned set of any
  other arm with inter prediction whose P-frames are not integer block motion
  compensation, named in a run's record before its first reading, against
  plain per-title AV1, and the openings. K4 judges each catalogue-tuned set
  against its generic set. wai.video.int_hyper's verdict is a kill or
  INCONCLUSIVE.
- One AV1 bar: 8-bit at preset 4, coded as §3.5 codes base8 in all else, and
  all-intra at preset 4. Preset 8 and a ×4 10-bit pipeline are reported.
- A kill wins over GO for an arm and claim, and K1 with GO is INCONCLUSIVE. K1
  reads only a rate proxy measured in the same run on the same titles, so a run
  that measures none has no K1, and run.json's K1 carries to no run here.
- GO's three quarters are of all the arm's titles, rounded up. A title
  INCONCLUSIVE on VMAF stays among them and does not count as meeting the bar.
  GO reads the cubic VMAF figure.

tools/wai_catalogue_prior_verify.py holds each rule file's SHA-256 itself
(RULE_SHA256), treats every file of the record named *-rule*.json as a rule
file, and checks it against that digest and the one §2.2 quotes in a
<!--cat-sha256:NAME--> marker. Its --self-test requires the record to pass and
each mutated copy to fail with its finding: a byte of the rule, a quoted digit,
the rule and its quoted digest changed together, an unlisted rule file, a
removed one, a removed, stray or misplaced marker, another format, and text
that is not JSON. The conformance step runs it before the record.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-07 03:19:08 -05:00
..
corpora Changelog entries and the conformance workflow's lists move into files a change adds 2026-11-05 01:41:29 -05:00
verifiers The catalogue-prior record holds its rule for a learned prior and an episodic series, pinned by SHA-256 2026-11-07 03:19:08 -05:00
baseline.json wai-conformance starts on a change to joule-ui-catalogue, the crate wai-app now links 2026-11-07 03:10:33 -05:00
README.md Changelog entries and the conformance workflow's lists move into files a change adds 2026-11-05 01:41:29 -05:00

README.md

What wai-conformance.yml runs, one file a step

wai-conformance.yml keeps no hand list of its verifier steps, corpora or kernel tests. Its lists are derived, when it runs, from the files here and from tags in the test files, by scripts/wai-conformance-lists.py, whose docstring states every key; its gate reads the steps' paths: through scripts/pr-touches-paths.py. A change that adds a verifier, a corpus or a kernel test adds a file, or a line to a file of its own, and edits no line of the workflow, unless a step's paths: names a file under none of the trigger's paths (check names it). Its features and [[test]] entries in wai/wai-rs/Cargo.toml stay shared (wai/CONTRIBUTING.md).

  • verifiers/<name>.sh: one step of the independent-verifiers job. Its body is the step's shell, run in wai/ (or working-directory:) with bash -eo pipefail, as a workflow step runs. Its header, the leading # key: value lines, gives step: (its name), order: (steps run in ascending order, a tie by file name), features: (each list compiles alone in features-alone, and every feature joins WAI_FEATURES), and where it needs them alone:, build: pq, release-features:, bins:, pq-bins:, paths: (the extension texts it reads, which the workflow's gate then runs on) and working-directory:. Prose comments follow the header, after a # line.
  • corpora/<name>.txt: one cargo-class corpus, verified by its own tool on every leg of cargo-classes (corpus:, order:, bin:, dir:).
  • A test file of wai/wai-rs/tests/ whose first line is // wai-conformance: int-kernels runs in the fast integer kernels' steps of both profiles (int-kernels-release: the release step only; int-kernels-amd: the AVX-512 host too). The steps' features are the tagged tests' required-features.
  • baseline.json: the lists the workflow kept by hand before they were derived. The repository checks require every derived list to hold every entry of it, and every step it names to run.

To see what the workflow runs: python3 scripts/wai-conformance-lists.py dry-run (of another commit: --ref <commit>; two of them diff line by line). To run one step as CI does: python3 scripts/wai-conformance-lists.py run --only <name>. To check the files: python3 scripts/wai-conformance-lists.py check.