open-standards/wai/live-conformance/detection/README.md

# live-conformance/detection — detection verdicts

The corpus of [`extensions/detection.md`](../../extensions/detection.md) §8: cases of origin claims,
robustness records and detection verdicts, accepted and refused, and what a verifier states of each.

**Files.** `vectors.json` names the stand-in detector and its definition, and lists the cases,
each its `name`, its `provenance`, `robustness` and `verdicts` (paths of the records given, in
order), `requireEvidence` (the policy `evidence` of §5.1) and `expect`. `records/` holds every
record once, named by its role and the first 16 hexadecimal digits of the SHA-256 of its text.

**Equivalence.** A conforming verifier given a case's records and policy reaches the outcome
`expect` states (§5): for `refuse`, the `reason` of §7, the `list` and the `index`; for `accept`,
what §5.3 states: each origin claim's `link`, `subject` and `class`, and each verdict's `link`,
`subject`, `detector`, `verdict`, `besideClaim` and `evidence`.

**The cases.**

- Accepted: `verdicts-beside-claims` (two verdicts that agree with their claims and two that
  disagree, the classes as signed), `claims-without-verdicts` (the same claims, the same
  classes), and `verdict-without-claim`.
- One refused case for each reason of §7, some reasons more than once; each name says its defect.

**Provenance.** `wai_detection_vectors gen` writes every file but this one from fixed seeds:
every key is SHA-256 of a label, and no key is one anyone holds. The detector,
`wai-detector:conformance`, scores a picture by the share of its bright pixels: it detects nothing
about how a picture was made and stands for a detector a verifier does not implement. Its
robustness records are measured: the generator runs it at each threshold over two bright pictures
and two dark ones it draws, under four attacks of robustness-evidence §4. The verdicts' scores are
its scores of five pictures the generator draws, named by the content hash of each as a PNG.

**Checked by.** `wai_detection_vectors verify` and `tests/detection.rs` in `wai-rs`, and
`../detection_verify.py`, the second implementation, written from the extension's text, which
verifies the robustness records with `../robustness_verify.py`, requires every mutated copy of an
accepted case to be refused and every one of its own checks, dropped, to be caught.