# conform-result-conformance

The conformance corpus of [conformance-results](../extensions/conformance-results.md) (§7):
records of `conform-result/1`, the signed outcome of a conformance run, each with the outcome
§4 gives it.

**Provenance.** Generated here by `wai_conform_result_vectors gen`, from records written in
code and signed with published one-byte seeds: 32 bytes of `0x43` (the key of every record but
one) and of `0x44` (a key a runner holds, in `valid_http_supplied_key`). Both are test keys and
attest nothing. Ed25519 signing is deterministic (RFC 8032), so the file is the same bytes on
every machine. Every run, corpus file, program, path and URL a record names is an example; no
record states a run that took place.

Each vector declares **one outcome**, set from the extension's text in the generator and never
copied from a verifier: `accept conformant=true`, `accept conformant=false`, or
`reject step=<n> reason=<code>` at a step of §4 with a code of §5. A verifier that reaches
another outcome does not conform.

## Files

| file | what it is |
|---|---|
| `vectors.json` | The corpus: each vector's `name`, what it is (`about`), the record's bytes (`record`, a string, since some vectors are bytes a JSON reader must refuse) and its outcome (`expected`); and the corpus keys' public keys. |
| `conform_result_verify.py` | The second implementation, written from the extension's text and sharing no code with `wai-rs`. It takes WAI's strict Ed25519 rule from [`../claim-conformance/_ed25519.py`](../claim-conformance/_ed25519.py), which is written from RFC 8032; it reads the JSON and writes the RFC 8785 form itself. Standard library only. |

## Vectors

- **Accepted:** a conformant run (`valid`); a run with a failed case (`valid_not_conformant`),
  a genuine record of a run that did not conform; a run with no case (`valid_no_cases`); and a
  decoder's run from the HTTP adapter, signed with a runner's key, with a name and version
  (`valid_http_supplied_key`).
- **Refused at step 4, `signature_invalid`:** a failed case turned into a pass after signing,
  its summary and verdict made to agree (`flipped_case`); the corpus digest changed
  (`corpus_digest_changed`); the summary alone changed (`summary_changed`); one bit of the
  signature changed (`bad_signature`); the signature's `S` replaced by `S + L`
  (`signature_s_not_reduced`); the identity as the key with `(R, S) = (B, 1)`, which the
  cofactorless equation accepts over any message (`key_small_order`).
- **Signed as they are, and refused:** a summary that does not count its cases (step 5,
  `summary_mismatch`): two counts wrong, and each of the three counts wrong alone with the
  others right; conformant though a case failed, and conformant with no case (step 6,
  `conformant_mismatch`); another version of the format (step 2, `unknown_format`); a member
  missing and a member added (step 3, `malformed`).
- **Bytes refused before a signature is read:** the verdict named twice (step 1); a count
  written with a fraction and a digest in upper-case hexadecimal (step 3); bytes that are not a
  JSON text (step 1).

## Checks

From `wai/wai-rs`:

```
cargo run --no-default-features --features conform_result --bin wai_conform_result_vectors -- verify ../conform-result-conformance
cargo test --no-default-features --features conform_result --test conform_result
```

`verify` requires `vectors.json` to regenerate byte for byte, no other file but this README and
the second implementation, and every vector to reach its outcome. The test adds that every
reason of §5 refuses a vector, that §5 lists exactly the reference's codes, and that a record
changed anywhere after it is signed is refused.

From `wai/`:

```
python3 -I conform-result-conformance/conform_result_verify.py conform-result-conformance
python3 -I conform-result-conformance/conform_result_verify.py --skip <check> conform-result-conformance
```

The first must report every vector's outcome and that the corpus refuses with every reason of §5.
The second leaves out one check of §4 (`duplicates`, `format`, `members`, `signature`,
`summary`, `verdict`) and succeeds only when the corpus then fails: each check is guarded by a
vector. `wai_conform_result_vectors verdicts` and `conform_result_verify.py verdicts` print
`name<TAB>outcome` for any corpus-format file, and `wai conform verifier --suite conform-result`
runs a verifier against this corpus.