f33765a846
The falsification of the reference found that leaving out the check of summary.cases alone left the corpus passing: every summary vector had passed and failed wrong too. Three vectors now carry each count wrong alone. Co-Authored-By: Claude Opus 5.5 <[email protected]>
73 lines
4.3 KiB
Markdown
73 lines
4.3 KiB
Markdown
# conform-result-conformance
|
|
|
|
The conformance corpus of [conformance-results](../extensions/conformance-results.md) (§7):
|
|
records of `conform-result/1`, the signed outcome of a conformance run, each with the outcome
|
|
§4 gives it.
|
|
|
|
**Provenance.** Generated here by `wai_conform_result_vectors gen`, from records written in
|
|
code and signed with published one-byte seeds: 32 bytes of `0x43` (the key of every record but
|
|
one) and of `0x44` (a key a runner holds, in `valid_http_supplied_key`). Both are test keys and
|
|
attest nothing. Ed25519 signing is deterministic (RFC 8032), so the file is the same bytes on
|
|
every machine. Every run, corpus file, program, path and URL a record names is an example; no
|
|
record states a run that took place.
|
|
|
|
Each vector declares **one outcome**, set from the extension's text in the generator and never
|
|
copied from a verifier: `accept conformant=true`, `accept conformant=false`, or
|
|
`reject step=<n> reason=<code>` at a step of §4 with a code of §5. A verifier that reaches
|
|
another outcome does not conform.
|
|
|
|
## Files
|
|
|
|
| file | what it is |
|
|
|---|---|
|
|
| `vectors.json` | The corpus: each vector's `name`, what it is (`about`), the record's bytes (`record`, a string, since some vectors are bytes a JSON reader must refuse) and its outcome (`expected`); and the corpus keys' public keys. |
|
|
| `conform_result_verify.py` | The second implementation, written from the extension's text and sharing no code with `wai-rs`. It takes WAI's strict Ed25519 rule from [`../claim-conformance/_ed25519.py`](../claim-conformance/_ed25519.py), which is written from RFC 8032; it reads the JSON and writes the RFC 8785 form itself. Standard library only. |
|
|
|
|
## Vectors
|
|
|
|
- **Accepted:** a conformant run (`valid`); a run with a failed case (`valid_not_conformant`),
|
|
a genuine record of a run that did not conform; a run with no case (`valid_no_cases`); and a
|
|
decoder's run from the HTTP adapter, signed with a runner's key, with a name and version
|
|
(`valid_http_supplied_key`).
|
|
- **Refused at step 4, `signature_invalid`:** a failed case turned into a pass after signing,
|
|
its summary and verdict made to agree (`flipped_case`); the corpus digest changed
|
|
(`corpus_digest_changed`); the summary alone changed (`summary_changed`); one bit of the
|
|
signature changed (`bad_signature`); the signature's `S` replaced by `S + L`
|
|
(`signature_s_not_reduced`); the identity as the key with `(R, S) = (B, 1)`, which the
|
|
cofactorless equation accepts over any message (`key_small_order`).
|
|
- **Signed as they are, and refused:** a summary that does not count its cases (step 5,
|
|
`summary_mismatch`): two counts wrong, and each of the three counts wrong alone with the
|
|
others right; conformant though a case failed, and conformant with no case (step 6,
|
|
`conformant_mismatch`); another version of the format (step 2, `unknown_format`); a member
|
|
missing and a member added (step 3, `malformed`).
|
|
- **Bytes refused before a signature is read:** the verdict named twice (step 1); a count
|
|
written with a fraction and a digest in upper-case hexadecimal (step 3); bytes that are not a
|
|
JSON text (step 1).
|
|
|
|
## Checks
|
|
|
|
From `wai/wai-rs`:
|
|
|
|
```
|
|
cargo run --no-default-features --features conform_result --bin wai_conform_result_vectors -- verify ../conform-result-conformance
|
|
cargo test --no-default-features --features conform_result --test conform_result
|
|
```
|
|
|
|
`verify` requires `vectors.json` to regenerate byte for byte, no other file but this README and
|
|
the second implementation, and every vector to reach its outcome. The test adds that every
|
|
reason of §5 refuses a vector, that §5 lists exactly the reference's codes, and that a record
|
|
changed anywhere after it is signed is refused.
|
|
|
|
From `wai/`:
|
|
|
|
```
|
|
python3 -I conform-result-conformance/conform_result_verify.py conform-result-conformance
|
|
python3 -I conform-result-conformance/conform_result_verify.py --skip <check> conform-result-conformance
|
|
```
|
|
|
|
The first must report every vector's outcome and that the corpus refuses with every reason of §5.
|
|
The second leaves out one check of §4 (`duplicates`, `format`, `members`, `signature`,
|
|
`summary`, `verdict`) and succeeds only when the corpus then fails: each check is guarded by a
|
|
vector. `wai_conform_result_vectors verdicts` and `conform_result_verify.py verdicts` print
|
|
`name<TAB>outcome` for any corpus-format file, and `wai conform verifier --suite conform-result`
|
|
runs a verifier against this corpus.
|