50 lines
3.1 KiB
Markdown
50 lines
3.1 KiB
Markdown
# live-conformance/robustness — robustness evidence for soft bindings
|
||
|
||
The corpus of [`extensions/robustness-evidence.md`](../../extensions/robustness-evidence.md) §13:
|
||
|
||
- `corpus/<name>.json`: the corpus documents (§3), each the JCS text of its document, so the
|
||
SHA-256 of the file is the document's digest: `conformance`, three pictures of 256 × 256 whose
|
||
samples this directory holds; `tent64-spectral`, `tent64-leaves` and `tent64-clips`, the
|
||
measurement sets of synthesis §7.5.8, by their samples' digests.
|
||
- `frames/<SHA-256>.rgb`: the samples of the conformance pictures, raw RGB8, each file named by
|
||
the SHA-256 of its bytes.
|
||
- `records/<name>.json`: the signed records, each its JCS text: `conformance.json`, the record
|
||
over the conformance corpus; `conformance-claim.json`, the claim of what that record shows
|
||
survived; `tent64-spectral.json`, `tent64-leaves.json` and `tent64-clips.json`, the records of
|
||
`wai-mark:tent64` over the measurement sets (§10); and `tent64-claim.json`, its claim, which is
|
||
synthesis §7.5.8's list.
|
||
- `vectors.json`: `algorithm`, the term and digest of `wai-mark:tent64`; `harness`, the term of
|
||
§6; and `cases`, each with its `policy` (`evidence`: `reproduced` or `stated`), the
|
||
algorithms the verifier `implements`, its `records` in order as JSON texts, the `corpora` it
|
||
was handed with them (`record`, the record's position, and a `file` of this directory or a
|
||
`text`), the `samples` it holds (each a SHA-256, the file `frames/<SHA-256>.rgb`), and what a
|
||
verifier gives it.
|
||
|
||
**Robustness equivalence.** For each case, a conforming verifier reaches the outcome it states:
|
||
|
||
- `refuse`: the `reason` of §12 and the `record`, the position of the record refused;
|
||
- `accept`: the `statements` of §9.3, one per record, exactly as written.
|
||
|
||
A verifier that reaches another outcome, reason, record or statement does not conform.
|
||
|
||
**Provenance.** `wai_robustness_vectors gen` writes every file from fixed seeds: the
|
||
conformance pictures are two dead-leaves pictures (`synthesis_mark::leaves`, seeds 200 and 201)
|
||
and a spectral one (`wai.generate.spectral`'s generator with its committed model, the prompt "a
|
||
lighthouse at dusk", seed 200), each 256 × 256; the measurement sets are the synthesis corpus's
|
||
(`live-conformance/synthesis-mark/`); every key is SHA-256 of `wai-robustness corpus key` and an
|
||
index; Ed25519 signing is deterministic. The records' tables are the harness's (§6), run by the
|
||
generator. No picture comes from outside the repository.
|
||
|
||
The records of `wai-mark:tent64` are reproduced by the generator, which makes the measurement
|
||
sets; their samples, 25 MB, are not in this directory, so a verifier that does not make them
|
||
reads those records as stated, and checks their tables against
|
||
`../synthesis-mark/battery.json`, which `wai_synthesis_mark_vectors` wrote: the same counts,
|
||
measured by another program.
|
||
|
||
From `wai/wai-rs`, the reference regenerates and checks the corpus; from this directory's
|
||
parent, the second implementation checks it:
|
||
|
||
```
|
||
cargo run --release --no-default-features --features robustness --bin wai_robustness_vectors -- verify ../live-conformance/robustness
|
||
python3 -I robustness_verify.py robustness
|
||
```
|