wai: bearer-receipts §13, reconciliations and holdings records over a return path of a few bytes #637
16 changed files with 5738 additions and 2 deletions
28
.github/wai-conformance/verifiers/compact-return-profile.sh
vendored
Normal file
28
.github/wai-conformance/verifiers/compact-return-profile.sh
vendored
Normal file
|
|
@ -0,0 +1,28 @@
|
|||
# step: compact return profile · wai-rs and compact_return_verify.py
|
||||
# order: 535
|
||||
# features: bearer_receipts,holdings
|
||||
# paths: wai/extensions/bearer-receipts.md
|
||||
#
|
||||
# The compact return profile (extensions/bearer-receipts.md §13): the reference module's unit
|
||||
# tests (a uint's one spelling, an instant's, the whole bitmap, a message reassembled in any
|
||||
# order with each defect refused, the bounds, and the corpus reaching every stated outcome and
|
||||
# failing a changed one) and tests/compact_return.rs (vectors.json regenerates byte for byte
|
||||
# from its fixed seeds and no other file is there, wai-rs gives every case its stated outcome
|
||||
# with every round trip holding both ways, §13.9 lists exactly the reference's codes, and
|
||||
# §13.8's floors are the lengths of the aggregate corpus's reports, read with the reference's
|
||||
# DAP reader); then the corpus's second implementation, compact_return_verify.py, written from
|
||||
# the extension's text and sharing no code with wai-rs, which reads groups and objects with
|
||||
# jwp_verify.py, claims with claim_verify.py, §5.3 with bearer_verify.py, holdings records with
|
||||
# holdings_verify.py and DAP reports with telemetry_aggregate_verify.py (the claim verifier
|
||||
# needs the `cryptography` package installed above), writes the references, the compact
|
||||
# records, their expansion and writing, fragments and reassembly itself, checks the floors
|
||||
# against telemetry-aggregation's table and corpus, requires every mutated copy of the corpus
|
||||
# to fail, and drops each of its own checks in turn; run isolated (-I).
|
||||
#
|
||||
# The paths this step brings to the workflow's trigger, and why:
|
||||
# wai/extensions/bearer-receipts.md: tests/compact_return.rs requires §13.9 to list the compact
|
||||
# profile's codes and §13.8 its floors, and compact_return_verify.py reads both.
|
||||
set -euo pipefail
|
||||
(cd wai-rs && cargo test --no-default-features --features "$WAI_FEATURES" --lib -- compact_return::)
|
||||
(cd wai-rs && cargo test --no-default-features --features "$WAI_FEATURES" --test compact_return)
|
||||
python3 -I live-conformance/compact_return_verify.py live-conformance/compact-return
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
### Reconciliations and holdings records over a return path of a few bytes
|
||||
|
||||
- **The compact return profile** ([extensions/bearer-receipts.md](extensions/bearer-receipts.md)
|
||||
§13): a reconciliation (§5) and a holdings record ([holdings](extensions/holdings.md) §4) in a
|
||||
binary form for a return path whose messages hold few bytes. It names by reference what the
|
||||
receiver already holds: the group (8 bytes of SHA-256 over its link hash), the bearer instance
|
||||
among that group's (4 bytes over its `bearer_id`) and the signer (8 bytes over its key); it writes
|
||||
integers in LEB128 and `issued_at` as seconds. A reference that resolves to no value, or to more
|
||||
than one, is refused, never resolved by choosing. A whole reconciliation carries no bitmap and its
|
||||
root is the group's; a partial one carries its bitmap, and its root is derived from the object
|
||||
receipts the receiver holds.
|
||||
- **Exact both ways.** A compact record expands, against the receiver's context, to the record's
|
||||
own bytes, a claim's canonical form or a holdings record's `WHD1`, which are then verified by
|
||||
their own text under the signature the compact record carried: the compact form states nothing of
|
||||
its own. A record has one compact form against a context. One that has none (an `issued_at` with a
|
||||
fraction or an offset, a group or instance the receiver does not hold, a `leaf_count` not the
|
||||
group's, a root its objects do not give, a hybrid claim) is refused `compact_not_encodable` and
|
||||
travels in its own form; nothing is sized by a claim's own `leaf_count` before it is found to be
|
||||
the group's.
|
||||
- **Fragments.** A message is the compact record and a 4-byte check, the first bytes of SHA-256 over
|
||||
the record, cut into frames under a 3-byte header: the sender's number for the message, the index
|
||||
and the count. Reassembly places fragments by index in any order of arrival and holds a fragment
|
||||
delivered twice, byte for byte, once. It refuses a fragment malformed, foreign (another number or
|
||||
count), repeated (other data at its index) or missing, and, by the check, one reordered or mixed
|
||||
in under the same header.
|
||||
- **DAP reports are never fragmented.** A task's floor is the length of its longest report; a path
|
||||
whose frame is below it carries none of the task's reports, and a sink holds them, or gives them to
|
||||
a node with a path that carries them, which uploads them unchanged. The floors of
|
||||
telemetry-aggregation's measures 1 to 4 are stated (552, 824, 2,664 and 648 bytes) and checked
|
||||
against the reports of its corpus.
|
||||
- **Beside it**: jwp-receipts §2.7 and holdings §1 point to the profile, and the CI's check of each
|
||||
feature list alone checks `bearer_receipts,holdings`.
|
||||
- **The corpus** [`live-conformance/compact-return/`](live-conformance/compact-return/): round trips
|
||||
of reconciliations in each signer form and over each bearer class and of holdings records,
|
||||
expansions (every form a reader takes, a changed byte refused by the record's own signature, a
|
||||
control for each reader rule of both records, references unknown and ambiguous, an instance held
|
||||
for other groups only, a holder's key under two `signer_id`s), records with no compact form,
|
||||
messages cut into frames of several sizes, and reassemblies with each defect. The reference is the
|
||||
`compact_return` module (features `bearer_receipts` and `holdings`); `compact_return_verify.py` is
|
||||
a second implementation.
|
||||
|
|
@ -17,6 +17,10 @@
|
|||
> `bearer_receipts`), with the two classes' bodies in `claim_receipt::bearer`; the corpus is
|
||||
> [`live-conformance/bearers/`](../live-conformance/bearers/), and its second implementation
|
||||
> is `live-conformance/bearer_verify.py`.
|
||||
> §13 writes a reconciliation and a holdings record compactly for return paths whose messages
|
||||
> hold few bytes; its reference is the `compact_return` module (features `bearer_receipts` and
|
||||
> `holdings`), its corpus [`live-conformance/compact-return/`](../live-conformance/compact-return/)
|
||||
> and its second implementation `live-conformance/compact_return_verify.py`.
|
||||
> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
|
||||
>
|
||||
> Texts read for this revision, each at the edition named: ETSI TS 103 769 V1.2.1 (2024-11);
|
||||
|
|
@ -32,7 +36,9 @@ In scope: the bearer classes (§2); why the bearer is declared beside the receip
|
|||
it (§3); the transmission claim (§4) and the reconciliation claim (§5), their bodies, rules
|
||||
and context checks; the verdict a verifier derives for one group from its receipts and those
|
||||
claims (§6); the upper bound on a receiver's share of a shared transmission's energy (§7);
|
||||
verification with no return path (§8); and the mappings onto bearers (§9).
|
||||
verification with no return path (§8); the mappings onto bearers (§9); and a compact form of
|
||||
reconciliations and holdings records for return paths whose messages hold few bytes, carried in
|
||||
fragments (§13).
|
||||
|
||||
When each object passes on a bearer with no return path, and with which forward error
|
||||
correction, is [carousel](carousel.md)'s: a publisher's `schedule` claim beside the transmission
|
||||
|
|
@ -570,3 +576,318 @@ mutates copies of the corpus and requires every copy to fail.
|
|||
controls the bearer can withhold revisions — a revocation among them — but cannot add a key
|
||||
to an anchored origin. A sink that has had no path for long holds a stale log, and its
|
||||
confirmation reports the head it used.
|
||||
|
||||
## 13. The compact return profile
|
||||
|
||||
A path back from a sink or a node may carry only a few bytes in each message: a radio frame, a
|
||||
text message, a satellite uplink. A reconciliation (§5) is a JSON claim that writes its group's
|
||||
link hash, its root, its signer's key and its signature in hexadecimal, beside the instance's URI;
|
||||
a holdings record ([holdings](holdings.md) §4) names its holder and each group by 32 bytes. This
|
||||
profile writes both records in a binary form that names by short references what the receiver
|
||||
already holds — the group, the bearer instance, the signer — and carries a message too long for
|
||||
one frame in fragments, with a check over the whole message.
|
||||
|
||||
The compact form is an encoding, never a statement of its own. A receiver expands a compact
|
||||
record, against what it holds, into the record's own bytes and verifies that record by its own
|
||||
text; the signature a compact record carries is the record's. Nothing here changes what a
|
||||
reconciliation or a holdings record states, how either is verified, or what a verifier derives
|
||||
from it (§6, holdings §5).
|
||||
|
||||
### 13.1 Terms
|
||||
|
||||
- **Path**: what carries messages from a **sender** (a sink, or a node) to a **receiver** (the
|
||||
distributor that reconciles, or a node that gathers for it). Its **frame** is the most bytes
|
||||
one of its messages carries.
|
||||
- **Context**: what the receiver holds that a compact record names. Nothing a compact record
|
||||
carries adds to it.
|
||||
- **Groups**: group receipts that pass jwp-receipts §3 steps 1 and 2, each with the object
|
||||
receipts held under it that pass §3 step 5 against it, and the `bearer_id` of each instance
|
||||
of the group the receiver holds a declaration of (§4).
|
||||
- **Signers**: keys, each with a `signer_id`: keys the receiver confirmed under jwp-receipts
|
||||
§7, or keys its senders enrolled with it.
|
||||
- **Compact record**: a reconciliation (§13.3) or a holdings record (§13.4) in this profile's
|
||||
form, at most 65,531 bytes. **Message**: a compact record followed by its **check**, the first 4
|
||||
bytes of SHA-256 over the compact record. **Fragment**: one frame's part of a message (§13.6).
|
||||
|
||||
A fixed-size integer is unsigned and big-endian.
|
||||
|
||||
### 13.2 Uints, instants and references
|
||||
|
||||
- A **uint** is an unsigned integer in LEB128: seven bits a byte, least significant first, the
|
||||
high bit set in every byte but the last. It is written in the fewest bytes, so a last byte of
|
||||
0x00 follows no other byte, and its value is at most 2^53 − 1, so it has at most 8 bytes.
|
||||
- An **instant** is a uint of seconds since 1970-01-01T00:00:00Z, leap seconds not counted, at
|
||||
most 253,402,300,799 (9999-12-31T23:59:59Z). It stands for the RFC 3339 date-time of that
|
||||
second written `YYYY-MM-DDTHH:MM:SSZ`, with `T` and `Z` in capitals, no fraction and a second
|
||||
below 60. A date-time written in any other way has no instant.
|
||||
- The **reference** of a value is the first *n* bytes of SHA-256 over it: a group's is 8 bytes
|
||||
over its link hash (32 bytes); a bearer instance's, 4 bytes over its `bearer_id` in UTF-8; a
|
||||
signer's, 8 bytes over its key (32 bytes).
|
||||
- A reference **resolves** to the one value of its kind in the context that has it: a group
|
||||
among the groups; a bearer instance among the instances of the group the record names; a
|
||||
signer among the signers — a key with its `signer_id` for a reconciliation, a key for a
|
||||
holdings record, which names no `signer_id`. Values that are equal are one value. A
|
||||
reference that no value has is refused `compact_reference_unknown`, and one that more than
|
||||
one value has, `compact_reference_ambiguous`: neither is resolved by choosing.
|
||||
|
||||
### 13.3 A compact reconciliation
|
||||
|
||||
| size | field | meaning |
|
||||
|---|---|---|
|
||||
| 1 | `kind` | 0x01 |
|
||||
| 1 | `form` | below |
|
||||
| 8 | `group` | the reference of the group: the claim's `subject` |
|
||||
| 4 | `bearer` | the reference of the bearer instance: the body's `bearer_id` |
|
||||
| uint | `seq` | the claim's `seq` |
|
||||
| uint | `issued_at` | the claim's `issued_at`, as an instant (§13.2) |
|
||||
| 32, 32 + uint + *n*, or 8 | `signer` | by the signer form, below |
|
||||
| 32 or 0 | `parent` | the claim's `parent_receipt_hash`, where `form` says the claim has one |
|
||||
| ⌈`leaf_count` / 8⌉ or 0 | `received` | the bitmap of §5.1, where `form` says the reconciliation is partial; `leaf_count` is the group's |
|
||||
| 64 | `sig` | the claim's `sig` |
|
||||
|
||||
`form` is read by its bits:
|
||||
|
||||
| bits | meaning |
|
||||
|---|---|
|
||||
| `form & 0x03` | the body's `bearer`: 0 `bidirectional`, 1 `multicast`, 2 `unidirectional` |
|
||||
| `form & 0x04` | set: **whole**, every object of the group received, and no `received` follows; clear: **partial** |
|
||||
| `(form >> 3) & 0x03` | the signer form: 0, the key (32 bytes), whose Ed25519 `did:key` (jwp-receipts §7.1) is the `signer_id`; 1, the key, then a uint *n* and the `signer_id` in *n* bytes of UTF-8; 2, the reference of the signer (8 bytes) |
|
||||
| `form & 0x20` | set: a `parent` follows; clear: `parent_receipt_hash` is `null` |
|
||||
| `form & 0xc0` | 0 |
|
||||
|
||||
A reader reads exactly the bytes it is given, field by field in the order of the table, and
|
||||
refuses them `compact_malformed` when they are more than 65,531 bytes, end before a field or go
|
||||
on after `sig`, or when:
|
||||
|
||||
1. `form` sets a bit of 0xc0, or names the bearer 3 or the signer form 3;
|
||||
2. a uint breaks §13.2, or `issued_at` is past its bound;
|
||||
3. in the signer form 1, the `signer_id` is not UTF-8, or is the key's own `did:key`, which is
|
||||
the signer form 0;
|
||||
4. a partial `received` sets no bit, sets a bit at or past `leaf_count`, or sets every bit below
|
||||
it, which is the whole form.
|
||||
|
||||
It resolves `group` as it reads it, since `received`'s length is the group's; then `bearer` among
|
||||
that group's instances; and, in the signer form 2, `signer` (§13.2).
|
||||
|
||||
Its **expansion** is the reconciliation claim in its canonical form (jwp-receipts §2.4.3), with:
|
||||
|
||||
- `kind` `claim`, `class` `reconciliation`, `subject_type` `receipt`, and `subject` the link hash
|
||||
of the group resolved;
|
||||
- `seq`, `issued_at` (the instant's date-time), `signer_pubkey`, `signer_id`,
|
||||
`parent_receipt_hash` and `sig` from the fields;
|
||||
- a body of `bearer` from `form`, `bearer_id` the instance resolved, `leaf_count` the group's,
|
||||
`attests` `received`, and:
|
||||
- whole: `received` the bitmap that sets every bit below `leaf_count`, and `received_root` the
|
||||
group's `root_hash`;
|
||||
- partial: `received` as carried, and `received_root` the root jwp-receipts §2.2 computes over
|
||||
the leaves of the object receipts the context holds at its set bits, in increasing order, each
|
||||
leaf in its object receipt's form. A set bit at which the context holds no object receipt is
|
||||
refused `compact_objects_not_held`.
|
||||
|
||||
The expansion is then verified as every reconciliation is: jwp-receipts §3.1, and §5 here. A
|
||||
compact record whose bytes were changed, where it still reads, expands to a claim its signer did
|
||||
not sign.
|
||||
|
||||
### 13.4 A compact holdings record
|
||||
|
||||
| size | field | meaning |
|
||||
|---|---|---|
|
||||
| 1 | `kind` | 0x02 |
|
||||
| 1 | `form` | 0: the holder's key follows (32 bytes); 1: the reference of the holder follows (8 bytes) |
|
||||
| 32 or 8 | `holder` | the holder |
|
||||
| uint | `as_of_ms` | the record's `as_of_ms` |
|
||||
| uint | `window_ms` | its `expires_ms` less its `as_of_ms` |
|
||||
| uint | `n` | its entries' count, at most 2^32 − 1 |
|
||||
| … | `entries` | `n` entries, below, in the record's order |
|
||||
| 64 | `sig` | the record's `sig` |
|
||||
|
||||
An entry:
|
||||
|
||||
| size | field | meaning |
|
||||
|---|---|---|
|
||||
| 1 | `form` | `form & 0x01` set: every object of the group (holdings §4.2's form 1), and no `held` follows; `form & 0x02` set: the group by reference; every other bit 0 |
|
||||
| 8, or 32 + uint | `group` | by reference: the reference of the group, whose `leaf_count` is the group's; else the group's link hash, then its `leaf_count` as a uint |
|
||||
| ⌈`leaf_count` / 8⌉ or 0 | `held` | the bitmap of holdings §4.2, in its form 0 |
|
||||
|
||||
A reader reads exactly the bytes it is given, in the order of the tables, resolving each
|
||||
reference as it reads it, and refuses them `compact_malformed` when they are more than 65,531
|
||||
bytes, end before a field or go on after `sig`, or when a `form` is not one the tables give, a
|
||||
uint breaks §13.2, or `n` is past its bound. Its **expansion** is the holdings record of holdings
|
||||
§4.1: `WHD1`, version 1, the holder's key, `as_of_ms`, `as_of_ms` + `window_ms`, `n`, each
|
||||
entry's link hash, `leaf_count`, form and `held`, and `sig`. The expansion is then read as every
|
||||
holdings record is, by holdings §4.4, and holdings §5 applies to it.
|
||||
|
||||
### 13.5 Writing a record compactly
|
||||
|
||||
The compact form of a record **against a context** — what its sender knows its receiver holds —
|
||||
is the one the rules below give, and its expansion against that context is the record, byte for
|
||||
byte. A reader takes every form: a value written inline expands as its reference would.
|
||||
|
||||
- **A reconciliation** has a compact form when it reads under jwp-receipts §3.1 steps 1 and 2 as
|
||||
a claim of class `reconciliation` without the hybrid members of
|
||||
[pq-signatures](pq-signatures.md) §3.1, its text is its canonical form, its `issued_at` has an
|
||||
instant, and its expansion against the context is that record: the context holds its group
|
||||
and, for that group, its instance; its `leaf_count` is the group's; and its `received` and
|
||||
`received_root` are ones §13.3 reads and derives. Its signer is written by
|
||||
reference where the context holds its key with its `signer_id` and the reference resolves to
|
||||
them, else in the form 0 where its `signer_id` is its key's `did:key`, else in the form 1.
|
||||
- **A holdings record** has a compact form when it reads under holdings §4.4, its signature
|
||||
verifying. Its holder is written by reference where the context holds its key and the reference
|
||||
resolves to it, and each entry's group by reference where the context holds that group with the
|
||||
entry's `leaf_count` and the reference resolves to it; each is written inline otherwise.
|
||||
|
||||
A record that has no compact form is refused `compact_not_encodable`, and travels in its own
|
||||
form. A sink that sends its reconciliations over a path whose frame is small **SHOULD** sign them
|
||||
with an `issued_at` that has an instant: to the second, in UTC.
|
||||
|
||||
### 13.6 Fragments
|
||||
|
||||
| size | field | meaning |
|
||||
|---|---|---|
|
||||
| 1 | `message` | the sender's number for the message, the same in each of its fragments |
|
||||
| 1 | `index` | the fragment's position in the message, from 0 |
|
||||
| 1 | `count` | the message's fragments, 1 to 255 |
|
||||
| … | `data` | at least one byte of the message |
|
||||
|
||||
A message is at most 65,535 bytes. A sender cuts it, in order, into fragments of the path's frame,
|
||||
the last holding what remains: the fragments' `data`, in increasing `index`, are the message. It
|
||||
numbers its messages on one path modulo 256, and does not use a number again while a fragment
|
||||
under it may still arrive. A path whose frame is 3 bytes or fewer carries no fragment, and a
|
||||
message that would need more than 255 fragments of a path's frame is not sent over that path.
|
||||
|
||||
### 13.7 Reassembly
|
||||
|
||||
The steps below are defined over a set of fragments taken as one message's. A receiver forms that
|
||||
set from the fragments it receives, by their sender, as the path addresses it, and by `message`;
|
||||
how it forms the set is not a step, and refuses nothing. A set formed by `message` holds no
|
||||
fragment under another number, and step 2 refuses one in a set formed otherwise. Fragments that
|
||||
are the same bytes are one fragment of the set, so a fragment the path delivers twice is held
|
||||
once. Given a message's fragments, in any order, a receiver checks, in this order:
|
||||
|
||||
1. each fragment has at least 4 bytes, a `count` of at least 1 and an `index` below its `count`
|
||||
(`fragment_malformed`);
|
||||
2. each has the first's `message` and `count` (`fragment_foreign`);
|
||||
3. no two that differ have one `index` (`fragment_repeated`);
|
||||
4. there are `count` of them (`fragment_missing`); a message of which no fragment is held is
|
||||
missing;
|
||||
5. their `data`, in increasing `index`, the message, is at most 65,535 bytes
|
||||
(`fragment_malformed`);
|
||||
6. the message has more than 4 bytes, and its last 4 are the check over the bytes before them
|
||||
(`fragment_check_failed`).
|
||||
|
||||
The bytes before the check are the compact record, read by its `kind`: §13.3 for 0x01, §13.4 for
|
||||
0x02, and `compact_malformed` for any other.
|
||||
|
||||
Fragments arrive in any order and are placed by their `index`. A fragment placed at a position
|
||||
other than the one it was cut from — its `index` changed, or the data of two fragments exchanged —
|
||||
is a **reordered** fragment. Where the headers still show a whole message, the check refuses the
|
||||
message it is in (step 6), as it refuses a fragment of another message that carries this
|
||||
message's `message` and `count`: steps 2 to 4 find what the headers show, and step 6 what they do
|
||||
not.
|
||||
|
||||
### 13.8 Aggregated measurement on a small frame
|
||||
|
||||
A report of [telemetry-aggregation](telemetry-aggregation.md), a DAP report, is not a record of
|
||||
this profile, and this profile never cuts one into fragments. A sink uploads each report whole to
|
||||
the leader (telemetry-aggregation §5), and a report has no shorter form: its input shares are
|
||||
encrypted to the aggregators. A task's **floor** is the length of its longest report. A sink
|
||||
sends a task's reports only over a path that carries one whole, so a path whose frame is below a
|
||||
task's floor carries none of its reports. Every report of a task of measures 1 to 4 of
|
||||
telemetry-aggregation §3 has one length, its floor:
|
||||
|
||||
| measure | floor, bytes |
|
||||
|---|---|
|
||||
| 1 `outcomes` | 552 |
|
||||
| 2 `stalls` | 824 |
|
||||
| 3 `durations` | 2,664 |
|
||||
| 4 `start_time` | 648 |
|
||||
|
||||
A sink whose every path is below a task's floor holds its reports until it has a path that
|
||||
carries them, or gives them, over a short link, to a **node** that has one — a receiver's cache,
|
||||
or a box that holds a carousel for the receivers near it. The node uploads each report as it
|
||||
received it. It learns of each report its id, its time, its length and its task, as each
|
||||
aggregator does (telemetry-aggregation §10), and which neighbour gave it, and nothing of its
|
||||
measurement. A node **MAY** carry its neighbours' messages of this profile, or the records they
|
||||
expand to, over its own path in the same way: a record a node carries is its sender's, unchanged,
|
||||
and the node signs nothing of it.
|
||||
|
||||
### 13.9 Reasons
|
||||
|
||||
The codes this profile adds, each with where it is raised:
|
||||
|
||||
| reason | raised at |
|
||||
|---|---|
|
||||
| `compact_malformed` | §13.3, §13.4, §13.7 |
|
||||
| `compact_reference_unknown` | §13.2 |
|
||||
| `compact_reference_ambiguous` | §13.2 |
|
||||
| `compact_objects_not_held` | §13.3 |
|
||||
| `compact_not_encodable` | §13.5 |
|
||||
| `fragment_malformed` | §13.7 steps 1 and 5 |
|
||||
| `fragment_foreign` | §13.7 step 2 |
|
||||
| `fragment_repeated` | §13.7 step 3 |
|
||||
| `fragment_missing` | §13.7 step 4 |
|
||||
| `fragment_check_failed` | §13.7 step 6 |
|
||||
|
||||
An expansion also raises the codes of its record's own verification: those of jwp-receipts §3.1
|
||||
and §10 here for a reconciliation, and holdings §7's for a holdings record.
|
||||
|
||||
### 13.10 Conformance
|
||||
|
||||
[`live-conformance/compact-return/`](../live-conformance/compact-return/) holds the corpus,
|
||||
written from fixed seeds by `wai_compact_return_vectors gen` as `vectors.json`. It has:
|
||||
|
||||
- **Round trips**: reconciliations whole and partial, over each bearer class, in each signer form,
|
||||
with and without a parent, a group of labelled leaves and a bitmap whose last byte is part used
|
||||
among them; and holdings records with their holder and groups by reference and inline. Each
|
||||
case carries the record, the context and the compact record, written from the record and
|
||||
expanded back to it, byte for byte, and the record's verdict under its own text.
|
||||
- **Expansions**: every form a reader takes, a value the receiver holds written inline among
|
||||
them; a compact record with a byte changed, which expands and is refused by its record's own
|
||||
signature; one negative control for each rule of §13.3 and §13.4 but the bound of 65,531
|
||||
bytes, which both implementations check themselves; references unknown and ambiguous, two
|
||||
bearer instances that share one reference among them; a bearer instance the receiver holds for
|
||||
other groups and not for the record's, which is unknown; a holder's key held under two
|
||||
`signer_id`s, which resolves, since a holdings record names no `signer_id`; and a partial
|
||||
reconciliation against a receiver that holds no object receipt.
|
||||
- **Encodings**: records with no compact form, each refused, a reconciliation whose `leaf_count` is
|
||||
2^53 − 1 beside a bitmap of one byte among them.
|
||||
- **Messages and reassemblies**: compact records cut into frames of several sizes, one fragment
|
||||
among them, and reassembled in order and in other orders, and with a fragment delivered twice,
|
||||
byte for byte, which is held once; and a missing fragment, a repeated one (other data at its
|
||||
`index`, and a fragment relabelled to another's `index`), a reordered one (data exchanged,
|
||||
indices exchanged), a foreign one (another number, another count, and the same header from
|
||||
another message), a changed byte, and malformed fragments.
|
||||
|
||||
A conforming verifier gives every case the outcome, the reason and everything else the corpus
|
||||
states. `wai_compact_return_vectors verify` and the test `tests/compact_return.rs` check that the
|
||||
file regenerates byte for byte, that `wai-rs` gives every case its outcome, that every reason of
|
||||
§13.9 appears, that §13.9 lists exactly the reference's codes, and that §13.8's floors are the
|
||||
lengths of the reports in telemetry-aggregation's corpus, read with the reference's DAP reader.
|
||||
`live-conformance/compact_return_verify.py` is the second implementation, written from this text:
|
||||
it shares no code with `wai-rs`; it reads groups, objects and leaves with the JWP corpus's second
|
||||
implementation (`jwp-conformance/jwp_verify.py`), claims with the claim corpus's
|
||||
(`claim-conformance/claim_verify.py`), §5.3 with `bearer_verify.py`, holdings records with
|
||||
`holdings_verify.py` and DAP reports with `telemetry_aggregate_verify.py`, and writes §13.2 to
|
||||
§13.8 itself; it checks the floors against telemetry-aggregation's own table and corpus; and it
|
||||
mutates copies of the corpus and requires every copy to fail, and drops each of its own checks in
|
||||
turn and requires the corpus to catch the weakened verifier.
|
||||
|
||||
### 13.11 Security and privacy considerations
|
||||
|
||||
- **The compact form carries no trust of its own.** It carries its record's signature, and the
|
||||
receiver verifies the expansion by the record's own text. A reference that resolved to another
|
||||
value would expand to another record, which that signature does not verify.
|
||||
- **The check is not a signature.** Its four bytes of SHA-256 find what a fragment's header does
|
||||
not show — a fragment misplaced, or mixed in from another message under the same header, and a
|
||||
byte changed on the path — and authenticate nothing. Anyone on a path can make a message whose
|
||||
check holds: what it expands to is refused unless its signer signed it.
|
||||
- **What a reference discloses.** A reference names what the receiver holds, and says no less
|
||||
than the record it stands for: anyone who holds the group receipts resolves a group's. A signer
|
||||
by reference links a sender's messages as its key does. A sink that signs each reconciliation
|
||||
with a fresh `did:key` (§12) sends its key inline, in the signer form 0.
|
||||
- **Bounded work.** A message is at most 65,535 bytes in at most 255 fragments, a compact record is
|
||||
read once, in order, and resolving a reference is a lookup. A receiver **SHOULD** bound how long
|
||||
it holds the fragments of a message that has not completed; this profile sets no bound. A writer
|
||||
sizes nothing by a reconciliation's own `leaf_count` before it has found it to be the group's
|
||||
(§13.5): a record a node carries for a neighbour (§13.8) can state a `leaf_count` of 2^53 − 1
|
||||
beside a bitmap of one byte and still read under jwp-receipts §3.1 steps 1 and 2.
|
||||
|
|
|
|||
|
|
@ -27,6 +27,8 @@ Out of scope: how records travel (a neighbourhood's own announcements, a file be
|
|||
carousel, a request); how a node chooses what to keep; how a neighbour is reached or what a link
|
||||
costs, which the asker orders its neighbours by itself; and objects not delivered under a group
|
||||
receipt, which have no holdings here.
|
||||
A record's compact form, for a return path whose messages hold few bytes, is
|
||||
[bearer-receipts](bearer-receipts.md) §13's; it expands to the record's own bytes.
|
||||
|
||||
Relationship to the core spec: `wai.delivery.holdings` is a **record** ([SPEC](../SPEC.md) §5).
|
||||
It names no payload format and is not dispatchable.
|
||||
|
|
|
|||
|
|
@ -1751,6 +1751,9 @@ value the JSON form gives it, and a verifier then reads and checks those
|
|||
members by §3 unchanged. It is canonical: a receipt has one binary form, with
|
||||
no optional encoding, so a writer reproduces the bytes it read. A receipt whose
|
||||
JSON members a reader of §3 step 1 or step 4 refuses has no binary form.
|
||||
A reconciliation (§2.4.15) has a compact form for return paths whose messages
|
||||
hold few bytes, [bearer-receipts](bearer-receipts.md) §13, which a receiver
|
||||
expands to the claim's canonical form before it verifies it.
|
||||
|
||||
#### 2.7.1. A group receipt
|
||||
|
||||
|
|
|
|||
|
|
@ -31,6 +31,7 @@ and the independent verifier of each, beside it.
|
|||
| [`influence/`](influence/) | [influence](../extensions/influence.md) (`wai.media.influence`) | `wai_influence_vectors gen` | `influence_verify.py` |
|
||||
| [`direction/`](direction/) | [direction](../extensions/direction.md) (`wai.media.direction`) | `wai_direction_vectors gen` | `direction_verify.py` |
|
||||
| [`bearers/`](bearers/) | [bearer-receipts](../extensions/bearer-receipts.md) (`wai.bearer.receipts`) | `wai_bearer_vectors gen` | `bearer_verify.py` |
|
||||
| [`compact-return/`](compact-return/) | [bearer-receipts](../extensions/bearer-receipts.md) §13 (the compact return profile) | `wai_compact_return_vectors gen` | `compact_return_verify.py` |
|
||||
| [`carousel/`](carousel/) | [carousel](../extensions/carousel.md) (`wai.bearer.carousel`) | `wai_carousel_vectors gen` | `carousel_verify.py` |
|
||||
| [`holdings/`](holdings/) | [holdings](../extensions/holdings.md) (`wai.delivery.holdings`) | `wai_holdings_vectors gen` | `holdings_verify.py` |
|
||||
| [`access/`](access/) | [access-services](../extensions/access-services.md) (`wai.access.services`) | `wai_access_services_vectors gen` | `access_services_verify.py` |
|
||||
|
|
|
|||
49
wai/live-conformance/compact-return/README.md
Normal file
49
wai/live-conformance/compact-return/README.md
Normal file
|
|
@ -0,0 +1,49 @@
|
|||
# live-conformance/compact-return — reconciliations and holdings over a small frame
|
||||
|
||||
The corpus of [`extensions/bearer-receipts.md`](../../extensions/bearer-receipts.md) §13.10, in one
|
||||
file, `vectors.json`.
|
||||
|
||||
**Provenance.** Generated here by `wai_compact_return_vectors gen` (`wai-rs`, features
|
||||
`bearer_receipts` and `holdings`), from Ed25519 keys on fixed 32-byte seeds, constant times, and
|
||||
envelopes whose payloads are opaque test bytes written by the generator; no media and no input from
|
||||
outside this repository.
|
||||
|
||||
`keys` names every key by its role, as 64 lowercase hexadecimal digits: the publisher's
|
||||
(`did:web:publisher.example#d1`), four sinks' and two holders'. `groups` holds each group receipt the
|
||||
cases name, with its object receipts in order: `g4`, `g3` (labelled leaves), `g9`, `g20`, and `far`,
|
||||
which no receiver holds. `contexts` holds each receiver's context (§13.1): for each group it holds,
|
||||
the group's name, the positions of the object receipts it holds under it and the `bearer_id`s of
|
||||
its instances; and its signers, each a `key` and a `signer_id`. `twin_bearers` holds two instances
|
||||
of `g4` whose references are equal; `twin_signer`, sink B's key and the holder's each under two
|
||||
`signer_id`s; `bearer_elsewhere`, the broadcast instance of `g9` and `g20` but not of `g4`.
|
||||
|
||||
- `records` (§13.3 to §13.5) are each a `kind` (`reconciliation` or `holdings`), the `record` in its
|
||||
own form (a claim's canonical text, or a holdings record in hexadecimal), the `context` it is
|
||||
written against and its `compact` record in hexadecimal. The record written compactly is
|
||||
`compact`, `compact` expands to the record, and `outcome.verdict` is the record's verdict under
|
||||
its own text: `accept`, or `read digest=…` for a holdings record.
|
||||
- `expansions` (§13.2 to §13.4) are each a `compact` record and a `context`; `outcome` is `expand`
|
||||
with the record's `kind`, the `record` and its `verdict`, or `refuse` and its `reason`.
|
||||
- `encodings` (§13.5) are each a record with no compact form against its `context`; `outcome` is
|
||||
`refuse` and `compact_not_encodable`.
|
||||
- `messages` (§13.6) are each a compact `record`, the sender's `message` number and a `frame`;
|
||||
`outcome.fragments` is the message cut into that frame, each fragment in hexadecimal. The
|
||||
fragments reassemble to the record.
|
||||
- `reassemblies` (§13.7) are each a set of `fragments`; `outcome` is `reassembled` and the compact
|
||||
`record`, or `refuse` and its `reason`.
|
||||
|
||||
A reconciliation's verdict is jwp-receipts §3.1 steps 1 to 4, then step 6 by bearer-receipts §5.3
|
||||
against the context's group and objects: `accept`, or `reject step=… reason=…`. Step 5, a signer's
|
||||
chain, is the signer's claims' and not this profile's. A holdings record's is holdings §4.4: `read
|
||||
digest=…`, or `reject reason=…`.
|
||||
|
||||
Everything is synthetic: keys attest nothing, and every origin is a reserved example domain. Each
|
||||
case states the outcome it was written to have, declared in the generator, never copied from a
|
||||
verifier. The bounds of 65,531 bytes for a compact record and 65,535 for a message are carried by no
|
||||
vector (65 KB of hexadecimal); both implementations check them themselves.
|
||||
|
||||
`wai_compact_return_vectors verify` and `tests/compact_return.rs` check that the file regenerates
|
||||
byte for byte and that `wai-rs` reaches every stated outcome; the test also checks §13.8's floors
|
||||
against [`../aggregate/`](../aggregate/)'s reports. `../compact_return_verify.py` is the second
|
||||
implementation, written from the extension's text, which also checks the floors, mutates copies of
|
||||
the corpus and drops each of its own checks in turn and requires each to be caught.
|
||||
2781
wai/live-conformance/compact-return/vectors.json
Normal file
2781
wai/live-conformance/compact-return/vectors.json
Normal file
File diff suppressed because it is too large
Load diff
763
wai/live-conformance/compact_return_verify.py
Normal file
763
wai/live-conformance/compact_return_verify.py
Normal file
|
|
@ -0,0 +1,763 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Independent verifier of the compact return corpus (extensions/bearer-receipts.md §13).
|
||||
|
||||
Written from the extension's text. It shares no code with wai-rs, which generates the corpus, and
|
||||
never runs it. It takes from the second implementations of the texts the profile builds on what
|
||||
they define: group and object receipts, link hashes and leaves from the JWP corpus's
|
||||
(`../jwp-conformance/jwp_verify.py`, jwp-receipts §2.2, §3); a claim's canonical form, the
|
||||
Ed25519 `did:key` and a claim's steps 1 to 4 from the claim corpus's
|
||||
(`../claim-conformance/claim_verify.py`, jwp-receipts §2.4.3, §3.1, §7.1); a reconciliation's
|
||||
check against its group and objects from the bearer corpus's (`bearer_verify.py`,
|
||||
bearer-receipts §5.3); the holdings record from the holdings corpus's (`holdings_verify.py`,
|
||||
holdings §4.4); and the reading of a DAP report from the aggregation corpus's
|
||||
(`telemetry_aggregate_verify.py`). Everything §13 adds is written here:
|
||||
|
||||
- §13.2: uints, instants and references, and how a reference resolves;
|
||||
- §13.3, §13.4: the compact reconciliation and holdings record, read and expanded against a
|
||||
receiver's context;
|
||||
- §13.5: writing a record compactly, which it does itself and compares with the corpus's bytes;
|
||||
- §13.6, §13.7: cutting a message into fragments and reassembling it;
|
||||
- §13.8: each floor, checked against the reports of telemetry-aggregation's corpus, and each
|
||||
measure's name against that extension's own table.
|
||||
|
||||
For every case it requires the outcome the corpus states, field for field; every round trip both
|
||||
ways; every cut to reassemble to its record; that every reason of §13.9's table appears in the
|
||||
corpus; then it mutates copies of the corpus, one defect each, and requires every copy to fail;
|
||||
and it drops each of its own checks in turn and requires the corpus to catch the weakened
|
||||
verifier.
|
||||
|
||||
python3 compact_return_verify.py <live-conformance/compact-return>
|
||||
|
||||
Exit status 0 only if every case verifies, every mutated copy fails and every weakened verifier
|
||||
is caught. Python 3.8 or later; the `cryptography` package, which the claim verifier imports.
|
||||
"""
|
||||
|
||||
import calendar
|
||||
import copy
|
||||
import datetime
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
for d in ("claim-conformance", "jwp-conformance"):
|
||||
sys.path.insert(0, os.path.join(HERE, "..", d))
|
||||
sys.path.insert(0, HERE)
|
||||
import bearer_verify as BV # noqa: E402
|
||||
import claim_verify as CV # noqa: E402
|
||||
import holdings_verify as HV # noqa: E402
|
||||
import jwp_verify as JV # noqa: E402
|
||||
import telemetry_aggregate_verify as TV # noqa: E402
|
||||
|
||||
EXTENSION = os.path.join(HERE, "..", "extensions", "bearer-receipts.md")
|
||||
AGGREGATION = os.path.join(HERE, "..", "extensions", "telemetry-aggregation.md")
|
||||
AGGREGATE_CORPUS = os.path.join(HERE, "aggregate", "vectors.json")
|
||||
|
||||
KIND_RECONCILIATION, KIND_HOLDINGS = 0x01, 0x02
|
||||
HEADER, CHECK = 3, 4
|
||||
MAX_MESSAGE = 65535
|
||||
MAX_RECORD = MAX_MESSAGE - CHECK
|
||||
MAX_SECONDS = 253402300799
|
||||
MAX_INT = 2 ** 53 - 1
|
||||
BEARERS = ("bidirectional", "multicast", "unidirectional")
|
||||
|
||||
# A weakened run drops one of these checks (see `WEAKENINGS`); None checks everything.
|
||||
DROPPED = None
|
||||
|
||||
|
||||
def drops(name):
|
||||
return DROPPED == name
|
||||
|
||||
|
||||
class Refused(Exception):
|
||||
def __init__(self, reason):
|
||||
super().__init__(reason)
|
||||
self.reason = reason
|
||||
|
||||
|
||||
def section(heading):
|
||||
"""One `###` section of the extension, from its heading to the next heading."""
|
||||
text = open(EXTENSION, encoding="utf-8").read()
|
||||
at = text.index("\n### " + heading) + 1
|
||||
end = text.find("\n#", at)
|
||||
return text[at:end if end >= 0 else len(text)]
|
||||
|
||||
|
||||
def reasons_table():
|
||||
"""§13.9's codes, in order."""
|
||||
return [r.split("`")[1] for r in section("13.9 Reasons").splitlines() if r.startswith("| `")]
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ §13.2
|
||||
|
||||
def ref(n, b):
|
||||
"""The first n bytes of SHA-256 over b: a reference, and with n = 4 a message's check."""
|
||||
return hashlib.sha256(b).digest()[:n]
|
||||
|
||||
|
||||
def check_of(record):
|
||||
return ref(CHECK, record)
|
||||
|
||||
|
||||
def put_uint(v):
|
||||
"""Unsigned LEB128 in its fewest bytes."""
|
||||
out = bytearray()
|
||||
while True:
|
||||
b, v = v & 0x7F, v >> 7
|
||||
if v == 0:
|
||||
out.append(b)
|
||||
return bytes(out)
|
||||
out.append(b | 0x80)
|
||||
|
||||
|
||||
def instant_text(s):
|
||||
return (datetime.datetime(1970, 1, 1) + datetime.timedelta(seconds=s)).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
|
||||
|
||||
def instant_of(text):
|
||||
"""The seconds of an issued_at written YYYY-MM-DDTHH:MM:SSZ, a second below 60, from 1970 to
|
||||
9999; None for any other text."""
|
||||
m = re.fullmatch(r"([0-9]{4})-([0-9]{2})-([0-9]{2})T([0-9]{2}):([0-9]{2}):([0-9]{2})Z", text)
|
||||
if not m or not CV.is_rfc3339(text):
|
||||
return None
|
||||
y, mo, d, h, mi, s = (int(x) for x in m.groups())
|
||||
if y < 1970 or s > 59:
|
||||
return None
|
||||
return calendar.timegm((y, mo, d, h, mi, s, 0, 0, 0))
|
||||
|
||||
|
||||
def one(found, key):
|
||||
"""§13.2: the one value found under a reference; values with one key are one value."""
|
||||
keys = {key(x) for x in found}
|
||||
if not keys:
|
||||
raise Refused("compact_reference_unknown")
|
||||
if len(keys) > 1 and not drops("ambiguous"):
|
||||
raise Refused("compact_reference_ambiguous")
|
||||
return found[0]
|
||||
|
||||
|
||||
class Reader:
|
||||
"""Reads a compact record; `kind` names it once read, so that a weakened run can drop a rule
|
||||
for one kind of record alone."""
|
||||
|
||||
def __init__(self, b):
|
||||
self.b, self.at, self.kind = b, 0, ""
|
||||
|
||||
def take(self, n):
|
||||
if self.at + n > len(self.b):
|
||||
raise Refused("compact_malformed")
|
||||
s = self.b[self.at:self.at + n]
|
||||
self.at += n
|
||||
return s
|
||||
|
||||
def byte(self):
|
||||
return self.take(1)[0]
|
||||
|
||||
def uint(self):
|
||||
v = 0
|
||||
for i in range(8):
|
||||
b = self.byte()
|
||||
v |= (b & 0x7F) << (7 * i)
|
||||
if not b & 0x80:
|
||||
if (i > 0 and b == 0 and not drops("minimal_" + self.kind)) \
|
||||
or (v > MAX_INT and not drops("bound_" + self.kind)):
|
||||
raise Refused("compact_malformed")
|
||||
return v
|
||||
raise Refused("compact_malformed")
|
||||
|
||||
def end(self):
|
||||
if self.at != len(self.b) and not drops("trailing_" + self.kind):
|
||||
raise Refused("compact_malformed")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ §13.1: the context
|
||||
|
||||
_CONTEXTS = {}
|
||||
|
||||
|
||||
def included(o, g):
|
||||
try:
|
||||
JV.inclusion(o, g)
|
||||
return True
|
||||
except JV.Refuse:
|
||||
return False
|
||||
|
||||
|
||||
def context_of(v, groups):
|
||||
"""A context from the corpus's form: each group by name, its object receipts at the
|
||||
positions listed (those that pass jwp-receipts §3 step 5), its bearer instances; a group
|
||||
whose receipt fails §3 steps 1 and 2 is not held."""
|
||||
memo = json.dumps([v, groups], sort_keys=True)
|
||||
if memo in _CONTEXTS:
|
||||
return _CONTEXTS[memo]
|
||||
ctx = {"groups": [], "signers": []}
|
||||
for line in v["groups"]:
|
||||
e = groups[line["group"]]
|
||||
try:
|
||||
g = JV.read_group(e["receipt"])
|
||||
JV.verify_group(g)
|
||||
except JV.Refuse:
|
||||
continue
|
||||
objs = [o for o in (JV.read_object(e["objects"][i]) for i in line["objects"]) if included(o, g)]
|
||||
link = JV.link(g)
|
||||
held = next((x for x in ctx["groups"] if x["link"] == link), None)
|
||||
if held is None:
|
||||
ctx["groups"].append({"g": g, "link": link, "objects": objs, "bearers": list(line["bearers"])})
|
||||
else:
|
||||
held["objects"] += objs
|
||||
held["bearers"] += line["bearers"]
|
||||
ctx["signers"] = [(bytes.fromhex(s["key"]), s["signer_id"]) for s in v["signers"]]
|
||||
_CONTEXTS[memo] = ctx
|
||||
return ctx
|
||||
|
||||
|
||||
def group_at(ctx, r):
|
||||
return one([x for x in ctx["groups"] if ref(8, x["link"]) == r], lambda x: x["link"])
|
||||
|
||||
|
||||
def bearer_at(ctx, grp, r):
|
||||
"""Among the instances of the group the record names, not of every group held."""
|
||||
held = [b for x in ctx["groups"] for b in x["bearers"]] if drops("own_group") else grp["bearers"]
|
||||
return one([b for b in held if ref(4, b.encode()) == r], lambda b: b)
|
||||
|
||||
|
||||
def signer_at(ctx, r):
|
||||
return one([s for s in ctx["signers"] if ref(8, s[0]) == r], lambda s: s)
|
||||
|
||||
|
||||
def key_at(ctx, r):
|
||||
"""For a holdings record, which names no signer_id: one key under two signer_ids is one key."""
|
||||
if drops("key_alone"):
|
||||
return signer_at(ctx, r)[0]
|
||||
return one([s[0] for s in ctx["signers"] if ref(8, s[0]) == r], lambda k: k)
|
||||
|
||||
|
||||
def group_by_link(ctx, link):
|
||||
return next((x for x in ctx["groups"] if x["link"] == link), None)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ §13.3: a compact reconciliation
|
||||
|
||||
def full_bitmap(n):
|
||||
b = bytearray(b"\xff" * ((n + 7) // 8))
|
||||
if n % 8:
|
||||
b[-1] = (0xFF << (8 - n % 8)) & 0xFF
|
||||
return bytes(b)
|
||||
|
||||
|
||||
def bit(b, i):
|
||||
return b[i // 8] & (0x80 >> (i % 8)) != 0
|
||||
|
||||
|
||||
def received_root(grp, received):
|
||||
leaves = []
|
||||
for i in range(grp["g"]["leaf_count"]):
|
||||
if bit(received, i):
|
||||
o = next((o for o in grp["objects"] if o["object_id"] == i), None)
|
||||
if o is None:
|
||||
if drops("objects_held"):
|
||||
continue
|
||||
raise Refused("compact_objects_not_held")
|
||||
leaves.append(JV.leaf(o))
|
||||
return BV.merkle_root(leaves)
|
||||
|
||||
|
||||
def expand_reconciliation(r, ctx):
|
||||
bad = Refused("compact_malformed")
|
||||
form = r.byte()
|
||||
if form & 0x03 == 3 or (form >> 3) & 0x03 == 3 or (form & 0xC0 and not drops("form_reserved")):
|
||||
raise bad
|
||||
bearer, whole, signer_form = BEARERS[form & 0x03], form & 0x04 != 0, (form >> 3) & 0x03
|
||||
grp = group_at(ctx, r.take(8))
|
||||
bearer_id = bearer_at(ctx, grp, r.take(4))
|
||||
seq = r.uint()
|
||||
issued = r.uint()
|
||||
if issued > MAX_SECONDS and not drops("issued_bound"):
|
||||
raise bad
|
||||
if signer_form == 0:
|
||||
key = r.take(32)
|
||||
signer_id = CV.did_key(key)
|
||||
elif signer_form == 1:
|
||||
key = r.take(32)
|
||||
raw = r.take(r.uint())
|
||||
try:
|
||||
signer_id = raw.decode("utf-8", errors="replace" if drops("utf8") else "strict")
|
||||
except UnicodeDecodeError:
|
||||
raise bad
|
||||
if signer_id == CV.did_key(key) and not drops("did_key_form"):
|
||||
raise bad
|
||||
else:
|
||||
key, signer_id = signer_at(ctx, r.take(8))
|
||||
parent = r.take(32) if form & 0x20 else None
|
||||
n = grp["g"]["leaf_count"]
|
||||
if whole:
|
||||
received = full_bitmap(n)
|
||||
else:
|
||||
received = r.take((n + 7) // 8)
|
||||
past = any(bit(received, i) for i in range(n, 8 * len(received)))
|
||||
if (past and not drops("past_bits")) or (not any(received) and not drops("no_bits")) \
|
||||
or (received == full_bitmap(n) and not drops("whole_spelling")):
|
||||
raise bad
|
||||
sig = r.take(64)
|
||||
r.end()
|
||||
root = bytes.fromhex(grp["g"]["root_hash"]) if whole else received_root(grp, received)
|
||||
claim = {
|
||||
"kind": "claim", "class": "reconciliation", "subject_type": "receipt", "subject": grp["link"].hex(),
|
||||
"seq": seq, "issued_at": instant_text(issued),
|
||||
"body": {"bearer": bearer, "bearer_id": bearer_id, "leaf_count": n, "received": received.hex(),
|
||||
"received_root": root.hex(), "attests": "received"},
|
||||
"signer_pubkey": key.hex(), "signer_id": signer_id,
|
||||
"parent_receipt_hash": parent.hex() if parent is not None else None, "sig": sig.hex(),
|
||||
}
|
||||
return CV.jcs(claim)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ §13.4: a compact holdings record
|
||||
|
||||
def expand_holdings(r, ctx):
|
||||
bad = Refused("compact_malformed")
|
||||
form = r.byte()
|
||||
if form == 0:
|
||||
holder = r.take(32)
|
||||
elif form == 1:
|
||||
holder = key_at(ctx, r.take(8))
|
||||
else:
|
||||
raise bad
|
||||
as_of, window, n = r.uint(), r.uint(), r.uint()
|
||||
if n > 2 ** 32 - 1:
|
||||
raise bad
|
||||
out = bytearray(b"WHD1\x01" + holder + as_of.to_bytes(8, "big") + (as_of + window).to_bytes(8, "big")
|
||||
+ n.to_bytes(4, "big"))
|
||||
for _ in range(n):
|
||||
f = r.byte()
|
||||
if f & ~0x03 and not drops("form_reserved"):
|
||||
raise bad
|
||||
if f & 0x02:
|
||||
grp = group_at(ctx, r.take(8))
|
||||
group, leaf_count = grp["link"], grp["g"]["leaf_count"]
|
||||
else:
|
||||
group, leaf_count = r.take(32), r.uint()
|
||||
out += group + leaf_count.to_bytes(8, "big")
|
||||
if f & 0x01:
|
||||
out.append(1)
|
||||
else:
|
||||
out.append(0)
|
||||
out += r.take((leaf_count + 7) // 8)
|
||||
out += r.take(64)
|
||||
r.end()
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def expand(compact, ctx):
|
||||
"""(kind, record): a reconciliation's canonical text, or a holdings record's bytes."""
|
||||
if len(compact) > MAX_RECORD:
|
||||
raise Refused("compact_malformed")
|
||||
r = Reader(compact)
|
||||
kind = r.byte()
|
||||
if kind == KIND_RECONCILIATION:
|
||||
r.kind = "reconciliation"
|
||||
return "reconciliation", expand_reconciliation(r, ctx)
|
||||
if kind == KIND_HOLDINGS:
|
||||
r.kind = "holdings"
|
||||
return "holdings", expand_holdings(r, ctx)
|
||||
raise Refused("compact_malformed")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ §13.5: writing a record
|
||||
|
||||
def encode_reconciliation(text, ctx):
|
||||
no = Refused("compact_not_encodable")
|
||||
try:
|
||||
CV.verify_claim(text)
|
||||
except CV.Reject as e:
|
||||
if e.step <= 2: # steps 1 and 2: it does not read
|
||||
raise no
|
||||
c = CV.parse_strict(text)
|
||||
if c["class"] != "reconciliation" or c["subject_type"] != "receipt" or CV.jcs(c) != text:
|
||||
raise no
|
||||
grp = group_by_link(ctx, bytes.fromhex(c["subject"]))
|
||||
issued = instant_of(c["issued_at"])
|
||||
if grp is None or issued is None:
|
||||
raise no
|
||||
b, key = c["body"], bytes.fromhex(c["signer_pubkey"])
|
||||
# The claim's own leaf_count is the group's before anything is sized by it.
|
||||
if b["leaf_count"] != grp["g"]["leaf_count"]:
|
||||
raise no
|
||||
received = bytes.fromhex(b["received"])
|
||||
whole = received == full_bitmap(grp["g"]["leaf_count"])
|
||||
me = (key, c["signer_id"])
|
||||
try:
|
||||
by_ref = signer_at(ctx, ref(8, key)) == me and not drops("by_reference")
|
||||
except Refused:
|
||||
by_ref = False
|
||||
signer_form = 2 if by_ref else 0 if c["signer_id"] == CV.did_key(key) else 1
|
||||
form = BEARERS.index(b["bearer"]) | (signer_form << 3) | (0x04 if whole else 0) \
|
||||
| (0x20 if c["parent_receipt_hash"] is not None else 0)
|
||||
out = bytes([KIND_RECONCILIATION, form]) + ref(8, grp["link"]) + ref(4, b["bearer_id"].encode()) \
|
||||
+ put_uint(c["seq"]) + put_uint(issued)
|
||||
if signer_form == 0:
|
||||
out += key
|
||||
elif signer_form == 1:
|
||||
sid = c["signer_id"].encode()
|
||||
out += key + put_uint(len(sid)) + sid
|
||||
else:
|
||||
out += ref(8, key)
|
||||
if c["parent_receipt_hash"] is not None:
|
||||
out += bytes.fromhex(c["parent_receipt_hash"])
|
||||
if not whole:
|
||||
out += received
|
||||
return out + bytes.fromhex(c["sig"])
|
||||
|
||||
|
||||
def encode_holdings(b, ctx):
|
||||
try:
|
||||
holder, as_of, expires, entries, _ = HV.read(b)
|
||||
except HV.Refused:
|
||||
raise Refused("compact_not_encodable")
|
||||
try:
|
||||
holder_ref = key_at(ctx, ref(8, holder)) == holder and not drops("by_reference")
|
||||
except Refused:
|
||||
holder_ref = False
|
||||
out = bytes([KIND_HOLDINGS, 1 if holder_ref else 0]) + (ref(8, holder) if holder_ref else holder)
|
||||
out += put_uint(as_of) + put_uint(expires - as_of) + put_uint(len(entries))
|
||||
for group, leaf_count, bits in entries:
|
||||
try:
|
||||
grp = group_at(ctx, ref(8, group))
|
||||
by_ref = grp["link"] == group and grp["g"]["leaf_count"] == leaf_count and not drops("by_reference")
|
||||
except Refused:
|
||||
by_ref = False
|
||||
out += bytes([(1 if bits is None else 0) | (2 if by_ref else 0)])
|
||||
out += ref(8, group) if by_ref else group + put_uint(leaf_count)
|
||||
if bits is not None:
|
||||
out += bits
|
||||
return out + b[-64:]
|
||||
|
||||
|
||||
def encode(kind, record, ctx):
|
||||
"""§13.5: the record's compact form against the context, which must expand to the record."""
|
||||
out = encode_reconciliation(record, ctx) if kind == "reconciliation" else encode_holdings(record, ctx)
|
||||
try:
|
||||
back = expand(out, ctx)[1]
|
||||
except Refused:
|
||||
back = None
|
||||
if back != record and not drops("roundtrip"):
|
||||
raise Refused("compact_not_encodable")
|
||||
return out
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ §13.6, §13.7: fragments
|
||||
|
||||
def message(record):
|
||||
return record + check_of(record)
|
||||
|
||||
|
||||
def cut(record, number, frame):
|
||||
m = message(record)
|
||||
if frame <= HEADER or len(m) > MAX_MESSAGE:
|
||||
return None
|
||||
per = frame - HEADER
|
||||
parts = [m[i:i + per] for i in range(0, len(m), per)]
|
||||
if len(parts) > 255:
|
||||
return None
|
||||
return [bytes([number, i, len(parts)]) + d for i, d in enumerate(parts)]
|
||||
|
||||
|
||||
def reassemble(fragments):
|
||||
"""§13.7, in its order: the compact record the fragments carry. Fragments that are the same
|
||||
bytes are one fragment."""
|
||||
if not drops("held_once"):
|
||||
fragments = list(dict.fromkeys(fragments))
|
||||
if not fragments:
|
||||
raise Refused("fragment_missing")
|
||||
if not drops("header") and any(len(f) <= HEADER or f[2] == 0 or f[1] >= f[2] for f in fragments):
|
||||
raise Refused("fragment_malformed")
|
||||
number, count = fragments[0][0], fragments[0][2]
|
||||
if not drops("foreign") and any(f[0] != number or f[2] != count for f in fragments):
|
||||
raise Refused("fragment_foreign")
|
||||
slots = {}
|
||||
for f in fragments:
|
||||
if f[1] in slots and not drops("repeated"):
|
||||
raise Refused("fragment_repeated")
|
||||
slots[f[1]] = f[HEADER:]
|
||||
if any(i not in slots for i in range(count)) and not drops("missing"):
|
||||
raise Refused("fragment_missing")
|
||||
m = b"".join(slots.get(i, b"") for i in range(count))
|
||||
if len(m) > MAX_MESSAGE:
|
||||
raise Refused("fragment_malformed")
|
||||
if len(m) <= CHECK or (m[-CHECK:] != check_of(m[:-CHECK]) and not drops("check")):
|
||||
raise Refused("fragment_check_failed")
|
||||
return m[:-CHECK]
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ verdicts of the records
|
||||
|
||||
def verdict(kind, record, ctx):
|
||||
"""A reconciliation through jwp-receipts §3.1 steps 1 to 4, then step 6 by bearer-receipts
|
||||
§5.3 against the context's group and objects; a holdings record under holdings §4.4."""
|
||||
if kind == "holdings":
|
||||
try:
|
||||
return "read digest=" + HV.digest(HV.read(record)).hex()
|
||||
except HV.Refused as r:
|
||||
return "reject reason=" + r.reason
|
||||
try:
|
||||
c = CV.verify_claim(record)
|
||||
grp = group_by_link(ctx, bytes.fromhex(c["subject"]))
|
||||
if grp is None:
|
||||
return "reject step=6 reason=context_invalid"
|
||||
BV.check_reconciliation(c, grp["g"], grp["objects"])
|
||||
except CV.Reject as r:
|
||||
return f"reject step={r.step} reason={r.reason}"
|
||||
return "accept"
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ §13.8: the floors
|
||||
|
||||
def floors():
|
||||
"""§13.8's rows: (code, name, floor)."""
|
||||
rows = []
|
||||
for line in section("13.8 ").splitlines():
|
||||
if line.startswith("| ") and "`" in line:
|
||||
cells = [c.strip() for c in line.strip("|").split("|")]
|
||||
rows.append((int(cells[0].split(" ")[0]), cells[0].split("`")[1], int(cells[1].replace(",", ""))))
|
||||
return rows
|
||||
|
||||
|
||||
def measures():
|
||||
"""telemetry-aggregation §3's table: code -> measure."""
|
||||
out = {}
|
||||
for line in open(AGGREGATION, encoding="utf-8").read().splitlines():
|
||||
m = re.match(r"\| ([0-9]+) \| `([a-z_]+)` \|", line)
|
||||
if m:
|
||||
out[int(m.group(1))] = m.group(2)
|
||||
return out
|
||||
|
||||
|
||||
def report_lengths(upload):
|
||||
"""The length of each report of a DAP upload (UploadRequest: reports, one after another)."""
|
||||
r, out = TV.Rd(upload), []
|
||||
while not r.done():
|
||||
start = r.i
|
||||
TV.read_report(r)
|
||||
out.append(r.i - start)
|
||||
return out
|
||||
|
||||
|
||||
def check_floors():
|
||||
fails = []
|
||||
rows = floors()
|
||||
if [r[0] for r in rows] != [1, 2, 3, 4]:
|
||||
fails.append(f"§13.8 states the floors of measures {[r[0] for r in rows]}, not 1 to 4")
|
||||
named = measures()
|
||||
flows = {f["name"]: f for f in json.load(open(AGGREGATE_CORPUS, encoding="utf-8"))["flows"]}
|
||||
for code, name, floor in rows:
|
||||
if named.get(code) != name:
|
||||
fails.append(f"§13.8 names measure {code} {name}; telemetry-aggregation names it {named.get(code)}")
|
||||
lengths = set(report_lengths(bytes.fromhex(flows[name]["upload"])))
|
||||
if lengths != {floor}:
|
||||
fails.append(f"{name}: the reports are {sorted(lengths)} bytes, §13.8 states {floor}")
|
||||
return fails
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ the corpus
|
||||
|
||||
def record_of(kind, v):
|
||||
return v if kind == "reconciliation" else bytes.fromhex(v)
|
||||
|
||||
|
||||
def record_value(kind, record):
|
||||
return record if kind == "reconciliation" else record.hex()
|
||||
|
||||
|
||||
def refuse(reason):
|
||||
return {"outcome": "refuse", "reason": reason}
|
||||
|
||||
|
||||
def run(kind, c, doc):
|
||||
ctx = context_of(doc["contexts"][c["context"]], doc["groups"]) if "context" in c else None
|
||||
if kind == "records":
|
||||
record = record_of(c["kind"], c["record"])
|
||||
stated = bytes.fromhex(c["compact"])
|
||||
if encode(c["kind"], record, ctx) != stated:
|
||||
return {"error": "written otherwise"}
|
||||
if expand(stated, ctx) != (c["kind"], record):
|
||||
return {"error": "expands to another record"}
|
||||
return {"verdict": verdict(c["kind"], record, ctx)}
|
||||
if kind == "expansions":
|
||||
try:
|
||||
k, record = expand(bytes.fromhex(c["compact"]), ctx)
|
||||
except Refused as r:
|
||||
return refuse(r.reason)
|
||||
return {"outcome": "expand", "kind": k, "record": record_value(k, record), "verdict": verdict(k, record, ctx)}
|
||||
if kind == "encodings":
|
||||
try:
|
||||
return {"outcome": "encode", "compact": encode(c["kind"], record_of(c["kind"], c["record"]), ctx).hex()}
|
||||
except Refused as r:
|
||||
return refuse(r.reason)
|
||||
if kind == "messages":
|
||||
record = bytes.fromhex(c["record"])
|
||||
fragments = cut(record, c["message"], c["frame"])
|
||||
if reassemble(fragments) != record:
|
||||
return {"error": "its fragments do not reassemble to it"}
|
||||
return {"fragments": [f.hex() for f in fragments]}
|
||||
try:
|
||||
return {"outcome": "reassembled", "record": reassemble([bytes.fromhex(f) for f in c["fragments"]]).hex()}
|
||||
except Refused as r:
|
||||
return refuse(r.reason)
|
||||
|
||||
|
||||
KINDS = ("records", "expansions", "encodings", "messages", "reassemblies")
|
||||
|
||||
|
||||
def strings(v, out):
|
||||
if isinstance(v, str):
|
||||
out.append(v)
|
||||
elif isinstance(v, list):
|
||||
for x in v:
|
||||
strings(x, out)
|
||||
elif isinstance(v, dict):
|
||||
for x in v.values():
|
||||
strings(x, out)
|
||||
|
||||
|
||||
def check(doc, quiet=False):
|
||||
"""The failures of a corpus document: each case's outcome, and every reason of §13.9."""
|
||||
fails = []
|
||||
for kind in KINDS:
|
||||
for c in doc[kind]:
|
||||
try:
|
||||
got = run(kind, c, doc)
|
||||
except (Refused, KeyError, TypeError, ValueError, IndexError, OverflowError, AttributeError,
|
||||
MemoryError) as e:
|
||||
got = {"error": repr(e)}
|
||||
if got != c["outcome"]:
|
||||
fails.append(c["name"])
|
||||
if not quiet:
|
||||
print(f"FAIL {c['name']}\n stated {json.dumps(c['outcome'])[:300]}\n got {json.dumps(got)[:300]}")
|
||||
stated = []
|
||||
for kind in KINDS:
|
||||
for c in doc[kind]:
|
||||
strings(c["outcome"], stated)
|
||||
for code in reasons_table():
|
||||
if code not in stated:
|
||||
fails.append("no case gives " + code)
|
||||
return fails
|
||||
|
||||
|
||||
def mutations(doc):
|
||||
"""Copies of the corpus, one defect each, every one of which must fail."""
|
||||
def case(d, name):
|
||||
return next(c for k in KINDS for c in d[k] if c["name"] == name)
|
||||
|
||||
def m_compact_byte(d):
|
||||
c = case(d, "whole_did_key")
|
||||
b = bytearray.fromhex(c["compact"])
|
||||
b[20] ^= 0x01
|
||||
c["compact"] = b.hex()
|
||||
|
||||
def m_record(d):
|
||||
c = case(d, "partial_enrolled")
|
||||
c["record"] = c["record"].replace('"seq":0', '"seq":1')
|
||||
|
||||
def m_verdict(d):
|
||||
case(d, "labelled_label_id")["outcome"]["verdict"] = "reject step=6 reason=reconciliation_root_mismatch"
|
||||
|
||||
def m_holdings(d):
|
||||
c = case(d, "holdings_by_reference")
|
||||
c["context"] = "no_signers"
|
||||
|
||||
def m_context(d):
|
||||
case(d, "objects_not_held")["context"] = "receiver"
|
||||
|
||||
def m_reason(d):
|
||||
case(d, "bearer_ambiguous")["outcome"]["reason"] = "compact_reference_unknown"
|
||||
|
||||
def m_encodable(d):
|
||||
case(d, "issued_at_fraction")["record"] = case(d, "whole_did_key")["record"]
|
||||
|
||||
def m_frame(d):
|
||||
case(d, "whole_frame_11")["frame"] = 12
|
||||
|
||||
def m_number(d):
|
||||
case(d, "holdings_frame_51")["message"] = 6
|
||||
|
||||
def m_reassembly(d):
|
||||
case(d, "reordered_data_exchanged")["fragments"] = list(case(d, "in_order")["fragments"])
|
||||
|
||||
def m_missing_stated_whole(d):
|
||||
case(d, "missing_middle")["outcome"] = {"outcome": "reassembled", "record": case(d, "whole_did_key")["compact"]}
|
||||
|
||||
def m_reason_gone(d):
|
||||
d["expansions"].remove(case(d, "objects_not_held"))
|
||||
|
||||
def m_twice_refused(d):
|
||||
case(d, "delivered_twice")["outcome"] = {"outcome": "refuse", "reason": "fragment_repeated"}
|
||||
|
||||
def m_bearer_elsewhere(d):
|
||||
case(d, "bearer_of_other_groups")["context"] = "receiver"
|
||||
|
||||
return [m_compact_byte, m_record, m_verdict, m_holdings, m_context, m_reason, m_encodable, m_frame, m_number,
|
||||
m_reassembly, m_missing_stated_whole, m_reason_gone, m_twice_refused, m_bearer_elsewhere]
|
||||
|
||||
|
||||
WEAKENINGS = ["minimal_reconciliation", "minimal_holdings", "bound_reconciliation", "bound_holdings",
|
||||
"trailing_reconciliation", "trailing_holdings", "form_reserved", "issued_bound", "utf8", "did_key_form",
|
||||
"past_bits", "no_bits", "whole_spelling", "ambiguous", "own_group", "key_alone", "objects_held",
|
||||
"by_reference", "roundtrip", "header", "foreign", "held_once", "repeated", "missing", "check"]
|
||||
|
||||
|
||||
def main(argv):
|
||||
global DROPPED
|
||||
if len(argv) != 2:
|
||||
print(__doc__)
|
||||
return 2
|
||||
doc = json.load(open(os.path.join(argv[1], "vectors.json"), encoding="utf-8"))
|
||||
extra = sorted(set(os.listdir(argv[1])) - {"vectors.json", "README.md"})
|
||||
fails = check(doc)
|
||||
if extra:
|
||||
fails.append(f"files the corpus does not hold: {extra}")
|
||||
# The bounds no vector carries (65 KB of hexadecimal): checked here.
|
||||
try:
|
||||
expand(bytes([KIND_HOLDINGS]) * (MAX_RECORD + 1), {"groups": [], "signers": []})
|
||||
fails.append("a compact record past 65,531 bytes was read")
|
||||
except Refused as r:
|
||||
if r.reason != "compact_malformed":
|
||||
fails.append("a compact record past the bound was refused " + r.reason)
|
||||
big = [bytes([7, i, 255]) + bytes(258) for i in range(255)]
|
||||
try:
|
||||
reassemble(big)
|
||||
fails.append("a message past 65,535 bytes was reassembled")
|
||||
except Refused as r:
|
||||
if r.reason != "fragment_malformed":
|
||||
fails.append("a message past the bound was refused " + r.reason)
|
||||
if cut(bytes(MAX_RECORD), 7, 1500) is None or cut(bytes(MAX_RECORD + 1), 7, 1500) is not None:
|
||||
fails.append("a message is cut up to 65,535 bytes and no further")
|
||||
fails += check_floors()
|
||||
print(f"{sum(len(doc[k]) for k in KINDS)} cases ({', '.join(f'{len(doc[k])} {k}' for k in KINDS)}): "
|
||||
f"{len(fails)} failure(s)")
|
||||
caught = 0
|
||||
for m in mutations(doc):
|
||||
d = copy.deepcopy(doc)
|
||||
m(d)
|
||||
if check(d, quiet=True):
|
||||
caught += 1
|
||||
else:
|
||||
fails.append(f"mutation {m.__name__} was not caught")
|
||||
print(f"{caught} of {len(mutations(doc))} mutated copies fail")
|
||||
weak = 0
|
||||
for w in WEAKENINGS:
|
||||
DROPPED = w
|
||||
if check(doc, quiet=True):
|
||||
weak += 1
|
||||
else:
|
||||
fails.append(f"a verifier without its {w} check passes the corpus")
|
||||
DROPPED = None
|
||||
print(f"{weak} of {len(WEAKENINGS)} weakened verifiers are caught")
|
||||
for f in fails:
|
||||
print("FAIL", f)
|
||||
return 1 if fails else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
|
|
@ -1414,6 +1414,11 @@ name = "wai_bearer_vectors"
|
|||
path = "src/bin/wai_bearer_vectors.rs"
|
||||
required-features = ["bearer_receipts"]
|
||||
|
||||
[[bin]]
|
||||
name = "wai_compact_return_vectors"
|
||||
path = "src/bin/wai_compact_return_vectors.rs"
|
||||
required-features = ["bearer_receipts", "holdings"]
|
||||
|
||||
[[bin]]
|
||||
name = "wai_carousel_vectors"
|
||||
path = "src/bin/wai_carousel_vectors.rs"
|
||||
|
|
@ -2491,6 +2496,11 @@ name = "bearer_receipts"
|
|||
path = "tests/bearer_receipts.rs"
|
||||
required-features = ["bearer_receipts"]
|
||||
|
||||
[[test]]
|
||||
name = "compact_return"
|
||||
path = "tests/compact_return.rs"
|
||||
required-features = ["bearer_receipts", "holdings", "telemetry_aggregate"]
|
||||
|
||||
[[test]]
|
||||
name = "carousel"
|
||||
path = "tests/carousel.rs"
|
||||
|
|
|
|||
75
wai/wai-rs/src/bin/wai_compact_return_vectors.rs
Normal file
75
wai/wai-rs/src/bin/wai_compact_return_vectors.rs
Normal file
|
|
@ -0,0 +1,75 @@
|
|||
//! Conformance corpus for the compact return profile (`live-conformance/compact-return/`,
|
||||
//! extensions/bearer-receipts.md §13.10).
|
||||
//!
|
||||
//! wai_compact_return_vectors gen <corpus-dir> write vectors.json, from fixed seeds
|
||||
//! wai_compact_return_vectors verify <corpus-dir> every file regenerates byte for byte, no
|
||||
//! other file but README.md is there, and
|
||||
//! every case gets the outcome it states
|
||||
//!
|
||||
//! The generator and this verifier are one implementation. The corpus's second
|
||||
//! implementation is `live-conformance/compact_return_verify.py`, written from the extension's
|
||||
//! text and sharing no code with this binary.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::path::Path;
|
||||
use wai::compact_return::corpus;
|
||||
|
||||
fn main() {
|
||||
let a: Vec<String> = std::env::args().collect();
|
||||
match (a.get(1).map(String::as_str), a.get(2)) {
|
||||
(Some("gen"), Some(d)) => {
|
||||
if let Err(e) = std::fs::create_dir_all(d) {
|
||||
eprintln!("create {d}: {e}");
|
||||
std::process::exit(1);
|
||||
}
|
||||
for (name, body) in corpus::generate() {
|
||||
let p = Path::new(d).join(&name);
|
||||
if let Err(e) = std::fs::write(&p, body) {
|
||||
eprintln!("write {}: {e}", p.display());
|
||||
std::process::exit(1);
|
||||
}
|
||||
}
|
||||
println!("wrote {d}");
|
||||
}
|
||||
(Some("verify"), Some(d)) => std::process::exit(verify(Path::new(d))),
|
||||
_ => {
|
||||
eprintln!("usage: wai_compact_return_vectors gen|verify <corpus-dir>");
|
||||
std::process::exit(2);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn verify(dir: &Path) -> i32 {
|
||||
let mut files = BTreeMap::new();
|
||||
let entries = match std::fs::read_dir(dir) {
|
||||
Ok(e) => e,
|
||||
Err(e) => {
|
||||
eprintln!("read {}: {e}", dir.display());
|
||||
return 1;
|
||||
}
|
||||
};
|
||||
for e in entries.flatten() {
|
||||
let name = e.file_name().to_string_lossy().into_owned();
|
||||
if name == "README.md" {
|
||||
continue;
|
||||
}
|
||||
match std::fs::read_to_string(e.path()) {
|
||||
Ok(s) => {
|
||||
files.insert(name, s);
|
||||
}
|
||||
Err(err) => {
|
||||
eprintln!("read {name}: {err}");
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
let generated: BTreeMap<String, String> = corpus::generate().into_iter().collect();
|
||||
let same = files == generated;
|
||||
println!("{} every file regenerates byte for byte from its seeds, and no other file is there", if same { "PASS" } else { "FAIL" });
|
||||
let fails = corpus::verify(&files);
|
||||
for f in &fails {
|
||||
println!("FAIL {f}");
|
||||
}
|
||||
println!("{} every case gets the outcome the corpus states", if fails.is_empty() { "PASS" } else { "FAIL" });
|
||||
(!same || !fails.is_empty()) as i32
|
||||
}
|
||||
682
wai/wai-rs/src/compact_return.rs
Normal file
682
wai/wai-rs/src/compact_return.rs
Normal file
|
|
@ -0,0 +1,682 @@
|
|||
//! The compact return profile — `extensions/bearer-receipts.md` §13: a reconciliation (§5) and a
|
||||
//! holdings record (`extensions/holdings.md` §4) in a binary form for return paths whose messages
|
||||
//! hold few bytes, carried in fragments with a check over the whole message.
|
||||
//!
|
||||
//! A compact record names by short references (§13.2) what its receiver already holds — the
|
||||
//! group, the bearer instance, the signer — in a [`Context`]. [`expand`] turns it, against that
|
||||
//! context, into the record's own bytes: a reconciliation claim's canonical form, a holdings
|
||||
//! record's `WHD1`. [`encode`] is its inverse for a record that has a compact form (§13.5), and
|
||||
//! refuses one that has none. The expansion is verified by the record's own text — `claim_receipt`
|
||||
//! and bearer-receipts §5.3 for a reconciliation, [`crate::holdings`] for a holdings record — and
|
||||
//! never here. [`cut`] and [`reassemble`] carry a message in fragments (§13.6, §13.7).
|
||||
//!
|
||||
//! # Rules enforced here rather than only documented
|
||||
//!
|
||||
//! - A reference resolves to one value or to none: two values with one reference refuse it
|
||||
//! (`compact_reference_ambiguous`), never resolved by choosing.
|
||||
//! - A compact record has one reading and each form one spelling: a uint in its fewest bytes, a
|
||||
//! signer written inline with its own `did:key` is form 0 and never form 1, and a partial
|
||||
//! bitmap that sets every bit is the whole form's, so a reader refuses the other spelling.
|
||||
//! - [`encode`] expands what it wrote against the same context and refuses unless that is the
|
||||
//! record it was given, byte for byte: what it returns round-trips.
|
||||
//! - The check over a message is not a signature: whatever a message expands to is verified by
|
||||
//! its own signature, and the compact form carries no statement of its own.
|
||||
//! - Fragments that are the same bytes are one fragment, so a fragment the path delivers twice is
|
||||
//! held once; two that differ at one index refuse the message (`fragment_repeated`).
|
||||
//! - [`encode`] sizes nothing by a reconciliation's own `leaf_count` before it has found it to be
|
||||
//! the group's: a record received from a neighbour states it, and steps 1 and 2 bound it only by
|
||||
//! 2^53 − 1.
|
||||
|
||||
use std::collections::BTreeSet;
|
||||
|
||||
use sha2::{Digest, Sha256};
|
||||
|
||||
use crate::claim_receipt::bearer::ATTESTS_RECEIVED;
|
||||
use crate::claim_receipt::{rfc3339_seconds, rfc3339_utc, Bearer, ClaimBody, ClaimReceipt, ReconciliationClaim, SubjectType};
|
||||
use crate::holdings::{self, Holdings};
|
||||
use crate::jwp_delivery::{root as merkle_root, GroupReceipt, ObjectReceipt};
|
||||
use crate::signer_id::did_key;
|
||||
|
||||
pub mod corpus;
|
||||
|
||||
/// The codes bearer-receipts §13.9 lists, in its order.
|
||||
pub const CODES: &[&str] = &[
|
||||
MALFORMED,
|
||||
REFERENCE_UNKNOWN,
|
||||
REFERENCE_AMBIGUOUS,
|
||||
OBJECTS_NOT_HELD,
|
||||
NOT_ENCODABLE,
|
||||
FRAGMENT_MALFORMED,
|
||||
FRAGMENT_FOREIGN,
|
||||
FRAGMENT_REPEATED,
|
||||
FRAGMENT_MISSING,
|
||||
FRAGMENT_CHECK_FAILED,
|
||||
];
|
||||
|
||||
pub const MALFORMED: &str = "compact_malformed";
|
||||
pub const REFERENCE_UNKNOWN: &str = "compact_reference_unknown";
|
||||
pub const REFERENCE_AMBIGUOUS: &str = "compact_reference_ambiguous";
|
||||
pub const OBJECTS_NOT_HELD: &str = "compact_objects_not_held";
|
||||
pub const NOT_ENCODABLE: &str = "compact_not_encodable";
|
||||
pub const FRAGMENT_MALFORMED: &str = "fragment_malformed";
|
||||
pub const FRAGMENT_FOREIGN: &str = "fragment_foreign";
|
||||
pub const FRAGMENT_REPEATED: &str = "fragment_repeated";
|
||||
pub const FRAGMENT_MISSING: &str = "fragment_missing";
|
||||
pub const FRAGMENT_CHECK_FAILED: &str = "fragment_check_failed";
|
||||
|
||||
/// §13.3: a compact reconciliation's `kind`.
|
||||
pub const KIND_RECONCILIATION: u8 = 0x01;
|
||||
/// §13.4: a compact holdings record's `kind`.
|
||||
pub const KIND_HOLDINGS: u8 = 0x02;
|
||||
/// §13.6: the bytes of a fragment before its data.
|
||||
pub const HEADER: usize = 3;
|
||||
/// §13.1: the bytes of a message's check.
|
||||
pub const CHECK: usize = 4;
|
||||
/// §13.6: the most bytes a message has.
|
||||
pub const MAX_MESSAGE: usize = 65_535;
|
||||
/// §13.3, §13.4: the most bytes a compact record has, so that its message is at most
|
||||
/// [`MAX_MESSAGE`].
|
||||
pub const MAX_RECORD: usize = MAX_MESSAGE - CHECK;
|
||||
/// §13.2: the greatest instant, 9999-12-31T23:59:59Z.
|
||||
pub const MAX_SECONDS: u64 = 253_402_300_799;
|
||||
/// §13.2: the greatest uint, 2^53 − 1.
|
||||
pub const MAX_INT: u64 = (1 << 53) - 1;
|
||||
|
||||
// ───────────────────────────── §13.2 ─────────────────────────────
|
||||
|
||||
/// The first `N` bytes of SHA-256 over `bytes`: a reference (§13.2), and with `N` = 4 a
|
||||
/// message's check (§13.1).
|
||||
pub fn reference<const N: usize>(bytes: &[u8]) -> [u8; N] {
|
||||
Sha256::digest(bytes)[..N].try_into().expect("N is at most 32")
|
||||
}
|
||||
|
||||
/// A group's reference: 8 bytes over its link hash.
|
||||
pub fn group_ref(link_hash: &[u8; 32]) -> [u8; 8] {
|
||||
reference(link_hash)
|
||||
}
|
||||
|
||||
/// A bearer instance's reference: 4 bytes over its `bearer_id` in UTF-8.
|
||||
pub fn bearer_ref(bearer_id: &str) -> [u8; 4] {
|
||||
reference(bearer_id.as_bytes())
|
||||
}
|
||||
|
||||
/// A signer's reference: 8 bytes over its key.
|
||||
pub fn signer_ref(key: &[u8; 32]) -> [u8; 8] {
|
||||
reference(key)
|
||||
}
|
||||
|
||||
/// A message's check (§13.1): the first 4 bytes of SHA-256 over its compact record.
|
||||
pub fn check(record: &[u8]) -> [u8; CHECK] {
|
||||
reference(record)
|
||||
}
|
||||
|
||||
/// Write `v` as a uint (§13.2): unsigned LEB128 in its fewest bytes.
|
||||
pub fn put_uint(out: &mut Vec<u8>, mut v: u64) {
|
||||
loop {
|
||||
let b = (v & 0x7f) as u8;
|
||||
v >>= 7;
|
||||
if v == 0 {
|
||||
out.push(b);
|
||||
return;
|
||||
}
|
||||
out.push(b | 0x80);
|
||||
}
|
||||
}
|
||||
|
||||
/// The instant of an `issued_at` (§13.2): seconds since 1970-01-01T00:00:00Z, for a text written
|
||||
/// `YYYY-MM-DDTHH:MM:SSZ` with a second below 60 and at most [`MAX_SECONDS`]; `None` for any other
|
||||
/// text, which has no instant.
|
||||
pub fn instant_of(text: &str) -> Option<u64> {
|
||||
let s = u64::try_from(rfc3339_seconds(text)?).ok()?;
|
||||
(s <= MAX_SECONDS && rfc3339_utc(s) == text).then_some(s)
|
||||
}
|
||||
|
||||
struct Reader<'a> {
|
||||
b: &'a [u8],
|
||||
at: usize,
|
||||
}
|
||||
|
||||
impl<'a> Reader<'a> {
|
||||
fn take(&mut self, n: usize) -> Result<&'a [u8], &'static str> {
|
||||
let s = self.at.checked_add(n).and_then(|e| self.b.get(self.at..e)).ok_or(MALFORMED)?;
|
||||
self.at += n;
|
||||
Ok(s)
|
||||
}
|
||||
fn byte(&mut self) -> Result<u8, &'static str> {
|
||||
Ok(self.take(1)?[0])
|
||||
}
|
||||
fn array<const N: usize>(&mut self) -> Result<[u8; N], &'static str> {
|
||||
Ok(self.take(N)?.try_into().expect("N bytes"))
|
||||
}
|
||||
/// §13.2: a uint in its fewest bytes, at most 2^53 − 1, so at most 8 bytes.
|
||||
fn uint(&mut self) -> Result<u64, &'static str> {
|
||||
let mut v = 0u64;
|
||||
for i in 0..8 {
|
||||
let b = self.byte()?;
|
||||
v |= u64::from(b & 0x7f) << (7 * i);
|
||||
if b & 0x80 == 0 {
|
||||
return if (i > 0 && b == 0) || v > MAX_INT { Err(MALFORMED) } else { Ok(v) };
|
||||
}
|
||||
}
|
||||
Err(MALFORMED)
|
||||
}
|
||||
fn end(&self) -> Result<(), &'static str> {
|
||||
if self.at == self.b.len() {
|
||||
Ok(())
|
||||
} else {
|
||||
Err(MALFORMED)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ───────────────────────────── §13.1: the context ─────────────────────────────
|
||||
|
||||
/// One group of a receiver's context (§13.1).
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct ContextGroup {
|
||||
pub receipt: GroupReceipt,
|
||||
/// Its link hash, computed once.
|
||||
pub link: [u8; 32],
|
||||
/// The object receipts held under it that pass jwp-receipts §3 step 5 against it.
|
||||
pub objects: Vec<ObjectReceipt>,
|
||||
/// The `bearer_id` of each instance of the group the receiver holds a declaration of.
|
||||
pub bearers: Vec<String>,
|
||||
}
|
||||
|
||||
/// A key a receiver holds, with its `signer_id` (§13.1).
|
||||
#[derive(Clone, Debug, PartialEq, Eq, PartialOrd, Ord)]
|
||||
pub struct Signer {
|
||||
pub key: [u8; 32],
|
||||
pub signer_id: String,
|
||||
}
|
||||
|
||||
/// What a receiver holds that a compact record names (§13.1).
|
||||
#[derive(Clone, Debug, Default)]
|
||||
pub struct Context {
|
||||
pub groups: Vec<ContextGroup>,
|
||||
pub signers: Vec<Signer>,
|
||||
}
|
||||
|
||||
/// Of the values found under one reference, the one value they are, or why not (§13.2):
|
||||
/// values with one key are one value.
|
||||
fn one<T, K: Ord>(found: Vec<T>, key: impl Fn(&T) -> K) -> Result<T, &'static str> {
|
||||
let distinct: BTreeSet<K> = found.iter().map(key).collect();
|
||||
match distinct.len() {
|
||||
0 => Err(REFERENCE_UNKNOWN),
|
||||
1 => Ok(found.into_iter().next().expect("one value")),
|
||||
_ => Err(REFERENCE_AMBIGUOUS),
|
||||
}
|
||||
}
|
||||
|
||||
impl Context {
|
||||
/// Add a group, with the object receipts and bearer instances held under it. A group whose
|
||||
/// receipt fails jwp-receipts §3 steps 1 and 2 is not held, nor is an object receipt that
|
||||
/// fails step 5 against it; a group already held gains the objects and instances.
|
||||
pub fn add_group(&mut self, receipt: GroupReceipt, objects: Vec<ObjectReceipt>, bearers: Vec<String>) {
|
||||
if receipt.verify().is_err() {
|
||||
return;
|
||||
}
|
||||
let objects: Vec<ObjectReceipt> = objects.into_iter().filter(|o| o.verify_inclusion(&receipt).is_ok()).collect();
|
||||
let link = receipt.link_hash();
|
||||
match self.groups.iter_mut().find(|g| g.link == link) {
|
||||
Some(g) => {
|
||||
g.objects.extend(objects);
|
||||
g.bearers.extend(bearers);
|
||||
}
|
||||
None => self.groups.push(ContextGroup { receipt, link, objects, bearers }),
|
||||
}
|
||||
}
|
||||
|
||||
/// The group a reference resolves to.
|
||||
pub fn group(&self, r: &[u8; 8]) -> Result<&ContextGroup, &'static str> {
|
||||
one(self.groups.iter().filter(|g| group_ref(&g.link) == *r).collect(), |g| g.link)
|
||||
}
|
||||
|
||||
/// The group whose link hash is `link`, if the context holds it.
|
||||
pub fn group_by_link(&self, link: &[u8; 32]) -> Option<&ContextGroup> {
|
||||
self.groups.iter().find(|g| g.link == *link)
|
||||
}
|
||||
|
||||
/// The signer, key and `signer_id`, a reference resolves to: for a reconciliation.
|
||||
pub fn signer(&self, r: &[u8; 8]) -> Result<&Signer, &'static str> {
|
||||
one(self.signers.iter().filter(|s| signer_ref(&s.key) == *r).collect(), |s| (s.key, s.signer_id.clone()))
|
||||
}
|
||||
|
||||
/// The key a reference resolves to: for a holdings record, which names no `signer_id`.
|
||||
pub fn key(&self, r: &[u8; 8]) -> Result<[u8; 32], &'static str> {
|
||||
one(self.signers.iter().filter(|s| signer_ref(&s.key) == *r).map(|s| s.key).collect(), |k| *k)
|
||||
}
|
||||
}
|
||||
|
||||
impl ContextGroup {
|
||||
/// The bearer instance of this group a reference resolves to.
|
||||
pub fn bearer(&self, r: &[u8; 4]) -> Result<&str, &'static str> {
|
||||
one(self.bearers.iter().filter(|b| bearer_ref(b) == *r).collect(), |b| b.to_string()).map(|b| b.as_str())
|
||||
}
|
||||
}
|
||||
|
||||
// ───────────────────────────── records ─────────────────────────────
|
||||
|
||||
/// A record in its own form: what a compact record expands to.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub enum Record {
|
||||
/// A reconciliation claim's canonical form (jwp-receipts §2.4.3).
|
||||
Reconciliation(String),
|
||||
/// A holdings record's bytes (holdings §4).
|
||||
Holdings(Vec<u8>),
|
||||
}
|
||||
|
||||
impl Record {
|
||||
pub fn bytes(&self) -> &[u8] {
|
||||
match self {
|
||||
Record::Reconciliation(s) => s.as_bytes(),
|
||||
Record::Holdings(b) => b,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// §13.3, §13.4: a compact record, expanded against `ctx` into its record's own bytes.
|
||||
pub fn expand(compact: &[u8], ctx: &Context) -> Result<Record, &'static str> {
|
||||
if compact.len() > MAX_RECORD {
|
||||
return Err(MALFORMED);
|
||||
}
|
||||
let mut r = Reader { b: compact, at: 0 };
|
||||
match r.byte()? {
|
||||
KIND_RECONCILIATION => expand_reconciliation(&mut r, ctx).map(Record::Reconciliation),
|
||||
KIND_HOLDINGS => expand_holdings(&mut r, ctx).map(Record::Holdings),
|
||||
_ => Err(MALFORMED),
|
||||
}
|
||||
}
|
||||
|
||||
/// §13.5: a record's compact form against `ctx`, or `compact_not_encodable`.
|
||||
pub fn encode(record: &Record, ctx: &Context) -> Result<Vec<u8>, &'static str> {
|
||||
let out = match record {
|
||||
Record::Reconciliation(text) => encode_reconciliation(text, ctx)?,
|
||||
Record::Holdings(bytes) => encode_holdings(bytes, ctx)?,
|
||||
};
|
||||
// What it wrote is the record's only if it expands to it, here and now.
|
||||
match expand(&out, ctx) {
|
||||
Ok(back) if back == *record => Ok(out),
|
||||
_ => Err(NOT_ENCODABLE),
|
||||
}
|
||||
}
|
||||
|
||||
/// The bitmap of `leaf_count` objects that holds every one.
|
||||
fn full_bitmap(leaf_count: u64) -> Vec<u8> {
|
||||
let mut b = vec![0xffu8; leaf_count.div_ceil(8) as usize];
|
||||
let used = (leaf_count % 8) as u32;
|
||||
if used != 0 {
|
||||
*b.last_mut().expect("a byte") = 0xffu8 << (8 - used);
|
||||
}
|
||||
b
|
||||
}
|
||||
|
||||
/// §13.3: a partial `received`'s root, over the leaves of the object receipts the context holds
|
||||
/// at its set bits, in increasing order.
|
||||
fn received_root(g: &ContextGroup, received: &[u8]) -> Result<[u8; 32], &'static str> {
|
||||
let mut leaves = Vec::new();
|
||||
for i in 0..g.receipt.leaf_count {
|
||||
if received[(i / 8) as usize] & (0x80 >> (i % 8)) != 0 {
|
||||
let o = g.objects.iter().find(|o| o.object_id == i).ok_or(OBJECTS_NOT_HELD)?;
|
||||
leaves.push(o.leaf());
|
||||
}
|
||||
}
|
||||
Ok(merkle_root(&leaves))
|
||||
}
|
||||
|
||||
fn expand_reconciliation(r: &mut Reader, ctx: &Context) -> Result<String, &'static str> {
|
||||
let form = r.byte()?;
|
||||
let bearer = match form & 0x03 {
|
||||
0 => Bearer::Bidirectional,
|
||||
1 => Bearer::Multicast,
|
||||
2 => Bearer::Unidirectional,
|
||||
_ => return Err(MALFORMED),
|
||||
};
|
||||
let whole = form & 0x04 != 0;
|
||||
let signer_form = (form >> 3) & 0x03;
|
||||
if form & 0xc0 != 0 || signer_form == 3 {
|
||||
return Err(MALFORMED);
|
||||
}
|
||||
let g = ctx.group(&r.array()?)?;
|
||||
let bearer_id = g.bearer(&r.array()?)?.to_string();
|
||||
let seq = r.uint()?;
|
||||
let issued_at = r.uint()?;
|
||||
if issued_at > MAX_SECONDS {
|
||||
return Err(MALFORMED);
|
||||
}
|
||||
let (signer_pubkey, signer_id) = match signer_form {
|
||||
0 => {
|
||||
let key: [u8; 32] = r.array()?;
|
||||
(key, did_key(&key))
|
||||
}
|
||||
1 => {
|
||||
let key: [u8; 32] = r.array()?;
|
||||
let n = usize::try_from(r.uint()?).map_err(|_| MALFORMED)?;
|
||||
let id = std::str::from_utf8(r.take(n)?).map_err(|_| MALFORMED)?.to_string();
|
||||
if id == did_key(&key) {
|
||||
return Err(MALFORMED);
|
||||
}
|
||||
(key, id)
|
||||
}
|
||||
_ => {
|
||||
let s = ctx.signer(&r.array()?)?;
|
||||
(s.key, s.signer_id.clone())
|
||||
}
|
||||
};
|
||||
let parent_receipt_hash = if form & 0x20 != 0 { Some(r.array::<32>()?) } else { None };
|
||||
let leaf_count = g.receipt.leaf_count;
|
||||
let received = if whole {
|
||||
full_bitmap(leaf_count)
|
||||
} else {
|
||||
let b = r.take(leaf_count.div_ceil(8) as usize)?.to_vec();
|
||||
let used = (leaf_count % 8) as u32;
|
||||
let past = used != 0 && b.last().is_some_and(|x| x & (0xffu8 >> used) != 0);
|
||||
if past || b.iter().all(|x| *x == 0) || b == full_bitmap(leaf_count) {
|
||||
return Err(MALFORMED);
|
||||
}
|
||||
b
|
||||
};
|
||||
let sig: [u8; 64] = r.array()?;
|
||||
r.end()?;
|
||||
let root = if whole { g.receipt.root_hash } else { received_root(g, &received)? };
|
||||
let claim = ClaimReceipt {
|
||||
subject_type: SubjectType::Receipt,
|
||||
subject: g.link,
|
||||
seq,
|
||||
issued_at: rfc3339_utc(issued_at),
|
||||
body: ClaimBody::Reconciliation(ReconciliationClaim {
|
||||
bearer,
|
||||
bearer_id,
|
||||
leaf_count,
|
||||
received,
|
||||
received_root: root,
|
||||
attests: ATTESTS_RECEIVED.into(),
|
||||
}),
|
||||
signer_pubkey,
|
||||
signer_id,
|
||||
parent_receipt_hash,
|
||||
sig,
|
||||
hybrid: None,
|
||||
};
|
||||
Ok(claim.to_json())
|
||||
}
|
||||
|
||||
fn expand_holdings(r: &mut Reader, ctx: &Context) -> Result<Vec<u8>, &'static str> {
|
||||
let holder: [u8; 32] = match r.byte()? {
|
||||
0 => r.array()?,
|
||||
1 => ctx.key(&r.array()?)?,
|
||||
_ => return Err(MALFORMED),
|
||||
};
|
||||
let as_of_ms = r.uint()?;
|
||||
let window_ms = r.uint()?;
|
||||
let n = u32::try_from(r.uint()?).map_err(|_| MALFORMED)?;
|
||||
let mut out = Vec::new();
|
||||
out.extend_from_slice(holdings::MAGIC);
|
||||
out.push(1);
|
||||
out.extend_from_slice(&holder);
|
||||
out.extend_from_slice(&as_of_ms.to_be_bytes());
|
||||
out.extend_from_slice(&(as_of_ms + window_ms).to_be_bytes());
|
||||
out.extend_from_slice(&n.to_be_bytes());
|
||||
for _ in 0..n {
|
||||
let form = r.byte()?;
|
||||
if form & !0x03 != 0 {
|
||||
return Err(MALFORMED);
|
||||
}
|
||||
let (group, leaf_count) = if form & 0x02 != 0 {
|
||||
let g = ctx.group(&r.array()?)?;
|
||||
(g.link, g.receipt.leaf_count)
|
||||
} else {
|
||||
(r.array::<32>()?, r.uint()?)
|
||||
};
|
||||
out.extend_from_slice(&group);
|
||||
out.extend_from_slice(&leaf_count.to_be_bytes());
|
||||
if form & 0x01 != 0 {
|
||||
out.push(1);
|
||||
} else {
|
||||
out.push(0);
|
||||
out.extend_from_slice(r.take(leaf_count.div_ceil(8) as usize)?);
|
||||
}
|
||||
}
|
||||
out.extend_from_slice(&r.array::<64>()?);
|
||||
r.end()?;
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn encode_reconciliation(text: &str, ctx: &Context) -> Result<Vec<u8>, &'static str> {
|
||||
let no = Err(NOT_ENCODABLE);
|
||||
let Ok(c) = ClaimReceipt::from_json(text) else { return no };
|
||||
let ClaimBody::Reconciliation(b) = &c.body else { return no };
|
||||
if c.hybrid.is_some() || c.subject_type != SubjectType::Receipt || c.to_json() != text {
|
||||
return no;
|
||||
}
|
||||
let Some(g) = ctx.group_by_link(&c.subject) else { return no };
|
||||
let Some(issued_at) = instant_of(&c.issued_at) else { return no };
|
||||
// The claim's own leaf_count is the group's before anything is sized by it.
|
||||
if b.leaf_count != g.receipt.leaf_count {
|
||||
return no;
|
||||
}
|
||||
let whole = b.received == full_bitmap(g.receipt.leaf_count);
|
||||
let me = Signer { key: c.signer_pubkey, signer_id: c.signer_id.clone() };
|
||||
let signer_form = if ctx.signer(&signer_ref(&me.key)) == Ok(&me) {
|
||||
2
|
||||
} else if c.signer_id == did_key(&c.signer_pubkey) {
|
||||
0
|
||||
} else {
|
||||
1
|
||||
};
|
||||
let bearer: u8 = match b.bearer {
|
||||
Bearer::Bidirectional => 0,
|
||||
Bearer::Multicast => 1,
|
||||
Bearer::Unidirectional => 2,
|
||||
};
|
||||
let mut form = bearer | (signer_form << 3);
|
||||
if whole {
|
||||
form |= 0x04;
|
||||
}
|
||||
if c.parent_receipt_hash.is_some() {
|
||||
form |= 0x20;
|
||||
}
|
||||
let mut out = vec![KIND_RECONCILIATION, form];
|
||||
out.extend_from_slice(&group_ref(&g.link));
|
||||
out.extend_from_slice(&bearer_ref(&b.bearer_id));
|
||||
put_uint(&mut out, c.seq);
|
||||
put_uint(&mut out, issued_at);
|
||||
match signer_form {
|
||||
0 => out.extend_from_slice(&c.signer_pubkey),
|
||||
1 => {
|
||||
out.extend_from_slice(&c.signer_pubkey);
|
||||
put_uint(&mut out, c.signer_id.len() as u64);
|
||||
out.extend_from_slice(c.signer_id.as_bytes());
|
||||
}
|
||||
_ => out.extend_from_slice(&signer_ref(&c.signer_pubkey)),
|
||||
}
|
||||
if let Some(p) = &c.parent_receipt_hash {
|
||||
out.extend_from_slice(p);
|
||||
}
|
||||
if !whole {
|
||||
out.extend_from_slice(&b.received);
|
||||
}
|
||||
out.extend_from_slice(&c.sig);
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn encode_holdings(bytes: &[u8], ctx: &Context) -> Result<Vec<u8>, &'static str> {
|
||||
let h = Holdings::read(bytes).map_err(|_| NOT_ENCODABLE)?;
|
||||
let mut out = vec![KIND_HOLDINGS];
|
||||
if ctx.key(&signer_ref(&h.holder)) == Ok(h.holder) {
|
||||
out.push(1);
|
||||
out.extend_from_slice(&signer_ref(&h.holder));
|
||||
} else {
|
||||
out.push(0);
|
||||
out.extend_from_slice(&h.holder);
|
||||
}
|
||||
put_uint(&mut out, h.as_of_ms);
|
||||
put_uint(&mut out, h.expires_ms - h.as_of_ms);
|
||||
put_uint(&mut out, h.entries.len() as u64);
|
||||
for e in &h.entries {
|
||||
let by_ref = ctx.group(&group_ref(&e.group)).is_ok_and(|g| g.link == e.group && g.receipt.leaf_count == e.leaf_count);
|
||||
let whole = e.held == holdings::Held::Whole;
|
||||
out.push(u8::from(whole) | if by_ref { 0x02 } else { 0 });
|
||||
if by_ref {
|
||||
out.extend_from_slice(&group_ref(&e.group));
|
||||
} else {
|
||||
out.extend_from_slice(&e.group);
|
||||
put_uint(&mut out, e.leaf_count);
|
||||
}
|
||||
if let holdings::Held::Bitmap(bits) = &e.held {
|
||||
out.extend_from_slice(bits);
|
||||
}
|
||||
}
|
||||
out.extend_from_slice(&h.sig);
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
// ───────────────────────────── §13.6, §13.7: fragments ─────────────────────────────
|
||||
|
||||
/// A compact record followed by its check (§13.1).
|
||||
pub fn message(record: &[u8]) -> Vec<u8> {
|
||||
[record, &check(record)[..]].concat()
|
||||
}
|
||||
|
||||
/// §13.6: a compact record's message cut, in order, into fragments of `frame` bytes, the last
|
||||
/// holding what remains, under the sender's number `number`. `None` when the path cannot carry
|
||||
/// it: a frame below 4 bytes, a message over [`MAX_MESSAGE`], or one that needs more than 255
|
||||
/// fragments.
|
||||
pub fn cut(record: &[u8], number: u8, frame: usize) -> Option<Vec<Vec<u8>>> {
|
||||
let m = message(record);
|
||||
if frame <= HEADER || m.len() > MAX_MESSAGE {
|
||||
return None;
|
||||
}
|
||||
let chunks: Vec<&[u8]> = m.chunks(frame - HEADER).collect();
|
||||
let count = u8::try_from(chunks.len()).ok()?;
|
||||
Some(chunks.iter().enumerate().map(|(i, d)| [&[number, i as u8, count][..], d].concat()).collect())
|
||||
}
|
||||
|
||||
/// §13.7: the compact record a message's fragments, in any order, carry; or the step that
|
||||
/// refuses them.
|
||||
pub fn reassemble(fragments: &[Vec<u8>]) -> Result<Vec<u8>, &'static str> {
|
||||
let first = fragments.first().ok_or(FRAGMENT_MISSING)?;
|
||||
if fragments.iter().any(|f| f.len() <= HEADER || f[2] == 0 || f[1] >= f[2]) {
|
||||
return Err(FRAGMENT_MALFORMED);
|
||||
}
|
||||
if fragments.iter().any(|f| f[0] != first[0] || f[2] != first[2]) {
|
||||
return Err(FRAGMENT_FOREIGN);
|
||||
}
|
||||
// Past step 2 every fragment has one number and one count, so two at one index with the same
|
||||
// data are the same bytes: one fragment.
|
||||
let mut slots: Vec<Option<&[u8]>> = vec![None; usize::from(first[2])];
|
||||
for f in fragments {
|
||||
let slot = &mut slots[usize::from(f[1])];
|
||||
match slot {
|
||||
Some(held) if *held == &f[HEADER..] => {}
|
||||
Some(_) => return Err(FRAGMENT_REPEATED),
|
||||
None => *slot = Some(&f[HEADER..]),
|
||||
}
|
||||
}
|
||||
let parts: Vec<&[u8]> = slots.into_iter().collect::<Option<_>>().ok_or(FRAGMENT_MISSING)?;
|
||||
let m = parts.concat();
|
||||
if m.len() > MAX_MESSAGE {
|
||||
return Err(FRAGMENT_MALFORMED);
|
||||
}
|
||||
if m.len() <= CHECK || m[m.len() - CHECK..] != check(&m[..m.len() - CHECK]) {
|
||||
return Err(FRAGMENT_CHECK_FAILED);
|
||||
}
|
||||
Ok(m[..m.len() - CHECK].to_vec())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn read_uint(b: &[u8]) -> Result<u64, &'static str> {
|
||||
let mut r = Reader { b, at: 0 };
|
||||
let v = r.uint()?;
|
||||
r.end().map(|()| v)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_uint_has_one_spelling() {
|
||||
for v in [0, 1, 127, 128, 300, 16_383, 16_384, 1_793_692_800, MAX_INT] {
|
||||
let mut b = Vec::new();
|
||||
put_uint(&mut b, v);
|
||||
assert_eq!(read_uint(&b), Ok(v), "{v}");
|
||||
}
|
||||
assert_eq!(read_uint(&[0x80, 0x00]), Err(MALFORMED), "0 in two bytes");
|
||||
assert_eq!(read_uint(&[0xff, 0x00]), Err(MALFORMED), "127 in two bytes");
|
||||
assert_eq!(read_uint(&[0xff; 7].iter().copied().chain([0x0f]).collect::<Vec<_>>()), Ok(MAX_INT));
|
||||
assert_eq!(read_uint(&[0xff; 7].iter().copied().chain([0x10]).collect::<Vec<_>>()), Err(MALFORMED), "2^53");
|
||||
assert_eq!(read_uint(&[0x80; 8]), Err(MALFORMED), "a ninth byte");
|
||||
assert_eq!(read_uint(&[0x80]), Err(MALFORMED), "ends inside");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_instant_is_one_spelling_of_one_second() {
|
||||
assert_eq!(instant_of("1970-01-01T00:00:00Z"), Some(0));
|
||||
assert_eq!(instant_of("2026-10-22T08:00:00Z"), Some(1_792_656_000));
|
||||
assert_eq!(instant_of("9999-12-31T23:59:59Z"), Some(MAX_SECONDS));
|
||||
for other in ["2026-10-22T08:00:00.0Z", "2026-10-22T08:00:00+00:00", "2026-10-22t08:00:00z", "2026-10-22T08:00:60Z", "1969-12-31T23:59:59Z"] {
|
||||
assert_eq!(instant_of(other), None, "{other}");
|
||||
}
|
||||
assert_eq!(rfc3339_utc(MAX_SECONDS), "9999-12-31T23:59:59Z");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_full_bitmap_leaves_the_bits_past_the_group_clear() {
|
||||
assert_eq!(full_bitmap(1), vec![0x80]);
|
||||
assert_eq!(full_bitmap(8), vec![0xff]);
|
||||
assert_eq!(full_bitmap(9), vec![0xff, 0x80]);
|
||||
assert_eq!(full_bitmap(20), vec![0xff, 0xff, 0xf0]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_message_reassembles_in_any_order_and_each_defect_refuses() {
|
||||
let record: Vec<u8> = (0u8..40).collect();
|
||||
let f = cut(&record, 7, 11).expect("a frame of 11");
|
||||
assert_eq!(f.len(), 6, "44 bytes in pieces of 8");
|
||||
assert!(f.iter().all(|x| x.len() <= 11));
|
||||
assert_eq!(reassemble(&f), Ok(record.clone()));
|
||||
let mut back = f.clone();
|
||||
back.reverse();
|
||||
assert_eq!(reassemble(&back), Ok(record.clone()), "arrival order is free");
|
||||
assert_eq!(reassemble(&f[1..]), Err(FRAGMENT_MISSING));
|
||||
assert_eq!(reassemble(&[]), Err(FRAGMENT_MISSING));
|
||||
assert_eq!(reassemble(&[f.clone(), vec![f[2].clone()]].concat()), Ok(record.clone()), "a fragment twice, byte for byte, is one");
|
||||
let mut other_data = f[2].clone();
|
||||
other_data[HEADER] ^= 0x01;
|
||||
assert_eq!(reassemble(&[f.clone(), vec![other_data]].concat()), Err(FRAGMENT_REPEATED));
|
||||
let mut swapped = f.clone();
|
||||
let (a, b) = (swapped[1][HEADER..].to_vec(), swapped[2][HEADER..].to_vec());
|
||||
swapped[1].truncate(HEADER);
|
||||
swapped[1].extend_from_slice(&b);
|
||||
swapped[2].truncate(HEADER);
|
||||
swapped[2].extend_from_slice(&a);
|
||||
assert_eq!(reassemble(&swapped), Err(FRAGMENT_CHECK_FAILED), "two fragments' data exchanged");
|
||||
let other = cut(&[9u8; 40], 8, 11).expect("another");
|
||||
let mut foreign = f.clone();
|
||||
foreign[3] = other[3].clone();
|
||||
assert_eq!(reassemble(&foreign), Err(FRAGMENT_FOREIGN));
|
||||
let same_number = cut(&[9u8; 40], 7, 11).expect("another under the same number");
|
||||
let mut mixed = f.clone();
|
||||
mixed[3] = same_number[3].clone();
|
||||
assert_eq!(reassemble(&mixed), Err(FRAGMENT_CHECK_FAILED));
|
||||
assert_eq!(reassemble(&[vec![7, 0, 1]]), Err(FRAGMENT_MALFORMED), "no data");
|
||||
assert_eq!(reassemble(&[vec![7, 1, 1, 0]]), Err(FRAGMENT_MALFORMED), "index past count");
|
||||
assert_eq!(reassemble(&[vec![7, 0, 0, 0]]), Err(FRAGMENT_MALFORMED), "count 0");
|
||||
assert_eq!(reassemble(&[[&[7u8, 0, 1][..], &check(&[])].concat()]), Err(FRAGMENT_CHECK_FAILED), "a check with no record");
|
||||
assert!(cut(&record, 7, 3).is_none(), "a frame of the header alone");
|
||||
assert!(cut(&vec![0u8; 300], 7, 4).is_none(), "more than 255 fragments");
|
||||
// §13.6's bound: 255 fragments of 258 bytes of data hold more than 65,535 bytes.
|
||||
let big: Vec<Vec<u8>> = (0..255u8).map(|i| [&[7u8, i, 255][..], &[0u8; 258][..]].concat()).collect();
|
||||
assert_eq!(reassemble(&big), Err(FRAGMENT_MALFORMED));
|
||||
assert!(cut(&vec![0u8; MAX_RECORD], 7, 1500).is_some());
|
||||
assert!(cut(&vec![0u8; MAX_RECORD + 1], 7, 1500).is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_record_past_the_bound_is_refused_before_it_is_read() {
|
||||
let ctx = Context::default();
|
||||
assert_eq!(expand(&vec![KIND_HOLDINGS; MAX_RECORD + 1], &ctx), Err(MALFORMED));
|
||||
assert_eq!(expand(&[], &ctx), Err(MALFORMED));
|
||||
assert_eq!(expand(&[0x03], &ctx), Err(MALFORMED));
|
||||
}
|
||||
}
|
||||
866
wai/wai-rs/src/compact_return/corpus.rs
Normal file
866
wai/wai-rs/src/compact_return/corpus.rs
Normal file
|
|
@ -0,0 +1,866 @@
|
|||
//! The conformance corpus of extensions/bearer-receipts.md §13.10: `live-conformance/compact-return/`.
|
||||
//!
|
||||
//! [`generate`] writes `vectors.json` from fixed seeds: every key is an Ed25519 key from a fixed
|
||||
//! 32-byte seed, every time a constant, every envelope is written here, and Ed25519 signing is
|
||||
//! deterministic (RFC 8032), so the file is the same bytes on every machine. Each case states the
|
||||
//! outcome it was written to have, declared here and never copied from [`super::expand`],
|
||||
//! [`super::encode`] or [`super::reassemble`]; [`verify`] requires the module to reach every
|
||||
//! outcome, every round trip to hold both ways, every cut to reassemble, and the corpus to give
|
||||
//! every reason of §13.9. `live-conformance/compact_return_verify.py` checks the same file from
|
||||
//! the extension's text, sharing no code with this module.
|
||||
|
||||
use super::*;
|
||||
use crate::claim_receipt::{hex, unhex, Rejection, TransmissionClaim};
|
||||
use crate::container::{Conditioning, Manifest, ModelRequirement, Wai};
|
||||
use crate::energy_class::{CounterFiltering, EnergyProvenance, Uncertainty};
|
||||
use crate::holdings::Entry;
|
||||
use crate::jwp_delivery::{GroupBuilder, ObjectLabel};
|
||||
use ed25519_dalek::SigningKey;
|
||||
use serde_json::{json, Map, Value};
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
const PUBLISHER: u8 = 0x11;
|
||||
const SINK_A: u8 = 0x41;
|
||||
const SINK_B: u8 = 0x42;
|
||||
const SINK_C: u8 = 0x43;
|
||||
const SINK_D: u8 = 0x44;
|
||||
const HOLDER: u8 = 0x51;
|
||||
const NEIGHBOUR: u8 = 0x52;
|
||||
|
||||
const D1: &str = "did:web:publisher.example#d1";
|
||||
/// Sink B signs under an origin that speaks for it, and has enrolled its key with the receiver.
|
||||
const SINK_B_ID: &str = "did:web:sinks.example#b";
|
||||
/// The same key under a second identifier, for an ambiguous reference.
|
||||
const SINK_B_OTHER_ID: &str = "did:web:sinks.example#b2";
|
||||
/// Sink C signs under a label, a `signer_id` of neither form of jwp-receipts §7.1.
|
||||
const SINK_C_ID: &str = "urn:example:wai-sink:c";
|
||||
/// The holder's key, as the receiver holds it.
|
||||
const HOLDER_ID: &str = "did:web:holders.example#h1";
|
||||
/// The same key under a second identifier: one key still, for a holdings record.
|
||||
const HOLDER_OTHER_ID: &str = "did:web:holders.example#h2";
|
||||
const MCAST: &str = "urn:example:wai-bearer:multicast-1";
|
||||
const BCAST: &str = "urn:example:wai-bearer:broadcast-1";
|
||||
const RELAY: &str = "urn:example:wai-bearer:relay-1";
|
||||
const NOT_HELD: &str = "urn:example:wai-bearer:broadcast-9";
|
||||
const GROUP_START: &str = "2026-10-21T12:00:00Z";
|
||||
const GROUP_END: &str = "2026-10-21T12:00:02Z";
|
||||
const SENT: &str = "2026-10-21T12:00:03Z";
|
||||
const RECONCILED: &str = "2026-10-22T08:00:00Z";
|
||||
/// The instant the holdings records state their holdings at.
|
||||
const T: u64 = 1_793_692_800_000;
|
||||
/// An hour: when they expire.
|
||||
const HOUR: u64 = 3_600_000;
|
||||
|
||||
fn key(seed: u8) -> SigningKey {
|
||||
SigningKey::from_bytes(&[seed; 32])
|
||||
}
|
||||
fn pk(seed: u8) -> [u8; 32] {
|
||||
key(seed).verifying_key().to_bytes()
|
||||
}
|
||||
|
||||
/// One WAI envelope with an opaque payload, which no case decodes.
|
||||
fn envelope(tag: &str, i: usize) -> Vec<u8> {
|
||||
let payload = format!("compact-return {tag} object {i}").into_bytes();
|
||||
Wai::new(Manifest::new("binary", "improve", ModelRequirement::new("wai.text.zstd"), Conditioning::new("bytes")), payload)
|
||||
.to_bytes()
|
||||
.expect("a small envelope writes")
|
||||
}
|
||||
|
||||
/// A published group: its receipt and its object receipts.
|
||||
#[derive(Clone)]
|
||||
struct Publication {
|
||||
group: GroupReceipt,
|
||||
objects: Vec<ObjectReceipt>,
|
||||
}
|
||||
|
||||
impl Publication {
|
||||
fn link(&self) -> [u8; 32] {
|
||||
self.group.link_hash()
|
||||
}
|
||||
/// The bitmap and root of a reconciliation of the objects `idx`.
|
||||
fn commitment(&self, idx: &[u64]) -> (Vec<u8>, [u8; 32]) {
|
||||
let leaves: Vec<[u8; 32]> = idx.iter().map(|&i| self.objects[i as usize].leaf()).collect();
|
||||
(ReconciliationClaim::bitmap(self.group.leaf_count, idx), merkle_root(&leaves))
|
||||
}
|
||||
}
|
||||
|
||||
fn publish(tag: &str, track: u64, n: usize, labelled: bool) -> Publication {
|
||||
let mut b = GroupBuilder::default();
|
||||
for i in 0..n {
|
||||
let h = crate::jwp_hash::content_hash(&envelope(tag, i));
|
||||
if labelled {
|
||||
let class = EnergyProvenance::OnChipCounter {
|
||||
uncertainty: Uncertainty { relative_ppm: 20_000, window_us: 1_000_000 },
|
||||
filtering: CounterFiltering::Unknown,
|
||||
};
|
||||
b.add(h, 100_000 + 10_000 * i as u64, "edge-1", Some(ObjectLabel::Class(class)));
|
||||
} else {
|
||||
b.add(h, 0, "", None);
|
||||
}
|
||||
}
|
||||
let (group, objects) =
|
||||
b.finish(&key(PUBLISHER), D1, track, 1, GROUP_START, GROUP_END, None, labelled).expect("the corpus's objects compose");
|
||||
Publication { group, objects }
|
||||
}
|
||||
|
||||
/// The groups: of 4, 3 (labelled leaves), 9 and 20 objects, which the receiver holds, and one of
|
||||
/// 4 it does not.
|
||||
struct Pubs {
|
||||
g4: Publication,
|
||||
g3: Publication,
|
||||
g9: Publication,
|
||||
g20: Publication,
|
||||
far: Publication,
|
||||
}
|
||||
|
||||
fn pubs() -> Pubs {
|
||||
Pubs { g4: publish("g4", 1, 4, false), g3: publish("g3", 2, 3, true), g9: publish("g9", 3, 9, false), g20: publish("g20", 4, 20, false), far: publish("far", 5, 4, false) }
|
||||
}
|
||||
|
||||
/// Two `bearer_id`s with one reference: the first pair, in order, of the corpus's twin names.
|
||||
fn twins() -> (String, String) {
|
||||
let mut seen: BTreeMap<[u8; 4], u32> = BTreeMap::new();
|
||||
for i in 0u32.. {
|
||||
let id = format!("urn:example:wai-bearer:twin-{i}");
|
||||
if let Some(j) = seen.insert(bearer_ref(&id), i) {
|
||||
return (format!("urn:example:wai-bearer:twin-{j}"), id);
|
||||
}
|
||||
}
|
||||
unreachable!("a 32-bit reference repeats")
|
||||
}
|
||||
|
||||
// ───────────────────────────── contexts ─────────────────────────────
|
||||
|
||||
/// One group of a context as the corpus writes it: the group's name, the positions of the object
|
||||
/// receipts held under it and its bearer instances.
|
||||
type GroupLine = (&'static str, Vec<u64>, Vec<String>);
|
||||
|
||||
fn context_json(groups: &[GroupLine], signers: &[([u8; 32], &str)]) -> Value {
|
||||
json!({
|
||||
"groups": groups.iter().map(|(g, objects, bearers)| json!({ "group": g, "objects": objects, "bearers": bearers })).collect::<Vec<_>>(),
|
||||
"signers": signers.iter().map(|(k, id)| json!({ "key": hex(k), "signer_id": id })).collect::<Vec<_>>(),
|
||||
})
|
||||
}
|
||||
|
||||
fn all(n: u64) -> Vec<u64> {
|
||||
(0..n).collect()
|
||||
}
|
||||
|
||||
/// The receiver's groups, with or without their object receipts and instances.
|
||||
fn receiver_groups(objects: bool, bearers: bool) -> Vec<GroupLine> {
|
||||
let o = |n: u64| if objects { all(n) } else { Vec::new() };
|
||||
let b = |ids: &[&str]| if bearers { ids.iter().map(|s| s.to_string()).collect() } else { Vec::new() };
|
||||
vec![("g4", o(4), b(&[MCAST, BCAST])), ("g3", o(3), b(&[RELAY])), ("g9", o(9), b(&[BCAST])), ("g20", o(20), b(&[BCAST]))]
|
||||
}
|
||||
|
||||
fn receiver_signers() -> Vec<([u8; 32], &'static str)> {
|
||||
vec![(pk(SINK_B), SINK_B_ID), (pk(HOLDER), HOLDER_ID)]
|
||||
}
|
||||
|
||||
fn contexts() -> Vec<(&'static str, Value)> {
|
||||
let (twin_a, twin_b) = twins();
|
||||
let mut twin_groups = receiver_groups(true, true);
|
||||
twin_groups[0].2.extend([twin_a.clone(), twin_b]);
|
||||
let mut alone = receiver_groups(true, true);
|
||||
alone[0].2.push(twin_a);
|
||||
let mut twin_signers = receiver_signers();
|
||||
twin_signers.push((pk(SINK_B), SINK_B_OTHER_ID));
|
||||
twin_signers.push((pk(HOLDER), HOLDER_OTHER_ID));
|
||||
// g4 without the broadcast, which g9 and g20 keep.
|
||||
let mut elsewhere = receiver_groups(true, true);
|
||||
elsewhere[0].2.retain(|b| b != BCAST);
|
||||
vec![
|
||||
("receiver", context_json(&receiver_groups(true, true), &receiver_signers())),
|
||||
("no_objects", context_json(&receiver_groups(false, true), &receiver_signers())),
|
||||
("no_bearers", context_json(&receiver_groups(true, false), &receiver_signers())),
|
||||
("no_signers", context_json(&receiver_groups(true, true), &[])),
|
||||
("empty", context_json(&[], &[])),
|
||||
("one_twin", context_json(&alone, &receiver_signers())),
|
||||
("twin_bearers", context_json(&twin_groups, &receiver_signers())),
|
||||
("twin_signer", context_json(&receiver_groups(true, true), &twin_signers)),
|
||||
("bearer_elsewhere", context_json(&elsewhere, &receiver_signers())),
|
||||
]
|
||||
}
|
||||
|
||||
fn groups_json(p: &Pubs) -> Value {
|
||||
let mut m = Map::new();
|
||||
for (name, g) in [("g4", &p.g4), ("g3", &p.g3), ("g9", &p.g9), ("g20", &p.g20), ("far", &p.far)] {
|
||||
m.insert(name.into(), json!({ "receipt": g.group.to_json(), "objects": g.objects.iter().map(|o| o.to_json()).collect::<Vec<_>>() }));
|
||||
}
|
||||
Value::Object(m)
|
||||
}
|
||||
|
||||
/// A context from the corpus's form: each group by name from `groups`, with the object receipts
|
||||
/// at the positions it lists.
|
||||
pub fn context_of(v: &Value, groups: &Value) -> Result<Context, String> {
|
||||
let mut ctx = Context::default();
|
||||
for g in v["groups"].as_array().ok_or("groups")? {
|
||||
let name = g["group"].as_str().ok_or("a group's name")?;
|
||||
let entry = &groups[name];
|
||||
let receipt = GroupReceipt::from_json(&entry["receipt"]).map_err(|e| format!("{name}: {e:?}"))?;
|
||||
let held = entry["objects"].as_array().ok_or("objects")?;
|
||||
let mut objects = Vec::new();
|
||||
for i in g["objects"].as_array().ok_or("a group's objects")? {
|
||||
let o = held.get(i.as_u64().ok_or("a position")? as usize).ok_or("a position past the objects")?;
|
||||
objects.push(ObjectReceipt::from_json(o).map_err(|e| format!("{name}: {e:?}"))?);
|
||||
}
|
||||
let bearers = g["bearers"]
|
||||
.as_array()
|
||||
.ok_or("bearers")?
|
||||
.iter()
|
||||
.map(|b| b.as_str().map(String::from).ok_or_else(|| "a bearer_id".to_string()))
|
||||
.collect::<Result<Vec<_>, _>>()?;
|
||||
ctx.add_group(receipt, objects, bearers);
|
||||
}
|
||||
for s in v["signers"].as_array().ok_or("signers")? {
|
||||
let key = s["key"].as_str().and_then(unhex).and_then(|b| <[u8; 32]>::try_from(b).ok()).ok_or("a signer's key")?;
|
||||
ctx.signers.push(Signer { key, signer_id: s["signer_id"].as_str().ok_or("a signer_id")?.to_string() });
|
||||
}
|
||||
Ok(ctx)
|
||||
}
|
||||
|
||||
// ───────────────────────────── records ─────────────────────────────
|
||||
|
||||
fn reconciliation_body(p: &Publication, bearer: Bearer, id: &str, idx: &[u64]) -> ReconciliationClaim {
|
||||
let (received, received_root) = p.commitment(idx);
|
||||
ReconciliationClaim { bearer, bearer_id: id.into(), leaf_count: p.group.leaf_count, received, received_root, attests: ATTESTS_RECEIVED.into() }
|
||||
}
|
||||
|
||||
/// A reconciliation a verifier accepts at jwp-receipts §3.1 steps 1 to 4.
|
||||
fn reconcile(seed: u8, id: &str, p: &Publication, body: ReconciliationClaim, seq: u64, at: &str, parent: Option<[u8; 32]>) -> ClaimReceipt {
|
||||
ClaimReceipt::seal(&key(seed), id, SubjectType::Receipt, p.link(), seq, at, ClaimBody::Reconciliation(body), parent)
|
||||
.expect("a reconciliation the corpus means to pass steps 1-4")
|
||||
}
|
||||
|
||||
/// One that breaks a rule of steps 1, 2 or 4, validly signed.
|
||||
fn reconcile_unchecked(seed: u8, subject_type: SubjectType, p: &Publication, body: ReconciliationClaim) -> ClaimReceipt {
|
||||
ClaimReceipt::seal_unchecked(&key(seed), did_key(&pk(seed)), subject_type, p.link(), 0, RECONCILED, ClaimBody::Reconciliation(body), None)
|
||||
.expect("a canonical form")
|
||||
}
|
||||
|
||||
/// The records the cases are built from.
|
||||
struct Records {
|
||||
/// Sink A, a fresh did:key: every object of g4, over the broadcast.
|
||||
whole: ClaimReceipt,
|
||||
/// Sink B, enrolled: objects 0 and 2 of g4, over the multicast.
|
||||
partial: ClaimReceipt,
|
||||
/// Sink B's next claim, chained to the last: objects 0 to 7 of g9.
|
||||
chained: ClaimReceipt,
|
||||
/// Sink C, under a label: object 1 of g3, whose leaves are labelled, over the relay.
|
||||
labelled: ClaimReceipt,
|
||||
/// Sink B's third: every object of g20.
|
||||
whole_twenty: ClaimReceipt,
|
||||
/// Sink D, a fresh did:key: every object of g4, over the first twin instance.
|
||||
twin: ClaimReceipt,
|
||||
/// The holder, whose key the receiver holds: objects 0 and 8 of g9, g20 whole, object 1 of a
|
||||
/// group the receiver does not hold.
|
||||
held: Holdings,
|
||||
/// A neighbour the receiver does not know: g4 whole, and the group it does not hold whole.
|
||||
neighbour: Holdings,
|
||||
}
|
||||
|
||||
fn records(p: &Pubs) -> Records {
|
||||
let (twin_a, _) = twins();
|
||||
let whole = reconcile(SINK_A, &did_key(&pk(SINK_A)), &p.g4, reconciliation_body(&p.g4, Bearer::Unidirectional, BCAST, &all(4)), 0, RECONCILED, None);
|
||||
let partial = reconcile(SINK_B, SINK_B_ID, &p.g4, reconciliation_body(&p.g4, Bearer::Multicast, MCAST, &[0, 2]), 0, RECONCILED, None);
|
||||
let chained = reconcile(SINK_B, SINK_B_ID, &p.g9, reconciliation_body(&p.g9, Bearer::Unidirectional, BCAST, &all(8)), 1, RECONCILED, Some(partial.link_hash()));
|
||||
let whole_twenty =
|
||||
reconcile(SINK_B, SINK_B_ID, &p.g20, reconciliation_body(&p.g20, Bearer::Unidirectional, BCAST, &all(20)), 2, RECONCILED, Some(chained.link_hash()));
|
||||
let labelled = reconcile(SINK_C, SINK_C_ID, &p.g3, reconciliation_body(&p.g3, Bearer::Bidirectional, RELAY, &[1]), 0, RECONCILED, None);
|
||||
let twin = reconcile(SINK_D, &did_key(&pk(SINK_D)), &p.g4, reconciliation_body(&p.g4, Bearer::Multicast, &twin_a, &all(4)), 0, RECONCILED, None);
|
||||
let e = |g: &Publication, held: &[u64]| Entry::of(g.link(), g.group.leaf_count, held).expect("a case's entry holds something");
|
||||
let held = Holdings::seal(&key(HOLDER), T, T + HOUR, vec![e(&p.g9, &[0, 8]), e(&p.g20, &all(20)), e(&p.far, &[1])]);
|
||||
let neighbour = Holdings::seal(&key(NEIGHBOUR), T, T + HOUR, vec![e(&p.g4, &all(4)), e(&p.far, &all(4))]);
|
||||
Records { whole, partial, chained, labelled, whole_twenty, twin, held, neighbour }
|
||||
}
|
||||
|
||||
fn rec(c: &ClaimReceipt) -> Record {
|
||||
Record::Reconciliation(c.to_json())
|
||||
}
|
||||
|
||||
fn hold(h: &Holdings) -> Record {
|
||||
Record::Holdings(h.to_bytes())
|
||||
}
|
||||
|
||||
fn kind_of(r: &Record) -> &'static str {
|
||||
match r {
|
||||
Record::Reconciliation(_) => "reconciliation",
|
||||
Record::Holdings(_) => "holdings",
|
||||
}
|
||||
}
|
||||
|
||||
fn record_value(r: &Record) -> Value {
|
||||
match r {
|
||||
Record::Reconciliation(s) => Value::String(s.clone()),
|
||||
Record::Holdings(b) => Value::String(hex(b)),
|
||||
}
|
||||
}
|
||||
|
||||
fn record_of(kind: &str, v: &Value) -> Result<Record, String> {
|
||||
let s = v.as_str().ok_or("a record")?;
|
||||
match kind {
|
||||
"reconciliation" => Ok(Record::Reconciliation(s.to_string())),
|
||||
"holdings" => unhex(s).map(Record::Holdings).ok_or_else(|| "not hex".to_string()),
|
||||
_ => Err(format!("kind {kind}")),
|
||||
}
|
||||
}
|
||||
|
||||
fn reject(r: Rejection) -> String {
|
||||
format!("reject step={} reason={}", r.step, r.reason.label())
|
||||
}
|
||||
|
||||
/// What a record's own text says of it: a reconciliation through jwp-receipts §3.1 steps 1 to 4,
|
||||
/// then step 6 by bearer-receipts §5.3 against the context's group and objects; a holdings record
|
||||
/// read under holdings §4.4.
|
||||
pub fn verdict(record: &Record, ctx: &Context) -> String {
|
||||
match record {
|
||||
Record::Reconciliation(text) => {
|
||||
let c = match ClaimReceipt::from_json(text) {
|
||||
Ok(c) => c,
|
||||
Err(r) => return reject(r),
|
||||
};
|
||||
if let Err(r) = c.verify() {
|
||||
return reject(r);
|
||||
}
|
||||
let Some(g) = ctx.group_by_link(&c.subject) else { return "reject step=6 reason=context_invalid".into() };
|
||||
let objects: Vec<&ObjectReceipt> = g.objects.iter().collect();
|
||||
match c.check_reconciliation_against(&g.receipt, &objects) {
|
||||
Ok(()) => "accept".into(),
|
||||
Err(r) => reject(r),
|
||||
}
|
||||
}
|
||||
Record::Holdings(b) => match Holdings::read(b) {
|
||||
Ok(h) => format!("read digest={}", hex(&h.digest())),
|
||||
Err(r) => format!("reject reason={r}"),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/// The contexts, built.
|
||||
fn built(contexts: &[(&str, Value)], groups: &Value) -> BTreeMap<String, Context> {
|
||||
contexts.iter().map(|(n, v)| (n.to_string(), context_of(v, groups).expect("the generator's contexts read"))).collect()
|
||||
}
|
||||
|
||||
fn compact(r: &Record, ctx: &Context) -> Vec<u8> {
|
||||
encode(r, ctx).expect("a record the corpus means to have a compact form")
|
||||
}
|
||||
|
||||
fn record_cases(r: &Records, ctx: &BTreeMap<String, Context>) -> Vec<Value> {
|
||||
let case = |name: &str, description: &str, record: Record, verdict: &str| {
|
||||
json!({
|
||||
"name": name,
|
||||
"description": description,
|
||||
"context": "receiver",
|
||||
"kind": kind_of(&record),
|
||||
"record": record_value(&record),
|
||||
"compact": hex(&compact(&record, &ctx["receiver"])),
|
||||
"outcome": { "verdict": verdict },
|
||||
})
|
||||
};
|
||||
let digest = |h: &Holdings| format!("read digest={}", hex(&h.digest()));
|
||||
vec![
|
||||
case("whole_did_key", "Every object of a group of four over a broadcast, signed under a fresh did:key: signer form 0, no bitmap.", rec(&r.whole), "accept"),
|
||||
case("partial_enrolled", "Two objects of four over a multicast, by a sink whose key the receiver holds: signer form 2.", rec(&r.partial), "accept"),
|
||||
case("partial_chained", "The same sink's next claim, its parent carried: eight objects of nine, a bitmap's last byte part used.", rec(&r.chained), "accept"),
|
||||
case("whole_twenty", "Its third: every object of twenty.", rec(&r.whole_twenty), "accept"),
|
||||
case("labelled_label_id", "One object of a group of labelled leaves over a relay, by a sink signing under a label: signer form 1.", rec(&r.labelled), "accept"),
|
||||
case("holdings_by_reference", "A holder the receiver knows: its key, and two of its three groups, by reference; the third inline.", hold(&r.held), &digest(&r.held)),
|
||||
case("holdings_inline_holder", "A holder the receiver does not know, inline: one group by reference, one inline, both whole.", hold(&r.neighbour), &digest(&r.neighbour)),
|
||||
]
|
||||
}
|
||||
|
||||
fn expand_outcome(record: &Record, verdict: &str) -> Value {
|
||||
json!({ "outcome": "expand", "kind": kind_of(record), "record": record_value(record), "verdict": verdict })
|
||||
}
|
||||
|
||||
fn refuse(reason: &str) -> Value {
|
||||
json!({ "outcome": "refuse", "reason": reason })
|
||||
}
|
||||
|
||||
/// The uint bytes of `v`.
|
||||
fn uint_bytes(v: u64) -> Vec<u8> {
|
||||
let mut b = Vec::new();
|
||||
put_uint(&mut b, v);
|
||||
b
|
||||
}
|
||||
|
||||
fn expansion_cases(r: &Records, ctx: &BTreeMap<String, Context>) -> Vec<Value> {
|
||||
let recv = &ctx["receiver"];
|
||||
let mut out = Vec::new();
|
||||
let mut case = |name: &str, description: &str, context: &str, bytes: &[u8], outcome: Value| {
|
||||
out.push(json!({ "name": name, "description": description, "context": context, "compact": hex(bytes), "outcome": outcome }));
|
||||
};
|
||||
let whole = compact(&rec(&r.whole), recv);
|
||||
let partial = compact(&rec(&r.partial), recv);
|
||||
let labelled = compact(&rec(&r.labelled), recv);
|
||||
let held = compact(&hold(&r.held), recv);
|
||||
let neighbour = compact(&hold(&r.neighbour), recv);
|
||||
let accept = "accept";
|
||||
let held_digest = format!("read digest={}", hex(&r.held.digest()));
|
||||
let neighbour_digest = format!("read digest={}", hex(&r.neighbour.digest()));
|
||||
|
||||
// Every form reads: a value the receiver holds, written inline, expands to the same record.
|
||||
case("signer_inline_though_held", "Sink B's claim with its key and signer_id inline (form 1), although the receiver holds them.", "receiver",
|
||||
&compact(&rec(&r.partial), &ctx["no_signers"]), expand_outcome(&rec(&r.partial), accept));
|
||||
case("holder_inline_though_held", "The holder's key inline, although the receiver holds it.", "receiver",
|
||||
&compact(&hold(&r.held), &ctx["no_signers"]), expand_outcome(&hold(&r.held), &held_digest));
|
||||
case("groups_inline_though_held", "Every group inline, although the receiver holds them.", "receiver",
|
||||
&compact(&hold(&r.neighbour), &ctx["empty"]), expand_outcome(&hold(&r.neighbour), &neighbour_digest));
|
||||
case("whole_needs_no_objects", "A whole reconciliation, against a receiver that holds no object receipt: its root is the group's.", "no_objects",
|
||||
&whole, expand_outcome(&rec(&r.whole), accept));
|
||||
|
||||
// A compact record adds no trust: a changed byte expands, and the record refuses itself.
|
||||
let changed = |b: &[u8], at: usize, x: u8| {
|
||||
let mut c = b.to_vec();
|
||||
c[at] ^= x;
|
||||
c
|
||||
};
|
||||
let refused_record = |b: &[u8], reason: &str| {
|
||||
let rec = expand(b, recv).expect("a changed record that still expands");
|
||||
expand_outcome(&rec, reason)
|
||||
};
|
||||
let sig = changed(&whole, whole.len() - 1, 0x01);
|
||||
case("signature_changed", "A bit of the signature flipped: it expands, and the claim's signature does not verify.", "receiver", &sig,
|
||||
refused_record(&sig, "reject step=3 reason=signature_verify_failed"));
|
||||
let seq = changed(&whole, 14, 0x01);
|
||||
case("seq_changed", "seq 1 for 0: it expands to a claim its signer did not sign.", "receiver", &seq,
|
||||
refused_record(&seq, "reject step=3 reason=signature_verify_failed"));
|
||||
let class = changed(&whole, 1, 0x03);
|
||||
case("bearer_class_changed", "The bearer class multicast for unidirectional: it expands to a claim its signer did not sign.", "receiver",
|
||||
&class, refused_record(&class, "reject step=3 reason=signature_verify_failed"));
|
||||
let hsig = changed(&held, held.len() - 1, 0x01);
|
||||
case("holdings_signature_changed", "A bit of the holder's signature flipped.", "receiver", &hsig,
|
||||
refused_record(&hsig, "reject reason=holdings_signature_invalid"));
|
||||
|
||||
// §13.3 and §13.4: the reader's rules.
|
||||
let m = refuse(MALFORMED);
|
||||
let with = |b: &[u8], at: usize, x: u8| {
|
||||
let mut c = b.to_vec();
|
||||
c[at] = x;
|
||||
c
|
||||
};
|
||||
let splice = |b: &[u8], at: usize, len: usize, new: &[u8]| [&b[..at], new, &b[at + len..]].concat();
|
||||
case("kind_unknown", "kind 0x03.", "receiver", &with(&whole, 0, 0x03), m.clone());
|
||||
case("form_reserved_bit", "form with 0x40 set.", "receiver", &with(&whole, 1, whole[1] | 0x40), m.clone());
|
||||
case("bearer_three", "A bearer class of 3.", "receiver", &with(&whole, 1, whole[1] | 0x03), m.clone());
|
||||
case("signer_form_three", "A signer form of 3.", "receiver", &with(&whole, 1, whole[1] | 0x18), m.clone());
|
||||
case("truncated", "The last byte of the signature missing.", "receiver", &whole[..whole.len() - 1], m.clone());
|
||||
case("trailing", "A byte after the signature.", "receiver", &[&whole[..], &[0]].concat(), m.clone());
|
||||
// seq is at 14: kind, form, group (8), bearer (4).
|
||||
case("seq_not_minimal", "seq 0 written in two bytes.", "receiver", &splice(&whole, 14, 1, &[0x80, 0x00]), m.clone());
|
||||
case("seq_past_bound", "seq 2^53.", "receiver", &splice(&whole, 14, 1, &uint_bytes(1 << 53)), m.clone());
|
||||
case("uint_nine_bytes", "seq in nine bytes.", "receiver", &splice(&whole, 14, 1, &[0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x01]), m.clone());
|
||||
let at_len = uint_bytes(instant_of(RECONCILED).expect("an instant")).len();
|
||||
case("issued_at_past_bound", "issued_at one second past 9999-12-31T23:59:59Z.", "receiver",
|
||||
&splice(&whole, 15, at_len, &uint_bytes(MAX_SECONDS + 1)), m.clone());
|
||||
{
|
||||
// Form 1 with the key's own did:key: that is form 0's spelling.
|
||||
let id = did_key(&pk(SINK_A));
|
||||
let key_end = 15 + at_len + 32;
|
||||
let mut b = with(&whole, 1, whole[1] | 0x08);
|
||||
b = splice(&b, key_end, 0, &[uint_bytes(id.len() as u64), id.into_bytes()].concat());
|
||||
case("signer_id_is_did_key", "Signer form 1 naming the key's own did:key, which is form 0.", "receiver", &b, m.clone());
|
||||
}
|
||||
{
|
||||
// Sink C's label: kind, form, group, bearer, seq (1), issued_at, key (32), then its length.
|
||||
let id_at = 15 + at_len + 32 + 1;
|
||||
case("signer_id_not_utf8", "A signer_id that is not UTF-8.", "receiver", &with(&labelled, id_at, 0xff), m.clone());
|
||||
}
|
||||
// The bitmap of objects 0 and 2 of four is the byte before the signature.
|
||||
let bitmap_at = partial.len() - 65;
|
||||
case("received_every_bit", "A partial bitmap of every object of four: the whole form's spelling.", "receiver", &with(&partial, bitmap_at, 0xf0), m.clone());
|
||||
case("received_no_bit", "A partial bitmap of no object.", "receiver", &with(&partial, bitmap_at, 0x00), m.clone());
|
||||
case("received_past_leaf_count", "Object 4 of a group of four set.", "receiver", &with(&partial, bitmap_at, 0xa8), m.clone());
|
||||
case("holdings_holder_form", "A holder form of 2.", "receiver", &with(&held, 1, 0x02), m.clone());
|
||||
let n_at = 2 + 8 + uint_bytes(T).len() + uint_bytes(HOUR).len();
|
||||
case("holdings_entry_form", "An entry's form with 0x04 set.", "receiver", &with(&held, n_at + 1, held[n_at + 1] | 0x04), m.clone());
|
||||
case("holdings_count_past_u32", "An entry count of 2^32, which WHD1 cannot state.", "receiver", &splice(&held, n_at, 1, &uint_bytes(1 << 32)), m.clone());
|
||||
case("holdings_truncated", "The last byte of the holder's signature missing.", "receiver", &held[..held.len() - 1], m.clone());
|
||||
case("holdings_trailing", "A byte after the holder's signature.", "receiver", &[&held[..], &[0]].concat(), m.clone());
|
||||
// as_of_ms is at 10: kind, form, holder (8); window_ms follows it.
|
||||
let window_at = 2 + 8 + uint_bytes(T).len();
|
||||
case("holdings_as_of_past_bound", "as_of_ms 2^53.", "receiver", &splice(&held, 10, uint_bytes(T).len(), &uint_bytes(1 << 53)), m.clone());
|
||||
{
|
||||
let mut longer = uint_bytes(HOUR);
|
||||
*longer.last_mut().expect("a byte") |= 0x80;
|
||||
longer.push(0x00);
|
||||
case("holdings_window_not_minimal", "window_ms written in one byte more than its fewest.", "receiver",
|
||||
&splice(&held, window_at, uint_bytes(HOUR).len(), &longer), m.clone());
|
||||
}
|
||||
{
|
||||
// The entry of the group the receiver does not hold is inline: its link hash, then its
|
||||
// leaf_count.
|
||||
let e = r.held.entries.iter().find(|e| recv.group_by_link(&e.group).is_none()).expect("an entry written inline");
|
||||
let at = held.windows(32).position(|w| w == &e.group[..]).expect("the inline group") + 32;
|
||||
case("holdings_leaf_count_nine_bytes", "An inline group's leaf_count in nine bytes.", "receiver",
|
||||
&splice(&held, at, uint_bytes(e.leaf_count).len(), &[0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x01]), m.clone());
|
||||
}
|
||||
|
||||
// §13.2: references.
|
||||
let unknown = refuse(REFERENCE_UNKNOWN);
|
||||
case("group_unknown", "A group the receiver does not hold.", "empty", &whole, unknown.clone());
|
||||
case("bearer_unknown", "A bearer instance of the group the receiver holds no declaration of.", "no_bearers", &whole, unknown.clone());
|
||||
case("bearer_of_other_groups", "Sink A's claim over the broadcast, an instance the receiver holds for g9 and g20 but not for g4.",
|
||||
"bearer_elsewhere", &whole, unknown.clone());
|
||||
case("signer_unknown", "A signer by reference the receiver does not hold.", "no_signers", &partial, unknown.clone());
|
||||
case("holder_unknown", "A holder by reference the receiver does not hold.", "no_signers", &held, unknown.clone());
|
||||
case("holdings_group_unknown", "An entry's group by reference the receiver does not hold.", "empty", &neighbour, unknown);
|
||||
let twin = compact(&rec(&r.twin), &ctx["one_twin"]);
|
||||
case("bearer_resolves_alone", "Sink D's claim over the first of two instances with one reference, the receiver holding that one alone.", "one_twin",
|
||||
&twin, expand_outcome(&rec(&r.twin), accept));
|
||||
case("bearer_ambiguous", "The same bytes, the receiver holding both instances: refused, not resolved by choosing.", "twin_bearers", &twin,
|
||||
refuse(REFERENCE_AMBIGUOUS));
|
||||
case("signer_ambiguous", "Sink B's key held under two signer_ids.", "twin_signer", &partial, refuse(REFERENCE_AMBIGUOUS));
|
||||
case("holder_under_two_ids", "The holder by reference, its key held under two signer_ids: one key, since a holdings record names no signer_id.",
|
||||
"twin_signer", &held, expand_outcome(&hold(&r.held), &held_digest));
|
||||
case("objects_not_held", "A partial reconciliation, against a receiver that holds no object receipt to derive its root from.", "no_objects",
|
||||
&partial, refuse(OBJECTS_NOT_HELD));
|
||||
out
|
||||
}
|
||||
|
||||
fn encoding_cases(p: &Pubs, r: &Records) -> Vec<Value> {
|
||||
let mut out = Vec::new();
|
||||
let mut case = |name: &str, description: &str, context: &str, record: Record| {
|
||||
out.push(json!({
|
||||
"name": name,
|
||||
"description": description,
|
||||
"context": context,
|
||||
"kind": kind_of(&record),
|
||||
"record": record_value(&record),
|
||||
"outcome": refuse(NOT_ENCODABLE),
|
||||
}));
|
||||
};
|
||||
let at = |t: &str| {
|
||||
rec(&reconcile(SINK_A, &did_key(&pk(SINK_A)), &p.g4, reconciliation_body(&p.g4, Bearer::Unidirectional, BCAST, &all(4)), 0, t, None))
|
||||
};
|
||||
case("issued_at_fraction", "issued_at with a fraction of a second.", "receiver", at("2026-10-22T08:00:00.5Z"));
|
||||
case("issued_at_offset", "issued_at with an offset.", "receiver", at("2026-10-22T08:00:00+00:00"));
|
||||
case("issued_at_lower_case", "issued_at with t and z in lower case.", "receiver", at("2026-10-22t08:00:00z"));
|
||||
case("issued_at_leap_second", "issued_at at a second of 60.", "receiver", at("2026-10-22T08:00:60Z"));
|
||||
case("issued_at_before_1970", "issued_at before 1970.", "receiver", at("1969-12-31T23:59:59Z"));
|
||||
case("not_canonical_text", "The claim with a space after its first brace: not its canonical form.", "receiver",
|
||||
Record::Reconciliation(r.whole.to_json().replacen('{', "{ ", 1)));
|
||||
let transmission = ClaimReceipt::seal(
|
||||
&key(PUBLISHER),
|
||||
D1,
|
||||
SubjectType::Receipt,
|
||||
p.g4.link(),
|
||||
0,
|
||||
SENT,
|
||||
ClaimBody::Transmission(TransmissionClaim { bearer: Bearer::Unidirectional, bearer_id: BCAST.into(), energy: None, attests: "transmitted".into() }),
|
||||
None,
|
||||
)
|
||||
.expect("a transmission claim");
|
||||
case("transmission_claim", "A transmission claim: this profile writes reconciliations only.", "receiver", rec(&transmission));
|
||||
case("group_not_held", "A reconciliation of a group the receiver does not hold.", "receiver",
|
||||
rec(&reconcile(SINK_A, &did_key(&pk(SINK_A)), &p.far, reconciliation_body(&p.far, Bearer::Unidirectional, BCAST, &all(4)), 0, RECONCILED, None)));
|
||||
case("bearer_not_held", "A reconciliation over an instance of the group the receiver holds no declaration of.", "receiver",
|
||||
rec(&reconcile(SINK_A, &did_key(&pk(SINK_A)), &p.g4, reconciliation_body(&p.g4, Bearer::Unidirectional, NOT_HELD, &all(4)), 0, RECONCILED, None)));
|
||||
let mut five = reconciliation_body(&p.g4, Bearer::Unidirectional, BCAST, &[0, 1, 2, 3]);
|
||||
five.leaf_count = 5;
|
||||
five.received = vec![0xf8];
|
||||
case("leaf_count_mismatch", "A leaf count of 5 for a group of four.", "receiver", rec(&reconcile_unchecked(SINK_A, SubjectType::Receipt, &p.g4, five)));
|
||||
let largest = r.whole.to_json().replacen("\"leaf_count\":4,", "\"leaf_count\":9007199254740991,", 1);
|
||||
assert_ne!(largest, r.whole.to_json(), "the claim states its leaf_count");
|
||||
case("leaf_count_largest", "Sink A's claim with a leaf_count of 2^53 − 1 beside its bitmap of one byte: not the group's, and nothing is sized by it.",
|
||||
"receiver", Record::Reconciliation(largest));
|
||||
let mut root = reconciliation_body(&p.g4, Bearer::Multicast, MCAST, &[0, 2]);
|
||||
root.received_root = [0; 32];
|
||||
case("root_mismatch", "A partial reconciliation whose root is not the one its objects give.", "receiver",
|
||||
rec(&reconcile_unchecked(SINK_A, SubjectType::Receipt, &p.g4, root)));
|
||||
let mut rendered = reconciliation_body(&p.g4, Bearer::Unidirectional, BCAST, &all(4));
|
||||
rendered.attests = "rendered".into();
|
||||
case("attests_other", "attests \"rendered\".", "receiver", rec(&reconcile_unchecked(SINK_A, SubjectType::Receipt, &p.g4, rendered)));
|
||||
let mut none = reconciliation_body(&p.g4, Bearer::Unidirectional, BCAST, &[]);
|
||||
none.received_root = merkle_root(&[]);
|
||||
case("no_bit_set", "A reconciliation of no object.", "receiver", rec(&reconcile_unchecked(SINK_A, SubjectType::Receipt, &p.g4, none)));
|
||||
case("subject_type_object", "subject_type \"object\", which the class does not admit.", "receiver",
|
||||
rec(&reconcile_unchecked(SINK_A, SubjectType::Object, &p.g4, reconciliation_body(&p.g4, Bearer::Unidirectional, BCAST, &all(4)))));
|
||||
case("objects_not_held_by_sender", "A partial reconciliation, the sender's context holding no object receipt to derive its root from.", "no_objects",
|
||||
rec(&r.partial));
|
||||
let mut bad = r.held.to_bytes();
|
||||
let n = bad.len() - 1;
|
||||
bad[n] ^= 0x01;
|
||||
case("holdings_unreadable", "A holdings record whose signature does not verify.", "receiver", Record::Holdings(bad));
|
||||
out
|
||||
}
|
||||
|
||||
fn message_cases(r: &Records, recv: &Context) -> Vec<Value> {
|
||||
let whole = compact(&rec(&r.whole), recv);
|
||||
let held = compact(&hold(&r.held), recv);
|
||||
let chained = compact(&rec(&r.chained), recv);
|
||||
let case = |name: &str, description: &str, record: &[u8], number: u8, frame: usize| {
|
||||
let fragments = cut(record, number, frame).expect("a frame the corpus cuts for");
|
||||
json!({
|
||||
"name": name,
|
||||
"description": description,
|
||||
"record": hex(record),
|
||||
"message": number,
|
||||
"frame": frame,
|
||||
"outcome": { "fragments": fragments.iter().map(|f| hex(f)).collect::<Vec<_>>() },
|
||||
})
|
||||
};
|
||||
vec![
|
||||
case("whole_frame_11", "whole_did_key's compact record in frames of 11 bytes.", &whole, 0x01, 11),
|
||||
case("whole_frame_13", "The same in frames of 13 bytes.", &whole, 0x02, 13),
|
||||
case("whole_frame_51", "The same in frames of 51 bytes.", &whole, 0x03, 51),
|
||||
case("whole_frame_140", "The same in one frame of 140 bytes.", &whole, 0x04, 140),
|
||||
case("holdings_frame_51", "holdings_by_reference's in frames of 51 bytes.", &held, 0x05, 51),
|
||||
case("holdings_frame_222", "The same in one frame of 222 bytes.", &held, 0x06, 222),
|
||||
case("chained_frame_222", "partial_chained's in one frame of 222 bytes.", &chained, 0x07, 222),
|
||||
]
|
||||
}
|
||||
|
||||
fn reassembly_cases(r: &Records, recv: &Context) -> Vec<Value> {
|
||||
let whole = compact(&rec(&r.whole), recv);
|
||||
let held = compact(&hold(&r.held), recv);
|
||||
let chained = compact(&rec(&r.chained), recv);
|
||||
// Sink D's compact record has the same length as sink A's, so cut under one number in one
|
||||
// frame its fragments carry the same headers.
|
||||
let other = {
|
||||
let mut ctx = recv.clone();
|
||||
ctx.groups[0].bearers.push(twins().0);
|
||||
compact(&rec(&r.twin), &ctx)
|
||||
};
|
||||
assert_eq!(other.len(), whole.len(), "two records of one length");
|
||||
let f11 = cut(&whole, 0x01, 11).expect("frames of 11");
|
||||
let f13 = cut(&whole, 0x02, 13).expect("frames of 13");
|
||||
let h51 = cut(&held, 0x05, 51).expect("frames of 51");
|
||||
let one = cut(&chained, 0x07, 222).expect("one frame");
|
||||
let same_header = cut(&other, 0x01, 11).expect("frames of 11");
|
||||
let other_number = cut(&other, 0x09, 11).expect("frames of 11");
|
||||
let other_count = cut(&held, 0x01, 11).expect("frames of 11");
|
||||
assert_ne!(other_count[0][2], f11[0][2], "another count");
|
||||
let mut out = Vec::new();
|
||||
let mut case = |name: &str, description: &str, fragments: Vec<Vec<u8>>, outcome: Value| {
|
||||
out.push(json!({ "name": name, "description": description, "fragments": fragments.iter().map(|f| hex(f)).collect::<Vec<_>>(), "outcome": outcome }));
|
||||
};
|
||||
let ok = |record: &[u8]| json!({ "outcome": "reassembled", "record": hex(record) });
|
||||
case("in_order", "whole_frame_11's fragments in order.", f11.clone(), ok(&whole));
|
||||
let mut reversed = f13.clone();
|
||||
reversed.reverse();
|
||||
case("arrival_reversed", "whole_frame_13's fragments last first: each is placed by its index.", reversed, ok(&whole));
|
||||
let mut shuffled = h51.clone();
|
||||
shuffled.rotate_left(2);
|
||||
shuffled.swap(0, 1);
|
||||
case("arrival_shuffled", "holdings_frame_51's fragments in another order.", shuffled, ok(&held));
|
||||
case("one_fragment", "chained_frame_222's one fragment.", one, ok(&chained));
|
||||
let without = |i: usize| {
|
||||
let mut f = f11.clone();
|
||||
f.remove(i);
|
||||
f
|
||||
};
|
||||
case("missing_middle", "Fragment 4 of whole_frame_11 missing.", without(4), refuse(FRAGMENT_MISSING));
|
||||
case("missing_last", "Its last fragment missing.", without(f11.len() - 1), refuse(FRAGMENT_MISSING));
|
||||
case("none", "No fragment.", Vec::new(), refuse(FRAGMENT_MISSING));
|
||||
let mut twice = f11.clone();
|
||||
twice.insert(3, f11[2].clone());
|
||||
case("delivered_twice", "Fragment 2 twice, byte for byte: one fragment, held once.", twice, ok(&whole));
|
||||
let mut differing = f11.clone();
|
||||
let mut copy = f11[2].clone();
|
||||
copy[HEADER] ^= 0x01;
|
||||
differing.push(copy);
|
||||
case("repeated_other_bytes", "A second fragment 2, with other data.", differing, refuse(FRAGMENT_REPEATED));
|
||||
let mut relabelled = f11.clone();
|
||||
relabelled[3][1] = 4;
|
||||
case("relabelled_onto_another", "Fragment 3 relabelled as fragment 4.", relabelled, refuse(FRAGMENT_REPEATED));
|
||||
let mut exchanged = f11.clone();
|
||||
let (a, b) = (f11[3][HEADER..].to_vec(), f11[4][HEADER..].to_vec());
|
||||
exchanged[3] = [&f11[3][..HEADER], &b[..]].concat();
|
||||
exchanged[4] = [&f11[4][..HEADER], &a[..]].concat();
|
||||
case("reordered_data_exchanged", "The data of fragments 3 and 4 exchanged, their headers kept: a reordered fragment.", exchanged,
|
||||
refuse(FRAGMENT_CHECK_FAILED));
|
||||
let mut swapped_index = f11.clone();
|
||||
swapped_index[3][1] = 4;
|
||||
swapped_index[4][1] = 3;
|
||||
case("reordered_index_exchanged", "The indices of fragments 3 and 4 exchanged: each placed where the other was cut from.", swapped_index,
|
||||
refuse(FRAGMENT_CHECK_FAILED));
|
||||
let mut foreign_number = f11.clone();
|
||||
foreign_number[2] = other_number[2].clone();
|
||||
case("foreign_number", "Fragment 2 of another message, under another number.", foreign_number, refuse(FRAGMENT_FOREIGN));
|
||||
let mut foreign_count = f11.clone();
|
||||
foreign_count[2] = other_count[2].clone();
|
||||
case("foreign_count", "Fragment 2 of another message under the same number, with another count.", foreign_count, refuse(FRAGMENT_FOREIGN));
|
||||
let mut foreign_same = f11.clone();
|
||||
foreign_same[2] = same_header[2].clone();
|
||||
case("foreign_same_header", "Fragment 2 of another message under the same number and count: only the check finds it.", foreign_same,
|
||||
refuse(FRAGMENT_CHECK_FAILED));
|
||||
let mut flipped = f11.clone();
|
||||
flipped[5][HEADER + 2] ^= 0x10;
|
||||
case("data_changed", "A bit of fragment 5's data flipped.", flipped, refuse(FRAGMENT_CHECK_FAILED));
|
||||
let mut header_only = f11.clone();
|
||||
header_only[1].truncate(HEADER);
|
||||
case("header_only", "A fragment of its header alone.", header_only, refuse(FRAGMENT_MALFORMED));
|
||||
let mut count_zero = f11.clone();
|
||||
count_zero[0][2] = 0;
|
||||
case("count_zero", "A fragment with a count of 0.", count_zero, refuse(FRAGMENT_MALFORMED));
|
||||
let mut past = f11.clone();
|
||||
let n = past.len() as u8;
|
||||
past[1][1] = n;
|
||||
case("index_past_count", "A fragment whose index is its count.", past, refuse(FRAGMENT_MALFORMED));
|
||||
case("check_without_record", "One fragment holding a check and no record.", vec![[&[0x0a, 0, 1][..], &check(&[])[..]].concat()],
|
||||
refuse(FRAGMENT_CHECK_FAILED));
|
||||
out
|
||||
}
|
||||
|
||||
/// The corpus: `vectors.json`, from fixed seeds.
|
||||
pub fn generate() -> Vec<(String, String)> {
|
||||
let p = pubs();
|
||||
let r = records(&p);
|
||||
let groups = groups_json(&p);
|
||||
let contexts = contexts();
|
||||
let ctx = built(&contexts, &groups);
|
||||
let mut keys = Map::new();
|
||||
for (name, seed) in [("publisher", PUBLISHER), ("sink_a", SINK_A), ("sink_b", SINK_B), ("sink_c", SINK_C), ("sink_d", SINK_D), ("holder", HOLDER), ("neighbour", NEIGHBOUR)] {
|
||||
keys.insert(name.into(), hex(&pk(seed)).into());
|
||||
}
|
||||
let doc = json!({
|
||||
"about": "Conformance corpus of extensions/bearer-receipts.md §13, the compact return profile: reconciliations (§5) and \
|
||||
holdings records (holdings §4) in compact form against a receiver's context (§13.1 to §13.5), and their \
|
||||
messages in fragments (§13.6, §13.7). Every key is Ed25519 from a fixed 32-byte seed and attests nothing; \
|
||||
envelopes carry opaque test payloads no case decodes. Each case states the outcome it was written to have, \
|
||||
declared in the generator. Regenerate with `wai_compact_return_vectors gen`.",
|
||||
"keys": keys,
|
||||
"groups": groups,
|
||||
"contexts": contexts.iter().map(|(n, v)| (n.to_string(), v.clone())).collect::<Map<String, Value>>(),
|
||||
"records": record_cases(&r, &ctx),
|
||||
"expansions": expansion_cases(&r, &ctx),
|
||||
"encodings": encoding_cases(&p, &r),
|
||||
"messages": message_cases(&r, &ctx["receiver"]),
|
||||
"reassemblies": reassembly_cases(&r, &ctx["receiver"]),
|
||||
});
|
||||
let mut s = serde_json::to_string_pretty(&doc).expect("serialises");
|
||||
s.push('\n');
|
||||
vec![("vectors.json".into(), s)]
|
||||
}
|
||||
|
||||
// ───────────────────────────── reading it back ─────────────────────────────
|
||||
|
||||
fn bytes_of(v: &Value) -> Result<Vec<u8>, String> {
|
||||
v.as_str().and_then(unhex).ok_or_else(|| format!("not hex: {v}"))
|
||||
}
|
||||
|
||||
fn strings(v: &Value, out: &mut Vec<String>) {
|
||||
match v {
|
||||
Value::String(s) => out.push(s.clone()),
|
||||
Value::Array(a) => a.iter().for_each(|x| strings(x, out)),
|
||||
Value::Object(m) => m.values().for_each(|x| strings(x, out)),
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
|
||||
/// Check a corpus's files against this module: each case's outcome, every round trip both ways,
|
||||
/// every cut reassembling to its record, and every reason of §13.9. Returns the failures.
|
||||
pub fn verify(files: &BTreeMap<String, String>) -> Vec<String> {
|
||||
let mut fails = Vec::new();
|
||||
let Some(text) = files.get("vectors.json") else { return vec!["no vectors.json".into()] };
|
||||
let doc: Value = match serde_json::from_str(text) {
|
||||
Ok(d) => d,
|
||||
Err(e) => return vec![format!("vectors.json: {e}")],
|
||||
};
|
||||
let mut contexts: BTreeMap<String, Context> = BTreeMap::new();
|
||||
for (name, v) in doc["contexts"].as_object().into_iter().flatten() {
|
||||
match context_of(v, &doc["groups"]) {
|
||||
Ok(c) => {
|
||||
contexts.insert(name.clone(), c);
|
||||
}
|
||||
Err(e) => fails.push(format!("context {name}: {e}")),
|
||||
}
|
||||
}
|
||||
let ctx = |c: &Value| contexts.get(c["context"].as_str().unwrap_or("")).ok_or_else(|| format!("no context {}", c["context"]));
|
||||
let empty = Vec::new();
|
||||
let mut stated = Vec::new();
|
||||
let mut check = |kind: &str, run: &dyn Fn(&Value) -> Result<Value, String>| {
|
||||
let cases = doc[kind].as_array().unwrap_or(&empty);
|
||||
if cases.is_empty() {
|
||||
fails.push(format!("no {kind} cases"));
|
||||
}
|
||||
for c in cases {
|
||||
let name = c["name"].as_str().unwrap_or("?");
|
||||
match run(c) {
|
||||
Err(e) => fails.push(format!("{name}: {e}")),
|
||||
Ok(got) if got != c["outcome"] => fails.push(format!("{name}: stated {} got {got}", c["outcome"])),
|
||||
Ok(_) => {}
|
||||
}
|
||||
strings(&c["outcome"], &mut stated);
|
||||
}
|
||||
};
|
||||
check("records", &|c| {
|
||||
let ctx = ctx(c)?;
|
||||
let record = record_of(c["kind"].as_str().unwrap_or(""), &c["record"])?;
|
||||
let stated = bytes_of(&c["compact"])?;
|
||||
let written = encode(&record, ctx).map_err(|r| format!("not written: {r}"))?;
|
||||
if written != stated {
|
||||
return Err(format!("written as {}", hex(&written)));
|
||||
}
|
||||
let back = expand(&stated, ctx).map_err(|r| format!("not expanded: {r}"))?;
|
||||
if back != record {
|
||||
return Err("expands to another record".into());
|
||||
}
|
||||
Ok(json!({ "verdict": verdict(&record, ctx) }))
|
||||
});
|
||||
check("expansions", &|c| {
|
||||
let ctx = ctx(c)?;
|
||||
Ok(match expand(&bytes_of(&c["compact"])?, ctx) {
|
||||
Ok(record) => expand_outcome(&record, &verdict(&record, ctx)),
|
||||
Err(r) => refuse(r),
|
||||
})
|
||||
});
|
||||
check("encodings", &|c| {
|
||||
let record = record_of(c["kind"].as_str().unwrap_or(""), &c["record"])?;
|
||||
Ok(match encode(&record, ctx(c)?) {
|
||||
Ok(b) => json!({ "outcome": "encode", "compact": hex(&b) }),
|
||||
Err(r) => refuse(r),
|
||||
})
|
||||
});
|
||||
check("messages", &|c| {
|
||||
let record = bytes_of(&c["record"])?;
|
||||
let number = c["message"].as_u64().and_then(|n| u8::try_from(n).ok()).ok_or("message")?;
|
||||
let frame = c["frame"].as_u64().ok_or("frame")? as usize;
|
||||
let fragments = cut(&record, number, frame).ok_or("not cut")?;
|
||||
if reassemble(&fragments) != Ok(record) {
|
||||
return Err("its fragments do not reassemble to it".into());
|
||||
}
|
||||
Ok(json!({ "fragments": fragments.iter().map(|f| hex(f)).collect::<Vec<_>>() }))
|
||||
});
|
||||
check("reassemblies", &|c| {
|
||||
let fragments: Vec<Vec<u8>> = c["fragments"].as_array().ok_or("fragments")?.iter().map(bytes_of).collect::<Result<_, _>>()?;
|
||||
Ok(match reassemble(&fragments) {
|
||||
Ok(record) => json!({ "outcome": "reassembled", "record": hex(&record) }),
|
||||
Err(r) => refuse(r),
|
||||
})
|
||||
});
|
||||
for code in CODES {
|
||||
if !stated.iter().any(|s| s == code) {
|
||||
fails.push(format!("no case gives {code}"));
|
||||
}
|
||||
}
|
||||
fails
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn the_corpus_reaches_every_stated_outcome() {
|
||||
let files: BTreeMap<String, String> = generate().into_iter().collect();
|
||||
let fails = verify(&files);
|
||||
assert!(fails.is_empty(), "{}", fails.join("\n"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_changed_outcome_fails_verification() {
|
||||
let mut files: BTreeMap<String, String> = generate().into_iter().collect();
|
||||
let text = files["vectors.json"].replacen("\"fragment_repeated\"", "\"fragment_missing\"", 1);
|
||||
files.insert("vectors.json".into(), text);
|
||||
assert!(!verify(&files).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_twins_share_a_reference() {
|
||||
let (a, b) = twins();
|
||||
assert_ne!(a, b);
|
||||
assert_eq!(bearer_ref(&a), bearer_ref(&b));
|
||||
}
|
||||
}
|
||||
|
|
@ -850,6 +850,13 @@ pub mod conform_result;
|
|||
/// cbindgen:ignore
|
||||
#[cfg(feature = "bearer_receipts")]
|
||||
pub mod bearer_receipts;
|
||||
// The compact return profile (extensions/bearer-receipts.md §13): a reconciliation and a holdings
|
||||
// record in a binary form that names by reference what the receiver holds, expanded to the
|
||||
// record's own bytes before it is verified, and carried in fragments with a check over the whole
|
||||
// message. With both records' features, and no feature of its own. Rust-only.
|
||||
/// cbindgen:ignore
|
||||
#[cfg(all(feature = "bearer_receipts", feature = "holdings"))]
|
||||
pub mod compact_return;
|
||||
// Carousel schedule (extensions/carousel.md): `wai.bearer.carousel` — when each object of a group
|
||||
// passes on a bearer instance with no return path and with which forward error correction; when a
|
||||
// receiver within a loss bound holds it, and the passes a receiver listens to. Rust-only.
|
||||
|
|
|
|||
99
wai/wai-rs/tests/compact_return.rs
Normal file
99
wai/wai-rs/tests/compact_return.rs
Normal file
|
|
@ -0,0 +1,99 @@
|
|||
//! The compact return profile (extensions/bearer-receipts.md §13): the committed corpus
|
||||
//! `live-conformance/compact-return/` regenerates byte for byte from its fixed seeds, holds no other
|
||||
//! file, every case in it gets the outcome it states, §13.9 lists exactly the reasons the reference
|
||||
//! reports, and §13.8's floors are the lengths of the reports of telemetry-aggregation's corpus,
|
||||
//! read with the reference's DAP reader.
|
||||
#![cfg(all(feature = "bearer_receipts", feature = "holdings", feature = "telemetry_aggregate"))]
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::path::PathBuf;
|
||||
use wai::compact_return::{corpus, CODES};
|
||||
use wai::telemetry_aggregate::{dap, measure::MEASURES};
|
||||
|
||||
fn wai_dir() -> PathBuf {
|
||||
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("..")
|
||||
}
|
||||
|
||||
fn committed() -> BTreeMap<String, String> {
|
||||
let dir = wai_dir().join("live-conformance/compact-return");
|
||||
std::fs::read_dir(&dir)
|
||||
.expect("the committed corpus")
|
||||
.flatten()
|
||||
.map(|e| (e.file_name().to_string_lossy().into_owned(), e.path()))
|
||||
.filter(|(n, _)| n != "README.md")
|
||||
.map(|(n, p)| (n, std::fs::read_to_string(p).expect("a text file")))
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn unhex(s: &str) -> Vec<u8> {
|
||||
(0..s.len()).step_by(2).map(|i| u8::from_str_radix(&s[i..i + 2], 16).expect("hex")).collect()
|
||||
}
|
||||
|
||||
/// The text of one `###` section of bearer-receipts, from its heading to the next heading.
|
||||
fn section(heading: &str) -> String {
|
||||
let text = std::fs::read_to_string(wai_dir().join("extensions/bearer-receipts.md")).expect("the extension");
|
||||
let at = text.find(&format!("\n### {heading}")).unwrap_or_else(|| panic!("a section {heading}"));
|
||||
let rest = &text[at + 1..];
|
||||
let end = rest.find("\n#").map_or(rest.len(), |e| e + 1);
|
||||
rest[..end].to_string()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_committed_corpus_regenerates_from_its_seeds() {
|
||||
let files = committed();
|
||||
let generated: BTreeMap<String, String> = corpus::generate().into_iter().collect();
|
||||
let extra: Vec<&String> = files.keys().filter(|k| !generated.contains_key(*k)).collect();
|
||||
let missing: Vec<&String> = generated.keys().filter(|k| !files.contains_key(*k)).collect();
|
||||
let differ: Vec<&String> = generated.iter().filter(|(k, v)| files.get(*k).is_some_and(|f| f != *v)).map(|(k, _)| k).collect();
|
||||
assert!(
|
||||
extra.is_empty() && missing.is_empty() && differ.is_empty(),
|
||||
"extra {extra:?}, missing {missing:?}, differ {differ:?}: run `wai_compact_return_vectors gen`"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_case_gets_its_stated_outcome() {
|
||||
let fails = corpus::verify(&committed());
|
||||
assert!(fails.is_empty(), "{}", fails.join("\n"));
|
||||
}
|
||||
|
||||
/// §13.9 lists exactly the codes `CODES` has, each once, in order.
|
||||
#[test]
|
||||
fn the_extension_lists_the_reasons() {
|
||||
let listed: Vec<String> =
|
||||
section("13.9 Reasons").lines().filter(|l| l.starts_with("| `")).filter_map(|l| l.split('`').nth(1).map(String::from)).collect();
|
||||
let want: Vec<String> = CODES.iter().map(|c| c.to_string()).collect();
|
||||
assert_eq!(listed, want, "§13.9's table and CODES differ");
|
||||
}
|
||||
|
||||
/// §13.8's floor of each of telemetry-aggregation's measures 1 to 4 is the length of its longest
|
||||
/// report in that extension's corpus, and every report of one task has that length.
|
||||
#[test]
|
||||
fn the_floors_are_the_reports_lengths() {
|
||||
let rows: Vec<(u8, String, usize)> = section("13.8 ")
|
||||
.lines()
|
||||
.filter(|l| l.starts_with("| ") && l.contains('`'))
|
||||
.map(|l| {
|
||||
let cells: Vec<&str> = l.trim_matches('|').split('|').map(str::trim).collect();
|
||||
let code: u8 = cells[0].split(' ').next().and_then(|c| c.parse().ok()).expect("a measure's code");
|
||||
let name = cells[0].split('`').nth(1).expect("a measure's name").to_string();
|
||||
let floor: usize = cells[1].replace(',', "").parse().expect("a floor");
|
||||
(code, name, floor)
|
||||
})
|
||||
.collect();
|
||||
let codes: Vec<u8> = rows.iter().map(|r| r.0).collect();
|
||||
assert_eq!(codes, vec![1, 2, 3, 4], "§13.8 states the floors of measures 1 to 4");
|
||||
let text = std::fs::read_to_string(wai_dir().join("live-conformance/aggregate/vectors.json")).expect("the aggregate corpus");
|
||||
let doc: serde_json::Value = serde_json::from_str(&text).expect("JSON");
|
||||
for (code, name, floor) in rows {
|
||||
let named = MEASURES.iter().find(|(c, _)| *c == code).map(|(_, n)| *n);
|
||||
assert_eq!(named, Some(name.as_str()), "measure {code}'s name");
|
||||
let flow = doc["flows"].as_array().expect("flows").iter().find(|f| f["name"] == name.as_str()).expect("a flow of the measure");
|
||||
let upload = unhex(flow["upload"].as_str().expect("an upload"));
|
||||
let reports = dap::decode_upload(&upload).expect("an upload the reference reads");
|
||||
assert!(!reports.is_empty());
|
||||
let lengths: std::collections::BTreeSet<usize> = reports.iter().map(|r| r.encode().len()).collect();
|
||||
assert_eq!(lengths.len(), 1, "every report of {name} has one length");
|
||||
assert_eq!(lengths.into_iter().next(), Some(floor), "{name}'s floor");
|
||||
}
|
||||
}
|
||||
|
|
@ -15,6 +15,7 @@ import json
|
|||
import os
|
||||
import re
|
||||
import socket
|
||||
import socketserver
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
|
|
@ -55,6 +56,14 @@ class Room(http.server.BaseHTTPRequestHandler):
|
|||
pass
|
||||
|
||||
|
||||
class Server(http.server.ThreadingHTTPServer):
|
||||
"""HTTPServer looks up a name for its address before it listens, and nothing here reads the
|
||||
name. On some hosts that lookup takes 35 s, past what `wai dev` waits for a service."""
|
||||
|
||||
def server_bind(self):
|
||||
socketserver.TCPServer.server_bind(self)
|
||||
|
||||
|
||||
def main(argv):
|
||||
if argv[:1] in (["keygen"], ["relay-lines"]):
|
||||
open(argv[1], "a").close()
|
||||
|
|
@ -73,7 +82,7 @@ def main(argv):
|
|||
while True:
|
||||
c, _ = s.accept()
|
||||
c.close()
|
||||
http.server.ThreadingHTTPServer(listen_addr(config, "listen"), Room).serve_forever()
|
||||
Server(listen_addr(config, "listen"), Room).serve_forever()
|
||||
return 0
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue