smart-byte: generate the conformance pack from the reference implementation #99
35 changed files with 6356 additions and 436 deletions
25
.github/workflows/conformance.yml
vendored
25
.github/workflows/conformance.yml
vendored
|
|
@ -84,6 +84,31 @@ jobs:
|
|||
exit 1
|
||||
fi
|
||||
|
||||
# The pack's values come from the Rust reference implementation, and
|
||||
# standards-build checks them against it under `cargo test`. This job checks
|
||||
# them against a second implementation, written separately in another
|
||||
# language, so a mistake shared by the generator and its own verifier still
|
||||
# fails here. Standard library only: nothing to install beyond python3.
|
||||
crosscheck:
|
||||
name: Second implementation reproduces every vector
|
||||
runs-on: linux
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Ensure python3 is available
|
||||
shell: bash
|
||||
run: |
|
||||
if ! command -v python3 >/dev/null 2>&1; then
|
||||
SUDO=""; [ "$(id -u)" -ne 0 ] && SUDO="sudo"
|
||||
$SUDO apt-get update -qq
|
||||
$SUDO apt-get install -y -qq python3
|
||||
fi
|
||||
python3 --version
|
||||
|
||||
- name: Cross-check v1
|
||||
run: python3 smart-byte/conformance/crosscheck/verify_v1.py smart-byte/conformance/v1
|
||||
|
||||
spec-vector-coupling:
|
||||
name: Spec changes require vector-pack updates
|
||||
runs-on: linux
|
||||
|
|
|
|||
|
|
@ -17,21 +17,24 @@ contain no `$ref` pointers, no external URLs, and no runtime dependencies.
|
|||
## What is in the pack
|
||||
|
||||
Each spec version has its own subdirectory (`v1/`, `v2/`, ...). Today only
|
||||
`v1/` exists; it corresponds to the spec text in
|
||||
`../spec/treatise_v1_parts_I_II_III_combined.md`.
|
||||
`v1/` exists; it corresponds to the spec text in `../spec/`
|
||||
(`treatise_v1_parts_I_II_III_combined.md` and
|
||||
`identity_and_key_rotation.md`).
|
||||
|
||||
A pack for a given version contains five JSON files plus `regeneration.md`:
|
||||
|
||||
| File | Tests |
|
||||
| -------------------------------- | -------------------------------------------------------- |
|
||||
| `canonicalisation_vectors.json` | CBOR canonical-encoding rules (§8 wire format) |
|
||||
| `said_vectors.json` | SAID computation: BLAKE3-256 over canonical CBOR (§8.2) |
|
||||
| `signature_vectors.json` | Ed25519 sign + verify over SAIDs (§8.3, §8.4) |
|
||||
| `lockstep_vectors.json` | BFT supermajority commit logic (§9, §13) |
|
||||
| `gossip_vectors.json` | Gossip propagation, conflict detection, ordering (§17) |
|
||||
| File | Tests |
|
||||
| -------------------------------- | --------------------------------------------------------------------- |
|
||||
| `canonicalisation_vectors.json` | Canonical CBOR: each encoding rule, accepted and refused (identity §2.3) |
|
||||
| `said_vectors.json` | SAID derivation and verification (identity §2.1–§2.5; treatise §8.2) |
|
||||
| `signature_vectors.json` | Ed25519 signatures over SAIDs (treatise §8.3, §8.4; RFC 8032) |
|
||||
| `lockstep_vectors.json` | Supermajority commit, frame outcomes, dissent log (treatise §13, §14) |
|
||||
| `gossip_vectors.json` | Receiving SAID-addressed objects (identity §2.6; treatise §11.4, §21.4) |
|
||||
|
||||
`regeneration.md` documents how to regenerate the vectors from the Rust
|
||||
reference implementation (`smart-byte-rs`, landing alongside spec v1.0).
|
||||
Every value in the pack is generated by the Rust reference implementation
|
||||
(`../smart-byte-rs`, crate `smart-byte-conformance`) and reproduced by an
|
||||
independent second implementation, `crosscheck/verify_v1.py`.
|
||||
`regeneration.md` documents both, and records every change to the pack.
|
||||
|
||||
## How to consume the vectors
|
||||
|
||||
|
|
@ -60,6 +63,92 @@ The conventional flow for an implementation under test (IUT):
|
|||
There is no scoring, no "mostly conforms" status, and no waiver process.
|
||||
A vector pack is a hard gate.
|
||||
|
||||
### Byte strings in JSON
|
||||
|
||||
JSON has no byte-string type, so an `input` writes a CBOR byte string as an
|
||||
object whose only member is `$bytes`, holding lowercase hex:
|
||||
`{"$bytes": "00ff"}`. `$bytes` is reserved: no map in the pack has a member
|
||||
of that name. Every other JSON value maps to CBOR directly: object → map with
|
||||
text keys, array → array, string → text string, integer → integer,
|
||||
`true` / `false` / `null` → the simple values. The pack contains no JSON number
|
||||
with a fraction or exponent, because identity spec §2.3 forbids floats, and no
|
||||
object names a member twice.
|
||||
|
||||
Where a vector supplies raw bytes instead (`*_cbor_hex`, `input_cbor_hex`),
|
||||
they are lowercase hex.
|
||||
|
||||
### Fields, file by file
|
||||
|
||||
- **`canonicalisation_vectors.json`** — a positive vector has `input` and
|
||||
`expected_cbor_hex`; a negative one has `input_cbor_hex`, which a strict
|
||||
decoder must refuse with `error`. Refusing means not decoding: an IUT must
|
||||
never re-encode non-canonical input into canonical form and accept it.
|
||||
- **`said_vectors.json`** — `said_field` names the member that carries the
|
||||
SAID (`d`). A positive vector's `input` holds the 53-`#` placeholder there.
|
||||
`canonical_cbor_hex` is the canonical encoding of `input` (the bytes that
|
||||
are hashed), `blake3_hex` is their BLAKE3-256, `expected_said` is that
|
||||
digest written `E` + lowercase unpadded base32, and `said_cbor_hex` is the
|
||||
object with its SAID in place, which must verify. A negative vector's
|
||||
`input_cbor_hex` must fail SAID verification with `error`. The adversarial
|
||||
vectors also give, for illustration, the SAIDs a lenient verifier would
|
||||
compute (`said_of_received_bytes`, `said_after_reencoding`).
|
||||
- **`signature_vectors.json`** — the Ed25519 rules every Smart Byte signature
|
||||
follows: RFC 8032, with strict decoding of keys and scalars. The message is
|
||||
`signed_message_hex`, the 32 digest bytes `envelope_said` spells — not its
|
||||
text. `verifying_key` is the key handed to the verifier; when
|
||||
`signing_key_seed` is not null, it is the key that seed derives. Verify
|
||||
`signature` under `verifying_key`: the result must be `expected_verify`,
|
||||
failing with `error` when false. A positive vector's `signature` is the one
|
||||
deterministic signature its seed makes.
|
||||
- **`lockstep_vectors.json`** — `node_count` is the cluster size n.
|
||||
`node_reported_hashes` are the opaque 32-byte state hashes each node
|
||||
broadcast, and `node_computed_hashes` the ones each computed; they differ
|
||||
only for a node that lies on the wire (treatise §13.4). Hashes are compared
|
||||
only for equality. The IUT computes `expected_threshold` (floor(2n/3)+1),
|
||||
then `expected_commit` and `expected_state_hash` from the reported hashes —
|
||||
the frame's public record. It also computes each node's `expected_outcomes`
|
||||
(`clean_commit`, `commit_with_dissent`, `minority_divergence` or
|
||||
`cluster_split`) and `expected_dissent_logs` from that node's own view: its
|
||||
peers' reported hashes plus the hash it computed.
|
||||
- **`gossip_vectors.json`** — `deliveries` arrive at `receiver` in order, each
|
||||
with the SAID it was announced or requested under (`claimed_said`) and its
|
||||
bytes (`cbor_hex`). Each result is `accepted` or `duplicate` with the SAID,
|
||||
or `rejected` with an error class, per identity spec §2.6. `expected_held`
|
||||
lists what the receiver holds afterwards, in acceptance order: each SAID
|
||||
with exactly the bytes delivered.
|
||||
|
||||
### Error classes
|
||||
|
||||
| Class | Meaning |
|
||||
| ------------------------- | ---------------------------------------------------------- |
|
||||
| `non_shortest_integer` | an integer not written with its shortest head |
|
||||
| `non_shortest_length` | a string, array or map length not written with its shortest head |
|
||||
| `unsorted_map_keys` | map keys not in the bytewise order of their encodings |
|
||||
| `duplicate_map_key` | a key appears twice in one map |
|
||||
| `float_forbidden` | a floating-point value |
|
||||
| `simple_value_forbidden` | a simple value other than `false`, `true` and `null` |
|
||||
| `tag_forbidden` | a CBOR tag |
|
||||
| `indefinite_length` | an indefinite-length item |
|
||||
| `invalid_utf8` | a text string that is not UTF-8 |
|
||||
| `trailing_bytes` | bytes after the one top-level item |
|
||||
| `said_mismatch` | content that does not hash to the SAID it states or arrived under |
|
||||
| `signature_invalid` | a signature that does not verify, including a non-canonical S |
|
||||
| `malformed_signature` | a signature that is not 64 bytes |
|
||||
| `malformed_verifying_key` | a key that is not 32 bytes, or encodes y outside the field |
|
||||
|
||||
The reference implementation also reports `truncated`, `malformed`,
|
||||
`too_deep`, `not_a_map`, `missing_said_field` and `malformed_said` for
|
||||
inputs no vector exercises. An input with several defects reports the first
|
||||
in stream order.
|
||||
|
||||
### Completeness
|
||||
|
||||
A pack is checked whole, not only vector by vector. Every file holds at least
|
||||
one vector, every vector the identity spec cites is present, and every error
|
||||
class in the table above is exercised by some vector — and no vector names a
|
||||
class the table does not list. Both checkers enforce this, so a pack cannot
|
||||
pass by losing vectors.
|
||||
|
||||
## Versioning rule
|
||||
|
||||
Vectors are versioned **per spec version**, not per implementation, not per
|
||||
|
|
@ -71,11 +160,13 @@ calendar release of the conformance pack.
|
|||
- `v2/` will appear when (and only when) the spec moves to a new version
|
||||
that introduces an incompatible wire-format change. The compatibility
|
||||
matrix lives in `VERSIONS.md`.
|
||||
- A bug in a vector pack (e.g. an `expected_*` field that disagrees with
|
||||
the spec) is fixed by superseding the vector with a new one whose `name`
|
||||
carries a suffix (`-fixed`, `-corrected`); the original is retained with
|
||||
a `"superseded_by"` field so historical IUTs that pinned to the old
|
||||
vector can detect the supersession.
|
||||
- In a frozen version, a bug in a vector pack (e.g. an `expected_*` field
|
||||
that disagrees with the spec) is fixed by superseding the vector with a
|
||||
new one whose `name` carries a suffix (`-fixed`, `-corrected`); the
|
||||
original is retained with a `"superseded_by"` field so historical IUTs
|
||||
that pinned to the old vector can detect the supersession. A DRAFT pack is
|
||||
regenerated in place instead, and every change is recorded in its
|
||||
`regeneration.md`.
|
||||
|
||||
## CI gate
|
||||
|
||||
|
|
@ -84,10 +175,17 @@ Every PR that touches `smart-byte/spec/` or `smart-byte/conformance/` must
|
|||
pass the gate. The gate validates:
|
||||
|
||||
- Every JSON file in `v*/` parses as valid JSON.
|
||||
- The second implementation, `crosscheck/verify_v1.py`, agrees with every
|
||||
vector, and finds the pack complete (see Completeness).
|
||||
- Every spec change is accompanied by a vector-pack update (a PR that
|
||||
modifies `smart-byte/spec/` without modifying `smart-byte/conformance/`
|
||||
is rejected at the gate).
|
||||
|
||||
Separately, `standards-build` runs `cargo test` in `smart-byte-rs` on every
|
||||
push. That checks every vector against the reference implementation, requires
|
||||
the published files to equal the generator's output byte for byte, and
|
||||
requires identity spec §2.4 to print the worked example's real values.
|
||||
|
||||
The gate's purpose is procedural: a spec change without a vector change is
|
||||
either a clarifying edit (which should declare itself by also touching the
|
||||
vector pack's `regeneration.md` to record the editorial reason) or a
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ Smart Byte spec versions.
|
|||
|
||||
| Vector pack | Spec version | Spec source | Status | Notes |
|
||||
| ----------- | ------------ | ------------------------------------------------------------- | -------- | --------------------------------------------------------------------- |
|
||||
| `v1/` | v1.0-draft | `../spec/treatise_v1_parts_I_II_III_combined.md` (Parts I-III) | DRAFT | Reference vectors; expected to be regenerated when `smart-byte-rs` lands. |
|
||||
| `v1/` | v1.0-draft | `../spec/treatise_v1_parts_I_II_III_combined.md` (Parts I-III), `../spec/identity_and_key_rotation.md` | DRAFT | Generated by the reference implementation (`smart-byte-conformance`) and reproduced by `crosscheck/verify_v1.py`. Regenerated in place while DRAFT (rule 4); every change is in `v1/regeneration.md`. |
|
||||
|
||||
Future entries are appended below this table when a new spec version is
|
||||
introduced. Older entries are never removed; an implementation that pins to
|
||||
|
|
@ -28,11 +28,13 @@ indefinitely.
|
|||
*additional* corners of the *same* canonical wire format, but no
|
||||
existing vector's `expected_*` outputs may be changed.
|
||||
|
||||
4. **Bug fixes via supersession.** A vector with an `expected_*` field that
|
||||
disagrees with the spec is fixed by adding a new vector (with a
|
||||
distinguishing `name` suffix) and adding a `"superseded_by": "<new-name>"`
|
||||
field to the original. Implementations pin to vector names, not array
|
||||
positions, so supersession is non-breaking.
|
||||
4. **Bug fixes via supersession, once frozen.** In a frozen version, a
|
||||
vector with an `expected_*` field that disagrees with the spec is fixed by
|
||||
adding a new vector (with a distinguishing `name` suffix) and adding a
|
||||
`"superseded_by": "<new-name>"` field to the original. Implementations pin
|
||||
to vector names, not array positions, so supersession is non-breaking. A
|
||||
DRAFT pack is regenerated in place, with every change recorded in its
|
||||
`regeneration.md`.
|
||||
|
||||
5. **Spec-version detection at the wire.** The wire format itself carries
|
||||
version information (see spec §8.3, §8.4 — the one-byte algorithm
|
||||
|
|
|
|||
733
smart-byte/conformance/crosscheck/verify_v1.py
Normal file
733
smart-byte/conformance/crosscheck/verify_v1.py
Normal file
|
|
@ -0,0 +1,733 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Second implementation of the Smart Byte v1 conformance pack.
|
||||
|
||||
The pack in ../v1 is generated by the Rust reference implementation
|
||||
(smart-byte-rs, crate smart-byte-conformance). This script is a separate
|
||||
implementation of everything the pack exercises, written in another language
|
||||
from the spec text rather than ported from the Rust: canonical CBOR and its
|
||||
strict decoder (identity spec §2.3), SAID derivation and verification (§2.1,
|
||||
§2.2), Ed25519 (RFC 8032), the lockstep commit rule and dissent log (treatise
|
||||
§13, §14), and the content-addressed gossip intake the gossip vectors describe.
|
||||
|
||||
It uses the Python standard library only. BLAKE3 and Ed25519 are implemented
|
||||
here from their specifications, and both are checked against published
|
||||
known-answer values before any vector is.
|
||||
|
||||
python3 verify_v1.py [PACK_DIR]
|
||||
|
||||
Exits 0 when every vector agrees with this implementation, 1 otherwise.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import json
|
||||
import pathlib
|
||||
import re
|
||||
import sys
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# BLAKE3 (hash mode, 32-byte output)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_M32 = 0xFFFFFFFF
|
||||
_IV = [0x6A09E667, 0xBB67AE85, 0x3C6EF372, 0xA54FF53A,
|
||||
0x510E527F, 0x9B05688C, 0x1F83D9AB, 0x5BE0CD19]
|
||||
_PERM = [2, 6, 3, 10, 7, 0, 4, 13, 1, 11, 12, 5, 9, 14, 15, 8]
|
||||
_CHUNK_START, _CHUNK_END, _PARENT, _ROOT = 1, 2, 4, 8
|
||||
_BLOCK, _CHUNK = 64, 1024
|
||||
|
||||
|
||||
def _rotr(x, n):
|
||||
return ((x >> n) | (x << (32 - n))) & _M32
|
||||
|
||||
|
||||
def _g(s, a, b, c, d, x, y):
|
||||
s[a] = (s[a] + s[b] + x) & _M32
|
||||
s[d] = _rotr(s[d] ^ s[a], 16)
|
||||
s[c] = (s[c] + s[d]) & _M32
|
||||
s[b] = _rotr(s[b] ^ s[c], 12)
|
||||
s[a] = (s[a] + s[b] + y) & _M32
|
||||
s[d] = _rotr(s[d] ^ s[a], 8)
|
||||
s[c] = (s[c] + s[d]) & _M32
|
||||
s[b] = _rotr(s[b] ^ s[c], 7)
|
||||
|
||||
|
||||
def _compress(cv, block, counter, block_len, flags):
|
||||
m = [int.from_bytes(block[4 * i:4 * i + 4], "little") for i in range(16)]
|
||||
s = list(cv) + _IV[:4] + [counter & _M32, (counter >> 32) & _M32, block_len, flags]
|
||||
for r in range(7):
|
||||
_g(s, 0, 4, 8, 12, m[0], m[1])
|
||||
_g(s, 1, 5, 9, 13, m[2], m[3])
|
||||
_g(s, 2, 6, 10, 14, m[4], m[5])
|
||||
_g(s, 3, 7, 11, 15, m[6], m[7])
|
||||
_g(s, 0, 5, 10, 15, m[8], m[9])
|
||||
_g(s, 1, 6, 11, 12, m[10], m[11])
|
||||
_g(s, 2, 7, 8, 13, m[12], m[13])
|
||||
_g(s, 3, 4, 9, 14, m[14], m[15])
|
||||
if r < 6:
|
||||
m = [m[i] for i in _PERM]
|
||||
return [s[i] ^ s[i + 8] for i in range(8)] + [s[i + 8] ^ cv[i] for i in range(8)]
|
||||
|
||||
|
||||
def _chunk_output(chunk, index):
|
||||
"""(cv, last block, block_len, flags) of a chunk, before its last compression."""
|
||||
blocks = [chunk[i:i + _BLOCK] for i in range(0, len(chunk), _BLOCK)] or [b""]
|
||||
cv = list(_IV)
|
||||
for i, block in enumerate(blocks):
|
||||
flags = (_CHUNK_START if i == 0 else 0) | (_CHUNK_END if i == len(blocks) - 1 else 0)
|
||||
padded = block.ljust(_BLOCK, b"\0")
|
||||
if i == len(blocks) - 1:
|
||||
return cv, padded, len(block), flags, index
|
||||
cv = _compress(cv, padded, index, _BLOCK, flags)[:8]
|
||||
|
||||
|
||||
def blake3(data):
|
||||
"""BLAKE3-256 of `data`."""
|
||||
chunks = [data[i:i + _CHUNK] for i in range(0, len(data), _CHUNK)] or [b""]
|
||||
stack = []
|
||||
for index, chunk in enumerate(chunks[:-1]):
|
||||
cv, block, n, flags, ctr = _chunk_output(chunk, index)
|
||||
node = _compress(cv, block, ctr, n, flags)[:8]
|
||||
total = index + 1
|
||||
while total & 1 == 0:
|
||||
left = stack.pop()
|
||||
node = _compress(_IV, _words(left) + _words(node), 0, _BLOCK, _PARENT)[:8]
|
||||
total >>= 1
|
||||
stack.append(node)
|
||||
out = _chunk_output(chunks[-1], len(chunks) - 1)
|
||||
while stack:
|
||||
cv, block, n, flags, ctr = out
|
||||
right = _compress(cv, block, ctr, n, flags)[:8]
|
||||
out = (_IV, _words(stack.pop()) + _words(right), _BLOCK, _PARENT, 0)
|
||||
cv, block, n, flags, ctr = out
|
||||
words = _compress(cv, block, ctr, n, flags | _ROOT)[:8]
|
||||
return b"".join(w.to_bytes(4, "little") for w in words)
|
||||
|
||||
|
||||
def _words(cv):
|
||||
return b"".join(w.to_bytes(4, "little") for w in cv)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Ed25519 (RFC 8032), with the strict checks the pack expects
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_P = 2 ** 255 - 19
|
||||
_L = 2 ** 252 + 27742317777372353535851937790883648493
|
||||
_D = -121665 * pow(121666, _P - 2, _P) % _P
|
||||
_SQRT_M1 = pow(2, (_P - 1) // 4, _P)
|
||||
|
||||
|
||||
def _recover_x(y, sign):
|
||||
if y >= _P:
|
||||
return None
|
||||
x2 = (y * y - 1) * pow(_D * y * y + 1, _P - 2, _P) % _P
|
||||
if x2 == 0:
|
||||
return None if sign else 0
|
||||
x = pow(x2, (_P + 3) // 8, _P)
|
||||
if (x * x - x2) % _P != 0:
|
||||
x = x * _SQRT_M1 % _P
|
||||
if (x * x - x2) % _P != 0:
|
||||
return None
|
||||
if x & 1 != sign:
|
||||
x = _P - x
|
||||
return x
|
||||
|
||||
|
||||
_BY = 4 * pow(5, _P - 2, _P) % _P
|
||||
_BX = _recover_x(_BY, 0)
|
||||
_B = (_BX, _BY, 1, _BX * _BY % _P)
|
||||
_ZERO = (0, 1, 1, 0)
|
||||
|
||||
|
||||
def _add(p, q):
|
||||
a = (p[1] - p[0]) * (q[1] - q[0]) % _P
|
||||
b = (p[1] + p[0]) * (q[1] + q[0]) % _P
|
||||
c = 2 * p[3] * q[3] * _D % _P
|
||||
d = 2 * p[2] * q[2] % _P
|
||||
e, f, g, h = b - a, d - c, d + c, b + a
|
||||
return (e * f % _P, g * h % _P, f * g % _P, e * h % _P)
|
||||
|
||||
|
||||
def _mul(k, p):
|
||||
q = _ZERO
|
||||
while k:
|
||||
if k & 1:
|
||||
q = _add(q, p)
|
||||
p = _add(p, p)
|
||||
k >>= 1
|
||||
return q
|
||||
|
||||
|
||||
def _eq(p, q):
|
||||
return (p[0] * q[2] - q[0] * p[2]) % _P == 0 and (p[1] * q[2] - q[1] * p[2]) % _P == 0
|
||||
|
||||
|
||||
def _encode_point(p):
|
||||
zi = pow(p[2], _P - 2, _P)
|
||||
x, y = p[0] * zi % _P, p[1] * zi % _P
|
||||
return (y | ((x & 1) << 255)).to_bytes(32, "little")
|
||||
|
||||
|
||||
def _decode_point(b):
|
||||
n = int.from_bytes(b, "little")
|
||||
y, sign = n & ((1 << 255) - 1), n >> 255
|
||||
x = _recover_x(y, sign)
|
||||
return None if x is None else (x, y, 1, x * y % _P)
|
||||
|
||||
|
||||
def _small_order(p):
|
||||
return _eq(_mul(8, p), _ZERO)
|
||||
|
||||
|
||||
def _sha512_int(*parts):
|
||||
return int.from_bytes(hashlib.sha512(b"".join(parts)).digest(), "little")
|
||||
|
||||
|
||||
def ed25519_public(seed):
|
||||
h = hashlib.sha512(seed).digest()
|
||||
a = int.from_bytes(h[:32], "little") & ((1 << 254) - 8) | (1 << 254)
|
||||
return _encode_point(_mul(a, _B))
|
||||
|
||||
|
||||
def ed25519_sign(seed, msg):
|
||||
h = hashlib.sha512(seed).digest()
|
||||
a = int.from_bytes(h[:32], "little") & ((1 << 254) - 8) | (1 << 254)
|
||||
pub = _encode_point(_mul(a, _B))
|
||||
r = _sha512_int(h[32:], msg) % _L
|
||||
big_r = _encode_point(_mul(r, _B))
|
||||
k = _sha512_int(big_r, pub, msg) % _L
|
||||
return big_r + ((r + k * a) % _L).to_bytes(32, "little")
|
||||
|
||||
|
||||
def ed25519_verify(pub, msg, sig):
|
||||
"""None if valid, else the pack's error class."""
|
||||
if len(sig) != 64:
|
||||
return "malformed_signature"
|
||||
if len(pub) != 32:
|
||||
return "malformed_verifying_key"
|
||||
a = _decode_point(pub)
|
||||
if a is None:
|
||||
return "malformed_verifying_key"
|
||||
r = _decode_point(sig[:32])
|
||||
s = int.from_bytes(sig[32:], "little")
|
||||
if r is None or s >= _L or _small_order(a) or _small_order(r):
|
||||
return "signature_invalid"
|
||||
k = _sha512_int(sig[:32], pub, msg) % _L
|
||||
if not _eq(_mul(s, _B), _add(r, _mul(k, a))):
|
||||
return "signature_invalid"
|
||||
return None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Canonical CBOR (identity spec §2.3)
|
||||
#
|
||||
# Values: int, bytes, str, list, Map (a list of (key, value) pairs), bool,
|
||||
# None. Python's bool is an int subclass, so bools are tested first.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class Map(list):
|
||||
"""A CBOR map as (key, value) pairs, in the order they were given."""
|
||||
|
||||
|
||||
class Refused(Exception):
|
||||
def __init__(self, cls):
|
||||
super().__init__(cls)
|
||||
self.cls = cls
|
||||
|
||||
|
||||
def _head(major, n):
|
||||
if n < 24:
|
||||
return bytes([major << 5 | n])
|
||||
for ai, width in ((24, 1), (25, 2), (26, 4), (27, 8)):
|
||||
if n < 1 << (8 * width):
|
||||
return bytes([major << 5 | ai]) + n.to_bytes(width, "big")
|
||||
raise ValueError("argument does not fit in 64 bits")
|
||||
|
||||
|
||||
def encode(v):
|
||||
if v is None:
|
||||
return b"\xf6"
|
||||
if v is True:
|
||||
return b"\xf5"
|
||||
if v is False:
|
||||
return b"\xf4"
|
||||
if isinstance(v, int):
|
||||
return _head(0, v) if v >= 0 else _head(1, -1 - v)
|
||||
if isinstance(v, bytes):
|
||||
return _head(2, len(v)) + v
|
||||
if isinstance(v, str):
|
||||
b = v.encode("utf-8")
|
||||
return _head(3, len(b)) + b
|
||||
if isinstance(v, Map):
|
||||
pairs = sorted((encode(k), encode(x)) for k, x in v)
|
||||
if any(pairs[i][0] == pairs[i + 1][0] for i in range(len(pairs) - 1)):
|
||||
raise Refused("duplicate_map_key")
|
||||
return _head(5, len(pairs)) + b"".join(k + x for k, x in pairs)
|
||||
if isinstance(v, list):
|
||||
return _head(4, len(v)) + b"".join(encode(x) for x in v)
|
||||
raise TypeError(f"no canonical form for {type(v).__name__}")
|
||||
|
||||
|
||||
def decode(data, strict=True):
|
||||
"""One canonical item, or Refused(class). strict=False accepts over-long
|
||||
heads and unsorted maps; it exists only to show what a lenient
|
||||
implementation would compute."""
|
||||
pos = 0
|
||||
|
||||
def need(n):
|
||||
nonlocal pos
|
||||
if n > len(data) - pos:
|
||||
raise Refused("truncated")
|
||||
out = data[pos:pos + n]
|
||||
pos += n
|
||||
return out
|
||||
|
||||
def item(depth):
|
||||
nonlocal pos
|
||||
if depth > 128:
|
||||
raise Refused("too_deep")
|
||||
ib = need(1)[0]
|
||||
major, ai = ib >> 5, ib & 31
|
||||
if major == 7:
|
||||
if ai == 20:
|
||||
return False
|
||||
if ai == 21:
|
||||
return True
|
||||
if ai == 22:
|
||||
return None
|
||||
if ai in (25, 26, 27):
|
||||
raise Refused("float_forbidden")
|
||||
if ai in (28, 29, 30, 31):
|
||||
raise Refused("malformed")
|
||||
if ai == 24 and need(1)[0] < 32:
|
||||
raise Refused("malformed") # RFC 8949 §3.3: not well-formed
|
||||
raise Refused("simple_value_forbidden")
|
||||
if major == 6:
|
||||
raise Refused("tag_forbidden")
|
||||
if ai in (28, 29, 30):
|
||||
raise Refused("malformed")
|
||||
if ai == 31:
|
||||
raise Refused("malformed" if major <= 1 else "indefinite_length")
|
||||
if ai < 24:
|
||||
n = ai
|
||||
else:
|
||||
width = {24: 1, 25: 2, 26: 4, 27: 8}[ai]
|
||||
n = int.from_bytes(need(width), "big")
|
||||
if strict and n < (24 if width == 1 else 1 << (8 * width // 2)):
|
||||
raise Refused("non_shortest_integer" if major <= 1 else "non_shortest_length")
|
||||
if major == 0:
|
||||
return n
|
||||
if major == 1:
|
||||
return -1 - n
|
||||
if major in (2, 3):
|
||||
raw = need(n)
|
||||
if major == 2:
|
||||
return raw
|
||||
try:
|
||||
return raw.decode("utf-8")
|
||||
except UnicodeDecodeError:
|
||||
raise Refused("invalid_utf8") from None
|
||||
if major == 4:
|
||||
return [item(depth + 1) for _ in range(n)]
|
||||
out, prev = Map(), None
|
||||
for _ in range(n):
|
||||
start = pos
|
||||
k = item(depth + 1)
|
||||
kb = data[start:pos]
|
||||
if strict and prev is not None:
|
||||
if kb == prev:
|
||||
raise Refused("duplicate_map_key")
|
||||
if kb < prev:
|
||||
raise Refused("unsorted_map_keys")
|
||||
prev = kb
|
||||
out.append((k, item(depth + 1)))
|
||||
return out
|
||||
|
||||
v = item(0)
|
||||
if pos != len(data):
|
||||
raise Refused("trailing_bytes")
|
||||
return v
|
||||
|
||||
|
||||
def from_json(j):
|
||||
"""The pack's typed JSON: {"$bytes": hex} is a byte string; `$bytes` is
|
||||
reserved and never a map member."""
|
||||
if isinstance(j, dict):
|
||||
if "$bytes" in j:
|
||||
h = j["$bytes"]
|
||||
if len(j) != 1 or not isinstance(h, str):
|
||||
raise ValueError("`$bytes` is reserved: a lone member holding hex")
|
||||
if len(h) % 2 or any(c not in "0123456789abcdef" for c in h):
|
||||
raise ValueError(f"$bytes must be lowercase hex: {h!r}")
|
||||
return bytes.fromhex(h)
|
||||
return Map((k, from_json(v)) for k, v in j.items())
|
||||
if isinstance(j, list):
|
||||
return [from_json(x) for x in j]
|
||||
if isinstance(j, float):
|
||||
raise ValueError("floats are forbidden")
|
||||
return j
|
||||
|
||||
|
||||
def load_json(path):
|
||||
"""Parse JSON, refusing any object that names a member twice."""
|
||||
def no_duplicates(pairs):
|
||||
out = {}
|
||||
for k, v in pairs:
|
||||
if k in out:
|
||||
raise ValueError(f"duplicate member {k!r}")
|
||||
out[k] = v
|
||||
return out
|
||||
return json.loads(path.read_text(encoding="utf-8"), object_pairs_hook=no_duplicates)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SAIDs (identity spec §2.1, §2.2) and receiving them (§2.6)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
PLACEHOLDER = "#" * 53
|
||||
|
||||
|
||||
def said_text(digest):
|
||||
return "E" + base64.b32encode(digest).decode().rstrip("=").lower()
|
||||
|
||||
|
||||
def parse_said(text):
|
||||
if not (isinstance(text, str) and len(text) == 53 and text[0] == "E"
|
||||
and all(c in "abcdefghijklmnopqrstuvwxyz234567" for c in text[1:])):
|
||||
return None
|
||||
digest = base64.b32decode(text[1:].upper() + "====")
|
||||
return digest if said_text(digest) == text else None
|
||||
|
||||
|
||||
def _with(obj, field, value):
|
||||
return Map((k, value if k == field else v) for k, v in obj)
|
||||
|
||||
|
||||
def derive(obj, field):
|
||||
"""(hashed bytes, digest, stamped object bytes)."""
|
||||
if not isinstance(obj, Map):
|
||||
raise Refused("not_a_map")
|
||||
if not any(k == field for k, _ in obj):
|
||||
raise Refused("missing_said_field")
|
||||
hashed = encode(_with(obj, field, PLACEHOLDER))
|
||||
digest = blake3(hashed)
|
||||
return hashed, digest, encode(_with(obj, field, said_text(digest)))
|
||||
|
||||
|
||||
def verify_said(data, field):
|
||||
"""The digest the bytes state and hash to, or Refused(class)."""
|
||||
obj = decode(data)
|
||||
if not isinstance(obj, Map):
|
||||
raise Refused("not_a_map")
|
||||
stated = [v for k, v in obj if k == field]
|
||||
if not stated:
|
||||
raise Refused("missing_said_field")
|
||||
digest = parse_said(stated[0])
|
||||
if digest is None:
|
||||
raise Refused("malformed_said")
|
||||
if derive(obj, field)[1] != digest:
|
||||
raise Refused("said_mismatch")
|
||||
return digest
|
||||
|
||||
|
||||
def receive(held, claimed, data):
|
||||
"""Identity spec §2.6, in order: canonical bytes, SAID match, the claimed
|
||||
SAID, and only then deduplication. Returns the result object."""
|
||||
try:
|
||||
digest = verify_said(data, "d")
|
||||
claim = parse_said(claimed)
|
||||
if claim is None:
|
||||
raise Refused("malformed_said")
|
||||
if claim != digest:
|
||||
raise Refused("said_mismatch")
|
||||
except Refused as e:
|
||||
return {"outcome": "rejected", "error": e.cls}
|
||||
text = said_text(digest)
|
||||
if any(h["said"] == text for h in held):
|
||||
return {"outcome": "duplicate", "said": text}
|
||||
held.append({"said": text, "cbor_hex": data.hex()})
|
||||
return {"outcome": "accepted", "said": text}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The checks. Each raises Mismatch; none uses `assert`, so `python -O`
|
||||
# cannot switch them off.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class Mismatch(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def check(condition, message):
|
||||
if not condition:
|
||||
raise Mismatch(message)
|
||||
|
||||
|
||||
def refused_class(fn, *args):
|
||||
try:
|
||||
fn(*args)
|
||||
except Refused as e:
|
||||
return e.cls
|
||||
return None
|
||||
|
||||
|
||||
def check_canonicalisation(v):
|
||||
if "input" in v:
|
||||
b = encode(from_json(v["input"]))
|
||||
check(b.hex() == v["expected_cbor_hex"], f"expected_cbor_hex: computed {b.hex()}")
|
||||
check(encode(decode(b)) == b, "decode then encode changed the bytes")
|
||||
else:
|
||||
got = refused_class(decode, bytes.fromhex(v["input_cbor_hex"]))
|
||||
check(got == v["error"], f"refused with {got}, vector says {v['error']}")
|
||||
|
||||
|
||||
def check_said(v):
|
||||
field = v["said_field"]
|
||||
if "input" in v:
|
||||
obj = from_json(v["input"])
|
||||
check([x for k, x in obj if k == field] == [PLACEHOLDER], "input must hold the placeholder")
|
||||
hashed, digest, stamped = derive(obj, field)
|
||||
check(hashed.hex() == v["canonical_cbor_hex"], "canonical_cbor_hex")
|
||||
check(digest.hex() == v["blake3_hex"], "blake3_hex")
|
||||
check(said_text(digest) == v["expected_said"], f"expected_said: computed {said_text(digest)}")
|
||||
check(stamped.hex() == v["said_cbor_hex"], "said_cbor_hex")
|
||||
check(verify_said(stamped, field) == digest, "said_cbor_hex does not verify")
|
||||
else:
|
||||
data = bytes.fromhex(v["input_cbor_hex"])
|
||||
got = refused_class(verify_said, data, field)
|
||||
check(got == v["error"], f"refused with {got}, vector says {v['error']}")
|
||||
if "said_of_received_bytes" in v or "said_after_reencoding" in v:
|
||||
lenient = decode(data, strict=False)
|
||||
stated = [x for k, x in lenient if k == field]
|
||||
check(len(stated) == 1 and isinstance(stated[0], str) and len(stated[0]) == 53,
|
||||
"no 53-character SAID to substitute")
|
||||
if "said_of_received_bytes" in v:
|
||||
at = data.find(stated[0].encode())
|
||||
check(at >= 0, "the stated SAID is not in the bytes")
|
||||
raw = data[:at] + PLACEHOLDER.encode() + data[at + 53:]
|
||||
check(said_text(blake3(raw)) == v["said_of_received_bytes"], "said_of_received_bytes")
|
||||
if "said_after_reencoding" in v:
|
||||
check(said_text(derive(lenient, field)[1]) == v["said_after_reencoding"],
|
||||
"said_after_reencoding")
|
||||
|
||||
|
||||
def check_signature(v, saids):
|
||||
check(v["envelope_said"] in saids, "envelope_said is not a SAID vector's expected_said")
|
||||
digest = parse_said(v["envelope_said"])
|
||||
check(digest is not None and digest.hex() == v["signed_message_hex"], "signed_message_hex")
|
||||
seed = v["signing_key_seed"]
|
||||
if seed is not None:
|
||||
check(ed25519_public(bytes.fromhex(seed)).hex() == v["verifying_key"],
|
||||
"verifying_key is not the key the seed derives")
|
||||
sig = bytes.fromhex(v["signature"])
|
||||
got = ed25519_verify(bytes.fromhex(v["verifying_key"]), digest, sig)
|
||||
if v["expected_verify"] is True:
|
||||
check(got is None and v["error"] is None, f"a positive vector fails: {got}")
|
||||
check(seed is not None, "a positive vector must carry its seed")
|
||||
check(ed25519_sign(bytes.fromhex(seed), digest) == sig, "the deterministic signature differs")
|
||||
else:
|
||||
check(v["expected_verify"] is False, "expected_verify is not a boolean")
|
||||
check(got == v["error"], f"refused with {got}, vector says {v['error']}")
|
||||
|
||||
|
||||
def tally(n, reports):
|
||||
"""(supermajority hash or None, sorted dissenters) — treatise §13.1."""
|
||||
threshold = 2 * n // 3 + 1
|
||||
votes = {}
|
||||
for h in reports.values():
|
||||
votes[h] = votes.get(h, 0) + 1
|
||||
winners = [h for h, c in votes.items() if c >= threshold]
|
||||
winner = winners[0] if winners else None
|
||||
dissenters = sorted(node for node, h in reports.items() if winner and h != winner)
|
||||
return winner, dissenters
|
||||
|
||||
|
||||
def check_lockstep(v):
|
||||
n, frame = v["node_count"], v["frame_number"]
|
||||
reported, computed = v["node_reported_hashes"], v["node_computed_hashes"]
|
||||
check(sorted(reported) == sorted(computed), "reported and computed name different nodes")
|
||||
check(0 < len(reported) <= n, "more reports than nodes")
|
||||
check(v["expected_threshold"] == 2 * n // 3 + 1, "expected_threshold")
|
||||
winner, _ = tally(n, reported)
|
||||
check(v["expected_commit"] == (winner is not None), "expected_commit")
|
||||
check(v["expected_state_hash"] == winner, "expected_state_hash")
|
||||
outcomes, logs = {}, {}
|
||||
for node, own in computed.items():
|
||||
# A node's view: its peers' reports and the hash it computed (§13.4).
|
||||
view = dict(reported)
|
||||
view[node] = own
|
||||
best, dissent = tally(n, view)
|
||||
if best is None:
|
||||
outcomes[node] = "cluster_split"
|
||||
elif own != best:
|
||||
outcomes[node] = "minority_divergence"
|
||||
else:
|
||||
outcomes[node] = "commit_with_dissent" if dissent else "clean_commit"
|
||||
logs[node] = [{"frame": frame, "dissenting_peers": dissent,
|
||||
"local_hash": own, "supermajority_hash": best}] \
|
||||
if outcomes[node] == "commit_with_dissent" else []
|
||||
check(v["expected_outcomes"] == outcomes, f"expected_outcomes: computed {outcomes}")
|
||||
check(v["expected_dissent_logs"] == logs, f"expected_dissent_logs: computed {logs}")
|
||||
|
||||
|
||||
def check_gossip(v):
|
||||
check(len(v["deliveries"]) == len(v["expected_results"]), "one result per delivery")
|
||||
held = []
|
||||
for d, want in zip(v["deliveries"], v["expected_results"]):
|
||||
got = receive(held, d["claimed_said"], bytes.fromhex(d["cbor_hex"]))
|
||||
check(got == want, f"published {want}, computed {got}")
|
||||
check(v["expected_held"] == held, f"expected_held: computed {held}")
|
||||
|
||||
|
||||
def self_test():
|
||||
# BLAKE3 known answers: the empty input, and the official test-vector input
|
||||
# pattern (byte i is i % 251) at lengths that cross block and chunk edges.
|
||||
check(blake3(b"").hex() == "af1349b9f5f9a1a6a0404dea36dcc9499bcb25c9adc112b7cc9a93cae41f3262",
|
||||
"BLAKE3 of the empty input")
|
||||
for n, want in BLAKE3_KAT:
|
||||
got = blake3(bytes(i % 251 for i in range(n))).hex()
|
||||
check(got == want, f"BLAKE3 known answer for length {n}: {got}")
|
||||
# Ed25519 known answers: RFC 8032 §7.1 tests 1 and 2.
|
||||
s1 = bytes.fromhex("9d61b19deffd5a60ba844af492ec2cc44449c5697b326919703bac031cae7f60")
|
||||
check(ed25519_public(s1).hex() == "d75a980182b10ab7d54bfed3c964073a0ee172f3daa62325af021a68f707511a",
|
||||
"RFC 8032 test 1 key")
|
||||
check(ed25519_sign(s1, b"").hex() == (
|
||||
"e5564300c360ac729086e2cc806e828a84877f1eb8e5d974d873e065224901555"
|
||||
"fb8821590a33bacc61e39701cf9b46bd25bf5f0595bbe24655141438e7a100b"), "RFC 8032 test 1 signature")
|
||||
s2 = bytes.fromhex("4ccd089b28ff96da9db6c346ec114e0f5b8a319f35aba624da8cf6ed4fb8a6fb")
|
||||
sig2 = ed25519_sign(s2, b"\x72")
|
||||
check(sig2.hex() == (
|
||||
"92a009a9f0d4cab8720e820b5f642540a2b27b5416503f8fb3762223ebdb69da"
|
||||
"085ac1e43e15996e458f3613d0f11d8c387b2eaeb4302aeeb00d291612bb0c00"), "RFC 8032 test 2 signature")
|
||||
check(ed25519_verify(ed25519_public(s2), b"\x72", sig2) is None, "RFC 8032 test 2 verifies")
|
||||
check(ed25519_verify(ed25519_public(s2), b"\x73", sig2) == "signature_invalid",
|
||||
"a changed message fails")
|
||||
|
||||
|
||||
# BLAKE3 hashes of the official test-vector input pattern (byte i is i % 251),
|
||||
# hash mode, 32 bytes, at lengths that cross block, chunk and tree edges.
|
||||
# Computed with the BLAKE3 team's reference implementation (Rust crate
|
||||
# `blake3` 1.8.7).
|
||||
BLAKE3_KAT = [
|
||||
(1, "2d3adedff11b61f14c886e35afa036736dcd87a74d27b5c1510225d0f592e213"),
|
||||
(63, "e9bc37a594daad83be9470df7f7b3798297c3d834ce80ba85d6e207627b7db7b"),
|
||||
(64, "4eed7141ea4a5cd4b788606bd23f46e212af9cacebacdc7d1f4c6dc7f2511b98"),
|
||||
(65, "de1e5fa0be70df6d2be8fffd0e99ceaa8eb6e8c93a63f2d8d1c30ecb6b263dee"),
|
||||
(1023, "10108970eeda3eb932baac1428c7a2163b0e924c9a9e25b35bba72b28f70bd11"),
|
||||
(1024, "42214739f095a406f3fc83deb889744ac00df831c10daa55189b5d121c855af7"),
|
||||
(1025, "d00278ae47eb27b34faecf67b4fe263f82d5412916c1ffd97c8cb7fb814b8444"),
|
||||
(2048, "e776b6028c7cd22a4d0ba182a8bf62205d2ef576467e838ed6f2529b85fba24a"),
|
||||
(2049, "5f4d72f40d7a5f82b15ca2b2e44b1de3c2ef86c426c95c1af0b6879522563030"),
|
||||
(3072, "b98cb0ff3623be03326b373de6b9095218513e64f1ee2edd2525c7ad1e5cffd2"),
|
||||
(3073, "7124b49501012f81cc7f11ca069ec9226cecb8a2c850cfe644e327d22d3e1cd3"),
|
||||
(4096, "015094013f57a5277b59d8475c0501042c0b642e531b0a1c8f58d2163229e969"),
|
||||
(4097, "9b4052b38f1c5fc8b1f9ff7ac7b27cd242487b3d890d15c96a1c25b8aa0fb995"),
|
||||
(5120, "9cadc15fed8b5d854562b26a9536d9707cadeda9b143978f319ab34230535833"),
|
||||
(8193, "bab6c09cb8ce8cf459261398d2e7aef35700bf488116ceb94a36d0f5f1b7bc3b"),
|
||||
(31744, "62b6960e1a44bcc1eb1a611a8d6235b6b4b78f32e7abc4fb4c6cdcce94895c47"),
|
||||
]
|
||||
|
||||
CHECKS = {
|
||||
"canonicalisation_vectors.json": check_canonicalisation,
|
||||
"said_vectors.json": check_said,
|
||||
"signature_vectors.json": None, # needs the SAID set; dispatched in main
|
||||
"lockstep_vectors.json": check_lockstep,
|
||||
"gossip_vectors.json": check_gossip,
|
||||
}
|
||||
|
||||
HERE = pathlib.Path(__file__).resolve().parent
|
||||
|
||||
|
||||
def spec_cited_said_vectors():
|
||||
"""The SAID vector prefixes the identity spec cites, ranges expanded."""
|
||||
spec = (HERE.parent.parent / "spec" / "identity_and_key_rotation.md").read_text(encoding="utf-8")
|
||||
words = [w for w in re.split(r"[^A-Za-z0-9-]+", spec) if w]
|
||||
cited = set()
|
||||
for i, w in enumerate(words):
|
||||
m = re.fullmatch(r"said-(\d+)", w)
|
||||
if not m:
|
||||
continue
|
||||
cited.add(f"said-{int(m.group(1)):03d}")
|
||||
end = re.fullmatch(r"said-(\d+)", words[i + 2]) if i + 2 < len(words) and words[i + 1] == "to" else None
|
||||
if end:
|
||||
cited.update(f"said-{k:03d}" for k in range(int(m.group(1)), int(end.group(1)) + 1))
|
||||
return cited
|
||||
|
||||
|
||||
def readme_error_classes():
|
||||
readme = (HERE.parent / "README.md").read_text(encoding="utf-8")
|
||||
section = readme.split("### Error classes", 1)[1]
|
||||
rows = []
|
||||
for line in section.splitlines():
|
||||
if line.startswith("|"):
|
||||
rows.append(line)
|
||||
elif rows:
|
||||
break
|
||||
return {line.split("`")[1] for line in rows if line.count("`") >= 2}
|
||||
|
||||
|
||||
def check_complete(files):
|
||||
"""Every file non-empty; every vector the spec cites present; every README
|
||||
error class exercised, and no other class used."""
|
||||
for name, vectors in files.items():
|
||||
check(len(vectors) > 0, f"{name} holds no vectors")
|
||||
names = [v["name"] for v in files["said_vectors.json"]]
|
||||
missing = sorted(p for p in spec_cited_said_vectors()
|
||||
if not any(n.startswith(p + "-") for n in names))
|
||||
check(not missing, f"the identity spec cites vectors the pack lacks: {missing}")
|
||||
used = set()
|
||||
for vectors in files.values():
|
||||
for v in vectors:
|
||||
if isinstance(v.get("error"), str):
|
||||
used.add(v["error"])
|
||||
for r in v.get("expected_results", []):
|
||||
if "error" in r:
|
||||
used.add(r["error"])
|
||||
listed = readme_error_classes()
|
||||
check(listed and used == listed,
|
||||
f"README classes unexercised: {sorted(listed - used)}; unlisted classes used: {sorted(used - listed)}")
|
||||
|
||||
|
||||
def main(pack):
|
||||
self_test()
|
||||
failed = total = 0
|
||||
files = {}
|
||||
for name in CHECKS:
|
||||
try:
|
||||
data = load_json(pack / name)
|
||||
check(isinstance(data, list), "the top-level value must be an array")
|
||||
names = [v["name"] for v in data]
|
||||
check(len(set(names)) == len(names), "duplicate vector names")
|
||||
check(all(isinstance(v.get("description"), str) for v in data), "a vector lacks a description")
|
||||
files[name] = data
|
||||
except (Mismatch, ValueError, KeyError, OSError) as e:
|
||||
failed += 1
|
||||
print(f"FAIL {name}: {e}")
|
||||
saids = {v["expected_said"] for v in files.get("said_vectors.json", []) if "expected_said" in v}
|
||||
for name, vectors in files.items():
|
||||
for v in vectors:
|
||||
total += 1
|
||||
try:
|
||||
if name == "signature_vectors.json":
|
||||
check_signature(v, saids)
|
||||
else:
|
||||
CHECKS[name](v)
|
||||
print(f"ok {name:<32} {v['name']}")
|
||||
except (Mismatch, Refused, KeyError, ValueError, TypeError) as e:
|
||||
failed += 1
|
||||
print(f"FAIL {name:<32} {v.get('name')}: {e!r}")
|
||||
if len(files) == len(CHECKS):
|
||||
try:
|
||||
check_complete(files)
|
||||
print("ok (pack) complete")
|
||||
except Mismatch as e:
|
||||
failed += 1
|
||||
print(f"FAIL (pack): {e}")
|
||||
print(f"{total} vectors, {failed} failed")
|
||||
return 1 if failed else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
default = HERE.parent / "v1"
|
||||
sys.exit(main(pathlib.Path(sys.argv[1]) if len(sys.argv) > 1 else default))
|
||||
|
|
@ -1,17 +1,17 @@
|
|||
[
|
||||
{
|
||||
"name": "canon-001-sorted-map-keys",
|
||||
"description": "Map keys must be sorted lexicographically in canonical CBOR. The input is given with keys out of order; the canonical encoding sorts them.",
|
||||
"description": "Map keys are written in the bytewise order of their canonical encodings (identity spec §2.3). The input lists the keys out of order; the canonical encoding sorts them.",
|
||||
"input": {
|
||||
"z": 1,
|
||||
"a": 2,
|
||||
"m": 3
|
||||
},
|
||||
"expected_cbor_hex": "a361610261186d036179016101"
|
||||
"expected_cbor_hex": "a3616102616d03617a01"
|
||||
},
|
||||
{
|
||||
"name": "canon-002-shortest-form-integers",
|
||||
"description": "Integers must be encoded in their shortest form. No leading zeros, no over-large width selection. 0, 23, 24, 255, 256, 65535, 65536 cover the boundaries of CBOR's minor-info encoding.",
|
||||
"description": "Integers use the shortest head that holds them: no leading zeros, no over-wide argument. 0, 23, 24, 255, 256, 65535 and 65536 sit on the boundaries of CBOR's argument widths.",
|
||||
"input": {
|
||||
"i0": 0,
|
||||
"i23": 23,
|
||||
|
|
@ -21,34 +21,154 @@
|
|||
"i65535": 65535,
|
||||
"i65536": 65536
|
||||
},
|
||||
"expected_cbor_hex": "a76269300062693017636932341818636932351818ff63693235361901006935363533361903e8676935363533362ffffeeeebbbbcccc"
|
||||
"expected_cbor_hex": "a7626930006369323317636932341818646932353518ff64693235361901006669363535333519ffff666936353533361a00010000"
|
||||
},
|
||||
{
|
||||
"name": "canon-003-no-tags",
|
||||
"description": "Canonical Smart Byte CBOR never uses CBOR tags. Application semantics carried in cargo fields are interpreted by layer-2 protocols; the substrate's wire format treats all values as untagged primitives. The input here is a plain byte string that an implementer might be tempted to wrap in tag 21 (expected base64url encoding) — the canonical form drops the tag.",
|
||||
"description": "A byte string is written untagged, as major type 2. Wrapping bytes in tag 21 (\"expected base64url conversion\") is tempting, but canonical encoding forbids tags (identity spec §2.3). canon-015 is the tagged form, which a decoder refuses rather than silently stripping the tag.",
|
||||
"input": {
|
||||
"payload": "AAECAwQFBgc="
|
||||
"payload": {
|
||||
"$bytes": "0001020304050607"
|
||||
}
|
||||
},
|
||||
"expected_cbor_hex": "a16770617964c6c6164480001020304050607"
|
||||
"expected_cbor_hex": "a1677061796c6f6164480001020304050607"
|
||||
},
|
||||
{
|
||||
"name": "canon-004-deterministic-float-encoding",
|
||||
"description": "Floating-point values are encoded in the shortest representation that preserves the value exactly. The spec recommends avoiding floats in cargo where possible (use fixed-point integers), but where they appear, the canonical encoding selects f16/f32/f64 by the round-trip test.",
|
||||
"input": {
|
||||
"exact_half": 1.5,
|
||||
"needs_f32": 3.4028234663852886e38,
|
||||
"needs_f64": 1.7976931348623157e308
|
||||
},
|
||||
"expected_cbor_hex": "a36a657861637421682dbcb1a96f3df9006a6e656564735f66333367fa7f7fffff6a6e656564735f6636341b7fefffffffffffff"
|
||||
"name": "canon-004-floats-forbidden",
|
||||
"description": "Floats are forbidden (identity spec §2.3): a value that would be a float elsewhere is carried as a fixed-point integer pair. The input is a map holding 1.5 as a half-precision float, and a decoder refuses it.",
|
||||
"input_cbor_hex": "a16a65786163745f68616c66f93e00",
|
||||
"error": "float_forbidden"
|
||||
},
|
||||
{
|
||||
"name": "canon-005-bytes-not-text-for-byte-arrays",
|
||||
"description": "Byte arrays MUST be encoded as CBOR major-type 2 (byte string), never as major-type 3 (text string). This matters for SAIDs, pubkeys, signatures, and any raw cryptographic material. The input shows three fields that look textual on the wire but are bytes by the spec: a SAID, a pubkey, a signature placeholder.",
|
||||
"description": "Byte values are byte strings (major type 2) — never text, and never an array of small integers, which is what a generic serializer writes for a fixed-size byte array by default. The members hold a 32-byte digest, a 32-byte public key and a 64-byte signature. A SAID's text form (identity spec §2.2) is text; the digest it spells is bytes.",
|
||||
"input": {
|
||||
"said": "0000000000000000000000000000000000000000000000000000000000000000",
|
||||
"key": "1111111111111111111111111111111111111111111111111111111111111111",
|
||||
"sig": "22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222"
|
||||
"said": {
|
||||
"$bytes": "0000000000000000000000000000000000000000000000000000000000000000"
|
||||
},
|
||||
"key": {
|
||||
"$bytes": "1111111111111111111111111111111111111111111111111111111111111111"
|
||||
},
|
||||
"sig": {
|
||||
"$bytes": "22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222"
|
||||
}
|
||||
},
|
||||
"expected_cbor_hex": "a3637361696458200000000000000000000000000000000000000000000000000000000000000000636b657958201111111111111111111111111111111111111111111111111111111111111111637369675840222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222"
|
||||
"expected_cbor_hex": "a3636b65795820111111111111111111111111111111111111111111111111111111111111111163736967584022222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222647361696458200000000000000000000000000000000000000000000000000000000000000000"
|
||||
},
|
||||
{
|
||||
"name": "canon-006-key-order-follows-encoded-bytes",
|
||||
"description": "Keys sort by their encoded bytes, not their text (identity spec §2.3). As text \"aa\" comes before \"b\"; encoded, \"b\" (0x61 0x62) comes before \"aa\" (0x62 0x61 0x61), because a string's length is in its first byte. canon-012 is the text-ordered encoding, which a decoder refuses.",
|
||||
"input": {
|
||||
"aa": 1,
|
||||
"b": 2
|
||||
},
|
||||
"expected_cbor_hex": "a261620262616101"
|
||||
},
|
||||
{
|
||||
"name": "canon-007-integer-boundaries-and-negatives",
|
||||
"description": "Negative integers (major type 1) and the 32-bit/64-bit boundary. -1 is 0x20, and -24 is the last one-byte form; 4294967295 is the largest four-byte argument and 4294967296 the smallest eight-byte one.",
|
||||
"input": [
|
||||
-1,
|
||||
-24,
|
||||
-25,
|
||||
-256,
|
||||
-257,
|
||||
-65536,
|
||||
-65537,
|
||||
4294967295,
|
||||
4294967296
|
||||
],
|
||||
"expected_cbor_hex": "892037381838ff39010039ffff3a000100001affffffff1b0000000100000000"
|
||||
},
|
||||
{
|
||||
"name": "canon-008-string-length-boundary",
|
||||
"description": "A length uses the shortest head too. A 23-character text string carries its length in its first byte; a 24-character one needs the one-byte argument.",
|
||||
"input": [
|
||||
"aaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"bbbbbbbbbbbbbbbbbbbbbbbb"
|
||||
],
|
||||
"expected_cbor_hex": "827761616161616161616161616161616161616161616161617818626262626262626262626262626262626262626262626262"
|
||||
},
|
||||
{
|
||||
"name": "canon-009-empty-containers-and-simple-values",
|
||||
"description": "An empty array, map, text string and byte string, and the simple values null, true and false — each a single byte.",
|
||||
"input": {
|
||||
"a": [],
|
||||
"b": {},
|
||||
"c": "",
|
||||
"d": {
|
||||
"$bytes": ""
|
||||
},
|
||||
"e": null,
|
||||
"f": true,
|
||||
"g": false
|
||||
},
|
||||
"expected_cbor_hex": "a76161806162a06163606164406165f66166f56167f4"
|
||||
},
|
||||
{
|
||||
"name": "canon-010-non-shortest-integer-refused",
|
||||
"description": "Zero written with a one-byte argument (0x18 0x00). Identity spec §2.3: \"a value of 0 is encoded as 0x00, never as 0x18 0x00\". A decoder refuses it rather than normalising it.",
|
||||
"input_cbor_hex": "1800",
|
||||
"error": "non_shortest_integer"
|
||||
},
|
||||
{
|
||||
"name": "canon-011-non-shortest-length-refused",
|
||||
"description": "The one-character text \"a\" with its length in a one-byte argument (0x78 0x01) rather than in the head. A length has one canonical form too.",
|
||||
"input_cbor_hex": "780161",
|
||||
"error": "non_shortest_length"
|
||||
},
|
||||
{
|
||||
"name": "canon-012-text-ordered-keys-refused",
|
||||
"description": "canon-006's map with its keys in text order (\"aa\" before \"b\"). The keys are distinct and the map is well-formed, but the order is not the canonical one, so the encoding is refused.",
|
||||
"input_cbor_hex": "a262616101616202",
|
||||
"error": "unsorted_map_keys"
|
||||
},
|
||||
{
|
||||
"name": "canon-013-duplicate-key-refused",
|
||||
"description": "A map holding the key \"a\" twice. It has no canonical form, and which of the two values a lenient decoder keeps differs between implementations.",
|
||||
"input_cbor_hex": "a2616101616102",
|
||||
"error": "duplicate_map_key"
|
||||
},
|
||||
{
|
||||
"name": "canon-014-indefinite-length-refused",
|
||||
"description": "An array written with an indefinite length (0x9f … 0xff). Identity spec §2.3 forbids indefinite-length items: the same array has a definite-length encoding.",
|
||||
"input_cbor_hex": "9f0102ff",
|
||||
"error": "indefinite_length"
|
||||
},
|
||||
{
|
||||
"name": "canon-015-tagged-byte-string-refused",
|
||||
"description": "canon-003's map with its byte string wrapped in tag 21. Tags are forbidden (identity spec §2.3): the decoder refuses the input; it does not strip the tag and accept what is left.",
|
||||
"input_cbor_hex": "a1677061796c6f6164d5480001020304050607",
|
||||
"error": "tag_forbidden"
|
||||
},
|
||||
{
|
||||
"name": "canon-016-single-precision-float-refused",
|
||||
"description": "A single-precision float (the largest finite one). Refused, as every float is.",
|
||||
"input_cbor_hex": "fa7f7fffff",
|
||||
"error": "float_forbidden"
|
||||
},
|
||||
{
|
||||
"name": "canon-017-double-precision-float-refused",
|
||||
"description": "A double-precision float (the largest finite one). Refused, as every float is.",
|
||||
"input_cbor_hex": "fb7fefffffffffffff",
|
||||
"error": "float_forbidden"
|
||||
},
|
||||
{
|
||||
"name": "canon-018-trailing-bytes-refused",
|
||||
"description": "Two items back to back. A canonical encoding is exactly one item: bytes after it would sit outside the value a SAID commits to, so the input is refused.",
|
||||
"input_cbor_hex": "0101",
|
||||
"error": "trailing_bytes"
|
||||
},
|
||||
{
|
||||
"name": "canon-019-invalid-utf8-refused",
|
||||
"description": "A text string whose bytes (0xc3 0x28) are not UTF-8. CBOR text strings are UTF-8 (RFC 8949 §3.1), so the input is refused.",
|
||||
"input_cbor_hex": "62c328",
|
||||
"error": "invalid_utf8"
|
||||
},
|
||||
{
|
||||
"name": "canon-020-simple-value-refused",
|
||||
"description": "The simple value undefined (0xf7). Identity spec §2.3 admits false, true and null and no other simple value, so the data model has nothing to decode it to and the input is refused.",
|
||||
"input_cbor_hex": "f7",
|
||||
"error": "simple_value_forbidden"
|
||||
}
|
||||
]
|
||||
|
|
|
|||
|
|
@ -1,65 +1,177 @@
|
|||
[
|
||||
{
|
||||
"name": "gossip-001-cross-node-propagation",
|
||||
"description": "An envelope published at node A is observed at node B after a single round of gossip. The envelope's SAID is identical at both observation points; the gossip layer does not modify the wire bytes.",
|
||||
"scenario": {
|
||||
"publisher": "node_A",
|
||||
"publish_frame": 184300,
|
||||
"envelope_said": "EBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB",
|
||||
"envelope_canonical_cbor_hex": "a6677665727369",
|
||||
"gossip_rounds": 1,
|
||||
"observers": ["node_B"]
|
||||
},
|
||||
"expected_observations": [
|
||||
"description": "An object published at node_A reaches node_B. Its bytes are canonical, they hash to the SAID they state, and that is the SAID they were announced under, so node_B accepts it (identity spec §2.6) and holds exactly the bytes it received. Gossip carries the object itself, never a re-encoding of it.",
|
||||
"receiver": "node_B",
|
||||
"deliveries": [
|
||||
{
|
||||
"observer": "node_B",
|
||||
"observed_said": "EBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB",
|
||||
"observation_frame": 184301,
|
||||
"outcome": "accepted"
|
||||
"from": "node_A",
|
||||
"claimed_said": "Eze2x3zkvaaxvxu6hb75kidinrkmtczqu63b6qnzbg3g75e6hnwra",
|
||||
"cbor_hex": "a261647835457a653278337a6b7661617876787536686237356b6964696e726b6d74637a717536336236716e7a6267336737356536686e777261616b582101d75a980182b10ab7d54bfed3c964073a0ee172f3daa62325af021a68f707511a"
|
||||
}
|
||||
],
|
||||
"expected_results": [
|
||||
{
|
||||
"outcome": "accepted",
|
||||
"said": "Eze2x3zkvaaxvxu6hb75kidinrkmtczqu63b6qnzbg3g75e6hnwra"
|
||||
}
|
||||
],
|
||||
"expected_held": [
|
||||
{
|
||||
"said": "Eze2x3zkvaaxvxu6hb75kidinrkmtczqu63b6qnzbg3g75e6hnwra",
|
||||
"cbor_hex": "a261647835457a653278337a6b7661617876787536686237356b6964696e726b6d74637a717536336236716e7a6267336737356536686e777261616b582101d75a980182b10ab7d54bfed3c964073a0ee172f3daa62325af021a68f707511a"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "gossip-002-said-conflict-rejected",
|
||||
"description": "Two different envelopes are gossiped under the same SAID. The receiving node MUST reject the second envelope. The SAID is content-addressed (BLAKE3 of the canonical CBOR); two structurally-different envelopes producing the same SAID is either a collision (cryptographically negligible) or an adversarial substitution attempt. Either way, the receiver keeps the first observation and rejects the second.",
|
||||
"scenario": {
|
||||
"publisher_a": "node_A",
|
||||
"publisher_b": "node_B",
|
||||
"publish_frame": 184310,
|
||||
"envelope_a_said": "ECCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC",
|
||||
"envelope_a_canonical_cbor_hex": "a6677665727369aaaa",
|
||||
"envelope_b_said": "ECCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC",
|
||||
"envelope_b_canonical_cbor_hex": "a6677665727369bbbb",
|
||||
"receiver": "node_C"
|
||||
},
|
||||
"expected_outcome": {
|
||||
"accepted": "envelope_a",
|
||||
"rejected": "envelope_b",
|
||||
"rejection_reason": "said_collision_envelope_a_already_observed",
|
||||
"alert_class": "potential_substitution_attack"
|
||||
}
|
||||
"description": "node_C holds said-008's object when a second object arrives under the same SAID: said-008's with n changed and its SAID left as it was (said-011's bytes). Verification comes before deduplication (identity spec §2.6), so the newcomer is reported as a SAID mismatch, which makes the substitution attempt visible, instead of being absorbed as a duplicate of what node_C holds.",
|
||||
"receiver": "node_C",
|
||||
"deliveries": [
|
||||
{
|
||||
"from": "node_A",
|
||||
"claimed_said": "Ek2atam3xjn73papa7bkh5ltoypfci4umrl3k5hoxohi6ajmpnu4q",
|
||||
"cbor_hex": "a261647835456b326174616d33786a6e37337061706137626b68356c746f797066636934756d726c336b35686f786f686936616a6d706e753471616e01"
|
||||
},
|
||||
{
|
||||
"from": "node_B",
|
||||
"claimed_said": "Ek2atam3xjn73papa7bkh5ltoypfci4umrl3k5hoxohi6ajmpnu4q",
|
||||
"cbor_hex": "a261647835456b326174616d33786a6e37337061706137626b68356c746f797066636934756d726c336b35686f786f686936616a6d706e753471616e02"
|
||||
}
|
||||
],
|
||||
"expected_results": [
|
||||
{
|
||||
"outcome": "accepted",
|
||||
"said": "Ek2atam3xjn73papa7bkh5ltoypfci4umrl3k5hoxohi6ajmpnu4q"
|
||||
},
|
||||
{
|
||||
"outcome": "rejected",
|
||||
"error": "said_mismatch"
|
||||
}
|
||||
],
|
||||
"expected_held": [
|
||||
{
|
||||
"said": "Ek2atam3xjn73papa7bkh5ltoypfci4umrl3k5hoxohi6ajmpnu4q",
|
||||
"cbor_hex": "a261647835456b326174616d33786a6e37337061706137626b68356c746f797066636934756d726c336b35686f786f686936616a6d706e753471616e01"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "gossip-003-sequence-gap-detection",
|
||||
"description": "An envelope arrives at node B referencing a previous-history-hash that node B has not seen. Node B MUST detect the gap, request the missing history via gossip backfill, and not commit the envelope's transition until the gap is closed. The expected behaviour is *not* to reject the envelope — it is to defer acceptance pending backfill.",
|
||||
"scenario": {
|
||||
"publisher": "node_A",
|
||||
"publish_frame": 184320,
|
||||
"envelope_said": "EDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD",
|
||||
"envelope_previous_history_hash": "EFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF",
|
||||
"receiver": "node_B",
|
||||
"receiver_known_history_heads": [
|
||||
"EAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
|
||||
"EBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB"
|
||||
]
|
||||
},
|
||||
"expected_outcome": {
|
||||
"observer": "node_B",
|
||||
"outcome": "deferred_pending_backfill",
|
||||
"missing_history_hash": "EFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF",
|
||||
"backfill_request_targets": ["node_A"],
|
||||
"backfill_max_frames": 256,
|
||||
"alert_class": "history_gap_detected"
|
||||
}
|
||||
"name": "gossip-003-backfill-answer-verified",
|
||||
"description": "node_B requests the object behind said-007's SAID: a party holding an identifier can request the object from any peer (treatise §21.4). node_A answers with said-008's object, which is genuine and self-consistent but not the one requested, and is refused. node_D's answer hashes to the requested SAID and is accepted (identity spec §2.6).",
|
||||
"receiver": "node_B",
|
||||
"deliveries": [
|
||||
{
|
||||
"from": "node_A",
|
||||
"claimed_said": "Eze2x3zkvaaxvxu6hb75kidinrkmtczqu63b6qnzbg3g75e6hnwra",
|
||||
"cbor_hex": "a261647835456b326174616d33786a6e37337061706137626b68356c746f797066636934756d726c336b35686f786f686936616a6d706e753471616e01"
|
||||
},
|
||||
{
|
||||
"from": "node_D",
|
||||
"claimed_said": "Eze2x3zkvaaxvxu6hb75kidinrkmtczqu63b6qnzbg3g75e6hnwra",
|
||||
"cbor_hex": "a261647835457a653278337a6b7661617876787536686237356b6964696e726b6d74637a717536336236716e7a6267336737356536686e777261616b582101d75a980182b10ab7d54bfed3c964073a0ee172f3daa62325af021a68f707511a"
|
||||
}
|
||||
],
|
||||
"expected_results": [
|
||||
{
|
||||
"outcome": "rejected",
|
||||
"error": "said_mismatch"
|
||||
},
|
||||
{
|
||||
"outcome": "accepted",
|
||||
"said": "Eze2x3zkvaaxvxu6hb75kidinrkmtczqu63b6qnzbg3g75e6hnwra"
|
||||
}
|
||||
],
|
||||
"expected_held": [
|
||||
{
|
||||
"said": "Eze2x3zkvaaxvxu6hb75kidinrkmtczqu63b6qnzbg3g75e6hnwra",
|
||||
"cbor_hex": "a261647835457a653278337a6b7661617876787536686237356b6964696e726b6d74637a717536336236716e7a6267336737356536686e777261616b582101d75a980182b10ab7d54bfed3c964073a0ee172f3daa62325af021a68f707511a"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "gossip-004-duplicate-delivery-deduplicated",
|
||||
"description": "The same object reaches node_B from two peers. The second delivery verifies and is a duplicate of a SAID already held, so it changes nothing: the overlay deduplicates by content address (treatise §11.4).",
|
||||
"receiver": "node_B",
|
||||
"deliveries": [
|
||||
{
|
||||
"from": "node_A",
|
||||
"claimed_said": "E4duyldlhqjm25aiznrd2dxsz6uxfhx6vyqj2itotg3hjahvqakiq",
|
||||
"cbor_hex": "a16164783545346475796c646c68716a6d323561697a6e7264326478737a367578666878367679716a3269746f746733686a61687671616b6971"
|
||||
},
|
||||
{
|
||||
"from": "node_C",
|
||||
"claimed_said": "E4duyldlhqjm25aiznrd2dxsz6uxfhx6vyqj2itotg3hjahvqakiq",
|
||||
"cbor_hex": "a16164783545346475796c646c68716a6d323561697a6e7264326478737a367578666878367679716a3269746f746733686a61687671616b6971"
|
||||
}
|
||||
],
|
||||
"expected_results": [
|
||||
{
|
||||
"outcome": "accepted",
|
||||
"said": "E4duyldlhqjm25aiznrd2dxsz6uxfhx6vyqj2itotg3hjahvqakiq"
|
||||
},
|
||||
{
|
||||
"outcome": "duplicate",
|
||||
"said": "E4duyldlhqjm25aiznrd2dxsz6uxfhx6vyqj2itotg3hjahvqakiq"
|
||||
}
|
||||
],
|
||||
"expected_held": [
|
||||
{
|
||||
"said": "E4duyldlhqjm25aiznrd2dxsz6uxfhx6vyqj2itotg3hjahvqakiq",
|
||||
"cbor_hex": "a16164783545346475796c646c68716a6d323561697a6e7264326478737a367578666878367679716a3269746f746733686a61687671616b6971"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "gossip-005-non-canonical-copy-rejected",
|
||||
"description": "said-009's bytes reach node_B announced under said-008's SAID, the SAID a re-encoding receiver would compute for them. The bytes are not canonical, so node_B refuses them (identity spec §2.5, §2.6). Accepting them would let a second byte string circulate under said-008's SAID.",
|
||||
"receiver": "node_B",
|
||||
"deliveries": [
|
||||
{
|
||||
"from": "node_A",
|
||||
"claimed_said": "Ek2atam3xjn73papa7bkh5ltoypfci4umrl3k5hoxohi6ajmpnu4q",
|
||||
"cbor_hex": "a261647835456b326174616d33786a6e37337061706137626b68356c746f797066636934756d726c336b35686f786f686936616a6d706e753471616e1801"
|
||||
}
|
||||
],
|
||||
"expected_results": [
|
||||
{
|
||||
"outcome": "rejected",
|
||||
"error": "non_shortest_integer"
|
||||
}
|
||||
],
|
||||
"expected_held": []
|
||||
},
|
||||
{
|
||||
"name": "gossip-006-substitute-arriving-first-rejected",
|
||||
"description": "gossip-002 in the other order: the altered object reaches node_C first and is refused, then said-008's object arrives and is accepted. Arrival order decides nothing; the content does.",
|
||||
"receiver": "node_C",
|
||||
"deliveries": [
|
||||
{
|
||||
"from": "node_B",
|
||||
"claimed_said": "Ek2atam3xjn73papa7bkh5ltoypfci4umrl3k5hoxohi6ajmpnu4q",
|
||||
"cbor_hex": "a261647835456b326174616d33786a6e37337061706137626b68356c746f797066636934756d726c336b35686f786f686936616a6d706e753471616e02"
|
||||
},
|
||||
{
|
||||
"from": "node_A",
|
||||
"claimed_said": "Ek2atam3xjn73papa7bkh5ltoypfci4umrl3k5hoxohi6ajmpnu4q",
|
||||
"cbor_hex": "a261647835456b326174616d33786a6e37337061706137626b68356c746f797066636934756d726c336b35686f786f686936616a6d706e753471616e01"
|
||||
}
|
||||
],
|
||||
"expected_results": [
|
||||
{
|
||||
"outcome": "rejected",
|
||||
"error": "said_mismatch"
|
||||
},
|
||||
{
|
||||
"outcome": "accepted",
|
||||
"said": "Ek2atam3xjn73papa7bkh5ltoypfci4umrl3k5hoxohi6ajmpnu4q"
|
||||
}
|
||||
],
|
||||
"expected_held": [
|
||||
{
|
||||
"said": "Ek2atam3xjn73papa7bkh5ltoypfci4umrl3k5hoxohi6ajmpnu4q",
|
||||
"cbor_hex": "a261647835456b326174616d33786a6e37337061706137626b68356c746f797066636934756d726c336b35686f786f686936616a6d706e753471616e01"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
|
|
|
|||
|
|
@ -1,109 +1,341 @@
|
|||
[
|
||||
{
|
||||
"name": "lockstep-001-unanimous-commit",
|
||||
"description": "All 4 nodes in the cluster agree on the post-frame state hash. The BFT threshold is floor(2n/3)+1 = 3, so 4-of-4 trivially commits. The expected_state_hash is the hash all four nodes reported.",
|
||||
"description": "All four nodes report the post-frame state hash they computed, and all agree. The threshold is floor(2·4/3)+1 = 3, so the frame commits, every node's outcome is a clean commit (treatise §13.3), and no dissent is logged. State hashes are opaque 32-byte values compared for equality. Each node's outcome comes from its own view: the hashes its peers reported plus the one it computed itself (§13.3, §13.4).",
|
||||
"node_count": 4,
|
||||
"frame_number": 184237,
|
||||
"ordered_transitions": [
|
||||
{
|
||||
"byte_said": "EBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB",
|
||||
"from_pubkey": "1111111111111111111111111111111111111111111111111111111111111111",
|
||||
"to_pubkey": "3333333333333333333333333333333333333333333333333333333333333333",
|
||||
"signature": "44444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444"
|
||||
},
|
||||
{
|
||||
"byte_said": "ECCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC",
|
||||
"from_pubkey": "3333333333333333333333333333333333333333333333333333333333333333",
|
||||
"to_pubkey": "5555555555555555555555555555555555555555555555555555555555555555",
|
||||
"signature": "66666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666"
|
||||
}
|
||||
],
|
||||
"node_hashes": {
|
||||
"node_1": "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
|
||||
"node_2": "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
|
||||
"node_3": "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
|
||||
"node_4": "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
|
||||
"node_reported_hashes": {
|
||||
"node_1": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"node_2": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"node_3": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"node_4": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
|
||||
},
|
||||
"node_computed_hashes": {
|
||||
"node_1": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"node_2": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"node_3": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"node_4": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
|
||||
},
|
||||
"expected_threshold": 3,
|
||||
"expected_commit": true,
|
||||
"expected_state_hash": "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
|
||||
"expected_dissent": []
|
||||
"expected_state_hash": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"expected_outcomes": {
|
||||
"node_1": "clean_commit",
|
||||
"node_2": "clean_commit",
|
||||
"node_3": "clean_commit",
|
||||
"node_4": "clean_commit"
|
||||
},
|
||||
"expected_dissent_logs": {
|
||||
"node_1": [],
|
||||
"node_2": [],
|
||||
"node_3": [],
|
||||
"node_4": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "lockstep-002-supermajority-commit",
|
||||
"description": "3 of 4 nodes agree on the post-frame state hash; the 4th node reports a different hash (either Byzantine or simply lagging on input ordering). floor(2*4/3)+1 = 3, threshold met, frame commits at the majority hash. The dissenting node's hash is recorded in the dissent log (spec §14) but does not block commit.",
|
||||
"description": "Three of four nodes agree. node_4's state diverged, and it reports the hash it computed. The threshold of 3 is met: nodes 1–3 commit with dissent, and each writes one dissent-log entry naming node_4 (treatise §14.1). node_4 sees a supermajority on a hash that is not its own — minority divergence — so it halts and writes nothing.",
|
||||
"node_count": 4,
|
||||
"frame_number": 184238,
|
||||
"ordered_transitions": [
|
||||
{
|
||||
"byte_said": "EBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB",
|
||||
"from_pubkey": "5555555555555555555555555555555555555555555555555555555555555555",
|
||||
"to_pubkey": "7777777777777777777777777777777777777777777777777777777777777777",
|
||||
"signature": "88888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888"
|
||||
}
|
||||
],
|
||||
"node_hashes": {
|
||||
"node_1": "BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB",
|
||||
"node_2": "BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB",
|
||||
"node_3": "BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB",
|
||||
"node_4": "DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD"
|
||||
"node_reported_hashes": {
|
||||
"node_1": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"node_2": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"node_3": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"node_4": "dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd"
|
||||
},
|
||||
"node_computed_hashes": {
|
||||
"node_1": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"node_2": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"node_3": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"node_4": "dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd"
|
||||
},
|
||||
"expected_threshold": 3,
|
||||
"expected_commit": true,
|
||||
"expected_state_hash": "BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB",
|
||||
"expected_dissent": [
|
||||
{
|
||||
"node": "node_4",
|
||||
"reported_hash": "DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD",
|
||||
"majority_hash": "BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB",
|
||||
"frame": 184238
|
||||
}
|
||||
]
|
||||
"expected_state_hash": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"expected_outcomes": {
|
||||
"node_1": "commit_with_dissent",
|
||||
"node_2": "commit_with_dissent",
|
||||
"node_3": "commit_with_dissent",
|
||||
"node_4": "minority_divergence"
|
||||
},
|
||||
"expected_dissent_logs": {
|
||||
"node_1": [
|
||||
{
|
||||
"frame": 184238,
|
||||
"dissenting_peers": [
|
||||
"node_4"
|
||||
],
|
||||
"local_hash": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"supermajority_hash": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
|
||||
}
|
||||
],
|
||||
"node_2": [
|
||||
{
|
||||
"frame": 184238,
|
||||
"dissenting_peers": [
|
||||
"node_4"
|
||||
],
|
||||
"local_hash": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"supermajority_hash": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
|
||||
}
|
||||
],
|
||||
"node_3": [
|
||||
{
|
||||
"frame": 184238,
|
||||
"dissenting_peers": [
|
||||
"node_4"
|
||||
],
|
||||
"local_hash": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"supermajority_hash": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
|
||||
}
|
||||
],
|
||||
"node_4": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "lockstep-003-threshold-not-met",
|
||||
"description": "Only 2 of 4 nodes agree on a single state hash. floor(2*4/3)+1 = 3, threshold not met, frame does NOT commit. The cluster must re-run input gathering or escalate per spec §13. The expected_state_hash is null because no single hash reached supermajority.",
|
||||
"description": "No hash is reported by more than two of the four nodes. None reaches the threshold of 3, so the frame does not commit and every node's outcome is a cluster split (§13.3). No dissent is logged: dissent is divergence from a committed hash (§14.2), and there is none.",
|
||||
"node_count": 4,
|
||||
"frame_number": 184239,
|
||||
"ordered_transitions": [
|
||||
{
|
||||
"byte_said": "ECCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC",
|
||||
"from_pubkey": "7777777777777777777777777777777777777777777777777777777777777777",
|
||||
"to_pubkey": "9999999999999999999999999999999999999999999999999999999999999999",
|
||||
"signature": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
|
||||
}
|
||||
],
|
||||
"node_hashes": {
|
||||
"node_1": "CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC",
|
||||
"node_2": "CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC",
|
||||
"node_3": "DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD",
|
||||
"node_4": "EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE"
|
||||
"node_reported_hashes": {
|
||||
"node_1": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc",
|
||||
"node_2": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc",
|
||||
"node_3": "dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd",
|
||||
"node_4": "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee"
|
||||
},
|
||||
"node_computed_hashes": {
|
||||
"node_1": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc",
|
||||
"node_2": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc",
|
||||
"node_3": "dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd",
|
||||
"node_4": "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee"
|
||||
},
|
||||
"expected_threshold": 3,
|
||||
"expected_commit": false,
|
||||
"expected_state_hash": null,
|
||||
"expected_dissent": [
|
||||
{
|
||||
"node": "node_1",
|
||||
"reported_hash": "CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC",
|
||||
"majority_hash": null,
|
||||
"frame": 184239
|
||||
},
|
||||
{
|
||||
"node": "node_2",
|
||||
"reported_hash": "CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC",
|
||||
"majority_hash": null,
|
||||
"frame": 184239
|
||||
},
|
||||
{
|
||||
"node": "node_3",
|
||||
"reported_hash": "DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD",
|
||||
"majority_hash": null,
|
||||
"frame": 184239
|
||||
},
|
||||
{
|
||||
"node": "node_4",
|
||||
"reported_hash": "EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE",
|
||||
"majority_hash": null,
|
||||
"frame": 184239
|
||||
}
|
||||
]
|
||||
"expected_outcomes": {
|
||||
"node_1": "cluster_split",
|
||||
"node_2": "cluster_split",
|
||||
"node_3": "cluster_split",
|
||||
"node_4": "cluster_split"
|
||||
},
|
||||
"expected_dissent_logs": {
|
||||
"node_1": [],
|
||||
"node_2": [],
|
||||
"node_3": [],
|
||||
"node_4": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "lockstep-004-three-nodes-one-diverged-split",
|
||||
"description": "The n = 3 exception (treatise §13.2). The threshold is 3, so one node whose state diverged leaves 2 of the 3 votes the frame needs. The cluster detects the divergence but cannot tolerate it: every node halts with a cluster split.",
|
||||
"node_count": 3,
|
||||
"frame_number": 184240,
|
||||
"node_reported_hashes": {
|
||||
"node_1": "1111111111111111111111111111111111111111111111111111111111111111",
|
||||
"node_2": "1111111111111111111111111111111111111111111111111111111111111111",
|
||||
"node_3": "2222222222222222222222222222222222222222222222222222222222222222"
|
||||
},
|
||||
"node_computed_hashes": {
|
||||
"node_1": "1111111111111111111111111111111111111111111111111111111111111111",
|
||||
"node_2": "1111111111111111111111111111111111111111111111111111111111111111",
|
||||
"node_3": "2222222222222222222222222222222222222222222222222222222222222222"
|
||||
},
|
||||
"expected_threshold": 3,
|
||||
"expected_commit": false,
|
||||
"expected_state_hash": null,
|
||||
"expected_outcomes": {
|
||||
"node_1": "cluster_split",
|
||||
"node_2": "cluster_split",
|
||||
"node_3": "cluster_split"
|
||||
},
|
||||
"expected_dissent_logs": {
|
||||
"node_1": [],
|
||||
"node_2": [],
|
||||
"node_3": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "lockstep-005-seven-nodes-two-dissenters",
|
||||
"description": "Seven nodes, threshold 5: five agree and two diverged nodes report hashes of their own. The frame commits. The dissent entry names both dissenters in ascending order, so every committing node writes the same entry (§14.2).",
|
||||
"node_count": 7,
|
||||
"frame_number": 184241,
|
||||
"node_reported_hashes": {
|
||||
"node_1": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a",
|
||||
"node_2": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a",
|
||||
"node_3": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a",
|
||||
"node_4": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a",
|
||||
"node_5": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a",
|
||||
"node_6": "6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b",
|
||||
"node_7": "7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c"
|
||||
},
|
||||
"node_computed_hashes": {
|
||||
"node_1": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a",
|
||||
"node_2": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a",
|
||||
"node_3": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a",
|
||||
"node_4": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a",
|
||||
"node_5": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a",
|
||||
"node_6": "6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b6b",
|
||||
"node_7": "7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c7c"
|
||||
},
|
||||
"expected_threshold": 5,
|
||||
"expected_commit": true,
|
||||
"expected_state_hash": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a",
|
||||
"expected_outcomes": {
|
||||
"node_1": "commit_with_dissent",
|
||||
"node_2": "commit_with_dissent",
|
||||
"node_3": "commit_with_dissent",
|
||||
"node_4": "commit_with_dissent",
|
||||
"node_5": "commit_with_dissent",
|
||||
"node_6": "minority_divergence",
|
||||
"node_7": "minority_divergence"
|
||||
},
|
||||
"expected_dissent_logs": {
|
||||
"node_1": [
|
||||
{
|
||||
"frame": 184241,
|
||||
"dissenting_peers": [
|
||||
"node_6",
|
||||
"node_7"
|
||||
],
|
||||
"local_hash": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a",
|
||||
"supermajority_hash": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a"
|
||||
}
|
||||
],
|
||||
"node_2": [
|
||||
{
|
||||
"frame": 184241,
|
||||
"dissenting_peers": [
|
||||
"node_6",
|
||||
"node_7"
|
||||
],
|
||||
"local_hash": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a",
|
||||
"supermajority_hash": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a"
|
||||
}
|
||||
],
|
||||
"node_3": [
|
||||
{
|
||||
"frame": 184241,
|
||||
"dissenting_peers": [
|
||||
"node_6",
|
||||
"node_7"
|
||||
],
|
||||
"local_hash": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a",
|
||||
"supermajority_hash": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a"
|
||||
}
|
||||
],
|
||||
"node_4": [
|
||||
{
|
||||
"frame": 184241,
|
||||
"dissenting_peers": [
|
||||
"node_6",
|
||||
"node_7"
|
||||
],
|
||||
"local_hash": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a",
|
||||
"supermajority_hash": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a"
|
||||
}
|
||||
],
|
||||
"node_5": [
|
||||
{
|
||||
"frame": 184241,
|
||||
"dissenting_peers": [
|
||||
"node_6",
|
||||
"node_7"
|
||||
],
|
||||
"local_hash": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a",
|
||||
"supermajority_hash": "5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a"
|
||||
}
|
||||
],
|
||||
"node_6": [],
|
||||
"node_7": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "lockstep-006-transport-liar-is-logged",
|
||||
"description": "Treatise §13.4: node_4 computed the honest hash but broadcast another. Its peers see 3-of-4 agreement and record node_4 in their dissent logs. node_4 tallies its own honest hash, sees unanimity and commits cleanly: its lie surfaces only in its peers' logs, which is why the dissent log is the substrate's misbehaviour signal (§14).",
|
||||
"node_count": 4,
|
||||
"frame_number": 184242,
|
||||
"node_reported_hashes": {
|
||||
"node_1": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"node_2": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"node_3": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"node_4": "dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd"
|
||||
},
|
||||
"node_computed_hashes": {
|
||||
"node_1": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"node_2": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"node_3": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"node_4": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
|
||||
},
|
||||
"expected_threshold": 3,
|
||||
"expected_commit": true,
|
||||
"expected_state_hash": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"expected_outcomes": {
|
||||
"node_1": "commit_with_dissent",
|
||||
"node_2": "commit_with_dissent",
|
||||
"node_3": "commit_with_dissent",
|
||||
"node_4": "clean_commit"
|
||||
},
|
||||
"expected_dissent_logs": {
|
||||
"node_1": [
|
||||
{
|
||||
"frame": 184242,
|
||||
"dissenting_peers": [
|
||||
"node_4"
|
||||
],
|
||||
"local_hash": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"supermajority_hash": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
|
||||
}
|
||||
],
|
||||
"node_2": [
|
||||
{
|
||||
"frame": 184242,
|
||||
"dissenting_peers": [
|
||||
"node_4"
|
||||
],
|
||||
"local_hash": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"supermajority_hash": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
|
||||
}
|
||||
],
|
||||
"node_3": [
|
||||
{
|
||||
"frame": 184242,
|
||||
"dissenting_peers": [
|
||||
"node_4"
|
||||
],
|
||||
"local_hash": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"supermajority_hash": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
|
||||
}
|
||||
],
|
||||
"node_4": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "lockstep-007-three-node-transport-liar",
|
||||
"description": "§13.2 and §13.4 together. At n = 3 a node that broadcasts a hash other than the one it computed splits the honest pair, who halt. The liar sees its own honest hash agree with both peers and commits. The reported hashes reach no supermajority, so the frame's public record is no commit.",
|
||||
"node_count": 3,
|
||||
"frame_number": 184243,
|
||||
"node_reported_hashes": {
|
||||
"node_1": "1111111111111111111111111111111111111111111111111111111111111111",
|
||||
"node_2": "1111111111111111111111111111111111111111111111111111111111111111",
|
||||
"node_3": "2222222222222222222222222222222222222222222222222222222222222222"
|
||||
},
|
||||
"node_computed_hashes": {
|
||||
"node_1": "1111111111111111111111111111111111111111111111111111111111111111",
|
||||
"node_2": "1111111111111111111111111111111111111111111111111111111111111111",
|
||||
"node_3": "1111111111111111111111111111111111111111111111111111111111111111"
|
||||
},
|
||||
"expected_threshold": 3,
|
||||
"expected_commit": false,
|
||||
"expected_state_hash": null,
|
||||
"expected_outcomes": {
|
||||
"node_1": "cluster_split",
|
||||
"node_2": "cluster_split",
|
||||
"node_3": "clean_commit"
|
||||
},
|
||||
"expected_dissent_logs": {
|
||||
"node_1": [],
|
||||
"node_2": [],
|
||||
"node_3": []
|
||||
}
|
||||
}
|
||||
]
|
||||
|
|
|
|||
|
|
@ -1,108 +1,242 @@
|
|||
# Regenerating v1 Conformance Vectors
|
||||
|
||||
The vectors in this directory are **reference vectors generated alongside the
|
||||
Rust reference implementation**, expected to be regenerated when
|
||||
`smart-byte-rs` lands. Until that point, the `expected_*` fields in each
|
||||
JSON file are placeholder values that are *internally consistent* (the same
|
||||
envelope produces the same SAID across vectors) but are not yet
|
||||
cryptographically computed.
|
||||
Every value in this directory is computed by the Rust reference implementation
|
||||
and checked by a second, independent implementation. Nothing in the pack is a
|
||||
placeholder.
|
||||
|
||||
This file documents the regeneration procedure so the placeholders can be
|
||||
replaced with real values in one mechanical pass when the reference
|
||||
implementation is available.
|
||||
- **Generator and verifier:** the `smart-byte-conformance` crate in
|
||||
`../../smart-byte-rs`. Names, descriptions and inputs are written in its
|
||||
`src/pack.rs`; every expected value is computed at generation time by
|
||||
`smart-byte-core` (canonical CBOR, SAIDs, signatures) and
|
||||
`smart-byte-lockstep` (the commit rule and dissent log).
|
||||
- **Second implementation:** `../crosscheck/verify_v1.py`, Python standard
|
||||
library only, written from the spec text in another language. It implements
|
||||
BLAKE3 and Ed25519 itself and checks both against known answers before it
|
||||
checks any vector.
|
||||
|
||||
## Prerequisites
|
||||
## Procedure
|
||||
|
||||
- Rust toolchain pinned by `../../rust-toolchain.toml`.
|
||||
- `smart-byte-rs` crate (private repo, to be open-sourced alongside spec
|
||||
v1.0 freeze).
|
||||
- `cargo run --bin gen-conformance-vectors` produces the v1 pack.
|
||||
|
||||
## Regeneration procedure
|
||||
From `smart-byte/smart-byte-rs`:
|
||||
|
||||
```bash
|
||||
# From the smart-byte-rs checkout
|
||||
cargo run --release --bin gen-conformance-vectors -- \
|
||||
--spec-version v1 \
|
||||
--output-dir /path/to/open-standards/smart-byte/conformance/v1
|
||||
# Write the pack. --output-dir defaults to this directory.
|
||||
cargo run --release -p smart-byte-conformance --bin gen-conformance-vectors -- \
|
||||
--spec-version v1 --output-dir ../conformance/v1
|
||||
|
||||
# Check a pack on disk against the reference implementation, vector by vector.
|
||||
cargo run --release -p smart-byte-conformance --bin verify-conformance-vectors -- \
|
||||
--pack ../conformance/v1
|
||||
|
||||
# Check it against the second implementation.
|
||||
python3 ../conformance/crosscheck/verify_v1.py ../conformance/v1
|
||||
```
|
||||
|
||||
The generator does the following, in order:
|
||||
`gen-conformance-vectors --check` writes nothing and exits non-zero if any file
|
||||
differs from the generator's output.
|
||||
|
||||
1. **Canonical CBOR vectors.** For each `input` JSON object, run the
|
||||
canonical CBOR encoder and write the hex output to `expected_cbor_hex`.
|
||||
The encoder uses RFC 8949 §4.2.1 (Core Deterministic Encoding) plus the
|
||||
additional Smart Byte rules: byte-string for byte arrays, no tags,
|
||||
sorted map keys.
|
||||
To change a vector, edit `smart-byte-conformance/src/pack.rs` and regenerate.
|
||||
Editing the JSON by hand fails CI: `cargo test` in `smart-byte-rs` requires the
|
||||
published files to equal the generator's output byte for byte, verifies every
|
||||
vector, and plants a defect in each file to prove the verifier would catch it.
|
||||
The conformance gate runs the second implementation on every pack change.
|
||||
|
||||
2. **SAID vectors.** For each `envelope`, substitute a zero-byte placeholder
|
||||
for the `said` field, canonical-CBOR-encode, BLAKE3-256 hash, and write
|
||||
the result to `expected_said`. The SAID format is the KERI prefix-byte
|
||||
scheme: `E` prefix byte (= BLAKE3-256) followed by the base64url
|
||||
encoding of the 32-byte hash. The `canonical_cbor_hex` field is also
|
||||
populated.
|
||||
## What the generator computes
|
||||
|
||||
3. **Signature vectors.** For each positive vector, derive the Ed25519
|
||||
verifying key from the seed via the standard RFC 8032 procedure, sign
|
||||
the `envelope_said` (32 bytes, raw, not the prefix-byte format) under
|
||||
the signing key, and write the signature to the `signature` field. For
|
||||
negative vectors, the generator does not modify the explicit
|
||||
`signature` / `verifying_key` fields; it only verifies that the
|
||||
`expected_verify: false` assertion holds when the named verifier runs
|
||||
the vector.
|
||||
|
||||
4. **Lockstep vectors.** For each vector, the generator runs the reference
|
||||
BFT-commit logic on the provided `node_hashes` and confirms that
|
||||
`expected_commit` and `expected_state_hash` match. For commits, it also
|
||||
confirms the `expected_dissent` log matches what the reference
|
||||
dissent-tracking logic produces.
|
||||
|
||||
5. **Gossip vectors.** For each scenario, the generator simulates the
|
||||
gossip propagation under spec §17 rules and confirms the
|
||||
`expected_outcome` / `expected_observations` match the reference
|
||||
gossip-engine output.
|
||||
1. **Canonicalisation vectors.** A positive vector's `input` is encoded to
|
||||
`expected_cbor_hex`, and the strict decoder must accept those bytes
|
||||
unchanged. A negative vector's `input_cbor_hex` must be refused with its
|
||||
`error` class; generation fails if the reference refuses it for any other
|
||||
reason, so a negative vector cannot pass by accident.
|
||||
2. **SAID vectors.** `input` holds the 53-`#` placeholder in its SAID field
|
||||
`d`. Its canonical encoding is `canonical_cbor_hex` (the bytes that are
|
||||
hashed), their BLAKE3-256 is `blake3_hex`, that digest as `E` + lowercase
|
||||
base32 is `expected_said`, and the object with its SAID in place is
|
||||
`said_cbor_hex`, which must verify. A negative vector's `input_cbor_hex` is
|
||||
refused by SAID verification with its `error` class.
|
||||
3. **Signature vectors.** The signed message is the 32 digest bytes a SAID
|
||||
spells (`signed_message_hex`), not its 53-character text. A positive
|
||||
vector's `signature` is the deterministic Ed25519 signature by
|
||||
`signing_key_seed`; a negative vector is refused with its `error` class.
|
||||
4. **Lockstep vectors.** The hashes nodes reported are tallied against the
|
||||
threshold floor(2n/3)+1 (treatise §13.1) for the frame's public record.
|
||||
Each node's outcome (§13.3) and dissent log (§14.1) come from its own
|
||||
view: its peers' reported hashes plus the hash it computed, so a node that
|
||||
lies on the wire (§13.4) is modelled exactly as the treatise describes.
|
||||
5. **Gossip vectors.** Each delivery goes through `smart-byte-core`'s inbox,
|
||||
which implements identity spec §2.6: canonical bytes, then the SAID they
|
||||
state, then the SAID they arrived under, and only then deduplication.
|
||||
|
||||
## Invariant: cross-vector consistency
|
||||
|
||||
The generator MUST emit consistent values across vectors. In particular:
|
||||
- Every `envelope_said` in `signature_vectors.json` is the `expected_said` of a
|
||||
vector in `said_vectors.json` — the verifier and the second implementation
|
||||
both enforce it.
|
||||
- Gossip deliveries carry SAID vectors' `said_cbor_hex` bytes, except where a
|
||||
vector alters them on purpose and says so.
|
||||
|
||||
- An envelope appearing in `said_vectors.json` and `lockstep_vectors.json`
|
||||
has the same SAID in both files.
|
||||
- A SAID appearing in `signature_vectors.json` as `envelope_said` and in
|
||||
`said_vectors.json` as `expected_said` is identical.
|
||||
## Cross-language check
|
||||
|
||||
The placeholder values in the current pack are internally consistent in
|
||||
the same way; the regeneration replaces each placeholder family with the
|
||||
real value computed from the canonical pipeline.
|
||||
This document has always required a second implementation to reproduce the
|
||||
`expected_*` values before a regeneration merges. `../crosscheck/verify_v1.py`
|
||||
is that implementation, and it reproduces every vector. Be clear about its
|
||||
limits: it was written by the same author as the generator, from the spec text
|
||||
rather than from the Rust. An implementation written by someone else is still
|
||||
the stronger test. The Go and TypeScript implementations an earlier draft of
|
||||
this document planned do not exist.
|
||||
|
||||
## What is NOT regenerated
|
||||
## Editorial change log
|
||||
|
||||
- Vector `name` fields (stable across regenerations; implementations pin
|
||||
to names, not array positions).
|
||||
- Vector `description` fields (human prose, not generator output).
|
||||
- The set of vectors (additions go through `VERSIONS.md` and the
|
||||
CI gate).
|
||||
### 2026-09-30 — the pack regenerated from the reference implementation
|
||||
|
||||
## Verifying a regeneration
|
||||
Every `expected_*` value was a placeholder. The SAIDs were runs of one repeated
|
||||
character, every `canonical_cbor_hex` was the same 7-byte stub, and the
|
||||
signatures were RFC 8032 test signatures over the empty message and over 0x72.
|
||||
None of them signed any SAID. All five canonicalisation encodings were wrong,
|
||||
and two of them were odd-length hex that could not be decoded at all. Every
|
||||
value is now computed, and checked twice (see above).
|
||||
|
||||
After regeneration, run:
|
||||
**Regenerated in place.** v1 is DRAFT, and a DRAFT pack is regenerated in
|
||||
place; supersession governs a frozen pack (`../VERSIONS.md` rule 4, now
|
||||
explicit about this). A few draft vectors did hold values an implementation
|
||||
could pass — lockstep-001's tally over identical hashes, and the sig-004 and
|
||||
sig-005 negatives. They keep their names and subjects here.
|
||||
|
||||
```bash
|
||||
cargo run --release --bin verify-conformance-vectors -- \
|
||||
--pack /path/to/open-standards/smart-byte/conformance/v1
|
||||
```
|
||||
**Inputs that had to change before anything could be computed:**
|
||||
|
||||
The verifier reads each vector, re-runs the canonical pipeline, and
|
||||
asserts byte-for-byte equality with the file contents. A clean exit
|
||||
status confirms the pack is self-consistent and matches the reference
|
||||
implementation's current behaviour.
|
||||
- JSON has no byte-string type, and the draft wrote byte values as hex text,
|
||||
which is indistinguishable from text. Byte strings are now written
|
||||
`{"$bytes": "<lowercase hex>"}`, and `$bytes` is reserved (`../README.md`).
|
||||
- SAID vectors: the SAID field is `d`, holding the 53-`#` placeholder (identity
|
||||
spec §2.1, §2.2). The draft used a `said` member holding `<PLACEHOLDER_SAID>`.
|
||||
The object member is now `input` rather than `envelope`, and the `version`
|
||||
member is gone.
|
||||
- `said-005`'s `previous_history_hash` values were SAID-shaped filler. They
|
||||
are now computed, and said-005's SAID is its own. The draft gave it
|
||||
said-004's SAID, but a SAID covers the whole object it is derived over.
|
||||
- `canon-003`'s byte string was given as base64 text; it is now bytes.
|
||||
|
||||
## Cross-language sanity check
|
||||
**Vectors whose meaning contradicted the spec, corrected (and renamed where the
|
||||
name stated the wrong rule):**
|
||||
|
||||
The vectors are the contract; the Rust reference implementation is the
|
||||
*proximate* generator but not the canonical authority. Before merging a
|
||||
regeneration, at least one additional implementation (currently planned:
|
||||
Go and TypeScript) must independently produce the same `expected_*`
|
||||
values from the same inputs. If implementations disagree, the disagreement
|
||||
is resolved by appeal to the spec text, not by which implementation
|
||||
emitted the value first.
|
||||
- `canon-004-deterministic-float-encoding` → `canon-004-floats-forbidden`. The
|
||||
draft encoded floats in their shortest exact width; identity spec §2.3
|
||||
forbids floats. canon-016/017 cover the other widths.
|
||||
- `lockstep-003` logged dissent for all four nodes of a frame that did not
|
||||
commit. Treatise §14.2 writes the dissent log only on a commit with dissent.
|
||||
- Lockstep vectors now separate the hash a node broadcast from the one it
|
||||
computed, and give each node's outcome and dissent log from its own view.
|
||||
The draft's single view called a node a "liar" while modelling a node whose
|
||||
state diverged; treatise §13.4's liar commits in its own view while its
|
||||
peers log it, which lockstep-006 and lockstep-007 now pin. lockstep-004 is
|
||||
renamed to say what it models: a diverged node.
|
||||
- Lockstep vectors no longer carry `ordered_transitions`. They were never
|
||||
inputs to the commit rule, and their "signatures" were filler.
|
||||
- `gossip-002-said-conflict-rejected` keeps its name but not its rule. The
|
||||
draft kept whichever object arrived first. Two different canonical objects
|
||||
cannot share a SAID short of a BLAKE3 collision, so the newcomer is refused
|
||||
because it does not hash to the SAID it states, in either arrival order
|
||||
(gossip-006). Verification precedes deduplication (identity spec §2.6).
|
||||
- `gossip-003-sequence-gap-detection` → `gossip-003-backfill-answer-verified`.
|
||||
The draft asserted a 256-frame backfill limit, request targets and alert
|
||||
classes that no spec text defines. The replacement tests what the spec
|
||||
fixes: an answer to a request by SAID is checked against that SAID.
|
||||
- Gossip vectors carry generic SAID-addressed objects, not envelope-shaped
|
||||
ones. A node that also checks a byte's provenance (treatise §8.3) must
|
||||
refuse an envelope with null or filler provenance, so such deliveries could
|
||||
not be MUST-pass. §2.6 applies to every SAID-addressed object.
|
||||
- `gossip-001` no longer asserts frames or round counts — gossip is not
|
||||
frame-synchronous — and `expected_held` now carries the bytes held.
|
||||
- Signature vectors: sig-003 and sig-004 share the class `signature_invalid`.
|
||||
Both are the same failed equation, and no verifier can tell them apart.
|
||||
sig-004's key is derived from its seed, and sig-005's class is
|
||||
`malformed_signature`.
|
||||
|
||||
**Vectors added:**
|
||||
|
||||
- canon-006 to canon-020: each §2.3 rule, accepted and refused.
|
||||
- said-006 to said-011: the suite identity spec §2.5 names.
|
||||
- sig-006 (a non-canonical S) and sig-007 (a key spelled with y ≥ p).
|
||||
- lockstep-004 to lockstep-007.
|
||||
- gossip-004 to gossip-006.
|
||||
|
||||
**Spec text changed with it (identity spec):**
|
||||
|
||||
- §2.4 printed an invented digest and SAID, and counted 62 bytes; it now
|
||||
prints said-008's values for its 61 bytes.
|
||||
- §2.1, §2.3, §2.4 and §4 step 1 told a verifier to re-encode an object
|
||||
canonically, which is exactly what §2.5's adversarial vector forbids. They
|
||||
now require received bytes to be canonical already.
|
||||
- §2.3 now forbids simple values other than false, true and null.
|
||||
- §2.6 is new: receiving SAID-addressed objects.
|
||||
- §3.3 and §3.4 now say a key is a byte string.
|
||||
- §2.5 and §6 now name the vectors and the modules.
|
||||
|
||||
**Reference implementation changed with it:**
|
||||
|
||||
- `smart-byte-core`:
|
||||
- canonical CBOR, with a strict decoder that never allocates from a
|
||||
declared count and reports the first defect in stream order;
|
||||
- §2 SAIDs and the §2.6 inbox;
|
||||
- SAID signature verification, with RFC 8032-strict key decoding;
|
||||
- `Envelope::from_cbor`, which accepts only an envelope's own encoding.
|
||||
- `smart-byte-lockstep`: the §13/§14 tally and each node's own view.
|
||||
`Cluster::step` keeps a dissent log.
|
||||
- `smart-byte-libp2p`:
|
||||
- gossip is validated before it is forwarded, so a forged or padded
|
||||
envelope stops at the first honest node;
|
||||
- a fetch's answer must be the envelope requested;
|
||||
- one bad message no longer ends a wait.
|
||||
- `smart-byte-keri-witness`: the verifier rejects `rec` by default, as spec
|
||||
v1 requires. Recovery is an explicit opt-in.
|
||||
|
||||
**Checked whole.** Both checkers require every file to be non-empty, every
|
||||
vector the identity spec cites to be present, and every README error class
|
||||
to be exercised — and no other. A pack cannot pass by losing vectors. The
|
||||
Python checker raises its own failures rather than using `assert`, so
|
||||
`python -O` cannot switch it off. Both refuse JSON that names a member twice.
|
||||
|
||||
**Next:** envelope and key-event-log identifiers move to §2 SAIDs. The
|
||||
envelope's wire format and signing preimages get a spec of their own, and an
|
||||
inception event placeholders `i` with `d`, because it derives its controller
|
||||
AID from its own SAID.
|
||||
|
||||
### 2026-09-21 — spec reference corrected; no wire-format change
|
||||
|
||||
`spec/identity_and_key_rotation.md` pointed three times at
|
||||
`../conformance/said_vectors.json`. That path does not resolve: the pack is at
|
||||
`../conformance/v1/said_vectors.json`. Line 266 of that spec states "all paths
|
||||
are relative to this file", so both the breakage and the intent were
|
||||
unambiguous, and an implementer following the normative reference reached
|
||||
nothing. The three references now carry the `v1/` segment.
|
||||
|
||||
**The wire format, the vectors and every `expected_*` value are unchanged.**
|
||||
This entry exists because the conformance gate requires a spec change to be
|
||||
accompanied by a pack change, and directs an editorial change to record its
|
||||
reason here. Recording it is the point; nothing in the pack was edited to
|
||||
satisfy the gate.
|
||||
|
||||
Two things found while making that correction, neither of them fixed here
|
||||
because both are normative decisions:
|
||||
|
||||
1. **The regeneration procedure above is not runnable as written.** It names
|
||||
`cargo run --bin gen-conformance-vectors` to produce the pack and
|
||||
`verify-conformance-vectors` to check it. **Neither binary exists** in
|
||||
`smart-byte-rs` — the workspace ships one binary, `smart-byte`. So the pass
|
||||
this document describes cannot be run in either direction, which is a
|
||||
stronger statement than "it has not been run yet".
|
||||
|
||||
2. **The precondition this document gives for regenerating has been met.** The
|
||||
status section says the placeholders stand until `smart-byte-rs` lands, and
|
||||
describes it as a private repo to be open-sourced at spec v1.0 freeze. It has
|
||||
landed, it is in this repository, and it is built and tested by the
|
||||
`standards-build` CI matrix. The blocker recorded here is stale.
|
||||
|
||||
Also worth carrying into whoever does the regeneration:
|
||||
`spec/identity_and_key_rotation.md` says implementers **MUST pass every vector**,
|
||||
and describes the suite as including "one adversarial case where a non-canonical
|
||||
CBOR encoding produces a different digest (to verify that implementations reject
|
||||
non-canonical input rather than silently re-encoding)". **That vector is not in
|
||||
this pack** — every SAID vector here is envelope-shaped, and the adversarial one
|
||||
is the only negative case the spec names. A regeneration that replaces the placeholder
|
||||
values without adding it would still leave the spec describing a suite this pack
|
||||
does not contain.
|
||||
|
|
|
|||
|
|
@ -1,10 +1,10 @@
|
|||
[
|
||||
{
|
||||
"name": "said-001-empty-envelope",
|
||||
"description": "SAID computation for the minimal envelope: schema-required fields only, no cargo, no provenance, no ownership transitions. This is the degenerate case used to verify that an implementation handles the canonical-CBOR + BLAKE3-256 pipeline on a payload with no application data.",
|
||||
"envelope": {
|
||||
"version": 1,
|
||||
"said": "<PLACEHOLDER_SAID>",
|
||||
"description": "A nested object with an envelope's top-level members — provenance, ownership chain, cargo, joule cost — all empty or zero. It exercises the whole pipeline (placeholder, canonical CBOR, BLAKE3-256, `E` + base32) on an object with no application data.",
|
||||
"said_field": "d",
|
||||
"input": {
|
||||
"d": "#####################################################",
|
||||
"provenance": null,
|
||||
"ownership_chain": [],
|
||||
"cargo": null,
|
||||
|
|
@ -13,36 +13,42 @@
|
|||
"estimated_microjoules": 0
|
||||
}
|
||||
},
|
||||
"canonical_cbor_hex": "a6677665727369",
|
||||
"expected_said": "EAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
|
||||
"canonical_cbor_hex": "a561647835232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232365636172676ff66a6a6f756c655f636f7374a2746d656173757265645f6d6963726f6a6f756c65730075657374696d617465645f6d6963726f6a6f756c6573006a70726f76656e616e6365f66f6f776e6572736869705f636861696e80",
|
||||
"blake3_hex": "f715776741e2a8a4157d584ed68c4574b4185c693bfb635ff93efa48be55d736",
|
||||
"expected_said": "E64kxoz2b4kukifl5lbhnndcfos2bqxdjhp5wgx7zh35erpsv243a",
|
||||
"said_cbor_hex": "a5616478354536346b786f7a3262346b756b69666c356c62686e6e6463666f7332627178646a687035776778377a68333565727073763234336165636172676ff66a6a6f756c655f636f7374a2746d656173757265645f6d6963726f6a6f756c65730075657374696d617465645f6d6963726f6a6f756c6573006a70726f76656e616e6365f66f6f776e6572736869705f636861696e80"
|
||||
},
|
||||
{
|
||||
"name": "said-002-usd-cargo",
|
||||
"description": "SAID computation for an envelope carrying USD cargo. Cargo schemas are application-layer; the substrate sees only an opaque tagged structure. This vector fixes the cargo to a 100.00 USD denomination issued by issuer pubkey 0x11...11.",
|
||||
"envelope": {
|
||||
"version": 1,
|
||||
"said": "<PLACEHOLDER_SAID>",
|
||||
"description": "The same object carrying a cargo map: a USD amount of 100.00 (100000000 micros) and a 32-byte issuer key. Members of mixed types — text, integer, byte string — all sort by their encoded keys.",
|
||||
"said_field": "d",
|
||||
"input": {
|
||||
"d": "#####################################################",
|
||||
"provenance": null,
|
||||
"ownership_chain": [],
|
||||
"cargo": {
|
||||
"type": "usd.cargo.v1",
|
||||
"denomination_micros": 100000000,
|
||||
"issuer_pubkey": "1111111111111111111111111111111111111111111111111111111111111111"
|
||||
"issuer_pubkey": {
|
||||
"$bytes": "1111111111111111111111111111111111111111111111111111111111111111"
|
||||
}
|
||||
},
|
||||
"joule_cost": {
|
||||
"measured_microjoules": 12,
|
||||
"estimated_microjoules": 15
|
||||
}
|
||||
},
|
||||
"canonical_cbor_hex": "a6677665727369",
|
||||
"expected_said": "EBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB"
|
||||
"canonical_cbor_hex": "a561647835232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232365636172676fa364747970656c7573642e636172676f2e76316d6973737565725f7075626b6579582011111111111111111111111111111111111111111111111111111111111111117364656e6f6d696e6174696f6e5f6d6963726f731a05f5e1006a6a6f756c655f636f7374a2746d656173757265645f6d6963726f6a6f756c65730c75657374696d617465645f6d6963726f6a6f756c65730f6a70726f76656e616e6365f66f6f776e6572736869705f636861696e80",
|
||||
"blake3_hex": "98560e8b8d05e5125c055df313b3e58bd411888644d463b965f084c9f9167baf",
|
||||
"expected_said": "Etbla5c4naxsrexaflxzrhm7frpkbdcegitkgholf6ccmt6iwpoxq",
|
||||
"said_cbor_hex": "a5616478354574626c613563346e61787372657861666c787a72686d376672706b626463656769746b67686f6c663663636d74366977706f787165636172676fa364747970656c7573642e636172676f2e76316d6973737565725f7075626b6579582011111111111111111111111111111111111111111111111111111111111111117364656e6f6d696e6174696f6e5f6d6963726f731a05f5e1006a6a6f756c655f636f7374a2746d656173757265645f6d6963726f6a6f756c65730c75657374696d617465645f6d6963726f6a6f756c65730f6a70726f76656e616e6365f66f6f776e6572736869705f636861696e80"
|
||||
},
|
||||
{
|
||||
"name": "said-003-custom-cargo",
|
||||
"description": "SAID computation for an envelope carrying a non-money cargo type (in this case a temperature sensor reading). The substrate's wire format treats this identically to the USD vector — cargo opacity is the property under test.",
|
||||
"envelope": {
|
||||
"version": 1,
|
||||
"said": "<PLACEHOLDER_SAID>",
|
||||
"description": "The same shape carrying a temperature reading as cargo. What a member means never affects its encoding: this cargo goes through the pipeline exactly as said-002's does.",
|
||||
"said_field": "d",
|
||||
"input": {
|
||||
"d": "#####################################################",
|
||||
"provenance": null,
|
||||
"ownership_chain": [],
|
||||
"cargo": {
|
||||
|
|
@ -56,76 +62,175 @@
|
|||
"estimated_microjoules": 10
|
||||
}
|
||||
},
|
||||
"canonical_cbor_hex": "a6677665727369",
|
||||
"expected_said": "ECCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC"
|
||||
"canonical_cbor_hex": "a561647835232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232365636172676fa464747970657573656e736f722e74656d70657261747572652e76316973656e736f725f69646f726f6f66746f702d61727261792d376d63656c736975735f6d696c6c69195bb36d72656164696e675f6672616d651a0002cfac6a6a6f756c655f636f7374a2746d656173757265645f6d6963726f6a6f756c65730875657374696d617465645f6d6963726f6a6f756c65730a6a70726f76656e616e6365f66f6f776e6572736869705f636861696e80",
|
||||
"blake3_hex": "30b1d5aad87f38aa3014135de70a49cd79c9df422fc62eeb3ef7d94bc3ec970f",
|
||||
"expected_said": "Egcy5lkwyp44kumaucno6ocsjzv44tx2cf7dc52z667muxq7ms4hq",
|
||||
"said_cbor_hex": "a56164783545676379356c6b77797034346b756d6175636e6f366f63736a7a763434747832636637646335327a3636376d757871376d7334687165636172676fa464747970657573656e736f722e74656d70657261747572652e76316973656e736f725f69646f726f6f66746f702d61727261792d376d63656c736975735f6d696c6c69195bb36d72656164696e675f6672616d651a0002cfac6a6a6f756c655f636f7374a2746d656173757265645f6d6963726f6a6f756c65730875657374696d617465645f6d6963726f6a6f756c65730a6a70726f76656e616e6365f66f6f776e6572736869705f636861696e80"
|
||||
},
|
||||
{
|
||||
"name": "said-004-with-provenance",
|
||||
"description": "SAID computation for an envelope with a provenance attestation. Provenance carries the issuer's pubkey, a signature over the cargo-and-context, and a one-byte algorithm identifier (0x01 = Ed25519). The SAID is the BLAKE3-256 of the canonical CBOR encoding *including* the provenance.",
|
||||
"envelope": {
|
||||
"version": 1,
|
||||
"said": "<PLACEHOLDER_SAID>",
|
||||
"description": "The same shape with a provenance map: algorithm identifier 0x01 (Ed25519), issuer key, issuance frame and signature. The SAID covers every member. The key and signature are fixed filler bytes: this vector tests SAID derivation over them, not their validity.",
|
||||
"said_field": "d",
|
||||
"input": {
|
||||
"d": "#####################################################",
|
||||
"provenance": {
|
||||
"alg": 1,
|
||||
"issuer_pubkey": "1111111111111111111111111111111111111111111111111111111111111111",
|
||||
"issuer_pubkey": {
|
||||
"$bytes": "1111111111111111111111111111111111111111111111111111111111111111"
|
||||
},
|
||||
"issuance_frame": 1024,
|
||||
"signature": "22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222"
|
||||
"signature": {
|
||||
"$bytes": "22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222"
|
||||
}
|
||||
},
|
||||
"ownership_chain": [],
|
||||
"cargo": {
|
||||
"type": "usd.cargo.v1",
|
||||
"denomination_micros": 100000000,
|
||||
"issuer_pubkey": "1111111111111111111111111111111111111111111111111111111111111111"
|
||||
"issuer_pubkey": {
|
||||
"$bytes": "1111111111111111111111111111111111111111111111111111111111111111"
|
||||
}
|
||||
},
|
||||
"joule_cost": {
|
||||
"measured_microjoules": 14,
|
||||
"estimated_microjoules": 18
|
||||
}
|
||||
},
|
||||
"canonical_cbor_hex": "a6677665727369",
|
||||
"expected_said": "EDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD"
|
||||
"canonical_cbor_hex": "a561647835232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232365636172676fa364747970656c7573642e636172676f2e76316d6973737565725f7075626b6579582011111111111111111111111111111111111111111111111111111111111111117364656e6f6d696e6174696f6e5f6d6963726f731a05f5e1006a6a6f756c655f636f7374a2746d656173757265645f6d6963726f6a6f756c65730e75657374696d617465645f6d6963726f6a6f756c6573126a70726f76656e616e6365a463616c6701697369676e61747572655840222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222226d6973737565725f7075626b6579582011111111111111111111111111111111111111111111111111111111111111116e69737375616e63655f6672616d651904006f6f776e6572736869705f636861696e80",
|
||||
"blake3_hex": "24bef3e8d603a964c5de687b67d86eb9d77c100d6be2ff85f19284e6d023f03a",
|
||||
"expected_said": "Ees7ph2gwaouwjro6nb5wpwdoxhlxyeannprp7bprskconubd6a5a",
|
||||
"said_cbor_hex": "a561647835456573377068326777616f75776a726f366e6235777077646f78686c787965616e6e70727037627072736b636f6e7562643661356165636172676fa364747970656c7573642e636172676f2e76316d6973737565725f7075626b6579582011111111111111111111111111111111111111111111111111111111111111117364656e6f6d696e6174696f6e5f6d6963726f731a05f5e1006a6a6f756c655f636f7374a2746d656173757265645f6d6963726f6a6f756c65730e75657374696d617465645f6d6963726f6a6f756c6573126a70726f76656e616e6365a463616c6701697369676e61747572655840222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222226d6973737565725f7075626b6579582011111111111111111111111111111111111111111111111111111111111111116e69737375616e63655f6672616d651904006f6f776e6572736869705f636861696e80"
|
||||
},
|
||||
{
|
||||
"name": "said-005-with-ownership-transitions",
|
||||
"description": "SAID computation for an envelope with two completed ownership transitions. Each transition signs (byte SAID, previous history hash, from pubkey, to pubkey, frame number). Note that the byte's SAID is computed over the *origin* envelope and never changes; this vector represents the envelope *as it would be encoded in the wire format when fetched at frame 1100*, with two transitions in its history.",
|
||||
"envelope": {
|
||||
"version": 1,
|
||||
"said": "<PLACEHOLDER_SAID>",
|
||||
"description": "said-004's object with two ownership transitions appended and a larger joule cost: an array of maps holding byte strings. The first transition's previous_history_hash is the digest of said-004's SAID; the second is blake3 of that digest followed by the first transition's canonical encoding. The SAID differs from said-004's because a SAID covers the whole object it is derived over. A byte's identity is a different quantity — the hash of its origin attestation (treatise §8.2) — and does not change as transitions accrue.",
|
||||
"said_field": "d",
|
||||
"input": {
|
||||
"d": "#####################################################",
|
||||
"provenance": {
|
||||
"alg": 1,
|
||||
"issuer_pubkey": "1111111111111111111111111111111111111111111111111111111111111111",
|
||||
"issuer_pubkey": {
|
||||
"$bytes": "1111111111111111111111111111111111111111111111111111111111111111"
|
||||
},
|
||||
"issuance_frame": 1024,
|
||||
"signature": "22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222"
|
||||
"signature": {
|
||||
"$bytes": "22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222"
|
||||
}
|
||||
},
|
||||
"ownership_chain": [
|
||||
{
|
||||
"alg": 1,
|
||||
"from_pubkey": "1111111111111111111111111111111111111111111111111111111111111111",
|
||||
"to_pubkey": "3333333333333333333333333333333333333333333333333333333333333333",
|
||||
"from_pubkey": {
|
||||
"$bytes": "1111111111111111111111111111111111111111111111111111111111111111"
|
||||
},
|
||||
"to_pubkey": {
|
||||
"$bytes": "3333333333333333333333333333333333333333333333333333333333333333"
|
||||
},
|
||||
"frame": 1050,
|
||||
"previous_history_hash": "EDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD",
|
||||
"signature": "44444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444"
|
||||
"previous_history_hash": {
|
||||
"$bytes": "24bef3e8d603a964c5de687b67d86eb9d77c100d6be2ff85f19284e6d023f03a"
|
||||
},
|
||||
"signature": {
|
||||
"$bytes": "44444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444"
|
||||
}
|
||||
},
|
||||
{
|
||||
"alg": 1,
|
||||
"from_pubkey": "3333333333333333333333333333333333333333333333333333333333333333",
|
||||
"to_pubkey": "5555555555555555555555555555555555555555555555555555555555555555",
|
||||
"from_pubkey": {
|
||||
"$bytes": "3333333333333333333333333333333333333333333333333333333333333333"
|
||||
},
|
||||
"to_pubkey": {
|
||||
"$bytes": "5555555555555555555555555555555555555555555555555555555555555555"
|
||||
},
|
||||
"frame": 1080,
|
||||
"previous_history_hash": "EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE",
|
||||
"signature": "66666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666"
|
||||
"previous_history_hash": {
|
||||
"$bytes": "48dd9261e8f0836e26d963e2924a0441c2ad1620654cbe76159581d10775aa15"
|
||||
},
|
||||
"signature": {
|
||||
"$bytes": "66666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666"
|
||||
}
|
||||
}
|
||||
],
|
||||
"cargo": {
|
||||
"type": "usd.cargo.v1",
|
||||
"denomination_micros": 100000000,
|
||||
"issuer_pubkey": "1111111111111111111111111111111111111111111111111111111111111111"
|
||||
"issuer_pubkey": {
|
||||
"$bytes": "1111111111111111111111111111111111111111111111111111111111111111"
|
||||
}
|
||||
},
|
||||
"joule_cost": {
|
||||
"measured_microjoules": 47,
|
||||
"estimated_microjoules": 56
|
||||
}
|
||||
},
|
||||
"canonical_cbor_hex": "a6677665727369",
|
||||
"expected_said": "EDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD"
|
||||
"canonical_cbor_hex": "a561647835232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232365636172676fa364747970656c7573642e636172676f2e76316d6973737565725f7075626b6579582011111111111111111111111111111111111111111111111111111111111111117364656e6f6d696e6174696f6e5f6d6963726f731a05f5e1006a6a6f756c655f636f7374a2746d656173757265645f6d6963726f6a6f756c6573182f75657374696d617465645f6d6963726f6a6f756c657318386a70726f76656e616e6365a463616c6701697369676e61747572655840222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222226d6973737565725f7075626b6579582011111111111111111111111111111111111111111111111111111111111111116e69737375616e63655f6672616d651904006f6f776e6572736869705f636861696e82a663616c6701656672616d6519041a697369676e617475726558404444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444469746f5f7075626b6579582033333333333333333333333333333333333333333333333333333333333333336b66726f6d5f7075626b6579582011111111111111111111111111111111111111111111111111111111111111117570726576696f75735f686973746f72795f68617368582024bef3e8d603a964c5de687b67d86eb9d77c100d6be2ff85f19284e6d023f03aa663616c6701656672616d65190438697369676e617475726558406666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666669746f5f7075626b6579582055555555555555555555555555555555555555555555555555555555555555556b66726f6d5f7075626b6579582033333333333333333333333333333333333333333333333333333333333333337570726576696f75735f686973746f72795f68617368582048dd9261e8f0836e26d963e2924a0441c2ad1620654cbe76159581d10775aa15",
|
||||
"blake3_hex": "c670b45906645f9144fc20b1638b48ac183fe09f70ed6903ebc10283083901f2",
|
||||
"expected_said": "Eyzyliwigmrpzcrh4ecywhc2ivqmd7ye7odwwsa7lyebigcbzahza",
|
||||
"said_cbor_hex": "a56164783545797a796c697769676d72707a63726834656379776863326976716d64377965376f6477777361376c796562696763627a61687a6165636172676fa364747970656c7573642e636172676f2e76316d6973737565725f7075626b6579582011111111111111111111111111111111111111111111111111111111111111117364656e6f6d696e6174696f6e5f6d6963726f731a05f5e1006a6a6f756c655f636f7374a2746d656173757265645f6d6963726f6a6f756c6573182f75657374696d617465645f6d6963726f6a6f756c657318386a70726f76656e616e6365a463616c6701697369676e61747572655840222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222226d6973737565725f7075626b6579582011111111111111111111111111111111111111111111111111111111111111116e69737375616e63655f6672616d651904006f6f776e6572736869705f636861696e82a663616c6701656672616d6519041a697369676e617475726558404444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444469746f5f7075626b6579582033333333333333333333333333333333333333333333333333333333333333336b66726f6d5f7075626b6579582011111111111111111111111111111111111111111111111111111111111111117570726576696f75735f686973746f72795f68617368582024bef3e8d603a964c5de687b67d86eb9d77c100d6be2ff85f19284e6d023f03aa663616c6701656672616d65190438697369676e617475726558406666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666669746f5f7075626b6579582055555555555555555555555555555555555555555555555555555555555555556b66726f6d5f7075626b6579582033333333333333333333333333333333333333333333333333333333333333337570726576696f75735f686973746f72795f68617368582048dd9261e8f0836e26d963e2924a0441c2ad1620654cbe76159581d10775aa15"
|
||||
},
|
||||
{
|
||||
"name": "said-006-only-said-field",
|
||||
"description": "The empty-map case of identity spec §2.5: an object whose only member is its SAID field. It is the smallest object a SAID can be derived over — a truly empty map has nowhere to put one.",
|
||||
"said_field": "d",
|
||||
"input": {
|
||||
"d": "#####################################################"
|
||||
},
|
||||
"canonical_cbor_hex": "a1616478352323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323",
|
||||
"blake3_hex": "e0e9858d678259ae81196c47a1de59f52e53dfd5c413a44dd336ce901eb00291",
|
||||
"expected_said": "E4duyldlhqjm25aiznrd2dxsz6uxfhx6vyqj2itotg3hjahvqakiq",
|
||||
"said_cbor_hex": "a16164783545346475796c646c68716a6d323561697a6e7264326478737a367578666878367679716a3269746f746733686a61687671616b6971"
|
||||
},
|
||||
{
|
||||
"name": "said-007-one-field-per-key",
|
||||
"description": "The one-field case: identity spec §3.4's per-key object {d, k}, whose SAID is a next-key commitment. k is a byte string: the algorithm identifier 0x01 (Ed25519) followed by the public key of RFC 8032's first test seed (§3.3).",
|
||||
"said_field": "d",
|
||||
"input": {
|
||||
"d": "#####################################################",
|
||||
"k": {
|
||||
"$bytes": "01d75a980182b10ab7d54bfed3c964073a0ee172f3daa62325af021a68f707511a"
|
||||
}
|
||||
},
|
||||
"canonical_cbor_hex": "a2616478352323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323616b582101d75a980182b10ab7d54bfed3c964073a0ee172f3daa62325af021a68f707511a",
|
||||
"blake3_hex": "c9357de555002f5bd3c70ffaa40d0d8a99316614f6c3e8372136cdfe93c76da2",
|
||||
"expected_said": "Eze2x3zkvaaxvxu6hb75kidinrkmtczqu63b6qnzbg3g75e6hnwra",
|
||||
"said_cbor_hex": "a261647835457a653278337a6b7661617876787536686237356b6964696e726b6d74637a717536336236716e7a6267336737356536686e777261616b582101d75a980182b10ab7d54bfed3c964073a0ee172f3daa62325af021a68f707511a"
|
||||
},
|
||||
{
|
||||
"name": "said-008-spec-worked-example",
|
||||
"description": "The worked example of identity spec §2.4: {\"d\": placeholder, \"n\": 1}. Its canonical encoding is 61 bytes, and §2.4 prints the digest and SAID given here.",
|
||||
"said_field": "d",
|
||||
"input": {
|
||||
"d": "#####################################################",
|
||||
"n": 1
|
||||
},
|
||||
"canonical_cbor_hex": "a2616478352323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323616e01",
|
||||
"blake3_hex": "56813033774b7fb781e0f8547eae6ec3ca24728c8af6ae9dd771d1e0258f6d39",
|
||||
"expected_said": "Ek2atam3xjn73papa7bkh5ltoypfci4umrl3k5hoxohi6ajmpnu4q",
|
||||
"said_cbor_hex": "a261647835456b326174616d33786a6e37337061706137626b68356c746f797066636934756d726c336b35686f786f686936616a6d706e753471616e01"
|
||||
},
|
||||
{
|
||||
"name": "said-009-non-canonical-copy-rejected",
|
||||
"description": "The adversarial case of identity spec §2.5, aimed at re-encoding. These are said-008's bytes with n written 0x18 0x01 instead of 0x01, still stating said-008's SAID. A verifier that re-encodes before hashing gets said-008's bytes back and accepts, so a second byte string would verify under said-008's SAID. A conforming verifier refuses the input as non-canonical before computing anything.",
|
||||
"said_field": "d",
|
||||
"input_cbor_hex": "a261647835456b326174616d33786a6e37337061706137626b68356c746f797066636934756d726c336b35686f786f686936616a6d706e753471616e1801",
|
||||
"error": "non_shortest_integer",
|
||||
"said_of_received_bytes": "Eiq35nwixqgv23jmqme6sgpiismnq7vv42z33qodyypexl3muy4ma",
|
||||
"said_after_reencoding": "Ek2atam3xjn73papa7bkh5ltoypfci4umrl3k5hoxohi6ajmpnu4q"
|
||||
},
|
||||
{
|
||||
"name": "said-010-self-consistent-non-canonical-rejected",
|
||||
"description": "The adversarial case aimed the other way. said-008's object with its keys in reverse order, stating the SAID of exactly these bytes (placeholder substituted in place). A verifier that hashes what it received, without checking canonical form, accepts it. A conforming verifier refuses it as non-canonical.",
|
||||
"said_field": "d",
|
||||
"input_cbor_hex": "a2616e016164783545686b706b787576766a346266617272357677366e70716f3665746e347a79746d797972373462666534646c6e6f64776a68646971",
|
||||
"error": "unsorted_map_keys",
|
||||
"said_of_received_bytes": "Ehkpkxuvvj4bfarr5vw6npqo6etn4zytmyyr74bfe4dlnodwjhdiq",
|
||||
"said_after_reencoding": "Ek2atam3xjn73papa7bkh5ltoypfci4umrl3k5hoxohi6ajmpnu4q"
|
||||
},
|
||||
{
|
||||
"name": "said-011-changed-content-rejected",
|
||||
"description": "said-008's object with n changed from 1 to 2 and its SAID left as it was. The bytes are canonical and well-formed, but they state a SAID they do not hash to.",
|
||||
"said_field": "d",
|
||||
"input_cbor_hex": "a261647835456b326174616d33786a6e37337061706137626b68356c746f797066636934756d726c336b35686f786f686936616a6d706e753471616e02",
|
||||
"error": "said_mismatch"
|
||||
}
|
||||
]
|
||||
|
|
|
|||
|
|
@ -1,52 +1,79 @@
|
|||
[
|
||||
{
|
||||
"name": "sig-001-known-key-known-said",
|
||||
"description": "Positive: sign a known envelope SAID with a known Ed25519 signing key, expect a deterministic signature. Ed25519 (RFC 8032) is deterministic, so the signature is reproducible across implementations.",
|
||||
"envelope_said": "EBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB",
|
||||
"description": "Ed25519 (RFC 8032) is deterministic: signing said-002's SAID with the key of RFC 8032's first test seed gives exactly this signature. The message is the 32 digest bytes the SAID spells (signed_message_hex), not its 53-character text.",
|
||||
"envelope_said": "Etbla5c4naxsrexaflxzrhm7frpkbdcegitkgholf6ccmt6iwpoxq",
|
||||
"signed_message_hex": "98560e8b8d05e5125c055df313b3e58bd411888644d463b965f084c9f9167baf",
|
||||
"signing_key_seed": "9d61b19deffd5a60ba844af492ec2cc44449c5697b326919703bac031cae7f60",
|
||||
"verifying_key": "d75a980182b10ab7d54bfed3c964073a0ee172f3daa62325af021a68f707511a",
|
||||
"signature": "e5564300c360ac729086e2cc806e828a84877f1eb8e5d974d873e065224901555fb8821590a33bacc61e39701cf9b46bd25bf5f0595bbe24655141438e7a100b",
|
||||
"verifying_key": "d75a980182b10ab7d54bfed3c964073a0ee172f3daa62325af021a68f707511a",
|
||||
"signature": "ceadf887b7391770a3c4a4a4cf2062f2e41db1447851cf2a843bf7b7718ae7ec8a534846a1bf46fbdedf3f9ee4a5832c9406cf33fd598fe6d520f98946e09b04",
|
||||
"expected_verify": true,
|
||||
"error": null
|
||||
},
|
||||
{
|
||||
"name": "sig-002-verify-positive",
|
||||
"description": "Positive: verify a signature against the envelope SAID and verifying key it was produced for. Result MUST be true.",
|
||||
"envelope_said": "EAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
|
||||
"description": "A signature over said-001's SAID by the key of RFC 8032's second test seed. It verifies under that key.",
|
||||
"envelope_said": "E64kxoz2b4kukifl5lbhnndcfos2bqxdjhp5wgx7zh35erpsv243a",
|
||||
"signed_message_hex": "f715776741e2a8a4157d584ed68c4574b4185c693bfb635ff93efa48be55d736",
|
||||
"signing_key_seed": "4ccd089b28ff96da9db6c346ec114e0f5b8a319f35aba624da8cf6ed4fb8a6fb",
|
||||
"verifying_key": "3d4017c3e843895a92b70aa74d1b7ebc9c982ccf2ec4968cc0cd55f12af4660c",
|
||||
"signature": "92a009a9f0d4cab8720e820b5f642540a2b27b5416503f8fb3762223ebdb69da085ac1e43e15996e458f3613d0f11d8c387b2eaeb4302aeeb00d291612bb0c00",
|
||||
"verifying_key": "3d4017c3e843895a92b70aa74d1b7ebc9c982ccf2ec4968cc0cd55f12af4660c",
|
||||
"signature": "ce9b9e7185967276bebe6a9c12f3570d821bcb7184e9c703bd278c1fa44ece65c0fb5b0855fe3fca5eb836d3f41b568a230af3156da0440cfaf5d4acd7123506",
|
||||
"expected_verify": true,
|
||||
"error": null
|
||||
},
|
||||
{
|
||||
"name": "sig-003-tampered-envelope",
|
||||
"description": "Negative: verify a signature against a *different* envelope SAID than the one it was produced for. The signature is structurally valid but binds a different SAID; verification MUST return false. This catches the most common attack: an adversary swaps the envelope and forwards the original signature.",
|
||||
"envelope_said": "EFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF",
|
||||
"description": "sig-002's signature presented with said-003's SAID: an adversary swaps the object and forwards the original signature. The signature is well-formed but binds another SAID, so verification MUST fail.",
|
||||
"envelope_said": "Egcy5lkwyp44kumaucno6ocsjzv44tx2cf7dc52z667muxq7ms4hq",
|
||||
"signed_message_hex": "30b1d5aad87f38aa3014135de70a49cd79c9df422fc62eeb3ef7d94bc3ec970f",
|
||||
"signing_key_seed": "4ccd089b28ff96da9db6c346ec114e0f5b8a319f35aba624da8cf6ed4fb8a6fb",
|
||||
"verifying_key": "3d4017c3e843895a92b70aa74d1b7ebc9c982ccf2ec4968cc0cd55f12af4660c",
|
||||
"signature": "92a009a9f0d4cab8720e820b5f642540a2b27b5416503f8fb3762223ebdb69da085ac1e43e15996e458f3613d0f11d8c387b2eaeb4302aeeb00d291612bb0c00",
|
||||
"verifying_key": "3d4017c3e843895a92b70aa74d1b7ebc9c982ccf2ec4968cc0cd55f12af4660c",
|
||||
"signature": "ce9b9e7185967276bebe6a9c12f3570d821bcb7184e9c703bd278c1fa44ece65c0fb5b0855fe3fca5eb836d3f41b568a230af3156da0440cfaf5d4acd7123506",
|
||||
"expected_verify": false,
|
||||
"error": "signature_does_not_bind_said"
|
||||
"error": "signature_invalid"
|
||||
},
|
||||
{
|
||||
"name": "sig-004-wrong-key",
|
||||
"description": "Negative: verify a signature against the SAID it was produced for, but with a *different* verifying key. Verification MUST return false. This catches the case where an adversary supplies a valid signature from a key that is not the byte's current owner.",
|
||||
"envelope_said": "EAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
|
||||
"description": "sig-002's signature over said-001's SAID, presented with another key — the one the all-zero seed derives. The signature is valid for the key that made it, which is not this one, so verification MUST fail.",
|
||||
"envelope_said": "E64kxoz2b4kukifl5lbhnndcfos2bqxdjhp5wgx7zh35erpsv243a",
|
||||
"signed_message_hex": "f715776741e2a8a4157d584ed68c4574b4185c693bfb635ff93efa48be55d736",
|
||||
"signing_key_seed": "0000000000000000000000000000000000000000000000000000000000000000",
|
||||
"verifying_key": "1111111111111111111111111111111111111111111111111111111111111111",
|
||||
"signature": "92a009a9f0d4cab8720e820b5f642540a2b27b5416503f8fb3762223ebdb69da085ac1e43e15996e458f3613d0f11d8c387b2eaeb4302aeeb00d291612bb0c00",
|
||||
"verifying_key": "3b6a27bcceb6a42d62a3a8d02a6f0d73653215771de243a63ac048a18b59da29",
|
||||
"signature": "ce9b9e7185967276bebe6a9c12f3570d821bcb7184e9c703bd278c1fa44ece65c0fb5b0855fe3fca5eb836d3f41b568a230af3156da0440cfaf5d4acd7123506",
|
||||
"expected_verify": false,
|
||||
"error": "verifying_key_does_not_match"
|
||||
"error": "signature_invalid"
|
||||
},
|
||||
{
|
||||
"name": "sig-005-malformed-signature",
|
||||
"description": "Negative: verify a structurally malformed signature (wrong length, in this case 63 bytes instead of 64). Verification MUST return false and the implementation MUST NOT panic or leak timing information distinguishing this case from the cryptographic-failure case.",
|
||||
"envelope_said": "EAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
|
||||
"description": "sig-002's signature cut to 63 bytes. It is refused as malformed before any cryptography runs, and an implementation MUST NOT panic on it.",
|
||||
"envelope_said": "E64kxoz2b4kukifl5lbhnndcfos2bqxdjhp5wgx7zh35erpsv243a",
|
||||
"signed_message_hex": "f715776741e2a8a4157d584ed68c4574b4185c693bfb635ff93efa48be55d736",
|
||||
"signing_key_seed": "4ccd089b28ff96da9db6c346ec114e0f5b8a319f35aba624da8cf6ed4fb8a6fb",
|
||||
"verifying_key": "3d4017c3e843895a92b70aa74d1b7ebc9c982ccf2ec4968cc0cd55f12af4660c",
|
||||
"signature": "92a009a9f0d4cab8720e820b5f642540a2b27b5416503f8fb3762223ebdb69da085ac1e43e15996e458f3613d0f11d8c387b2eaeb4302aeeb00d291612bb",
|
||||
"verifying_key": "3d4017c3e843895a92b70aa74d1b7ebc9c982ccf2ec4968cc0cd55f12af4660c",
|
||||
"signature": "ce9b9e7185967276bebe6a9c12f3570d821bcb7184e9c703bd278c1fa44ece65c0fb5b0855fe3fca5eb836d3f41b568a230af3156da0440cfaf5d4acd71235",
|
||||
"expected_verify": false,
|
||||
"error": "malformed_signature_wrong_length"
|
||||
"error": "malformed_signature"
|
||||
},
|
||||
{
|
||||
"name": "sig-006-non-canonical-s-rejected",
|
||||
"description": "sig-002's signature with its scalar S replaced by S + L: the same scalar, written outside [0, L). RFC 8032 §5.1.7 requires refusing it; accepting it would make signatures malleable, giving a second valid-looking signature for one message and key.",
|
||||
"envelope_said": "E64kxoz2b4kukifl5lbhnndcfos2bqxdjhp5wgx7zh35erpsv243a",
|
||||
"signed_message_hex": "f715776741e2a8a4157d584ed68c4574b4185c693bfb635ff93efa48be55d736",
|
||||
"signing_key_seed": "4ccd089b28ff96da9db6c346ec114e0f5b8a319f35aba624da8cf6ed4fb8a6fb",
|
||||
"verifying_key": "3d4017c3e843895a92b70aa74d1b7ebc9c982ccf2ec4968cc0cd55f12af4660c",
|
||||
"signature": "ce9b9e7185967276bebe6a9c12f3570d821bcb7184e9c703bd278c1fa44ece65adcf51656f61522235552e76d315359f230af3156da0440cfaf5d4acd7123516",
|
||||
"expected_verify": false,
|
||||
"error": "signature_invalid"
|
||||
},
|
||||
{
|
||||
"name": "sig-007-non-canonical-key-rejected",
|
||||
"description": "A verifying key spelled with y = p + 3: an out-of-range encoding of the valid, large-order point with y = 3. RFC 8032 §5.1.3 requires decoding to fail when y is not below p, so the key is refused as malformed before any verification runs — one key has one spelling. There is no seed: no key pair has this spelling.",
|
||||
"envelope_said": "E64kxoz2b4kukifl5lbhnndcfos2bqxdjhp5wgx7zh35erpsv243a",
|
||||
"signed_message_hex": "f715776741e2a8a4157d584ed68c4574b4185c693bfb635ff93efa48be55d736",
|
||||
"signing_key_seed": null,
|
||||
"verifying_key": "f0ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7f",
|
||||
"signature": "ce9b9e7185967276bebe6a9c12f3570d821bcb7184e9c703bd278c1fa44ece65c0fb5b0855fe3fca5eb836d3f41b568a230af3156da0440cfaf5d4acd7123506",
|
||||
"expected_verify": false,
|
||||
"error": "malformed_verifying_key"
|
||||
}
|
||||
]
|
||||
|
|
|
|||
11
smart-byte/smart-byte-rs/Cargo.lock
generated
11
smart-byte/smart-byte-rs/Cargo.lock
generated
|
|
@ -5942,6 +5942,17 @@ dependencies = [
|
|||
"zeroize",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "smart-byte-conformance"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"hex",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"smart-byte-core",
|
||||
"smart-byte-lockstep",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "smart-byte-core"
|
||||
version = "0.1.0"
|
||||
|
|
|
|||
|
|
@ -23,6 +23,7 @@ members = [
|
|||
"crates/smart-byte-tendermint",
|
||||
"crates/smart-byte-acdc",
|
||||
"crates/smart-byte-oidc4vc",
|
||||
"crates/smart-byte-conformance",
|
||||
]
|
||||
|
||||
[workspace.package]
|
||||
|
|
|
|||
|
|
@ -0,0 +1,24 @@
|
|||
[package]
|
||||
name = "smart-byte-conformance"
|
||||
version.workspace = true
|
||||
edition.workspace = true
|
||||
rust-version.workspace = true
|
||||
license.workspace = true
|
||||
repository.workspace = true
|
||||
description = "Generates and verifies the Smart Byte conformance vector pack (smart-byte/conformance/v1)."
|
||||
publish = false
|
||||
|
||||
[dependencies]
|
||||
smart-byte-core = { workspace = true }
|
||||
smart-byte-lockstep = { workspace = true }
|
||||
serde = { workspace = true }
|
||||
serde_json = "1"
|
||||
hex = "0.4"
|
||||
|
||||
[[bin]]
|
||||
name = "gen-conformance-vectors"
|
||||
path = "src/bin/gen-conformance-vectors.rs"
|
||||
|
||||
[[bin]]
|
||||
name = "verify-conformance-vectors"
|
||||
path = "src/bin/verify-conformance-vectors.rs"
|
||||
|
|
@ -0,0 +1,67 @@
|
|||
//! Write the v1 conformance pack.
|
||||
//!
|
||||
//! ```text
|
||||
//! gen-conformance-vectors [--spec-version v1] [--output-dir DIR] [--check]
|
||||
//! ```
|
||||
//!
|
||||
//! `--output-dir` defaults to the published pack, `smart-byte/conformance/v1`.
|
||||
//! `--check` writes nothing and exits non-zero if any file in the directory
|
||||
//! differs from what the reference implementation generates.
|
||||
|
||||
use std::path::PathBuf;
|
||||
use std::process::ExitCode;
|
||||
|
||||
fn main() -> ExitCode {
|
||||
let mut out = smart_byte_conformance::published_pack_dir();
|
||||
let mut check = false;
|
||||
let mut args = std::env::args().skip(1);
|
||||
while let Some(arg) = args.next() {
|
||||
match arg.as_str() {
|
||||
"--spec-version" => match args.next().as_deref() {
|
||||
Some("v1") => {}
|
||||
other => {
|
||||
eprintln!("only spec version v1 has a pack; got {other:?}");
|
||||
return ExitCode::from(2);
|
||||
}
|
||||
},
|
||||
"--output-dir" => match args.next() {
|
||||
Some(dir) => out = PathBuf::from(dir),
|
||||
None => {
|
||||
eprintln!("--output-dir needs a directory");
|
||||
return ExitCode::from(2);
|
||||
}
|
||||
},
|
||||
"--check" => check = true,
|
||||
other => {
|
||||
eprintln!("unknown argument {other:?}");
|
||||
eprintln!(
|
||||
"usage: gen-conformance-vectors [--spec-version v1] [--output-dir DIR] [--check]"
|
||||
);
|
||||
return ExitCode::from(2);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let mut stale = 0;
|
||||
for (name, contents) in smart_byte_conformance::pack::generate() {
|
||||
let path = out.join(name);
|
||||
if check {
|
||||
if std::fs::read_to_string(&path).ok().as_deref() == Some(contents.as_str()) {
|
||||
println!("ok {}", path.display());
|
||||
} else {
|
||||
println!("STALE {}", path.display());
|
||||
stale += 1;
|
||||
}
|
||||
} else if let Err(e) = std::fs::write(&path, contents) {
|
||||
eprintln!("write {}: {e}", path.display());
|
||||
return ExitCode::FAILURE;
|
||||
} else {
|
||||
println!("wrote {}", path.display());
|
||||
}
|
||||
}
|
||||
if stale > 0 {
|
||||
eprintln!("{stale} file(s) differ from the reference implementation's output");
|
||||
return ExitCode::FAILURE;
|
||||
}
|
||||
ExitCode::SUCCESS
|
||||
}
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
//! Check a conformance pack on disk against the reference implementation.
|
||||
//!
|
||||
//! ```text
|
||||
//! verify-conformance-vectors [--pack DIR]
|
||||
//! ```
|
||||
//!
|
||||
//! `--pack` defaults to the published pack, `smart-byte/conformance/v1`.
|
||||
//! Exits non-zero if any vector fails.
|
||||
|
||||
use std::path::PathBuf;
|
||||
use std::process::ExitCode;
|
||||
|
||||
fn main() -> ExitCode {
|
||||
let mut dir = smart_byte_conformance::published_pack_dir();
|
||||
let mut args = std::env::args().skip(1);
|
||||
while let Some(arg) = args.next() {
|
||||
match (arg.as_str(), args.next()) {
|
||||
("--pack", Some(d)) => dir = PathBuf::from(d),
|
||||
_ => {
|
||||
eprintln!("usage: verify-conformance-vectors [--pack DIR]");
|
||||
return ExitCode::from(2);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let report = smart_byte_conformance::verify::verify_pack(&dir);
|
||||
for c in &report.checks {
|
||||
match &c.result {
|
||||
Ok(()) => println!("ok {:<32} {}", c.file, c.vector),
|
||||
Err(e) => println!("FAIL {:<32} {}: {e}", c.file, c.vector),
|
||||
}
|
||||
}
|
||||
let failed = report.failures().len();
|
||||
println!("{} checks, {failed} failed", report.checks.len());
|
||||
if report.passed() {
|
||||
ExitCode::SUCCESS
|
||||
} else {
|
||||
ExitCode::FAILURE
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,334 @@
|
|||
//! The pack's JSON: an order-preserving value type with a deterministic
|
||||
//! printer, and the typed mapping between JSON and the CBOR data model.
|
||||
//!
|
||||
//! JSON has no byte-string type, so the pack writes a byte string as an object
|
||||
//! whose only member is `$bytes`, holding lowercase hex. `$bytes` is reserved:
|
||||
//! no map in the pack has a member of that name. Every other JSON value maps
|
||||
//! to CBOR the obvious way: object → map with text keys, array → array,
|
||||
//! string → text, integer → integer, `true`/`false`/`null` → simple values.
|
||||
//! A JSON number with a fraction or exponent is refused: identity spec §2.3
|
||||
//! forbids floats.
|
||||
//!
|
||||
//! [`parse_strict`] reads the pack's JSON refusing duplicate object members,
|
||||
//! which a plain JSON parser would silently collapse.
|
||||
|
||||
use smart_byte_core::canonical::Value;
|
||||
|
||||
/// A JSON value whose objects keep the order they were built in, so the
|
||||
/// generator's output is byte-for-byte reproducible.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub enum J {
|
||||
Null,
|
||||
Bool(bool),
|
||||
Int(i128),
|
||||
Str(String),
|
||||
Arr(Vec<J>),
|
||||
Obj(Vec<(String, J)>),
|
||||
}
|
||||
|
||||
impl J {
|
||||
pub fn str(s: impl Into<String>) -> J {
|
||||
J::Str(s.into())
|
||||
}
|
||||
|
||||
pub fn obj<K: Into<String>>(entries: impl IntoIterator<Item = (K, J)>) -> J {
|
||||
J::Obj(entries.into_iter().map(|(k, v)| (k.into(), v)).collect())
|
||||
}
|
||||
|
||||
pub fn strs<S: Into<String>>(items: impl IntoIterator<Item = S>) -> J {
|
||||
J::Arr(items.into_iter().map(|s| J::Str(s.into())).collect())
|
||||
}
|
||||
}
|
||||
|
||||
impl From<&str> for J {
|
||||
fn from(s: &str) -> J {
|
||||
J::Str(s.to_owned())
|
||||
}
|
||||
}
|
||||
|
||||
impl From<String> for J {
|
||||
fn from(s: String) -> J {
|
||||
J::Str(s)
|
||||
}
|
||||
}
|
||||
|
||||
impl From<bool> for J {
|
||||
fn from(b: bool) -> J {
|
||||
J::Bool(b)
|
||||
}
|
||||
}
|
||||
|
||||
impl From<u64> for J {
|
||||
fn from(n: u64) -> J {
|
||||
J::Int(i128::from(n))
|
||||
}
|
||||
}
|
||||
|
||||
impl From<usize> for J {
|
||||
fn from(n: usize) -> J {
|
||||
J::Int(n as i128)
|
||||
}
|
||||
}
|
||||
|
||||
/// Print `j` with two-space indentation and a trailing newline. Non-ASCII text
|
||||
/// is written as UTF-8, not escaped.
|
||||
pub fn pretty(j: &J) -> String {
|
||||
let mut out = String::new();
|
||||
write(j, 0, &mut out);
|
||||
out.push('\n');
|
||||
out
|
||||
}
|
||||
|
||||
fn write(j: &J, indent: usize, out: &mut String) {
|
||||
match j {
|
||||
J::Null => out.push_str("null"),
|
||||
J::Bool(b) => out.push_str(if *b { "true" } else { "false" }),
|
||||
J::Int(n) => out.push_str(&n.to_string()),
|
||||
J::Str(s) => write_str(s, out),
|
||||
J::Arr(items) if items.is_empty() => out.push_str("[]"),
|
||||
J::Obj(entries) if entries.is_empty() => out.push_str("{}"),
|
||||
J::Arr(items) => {
|
||||
out.push_str("[\n");
|
||||
for (i, item) in items.iter().enumerate() {
|
||||
pad(indent + 2, out);
|
||||
write(item, indent + 2, out);
|
||||
out.push_str(if i + 1 < items.len() { ",\n" } else { "\n" });
|
||||
}
|
||||
pad(indent, out);
|
||||
out.push(']');
|
||||
}
|
||||
J::Obj(entries) => {
|
||||
out.push_str("{\n");
|
||||
for (i, (k, v)) in entries.iter().enumerate() {
|
||||
pad(indent + 2, out);
|
||||
write_str(k, out);
|
||||
out.push_str(": ");
|
||||
write(v, indent + 2, out);
|
||||
out.push_str(if i + 1 < entries.len() { ",\n" } else { "\n" });
|
||||
}
|
||||
pad(indent, out);
|
||||
out.push('}');
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn pad(n: usize, out: &mut String) {
|
||||
out.extend(std::iter::repeat_n(' ', n));
|
||||
}
|
||||
|
||||
fn write_str(s: &str, out: &mut String) {
|
||||
out.push('"');
|
||||
for c in s.chars() {
|
||||
match c {
|
||||
'"' => out.push_str("\\\""),
|
||||
'\\' => out.push_str("\\\\"),
|
||||
'\n' => out.push_str("\\n"),
|
||||
'\r' => out.push_str("\\r"),
|
||||
'\t' => out.push_str("\\t"),
|
||||
c if (c as u32) < 0x20 => out.push_str(&format!("\\u{:04x}", c as u32)),
|
||||
c => out.push(c),
|
||||
}
|
||||
}
|
||||
out.push('"');
|
||||
}
|
||||
|
||||
/// A byte string in the pack's typed JSON.
|
||||
pub fn bytes_json(b: &[u8]) -> J {
|
||||
J::obj([("$bytes", J::Str(hex::encode(b)))])
|
||||
}
|
||||
|
||||
/// The pack's typed JSON for a CBOR value, keeping map entries in the order
|
||||
/// the value holds them.
|
||||
///
|
||||
/// # Panics
|
||||
/// On a map key that is not text, or a text key spelled `$bytes`: the typed
|
||||
/// JSON has no spelling for either, and the pack's generator never builds one.
|
||||
pub fn to_json(v: &Value) -> J {
|
||||
match v {
|
||||
Value::Unsigned(n) => J::Int(i128::from(*n)),
|
||||
Value::Negative(n) => J::Int(-1 - i128::from(*n)),
|
||||
Value::Bytes(b) => bytes_json(b),
|
||||
Value::Text(t) => J::Str(t.clone()),
|
||||
Value::Array(items) => J::Arr(items.iter().map(to_json).collect()),
|
||||
Value::Map(entries) => J::Obj(
|
||||
entries
|
||||
.iter()
|
||||
.map(|(k, v)| match k {
|
||||
Value::Text(t) if t == "$bytes" => panic!("`$bytes` is a reserved key"),
|
||||
Value::Text(t) => (t.clone(), to_json(v)),
|
||||
other => panic!("typed JSON has text keys only, not {other:?}"),
|
||||
})
|
||||
.collect(),
|
||||
),
|
||||
Value::Bool(b) => J::Bool(*b),
|
||||
Value::Null => J::Null,
|
||||
}
|
||||
}
|
||||
|
||||
/// Parse JSON, refusing any object that names a member twice.
|
||||
pub fn parse_strict(bytes: &[u8]) -> Result<serde_json::Value, String> {
|
||||
use serde::de::DeserializeSeed;
|
||||
let mut de = serde_json::Deserializer::from_slice(bytes);
|
||||
let value = Strict.deserialize(&mut de).map_err(|e| e.to_string())?;
|
||||
de.end().map_err(|e| e.to_string())?;
|
||||
Ok(value)
|
||||
}
|
||||
|
||||
struct Strict;
|
||||
|
||||
impl<'de> serde::de::DeserializeSeed<'de> for Strict {
|
||||
type Value = serde_json::Value;
|
||||
|
||||
fn deserialize<D: serde::Deserializer<'de>>(self, d: D) -> Result<Self::Value, D::Error> {
|
||||
d.deserialize_any(Strict)
|
||||
}
|
||||
}
|
||||
|
||||
impl<'de> serde::de::Visitor<'de> for Strict {
|
||||
type Value = serde_json::Value;
|
||||
|
||||
fn expecting(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
|
||||
f.write_str("JSON with no duplicate object members")
|
||||
}
|
||||
|
||||
fn visit_bool<E>(self, b: bool) -> Result<Self::Value, E> {
|
||||
Ok(b.into())
|
||||
}
|
||||
|
||||
fn visit_i64<E>(self, n: i64) -> Result<Self::Value, E> {
|
||||
Ok(n.into())
|
||||
}
|
||||
|
||||
fn visit_u64<E>(self, n: u64) -> Result<Self::Value, E> {
|
||||
Ok(n.into())
|
||||
}
|
||||
|
||||
fn visit_f64<E>(self, n: f64) -> Result<Self::Value, E> {
|
||||
Ok(n.into())
|
||||
}
|
||||
|
||||
fn visit_str<E>(self, s: &str) -> Result<Self::Value, E> {
|
||||
Ok(s.into())
|
||||
}
|
||||
|
||||
fn visit_unit<E>(self) -> Result<Self::Value, E> {
|
||||
Ok(serde_json::Value::Null)
|
||||
}
|
||||
|
||||
fn visit_seq<A: serde::de::SeqAccess<'de>>(self, mut seq: A) -> Result<Self::Value, A::Error> {
|
||||
let mut items = Vec::new();
|
||||
while let Some(item) = seq.next_element_seed(Strict)? {
|
||||
items.push(item);
|
||||
}
|
||||
Ok(serde_json::Value::Array(items))
|
||||
}
|
||||
|
||||
fn visit_map<A: serde::de::MapAccess<'de>>(self, mut map: A) -> Result<Self::Value, A::Error> {
|
||||
let mut out = serde_json::Map::new();
|
||||
while let Some(key) = map.next_key::<String>()? {
|
||||
if out.contains_key(&key) {
|
||||
return Err(serde::de::Error::custom(format!(
|
||||
"duplicate member {key:?}"
|
||||
)));
|
||||
}
|
||||
let value = map.next_value_seed(Strict)?;
|
||||
out.insert(key, value);
|
||||
}
|
||||
Ok(serde_json::Value::Object(out))
|
||||
}
|
||||
}
|
||||
|
||||
/// The CBOR value a typed-JSON value denotes.
|
||||
pub fn from_json(j: &serde_json::Value) -> Result<Value, String> {
|
||||
use serde_json::Value as S;
|
||||
Ok(match j {
|
||||
S::Null => Value::Null,
|
||||
S::Bool(b) => Value::Bool(*b),
|
||||
S::Number(n) => {
|
||||
if let Some(u) = n.as_u64() {
|
||||
Value::Unsigned(u)
|
||||
} else if let Some(i) = n.as_i64() {
|
||||
Value::int(i)
|
||||
} else {
|
||||
return Err(format!("{n} is not an integer; floats are forbidden"));
|
||||
}
|
||||
}
|
||||
S::String(s) => Value::Text(s.clone()),
|
||||
S::Array(items) => Value::Array(items.iter().map(from_json).collect::<Result<_, _>>()?),
|
||||
S::Object(map) => {
|
||||
if let Some(h) = map.get("$bytes") {
|
||||
let (1, S::String(h)) = (map.len(), h) else {
|
||||
return Err("`$bytes` is reserved: a lone member holding hex".into());
|
||||
};
|
||||
if h.len() % 2 != 0 || h.bytes().any(|c| !matches!(c, b'0'..=b'9' | b'a'..=b'f')) {
|
||||
return Err(format!(
|
||||
"$bytes must be lowercase hex of even length: {h:?}"
|
||||
));
|
||||
}
|
||||
return Ok(Value::Bytes(hex::decode(h).map_err(|e| e.to_string())?));
|
||||
}
|
||||
Value::Map(
|
||||
map.iter()
|
||||
.map(|(k, v)| Ok((Value::Text(k.clone()), from_json(v)?)))
|
||||
.collect::<Result<_, String>>()?,
|
||||
)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn typed_json_round_trips_the_data_model() {
|
||||
let v = Value::map([
|
||||
("z", Value::int(-5)),
|
||||
("bytes", Value::Bytes(vec![0, 0xff])),
|
||||
(
|
||||
"list",
|
||||
Value::Array(vec![Value::Null, Value::Bool(false), Value::text("t")]),
|
||||
),
|
||||
("empty", Value::Bytes(vec![])),
|
||||
("big", Value::Unsigned(u64::MAX)),
|
||||
]);
|
||||
let printed = pretty(&to_json(&v));
|
||||
let parsed: serde_json::Value = serde_json::from_str(&printed).unwrap();
|
||||
let back = from_json(&parsed).unwrap();
|
||||
// serde_json reorders object members, so compare canonical encodings.
|
||||
assert_eq!(
|
||||
smart_byte_core::canonical::encode(&back).unwrap(),
|
||||
smart_byte_core::canonical::encode(&v).unwrap()
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn floats_loose_hex_and_misused_bytes_keys_are_refused() {
|
||||
assert!(from_json(&serde_json::json!(1.5)).is_err());
|
||||
assert!(from_json(&serde_json::json!({"$bytes": "ABCD"})).is_err());
|
||||
assert!(from_json(&serde_json::json!({"$bytes": "abc"})).is_err());
|
||||
// `$bytes` is reserved: never a map member alongside others, never
|
||||
// anything but hex.
|
||||
assert!(from_json(&serde_json::json!({"$bytes": "00", "x": 1})).is_err());
|
||||
assert!(from_json(&serde_json::json!({"$bytes": 7})).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn duplicate_members_are_refused() {
|
||||
assert!(parse_strict(br#"{"a": 1, "b": {"c": 2}}"#).is_ok());
|
||||
assert!(parse_strict(br#"{"a": 1, "a": 2}"#).is_err());
|
||||
assert!(parse_strict(br#"[{"x": {"y": 1, "y": 1}}]"#).is_err());
|
||||
assert!(parse_strict(br#"{"a": 1} 2"#).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_printer_escapes_only_what_json_requires() {
|
||||
let j = J::obj([("k", J::str("a\"b\\c\nd — é\u{1}"))]);
|
||||
assert_eq!(
|
||||
pretty(&j),
|
||||
"{\n \"k\": \"a\\\"b\\\\c\\nd — é\\u0001\"\n}\n"
|
||||
);
|
||||
let parsed: serde_json::Value = serde_json::from_str(&pretty(&j)).unwrap();
|
||||
assert_eq!(parsed["k"], "a\"b\\c\nd — é\u{1}");
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,19 @@
|
|||
//! The Smart Byte conformance vector pack (`smart-byte/conformance/v1`):
|
||||
//! generated from the reference implementation by [`pack::generate`], and
|
||||
//! checked against it by [`verify::verify_pack`].
|
||||
//!
|
||||
//! * `gen-conformance-vectors` writes the pack.
|
||||
//! * `verify-conformance-vectors` checks a pack on disk, whoever wrote it.
|
||||
//! * `tests/pack.rs` runs both against the published pack under `cargo test`,
|
||||
//! so CI fails if the pack and the implementation ever disagree.
|
||||
|
||||
pub mod json;
|
||||
pub mod pack;
|
||||
pub mod verify;
|
||||
|
||||
use std::path::PathBuf;
|
||||
|
||||
/// The published v1 pack, beside the spec in this repository.
|
||||
pub fn published_pack_dir() -> PathBuf {
|
||||
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../../conformance/v1")
|
||||
}
|
||||
|
|
@ -0,0 +1,983 @@
|
|||
//! The v1 vector pack, built from the reference implementation.
|
||||
//!
|
||||
//! Inputs, names and descriptions are written here; every expected value is
|
||||
//! computed by `smart-byte-core` (canonical CBOR, SAIDs, signatures, the
|
||||
//! inbox) and `smart-byte-lockstep` (the commit rule) at generation time.
|
||||
//! A negative vector also states the error class it is meant to produce, and
|
||||
//! generation fails if the reference refuses it for any other reason — so a
|
||||
//! vector cannot pass by accident while testing nothing.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use smart_byte_core::Said;
|
||||
use smart_byte_core::canonical::{self, Value};
|
||||
use smart_byte_core::inbox::{Delivery, Inbox};
|
||||
use smart_byte_core::said::{self, SAID_FIELD, SAID_PLACEHOLDER, SaidDerivation};
|
||||
use smart_byte_core::sign::{self, SigningKey};
|
||||
use smart_byte_lockstep::{node_view, supermajority_threshold, tally};
|
||||
|
||||
use crate::json::{J, pretty, to_json};
|
||||
|
||||
/// The pack's vector files, in the order the README lists them.
|
||||
pub const FILES: [&str; 5] = [
|
||||
"canonicalisation_vectors.json",
|
||||
"said_vectors.json",
|
||||
"signature_vectors.json",
|
||||
"lockstep_vectors.json",
|
||||
"gossip_vectors.json",
|
||||
];
|
||||
|
||||
/// The seed of RFC 8032 §7.1 test 1.
|
||||
pub const RFC8032_TEST1_SEED: &str =
|
||||
"9d61b19deffd5a60ba844af492ec2cc44449c5697b326919703bac031cae7f60";
|
||||
/// The seed of RFC 8032 §7.1 test 2.
|
||||
pub const RFC8032_TEST2_SEED: &str =
|
||||
"4ccd089b28ff96da9db6c346ec114e0f5b8a319f35aba624da8cf6ed4fb8a6fb";
|
||||
|
||||
/// Every file of the pack: (file name, contents).
|
||||
pub fn generate() -> Vec<(&'static str, String)> {
|
||||
let saids = SaidVectors::build();
|
||||
vec![
|
||||
(FILES[0], pretty(&canonicalisation())),
|
||||
(FILES[1], pretty(&saids.json)),
|
||||
(FILES[2], pretty(&signatures(&saids))),
|
||||
(FILES[3], pretty(&lockstep())),
|
||||
(FILES[4], pretty(&gossip(&saids))),
|
||||
]
|
||||
}
|
||||
|
||||
fn cbor(v: &Value) -> Vec<u8> {
|
||||
canonical::encode(v).expect("every value the pack builds has a canonical form")
|
||||
}
|
||||
|
||||
fn text(s: &str) -> Value {
|
||||
Value::text(s)
|
||||
}
|
||||
|
||||
fn uint(n: u64) -> Value {
|
||||
Value::Unsigned(n)
|
||||
}
|
||||
|
||||
fn seed(hex_seed: &str) -> [u8; 32] {
|
||||
hex::decode(hex_seed).unwrap().try_into().unwrap()
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// canonicalisation_vectors.json
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
fn canon_positive(name: &str, description: &str, input: Value) -> J {
|
||||
let bytes = cbor(&input);
|
||||
// The decoder must accept what the encoder wrote, unchanged.
|
||||
assert_eq!(cbor(&canonical::decode(&bytes).unwrap()), bytes, "{name}");
|
||||
J::obj([
|
||||
("name", J::from(name)),
|
||||
("description", J::from(description)),
|
||||
("input", to_json(&input)),
|
||||
("expected_cbor_hex", J::Str(hex::encode(bytes))),
|
||||
])
|
||||
}
|
||||
|
||||
fn canon_negative(name: &str, description: &str, input: &[u8], class: &str) -> J {
|
||||
let refused = canonical::decode(input).expect_err(name);
|
||||
assert_eq!(
|
||||
refused.class(),
|
||||
class,
|
||||
"{name} is refused for the wrong reason"
|
||||
);
|
||||
J::obj([
|
||||
("name", J::from(name)),
|
||||
("description", J::from(description)),
|
||||
("input_cbor_hex", J::Str(hex::encode(input))),
|
||||
("error", J::from(class)),
|
||||
])
|
||||
}
|
||||
|
||||
fn concat(parts: &[&[u8]]) -> Vec<u8> {
|
||||
parts.concat()
|
||||
}
|
||||
|
||||
fn canonicalisation() -> J {
|
||||
let payload = Value::Bytes((0u8..8).collect());
|
||||
J::Arr(vec![
|
||||
canon_positive(
|
||||
"canon-001-sorted-map-keys",
|
||||
"Map keys are written in the bytewise order of their canonical encodings (identity spec §2.3). The input lists the keys out of order; the canonical encoding sorts them.",
|
||||
Value::map([("z", uint(1)), ("a", uint(2)), ("m", uint(3))]),
|
||||
),
|
||||
canon_positive(
|
||||
"canon-002-shortest-form-integers",
|
||||
"Integers use the shortest head that holds them: no leading zeros, no over-wide argument. 0, 23, 24, 255, 256, 65535 and 65536 sit on the boundaries of CBOR's argument widths.",
|
||||
Value::map([
|
||||
("i0", uint(0)),
|
||||
("i23", uint(23)),
|
||||
("i24", uint(24)),
|
||||
("i255", uint(255)),
|
||||
("i256", uint(256)),
|
||||
("i65535", uint(65535)),
|
||||
("i65536", uint(65536)),
|
||||
]),
|
||||
),
|
||||
canon_positive(
|
||||
"canon-003-no-tags",
|
||||
"A byte string is written untagged, as major type 2. Wrapping bytes in tag 21 (\"expected base64url conversion\") is tempting, but canonical encoding forbids tags (identity spec §2.3). canon-015 is the tagged form, which a decoder refuses rather than silently stripping the tag.",
|
||||
Value::map([("payload", payload.clone())]),
|
||||
),
|
||||
canon_negative(
|
||||
"canon-004-floats-forbidden",
|
||||
"Floats are forbidden (identity spec §2.3): a value that would be a float elsewhere is carried as a fixed-point integer pair. The input is a map holding 1.5 as a half-precision float, and a decoder refuses it.",
|
||||
&concat(&[&[0xa1], &cbor(&text("exact_half")), &[0xf9, 0x3e, 0x00]]),
|
||||
"float_forbidden",
|
||||
),
|
||||
canon_positive(
|
||||
"canon-005-bytes-not-text-for-byte-arrays",
|
||||
"Byte values are byte strings (major type 2) — never text, and never an array of small integers, which is what a generic serializer writes for a fixed-size byte array by default. The members hold a 32-byte digest, a 32-byte public key and a 64-byte signature. A SAID's text form (identity spec §2.2) is text; the digest it spells is bytes.",
|
||||
Value::map([
|
||||
("said", Value::Bytes(vec![0x00; 32])),
|
||||
("key", Value::Bytes(vec![0x11; 32])),
|
||||
("sig", Value::Bytes(vec![0x22; 64])),
|
||||
]),
|
||||
),
|
||||
canon_positive(
|
||||
"canon-006-key-order-follows-encoded-bytes",
|
||||
"Keys sort by their encoded bytes, not their text (identity spec §2.3). As text \"aa\" comes before \"b\"; encoded, \"b\" (0x61 0x62) comes before \"aa\" (0x62 0x61 0x61), because a string's length is in its first byte. canon-012 is the text-ordered encoding, which a decoder refuses.",
|
||||
Value::map([("aa", uint(1)), ("b", uint(2))]),
|
||||
),
|
||||
canon_positive(
|
||||
"canon-007-integer-boundaries-and-negatives",
|
||||
"Negative integers (major type 1) and the 32-bit/64-bit boundary. -1 is 0x20, and -24 is the last one-byte form; 4294967295 is the largest four-byte argument and 4294967296 the smallest eight-byte one.",
|
||||
Value::Array(
|
||||
[
|
||||
-1i64, -24, -25, -256, -257, -65536, -65537, 4294967295, 4294967296,
|
||||
]
|
||||
.into_iter()
|
||||
.map(Value::int)
|
||||
.collect(),
|
||||
),
|
||||
),
|
||||
canon_positive(
|
||||
"canon-008-string-length-boundary",
|
||||
"A length uses the shortest head too. A 23-character text string carries its length in its first byte; a 24-character one needs the one-byte argument.",
|
||||
Value::Array(vec![text(&"a".repeat(23)), text(&"b".repeat(24))]),
|
||||
),
|
||||
canon_positive(
|
||||
"canon-009-empty-containers-and-simple-values",
|
||||
"An empty array, map, text string and byte string, and the simple values null, true and false — each a single byte.",
|
||||
Value::map([
|
||||
("a", Value::Array(vec![])),
|
||||
("b", Value::Map(vec![])),
|
||||
("c", text("")),
|
||||
("d", Value::Bytes(vec![])),
|
||||
("e", Value::Null),
|
||||
("f", Value::Bool(true)),
|
||||
("g", Value::Bool(false)),
|
||||
]),
|
||||
),
|
||||
canon_negative(
|
||||
"canon-010-non-shortest-integer-refused",
|
||||
"Zero written with a one-byte argument (0x18 0x00). Identity spec §2.3: \"a value of 0 is encoded as 0x00, never as 0x18 0x00\". A decoder refuses it rather than normalising it.",
|
||||
&[0x18, 0x00],
|
||||
"non_shortest_integer",
|
||||
),
|
||||
canon_negative(
|
||||
"canon-011-non-shortest-length-refused",
|
||||
"The one-character text \"a\" with its length in a one-byte argument (0x78 0x01) rather than in the head. A length has one canonical form too.",
|
||||
&[0x78, 0x01, 0x61],
|
||||
"non_shortest_length",
|
||||
),
|
||||
canon_negative(
|
||||
"canon-012-text-ordered-keys-refused",
|
||||
"canon-006's map with its keys in text order (\"aa\" before \"b\"). The keys are distinct and the map is well-formed, but the order is not the canonical one, so the encoding is refused.",
|
||||
&concat(&[
|
||||
&[0xa2],
|
||||
&cbor(&text("aa")),
|
||||
&[0x01],
|
||||
&cbor(&text("b")),
|
||||
&[0x02],
|
||||
]),
|
||||
"unsorted_map_keys",
|
||||
),
|
||||
canon_negative(
|
||||
"canon-013-duplicate-key-refused",
|
||||
"A map holding the key \"a\" twice. It has no canonical form, and which of the two values a lenient decoder keeps differs between implementations.",
|
||||
&concat(&[
|
||||
&[0xa2],
|
||||
&cbor(&text("a")),
|
||||
&[0x01],
|
||||
&cbor(&text("a")),
|
||||
&[0x02],
|
||||
]),
|
||||
"duplicate_map_key",
|
||||
),
|
||||
canon_negative(
|
||||
"canon-014-indefinite-length-refused",
|
||||
"An array written with an indefinite length (0x9f … 0xff). Identity spec §2.3 forbids indefinite-length items: the same array has a definite-length encoding.",
|
||||
&[0x9f, 0x01, 0x02, 0xff],
|
||||
"indefinite_length",
|
||||
),
|
||||
canon_negative(
|
||||
"canon-015-tagged-byte-string-refused",
|
||||
"canon-003's map with its byte string wrapped in tag 21. Tags are forbidden (identity spec §2.3): the decoder refuses the input; it does not strip the tag and accept what is left.",
|
||||
&concat(&[&[0xa1], &cbor(&text("payload")), &[0xd5], &cbor(&payload)]),
|
||||
"tag_forbidden",
|
||||
),
|
||||
canon_negative(
|
||||
"canon-016-single-precision-float-refused",
|
||||
"A single-precision float (the largest finite one). Refused, as every float is.",
|
||||
&[0xfa, 0x7f, 0x7f, 0xff, 0xff],
|
||||
"float_forbidden",
|
||||
),
|
||||
canon_negative(
|
||||
"canon-017-double-precision-float-refused",
|
||||
"A double-precision float (the largest finite one). Refused, as every float is.",
|
||||
&[0xfb, 0x7f, 0xef, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff],
|
||||
"float_forbidden",
|
||||
),
|
||||
canon_negative(
|
||||
"canon-018-trailing-bytes-refused",
|
||||
"Two items back to back. A canonical encoding is exactly one item: bytes after it would sit outside the value a SAID commits to, so the input is refused.",
|
||||
&[0x01, 0x01],
|
||||
"trailing_bytes",
|
||||
),
|
||||
canon_negative(
|
||||
"canon-019-invalid-utf8-refused",
|
||||
"A text string whose bytes (0xc3 0x28) are not UTF-8. CBOR text strings are UTF-8 (RFC 8949 §3.1), so the input is refused.",
|
||||
&[0x62, 0xc3, 0x28],
|
||||
"invalid_utf8",
|
||||
),
|
||||
canon_negative(
|
||||
"canon-020-simple-value-refused",
|
||||
"The simple value undefined (0xf7). Identity spec §2.3 admits false, true and null and no other simple value, so the data model has nothing to decode it to and the input is refused.",
|
||||
&[0xf7],
|
||||
"simple_value_forbidden",
|
||||
),
|
||||
])
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// said_vectors.json
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// The SAID vectors, plus each positive vector's derivation for the files
|
||||
/// that refer to them.
|
||||
pub struct SaidVectors {
|
||||
pub json: J,
|
||||
pub by_name: BTreeMap<&'static str, SaidDerivation>,
|
||||
/// said-009's input: said-008's object with `n` written non-canonically.
|
||||
pub non_canonical_copy: Vec<u8>,
|
||||
/// said-011's input: said-008's object with `n` changed, `d` kept.
|
||||
pub changed_copy: Vec<u8>,
|
||||
}
|
||||
|
||||
fn said_positive(
|
||||
by_name: &mut BTreeMap<&'static str, SaidDerivation>,
|
||||
name: &'static str,
|
||||
description: &str,
|
||||
input: Value,
|
||||
) -> J {
|
||||
let d = said::derive(&input, SAID_FIELD).expect(name);
|
||||
assert_eq!(
|
||||
said::verify(&d.encoded, SAID_FIELD).as_ref(),
|
||||
Ok(&d.said),
|
||||
"{name}"
|
||||
);
|
||||
let json = J::obj([
|
||||
("name", J::from(name)),
|
||||
("description", J::from(description)),
|
||||
("said_field", J::from(SAID_FIELD)),
|
||||
("input", to_json(&input)),
|
||||
("canonical_cbor_hex", J::Str(hex::encode(&d.hashed))),
|
||||
("blake3_hex", J::Str(hex::encode(d.said.as_bytes()))),
|
||||
("expected_said", J::Str(d.said.to_said_text())),
|
||||
("said_cbor_hex", J::Str(hex::encode(&d.encoded))),
|
||||
]);
|
||||
by_name.insert(name, d);
|
||||
json
|
||||
}
|
||||
|
||||
fn said_negative(
|
||||
name: &str,
|
||||
description: &str,
|
||||
input: &[u8],
|
||||
class: &str,
|
||||
extra: Vec<(&str, J)>,
|
||||
) -> J {
|
||||
let refused = said::verify(input, SAID_FIELD).expect_err(name);
|
||||
assert_eq!(
|
||||
refused.class(),
|
||||
class,
|
||||
"{name} is refused for the wrong reason"
|
||||
);
|
||||
let mut entries = vec![
|
||||
("name", J::from(name)),
|
||||
("description", J::from(description)),
|
||||
("said_field", J::from(SAID_FIELD)),
|
||||
("input_cbor_hex", J::Str(hex::encode(input))),
|
||||
("error", J::from(class)),
|
||||
];
|
||||
entries.extend(extra);
|
||||
J::obj(entries)
|
||||
}
|
||||
|
||||
impl SaidVectors {
|
||||
pub fn build() -> Self {
|
||||
let mut by_name = BTreeMap::new();
|
||||
let placeholder = || text(SAID_PLACEHOLDER);
|
||||
let key = |b: u8| Value::Bytes(vec![b; 32]);
|
||||
let filler_signature = |b: u8| Value::Bytes(vec![b; 64]);
|
||||
let joules = |measured: u64, estimated: u64| {
|
||||
Value::map([
|
||||
("measured_microjoules", uint(measured)),
|
||||
("estimated_microjoules", uint(estimated)),
|
||||
])
|
||||
};
|
||||
let usd = Value::map([
|
||||
("type", text("usd.cargo.v1")),
|
||||
("denomination_micros", uint(100_000_000)),
|
||||
("issuer_pubkey", key(0x11)),
|
||||
]);
|
||||
let provenance = Value::map([
|
||||
("alg", uint(1)),
|
||||
("issuer_pubkey", key(0x11)),
|
||||
("issuance_frame", uint(1024)),
|
||||
("signature", filler_signature(0x22)),
|
||||
]);
|
||||
let envelope = |provenance: Value, chain: Vec<Value>, cargo: Value, cost: Value| {
|
||||
Value::map([
|
||||
("d", placeholder()),
|
||||
("provenance", provenance),
|
||||
("ownership_chain", Value::Array(chain)),
|
||||
("cargo", cargo),
|
||||
("joule_cost", cost),
|
||||
])
|
||||
};
|
||||
|
||||
let mut vectors = vec![
|
||||
said_positive(
|
||||
&mut by_name,
|
||||
"said-001-empty-envelope",
|
||||
"A nested object with an envelope's top-level members — provenance, ownership chain, cargo, joule cost — all empty or zero. It exercises the whole pipeline (placeholder, canonical CBOR, BLAKE3-256, `E` + base32) on an object with no application data.",
|
||||
envelope(Value::Null, vec![], Value::Null, joules(0, 0)),
|
||||
),
|
||||
said_positive(
|
||||
&mut by_name,
|
||||
"said-002-usd-cargo",
|
||||
"The same object carrying a cargo map: a USD amount of 100.00 (100000000 micros) and a 32-byte issuer key. Members of mixed types — text, integer, byte string — all sort by their encoded keys.",
|
||||
envelope(Value::Null, vec![], usd.clone(), joules(12, 15)),
|
||||
),
|
||||
said_positive(
|
||||
&mut by_name,
|
||||
"said-003-custom-cargo",
|
||||
"The same shape carrying a temperature reading as cargo. What a member means never affects its encoding: this cargo goes through the pipeline exactly as said-002's does.",
|
||||
envelope(
|
||||
Value::Null,
|
||||
vec![],
|
||||
Value::map([
|
||||
("type", text("sensor.temperature.v1")),
|
||||
("sensor_id", text("rooftop-array-7")),
|
||||
("celsius_milli", uint(23475)),
|
||||
("reading_frame", uint(184236)),
|
||||
]),
|
||||
joules(8, 10),
|
||||
),
|
||||
),
|
||||
said_positive(
|
||||
&mut by_name,
|
||||
"said-004-with-provenance",
|
||||
"The same shape with a provenance map: algorithm identifier 0x01 (Ed25519), issuer key, issuance frame and signature. The SAID covers every member. The key and signature are fixed filler bytes: this vector tests SAID derivation over them, not their validity.",
|
||||
envelope(provenance.clone(), vec![], usd.clone(), joules(14, 18)),
|
||||
),
|
||||
];
|
||||
|
||||
// said-005 chains from said-004 (treatise §10.1): the byte's identity is
|
||||
// its initial history hash, and each step is blake3(previous ‖ payload).
|
||||
let origin = by_name["said-004-with-provenance"].said;
|
||||
let transition = |from: u8, to: u8, frame: u64, previous: [u8; 32], signature: u8| {
|
||||
Value::map([
|
||||
("alg", uint(1)),
|
||||
("from_pubkey", key(from)),
|
||||
("to_pubkey", key(to)),
|
||||
("frame", uint(frame)),
|
||||
("previous_history_hash", Value::Bytes(previous.to_vec())),
|
||||
("signature", filler_signature(signature)),
|
||||
])
|
||||
};
|
||||
let first = transition(0x11, 0x33, 1050, origin.0, 0x44);
|
||||
let after_first = Said::hash(&[origin.as_bytes().as_slice(), &cbor(&first)].concat()).0;
|
||||
let second = transition(0x33, 0x55, 1080, after_first, 0x66);
|
||||
vectors.push(said_positive(
|
||||
&mut by_name,
|
||||
"said-005-with-ownership-transitions",
|
||||
"said-004's object with two ownership transitions appended and a larger joule cost: an array of maps holding byte strings. The first transition's previous_history_hash is the digest of said-004's SAID; the second is blake3 of that digest followed by the first transition's canonical encoding. The SAID differs from said-004's because a SAID covers the whole object it is derived over. A byte's identity is a different quantity — the hash of its origin attestation (treatise §8.2) — and does not change as transitions accrue.",
|
||||
envelope(provenance, vec![first, second], usd, joules(47, 56)),
|
||||
));
|
||||
|
||||
vectors.push(said_positive(
|
||||
&mut by_name,
|
||||
"said-006-only-said-field",
|
||||
"The empty-map case of identity spec §2.5: an object whose only member is its SAID field. It is the smallest object a SAID can be derived over — a truly empty map has nowhere to put one.",
|
||||
Value::map([("d", placeholder())]),
|
||||
));
|
||||
|
||||
let rfc_key = SigningKey::from_bytes(&seed(RFC8032_TEST1_SEED))
|
||||
.verifying_key()
|
||||
.to_bytes();
|
||||
let prefixed_key = [&[0x01u8][..], &rfc_key].concat();
|
||||
vectors.push(said_positive(
|
||||
&mut by_name,
|
||||
"said-007-one-field-per-key",
|
||||
"The one-field case: identity spec §3.4's per-key object {d, k}, whose SAID is a next-key commitment. k is a byte string: the algorithm identifier 0x01 (Ed25519) followed by the public key of RFC 8032's first test seed (§3.3).",
|
||||
Value::map([("d", placeholder()), ("k", Value::Bytes(prefixed_key.clone()))]),
|
||||
));
|
||||
assert_eq!(
|
||||
said::per_key_said(&prefixed_key),
|
||||
by_name["said-007-one-field-per-key"].said,
|
||||
"per_key_said must agree with the vector it implements"
|
||||
);
|
||||
|
||||
vectors.push(said_positive(
|
||||
&mut by_name,
|
||||
"said-008-spec-worked-example",
|
||||
"The worked example of identity spec §2.4: {\"d\": placeholder, \"n\": 1}. Its canonical encoding is 61 bytes, and §2.4 prints the digest and SAID given here.",
|
||||
Value::map([("d", placeholder()), ("n", uint(1))]),
|
||||
));
|
||||
|
||||
// The adversarial cases of identity spec §2.5, built on said-008.
|
||||
let example = &by_name["said-008-spec-worked-example"];
|
||||
let canonical_said = example.said.to_said_text();
|
||||
// n = 1 is the final byte of the canonical form; spell it 0x18 0x01.
|
||||
let lax_n = |canonical: &[u8]| {
|
||||
let mut v = canonical[..canonical.len() - 1].to_vec();
|
||||
v.extend_from_slice(&[0x18, 0x01]);
|
||||
v
|
||||
};
|
||||
let non_canonical_copy = lax_n(&example.encoded);
|
||||
let hashed_as_received = Said::hash(&lax_n(&example.hashed)).to_said_text();
|
||||
vectors.push(said_negative(
|
||||
"said-009-non-canonical-copy-rejected",
|
||||
"The adversarial case of identity spec §2.5, aimed at re-encoding. These are said-008's bytes with n written 0x18 0x01 instead of 0x01, still stating said-008's SAID. A verifier that re-encodes before hashing gets said-008's bytes back and accepts, so a second byte string would verify under said-008's SAID. A conforming verifier refuses the input as non-canonical before computing anything.",
|
||||
&non_canonical_copy,
|
||||
"non_shortest_integer",
|
||||
vec![
|
||||
("said_of_received_bytes", J::Str(hashed_as_received)),
|
||||
("said_after_reencoding", J::Str(canonical_said.clone())),
|
||||
],
|
||||
));
|
||||
|
||||
// The same object with its keys reversed, stating the SAID of exactly
|
||||
// those bytes: self-consistent for a verifier that hashes as received.
|
||||
let reversed = |d: &Value| {
|
||||
concat(&[
|
||||
&[0xa2],
|
||||
&cbor(&text("n")),
|
||||
&[0x01],
|
||||
&cbor(&text("d")),
|
||||
&cbor(d),
|
||||
])
|
||||
};
|
||||
let reversed_said = Said::hash(&reversed(&placeholder())).to_said_text();
|
||||
let reversed_bytes = reversed(&text(&reversed_said));
|
||||
vectors.push(said_negative(
|
||||
"said-010-self-consistent-non-canonical-rejected",
|
||||
"The adversarial case aimed the other way. said-008's object with its keys in reverse order, stating the SAID of exactly these bytes (placeholder substituted in place). A verifier that hashes what it received, without checking canonical form, accepts it. A conforming verifier refuses it as non-canonical.",
|
||||
&reversed_bytes,
|
||||
"unsorted_map_keys",
|
||||
vec![
|
||||
("said_of_received_bytes", J::Str(reversed_said)),
|
||||
("said_after_reencoding", J::Str(canonical_said)),
|
||||
],
|
||||
));
|
||||
|
||||
let mut changed_copy = example.encoded.clone();
|
||||
*changed_copy.last_mut().unwrap() = 0x02;
|
||||
vectors.push(said_negative(
|
||||
"said-011-changed-content-rejected",
|
||||
"said-008's object with n changed from 1 to 2 and its SAID left as it was. The bytes are canonical and well-formed, but they state a SAID they do not hash to.",
|
||||
&changed_copy,
|
||||
"said_mismatch",
|
||||
vec![],
|
||||
));
|
||||
|
||||
SaidVectors {
|
||||
json: J::Arr(vectors),
|
||||
by_name,
|
||||
non_canonical_copy,
|
||||
changed_copy,
|
||||
}
|
||||
}
|
||||
|
||||
fn said(&self, name: &str) -> &SaidDerivation {
|
||||
&self.by_name[name]
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// signature_vectors.json
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
fn signature_vector(
|
||||
name: &str,
|
||||
description: &str,
|
||||
said: &Said,
|
||||
signing_seed: Option<[u8; 32]>,
|
||||
verifying_key: [u8; 32],
|
||||
signature: &[u8],
|
||||
expected_verify: bool,
|
||||
error: Option<&str>,
|
||||
) -> J {
|
||||
if let Some(seed) = signing_seed {
|
||||
assert_eq!(
|
||||
SigningKey::from_bytes(&seed).verifying_key().to_bytes(),
|
||||
verifying_key,
|
||||
"{name}: a seed's vector carries the key the seed derives"
|
||||
);
|
||||
}
|
||||
let result = sign::verify_said_signature(&said.to_said_text(), signature, &verifying_key);
|
||||
assert_eq!(result.is_ok(), expected_verify, "{name}");
|
||||
assert_eq!(
|
||||
result.err().map(|e| e.class()),
|
||||
error,
|
||||
"{name} fails for the wrong reason"
|
||||
);
|
||||
J::obj([
|
||||
("name", J::from(name)),
|
||||
("description", J::from(description)),
|
||||
("envelope_said", J::Str(said.to_said_text())),
|
||||
("signed_message_hex", J::Str(hex::encode(said.as_bytes()))),
|
||||
(
|
||||
"signing_key_seed",
|
||||
signing_seed
|
||||
.map(|s| J::Str(hex::encode(s)))
|
||||
.unwrap_or(J::Null),
|
||||
),
|
||||
("verifying_key", J::Str(hex::encode(verifying_key))),
|
||||
("signature", J::Str(hex::encode(signature))),
|
||||
("expected_verify", J::from(expected_verify)),
|
||||
("error", error.map(J::from).unwrap_or(J::Null)),
|
||||
])
|
||||
}
|
||||
|
||||
/// The Ed25519 group order L, little-endian.
|
||||
const ED25519_L: [u8; 32] = [
|
||||
0xed, 0xd3, 0xf5, 0x5c, 0x1a, 0x63, 0x12, 0x58, 0xd6, 0x9c, 0xf7, 0xa2, 0xde, 0xf9, 0xde, 0x14,
|
||||
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10,
|
||||
];
|
||||
|
||||
/// `signature` with its scalar S replaced by S + L: the same scalar, written
|
||||
/// outside [0, L).
|
||||
fn with_s_plus_l(signature: &[u8; 64]) -> [u8; 64] {
|
||||
let mut out = *signature;
|
||||
let mut carry = 0u16;
|
||||
for i in 0..32 {
|
||||
let v = u16::from(signature[32 + i]) + u16::from(ED25519_L[i]) + carry;
|
||||
out[32 + i] = v as u8;
|
||||
carry = v >> 8;
|
||||
}
|
||||
assert_eq!(carry, 0, "S + L fits in 32 bytes for any S < L");
|
||||
out
|
||||
}
|
||||
|
||||
/// The point y = 3 (sign bit clear) spelled with y = p + 3: an out-of-range
|
||||
/// encoding of a valid, large-order point.
|
||||
const KEY_Y_PLUS_P: [u8; 32] = [
|
||||
0xf0, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
|
||||
0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x7f,
|
||||
];
|
||||
|
||||
fn key_of(seed: &[u8; 32]) -> [u8; 32] {
|
||||
SigningKey::from_bytes(seed).verifying_key().to_bytes()
|
||||
}
|
||||
|
||||
fn signatures(saids: &SaidVectors) -> J {
|
||||
let empty = saids.said("said-001-empty-envelope").said;
|
||||
let usd = saids.said("said-002-usd-cargo").said;
|
||||
let custom = saids.said("said-003-custom-cargo").said;
|
||||
let (seed1, seed2, zero) = (
|
||||
seed(RFC8032_TEST1_SEED),
|
||||
seed(RFC8032_TEST2_SEED),
|
||||
[0u8; 32],
|
||||
);
|
||||
let sig1 = sign::sign_said(&usd, &SigningKey::from_bytes(&seed1)).to_bytes();
|
||||
let sig2 = sign::sign_said(&empty, &SigningKey::from_bytes(&seed2)).to_bytes();
|
||||
J::Arr(vec![
|
||||
signature_vector(
|
||||
"sig-001-known-key-known-said",
|
||||
"Ed25519 (RFC 8032) is deterministic: signing said-002's SAID with the key of RFC 8032's first test seed gives exactly this signature. The message is the 32 digest bytes the SAID spells (signed_message_hex), not its 53-character text.",
|
||||
&usd,
|
||||
Some(seed1),
|
||||
key_of(&seed1),
|
||||
&sig1,
|
||||
true,
|
||||
None,
|
||||
),
|
||||
signature_vector(
|
||||
"sig-002-verify-positive",
|
||||
"A signature over said-001's SAID by the key of RFC 8032's second test seed. It verifies under that key.",
|
||||
&empty,
|
||||
Some(seed2),
|
||||
key_of(&seed2),
|
||||
&sig2,
|
||||
true,
|
||||
None,
|
||||
),
|
||||
signature_vector(
|
||||
"sig-003-tampered-envelope",
|
||||
"sig-002's signature presented with said-003's SAID: an adversary swaps the object and forwards the original signature. The signature is well-formed but binds another SAID, so verification MUST fail.",
|
||||
&custom,
|
||||
Some(seed2),
|
||||
key_of(&seed2),
|
||||
&sig2,
|
||||
false,
|
||||
Some("signature_invalid"),
|
||||
),
|
||||
signature_vector(
|
||||
"sig-004-wrong-key",
|
||||
"sig-002's signature over said-001's SAID, presented with another key — the one the all-zero seed derives. The signature is valid for the key that made it, which is not this one, so verification MUST fail.",
|
||||
&empty,
|
||||
Some(zero),
|
||||
key_of(&zero),
|
||||
&sig2,
|
||||
false,
|
||||
Some("signature_invalid"),
|
||||
),
|
||||
signature_vector(
|
||||
"sig-005-malformed-signature",
|
||||
"sig-002's signature cut to 63 bytes. It is refused as malformed before any cryptography runs, and an implementation MUST NOT panic on it.",
|
||||
&empty,
|
||||
Some(seed2),
|
||||
key_of(&seed2),
|
||||
&sig2[..63],
|
||||
false,
|
||||
Some("malformed_signature"),
|
||||
),
|
||||
signature_vector(
|
||||
"sig-006-non-canonical-s-rejected",
|
||||
"sig-002's signature with its scalar S replaced by S + L: the same scalar, written outside [0, L). RFC 8032 §5.1.7 requires refusing it; accepting it would make signatures malleable, giving a second valid-looking signature for one message and key.",
|
||||
&empty,
|
||||
Some(seed2),
|
||||
key_of(&seed2),
|
||||
&with_s_plus_l(&sig2),
|
||||
false,
|
||||
Some("signature_invalid"),
|
||||
),
|
||||
signature_vector(
|
||||
"sig-007-non-canonical-key-rejected",
|
||||
"A verifying key spelled with y = p + 3: an out-of-range encoding of the valid, large-order point with y = 3. RFC 8032 §5.1.3 requires decoding to fail when y is not below p, so the key is refused as malformed before any verification runs — one key has one spelling. There is no seed: no key pair has this spelling.",
|
||||
&empty,
|
||||
None,
|
||||
KEY_Y_PLUS_P,
|
||||
&sig2,
|
||||
false,
|
||||
Some("malformed_verifying_key"),
|
||||
),
|
||||
])
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// lockstep_vectors.json
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// One node: its name, the hash it reported, and the hash it computed.
|
||||
type NodeHashes<'a> = (&'a str, u8, u8);
|
||||
|
||||
fn lockstep_vector(
|
||||
name: &str,
|
||||
description: &str,
|
||||
frame: u64,
|
||||
cluster_size: usize,
|
||||
nodes: &[NodeHashes<'_>],
|
||||
) -> J {
|
||||
let reported: Vec<(String, [u8; 32])> = nodes
|
||||
.iter()
|
||||
.map(|(node, r, _)| ((*node).to_owned(), [*r; 32]))
|
||||
.collect();
|
||||
// What the broadcast hashes decide: the frame's public record.
|
||||
let public = tally(cluster_size, &reported).expect(name);
|
||||
|
||||
let mut outcomes = Vec::new();
|
||||
let mut logs = Vec::new();
|
||||
for (node, _, c) in nodes {
|
||||
let computed = [*c; 32];
|
||||
let view = node_view(cluster_size, &reported, &(*node).to_owned(), &computed).expect(name);
|
||||
outcomes.push((
|
||||
(*node).to_owned(),
|
||||
J::from(view.outcome(&computed).as_str()),
|
||||
));
|
||||
let log: Vec<J> = view
|
||||
.dissent_entry(frame, &computed)
|
||||
.into_iter()
|
||||
.map(|e| {
|
||||
J::obj([
|
||||
("frame", J::from(e.frame)),
|
||||
("dissenting_peers", J::strs(e.dissenting_peers)),
|
||||
("local_hash", J::Str(hex::encode(e.local_hash))),
|
||||
(
|
||||
"supermajority_hash",
|
||||
J::Str(hex::encode(e.supermajority_hash)),
|
||||
),
|
||||
])
|
||||
})
|
||||
.collect();
|
||||
logs.push(((*node).to_owned(), J::Arr(log)));
|
||||
}
|
||||
|
||||
let hashes = |pick: fn(&NodeHashes<'_>) -> u8| {
|
||||
J::obj(
|
||||
nodes
|
||||
.iter()
|
||||
.map(|n| (n.0.to_owned(), J::Str(hex::encode([pick(n); 32])))),
|
||||
)
|
||||
};
|
||||
J::obj([
|
||||
("name", J::from(name)),
|
||||
("description", J::from(description)),
|
||||
("node_count", J::from(cluster_size)),
|
||||
("frame_number", J::from(frame)),
|
||||
("node_reported_hashes", hashes(|n| n.1)),
|
||||
("node_computed_hashes", hashes(|n| n.2)),
|
||||
(
|
||||
"expected_threshold",
|
||||
J::from(supermajority_threshold(cluster_size)),
|
||||
),
|
||||
("expected_commit", J::from(public.committed())),
|
||||
(
|
||||
"expected_state_hash",
|
||||
public
|
||||
.supermajority_hash
|
||||
.map(|h| J::Str(hex::encode(h)))
|
||||
.unwrap_or(J::Null),
|
||||
),
|
||||
("expected_outcomes", J::Obj(outcomes)),
|
||||
("expected_dissent_logs", J::Obj(logs)),
|
||||
])
|
||||
}
|
||||
|
||||
fn lockstep() -> J {
|
||||
J::Arr(vec![
|
||||
lockstep_vector(
|
||||
"lockstep-001-unanimous-commit",
|
||||
"All four nodes report the post-frame state hash they computed, and all agree. The threshold is floor(2·4/3)+1 = 3, so the frame commits, every node's outcome is a clean commit (treatise §13.3), and no dissent is logged. State hashes are opaque 32-byte values compared for equality. Each node's outcome comes from its own view: the hashes its peers reported plus the one it computed itself (§13.3, §13.4).",
|
||||
184237,
|
||||
4,
|
||||
&[
|
||||
("node_1", 0xaa, 0xaa),
|
||||
("node_2", 0xaa, 0xaa),
|
||||
("node_3", 0xaa, 0xaa),
|
||||
("node_4", 0xaa, 0xaa),
|
||||
],
|
||||
),
|
||||
lockstep_vector(
|
||||
"lockstep-002-supermajority-commit",
|
||||
"Three of four nodes agree. node_4's state diverged, and it reports the hash it computed. The threshold of 3 is met: nodes 1–3 commit with dissent, and each writes one dissent-log entry naming node_4 (treatise §14.1). node_4 sees a supermajority on a hash that is not its own — minority divergence — so it halts and writes nothing.",
|
||||
184238,
|
||||
4,
|
||||
&[
|
||||
("node_1", 0xbb, 0xbb),
|
||||
("node_2", 0xbb, 0xbb),
|
||||
("node_3", 0xbb, 0xbb),
|
||||
("node_4", 0xdd, 0xdd),
|
||||
],
|
||||
),
|
||||
lockstep_vector(
|
||||
"lockstep-003-threshold-not-met",
|
||||
"No hash is reported by more than two of the four nodes. None reaches the threshold of 3, so the frame does not commit and every node's outcome is a cluster split (§13.3). No dissent is logged: dissent is divergence from a committed hash (§14.2), and there is none.",
|
||||
184239,
|
||||
4,
|
||||
&[
|
||||
("node_1", 0xcc, 0xcc),
|
||||
("node_2", 0xcc, 0xcc),
|
||||
("node_3", 0xdd, 0xdd),
|
||||
("node_4", 0xee, 0xee),
|
||||
],
|
||||
),
|
||||
lockstep_vector(
|
||||
"lockstep-004-three-nodes-one-diverged-split",
|
||||
"The n = 3 exception (treatise §13.2). The threshold is 3, so one node whose state diverged leaves 2 of the 3 votes the frame needs. The cluster detects the divergence but cannot tolerate it: every node halts with a cluster split.",
|
||||
184240,
|
||||
3,
|
||||
&[
|
||||
("node_1", 0x11, 0x11),
|
||||
("node_2", 0x11, 0x11),
|
||||
("node_3", 0x22, 0x22),
|
||||
],
|
||||
),
|
||||
lockstep_vector(
|
||||
"lockstep-005-seven-nodes-two-dissenters",
|
||||
"Seven nodes, threshold 5: five agree and two diverged nodes report hashes of their own. The frame commits. The dissent entry names both dissenters in ascending order, so every committing node writes the same entry (§14.2).",
|
||||
184241,
|
||||
7,
|
||||
&[
|
||||
("node_1", 0x5a, 0x5a),
|
||||
("node_2", 0x5a, 0x5a),
|
||||
("node_3", 0x5a, 0x5a),
|
||||
("node_4", 0x5a, 0x5a),
|
||||
("node_5", 0x5a, 0x5a),
|
||||
("node_6", 0x6b, 0x6b),
|
||||
("node_7", 0x7c, 0x7c),
|
||||
],
|
||||
),
|
||||
lockstep_vector(
|
||||
"lockstep-006-transport-liar-is-logged",
|
||||
"Treatise §13.4: node_4 computed the honest hash but broadcast another. Its peers see 3-of-4 agreement and record node_4 in their dissent logs. node_4 tallies its own honest hash, sees unanimity and commits cleanly: its lie surfaces only in its peers' logs, which is why the dissent log is the substrate's misbehaviour signal (§14).",
|
||||
184242,
|
||||
4,
|
||||
&[
|
||||
("node_1", 0xbb, 0xbb),
|
||||
("node_2", 0xbb, 0xbb),
|
||||
("node_3", 0xbb, 0xbb),
|
||||
("node_4", 0xdd, 0xbb),
|
||||
],
|
||||
),
|
||||
lockstep_vector(
|
||||
"lockstep-007-three-node-transport-liar",
|
||||
"§13.2 and §13.4 together. At n = 3 a node that broadcasts a hash other than the one it computed splits the honest pair, who halt. The liar sees its own honest hash agree with both peers and commits. The reported hashes reach no supermajority, so the frame's public record is no commit.",
|
||||
184243,
|
||||
3,
|
||||
&[
|
||||
("node_1", 0x11, 0x11),
|
||||
("node_2", 0x11, 0x11),
|
||||
("node_3", 0x22, 0x11),
|
||||
],
|
||||
),
|
||||
])
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// gossip_vectors.json
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
fn gossip_vector(
|
||||
name: &str,
|
||||
description: &str,
|
||||
receiver: &str,
|
||||
deliveries: &[(&str, String, Vec<u8>)],
|
||||
) -> J {
|
||||
let mut inbox = Inbox::new();
|
||||
let mut results = Vec::new();
|
||||
for (_, claimed, bytes) in deliveries {
|
||||
results.push(match inbox.deliver(claimed, bytes) {
|
||||
Delivery::Accepted(s) => J::obj([
|
||||
("outcome", J::from("accepted")),
|
||||
("said", J::Str(s.to_said_text())),
|
||||
]),
|
||||
Delivery::Duplicate(s) => J::obj([
|
||||
("outcome", J::from("duplicate")),
|
||||
("said", J::Str(s.to_said_text())),
|
||||
]),
|
||||
Delivery::Rejected(e) => J::obj([
|
||||
("outcome", J::from("rejected")),
|
||||
("error", J::from(e.class())),
|
||||
]),
|
||||
});
|
||||
}
|
||||
J::obj([
|
||||
("name", J::from(name)),
|
||||
("description", J::from(description)),
|
||||
("receiver", J::from(receiver)),
|
||||
(
|
||||
"deliveries",
|
||||
J::Arr(
|
||||
deliveries
|
||||
.iter()
|
||||
.map(|(from, claimed, bytes)| {
|
||||
J::obj([
|
||||
("from", J::from(*from)),
|
||||
("claimed_said", J::Str(claimed.clone())),
|
||||
("cbor_hex", J::Str(hex::encode(bytes))),
|
||||
])
|
||||
})
|
||||
.collect(),
|
||||
),
|
||||
),
|
||||
("expected_results", J::Arr(results)),
|
||||
(
|
||||
"expected_held",
|
||||
J::Arr(
|
||||
inbox
|
||||
.held()
|
||||
.iter()
|
||||
.map(|(said, bytes)| {
|
||||
J::obj([
|
||||
("said", J::Str(said.to_said_text())),
|
||||
("cbor_hex", J::Str(hex::encode(bytes))),
|
||||
])
|
||||
})
|
||||
.collect(),
|
||||
),
|
||||
),
|
||||
])
|
||||
}
|
||||
|
||||
fn gossip(saids: &SaidVectors) -> J {
|
||||
let empty = saids.said("said-006-only-said-field");
|
||||
let per_key = saids.said("said-007-one-field-per-key");
|
||||
let example = saids.said("said-008-spec-worked-example");
|
||||
let text = |d: &SaidDerivation| d.said.to_said_text();
|
||||
J::Arr(vec![
|
||||
gossip_vector(
|
||||
"gossip-001-cross-node-propagation",
|
||||
"An object published at node_A reaches node_B. Its bytes are canonical, they hash to the SAID they state, and that is the SAID they were announced under, so node_B accepts it (identity spec §2.6) and holds exactly the bytes it received. Gossip carries the object itself, never a re-encoding of it.",
|
||||
"node_B",
|
||||
&[("node_A", text(per_key), per_key.encoded.clone())],
|
||||
),
|
||||
gossip_vector(
|
||||
"gossip-002-said-conflict-rejected",
|
||||
"node_C holds said-008's object when a second object arrives under the same SAID: said-008's with n changed and its SAID left as it was (said-011's bytes). Verification comes before deduplication (identity spec §2.6), so the newcomer is reported as a SAID mismatch, which makes the substitution attempt visible, instead of being absorbed as a duplicate of what node_C holds.",
|
||||
"node_C",
|
||||
&[
|
||||
("node_A", text(example), example.encoded.clone()),
|
||||
("node_B", text(example), saids.changed_copy.clone()),
|
||||
],
|
||||
),
|
||||
gossip_vector(
|
||||
"gossip-003-backfill-answer-verified",
|
||||
"node_B requests the object behind said-007's SAID: a party holding an identifier can request the object from any peer (treatise §21.4). node_A answers with said-008's object, which is genuine and self-consistent but not the one requested, and is refused. node_D's answer hashes to the requested SAID and is accepted (identity spec §2.6).",
|
||||
"node_B",
|
||||
&[
|
||||
("node_A", text(per_key), example.encoded.clone()),
|
||||
("node_D", text(per_key), per_key.encoded.clone()),
|
||||
],
|
||||
),
|
||||
gossip_vector(
|
||||
"gossip-004-duplicate-delivery-deduplicated",
|
||||
"The same object reaches node_B from two peers. The second delivery verifies and is a duplicate of a SAID already held, so it changes nothing: the overlay deduplicates by content address (treatise §11.4).",
|
||||
"node_B",
|
||||
&[
|
||||
("node_A", text(empty), empty.encoded.clone()),
|
||||
("node_C", text(empty), empty.encoded.clone()),
|
||||
],
|
||||
),
|
||||
gossip_vector(
|
||||
"gossip-005-non-canonical-copy-rejected",
|
||||
"said-009's bytes reach node_B announced under said-008's SAID, the SAID a re-encoding receiver would compute for them. The bytes are not canonical, so node_B refuses them (identity spec §2.5, §2.6). Accepting them would let a second byte string circulate under said-008's SAID.",
|
||||
"node_B",
|
||||
&[("node_A", text(example), saids.non_canonical_copy.clone())],
|
||||
),
|
||||
gossip_vector(
|
||||
"gossip-006-substitute-arriving-first-rejected",
|
||||
"gossip-002 in the other order: the altered object reaches node_C first and is refused, then said-008's object arrives and is accepted. Arrival order decides nothing; the content does.",
|
||||
"node_C",
|
||||
&[
|
||||
("node_B", text(example), saids.changed_copy.clone()),
|
||||
("node_A", text(example), example.encoded.clone()),
|
||||
],
|
||||
),
|
||||
])
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::json::bytes_json;
|
||||
|
||||
#[test]
|
||||
fn generation_is_deterministic() {
|
||||
assert_eq!(generate(), generate());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_typed_json_for_bytes_is_one_member() {
|
||||
assert_eq!(bytes_json(&[0xab]), J::obj([("$bytes", J::from("ab"))]));
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,637 @@
|
|||
//! Check a vector pack on disk against the reference implementation.
|
||||
//!
|
||||
//! This reads only the published JSON: every expected value is recomputed
|
||||
//! from the vector's own inputs, and every negative vector must be refused
|
||||
//! with the error class it names. It does not consult the generator, so it
|
||||
//! checks a pack whoever produced it.
|
||||
//!
|
||||
//! It also checks that the pack is complete, against texts that live outside
|
||||
//! the pack: every vector the identity spec cites must be present, and every
|
||||
//! error class the conformance README lists must be exercised (and no other).
|
||||
//! A pack cannot pass by losing vectors.
|
||||
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use serde_json::Value as S;
|
||||
use smart_byte_core::Said;
|
||||
use smart_byte_core::canonical::{self, Value};
|
||||
use smart_byte_core::inbox::{Delivery, Inbox};
|
||||
use smart_byte_core::said::{self, SAID_PLACEHOLDER};
|
||||
use smart_byte_core::sign::{self, SigningKey};
|
||||
use smart_byte_lockstep::{node_view, supermajority_threshold, tally};
|
||||
|
||||
use crate::json::{from_json, parse_strict};
|
||||
use crate::pack::FILES;
|
||||
|
||||
/// One vector's result.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Check {
|
||||
pub file: String,
|
||||
pub vector: String,
|
||||
pub result: Result<(), String>,
|
||||
}
|
||||
|
||||
/// Every check a pack verification made.
|
||||
#[derive(Debug, Default)]
|
||||
pub struct Report {
|
||||
pub checks: Vec<Check>,
|
||||
}
|
||||
|
||||
impl Report {
|
||||
pub fn failures(&self) -> Vec<&Check> {
|
||||
self.checks.iter().filter(|c| c.result.is_err()).collect()
|
||||
}
|
||||
|
||||
pub fn passed(&self) -> bool {
|
||||
!self.checks.is_empty() && self.failures().is_empty()
|
||||
}
|
||||
|
||||
fn record(&mut self, file: &str, vector: &str, result: Result<(), String>) {
|
||||
self.checks.push(Check {
|
||||
file: file.to_owned(),
|
||||
vector: vector.to_owned(),
|
||||
result,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
type R = Result<(), String>;
|
||||
|
||||
fn field<'a>(v: &'a S, key: &str) -> Result<&'a S, String> {
|
||||
v.get(key).ok_or_else(|| format!("missing `{key}`"))
|
||||
}
|
||||
|
||||
fn text<'a>(v: &'a S, key: &str) -> Result<&'a str, String> {
|
||||
field(v, key)?
|
||||
.as_str()
|
||||
.ok_or_else(|| format!("`{key}` is not a string"))
|
||||
}
|
||||
|
||||
fn hex_field(v: &S, key: &str) -> Result<Vec<u8>, String> {
|
||||
hex::decode(text(v, key)?).map_err(|e| format!("`{key}` is not hex: {e}"))
|
||||
}
|
||||
|
||||
fn hash32(v: &S, what: &str) -> Result<[u8; 32], String> {
|
||||
v.as_str()
|
||||
.and_then(|s| hex::decode(s).ok())
|
||||
.and_then(|b| b.try_into().ok())
|
||||
.ok_or_else(|| format!("{what} is not 32 bytes of hex"))
|
||||
}
|
||||
|
||||
fn same(what: &str, computed: impl PartialEq<String> + std::fmt::Display, published: &str) -> R {
|
||||
if computed == published.to_owned() {
|
||||
Ok(())
|
||||
} else {
|
||||
Err(format!(
|
||||
"{what}: published {published}, reference computes {computed}"
|
||||
))
|
||||
}
|
||||
}
|
||||
|
||||
/// The texts the completeness checks are held to: the identity spec and the
|
||||
/// conformance README in this repository.
|
||||
fn repo_text(relative: &str) -> Result<String, String> {
|
||||
let path = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(relative);
|
||||
std::fs::read_to_string(&path).map_err(|e| format!("read {}: {e}", path.display()))
|
||||
}
|
||||
|
||||
/// The SAID vector prefixes the identity spec cites (`said-006`, and every
|
||||
/// member of a range written `said-001` to `said-005`).
|
||||
pub fn spec_cited_said_vectors() -> Result<BTreeSet<String>, String> {
|
||||
let spec = repo_text("../../../spec/identity_and_key_rotation.md")?;
|
||||
let mut cited = BTreeSet::new();
|
||||
let words: Vec<&str> = spec
|
||||
.split(|c: char| !(c.is_ascii_alphanumeric() || c == '-'))
|
||||
.filter(|w| !w.is_empty())
|
||||
.collect();
|
||||
let number = |w: &str| w.strip_prefix("said-").and_then(|n| n.parse::<u32>().ok());
|
||||
for (i, w) in words.iter().enumerate() {
|
||||
let Some(n) = number(w) else { continue };
|
||||
cited.insert(format!("said-{n:03}"));
|
||||
if words.get(i + 1) == Some(&"to")
|
||||
&& let Some(end) = words.get(i + 2).and_then(|w| number(w))
|
||||
{
|
||||
for k in n..=end {
|
||||
cited.insert(format!("said-{k:03}"));
|
||||
}
|
||||
}
|
||||
}
|
||||
if cited.is_empty() {
|
||||
return Err("the identity spec cites no SAID vector".into());
|
||||
}
|
||||
Ok(cited)
|
||||
}
|
||||
|
||||
/// The error classes the conformance README's table lists.
|
||||
pub fn readme_error_classes() -> Result<BTreeSet<String>, String> {
|
||||
let readme = repo_text("../../../conformance/README.md")?;
|
||||
let section = readme
|
||||
.split("### Error classes")
|
||||
.nth(1)
|
||||
.ok_or("the README has no error-class table")?;
|
||||
let classes: BTreeSet<String> = section
|
||||
.lines()
|
||||
.skip_while(|l| !l.starts_with('|'))
|
||||
.take_while(|l| l.starts_with('|'))
|
||||
.filter_map(|l| l.split('`').nth(1).map(str::to_owned))
|
||||
.collect();
|
||||
if classes.is_empty() {
|
||||
return Err("the README's error-class table is empty".into());
|
||||
}
|
||||
Ok(classes)
|
||||
}
|
||||
|
||||
/// Check every file of the pack in `dir`.
|
||||
pub fn verify_pack(dir: &Path) -> Report {
|
||||
let mut report = Report::default();
|
||||
let mut files: BTreeMap<&str, Vec<S>> = BTreeMap::new();
|
||||
|
||||
for file in FILES {
|
||||
let loaded = std::fs::read(dir.join(file))
|
||||
.map_err(|e| format!("read: {e}"))
|
||||
.and_then(|b| parse_strict(&b).map_err(|e| format!("parse: {e}")))
|
||||
.and_then(|j| match j {
|
||||
S::Array(items) => Ok(items),
|
||||
_ => Err("the top-level value must be an array".to_owned()),
|
||||
});
|
||||
match loaded {
|
||||
Ok(items) => {
|
||||
report.record(file, "(file)", structure(&items));
|
||||
files.insert(file, items);
|
||||
}
|
||||
Err(e) => report.record(file, "(file)", Err(e)),
|
||||
}
|
||||
}
|
||||
|
||||
let said_texts: BTreeSet<String> = files
|
||||
.get("said_vectors.json")
|
||||
.into_iter()
|
||||
.flatten()
|
||||
.filter_map(|v| {
|
||||
v.get("expected_said")
|
||||
.and_then(S::as_str)
|
||||
.map(str::to_owned)
|
||||
})
|
||||
.collect();
|
||||
|
||||
for (file, items) in &files {
|
||||
for v in items {
|
||||
let name = v.get("name").and_then(S::as_str).unwrap_or("(unnamed)");
|
||||
let result = match *file {
|
||||
"canonicalisation_vectors.json" => canonicalisation(v),
|
||||
"said_vectors.json" => said_vector(v),
|
||||
"signature_vectors.json" => signature(v, &said_texts),
|
||||
"lockstep_vectors.json" => lockstep(v),
|
||||
"gossip_vectors.json" => gossip(v),
|
||||
_ => unreachable!("FILES lists exactly these"),
|
||||
};
|
||||
report.record(file, name, result);
|
||||
}
|
||||
}
|
||||
|
||||
report.record(
|
||||
"(pack)",
|
||||
"spec-cited vectors present",
|
||||
cited_present(&files),
|
||||
);
|
||||
report.record(
|
||||
"(pack)",
|
||||
"README error classes exercised",
|
||||
classes_exercised(&files),
|
||||
);
|
||||
report
|
||||
}
|
||||
|
||||
/// Names and descriptions present, names unique, supersessions resolvable,
|
||||
/// and at least one vector.
|
||||
fn structure(items: &[S]) -> R {
|
||||
if items.is_empty() {
|
||||
return Err("the file holds no vectors".into());
|
||||
}
|
||||
let mut names = BTreeSet::new();
|
||||
for v in items {
|
||||
let name = text(v, "name")?;
|
||||
text(v, "description").map_err(|e| format!("{name}: {e}"))?;
|
||||
if !names.insert(name) {
|
||||
return Err(format!("duplicate vector name {name}"));
|
||||
}
|
||||
}
|
||||
for v in items {
|
||||
if let Some(target) = v.get("superseded_by").and_then(S::as_str)
|
||||
&& !names.contains(target)
|
||||
{
|
||||
return Err(format!(
|
||||
"{} is superseded by {target}, which is not in the file",
|
||||
v["name"]
|
||||
));
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn cited_present(files: &BTreeMap<&str, Vec<S>>) -> R {
|
||||
let names: Vec<&str> = files
|
||||
.get("said_vectors.json")
|
||||
.into_iter()
|
||||
.flatten()
|
||||
.filter_map(|v| v.get("name").and_then(S::as_str))
|
||||
.collect();
|
||||
let missing: Vec<String> = spec_cited_said_vectors()?
|
||||
.into_iter()
|
||||
.filter(|prefix| !names.iter().any(|n| n.starts_with(&format!("{prefix}-"))))
|
||||
.collect();
|
||||
if missing.is_empty() {
|
||||
Ok(())
|
||||
} else {
|
||||
Err(format!(
|
||||
"the identity spec cites vectors the pack lacks: {missing:?}"
|
||||
))
|
||||
}
|
||||
}
|
||||
|
||||
fn classes_exercised(files: &BTreeMap<&str, Vec<S>>) -> R {
|
||||
let mut used = BTreeSet::new();
|
||||
for v in files.values().flatten() {
|
||||
if let Some(e) = v.get("error").and_then(S::as_str) {
|
||||
used.insert(e.to_owned());
|
||||
}
|
||||
for r in v
|
||||
.get("expected_results")
|
||||
.and_then(S::as_array)
|
||||
.into_iter()
|
||||
.flatten()
|
||||
{
|
||||
if let Some(e) = r.get("error").and_then(S::as_str) {
|
||||
used.insert(e.to_owned());
|
||||
}
|
||||
}
|
||||
}
|
||||
let listed = readme_error_classes()?;
|
||||
let unexercised: Vec<&String> = listed.difference(&used).collect();
|
||||
let undocumented: Vec<&String> = used.difference(&listed).collect();
|
||||
if unexercised.is_empty() && undocumented.is_empty() {
|
||||
Ok(())
|
||||
} else {
|
||||
Err(format!(
|
||||
"README classes no vector exercises: {unexercised:?}; classes the README does not list: {undocumented:?}"
|
||||
))
|
||||
}
|
||||
}
|
||||
|
||||
fn refused_as(what: &str, class: Result<(), &'static str>, v: &S) -> R {
|
||||
let want = text(v, "error")?;
|
||||
match class {
|
||||
Ok(()) => Err(format!("{what} accepted input it must refuse with {want}")),
|
||||
Err(got) if got == want => Ok(()),
|
||||
Err(got) => Err(format!("refused with {got}, but the vector says {want}")),
|
||||
}
|
||||
}
|
||||
|
||||
fn canonicalisation(v: &S) -> R {
|
||||
if let Some(input) = v.get("input") {
|
||||
let value = from_json(input)?;
|
||||
let bytes = canonical::encode(&value).map_err(|e| e.to_string())?;
|
||||
same(
|
||||
"expected_cbor_hex",
|
||||
hex::encode(&bytes),
|
||||
text(v, "expected_cbor_hex")?,
|
||||
)?;
|
||||
let back = canonical::decode(&bytes)
|
||||
.map_err(|e| format!("decoder refuses the encoder's output: {e}"))?;
|
||||
if canonical::encode(&back).map_err(|e| e.to_string())? != bytes {
|
||||
return Err("decode then encode does not reproduce the bytes".into());
|
||||
}
|
||||
Ok(())
|
||||
} else {
|
||||
let input = hex_field(v, "input_cbor_hex")?;
|
||||
refused_as(
|
||||
"the decoder",
|
||||
canonical::decode(&input).map(drop).map_err(|e| e.class()),
|
||||
v,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
fn said_vector(v: &S) -> R {
|
||||
let field_name = text(v, "said_field")?;
|
||||
if let Some(input) = v.get("input") {
|
||||
let value = from_json(input)?;
|
||||
if value.get(field_name) != Some(&Value::text(SAID_PLACEHOLDER)) {
|
||||
return Err(format!(
|
||||
"`input` must hold the placeholder in `{field_name}` (identity spec §2.1 step 1)"
|
||||
));
|
||||
}
|
||||
let d = said::derive(&value, field_name).map_err(|e| e.to_string())?;
|
||||
same(
|
||||
"canonical_cbor_hex",
|
||||
hex::encode(&d.hashed),
|
||||
text(v, "canonical_cbor_hex")?,
|
||||
)?;
|
||||
same(
|
||||
"blake3_hex",
|
||||
hex::encode(d.said.as_bytes()),
|
||||
text(v, "blake3_hex")?,
|
||||
)?;
|
||||
same(
|
||||
"expected_said",
|
||||
d.said.to_said_text(),
|
||||
text(v, "expected_said")?,
|
||||
)?;
|
||||
same(
|
||||
"said_cbor_hex",
|
||||
hex::encode(&d.encoded),
|
||||
text(v, "said_cbor_hex")?,
|
||||
)?;
|
||||
let verified = said::verify(&hex_field(v, "said_cbor_hex")?, field_name)
|
||||
.map_err(|e| format!("said_cbor_hex does not verify: {e}"))?;
|
||||
same(
|
||||
"verified SAID",
|
||||
verified.to_said_text(),
|
||||
text(v, "expected_said")?,
|
||||
)
|
||||
} else {
|
||||
let input = hex_field(v, "input_cbor_hex")?;
|
||||
refused_as(
|
||||
"SAID verification",
|
||||
said::verify(&input, field_name)
|
||||
.map(drop)
|
||||
.map_err(|e| e.class()),
|
||||
v,
|
||||
)?;
|
||||
if let Some(received) = v.get("said_of_received_bytes").and_then(S::as_str) {
|
||||
same(
|
||||
"said_of_received_bytes",
|
||||
said_of_received_bytes(&input, field_name)?,
|
||||
received,
|
||||
)?;
|
||||
}
|
||||
if let Some(reencoded) = v.get("said_after_reencoding").and_then(S::as_str) {
|
||||
let value = lenient_decode(&input)?;
|
||||
let d = said::derive(&value, field_name).map_err(|e| e.to_string())?;
|
||||
same("said_after_reencoding", d.said.to_said_text(), reencoded)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// The SAID a verifier that skips the canonical check would compute: the
|
||||
/// received bytes with the stated SAID overwritten in place by the placeholder.
|
||||
fn said_of_received_bytes(input: &[u8], field_name: &str) -> Result<String, String> {
|
||||
let stated = match lenient_decode(input)?.get(field_name) {
|
||||
Some(Value::Text(t)) if t.len() == SAID_PLACEHOLDER.len() => t.clone(),
|
||||
_ => {
|
||||
return Err(format!(
|
||||
"no {}-character `{field_name}` to substitute",
|
||||
SAID_PLACEHOLDER.len()
|
||||
));
|
||||
}
|
||||
};
|
||||
let at = input
|
||||
.windows(stated.len())
|
||||
.position(|w| w == stated.as_bytes())
|
||||
.ok_or("the stated SAID is not in the bytes")?;
|
||||
let mut bytes = input.to_vec();
|
||||
bytes[at..at + stated.len()].copy_from_slice(SAID_PLACEHOLDER.as_bytes());
|
||||
Ok(Said::hash(&bytes).to_said_text())
|
||||
}
|
||||
|
||||
/// A decoder that accepts over-long heads and maps in any order — only so the
|
||||
/// verifier can show what a lenient implementation would compute. Nothing
|
||||
/// that decides acceptance uses it.
|
||||
fn lenient_decode(input: &[u8]) -> Result<Value, String> {
|
||||
fn item(b: &[u8], pos: &mut usize, depth: usize) -> Result<Value, String> {
|
||||
if depth > canonical::MAX_DEPTH {
|
||||
return Err("too deep".into());
|
||||
}
|
||||
let initial = *b.get(*pos).ok_or("truncated")?;
|
||||
*pos += 1;
|
||||
let (major, ai) = (initial >> 5, initial & 0x1f);
|
||||
if major == 7 {
|
||||
return match ai {
|
||||
20 => Ok(Value::Bool(false)),
|
||||
21 => Ok(Value::Bool(true)),
|
||||
22 => Ok(Value::Null),
|
||||
_ => Err(format!(
|
||||
"lenient decoder: unsupported simple/float {initial:#x}"
|
||||
)),
|
||||
};
|
||||
}
|
||||
let width = match ai {
|
||||
0..=23 => 0,
|
||||
24 => 1,
|
||||
25 => 2,
|
||||
26 => 4,
|
||||
27 => 8,
|
||||
_ => return Err(format!("lenient decoder: unsupported head {initial:#x}")),
|
||||
};
|
||||
let mut arg = if width == 0 { u64::from(ai) } else { 0 };
|
||||
for _ in 0..width {
|
||||
arg = (arg << 8) | u64::from(*b.get(*pos).ok_or("truncated")?);
|
||||
*pos += 1;
|
||||
}
|
||||
let mut take = |n: u64| -> Result<Vec<u8>, String> {
|
||||
let n = usize::try_from(n).map_err(|e| e.to_string())?;
|
||||
let end = pos.checked_add(n).ok_or("truncated")?;
|
||||
let s = b.get(*pos..end).ok_or("truncated")?.to_vec();
|
||||
*pos = end;
|
||||
Ok(s)
|
||||
};
|
||||
Ok(match major {
|
||||
0 => Value::Unsigned(arg),
|
||||
1 => Value::Negative(arg),
|
||||
2 => Value::Bytes(take(arg)?),
|
||||
3 => Value::Text(String::from_utf8(take(arg)?).map_err(|e| e.to_string())?),
|
||||
4 => {
|
||||
let mut items = Vec::new();
|
||||
for _ in 0..arg {
|
||||
items.push(item(b, pos, depth + 1)?);
|
||||
}
|
||||
Value::Array(items)
|
||||
}
|
||||
5 => {
|
||||
let mut entries = Vec::new();
|
||||
for _ in 0..arg {
|
||||
let k = item(b, pos, depth + 1)?;
|
||||
entries.push((k, item(b, pos, depth + 1)?));
|
||||
}
|
||||
Value::Map(entries)
|
||||
}
|
||||
_ => return Err(format!("lenient decoder: unsupported major type {major}")),
|
||||
})
|
||||
}
|
||||
let mut pos = 0;
|
||||
let v = item(input, &mut pos, 0)?;
|
||||
if pos != input.len() {
|
||||
return Err("trailing bytes".into());
|
||||
}
|
||||
Ok(v)
|
||||
}
|
||||
|
||||
fn signature(v: &S, said_texts: &BTreeSet<String>) -> R {
|
||||
let said_text = text(v, "envelope_said")?;
|
||||
if !said_texts.contains(said_text) {
|
||||
return Err(format!(
|
||||
"envelope_said {said_text} is not the expected_said of any SAID vector"
|
||||
));
|
||||
}
|
||||
let said = Said::from_said_text(said_text).map_err(|e| e.to_string())?;
|
||||
same(
|
||||
"signed_message_hex",
|
||||
hex::encode(said.as_bytes()),
|
||||
text(v, "signed_message_hex")?,
|
||||
)?;
|
||||
let key = match field(v, "signing_key_seed")? {
|
||||
S::Null => None,
|
||||
seed => {
|
||||
let seed: [u8; 32] = hash32(seed, "signing_key_seed")?;
|
||||
let key = SigningKey::from_bytes(&seed);
|
||||
same(
|
||||
"verifying_key",
|
||||
hex::encode(key.verifying_key().to_bytes()),
|
||||
text(v, "verifying_key")?,
|
||||
)?;
|
||||
Some(key)
|
||||
}
|
||||
};
|
||||
let sig = hex_field(v, "signature")?;
|
||||
let result = sign::verify_said_signature(said_text, &sig, &hex_field(v, "verifying_key")?);
|
||||
match field(v, "expected_verify")?.as_bool() {
|
||||
Some(true) => {
|
||||
result.map_err(|e| format!("a positive vector fails to verify: {e}"))?;
|
||||
if !field(v, "error")?.is_null() {
|
||||
return Err("a positive vector names an error".into());
|
||||
}
|
||||
// Ed25519 is deterministic: the published signature is THE
|
||||
// signature, so a positive vector must carry its seed.
|
||||
let key = key.ok_or("a positive vector must carry its signing seed")?;
|
||||
same(
|
||||
"signature",
|
||||
hex::encode(sign::sign_said(&said, &key).to_bytes()),
|
||||
text(v, "signature")?,
|
||||
)
|
||||
}
|
||||
Some(false) => refused_as("signature verification", result.map_err(|e| e.class()), v),
|
||||
None => Err("expected_verify is not a boolean".into()),
|
||||
}
|
||||
}
|
||||
|
||||
fn lockstep(v: &S) -> R {
|
||||
let n = field(v, "node_count")?
|
||||
.as_u64()
|
||||
.ok_or("node_count is not an integer")? as usize;
|
||||
let frame = field(v, "frame_number")?
|
||||
.as_u64()
|
||||
.ok_or("frame_number is not an integer")?;
|
||||
let reported_map = field(v, "node_reported_hashes")?
|
||||
.as_object()
|
||||
.ok_or("node_reported_hashes is not an object")?;
|
||||
let computed_map = field(v, "node_computed_hashes")?
|
||||
.as_object()
|
||||
.ok_or("node_computed_hashes is not an object")?;
|
||||
if reported_map.keys().ne(computed_map.keys()) {
|
||||
return Err("reported and computed hashes must name the same nodes".into());
|
||||
}
|
||||
let mut reported = Vec::new();
|
||||
for (node, h) in reported_map {
|
||||
reported.push((node.clone(), hash32(h, node)?));
|
||||
}
|
||||
let public = tally(n, &reported).map_err(|e| e.to_string())?;
|
||||
|
||||
same(
|
||||
"expected_threshold",
|
||||
supermajority_threshold(n).to_string(),
|
||||
&field(v, "expected_threshold")?.to_string(),
|
||||
)?;
|
||||
same(
|
||||
"expected_commit",
|
||||
public.committed().to_string(),
|
||||
&field(v, "expected_commit")?.to_string(),
|
||||
)?;
|
||||
let state = public.supermajority_hash.map(hex::encode);
|
||||
if field(v, "expected_state_hash")?.as_str() != state.as_deref() {
|
||||
return Err(format!("expected_state_hash: reference computes {state:?}"));
|
||||
}
|
||||
|
||||
let outcomes = field(v, "expected_outcomes")?
|
||||
.as_object()
|
||||
.ok_or("expected_outcomes is not an object")?;
|
||||
let logs = field(v, "expected_dissent_logs")?
|
||||
.as_object()
|
||||
.ok_or("expected_dissent_logs is not an object")?;
|
||||
if outcomes.len() != reported.len() || logs.len() != reported.len() {
|
||||
return Err("expected_outcomes and expected_dissent_logs must name every node".into());
|
||||
}
|
||||
for (node, computed) in computed_map {
|
||||
let computed = hash32(computed, node)?;
|
||||
let view = node_view(n, &reported, node, &computed).map_err(|e| e.to_string())?;
|
||||
let published = outcomes
|
||||
.get(node)
|
||||
.and_then(S::as_str)
|
||||
.ok_or_else(|| format!("no outcome for {node}"))?;
|
||||
same(
|
||||
&format!("{node}'s outcome"),
|
||||
view.outcome(&computed).as_str().to_owned(),
|
||||
published,
|
||||
)?;
|
||||
let log: Vec<S> = view
|
||||
.dissent_entry(frame, &computed)
|
||||
.into_iter()
|
||||
.map(|e| {
|
||||
serde_json::json!({
|
||||
"frame": e.frame,
|
||||
"dissenting_peers": e.dissenting_peers,
|
||||
"local_hash": hex::encode(e.local_hash),
|
||||
"supermajority_hash": hex::encode(e.supermajority_hash),
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
if logs.get(node) != Some(&S::Array(log.clone())) {
|
||||
return Err(format!(
|
||||
"{node}'s dissent log: reference computes {}",
|
||||
S::Array(log)
|
||||
));
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn gossip(v: &S) -> R {
|
||||
let deliveries = field(v, "deliveries")?
|
||||
.as_array()
|
||||
.ok_or("deliveries is not an array")?;
|
||||
let expected = field(v, "expected_results")?
|
||||
.as_array()
|
||||
.ok_or("expected_results is not an array")?;
|
||||
if deliveries.len() != expected.len() {
|
||||
return Err("one expected result per delivery".into());
|
||||
}
|
||||
let mut inbox = Inbox::new();
|
||||
for (i, (d, want)) in deliveries.iter().zip(expected).enumerate() {
|
||||
let got = match inbox.deliver(text(d, "claimed_said")?, &hex_field(d, "cbor_hex")?) {
|
||||
Delivery::Accepted(s) => {
|
||||
serde_json::json!({"outcome": "accepted", "said": s.to_said_text()})
|
||||
}
|
||||
Delivery::Duplicate(s) => {
|
||||
serde_json::json!({"outcome": "duplicate", "said": s.to_said_text()})
|
||||
}
|
||||
Delivery::Rejected(e) => {
|
||||
serde_json::json!({"outcome": "rejected", "error": e.class()})
|
||||
}
|
||||
};
|
||||
if &got != want {
|
||||
return Err(format!(
|
||||
"delivery {i}: published {want}, reference computes {got}"
|
||||
));
|
||||
}
|
||||
}
|
||||
let held: Vec<S> = inbox
|
||||
.held()
|
||||
.iter()
|
||||
.map(|(s, b)| serde_json::json!({"said": s.to_said_text(), "cbor_hex": hex::encode(b)}))
|
||||
.collect();
|
||||
if field(v, "expected_held")? != &S::Array(held.clone()) {
|
||||
return Err(format!("expected_held: reference holds {}", S::Array(held)));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
|
@ -0,0 +1,379 @@
|
|||
//! The published pack, `smart-byte/conformance/v1`, held to the reference
|
||||
//! implementation under `cargo test` — so CI fails whenever the two disagree.
|
||||
|
||||
use std::collections::BTreeSet;
|
||||
use std::path::Path;
|
||||
|
||||
use serde_json::{Value, json};
|
||||
use smart_byte_conformance::verify::{readme_error_classes, spec_cited_said_vectors};
|
||||
use smart_byte_conformance::{pack, published_pack_dir, verify::verify_pack};
|
||||
|
||||
fn published(file: &str) -> String {
|
||||
std::fs::read_to_string(published_pack_dir().join(file))
|
||||
.unwrap_or_else(|e| panic!("read {file}: {e}"))
|
||||
}
|
||||
|
||||
fn vectors(file: &str) -> Vec<Value> {
|
||||
serde_json::from_str::<Value>(&published(file))
|
||||
.unwrap()
|
||||
.as_array()
|
||||
.unwrap()
|
||||
.clone()
|
||||
}
|
||||
|
||||
fn by_name(file: &str, name: &str) -> Value {
|
||||
vectors(file)
|
||||
.into_iter()
|
||||
.find(|v| v["name"] == name)
|
||||
.unwrap_or_else(|| panic!("{file} has no vector {name}"))
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_published_vector_agrees_with_the_reference() {
|
||||
let report = verify_pack(&published_pack_dir());
|
||||
let failures = report.failures();
|
||||
assert!(failures.is_empty(), "{failures:#?}");
|
||||
// One structure check per file, one check per vector, two completeness
|
||||
// checks: none skipped.
|
||||
let total: usize = pack::FILES.iter().map(|f| vectors(f).len()).sum();
|
||||
assert_eq!(report.checks.len(), pack::FILES.len() + total + 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_published_pack_is_exactly_what_the_reference_generates() {
|
||||
for (file, generated) in pack::generate() {
|
||||
assert!(
|
||||
published(file) == generated,
|
||||
"{file} differs from the reference implementation's output; regenerate with \
|
||||
`cargo run -p smart-byte-conformance --bin gen-conformance-vectors`"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Write the published pack to `dir` with `edit` applied to the files'
|
||||
/// parsed contents, and return the names of the checks the verifier fails.
|
||||
fn failures_with(dir: &Path, edit: impl Fn(&str, &mut Vec<Value>)) -> Vec<String> {
|
||||
std::fs::create_dir_all(dir).unwrap();
|
||||
for f in pack::FILES {
|
||||
let mut items = vectors(f);
|
||||
edit(f, &mut items);
|
||||
std::fs::write(dir.join(f), serde_json::to_string_pretty(&items).unwrap()).unwrap();
|
||||
}
|
||||
verify_pack(dir)
|
||||
.failures()
|
||||
.into_iter()
|
||||
.map(|c| c.vector.clone())
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn scratch(name: &str) -> std::path::PathBuf {
|
||||
Path::new(env!("CARGO_TARGET_TMPDIR")).join(name)
|
||||
}
|
||||
|
||||
/// Falsification: the verifier is not vacuous. Each planted defect — a wrong
|
||||
/// value, a wrong error class, a negative vector turned positive — must be
|
||||
/// caught, and caught on the vector it was planted in.
|
||||
#[test]
|
||||
fn the_verifier_catches_a_planted_defect_in_every_file() {
|
||||
let flip_last = |s: &str| {
|
||||
let mut s = s.to_owned();
|
||||
let last = s.pop().unwrap();
|
||||
s.push(if last == '0' { '1' } else { '0' });
|
||||
s
|
||||
};
|
||||
type Plant = (&'static str, &'static str, Box<dyn Fn(&mut Value)>);
|
||||
let plants: Vec<Plant> = vec![
|
||||
(
|
||||
"canonicalisation_vectors.json",
|
||||
"canon-001-sorted-map-keys",
|
||||
Box::new(move |v| {
|
||||
v["expected_cbor_hex"] = json!(flip_last(v["expected_cbor_hex"].as_str().unwrap()))
|
||||
}),
|
||||
),
|
||||
(
|
||||
"canonicalisation_vectors.json",
|
||||
"canon-012-text-ordered-keys-refused",
|
||||
Box::new(|v| v["error"] = json!("duplicate_map_key")),
|
||||
),
|
||||
(
|
||||
"canonicalisation_vectors.json",
|
||||
"canon-004-floats-forbidden",
|
||||
Box::new(|v| v["input_cbor_hex"] = json!("a16a65786163745f68616c6601")),
|
||||
),
|
||||
(
|
||||
"canonicalisation_vectors.json",
|
||||
"canon-020-simple-value-refused",
|
||||
Box::new(|v| v["input_cbor_hex"] = json!("f6")),
|
||||
),
|
||||
(
|
||||
"said_vectors.json",
|
||||
"said-008-spec-worked-example",
|
||||
Box::new(move |v| {
|
||||
v["expected_said"] = json!(flip_last(v["expected_said"].as_str().unwrap()))
|
||||
}),
|
||||
),
|
||||
(
|
||||
"said_vectors.json",
|
||||
"said-009-non-canonical-copy-rejected",
|
||||
Box::new(|v| v["error"] = json!("said_mismatch")),
|
||||
),
|
||||
(
|
||||
"said_vectors.json",
|
||||
"said-010-self-consistent-non-canonical-rejected",
|
||||
Box::new(move |v| {
|
||||
v["said_of_received_bytes"] =
|
||||
json!(flip_last(v["said_of_received_bytes"].as_str().unwrap()))
|
||||
}),
|
||||
),
|
||||
(
|
||||
"said_vectors.json",
|
||||
"said-005-with-ownership-transitions",
|
||||
Box::new(|v| v["input"]["joule_cost"]["measured_microjoules"] = json!(48)),
|
||||
),
|
||||
(
|
||||
"signature_vectors.json",
|
||||
"sig-001-known-key-known-said",
|
||||
Box::new(move |v| v["signature"] = json!(flip_last(v["signature"].as_str().unwrap()))),
|
||||
),
|
||||
(
|
||||
"signature_vectors.json",
|
||||
"sig-003-tampered-envelope",
|
||||
Box::new(|v| {
|
||||
v["expected_verify"] = json!(true);
|
||||
v["error"] = Value::Null;
|
||||
}),
|
||||
),
|
||||
(
|
||||
"signature_vectors.json",
|
||||
"sig-006-non-canonical-s-rejected",
|
||||
Box::new(|v| v["error"] = json!("malformed_signature")),
|
||||
),
|
||||
(
|
||||
"signature_vectors.json",
|
||||
"sig-007-non-canonical-key-rejected",
|
||||
// The canonical spelling of the same point: now a well-formed key
|
||||
// that merely fails to verify, so the published class is wrong.
|
||||
Box::new(|v| {
|
||||
v["verifying_key"] =
|
||||
json!("0300000000000000000000000000000000000000000000000000000000000000")
|
||||
}),
|
||||
),
|
||||
(
|
||||
"lockstep_vectors.json",
|
||||
"lockstep-002-supermajority-commit",
|
||||
Box::new(|v| v["expected_dissent_logs"]["node_1"] = json!([])),
|
||||
),
|
||||
(
|
||||
"lockstep_vectors.json",
|
||||
"lockstep-004-three-nodes-one-diverged-split",
|
||||
Box::new(|v| v["expected_commit"] = json!(true)),
|
||||
),
|
||||
(
|
||||
"lockstep_vectors.json",
|
||||
"lockstep-002-supermajority-commit",
|
||||
Box::new(|v| v["expected_outcomes"]["node_4"] = json!("commit_with_dissent")),
|
||||
),
|
||||
(
|
||||
"lockstep_vectors.json",
|
||||
"lockstep-006-transport-liar-is-logged",
|
||||
Box::new(|v| v["expected_outcomes"]["node_4"] = json!("minority_divergence")),
|
||||
),
|
||||
(
|
||||
"gossip_vectors.json",
|
||||
"gossip-002-said-conflict-rejected",
|
||||
Box::new(|v| {
|
||||
v["expected_results"][1] =
|
||||
json!({"outcome": "duplicate", "said": v["expected_results"][0]["said"]})
|
||||
}),
|
||||
),
|
||||
(
|
||||
"gossip_vectors.json",
|
||||
"gossip-005-non-canonical-copy-rejected",
|
||||
Box::new(|v| {
|
||||
let s = v["deliveries"][0]["claimed_said"].clone();
|
||||
let b = v["deliveries"][0]["cbor_hex"].clone();
|
||||
v["expected_results"][0] = json!({"outcome": "accepted", "said": s});
|
||||
v["expected_held"] = json!([{"said": s, "cbor_hex": b}]);
|
||||
}),
|
||||
),
|
||||
(
|
||||
"gossip_vectors.json",
|
||||
"gossip-001-cross-node-propagation",
|
||||
Box::new(move |v| {
|
||||
let h = v["expected_held"][0]["cbor_hex"]
|
||||
.as_str()
|
||||
.unwrap()
|
||||
.to_owned();
|
||||
v["expected_held"][0]["cbor_hex"] = json!(flip_last(&h));
|
||||
}),
|
||||
),
|
||||
];
|
||||
for (i, (file, name, mutate)) in plants.iter().enumerate() {
|
||||
let failed = failures_with(&scratch(&format!("planted/{i}")), |f, items| {
|
||||
if f == *file {
|
||||
mutate(items.iter_mut().find(|v| v["name"] == *name).expect(name));
|
||||
}
|
||||
});
|
||||
assert_eq!(
|
||||
failed,
|
||||
vec![name.to_string()],
|
||||
"planted defect {i} in {file}/{name}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Falsification: a pack cannot pass by losing vectors.
|
||||
#[test]
|
||||
fn an_incomplete_pack_fails() {
|
||||
// Every file emptied.
|
||||
let failed = failures_with(&scratch("incomplete/empty"), |_, items| items.clear());
|
||||
for f in pack::FILES {
|
||||
assert!(failed.contains(&"(file)".to_string()), "{f}");
|
||||
}
|
||||
assert!(failed.contains(&"spec-cited vectors present".to_string()));
|
||||
assert!(failed.contains(&"README error classes exercised".to_string()));
|
||||
|
||||
// One vector the spec cites, deleted.
|
||||
let failed = failures_with(&scratch("incomplete/said-010"), |f, items| {
|
||||
if f == "said_vectors.json" {
|
||||
items.retain(|v| !v["name"].as_str().unwrap().starts_with("said-010-"));
|
||||
}
|
||||
});
|
||||
assert_eq!(failed, vec!["spec-cited vectors present".to_string()]);
|
||||
|
||||
// The only vector exercising a README error class, deleted.
|
||||
let failed = failures_with(&scratch("incomplete/class"), |f, items| {
|
||||
if f == "canonicalisation_vectors.json" {
|
||||
items.retain(|v| v["name"] != "canon-020-simple-value-refused");
|
||||
}
|
||||
});
|
||||
assert_eq!(failed, vec!["README error classes exercised".to_string()]);
|
||||
}
|
||||
|
||||
/// Falsification: a vector that names an error class the README does not
|
||||
/// list is caught, so the README stays the full list.
|
||||
#[test]
|
||||
fn an_undocumented_error_class_fails() {
|
||||
let failed = failures_with(&scratch("undocumented"), |f, items| {
|
||||
if f == "canonicalisation_vectors.json" {
|
||||
let mut extra = items
|
||||
.iter()
|
||||
.find(|v| v["name"] == "canon-018-trailing-bytes-refused")
|
||||
.unwrap()
|
||||
.clone();
|
||||
extra["name"] = json!("canon-099-made-up");
|
||||
extra["error"] = json!("made_up_class");
|
||||
items.push(extra);
|
||||
}
|
||||
});
|
||||
assert!(failed.contains(&"README error classes exercised".to_string()));
|
||||
assert!(failed.contains(&"canon-099-made-up".to_string()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_pack_with_a_duplicated_json_member_fails() {
|
||||
let dir = scratch("duplicate-member");
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
for f in pack::FILES {
|
||||
let mut body = published(f);
|
||||
if f == "lockstep_vectors.json" {
|
||||
// Name a member twice; a plain JSON parser would keep the last.
|
||||
body = body.replacen(
|
||||
"\"node_count\": 4,",
|
||||
"\"node_count\": 4,\n \"node_count\": 4,",
|
||||
1,
|
||||
);
|
||||
}
|
||||
std::fs::write(dir.join(f), body).unwrap();
|
||||
}
|
||||
let report = verify_pack(&dir);
|
||||
let failed: Vec<&str> = report.failures().iter().map(|c| c.file.as_str()).collect();
|
||||
assert_eq!(failed, vec!["lockstep_vectors.json"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_said_pack_holds_the_suite_the_spec_names() {
|
||||
let cited = spec_cited_said_vectors().unwrap();
|
||||
for n in 1..=11 {
|
||||
assert!(
|
||||
cited.contains(&format!("said-{n:03}")),
|
||||
"the spec no longer cites said-{n:03}"
|
||||
);
|
||||
}
|
||||
let keys = |name: &str| -> BTreeSet<String> {
|
||||
by_name("said_vectors.json", name)["input"]
|
||||
.as_object()
|
||||
.unwrap()
|
||||
.keys()
|
||||
.cloned()
|
||||
.collect()
|
||||
};
|
||||
let set = |ks: &[&str]| ks.iter().map(|k| k.to_string()).collect::<BTreeSet<_>>();
|
||||
assert_eq!(keys("said-006-only-said-field"), set(&["d"]));
|
||||
assert_eq!(keys("said-007-one-field-per-key"), set(&["d", "k"]));
|
||||
assert_eq!(keys("said-008-spec-worked-example"), set(&["d", "n"]));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_canonical_rule_has_a_negative_vector() {
|
||||
let classes: BTreeSet<String> = vectors("canonicalisation_vectors.json")
|
||||
.into_iter()
|
||||
.filter_map(|v| v["error"].as_str().map(str::to_owned))
|
||||
.collect();
|
||||
for rule in [
|
||||
"non_shortest_integer",
|
||||
"non_shortest_length",
|
||||
"unsorted_map_keys",
|
||||
"duplicate_map_key",
|
||||
"float_forbidden",
|
||||
"simple_value_forbidden",
|
||||
"tag_forbidden",
|
||||
"indefinite_length",
|
||||
"invalid_utf8",
|
||||
"trailing_bytes",
|
||||
] {
|
||||
assert!(
|
||||
classes.contains(rule),
|
||||
"no canonicalisation vector is refused with {rule}"
|
||||
);
|
||||
assert!(
|
||||
readme_error_classes().unwrap().contains(rule),
|
||||
"README omits {rule}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_spec_prints_the_worked_example_the_pack_computes() {
|
||||
let spec = std::fs::read_to_string(
|
||||
Path::new(env!("CARGO_MANIFEST_DIR")).join("../../../spec/identity_and_key_rotation.md"),
|
||||
)
|
||||
.unwrap();
|
||||
let example = by_name("said_vectors.json", "said-008-spec-worked-example");
|
||||
for field in ["blake3_hex", "expected_said", "canonical_cbor_hex"] {
|
||||
let value = example[field].as_str().unwrap();
|
||||
assert!(
|
||||
spec.contains(value),
|
||||
"identity spec §2.4 does not print the pack's {field} {value}"
|
||||
);
|
||||
}
|
||||
assert!(
|
||||
spec.contains("total 61 bytes"),
|
||||
"§2.4 must give the encoding's real length"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn signatures_sign_saids_the_said_pack_derives() {
|
||||
let saids: BTreeSet<String> = vectors("said_vectors.json")
|
||||
.into_iter()
|
||||
.filter_map(|v| v["expected_said"].as_str().map(str::to_owned))
|
||||
.collect();
|
||||
for v in vectors("signature_vectors.json") {
|
||||
assert!(
|
||||
saids.contains(v["envelope_said"].as_str().unwrap()),
|
||||
"{}",
|
||||
v["name"]
|
||||
);
|
||||
}
|
||||
}
|
||||
555
smart-byte/smart-byte-rs/crates/smart-byte-core/src/canonical.rs
Normal file
555
smart-byte/smart-byte-rs/crates/smart-byte-core/src/canonical.rs
Normal file
|
|
@ -0,0 +1,555 @@
|
|||
//! Canonical CBOR — the encoding every SAID is computed over.
|
||||
//!
|
||||
//! Identity spec §2.3 fixes one byte form per value: RFC 8949 §4.2.1 core
|
||||
//! deterministic encoding, with floats, tags, indefinite lengths and simple
|
||||
//! values other than `false`, `true` and `null` forbidden.
|
||||
//! [`encode`] produces that form from a [`Value`]. [`decode`] accepts **only**
|
||||
//! that form: a strict decoder that refuses non-canonical bytes instead of
|
||||
//! re-encoding them, which is the property the spec's adversarial SAID vector
|
||||
//! (§2.5) exists to test. Re-encoding would make a non-canonical copy of an
|
||||
//! object hash to the canonical SAID, so two different byte strings would both
|
||||
//! verify under one identifier.
|
||||
//!
|
||||
//! The rules, and the error class [`decode`] reports for each:
|
||||
//!
|
||||
//! | rule (§2.3 / RFC 8949) | class |
|
||||
//! | -------------------------------------------------------- | ---------------------- |
|
||||
//! | integers use the shortest head | `non_shortest_integer` |
|
||||
//! | string, array and map lengths use the shortest head | `non_shortest_length` |
|
||||
//! | map keys sorted by their encoded bytes, not their text | `unsorted_map_keys` |
|
||||
//! | no key twice in one map | `duplicate_map_key` |
|
||||
//! | no floats | `float_forbidden` |
|
||||
//! | no simple values but `false`, `true`, `null` | `simple_value_forbidden` |
|
||||
//! | no tags | `tag_forbidden` |
|
||||
//! | no indefinite-length items | `indefinite_length` |
|
||||
//! | text strings are UTF-8 | `invalid_utf8` |
|
||||
//! | the input is exactly one item | `trailing_bytes` |
|
||||
//!
|
||||
//! Inputs that are not well-formed CBOR at all report `truncated` or
|
||||
//! `malformed`. Nesting deeper than [`MAX_DEPTH`] reports `too_deep`. That
|
||||
//! limit is this implementation's guard against unbounded recursion, not a
|
||||
//! canonical-form rule.
|
||||
//!
|
||||
//! An input with several defects reports the first one in stream order, so
|
||||
//! every conforming decoder names the same class for it.
|
||||
//!
|
||||
//! The decoder never allocates from a declared count. Arrays and maps grow as
|
||||
//! their items actually decode, so memory stays proportional to the input
|
||||
//! however deeply a hostile input nests oversized counts.
|
||||
|
||||
use std::cmp::Ordering;
|
||||
|
||||
/// How deeply [`decode`] lets arrays and maps nest before refusing the input.
|
||||
pub const MAX_DEPTH: usize = 128;
|
||||
|
||||
/// A value in the Smart Byte data model: the CBOR values canonical encoding
|
||||
/// admits. Floats, tags and other simple values have no variant because
|
||||
/// identity spec §2.3 forbids them.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub enum Value {
|
||||
/// Major type 0: `0 ..= u64::MAX`.
|
||||
Unsigned(u64),
|
||||
/// Major type 1: the integer `-1 - n`, so `Negative(0)` is `-1`.
|
||||
Negative(u64),
|
||||
/// Major type 2.
|
||||
Bytes(Vec<u8>),
|
||||
/// Major type 3.
|
||||
Text(String),
|
||||
/// Major type 4.
|
||||
Array(Vec<Value>),
|
||||
/// Major type 5. Entries may be held in any order; [`encode`] sorts them.
|
||||
Map(Vec<(Value, Value)>),
|
||||
/// Simple values 20 and 21.
|
||||
Bool(bool),
|
||||
/// Simple value 22.
|
||||
Null,
|
||||
}
|
||||
|
||||
impl Value {
|
||||
/// A signed integer, placed in major type 0 or 1 as its sign requires.
|
||||
pub fn int(i: i64) -> Self {
|
||||
if i >= 0 {
|
||||
Value::Unsigned(i as u64)
|
||||
} else {
|
||||
// -1 - n = i => n = -1 - i, which cannot overflow for i < 0.
|
||||
Value::Negative((-1 - i) as u64)
|
||||
}
|
||||
}
|
||||
|
||||
/// A text value.
|
||||
pub fn text(s: impl Into<String>) -> Self {
|
||||
Value::Text(s.into())
|
||||
}
|
||||
|
||||
/// A map from text keys, in the order given.
|
||||
pub fn map<K: Into<String>>(entries: impl IntoIterator<Item = (K, Value)>) -> Self {
|
||||
Value::Map(
|
||||
entries
|
||||
.into_iter()
|
||||
.map(|(k, v)| (Value::Text(k.into()), v))
|
||||
.collect(),
|
||||
)
|
||||
}
|
||||
|
||||
/// The value held under text key `key`, if this is a map holding one.
|
||||
pub fn get(&self, key: &str) -> Option<&Value> {
|
||||
match self {
|
||||
Value::Map(entries) => entries
|
||||
.iter()
|
||||
.find(|(k, _)| matches!(k, Value::Text(t) if t == key))
|
||||
.map(|(_, v)| v),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Mutable access to the value held under text key `key`.
|
||||
pub fn get_mut(&mut self, key: &str) -> Option<&mut Value> {
|
||||
match self {
|
||||
Value::Map(entries) => entries
|
||||
.iter_mut()
|
||||
.find(|(k, _)| matches!(k, Value::Text(t) if t == key))
|
||||
.map(|(_, v)| v),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Why an encoding was refused. [`CanonicalError::class`] gives the stable
|
||||
/// name the conformance pack uses for each.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, thiserror::Error)]
|
||||
pub enum CanonicalError {
|
||||
#[error("input ends inside an item")]
|
||||
Truncated,
|
||||
#[error("bytes follow the top-level item")]
|
||||
TrailingBytes,
|
||||
#[error("not well-formed CBOR")]
|
||||
Malformed,
|
||||
#[error("integer not in its shortest form")]
|
||||
NonShortestInteger,
|
||||
#[error("length not in its shortest form")]
|
||||
NonShortestLength,
|
||||
#[error("indefinite-length item")]
|
||||
IndefiniteLength,
|
||||
#[error("CBOR tags are forbidden")]
|
||||
TagForbidden,
|
||||
#[error("floating-point values are forbidden")]
|
||||
FloatForbidden,
|
||||
#[error("simple values other than false, true and null are forbidden")]
|
||||
SimpleValueForbidden,
|
||||
#[error("map keys are not in canonical order")]
|
||||
UnsortedMapKeys,
|
||||
#[error("a map key appears twice")]
|
||||
DuplicateMapKey,
|
||||
#[error("text string is not valid UTF-8")]
|
||||
InvalidUtf8,
|
||||
#[error("nested deeper than the decoder allows")]
|
||||
TooDeep,
|
||||
}
|
||||
|
||||
impl CanonicalError {
|
||||
/// The error class as the conformance pack names it.
|
||||
pub fn class(&self) -> &'static str {
|
||||
match self {
|
||||
CanonicalError::Truncated => "truncated",
|
||||
CanonicalError::TrailingBytes => "trailing_bytes",
|
||||
CanonicalError::Malformed => "malformed",
|
||||
CanonicalError::NonShortestInteger => "non_shortest_integer",
|
||||
CanonicalError::NonShortestLength => "non_shortest_length",
|
||||
CanonicalError::IndefiniteLength => "indefinite_length",
|
||||
CanonicalError::TagForbidden => "tag_forbidden",
|
||||
CanonicalError::FloatForbidden => "float_forbidden",
|
||||
CanonicalError::SimpleValueForbidden => "simple_value_forbidden",
|
||||
CanonicalError::UnsortedMapKeys => "unsorted_map_keys",
|
||||
CanonicalError::DuplicateMapKey => "duplicate_map_key",
|
||||
CanonicalError::InvalidUtf8 => "invalid_utf8",
|
||||
CanonicalError::TooDeep => "too_deep",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Encode `value` canonically.
|
||||
///
|
||||
/// Fails only for a value with no canonical form: a map holding the same key
|
||||
/// twice (`duplicate_map_key`).
|
||||
pub fn encode(value: &Value) -> Result<Vec<u8>, CanonicalError> {
|
||||
let mut out = Vec::new();
|
||||
encode_into(value, &mut out)?;
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn head(out: &mut Vec<u8>, major: u8, arg: u64) {
|
||||
let m = major << 5;
|
||||
if arg < 24 {
|
||||
out.push(m | arg as u8);
|
||||
} else if arg <= 0xff {
|
||||
out.push(m | 24);
|
||||
out.push(arg as u8);
|
||||
} else if arg <= 0xffff {
|
||||
out.push(m | 25);
|
||||
out.extend_from_slice(&(arg as u16).to_be_bytes());
|
||||
} else if arg <= 0xffff_ffff {
|
||||
out.push(m | 26);
|
||||
out.extend_from_slice(&(arg as u32).to_be_bytes());
|
||||
} else {
|
||||
out.push(m | 27);
|
||||
out.extend_from_slice(&arg.to_be_bytes());
|
||||
}
|
||||
}
|
||||
|
||||
fn encode_into(value: &Value, out: &mut Vec<u8>) -> Result<(), CanonicalError> {
|
||||
match value {
|
||||
Value::Unsigned(n) => head(out, 0, *n),
|
||||
Value::Negative(n) => head(out, 1, *n),
|
||||
Value::Bytes(b) => {
|
||||
head(out, 2, b.len() as u64);
|
||||
out.extend_from_slice(b);
|
||||
}
|
||||
Value::Text(t) => {
|
||||
head(out, 3, t.len() as u64);
|
||||
out.extend_from_slice(t.as_bytes());
|
||||
}
|
||||
Value::Array(items) => {
|
||||
head(out, 4, items.len() as u64);
|
||||
for item in items {
|
||||
encode_into(item, out)?;
|
||||
}
|
||||
}
|
||||
Value::Map(entries) => {
|
||||
let mut encoded = Vec::with_capacity(entries.len());
|
||||
for (k, v) in entries {
|
||||
encoded.push((encode(k)?, encode(v)?));
|
||||
}
|
||||
// §2.3: "sorted lexicographically by their canonical CBOR byte
|
||||
// representation, *not* by the corresponding string".
|
||||
encoded.sort_by(|a, b| a.0.cmp(&b.0));
|
||||
if encoded.windows(2).any(|w| w[0].0 == w[1].0) {
|
||||
return Err(CanonicalError::DuplicateMapKey);
|
||||
}
|
||||
head(out, 5, encoded.len() as u64);
|
||||
for (k, v) in encoded {
|
||||
out.extend_from_slice(&k);
|
||||
out.extend_from_slice(&v);
|
||||
}
|
||||
}
|
||||
Value::Bool(false) => out.push(0xf4),
|
||||
Value::Bool(true) => out.push(0xf5),
|
||||
Value::Null => out.push(0xf6),
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Decode `bytes`, accepting only the canonical encoding of exactly one value.
|
||||
pub fn decode(bytes: &[u8]) -> Result<Value, CanonicalError> {
|
||||
let mut d = Decoder { buf: bytes, pos: 0 };
|
||||
let value = d.item(0)?;
|
||||
if d.pos != bytes.len() {
|
||||
return Err(CanonicalError::TrailingBytes);
|
||||
}
|
||||
Ok(value)
|
||||
}
|
||||
|
||||
struct Decoder<'a> {
|
||||
buf: &'a [u8],
|
||||
pos: usize,
|
||||
}
|
||||
|
||||
impl<'a> Decoder<'a> {
|
||||
fn take(&mut self, n: usize) -> Result<&'a [u8], CanonicalError> {
|
||||
let end = self.pos.checked_add(n).ok_or(CanonicalError::Truncated)?;
|
||||
let s = self
|
||||
.buf
|
||||
.get(self.pos..end)
|
||||
.ok_or(CanonicalError::Truncated)?;
|
||||
self.pos = end;
|
||||
Ok(s)
|
||||
}
|
||||
|
||||
fn byte(&mut self) -> Result<u8, CanonicalError> {
|
||||
Ok(self.take(1)?[0])
|
||||
}
|
||||
|
||||
/// The argument of a head whose additional information is `ai`, refusing
|
||||
/// any head longer than the argument needs.
|
||||
fn argument(&mut self, ai: u8, non_shortest: CanonicalError) -> Result<u64, CanonicalError> {
|
||||
let (arg, floor) = match ai {
|
||||
0..=23 => return Ok(u64::from(ai)),
|
||||
24 => (u64::from(self.byte()?), 24),
|
||||
25 => (
|
||||
u64::from(u16::from_be_bytes(self.take(2)?.try_into().unwrap())),
|
||||
0x100,
|
||||
),
|
||||
26 => (
|
||||
u64::from(u32::from_be_bytes(self.take(4)?.try_into().unwrap())),
|
||||
0x1_0000,
|
||||
),
|
||||
27 => (
|
||||
u64::from_be_bytes(self.take(8)?.try_into().unwrap()),
|
||||
0x1_0000_0000,
|
||||
),
|
||||
_ => unreachable!("additional information 28..=31 is handled by the caller"),
|
||||
};
|
||||
if arg < floor {
|
||||
Err(non_shortest)
|
||||
} else {
|
||||
Ok(arg)
|
||||
}
|
||||
}
|
||||
|
||||
/// A string length or item count. It never sizes an allocation: strings
|
||||
/// are sliced from the input, which bounds them, and containers grow item
|
||||
/// by item, so the first defect in stream order is the one reported.
|
||||
fn length(&mut self, ai: u8) -> Result<usize, CanonicalError> {
|
||||
let n = self.argument(ai, CanonicalError::NonShortestLength)?;
|
||||
usize::try_from(n).map_err(|_| CanonicalError::Truncated)
|
||||
}
|
||||
|
||||
fn item(&mut self, depth: usize) -> Result<Value, CanonicalError> {
|
||||
if depth > MAX_DEPTH {
|
||||
return Err(CanonicalError::TooDeep);
|
||||
}
|
||||
let initial = self.byte()?;
|
||||
let major = initial >> 5;
|
||||
let ai = initial & 0x1f;
|
||||
|
||||
if major == 7 {
|
||||
return match ai {
|
||||
20 => Ok(Value::Bool(false)),
|
||||
21 => Ok(Value::Bool(true)),
|
||||
22 => Ok(Value::Null),
|
||||
0..=19 | 23 => Err(CanonicalError::SimpleValueForbidden),
|
||||
24 => match self.byte()? {
|
||||
// RFC 8949 §3.3: a two-byte simple value below 32 is not
|
||||
// well-formed (it would have a one-byte encoding).
|
||||
n if n < 32 => Err(CanonicalError::Malformed),
|
||||
_ => Err(CanonicalError::SimpleValueForbidden),
|
||||
},
|
||||
25..=27 => Err(CanonicalError::FloatForbidden),
|
||||
// 28..=30 are reserved; 31 is a "break" with nothing to end.
|
||||
_ => Err(CanonicalError::Malformed),
|
||||
};
|
||||
}
|
||||
if major == 6 {
|
||||
return Err(CanonicalError::TagForbidden);
|
||||
}
|
||||
match ai {
|
||||
28..=30 => return Err(CanonicalError::Malformed),
|
||||
31 => {
|
||||
return Err(if major <= 1 {
|
||||
CanonicalError::Malformed
|
||||
} else {
|
||||
CanonicalError::IndefiniteLength
|
||||
});
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
|
||||
match major {
|
||||
0 => Ok(Value::Unsigned(
|
||||
self.argument(ai, CanonicalError::NonShortestInteger)?,
|
||||
)),
|
||||
1 => Ok(Value::Negative(
|
||||
self.argument(ai, CanonicalError::NonShortestInteger)?,
|
||||
)),
|
||||
2 => {
|
||||
let n = self.length(ai)?;
|
||||
Ok(Value::Bytes(self.take(n)?.to_vec()))
|
||||
}
|
||||
3 => {
|
||||
let n = self.length(ai)?;
|
||||
let raw = self.take(n)?;
|
||||
let s = std::str::from_utf8(raw).map_err(|_| CanonicalError::InvalidUtf8)?;
|
||||
Ok(Value::Text(s.to_owned()))
|
||||
}
|
||||
4 => {
|
||||
let n = self.length(ai)?;
|
||||
let mut items = Vec::new();
|
||||
for _ in 0..n {
|
||||
items.push(self.item(depth + 1)?);
|
||||
}
|
||||
Ok(Value::Array(items))
|
||||
}
|
||||
5 => {
|
||||
let n = self.length(ai)?;
|
||||
let mut entries = Vec::new();
|
||||
let mut previous_key: Option<&'a [u8]> = None;
|
||||
for _ in 0..n {
|
||||
let start = self.pos;
|
||||
let key = self.item(depth + 1)?;
|
||||
// The decoder is strict, so the bytes it just consumed ARE
|
||||
// the key's canonical encoding: order is checked on those.
|
||||
let key_bytes = &self.buf[start..self.pos];
|
||||
if let Some(prev) = previous_key {
|
||||
match prev.cmp(key_bytes) {
|
||||
Ordering::Less => {}
|
||||
Ordering::Equal => return Err(CanonicalError::DuplicateMapKey),
|
||||
Ordering::Greater => return Err(CanonicalError::UnsortedMapKeys),
|
||||
}
|
||||
}
|
||||
previous_key = Some(key_bytes);
|
||||
let value = self.item(depth + 1)?;
|
||||
entries.push((key, value));
|
||||
}
|
||||
Ok(Value::Map(entries))
|
||||
}
|
||||
_ => unreachable!("major types 6 and 7 are handled above"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn hex(s: &str) -> Vec<u8> {
|
||||
(0..s.len())
|
||||
.step_by(2)
|
||||
.map(|i| u8::from_str_radix(&s[i..i + 2], 16).unwrap())
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn enc(v: &Value) -> String {
|
||||
encode(v)
|
||||
.unwrap()
|
||||
.iter()
|
||||
.map(|b| format!("{b:02x}"))
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn integers_take_the_shortest_head_at_every_boundary() {
|
||||
for (n, want) in [
|
||||
(0u64, "00"),
|
||||
(23, "17"),
|
||||
(24, "1818"),
|
||||
(255, "18ff"),
|
||||
(256, "190100"),
|
||||
(65535, "19ffff"),
|
||||
(65536, "1a00010000"),
|
||||
(4294967295, "1affffffff"),
|
||||
(4294967296, "1b0000000100000000"),
|
||||
(u64::MAX, "1bffffffffffffffff"),
|
||||
] {
|
||||
assert_eq!(enc(&Value::Unsigned(n)), want, "{n}");
|
||||
assert_eq!(decode(&hex(want)).unwrap(), Value::Unsigned(n));
|
||||
}
|
||||
assert_eq!(enc(&Value::int(-1)), "20");
|
||||
assert_eq!(enc(&Value::int(-24)), "37");
|
||||
assert_eq!(enc(&Value::int(-25)), "3818");
|
||||
assert_eq!(enc(&Value::int(i64::MIN)), "3b7fffffffffffffff");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn map_keys_sort_by_encoded_bytes_not_by_text() {
|
||||
// As strings "aa" < "b"; encoded, 0x61 0x62 ("b") < 0x62 0x61 0x61.
|
||||
let v = Value::map([("aa", Value::int(1)), ("b", Value::int(2))]);
|
||||
assert_eq!(enc(&v), "a261620262616101");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_spec_worked_example_is_61_bytes() {
|
||||
let v = Value::map([("n", Value::int(1)), ("d", Value::text("#".repeat(53)))]);
|
||||
let bytes = encode(&v).unwrap();
|
||||
assert_eq!(bytes.len(), 61);
|
||||
assert_eq!(&bytes[..5], &[0xa2, 0x61, 0x64, 0x78, 0x35]);
|
||||
assert_eq!(&bytes[58..], &[0x61, 0x6e, 0x01]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn duplicate_keys_have_no_canonical_form() {
|
||||
let v = Value::map([("a", Value::int(1)), ("a", Value::int(2))]);
|
||||
assert_eq!(encode(&v), Err(CanonicalError::DuplicateMapKey));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn decode_round_trips_everything_encode_produces() {
|
||||
let v = Value::map([
|
||||
("bytes", Value::Bytes(vec![0, 1, 2])),
|
||||
("text", Value::text("ünïcode")),
|
||||
(
|
||||
"list",
|
||||
Value::Array(vec![Value::Null, Value::Bool(true), Value::Bool(false)]),
|
||||
),
|
||||
("nested", Value::map([("x", Value::int(-1000))])),
|
||||
("big", Value::Unsigned(u64::MAX)),
|
||||
]);
|
||||
let bytes = encode(&v).unwrap();
|
||||
let back = decode(&bytes).unwrap();
|
||||
assert_eq!(encode(&back).unwrap(), bytes);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_non_canonical_form_is_refused_with_its_own_class() {
|
||||
use CanonicalError::*;
|
||||
for (h, want) in [
|
||||
("1800", NonShortestInteger),
|
||||
("190017", NonShortestInteger),
|
||||
("3800", NonShortestInteger),
|
||||
("1b00000000ffffffff", NonShortestInteger),
|
||||
("780161", NonShortestLength),
|
||||
("5801ff", NonShortestLength),
|
||||
("980100", NonShortestLength),
|
||||
("a262616101616202", UnsortedMapKeys),
|
||||
("a2616101616102", DuplicateMapKey),
|
||||
("9f0102ff", IndefiniteLength),
|
||||
("7f6161ff", IndefiniteLength),
|
||||
("bf616101ff", IndefiniteLength),
|
||||
("d54100", TagForbidden),
|
||||
("f93e00", FloatForbidden),
|
||||
("fa7f7fffff", FloatForbidden),
|
||||
("fb7fefffffffffffff", FloatForbidden),
|
||||
("62c328", InvalidUtf8),
|
||||
("0101", TrailingBytes),
|
||||
("1a0001", Truncated),
|
||||
("43", Truncated),
|
||||
("", Truncated),
|
||||
("1c", Malformed),
|
||||
("ff", Malformed),
|
||||
("f81f", Malformed),
|
||||
("f7", SimpleValueForbidden),
|
||||
("e0", SimpleValueForbidden),
|
||||
("f3", SimpleValueForbidden),
|
||||
("f820", SimpleValueForbidden),
|
||||
("f8ff", SimpleValueForbidden),
|
||||
("1f", Malformed),
|
||||
] {
|
||||
assert_eq!(decode(&hex(h)), Err(want), "{h}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn nesting_is_bounded() {
|
||||
let mut deep = vec![0x81u8; MAX_DEPTH + 1];
|
||||
deep.push(0x00);
|
||||
assert_eq!(decode(&deep), Err(CanonicalError::TooDeep));
|
||||
let mut ok = vec![0x81u8; MAX_DEPTH];
|
||||
ok.push(0x00);
|
||||
assert!(decode(&ok).is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_first_defect_in_stream_order_is_the_one_reported() {
|
||||
// An oversized count is only a defect once the input runs out; a
|
||||
// forbidden item met before that is reported first.
|
||||
use CanonicalError::*;
|
||||
for (h, want) in [
|
||||
("90cf", TagForbidden),
|
||||
("a9fb76", FloatForbidden),
|
||||
("9a00010000d5", TagForbidden),
|
||||
("9a0001000001", Truncated),
|
||||
] {
|
||||
assert_eq!(decode(&hex(h)), Err(want), "{h}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_huge_declared_length_is_refused_without_allocating() {
|
||||
assert_eq!(
|
||||
decode(&hex("5bffffffffffffffff")),
|
||||
Err(CanonicalError::Truncated)
|
||||
);
|
||||
assert_eq!(
|
||||
decode(&hex("9bffffffffffffffff")),
|
||||
Err(CanonicalError::Truncated)
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -35,6 +35,8 @@ pub enum EnvelopeError {
|
|||
Cbor(String),
|
||||
#[error("envelope SAID mismatch (expected {expected}, computed {computed})")]
|
||||
SaidMismatch { expected: Said, computed: Said },
|
||||
#[error("the bytes decode, but are not the envelope's own encoding")]
|
||||
NotItsEncoding,
|
||||
}
|
||||
|
||||
impl From<serde_cbor::Error> for EnvelopeError {
|
||||
|
|
@ -89,9 +91,17 @@ impl Envelope {
|
|||
Ok(serde_cbor::to_vec(self)?)
|
||||
}
|
||||
|
||||
/// Decode from canonical CBOR.
|
||||
/// Decode an envelope, accepting only its own encoding: the bytes must be
|
||||
/// exactly what [`Envelope::to_cbor`] writes for the decoded envelope.
|
||||
///
|
||||
/// The underlying decoder tolerates extra members, indefinite lengths and
|
||||
/// over-long heads. Without this check one envelope would have unboundedly
|
||||
/// many byte strings, each a fresh gossip message id under the same SAID.
|
||||
pub fn from_cbor(bytes: &[u8]) -> Result<Self, EnvelopeError> {
|
||||
let env: Self = serde_cbor::from_slice(bytes)?;
|
||||
if env.to_cbor()? != bytes {
|
||||
return Err(EnvelopeError::NotItsEncoding);
|
||||
}
|
||||
Ok(env)
|
||||
}
|
||||
|
||||
|
|
@ -156,6 +166,32 @@ mod tests {
|
|||
back.verify_said().unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_envelope_decodes_only_from_its_own_encoding() {
|
||||
let bytes = fixture_envelope().to_cbor().unwrap();
|
||||
assert_eq!(bytes[0], 0xa5, "a five-member map");
|
||||
|
||||
// An extra member the decoder would otherwise ignore.
|
||||
let mut padded = bytes.clone();
|
||||
padded[0] = 0xa6;
|
||||
padded.extend_from_slice(&[0x61, b'x', 0x00]);
|
||||
assert!(matches!(
|
||||
Envelope::from_cbor(&padded),
|
||||
Err(EnvelopeError::NotItsEncoding)
|
||||
));
|
||||
|
||||
// The same map written with an indefinite length.
|
||||
let mut indefinite = bytes.clone();
|
||||
indefinite[0] = 0xbf;
|
||||
indefinite.push(0xff);
|
||||
assert!(matches!(
|
||||
Envelope::from_cbor(&indefinite),
|
||||
Err(EnvelopeError::NotItsEncoding)
|
||||
));
|
||||
|
||||
assert!(Envelope::from_cbor(&bytes).is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mutation_breaks_said() {
|
||||
let mut env = fixture_envelope();
|
||||
|
|
|
|||
133
smart-byte/smart-byte-rs/crates/smart-byte-core/src/inbox.rs
Normal file
133
smart-byte/smart-byte-rs/crates/smart-byte-core/src/inbox.rs
Normal file
|
|
@ -0,0 +1,133 @@
|
|||
//! Receiving SAID-addressed objects (identity spec §2.6).
|
||||
//!
|
||||
//! An object that arrives over the network — gossiped, or as the answer to a
|
||||
//! request by SAID — is accepted in four steps, in this order:
|
||||
//!
|
||||
//! 1. its bytes are the canonical encoding of the object, or it is refused
|
||||
//! (never re-encoded);
|
||||
//! 2. it hashes to the SAID it states in `d`;
|
||||
//! 3. that SAID is the one it was announced or requested under;
|
||||
//! 4. only then: a SAID already held makes it a duplicate.
|
||||
//!
|
||||
//! Verification comes before deduplication, so a forged object announced
|
||||
//! under a SAID the node already holds is reported as a mismatch — the
|
||||
//! substitution attempt is visible — rather than absorbed as a duplicate. No
|
||||
//! first-come rule is needed: two different canonical objects cannot share a
|
||||
//! SAID short of a BLAKE3 collision.
|
||||
|
||||
use crate::said::{self, SAID_FIELD, Said, SaidCheckError};
|
||||
|
||||
/// What became of one delivery.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub enum Delivery {
|
||||
/// Verified and stored.
|
||||
Accepted(Said),
|
||||
/// Verified, and already held.
|
||||
Duplicate(Said),
|
||||
/// Refused; [`SaidCheckError::class`] names why.
|
||||
Rejected(SaidCheckError),
|
||||
}
|
||||
|
||||
/// A node's store of SAID-addressed objects, in the order it accepted them.
|
||||
#[derive(Debug, Default)]
|
||||
pub struct Inbox {
|
||||
held: Vec<(Said, Vec<u8>)>,
|
||||
}
|
||||
|
||||
impl Inbox {
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
/// Deliver `bytes`, announced or requested under `claimed_said`.
|
||||
pub fn deliver(&mut self, claimed_said: &str, bytes: &[u8]) -> Delivery {
|
||||
let said = match said::verify(bytes, SAID_FIELD) {
|
||||
Ok(s) => s,
|
||||
Err(e) => return Delivery::Rejected(e),
|
||||
};
|
||||
let claimed = match Said::from_said_text(claimed_said) {
|
||||
Ok(c) => c,
|
||||
Err(_) => return Delivery::Rejected(SaidCheckError::MalformedSaid("claimed".into())),
|
||||
};
|
||||
if claimed != said {
|
||||
return Delivery::Rejected(SaidCheckError::Mismatch {
|
||||
stated: claimed.to_said_text(),
|
||||
computed: said.to_said_text(),
|
||||
});
|
||||
}
|
||||
if self.get(&said).is_some() {
|
||||
return Delivery::Duplicate(said);
|
||||
}
|
||||
self.held.push((said, bytes.to_vec()));
|
||||
Delivery::Accepted(said)
|
||||
}
|
||||
|
||||
/// The objects held, in acceptance order: each SAID with exactly the bytes
|
||||
/// that were delivered.
|
||||
pub fn held(&self) -> &[(Said, Vec<u8>)] {
|
||||
&self.held
|
||||
}
|
||||
|
||||
/// The bytes held under `said`.
|
||||
pub fn get(&self, said: &Said) -> Option<&[u8]> {
|
||||
self.held
|
||||
.iter()
|
||||
.find(|(s, _)| s == said)
|
||||
.map(|(_, b)| b.as_slice())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::canonical::Value;
|
||||
use crate::said::SAID_PLACEHOLDER;
|
||||
|
||||
fn object(n: u64) -> said::SaidDerivation {
|
||||
said::derive(
|
||||
&Value::map([
|
||||
("d", Value::text(SAID_PLACEHOLDER)),
|
||||
("n", Value::Unsigned(n)),
|
||||
]),
|
||||
SAID_FIELD,
|
||||
)
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn verification_comes_before_deduplication() {
|
||||
let one = object(1);
|
||||
let mut inbox = Inbox::new();
|
||||
let text = one.said.to_said_text();
|
||||
assert_eq!(
|
||||
inbox.deliver(&text, &one.encoded),
|
||||
Delivery::Accepted(one.said)
|
||||
);
|
||||
assert_eq!(
|
||||
inbox.deliver(&text, &one.encoded),
|
||||
Delivery::Duplicate(one.said)
|
||||
);
|
||||
|
||||
// n changed to 2, d left as it was, announced under the held SAID: a
|
||||
// mismatch, not a duplicate.
|
||||
let mut forged = one.encoded.clone();
|
||||
*forged.last_mut().unwrap() = 0x02;
|
||||
match inbox.deliver(&text, &forged) {
|
||||
Delivery::Rejected(e) => assert_eq!(e.class(), "said_mismatch"),
|
||||
other => panic!("forged object was {other:?}"),
|
||||
}
|
||||
assert_eq!(inbox.held().len(), 1);
|
||||
assert_eq!(inbox.get(&one.said), Some(one.encoded.as_slice()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_genuine_object_under_another_said_is_refused() {
|
||||
let (one, two) = (object(1), object(2));
|
||||
let mut inbox = Inbox::new();
|
||||
match inbox.deliver(&one.said.to_said_text(), &two.encoded) {
|
||||
Delivery::Rejected(e) => assert_eq!(e.class(), "said_mismatch"),
|
||||
other => panic!("{other:?}"),
|
||||
}
|
||||
assert!(inbox.held().is_empty());
|
||||
}
|
||||
}
|
||||
|
|
@ -6,6 +6,9 @@
|
|||
//! * [`Envelope`] — content-addressed signed carrier of arbitrary cargo
|
||||
//! with intrinsic provenance, ownership history, and energy cost.
|
||||
//! * [`Said`] — Self-Addressing IDentifier (BLAKE3 over canonical CBOR).
|
||||
//! [`said::derive`] / [`said::verify`] implement identity spec §2 over
|
||||
//! [`canonical`] CBOR; see the [`said`] module for which identifiers in this
|
||||
//! crate do and do not follow it yet.
|
||||
//! * [`sign`] / [`verify`] — Ed25519 signature over the envelope's SAID.
|
||||
//! * [`Cargo`] — discriminated union of byte payload, USD claim,
|
||||
//! joule claim, or custom typed body.
|
||||
|
|
@ -20,8 +23,10 @@
|
|||
//! language can therefore compute the same SAID for the same
|
||||
//! semantic envelope without trusting each other.
|
||||
|
||||
pub mod canonical;
|
||||
pub mod cargo;
|
||||
pub mod envelope;
|
||||
pub mod inbox;
|
||||
pub mod joule_cost;
|
||||
pub mod ownership;
|
||||
pub mod provenance;
|
||||
|
|
@ -33,8 +38,8 @@ pub use envelope::{Envelope, EnvelopeError};
|
|||
pub use joule_cost::JouleCost;
|
||||
pub use ownership::{OwnershipChain, Transition};
|
||||
pub use provenance::Provenance;
|
||||
pub use said::{Said, SaidError};
|
||||
pub use sign::{Signature, SigningKey, VerifyingKey, sign, verify};
|
||||
pub use said::{Said, SaidCheckError, SaidDerivation, SaidError};
|
||||
pub use sign::{SaidSignatureError, Signature, SigningKey, VerifyingKey, sign, verify};
|
||||
|
||||
/// BLAKE3 32-byte hash output. Re-exported so downstream crates can
|
||||
/// refer to it without depending on `blake3` directly.
|
||||
|
|
|
|||
|
|
@ -1,14 +1,43 @@
|
|||
//! Self-Addressing IDentifiers.
|
||||
//!
|
||||
//! A [`Said`] is a 32-byte BLAKE3 digest, displayed as bytewise base32
|
||||
//! (RFC 4648, lower-case, no padding). The substrate uses SAIDs for
|
||||
//! envelope ids and for principal identities.
|
||||
//! A [`Said`] is a 32-byte BLAKE3 digest. It has two text forms, and they are
|
||||
//! not interchangeable:
|
||||
//!
|
||||
//! * **The identity spec's form** (§2.2): the derivation code `E` followed by
|
||||
//! the digest in *lowercase* unpadded base32 — 53 characters.
|
||||
//! [`Said::to_said_text`] / [`Said::from_said_text`] produce and parse it,
|
||||
//! and [`derive`] / [`verify`] implement the §2.1 procedure that yields it:
|
||||
//! hash the canonical CBOR ([`crate::canonical`]) of the object with its
|
||||
//! SAID field holding [`SAID_PLACEHOLDER`]. The conformance pack in
|
||||
//! `smart-byte/conformance/v1` is generated from these functions.
|
||||
//! * **The older form** that [`Display`](fmt::Display) and
|
||||
//! [`Said::to_base32`] print: *upper-case* unpadded base32 with no
|
||||
//! derivation code. It predates the identity spec. [`Envelope`] ids and the
|
||||
//! key-event log in `smart-byte-keri-witness` still derive and print SAIDs
|
||||
//! this way (`serde_cbor` over a zeroed id), so their identifiers are not
|
||||
//! §2 SAIDs yet.
|
||||
//!
|
||||
//! [`Envelope`]: crate::Envelope
|
||||
|
||||
use std::fmt;
|
||||
|
||||
use data_encoding::BASE32_NOPAD;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::canonical::{self, CanonicalError, Value};
|
||||
|
||||
/// The derivation code of a BLAKE3-256 SAID (identity spec §2.2).
|
||||
pub const BLAKE3_256_CODE: char = 'E';
|
||||
|
||||
/// Characters in a SAID's text form: the code plus 52 base32 characters.
|
||||
pub const SAID_TEXT_LEN: usize = 53;
|
||||
|
||||
/// The member that carries an object's SAID (identity spec §2.1).
|
||||
pub const SAID_FIELD: &str = "d";
|
||||
|
||||
/// What a SAID field holds while its object is hashed: `#` 53 times (§2.2).
|
||||
pub const SAID_PLACEHOLDER: &str = "#####################################################";
|
||||
|
||||
/// Errors returned when parsing a [`Said`] from text.
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum SaidError {
|
||||
|
|
@ -20,9 +49,8 @@ pub enum SaidError {
|
|||
|
||||
/// 32-byte self-addressing identifier.
|
||||
///
|
||||
/// Equality is byte-equality. The textual form is upper-case base32
|
||||
/// without padding so it can be embedded in URLs and copied by humans.
|
||||
#[derive(Clone, Copy, PartialEq, Eq, Hash, Default, Serialize, Deserialize)]
|
||||
/// Equality is byte-equality. See the module docs for its two text forms.
|
||||
#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Default, Serialize, Deserialize)]
|
||||
#[serde(transparent)]
|
||||
pub struct Said(pub [u8; 32]);
|
||||
|
||||
|
|
@ -56,6 +84,149 @@ impl Said {
|
|||
out.copy_from_slice(&bytes);
|
||||
Ok(Self(out))
|
||||
}
|
||||
|
||||
/// The identity spec's text form: `E` + lowercase unpadded base32 (§2.2).
|
||||
pub fn to_said_text(&self) -> String {
|
||||
let mut s = String::with_capacity(SAID_TEXT_LEN);
|
||||
s.push(BLAKE3_256_CODE);
|
||||
s.push_str(&BASE32_NOPAD.encode(&self.0).to_ascii_lowercase());
|
||||
s
|
||||
}
|
||||
|
||||
/// Parse the identity spec's text form.
|
||||
///
|
||||
/// Only the one canonical spelling is accepted: exactly 53 characters,
|
||||
/// the `E` code, lowercase base32, and zero trailing bits in the last
|
||||
/// character. Anything looser would give one digest several spellings,
|
||||
/// and a SAID compared as text would then fail to match itself.
|
||||
pub fn from_said_text(s: &str) -> Result<Self, SaidError> {
|
||||
let body = s
|
||||
.strip_prefix(BLAKE3_256_CODE)
|
||||
.filter(|b| {
|
||||
b.len() == SAID_TEXT_LEN - 1
|
||||
&& b.bytes().all(|c| matches!(c, b'a'..=b'z' | b'2'..=b'7'))
|
||||
})
|
||||
.ok_or_else(|| {
|
||||
SaidError::Decode(format!(
|
||||
"not a {SAID_TEXT_LEN}-character `E` + lowercase base32 SAID: {s:?}"
|
||||
))
|
||||
})?;
|
||||
// `from_base32` decodes with trailing-bit checking, which is what
|
||||
// refuses a non-zero final character.
|
||||
Self::from_base32(body)
|
||||
}
|
||||
}
|
||||
|
||||
/// The steps of one SAID derivation (identity spec §2.1), each kept so a test
|
||||
/// vector can publish all of them.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct SaidDerivation {
|
||||
/// Canonical CBOR of the object with its SAID field holding
|
||||
/// [`SAID_PLACEHOLDER`] — the bytes that are hashed.
|
||||
pub hashed: Vec<u8>,
|
||||
/// BLAKE3-256 of `hashed`.
|
||||
pub said: Said,
|
||||
/// The object with the SAID in its field.
|
||||
pub object: Value,
|
||||
/// Canonical CBOR of `object`.
|
||||
pub encoded: Vec<u8>,
|
||||
}
|
||||
|
||||
/// Why a §2 derivation or verification refused its input.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, thiserror::Error)]
|
||||
pub enum SaidCheckError {
|
||||
#[error("not canonical CBOR: {0}")]
|
||||
NonCanonical(#[from] CanonicalError),
|
||||
#[error("a SAID-bearing object must be a map")]
|
||||
NotAMap,
|
||||
#[error("the object has no `{0}` field")]
|
||||
MissingSaidField(String),
|
||||
#[error("the `{0}` field does not hold a well-formed SAID")]
|
||||
MalformedSaid(String),
|
||||
#[error("the object states SAID {stated} but hashes to {computed}")]
|
||||
Mismatch { stated: String, computed: String },
|
||||
}
|
||||
|
||||
impl SaidCheckError {
|
||||
/// The error class as the conformance pack names it.
|
||||
pub fn class(&self) -> &'static str {
|
||||
match self {
|
||||
SaidCheckError::NonCanonical(e) => e.class(),
|
||||
SaidCheckError::NotAMap => "not_a_map",
|
||||
SaidCheckError::MissingSaidField(_) => "missing_said_field",
|
||||
SaidCheckError::MalformedSaid(_) => "malformed_said",
|
||||
SaidCheckError::Mismatch { .. } => "said_mismatch",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Derive the SAID of `object`, which carries its SAID in the member named
|
||||
/// `field` (identity spec §2.1). Whatever that member holds now is replaced by
|
||||
/// the placeholder before hashing, so deriving over an already-stamped object
|
||||
/// gives the same SAID as deriving over a fresh one.
|
||||
pub fn derive(object: &Value, field: &str) -> Result<SaidDerivation, SaidCheckError> {
|
||||
if !matches!(object, Value::Map(_)) {
|
||||
return Err(SaidCheckError::NotAMap);
|
||||
}
|
||||
let mut stamped = object.clone();
|
||||
let slot = stamped
|
||||
.get_mut(field)
|
||||
.ok_or_else(|| SaidCheckError::MissingSaidField(field.to_owned()))?;
|
||||
*slot = Value::text(SAID_PLACEHOLDER);
|
||||
let hashed = canonical::encode(&stamped)?;
|
||||
let said = Said::hash(&hashed);
|
||||
if let Some(slot) = stamped.get_mut(field) {
|
||||
*slot = Value::Text(said.to_said_text());
|
||||
}
|
||||
let encoded = canonical::encode(&stamped)?;
|
||||
Ok(SaidDerivation {
|
||||
hashed,
|
||||
said,
|
||||
object: stamped,
|
||||
encoded,
|
||||
})
|
||||
}
|
||||
|
||||
/// Verify received bytes against the SAID they state in `field`.
|
||||
///
|
||||
/// The bytes must already be canonical: a non-canonical copy is refused with
|
||||
/// the [`CanonicalError`] it violates, never re-encoded and accepted. That is
|
||||
/// the behaviour identity spec §2.5's adversarial vector tests — re-encoding
|
||||
/// would let two different byte strings verify under one SAID.
|
||||
pub fn verify(bytes: &[u8], field: &str) -> Result<Said, SaidCheckError> {
|
||||
let object = canonical::decode(bytes)?;
|
||||
if !matches!(object, Value::Map(_)) {
|
||||
return Err(SaidCheckError::NotAMap);
|
||||
}
|
||||
let stated = match object.get(field) {
|
||||
None => return Err(SaidCheckError::MissingSaidField(field.to_owned())),
|
||||
Some(Value::Text(t)) => {
|
||||
Said::from_said_text(t).map_err(|_| SaidCheckError::MalformedSaid(field.to_owned()))?
|
||||
}
|
||||
Some(_) => return Err(SaidCheckError::MalformedSaid(field.to_owned())),
|
||||
};
|
||||
let computed = derive(&object, field)?.said;
|
||||
if computed != stated {
|
||||
return Err(SaidCheckError::Mismatch {
|
||||
stated: stated.to_said_text(),
|
||||
computed: computed.to_said_text(),
|
||||
});
|
||||
}
|
||||
Ok(stated)
|
||||
}
|
||||
|
||||
/// The per-key SAID of identity spec §3.4: the SAID of
|
||||
/// `{"d": <placeholder>, "k": <key>}`, where `key` is the key's algorithm
|
||||
/// identifier byte (§3.3; `0x01` = Ed25519) followed by the public key, carried
|
||||
/// as a byte string.
|
||||
pub fn per_key_said(prefixed_key: &[u8]) -> Said {
|
||||
let object = Value::map([
|
||||
("d", Value::text(SAID_PLACEHOLDER)),
|
||||
("k", Value::Bytes(prefixed_key.to_vec())),
|
||||
]);
|
||||
derive(&object, "d")
|
||||
.expect("a map with two distinct text keys always has a canonical form")
|
||||
.said
|
||||
}
|
||||
|
||||
impl fmt::Display for Said {
|
||||
|
|
@ -94,4 +265,98 @@ mod tests {
|
|||
let err = Said::from_base32("AAAA").unwrap_err();
|
||||
assert!(matches!(err, SaidError::WrongLength(_)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn spec_text_form_is_53_lowercase_characters_and_round_trips() {
|
||||
let said = Said::hash(b"smart-byte");
|
||||
let text = said.to_said_text();
|
||||
assert_eq!(text.len(), SAID_TEXT_LEN);
|
||||
assert!(text.starts_with('E'));
|
||||
assert!(
|
||||
text[1..]
|
||||
.bytes()
|
||||
.all(|c| matches!(c, b'a'..=b'z' | b'2'..=b'7'))
|
||||
);
|
||||
assert_eq!(Said::from_said_text(&text).unwrap(), said);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn spec_text_form_has_exactly_one_spelling() {
|
||||
let text = Said::hash(b"x").to_said_text();
|
||||
// Upper case, a missing or wrong code, and a wrong length are refused.
|
||||
assert!(Said::from_said_text(&text.to_ascii_uppercase()).is_err());
|
||||
assert!(Said::from_said_text(&text[1..]).is_err());
|
||||
assert!(Said::from_said_text(&format!("F{}", &text[1..])).is_err());
|
||||
assert!(Said::from_said_text(&text[..52]).is_err());
|
||||
// The 52nd base32 character carries 4 padding bits that must be zero:
|
||||
// of its 32 possible values only 'a' (0) and 'q' (16) are canonical.
|
||||
let mut body = "a".repeat(51);
|
||||
body.push('b');
|
||||
assert!(Said::from_said_text(&format!("E{body}")).is_err());
|
||||
body.pop();
|
||||
body.push('q');
|
||||
assert!(Said::from_said_text(&format!("E{body}")).is_ok());
|
||||
}
|
||||
|
||||
fn worked_example() -> Value {
|
||||
Value::map([("d", Value::text(SAID_PLACEHOLDER)), ("n", Value::int(1))])
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn derive_hashes_the_placeholder_form_and_stamps_the_result() {
|
||||
let d = derive(&worked_example(), "d").unwrap();
|
||||
assert_eq!(d.hashed.len(), 61);
|
||||
assert_eq!(d.said, Said::hash(&d.hashed));
|
||||
assert_eq!(
|
||||
d.encoded.len(),
|
||||
d.hashed.len(),
|
||||
"the placeholder keeps the length stable"
|
||||
);
|
||||
assert_eq!(verify(&d.encoded, "d").unwrap(), d.said);
|
||||
// Deriving again over the stamped object gives the same SAID.
|
||||
assert_eq!(derive(&d.object, "d").unwrap().said, d.said);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn verify_refuses_changed_content_and_non_canonical_copies() {
|
||||
let d = derive(&worked_example(), "d").unwrap();
|
||||
|
||||
// Change n from 1 to 2 and keep the stated SAID.
|
||||
let mut changed = d.encoded.clone();
|
||||
*changed.last_mut().unwrap() = 0x02;
|
||||
assert_eq!(verify(&changed, "d").unwrap_err().class(), "said_mismatch");
|
||||
|
||||
// The same object with n spelled 0x18 0x01: refused as non-canonical,
|
||||
// even though re-encoding it would reproduce the stated SAID.
|
||||
let mut lax = d.encoded.clone();
|
||||
lax.pop();
|
||||
lax.extend_from_slice(&[0x18, 0x01]);
|
||||
assert_eq!(
|
||||
verify(&lax, "d").unwrap_err().class(),
|
||||
"non_shortest_integer"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn verify_names_each_structural_failure() {
|
||||
let not_a_map = canonical::encode(&Value::int(1)).unwrap();
|
||||
assert_eq!(verify(¬_a_map, "d").unwrap_err().class(), "not_a_map");
|
||||
let no_field = canonical::encode(&Value::map([("n", Value::int(1))])).unwrap();
|
||||
assert_eq!(
|
||||
verify(&no_field, "d").unwrap_err().class(),
|
||||
"missing_said_field"
|
||||
);
|
||||
let bad = canonical::encode(&Value::map([("d", Value::text("Enot-a-said"))])).unwrap();
|
||||
assert_eq!(verify(&bad, "d").unwrap_err().class(), "malformed_said");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn per_key_said_commits_to_the_algorithm_byte() {
|
||||
let key = [7u8; 32];
|
||||
let mut ed = vec![0x01];
|
||||
ed.extend_from_slice(&key);
|
||||
let mut other = vec![0x02];
|
||||
other.extend_from_slice(&key);
|
||||
assert_ne!(per_key_said(&ed), per_key_said(&other));
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ use ed25519_dalek::Signer;
|
|||
pub use ed25519_dalek::{Signature, SigningKey, Verifier, VerifyingKey};
|
||||
|
||||
use crate::envelope::Envelope;
|
||||
use crate::said::Said;
|
||||
|
||||
/// Errors returned by verification.
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
|
|
@ -32,3 +33,167 @@ pub fn verify(
|
|||
key.verify(envelope.id.as_bytes(), sig)
|
||||
.map_err(|e| SignError::Bad(e.to_string()))
|
||||
}
|
||||
|
||||
/// Sign a SAID. The message is the 32 digest bytes the SAID encodes, not its
|
||||
/// 53-character text — the same bytes [`sign`] signs for an envelope.
|
||||
pub fn sign_said(said: &Said, key: &SigningKey) -> Signature {
|
||||
key.sign(said.as_bytes())
|
||||
}
|
||||
|
||||
/// Why a signature over a SAID was refused.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, thiserror::Error)]
|
||||
pub enum SaidSignatureError {
|
||||
#[error("not a well-formed SAID")]
|
||||
MalformedSaid,
|
||||
#[error("an Ed25519 signature is 64 bytes; got {0}")]
|
||||
MalformedSignature(usize),
|
||||
#[error("not an Ed25519 public key")]
|
||||
MalformedVerifyingKey,
|
||||
#[error("the signature does not verify")]
|
||||
SignatureInvalid,
|
||||
}
|
||||
|
||||
impl SaidSignatureError {
|
||||
/// The error class as the conformance pack names it.
|
||||
pub fn class(&self) -> &'static str {
|
||||
match self {
|
||||
SaidSignatureError::MalformedSaid => "malformed_said",
|
||||
SaidSignatureError::MalformedSignature(_) => "malformed_signature",
|
||||
SaidSignatureError::MalformedVerifyingKey => "malformed_verifying_key",
|
||||
SaidSignatureError::SignatureInvalid => "signature_invalid",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Verify `signature` over the SAID whose identity-spec text form is
|
||||
/// `said_text`, under `verifying_key`.
|
||||
///
|
||||
/// Verification is RFC 8032 Ed25519, strict throughout:
|
||||
///
|
||||
/// * a key whose encoded `y` is not below the field prime is refused before
|
||||
/// anything else (RFC 8032 §5.1.3), so one key has one encoding;
|
||||
/// * a signature scalar `S` outside `[0, L)` is refused (RFC 8032 §5.1.7);
|
||||
/// * small-order keys and commitments are refused, which plain verification
|
||||
/// would let a weak key satisfy.
|
||||
///
|
||||
/// Structural problems are reported before any cryptography runs, so a short
|
||||
/// signature is `malformed_signature` rather than a verification failure.
|
||||
pub fn verify_said_signature(
|
||||
said_text: &str,
|
||||
signature: &[u8],
|
||||
verifying_key: &[u8],
|
||||
) -> Result<(), SaidSignatureError> {
|
||||
let said = Said::from_said_text(said_text).map_err(|_| SaidSignatureError::MalformedSaid)?;
|
||||
let sig: [u8; 64] = signature
|
||||
.try_into()
|
||||
.map_err(|_| SaidSignatureError::MalformedSignature(signature.len()))?;
|
||||
let key: [u8; 32] = verifying_key
|
||||
.try_into()
|
||||
.map_err(|_| SaidSignatureError::MalformedVerifyingKey)?;
|
||||
if !encodes_y_below_p(&key) {
|
||||
return Err(SaidSignatureError::MalformedVerifyingKey);
|
||||
}
|
||||
let key =
|
||||
VerifyingKey::from_bytes(&key).map_err(|_| SaidSignatureError::MalformedVerifyingKey)?;
|
||||
key.verify_strict(said.as_bytes(), &Signature::from_bytes(&sig))
|
||||
.map_err(|_| SaidSignatureError::SignatureInvalid)
|
||||
}
|
||||
|
||||
/// Whether a compressed Edwards point's `y` (its low 255 bits) is below the
|
||||
/// field prime `p = 2^255 - 19`. Decoders that skip this accept nineteen
|
||||
/// out-of-range spellings, some of which decompress to valid points.
|
||||
fn encodes_y_below_p(point: &[u8; 32]) -> bool {
|
||||
let top = point[31] & 0x7f;
|
||||
!(top == 0x7f && point[1..31].iter().all(|&b| b == 0xff) && point[0] >= 0xed)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// The Ed25519 group order L, little-endian.
|
||||
const L: [u8; 32] = [
|
||||
0xed, 0xd3, 0xf5, 0x5c, 0x1a, 0x63, 0x12, 0x58, 0xd6, 0x9c, 0xf7, 0xa2, 0xde, 0xf9, 0xde,
|
||||
0x14, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
|
||||
0x00, 0x10,
|
||||
];
|
||||
|
||||
fn add_l(s: &[u8]) -> [u8; 32] {
|
||||
let mut out = [0u8; 32];
|
||||
let mut carry = 0u16;
|
||||
for i in 0..32 {
|
||||
let v = u16::from(s[i]) + u16::from(L[i]) + carry;
|
||||
out[i] = v as u8;
|
||||
carry = v >> 8;
|
||||
}
|
||||
assert_eq!(carry, 0);
|
||||
out
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_signature_over_a_said_verifies_and_nothing_else_does() {
|
||||
let key = SigningKey::from_bytes(&[9u8; 32]);
|
||||
let said = Said::hash(b"one");
|
||||
let text = said.to_said_text();
|
||||
let vk = key.verifying_key().to_bytes();
|
||||
let sig = sign_said(&said, &key).to_bytes();
|
||||
|
||||
assert_eq!(verify_said_signature(&text, &sig, &vk), Ok(()));
|
||||
let other = Said::hash(b"two").to_said_text();
|
||||
assert_eq!(
|
||||
verify_said_signature(&other, &sig, &vk),
|
||||
Err(SaidSignatureError::SignatureInvalid)
|
||||
);
|
||||
let stranger = SigningKey::from_bytes(&[8u8; 32])
|
||||
.verifying_key()
|
||||
.to_bytes();
|
||||
assert_eq!(
|
||||
verify_said_signature(&text, &sig, &stranger),
|
||||
Err(SaidSignatureError::SignatureInvalid)
|
||||
);
|
||||
assert_eq!(
|
||||
verify_said_signature(&text, &sig[..63], &vk),
|
||||
Err(SaidSignatureError::MalformedSignature(63))
|
||||
);
|
||||
assert_eq!(
|
||||
verify_said_signature("Enope", &sig, &vk),
|
||||
Err(SaidSignatureError::MalformedSaid)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_key_spelled_with_y_at_or_above_p_is_refused() {
|
||||
// y = p + 3 (sign bit clear) is an out-of-range spelling of y = 3,
|
||||
// which is on the curve at large order. The Ed25519 library alone
|
||||
// accepts it as a key; the range check is what refuses it.
|
||||
let mut key = [0xffu8; 32];
|
||||
key[0] = 0xf0;
|
||||
key[31] = 0x7f;
|
||||
assert!(VerifyingKey::from_bytes(&key).is_ok());
|
||||
let said = Said::hash(b"k");
|
||||
assert_eq!(
|
||||
verify_said_signature(&said.to_said_text(), &[0u8; 64], &key),
|
||||
Err(SaidSignatureError::MalformedVerifyingKey)
|
||||
);
|
||||
// The largest in-range y is accepted as an encoding (and then fails
|
||||
// as a key or a signature, not as a spelling).
|
||||
let mut edge = [0xffu8; 32];
|
||||
edge[0] = 0xec;
|
||||
edge[31] = 0x7f;
|
||||
assert!(encodes_y_below_p(&edge));
|
||||
assert!(!encodes_y_below_p(&key));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_same_scalar_spelled_s_plus_l_is_refused() {
|
||||
let key = SigningKey::from_bytes(&[3u8; 32]);
|
||||
let said = Said::hash(b"malleable");
|
||||
let mut sig = sign_said(&said, &key).to_bytes();
|
||||
let s_plus_l = add_l(&sig[32..]);
|
||||
sig[32..].copy_from_slice(&s_plus_l);
|
||||
assert_eq!(
|
||||
verify_said_signature(&said.to_said_text(), &sig, &key.verifying_key().to_bytes()),
|
||||
Err(SaidSignatureError::SignatureInvalid)
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -43,25 +43,37 @@ pub struct VerificationReport {
|
|||
}
|
||||
|
||||
/// Replay verifier.
|
||||
#[derive(Default)]
|
||||
pub struct LogVerifier {
|
||||
/// When `true`, reject `rec` events outright per spec §3.2.
|
||||
/// When `false`, accept recoveries (this crate's extension).
|
||||
/// `true` (the default): reject `rec` events outright. Identity spec §3.2
|
||||
/// reserves `rec`, and a v1 verifier MUST reject it as malformed. `false`
|
||||
/// accepts recoveries verified against the keys they reveal — an
|
||||
/// extension outside spec v1, for exercising recovery before the spec
|
||||
/// defines it.
|
||||
pub strict: bool,
|
||||
/// When `true`, also require the witness-receipt threshold be met
|
||||
/// for every event in the log.
|
||||
pub require_witness_quorum: bool,
|
||||
}
|
||||
|
||||
impl Default for LogVerifier {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
strict: true,
|
||||
require_witness_quorum: false,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl LogVerifier {
|
||||
/// Build a permissive verifier that accepts recoveries and does
|
||||
/// not require witness quorum.
|
||||
/// Build a spec-v1 verifier: rejects `rec`, and does not require witness
|
||||
/// quorum.
|
||||
#[must_use]
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
/// Enable strict spec mode (rejects `rec`).
|
||||
/// `true` (the default) rejects `rec` per spec v1; `false` opts into the
|
||||
/// recovery extension.
|
||||
#[must_use]
|
||||
pub fn strict(mut self, on: bool) -> Self {
|
||||
self.strict = on;
|
||||
|
|
|
|||
|
|
@ -146,12 +146,16 @@ impl SmartByteBehaviour {
|
|||
}
|
||||
|
||||
/// Build a sensible default Gossipsub config: signed messages, IDs from
|
||||
/// content (so identical envelopes deduplicate), short heartbeat.
|
||||
/// content (so identical envelopes deduplicate), short heartbeat, and
|
||||
/// application validation: a received message is forwarded only after
|
||||
/// [`crate::transport::settle_gossip`] accepts it as a genuine envelope, so a
|
||||
/// forged one stops at the first honest node instead of crossing the mesh.
|
||||
pub fn default_gossipsub_config() -> gossipsub::Config {
|
||||
use std::time::Duration;
|
||||
gossipsub::ConfigBuilder::default()
|
||||
.heartbeat_interval(Duration::from_secs(1))
|
||||
.validation_mode(gossipsub::ValidationMode::Strict)
|
||||
.validate_messages()
|
||||
.message_id_fn(|m: &gossipsub::Message| {
|
||||
let h = blake3_hash_message(m);
|
||||
gossipsub::MessageId::from(h.to_vec())
|
||||
|
|
|
|||
|
|
@ -41,6 +41,17 @@ pub enum Error {
|
|||
reason: String,
|
||||
},
|
||||
|
||||
/// A peer answered a fetch for one SAID with a different envelope.
|
||||
#[error("{peer} answered a fetch for {requested} with {served}")]
|
||||
Substituted {
|
||||
/// Peer that answered.
|
||||
peer: PeerId,
|
||||
/// The SAID that was asked for.
|
||||
requested: smart_byte_core::Said,
|
||||
/// The id the answer stated.
|
||||
served: smart_byte_core::Said,
|
||||
},
|
||||
|
||||
/// The control channel for driving the swarm was closed.
|
||||
#[error("control channel closed")]
|
||||
ControlClosed,
|
||||
|
|
|
|||
|
|
@ -42,7 +42,26 @@ pub async fn run(node: &mut Libp2pNode, tx: mpsc::Sender<NodeEvent>) -> Result<(
|
|||
loop {
|
||||
tokio::select! {
|
||||
ev = node.swarm.select_next_some() => {
|
||||
if let Some(out) = translate(ev)
|
||||
let out = match ev {
|
||||
// Every gossip message is settled here before anything
|
||||
// else: accepted (and forwarded) only if it is a genuine
|
||||
// envelope, rejected otherwise.
|
||||
SwarmEvent::Behaviour(SmartByteBehaviourEvent::Gossipsub(
|
||||
gossipsub::Event::Message { propagation_source, message_id, message },
|
||||
)) => crate::transport::settle_gossip(
|
||||
&mut node.swarm,
|
||||
&message_id,
|
||||
&propagation_source,
|
||||
&message.data,
|
||||
)
|
||||
.map(|envelope| NodeEvent::EnvelopeReceived {
|
||||
envelope: Box::new(envelope),
|
||||
from: propagation_source,
|
||||
topic: message.topic.into_string(),
|
||||
}),
|
||||
other => translate(other),
|
||||
};
|
||||
if let Some(out) = out
|
||||
&& tx.send(out).await.is_err()
|
||||
{
|
||||
return Err(Error::ControlClosed);
|
||||
|
|
@ -59,23 +78,6 @@ fn translate(ev: SwarmEvent<SmartByteBehaviourEvent>) -> Option<NodeEvent> {
|
|||
match ev {
|
||||
SwarmEvent::ConnectionEstablished { peer_id, .. } => Some(NodeEvent::PeerConnected(peer_id)),
|
||||
SwarmEvent::ConnectionClosed { peer_id, .. } => Some(NodeEvent::PeerDisconnected(peer_id)),
|
||||
SwarmEvent::Behaviour(SmartByteBehaviourEvent::Gossipsub(
|
||||
gossipsub::Event::Message {
|
||||
propagation_source,
|
||||
message,
|
||||
..
|
||||
},
|
||||
)) => match Envelope::from_cbor(&message.data) {
|
||||
Ok(envelope) => Some(NodeEvent::EnvelopeReceived {
|
||||
envelope: Box::new(envelope),
|
||||
from: propagation_source,
|
||||
topic: message.topic.into_string(),
|
||||
}),
|
||||
Err(e) => {
|
||||
tracing::warn!(error = %e, "received malformed envelope on gossipsub");
|
||||
None
|
||||
}
|
||||
},
|
||||
SwarmEvent::Behaviour(SmartByteBehaviourEvent::Mdns(mdns::Event::Discovered(peers))) => {
|
||||
// Surface the first newly-discovered peer; additional peers
|
||||
// will be returned through subsequent mDNS events.
|
||||
|
|
|
|||
|
|
@ -125,10 +125,13 @@ impl Libp2pNode {
|
|||
{
|
||||
let remaining = deadline.saturating_duration_since(tokio::time::Instant::now());
|
||||
match tokio::time::timeout(remaining, swarm.select_next_some()).await {
|
||||
Ok(SwarmEvent::NewListenAddr { address, .. }) => {
|
||||
listen_addrs.push(address);
|
||||
Ok(ev) => {
|
||||
if let Some(SwarmEvent::NewListenAddr { address, .. }) =
|
||||
crate::transport::settle_if_gossip(&mut swarm, ev)
|
||||
{
|
||||
listen_addrs.push(address);
|
||||
}
|
||||
}
|
||||
Ok(_) => {}
|
||||
Err(_) => break,
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -28,17 +28,91 @@
|
|||
use std::time::Duration;
|
||||
|
||||
use libp2p::futures::{Stream, StreamExt};
|
||||
use libp2p::gossipsub::{IdentTopic, MessageId};
|
||||
use libp2p::gossipsub::{self, IdentTopic, MessageAcceptance, MessageId};
|
||||
use libp2p::request_response::{self, OutboundRequestId};
|
||||
use libp2p::swarm::SwarmEvent;
|
||||
use libp2p::{PeerId, kad};
|
||||
use libp2p::{PeerId, Swarm, kad};
|
||||
use smart_byte_core::{Envelope, Said};
|
||||
|
||||
use crate::behaviour::SmartByteBehaviourEvent;
|
||||
use crate::behaviour::{SmartByteBehaviour, SmartByteBehaviourEvent};
|
||||
use crate::codec::{EnvelopeRequest, EnvelopeResponse};
|
||||
use crate::error::{Error, Result};
|
||||
use crate::node::Libp2pNode;
|
||||
|
||||
/// Decode an envelope received from the network, accepting it only if the
|
||||
/// bytes are its own encoding and its content hashes to the id it states
|
||||
/// (treatise §10.2 step 1).
|
||||
pub(crate) fn accept_envelope(
|
||||
bytes: &[u8],
|
||||
) -> std::result::Result<Envelope, smart_byte_core::EnvelopeError> {
|
||||
let env = Envelope::from_cbor(bytes)?;
|
||||
env.verify_said()?;
|
||||
Ok(env)
|
||||
}
|
||||
|
||||
/// Rule on one received gossip message.
|
||||
///
|
||||
/// [`crate::behaviour::default_gossipsub_config`] holds every received message
|
||||
/// unforwarded until the application rules on it. This accepts — and so
|
||||
/// forwards — only a genuine envelope, and rejects anything else, which also
|
||||
/// counts against the peer that sent it. Every loop that drives the swarm
|
||||
/// settles each message it sees, so nothing stalls and nothing forged is
|
||||
/// relayed. Returns the envelope when it was accepted.
|
||||
pub fn settle_gossip(
|
||||
swarm: &mut Swarm<SmartByteBehaviour>,
|
||||
message_id: &MessageId,
|
||||
propagation_source: &PeerId,
|
||||
data: &[u8],
|
||||
) -> Option<Envelope> {
|
||||
let verdict = accept_envelope(data);
|
||||
let acceptance = match &verdict {
|
||||
Ok(_) => MessageAcceptance::Accept,
|
||||
Err(e) => {
|
||||
tracing::warn!(error = %e, peer = %propagation_source, "rejected a gossiped message");
|
||||
MessageAcceptance::Reject
|
||||
}
|
||||
};
|
||||
swarm
|
||||
.behaviour_mut()
|
||||
.gossipsub
|
||||
.report_message_validation_result(message_id, propagation_source, acceptance);
|
||||
verdict.ok()
|
||||
}
|
||||
|
||||
/// Settle `ev` if it is a gossip message, for loops waiting on something
|
||||
/// else; pass every other event through.
|
||||
pub(crate) fn settle_if_gossip(
|
||||
swarm: &mut Swarm<SmartByteBehaviour>,
|
||||
ev: SwarmEvent<SmartByteBehaviourEvent>,
|
||||
) -> Option<SwarmEvent<SmartByteBehaviourEvent>> {
|
||||
match ev {
|
||||
SwarmEvent::Behaviour(SmartByteBehaviourEvent::Gossipsub(gossipsub::Event::Message {
|
||||
propagation_source,
|
||||
message_id,
|
||||
message,
|
||||
})) => {
|
||||
settle_gossip(swarm, &message_id, &propagation_source, &message.data);
|
||||
None
|
||||
}
|
||||
other => Some(other),
|
||||
}
|
||||
}
|
||||
|
||||
/// Accept a fetch's answer only if it is the envelope that was asked for: it
|
||||
/// must state the requested SAID, and its content must hash to it. Otherwise a
|
||||
/// peer could answer any request with any envelope it liked.
|
||||
fn check_served(peer: PeerId, requested: Said, envelope: Envelope) -> Result<Envelope> {
|
||||
if envelope.id != requested {
|
||||
return Err(Error::Substituted {
|
||||
peer,
|
||||
requested,
|
||||
served: envelope.id,
|
||||
});
|
||||
}
|
||||
envelope.verify_said()?;
|
||||
Ok(envelope)
|
||||
}
|
||||
|
||||
/// Publish an envelope on a gossipsub topic. Returns the `MessageId`
|
||||
/// assigned by gossipsub.
|
||||
pub async fn publish_envelope(
|
||||
|
|
@ -98,12 +172,17 @@ pub async fn subscribe<'a>(
|
|||
if let SwarmEvent::Behaviour(SmartByteBehaviourEvent::Gossipsub(
|
||||
libp2p::gossipsub::Event::Message {
|
||||
propagation_source,
|
||||
message_id,
|
||||
message,
|
||||
..
|
||||
},
|
||||
)) = ev
|
||||
&& let Some(env) = settle_gossip(
|
||||
&mut node.swarm,
|
||||
&message_id,
|
||||
&propagation_source,
|
||||
&message.data,
|
||||
)
|
||||
&& message.topic.as_str() == topic_owned
|
||||
&& let Ok(env) = Envelope::from_cbor(&message.data)
|
||||
{
|
||||
return Some((
|
||||
(env, propagation_source),
|
||||
|
|
@ -116,8 +195,9 @@ pub async fn subscribe<'a>(
|
|||
Ok(Box::pin(stream))
|
||||
}
|
||||
|
||||
/// Poll the swarm until the next gossipsub envelope arrives on
|
||||
/// `topic`, or `timeout` elapses.
|
||||
/// Poll the swarm until the next genuine gossipsub envelope arrives on
|
||||
/// `topic`, or `timeout` elapses. A message that is not a genuine envelope is
|
||||
/// rejected and skipped: one bad peer cannot end the wait.
|
||||
pub async fn next_envelope_on(
|
||||
node: &mut Libp2pNode,
|
||||
topic: &str,
|
||||
|
|
@ -135,13 +215,21 @@ pub async fn next_envelope_on(
|
|||
Ok(SwarmEvent::Behaviour(SmartByteBehaviourEvent::Gossipsub(
|
||||
libp2p::gossipsub::Event::Message {
|
||||
propagation_source,
|
||||
message_id,
|
||||
message,
|
||||
..
|
||||
},
|
||||
))) if message.topic.as_str() == topic => {
|
||||
let env = Envelope::from_cbor(&message.data)
|
||||
.map_err(Error::Envelope)?;
|
||||
return Ok((env, propagation_source));
|
||||
))) => {
|
||||
let env = settle_gossip(
|
||||
&mut node.swarm,
|
||||
&message_id,
|
||||
&propagation_source,
|
||||
&message.data,
|
||||
);
|
||||
if let Some(env) = env
|
||||
&& message.topic.as_str() == topic
|
||||
{
|
||||
return Ok((env, propagation_source));
|
||||
}
|
||||
}
|
||||
Ok(_) => {}
|
||||
Err(_) => {
|
||||
|
|
@ -206,8 +294,11 @@ async fn direct_fetch(node: &mut Libp2pNode, peer: PeerId, said: Said) -> Result
|
|||
}
|
||||
match tokio::time::timeout(remaining, node.swarm.select_next_some()).await {
|
||||
Ok(ev) => {
|
||||
let Some(ev) = settle_if_gossip(&mut node.swarm, ev) else {
|
||||
continue;
|
||||
};
|
||||
if let Some(envelope) = handle_event_for_response(ev, request_id, peer)? {
|
||||
return Ok(envelope);
|
||||
return check_served(peer, said, envelope);
|
||||
}
|
||||
}
|
||||
Err(_) => {
|
||||
|
|
@ -269,8 +360,14 @@ async fn find_provider(node: &mut Libp2pNode, said: Said) -> Result<PeerId> {
|
|||
if remaining.is_zero() {
|
||||
return Err(Error::Kademlia("provider lookup timed out".to_string()));
|
||||
}
|
||||
match tokio::time::timeout(remaining, node.swarm.select_next_some()).await {
|
||||
Ok(SwarmEvent::Behaviour(SmartByteBehaviourEvent::Kad(
|
||||
let ev = match tokio::time::timeout(remaining, node.swarm.select_next_some()).await {
|
||||
Ok(ev) => settle_if_gossip(&mut node.swarm, ev),
|
||||
Err(_) => {
|
||||
return Err(Error::Kademlia("provider lookup timed out".to_string()));
|
||||
}
|
||||
};
|
||||
match ev {
|
||||
Some(SwarmEvent::Behaviour(SmartByteBehaviourEvent::Kad(
|
||||
kad::Event::OutboundQueryProgressed {
|
||||
id,
|
||||
result: kad::QueryResult::GetProviders(Ok(progress)),
|
||||
|
|
@ -286,7 +383,7 @@ async fn find_provider(node: &mut Libp2pNode, said: Said) -> Result<PeerId> {
|
|||
return Err(Error::Kademlia("no providers found".to_string()));
|
||||
}
|
||||
},
|
||||
Ok(SwarmEvent::Behaviour(SmartByteBehaviourEvent::Kad(
|
||||
Some(SwarmEvent::Behaviour(SmartByteBehaviourEvent::Kad(
|
||||
kad::Event::OutboundQueryProgressed {
|
||||
id,
|
||||
result: kad::QueryResult::GetProviders(Err(e)),
|
||||
|
|
@ -295,10 +392,66 @@ async fn find_provider(node: &mut Libp2pNode, said: Said) -> Result<PeerId> {
|
|||
))) if id == qid => {
|
||||
return Err(Error::Kademlia(format!("{e:?}")));
|
||||
}
|
||||
Ok(_) => {}
|
||||
Err(_) => {
|
||||
return Err(Error::Kademlia("provider lookup timed out".to_string()));
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use chrono::TimeZone;
|
||||
use smart_byte_core::{Cargo, EnvelopeError, JouleCost, OwnershipChain, Provenance};
|
||||
|
||||
fn envelope(seed: &[u8]) -> Envelope {
|
||||
let issued_at = chrono::Utc.with_ymd_and_hms(2026, 9, 30, 0, 0, 0).unwrap();
|
||||
Envelope::new(
|
||||
Provenance::new(Said::hash(seed), issued_at, vec![]),
|
||||
OwnershipChain::empty(),
|
||||
Cargo::Usd { minor: 100 },
|
||||
JouleCost::measured(1),
|
||||
)
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_received_envelope_must_hash_to_its_own_id() {
|
||||
let genuine = envelope(b"issuer");
|
||||
assert_eq!(
|
||||
accept_envelope(&genuine.to_cbor().unwrap()).unwrap(),
|
||||
genuine
|
||||
);
|
||||
|
||||
// Same id, different content: refused rather than delivered under it.
|
||||
let mut forged = genuine.clone();
|
||||
forged.cargo = Cargo::Usd { minor: 1_000_000 };
|
||||
assert!(matches!(
|
||||
accept_envelope(&forged.to_cbor().unwrap()),
|
||||
Err(EnvelopeError::SaidMismatch { .. })
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_fetch_answered_with_another_envelope_is_refused() {
|
||||
let peer = PeerId::random();
|
||||
let wanted = envelope(b"wanted");
|
||||
let other = envelope(b"other");
|
||||
|
||||
assert_eq!(
|
||||
check_served(peer, wanted.id, wanted.clone()).unwrap(),
|
||||
wanted
|
||||
);
|
||||
assert!(matches!(
|
||||
check_served(peer, wanted.id, other.clone()),
|
||||
Err(Error::Substituted { requested, served, .. })
|
||||
if requested == wanted.id && served == other.id
|
||||
));
|
||||
// Stating the requested id is not enough: the content must hash to it.
|
||||
let mut relabelled = other;
|
||||
relabelled.id = wanted.id;
|
||||
assert!(matches!(
|
||||
check_served(peer, wanted.id, relabelled),
|
||||
Err(Error::Envelope(EnvelopeError::SaidMismatch { .. }))
|
||||
));
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,145 @@
|
|||
//! A forged envelope stops at the first honest node; a genuine one crosses it.
|
||||
//!
|
||||
//! A — B — C, with A and C not connected. B runs the crate's own event loop,
|
||||
//! which settles every gossip message before gossipsub may forward it. A
|
||||
//! publishes three messages: an envelope with its cargo changed under the
|
||||
//! genuine id, the genuine envelope padded with an extra member, and the
|
||||
//! genuine envelope. Only the last may reach C.
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
use chrono::TimeZone;
|
||||
use libp2p::futures::StreamExt;
|
||||
use libp2p::gossipsub::{self, IdentTopic};
|
||||
use libp2p::swarm::SwarmEvent;
|
||||
use smart_byte_core::{Cargo, Envelope, JouleCost, OwnershipChain, Provenance, Said};
|
||||
use smart_byte_libp2p::behaviour::{SmartByteBehaviourEvent, default_gossipsub_config};
|
||||
use smart_byte_libp2p::transport::{settle_gossip, subscribe_topic};
|
||||
use smart_byte_libp2p::{Libp2pNode, NodeConfig, NodeEvent, run};
|
||||
use tokio::time::Instant;
|
||||
|
||||
async fn node() -> Libp2pNode {
|
||||
Libp2pNode::new(NodeConfig {
|
||||
listen_addrs: vec!["/ip4/127.0.0.1/tcp/0".parse().unwrap()],
|
||||
bootstrap_peers: vec![],
|
||||
identity: libp2p::identity::Keypair::generate_ed25519(),
|
||||
enable_mdns: false,
|
||||
enable_relay: false,
|
||||
enable_dcutr: false,
|
||||
gossipsub_config: default_gossipsub_config(),
|
||||
})
|
||||
.await
|
||||
.expect("spawn node")
|
||||
}
|
||||
|
||||
/// Poll `node` once, settling any gossip message; return its data if it was one.
|
||||
fn received(node: &mut Libp2pNode, ev: SwarmEvent<SmartByteBehaviourEvent>) -> Option<Vec<u8>> {
|
||||
if let SwarmEvent::Behaviour(SmartByteBehaviourEvent::Gossipsub(gossipsub::Event::Message {
|
||||
propagation_source,
|
||||
message_id,
|
||||
message,
|
||||
})) = ev
|
||||
{
|
||||
settle_gossip(
|
||||
&mut node.swarm,
|
||||
&message_id,
|
||||
&propagation_source,
|
||||
&message.data,
|
||||
);
|
||||
return Some(message.data);
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn a_forged_envelope_is_not_relayed() {
|
||||
let (mut a, mut b, mut c) = (node().await, node().await, node().await);
|
||||
let b_addr = b.listen_addrs[0].clone();
|
||||
a.swarm.dial(b_addr.clone()).expect("dial a -> b");
|
||||
c.swarm.dial(b_addr).expect("dial c -> b");
|
||||
let topic = "smart-byte/test/relay";
|
||||
for n in [&mut a, &mut b, &mut c] {
|
||||
subscribe_topic(n, topic).expect("subscribe");
|
||||
}
|
||||
|
||||
let (tx, mut rx) = tokio::sync::mpsc::channel::<NodeEvent>(1024);
|
||||
tokio::spawn(async move {
|
||||
let mut b = b;
|
||||
let _ = run(&mut b, tx).await;
|
||||
});
|
||||
tokio::spawn(async move { while rx.recv().await.is_some() {} });
|
||||
|
||||
// Let the connections settle and the topic mesh form.
|
||||
let warm = Instant::now() + Duration::from_secs(4);
|
||||
while Instant::now() < warm {
|
||||
tokio::select! {
|
||||
_ = a.swarm.select_next_some() => {}
|
||||
_ = c.swarm.select_next_some() => {}
|
||||
_ = tokio::time::sleep(Duration::from_millis(20)) => {}
|
||||
}
|
||||
}
|
||||
assert!(
|
||||
!a.swarm.is_connected(&c.local_peer_id),
|
||||
"A and C must not be directly connected"
|
||||
);
|
||||
|
||||
let issued = chrono::Utc.with_ymd_and_hms(2026, 9, 30, 0, 0, 0).unwrap();
|
||||
let genuine = Envelope::new(
|
||||
Provenance::new(Said::hash(b"issuer"), issued, vec![]),
|
||||
OwnershipChain::empty(),
|
||||
Cargo::Usd { minor: 100 },
|
||||
JouleCost::measured(1),
|
||||
)
|
||||
.unwrap();
|
||||
let mut forged = genuine.clone();
|
||||
forged.cargo = Cargo::Usd { minor: 1_000_000 };
|
||||
let forged = forged.to_cbor().unwrap();
|
||||
let genuine_bytes = genuine.to_cbor().unwrap();
|
||||
let mut padded = genuine_bytes.clone();
|
||||
padded[0] = 0xa6;
|
||||
padded.extend_from_slice(&[0x61, b'x', 0x00]);
|
||||
|
||||
let gossip = &mut a.swarm.behaviour_mut().gossipsub;
|
||||
for bytes in [&forged, &padded, &genuine_bytes] {
|
||||
gossip
|
||||
.publish(IdentTopic::new(topic), bytes.clone())
|
||||
.expect("publish");
|
||||
}
|
||||
|
||||
// Wait for the genuine envelope, then a grace period for anything that
|
||||
// would have been relayed alongside it.
|
||||
let mut seen: Vec<Vec<u8>> = Vec::new();
|
||||
let deadline = Instant::now() + Duration::from_secs(15);
|
||||
let mut grace: Option<Instant> = None;
|
||||
loop {
|
||||
let now = Instant::now();
|
||||
if now > deadline || grace.is_some_and(|g| now > g) {
|
||||
break;
|
||||
}
|
||||
tokio::select! {
|
||||
_ = a.swarm.select_next_some() => {}
|
||||
ev = c.swarm.select_next_some() => {
|
||||
if let Some(data) = received(&mut c, ev) {
|
||||
if data == genuine_bytes && grace.is_none() {
|
||||
grace = Some(Instant::now() + Duration::from_millis(1500));
|
||||
}
|
||||
seen.push(data);
|
||||
}
|
||||
}
|
||||
_ = tokio::time::sleep(Duration::from_millis(20)) => {}
|
||||
}
|
||||
}
|
||||
|
||||
assert!(
|
||||
seen.contains(&genuine_bytes),
|
||||
"the genuine envelope never reached C"
|
||||
);
|
||||
assert!(
|
||||
!seen.contains(&forged),
|
||||
"B relayed an envelope whose content does not hash to its id"
|
||||
);
|
||||
assert!(
|
||||
!seen.contains(&padded),
|
||||
"B relayed a padded copy of a genuine envelope"
|
||||
);
|
||||
}
|
||||
|
|
@ -9,8 +9,14 @@
|
|||
//! Faulty nodes are modeled as nodes that *flip a bit* in their state
|
||||
//! before voting; the cluster still commits if at least `floor(2n/3)+1`
|
||||
//! honest nodes agree.
|
||||
//!
|
||||
//! The commit decision itself is [`tally`]: a pure function from the state
|
||||
//! hashes a frame's nodes reported to the treatise's outcomes — the
|
||||
//! supermajority threshold (§13.1), the four frame outcomes (§13.3) and the
|
||||
//! dissent-log entry a committing node writes (§14.1–§14.2). [`Cluster::step`]
|
||||
//! uses it, and the conformance pack's lockstep vectors are checked against it.
|
||||
|
||||
use std::collections::{HashMap, HashSet};
|
||||
use std::collections::{BTreeMap, HashSet};
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use smart_byte_core::{Blake3Hash, Said, ownership::Transition};
|
||||
|
|
@ -28,6 +34,182 @@ pub enum LockstepError {
|
|||
},
|
||||
}
|
||||
|
||||
/// The supermajority a cluster of `n` nodes needs to commit a frame:
|
||||
/// `floor(2n/3) + 1` (treatise §13.1).
|
||||
pub fn supermajority_threshold(n: usize) -> usize {
|
||||
2 * n / 3 + 1
|
||||
}
|
||||
|
||||
/// The four outcomes a frame can reach, as one node sees them (treatise §13.3).
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
|
||||
pub enum FrameOutcome {
|
||||
/// Supermajority on this node's hash, and every report agreed.
|
||||
CleanCommit,
|
||||
/// Supermajority on this node's hash, but some peers reported another.
|
||||
CommitWithDissent,
|
||||
/// Supermajority on a hash that is not this node's: it halts.
|
||||
MinorityDivergence,
|
||||
/// No hash reached the supermajority: every node halts.
|
||||
ClusterSplit,
|
||||
}
|
||||
|
||||
impl FrameOutcome {
|
||||
/// The outcome as the conformance pack names it.
|
||||
pub fn as_str(&self) -> &'static str {
|
||||
match self {
|
||||
FrameOutcome::CleanCommit => "clean_commit",
|
||||
FrameOutcome::CommitWithDissent => "commit_with_dissent",
|
||||
FrameOutcome::MinorityDivergence => "minority_divergence",
|
||||
FrameOutcome::ClusterSplit => "cluster_split",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// One dissent-log entry (treatise §14.1): the frame, the peers whose
|
||||
/// reported hash differed from the supermajority hash, the hash the logging
|
||||
/// node computed, and the supermajority hash.
|
||||
///
|
||||
/// Only a node that commits writes one, so `local_hash` always equals
|
||||
/// `supermajority_hash` — §14.1 says so, and keeping both fields makes an
|
||||
/// entry from a node that broke that rule detectable.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct DissentEntry<P> {
|
||||
pub frame: u64,
|
||||
/// In ascending order, so every honest node writes the same entry (§14.2).
|
||||
pub dissenting_peers: Vec<P>,
|
||||
pub local_hash: Blake3Hash,
|
||||
pub supermajority_hash: Blake3Hash,
|
||||
}
|
||||
|
||||
/// The state-hash reports for one frame, tallied.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct Tally<P> {
|
||||
/// Nodes in the cluster — the `n` of the threshold, which counts nodes
|
||||
/// that did not report.
|
||||
pub cluster_size: usize,
|
||||
/// Votes for the most-reported hash.
|
||||
pub leading_votes: usize,
|
||||
/// The hash reported by at least [`supermajority_threshold`] nodes, if
|
||||
/// any. At most one hash can be: the threshold exceeds half the cluster.
|
||||
pub supermajority_hash: Option<Blake3Hash>,
|
||||
/// Peers whose report differs from the supermajority hash, ascending.
|
||||
/// Empty when there is no supermajority: dissent is divergence *from* a
|
||||
/// committed hash, so a split frame records none (§14.2).
|
||||
pub dissenters: Vec<P>,
|
||||
}
|
||||
|
||||
/// Why a set of reports could not be tallied.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
|
||||
pub enum TallyError {
|
||||
#[error("cluster is empty")]
|
||||
EmptyCluster,
|
||||
#[error("{reports} state-hash reports for a cluster of {cluster_size}")]
|
||||
TooManyReports { reports: usize, cluster_size: usize },
|
||||
#[error("a peer reported twice for one frame")]
|
||||
DuplicateReport,
|
||||
}
|
||||
|
||||
/// Tally one frame's state-hash reports for a cluster of `cluster_size`.
|
||||
pub fn tally<P: Ord + Clone>(
|
||||
cluster_size: usize,
|
||||
reports: &[(P, Blake3Hash)],
|
||||
) -> Result<Tally<P>, TallyError> {
|
||||
if cluster_size == 0 {
|
||||
return Err(TallyError::EmptyCluster);
|
||||
}
|
||||
if reports.len() > cluster_size {
|
||||
return Err(TallyError::TooManyReports {
|
||||
reports: reports.len(),
|
||||
cluster_size,
|
||||
});
|
||||
}
|
||||
let mut by_peer: BTreeMap<&P, &Blake3Hash> = BTreeMap::new();
|
||||
for (peer, hash) in reports {
|
||||
if by_peer.insert(peer, hash).is_some() {
|
||||
return Err(TallyError::DuplicateReport);
|
||||
}
|
||||
}
|
||||
let mut votes: BTreeMap<&Blake3Hash, usize> = BTreeMap::new();
|
||||
for hash in by_peer.values() {
|
||||
*votes.entry(*hash).or_default() += 1;
|
||||
}
|
||||
let leading_votes = votes.values().copied().max().unwrap_or(0);
|
||||
let threshold = supermajority_threshold(cluster_size);
|
||||
let supermajority_hash = votes
|
||||
.iter()
|
||||
.find(|(_, count)| **count >= threshold)
|
||||
.map(|(hash, _)| **hash);
|
||||
let dissenters = match supermajority_hash {
|
||||
Some(winner) => by_peer
|
||||
.iter()
|
||||
.filter(|(_, hash)| ***hash != winner)
|
||||
.map(|(peer, _)| (*peer).clone())
|
||||
.collect(),
|
||||
None => Vec::new(),
|
||||
};
|
||||
Ok(Tally {
|
||||
cluster_size,
|
||||
leading_votes,
|
||||
supermajority_hash,
|
||||
dissenters,
|
||||
})
|
||||
}
|
||||
|
||||
/// One node's view of a frame (treatise §13.3, §13.4): the hashes its peers
|
||||
/// reported, plus the hash it computed itself in place of whatever it
|
||||
/// broadcast.
|
||||
///
|
||||
/// For a node that reports what it computes the view is the shared tally. For
|
||||
/// the transport-level liar of §13.4 — honest internal state, garbage on the
|
||||
/// wire — it is not: the liar tallies its own honest hash, sees agreement, and
|
||||
/// commits, while its peers record it as a dissenter. Its misbehaviour surfaces
|
||||
/// only in their dissent logs, which is exactly the treatise's point.
|
||||
pub fn node_view<P: Ord + Clone>(
|
||||
cluster_size: usize,
|
||||
reports: &[(P, Blake3Hash)],
|
||||
node: &P,
|
||||
computed: &Blake3Hash,
|
||||
) -> Result<Tally<P>, TallyError> {
|
||||
let mut view: Vec<(P, Blake3Hash)> = reports
|
||||
.iter()
|
||||
.filter(|(peer, _)| peer != node)
|
||||
.cloned()
|
||||
.collect();
|
||||
view.push((node.clone(), *computed));
|
||||
tally(cluster_size, &view)
|
||||
}
|
||||
|
||||
impl<P: Ord + Clone> Tally<P> {
|
||||
/// Whether the frame commits.
|
||||
pub fn committed(&self) -> bool {
|
||||
self.supermajority_hash.is_some()
|
||||
}
|
||||
|
||||
/// The §13.3 outcome for a node whose own post-frame hash is `local`.
|
||||
pub fn outcome(&self, local: &Blake3Hash) -> FrameOutcome {
|
||||
match &self.supermajority_hash {
|
||||
None => FrameOutcome::ClusterSplit,
|
||||
Some(winner) if winner != local => FrameOutcome::MinorityDivergence,
|
||||
Some(_) if self.dissenters.is_empty() => FrameOutcome::CleanCommit,
|
||||
Some(_) => FrameOutcome::CommitWithDissent,
|
||||
}
|
||||
}
|
||||
|
||||
/// The dissent-log entry a node whose own hash is `local` writes for
|
||||
/// `frame`: present only for [`FrameOutcome::CommitWithDissent`].
|
||||
pub fn dissent_entry(&self, frame: u64, local: &Blake3Hash) -> Option<DissentEntry<P>> {
|
||||
match self.outcome(local) {
|
||||
FrameOutcome::CommitWithDissent => Some(DissentEntry {
|
||||
frame,
|
||||
dissenting_peers: self.dissenters.clone(),
|
||||
local_hash: *local,
|
||||
supermajority_hash: self.supermajority_hash?,
|
||||
}),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A single ordered batch of state transitions.
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub struct Frame {
|
||||
|
|
@ -94,12 +276,17 @@ pub struct FrameCommit {
|
|||
pub frame: u64,
|
||||
pub state_hash: Blake3Hash,
|
||||
pub committed_by: HashSet<Said>,
|
||||
/// Nodes whose announced hash differed from `state_hash` (§14.1).
|
||||
pub dissenters: Vec<Said>,
|
||||
}
|
||||
|
||||
/// A federated cluster of [`Node`]s running deterministic lockstep.
|
||||
pub struct Cluster {
|
||||
pub nodes: Vec<Node>,
|
||||
pub frame: u64,
|
||||
/// The dissent log the committing nodes keep (treatise §14). Every honest
|
||||
/// node writes the same entry for a frame, so the simulation keeps one.
|
||||
pub dissent_log: Vec<DissentEntry<Said>>,
|
||||
}
|
||||
|
||||
impl Cluster {
|
||||
|
|
@ -111,7 +298,11 @@ impl Cluster {
|
|||
Node::new(id)
|
||||
})
|
||||
.collect();
|
||||
Self { nodes, frame: 0 }
|
||||
Self {
|
||||
nodes,
|
||||
frame: 0,
|
||||
dissent_log: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Build a cluster of `n` nodes where `faulty_count` of them are
|
||||
|
|
@ -136,29 +327,32 @@ impl Cluster {
|
|||
ordered_transitions: transitions,
|
||||
};
|
||||
let total = self.nodes.len();
|
||||
let mut tally: HashMap<Blake3Hash, HashSet<Said>> = HashMap::new();
|
||||
for node in &mut self.nodes {
|
||||
let vote = node.step(&frame);
|
||||
tally.entry(vote).or_default().insert(node.id);
|
||||
}
|
||||
// Pick the hash with the largest backing set.
|
||||
let (best_hash, voters) = tally
|
||||
.into_iter()
|
||||
.max_by_key(|(_, voters)| voters.len())
|
||||
.expect("non-empty cluster has at least one vote");
|
||||
// Strict > 2/3 (BFT supermajority).
|
||||
let needed = (2 * total) / 3 + 1;
|
||||
if voters.len() < needed {
|
||||
let reports: Vec<(Said, Blake3Hash)> = self
|
||||
.nodes
|
||||
.iter_mut()
|
||||
.map(|node| (node.id, node.step(&frame)))
|
||||
.collect();
|
||||
let result = tally(total, &reports).expect("one report per node in a non-empty cluster");
|
||||
let Some(state_hash) = result.supermajority_hash else {
|
||||
return Err(LockstepError::NoSupermajority {
|
||||
frame: frame.frame_number,
|
||||
best: voters.len(),
|
||||
best: result.leading_votes,
|
||||
total,
|
||||
});
|
||||
};
|
||||
let committed_by = reports
|
||||
.iter()
|
||||
.filter(|(_, hash)| *hash == state_hash)
|
||||
.map(|(id, _)| *id)
|
||||
.collect();
|
||||
if let Some(entry) = result.dissent_entry(frame.frame_number, &state_hash) {
|
||||
self.dissent_log.push(entry);
|
||||
}
|
||||
let commit = FrameCommit {
|
||||
frame: frame.frame_number,
|
||||
state_hash: best_hash,
|
||||
committed_by: voters,
|
||||
state_hash,
|
||||
committed_by,
|
||||
dissenters: result.dissenters,
|
||||
};
|
||||
self.frame += 1;
|
||||
Ok(commit)
|
||||
|
|
@ -203,11 +397,144 @@ mod tests {
|
|||
#[test]
|
||||
fn one_faulty_node_still_commits() {
|
||||
let mut cluster = Cluster::new_with_faulty(4, 1);
|
||||
let liar = cluster.nodes[3].id;
|
||||
// 3-of-4 honest nodes is exactly the floor(2*4/3)+1 = 3 threshold.
|
||||
for f in 0..10 {
|
||||
let commit = cluster.step(synthetic_transitions(f, 3)).unwrap();
|
||||
assert_eq!(commit.committed_by.len(), 3);
|
||||
assert_eq!(commit.dissenters, vec![liar]);
|
||||
}
|
||||
// §14: the liar is caught by the dissent log on every frame it lied.
|
||||
assert_eq!(cluster.dissent_log.len(), 10);
|
||||
assert!(
|
||||
cluster
|
||||
.dissent_log
|
||||
.iter()
|
||||
.all(|e| e.dissenting_peers == vec![liar] && e.local_hash == e.supermajority_hash)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_honest_cluster_logs_no_dissent() {
|
||||
let mut cluster = Cluster::new_honest(4);
|
||||
for f in 0..3 {
|
||||
assert!(
|
||||
cluster
|
||||
.step(synthetic_transitions(f, 2))
|
||||
.unwrap()
|
||||
.dissenters
|
||||
.is_empty()
|
||||
);
|
||||
}
|
||||
assert!(cluster.dissent_log.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_threshold_table_of_section_13_1() {
|
||||
for (n, want) in [
|
||||
(2, 2),
|
||||
(3, 3),
|
||||
(4, 3),
|
||||
(5, 4),
|
||||
(6, 5),
|
||||
(7, 5),
|
||||
(10, 7),
|
||||
(16, 11),
|
||||
(32, 22),
|
||||
(64, 43),
|
||||
] {
|
||||
assert_eq!(supermajority_threshold(n), want, "n = {n}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tally_gives_each_node_its_own_outcome() {
|
||||
let (a, b) = ([0xaa; 32], [0xbb; 32]);
|
||||
let t = tally(4, &[(1, a), (2, a), (3, a), (4, b)]).unwrap();
|
||||
assert_eq!(t.supermajority_hash, Some(a));
|
||||
assert_eq!(t.dissenters, vec![4]);
|
||||
assert_eq!(t.outcome(&a), FrameOutcome::CommitWithDissent);
|
||||
assert_eq!(t.outcome(&b), FrameOutcome::MinorityDivergence);
|
||||
assert!(
|
||||
t.dissent_entry(9, &b).is_none(),
|
||||
"a halted node writes no entry"
|
||||
);
|
||||
let e = t.dissent_entry(9, &a).unwrap();
|
||||
assert_eq!(
|
||||
(e.frame, e.dissenting_peers, e.supermajority_hash),
|
||||
(9, vec![4], a)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_split_frame_records_no_dissent() {
|
||||
// §13.2: at n = 3 one liar leaves 2 of the 3 votes the frame needs.
|
||||
let (a, b) = ([1; 32], [2; 32]);
|
||||
let t = tally(3, &[(1, a), (2, a), (3, b)]).unwrap();
|
||||
assert_eq!(t.supermajority_hash, None);
|
||||
assert_eq!(t.leading_votes, 2);
|
||||
assert!(t.dissenters.is_empty());
|
||||
assert_eq!(t.outcome(&a), FrameOutcome::ClusterSplit);
|
||||
assert!(t.dissent_entry(1, &a).is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn missing_reports_count_against_the_threshold() {
|
||||
// n stays the cluster size: 3 agreeing reports commit at n = 4 but
|
||||
// not at n = 5, whoever failed to report.
|
||||
let a = [7; 32];
|
||||
assert!(tally(4, &[(1, a), (2, a), (3, a)]).unwrap().committed());
|
||||
assert!(!tally(5, &[(1, a), (2, a), (3, a)]).unwrap().committed());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_transport_liar_commits_in_its_own_view_and_is_logged_in_its_peers() {
|
||||
// §13.4: node 4 computed the honest hash but broadcast another.
|
||||
let (honest, lie) = ([0xbb; 32], [0xdd; 32]);
|
||||
let reports = [(1, honest), (2, honest), (3, honest), (4, lie)];
|
||||
let peers = tally(4, &reports).unwrap();
|
||||
assert_eq!(peers.outcome(&honest), FrameOutcome::CommitWithDissent);
|
||||
assert_eq!(peers.dissenters, vec![4]);
|
||||
let own = node_view(4, &reports, &4, &honest).unwrap();
|
||||
assert_eq!(own.outcome(&honest), FrameOutcome::CleanCommit);
|
||||
assert!(own.dissent_entry(1, &honest).is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn at_three_nodes_the_liar_alone_believes_the_frame_committed() {
|
||||
// §13.2 + §13.4: the honest pair sees 2-vs-1 and halts; the liar sees
|
||||
// its own honest hash agree with both peers.
|
||||
let (honest, lie) = ([0x11; 32], [0x22; 32]);
|
||||
let reports = [(1, honest), (2, honest), (3, lie)];
|
||||
assert_eq!(
|
||||
node_view(3, &reports, &1, &honest)
|
||||
.unwrap()
|
||||
.outcome(&honest),
|
||||
FrameOutcome::ClusterSplit
|
||||
);
|
||||
assert_eq!(
|
||||
node_view(3, &reports, &3, &honest)
|
||||
.unwrap()
|
||||
.outcome(&honest),
|
||||
FrameOutcome::CleanCommit
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn malformed_report_sets_are_refused() {
|
||||
let a = [7; 32];
|
||||
assert_eq!(tally::<u8>(0, &[]), Err(TallyError::EmptyCluster));
|
||||
assert_eq!(
|
||||
tally(1, &[(1, a), (2, a)]),
|
||||
Err(TallyError::TooManyReports {
|
||||
reports: 2,
|
||||
cluster_size: 1
|
||||
})
|
||||
);
|
||||
assert_eq!(
|
||||
tally(3, &[(1, a), (1, a)]),
|
||||
Err(TallyError::DuplicateReport)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ A **Self-Addressing IDentifier** is the cryptographic digest of a canonically-en
|
|||
4. Encode the digest using the prefix-plus-bytewise-base32 encoding in §2.2.
|
||||
5. Replace the placeholder in the object with the resulting SAID.
|
||||
|
||||
After step 5, the object is *self-certifying*: any party in possession of the object can run steps 1–4 against a copy with the SAID field reset to the placeholder, compute the digest, and confirm that the object's `d` field matches. Any modification of any other field — by even a single bit — produces a different digest, and the verification fails.
|
||||
After step 5, the object is *self-certifying*: any party in possession of the object can run steps 1–4 against a copy with the SAID field reset to the placeholder, compute the digest, and confirm that the object's `d` field matches. Any modification of any other field — by even a single bit — produces a different digest, and the verification fails. A party that holds the object as bytes it received first confirms they are the object's canonical encoding (§2.3) and refuses them otherwise; it never re-encodes received bytes into canonical form and verifies the result (§2.5, §2.6).
|
||||
|
||||
This is the same content-addressing pattern used for envelope identity in treatise §8.2; the SAID construction generalizes it to *any* object that needs a content-derived identifier including those that must reference themselves (key-event log entries, anchors, attestations).
|
||||
|
||||
|
|
@ -45,10 +45,11 @@ The canonical encoding for SAID computation is CBOR (RFC 8949) with deterministi
|
|||
- Strings use the shortest length-prefix.
|
||||
- Floats are forbidden. Any value that would be a float in another encoding must be represented as a fixed-point integer pair.
|
||||
- CBOR tags are forbidden. Tagged values change parser behavior and undermine determinism.
|
||||
- Simple values other than `false`, `true` and `null` are forbidden. The data model has nothing else to decode them to.
|
||||
- Indefinite-length items are forbidden.
|
||||
- The encoded form is the *only* form over which the digest is computed; serializations that round-trip through other formats (JSON, MessagePack) must re-encode to canonical CBOR before digest computation.
|
||||
- The encoded form is the *only* form over which the digest is computed. An object held in another format (JSON, MessagePack) is encoded to canonical CBOR before its digest is computed. CBOR bytes received from another party must already be canonical: a verifier refuses any other encoding and never re-encodes it (§2.5, §2.6).
|
||||
|
||||
These are the deterministic encoding rules from RFC 8949 §4.2.1 with the tag-and-float prohibitions made explicit. They are also the rules used by envelope serialization in treatise §8 — the choice is intentional, so that a SAID computed over an envelope-shaped object yields the same identifier whether the SAID is being computed by Smart Byte's envelope-identity machinery or its key-event log machinery.
|
||||
These are the deterministic encoding rules from RFC 8949 §4.2.1 with the tag, float and simple-value prohibitions made explicit. They are also the rules used by envelope serialization in treatise §8 — the choice is intentional, so that a SAID computed over an envelope-shaped object yields the same identifier whether the SAID is being computed by Smart Byte's envelope-identity machinery or its key-event log machinery.
|
||||
|
||||
### 2.4 Worked example
|
||||
|
||||
|
|
@ -79,34 +80,51 @@ A2 # map(2)
|
|||
01 # unsigned(1)
|
||||
```
|
||||
|
||||
Concatenated: `A2 61 64 78 35` + 53×`23` + `61 6E 01`, total 62 bytes.
|
||||
|
||||
BLAKE3-256 of those 62 bytes is (worked example; verifiers should reproduce against the test vector file referenced in §2.5):
|
||||
Concatenated: `A2 61 64 78 35` + 53×`23` + `61 6E 01`, total 61 bytes:
|
||||
|
||||
```
|
||||
b3: 3f a1 7b 9c 5d 2e 88 41 ... (32 bytes)
|
||||
a2616478352323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323616e01
|
||||
```
|
||||
|
||||
The 32-byte digest encoded in lowercase bytewise base32 (no padding) yields 52 characters, prefixed with `E` to produce the SAID:
|
||||
BLAKE3-256 of those 61 bytes is:
|
||||
|
||||
```
|
||||
E h7 q xt 4 ... (53 ASCII chars total — full string in conformance vectors)
|
||||
56813033774b7fb781e0f8547eae6ec3ca24728c8af6ae9dd771d1e0258f6d39
|
||||
```
|
||||
|
||||
The 32-byte digest encoded in lowercase bytewise base32 (no padding) yields 52 characters, prefixed with `E` to produce the 53-character SAID:
|
||||
|
||||
```
|
||||
Ek2atam3xjn73papa7bkh5ltoypfci4umrl3k5hoxohi6ajmpnu4q
|
||||
```
|
||||
|
||||
The final object, with the placeholder replaced, is:
|
||||
|
||||
```
|
||||
{
|
||||
"d": "E<52-char base32 of the BLAKE3-256 digest>",
|
||||
"d": "Ek2atam3xjn73papa7bkh5ltoypfci4umrl3k5hoxohi6ajmpnu4q",
|
||||
"n": 1
|
||||
}
|
||||
```
|
||||
|
||||
Any verifier in possession of the object replaces `d` with 53 `#` characters, re-encodes canonically, computes BLAKE3-256, base32-encodes with the `E` prefix, and confirms equality with the asserted `d`.
|
||||
These are the values of vector `said-008-spec-worked-example` in [`../conformance/v1/said_vectors.json`](../conformance/v1/said_vectors.json), generated by the reference implementation and reproduced by an independent second implementation (`../conformance/crosscheck/verify_v1.py`).
|
||||
|
||||
A verifier holding the object's bytes confirms they are canonical, replaces `d` with 53 `#` characters, encodes the result canonically, computes BLAKE3-256, base32-encodes it with the `E` prefix, and confirms equality with the asserted `d`.
|
||||
|
||||
### 2.5 Test-vector hook
|
||||
|
||||
The full set of canonical worked examples — placeholder bytes, canonical CBOR bytes, BLAKE3 digest, final SAID — lives in `../conformance/said_vectors.json` (created in issue #6). Implementers MUST pass every vector. The suite covers the empty-map case, the one-field case, the worked example above, several envelope-shaped objects from treatise §8, and one adversarial case where a non-canonical CBOR encoding produces a different digest (to verify that implementations reject non-canonical input rather than silently re-encoding).
|
||||
The full set of canonical worked examples — placeholder bytes, canonical CBOR bytes, BLAKE3 digest, final SAID — lives in `../conformance/v1/said_vectors.json`. Implementers MUST pass every vector. The suite covers the empty-map case (`said-006`), the one-field case (`said-007`, a §3.4 per-key object), the worked example above (`said-008`), five nested objects built from an envelope's members (`said-001` to `said-005`), and two adversarial cases where a non-canonical CBOR encoding produces a different digest (`said-009`, `said-010`), to verify that implementations reject non-canonical input rather than silently re-encoding it or hashing it as received. `said-011` is a canonical object that states a SAID its content does not hash to. The canonical-encoding rules of §2.3 are exercised one by one in `../conformance/v1/canonicalisation_vectors.json`.
|
||||
|
||||
### 2.6 Receiving SAID-addressed objects
|
||||
|
||||
An object that arrives from another party — gossiped (treatise §11.4) or as the answer to a request by SAID (treatise §21.4) — is accepted in four steps, in this order:
|
||||
|
||||
1. Its bytes are the canonical encoding of the object (§2.3). If they are not, the object is refused; it is never re-encoded.
|
||||
2. It hashes to the SAID it states in `d`.
|
||||
3. That SAID is the one it was announced or requested under.
|
||||
4. Only then: if the SAID is already held, the object is a duplicate and changes nothing.
|
||||
|
||||
Verification precedes deduplication, so a forged object announced under a SAID the receiver already holds is reported as a SAID mismatch — the substitution attempt is visible — instead of being absorbed as a duplicate. No first-come rule is needed: two different canonical objects cannot share a SAID short of a BLAKE3 collision. `../conformance/v1/gossip_vectors.json` exercises each step.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -178,17 +196,17 @@ Notes on individual fields:
|
|||
- `s` starts at `0` for `icp` and increments by exactly 1 per event — no gaps, no reuse.
|
||||
- `p` is the empty string for `icp` and the previous event's `d` for every other event.
|
||||
- `kt` is an unweighted integer threshold; KERI's weighted-threshold extension is reserved for a later revision.
|
||||
- Each entry in `k` is a public-key encoding prefixed with the one-byte algorithm identifier from treatise §8.3 (`0x01` for ed25519; remaining range reserved for post-quantum schemes such as ML-DSA). Order matters; the same key in a different position produces a different `n` commitment and breaks the chain.
|
||||
- Each entry in `k` is a byte string: the one-byte algorithm identifier from treatise §8.3 (`0x01` for ed25519; the remaining range is reserved for post-quantum schemes such as ML-DSA) followed by the public key. Order matters; the same key in a different position produces a different `n` commitment and breaks the chain.
|
||||
- `nt` is committed in advance so an adversary cannot lower the threshold without revealing pre-rotated keys.
|
||||
- Each entry in `n` is the SAID of the corresponding next signing key per §3.4. An empty `n` signals the controller does not intend to rotate further; thereafter only `ixn` events are accepted.
|
||||
- `a` is populated for `ixn` and empty for `icp` and `rot`.
|
||||
|
||||
### 3.4 Per-key SAID derivation
|
||||
|
||||
Each entry in `n` is the SAID of a tiny CBOR map containing the corresponding next signing key, computed per §2 with the placeholder rules. Specifically, for each next-key `K_i` (the full prefixed public-key encoding the controller intends to use after the next rotation), the entry `n[i]` is:
|
||||
Each entry in `n` is the SAID of a tiny CBOR map containing the corresponding next signing key, computed per §2 with the placeholder rules. Specifically, for each next-key `K_i` (the algorithm-prefixed key byte string of §3.3 the controller intends to use after the next rotation), the entry `n[i]` is:
|
||||
|
||||
```
|
||||
SAID({"d": "<placeholder>", "k": "<K_i>"})
|
||||
SAID({"d": "<placeholder>", "k": h'<K_i>'})
|
||||
```
|
||||
|
||||
with `d` initialized to the 53-`#` placeholder, the map canonically encoded per §2.3, the digest computed per §2.2, and `d` replaced with the resulting SAID. This per-key SAID is what appears in the previous event's `n` array; at rotation, the verifier recomputes the per-key SAID for each `k[i]` in the new event and confirms it matches `n[i]` from the previous event, position-wise.
|
||||
|
|
@ -207,7 +225,7 @@ This separation lets a verifier confirm the event's SAID independently of signat
|
|||
|
||||
Given a sequence of events and a current controller AID, a verifier accepts the log if and only if every check below passes for every event in order from `s: 0` to the latest event seen.
|
||||
|
||||
1. **SAID verification.** Replace the event's `d` field with the 53-character placeholder, canonically re-encode per §2.3, compute BLAKE3-256, base32-encode with the `E` prefix, and confirm equality with the asserted `d`. Reject the event if the recomputed SAID does not match.
|
||||
1. **SAID verification.** Confirm the event's bytes are canonical (§2.3) and reject them otherwise (§2.6). Replace the event's `d` field with the 53-character placeholder, encode canonically, compute BLAKE3-256, base32-encode with the `E` prefix, and confirm equality with the asserted `d`. Reject the event if the recomputed SAID does not match.
|
||||
|
||||
2. **Sequence verification.** Confirm the event's `s` is exactly the previous event's `s` + 1. For the inception event, confirm `s` is `0`. Reject the event on any gap, repeat, or out-of-order value.
|
||||
|
||||
|
|
@ -245,10 +263,13 @@ The Authenticated Chained Data Containers (ACDC) specification extends KERI's cr
|
|||
|
||||
The Rust reference implementation of this section lives in the `smart-byte-rs` workspace:
|
||||
|
||||
- `smart-byte-rs/crates/smart-byte-core/src/said.rs` — SAID computation, placeholder substitution, canonical CBOR encoding, per-key SAID derivation, and the conformance-vector harness referenced in §2.5.
|
||||
- `smart-byte-rs/crates/smart-byte-core/src/keri.rs` — key-event log: event type definitions, the wrapper that pairs events with signatures, the verifier algorithm in §4, and helpers for emitting `icp`, `rot`, and `ixn` events from an in-memory controller state.
|
||||
- `smart-byte-rs/crates/smart-byte-core/src/canonical.rs` — the canonical CBOR encoding of §2.3, and a strict decoder that refuses non-canonical input instead of re-encoding it.
|
||||
- `smart-byte-rs/crates/smart-byte-core/src/said.rs` — SAID derivation and verification (§2.1, §2.2), placeholder substitution, and per-key SAID derivation (§3.4).
|
||||
- `smart-byte-rs/crates/smart-byte-core/src/inbox.rs` — receiving SAID-addressed objects (§2.6).
|
||||
- `smart-byte-rs/crates/smart-byte-conformance` — the conformance-vector harness referenced in §2.5. It generates `../conformance/v1` from the two modules above and checks the published pack under `cargo test`.
|
||||
- `smart-byte-rs/crates/smart-byte-keri-witness` — the key-event log: event type definitions, the wrapper that pairs events with signatures, the verifier algorithm in §4, and helpers for emitting `icp`, `rot`, and `ixn` events. **Its event SAIDs do not follow §2 yet**: it hashes a general-purpose CBOR serialization with a zeroed placeholder, not the canonical encoding with the 53-`#` placeholder, so its identifiers are not interoperable with this section.
|
||||
|
||||
If these modules are not yet present in `smart-byte-rs` at the time this section lands, the implementer should treat the function signatures and field-name conventions in §§2–4 as the v1 interface contract and implement against them directly; the reference crate will conform to those signatures when it lands. The conformance test vectors in `../conformance/said_vectors.json` (issue #6) are the authoritative reference, independent of any single language's implementation.
|
||||
Where the reference crate and this section disagree, this section governs. The conformance test vectors in `../conformance/v1/said_vectors.json` are the authoritative reference, independent of any single language's implementation.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -263,4 +284,4 @@ If these modules are not yet present in `smart-byte-rs` at the time this section
|
|||
- O'Connor, Jack, et al. *BLAKE3: One Function, Fast Everywhere*. 2020. <https://github.com/BLAKE3-team/BLAKE3-specs>
|
||||
- Global Legal Entity Identifier Foundation (GLEIF). *vLEI Ecosystem Governance Framework*. Production deployments via Provenant and Veridian, 2024–2026.
|
||||
|
||||
Cross-references within this spec are to other files in `./` (e.g. [`treatise_v1_parts_I_II_III_combined.md`](./treatise_v1_parts_I_II_III_combined.md), `../conformance/said_vectors.json`); all paths are relative to this file.
|
||||
Cross-references within this spec are to other files in `./` (e.g. [`treatise_v1_parts_I_II_III_combined.md`](./treatise_v1_parts_I_II_III_combined.md), `../conformance/v1/said_vectors.json`); all paths are relative to this file.
|
||||
|
|
|
|||
Loading…
Reference in a new issue