smart-byte: generate the conformance pack from the reference implementation #99

Merged

dcharlot merged 12 commits from ci/smart-byte-pack into main 2026-10-01 01:03:48 -04:00

Conversation 0 Commits 12 Files changed 35 +6356 -436

2026-09-30

Transaction Science

smart-byte: the conformance pack, corrected against an independent review ...

The review found the pack and its checkers wrong in ways two agreeing
implementations could not catch.

Spec:
- Identity spec §2.1, §2.3, §2.4 and §4 step 1 told a verifier to re-encode
  an object canonically. A verifier written to them accepts said-009, the
  re-encoding attack §2.5 exists to refuse. All four now require received
  bytes to be canonical already.
- New §2.6 fixes how a SAID-addressed object is received: canonical bytes,
  then the SAID they state, then the SAID they arrived under, and only then
  deduplication.
- §2.3 forbids simple values other than false, true and null.
- §3.3/§3.4 say a key is a byte string, as said-007 already encoded it.

Vectors:
- Lockstep vectors separate the hash a node broadcast from the one it
  computed, and give each node's outcome and dissent log from its own view.
  The old single view called a node a "liar" while modelling divergence;
  lockstep-006/007 now pin treatise §13.4's transport liar, who commits in
  its own view while its peers log it.
- Gossip vectors carry generic SAID-addressed objects. Envelope-shaped ones
  with null or filler provenance are bytes a provenance-checking node must
  refuse (treatise §8.3), so they could not be MUST-pass. `expected_held`
  now carries the bytes held.
- New canon-020 (simple value) and sig-007 (a key spelled with y = p + 3, a
  second spelling of a valid point that ed25519-dalek alone accepts).
- SAID vector descriptions no longer claim fidelity to treatise §8 or a
  frame the data never mentions.

Checkers:
- Both now check the pack whole. An empty pack, a deleted vector the spec
  cites, a deleted class, or an unlisted class all fail; before, an empty
  pack passed both checkers.
- The Python checker no longer uses `assert`. Under `python -O` it passed a
  corrupted pack.
- Both refuse JSON that names a member twice.
- The Rust verifier no longer panics on an empty stated SAID.
- The gossip expectations come from smart-byte-core's inbox, so every value
  in the pack is now the reference implementation's.

Falsified:
- The Rust verifier catches 19 of 19 planted defects, each on its own vector,
  plus the empty-pack, deleted-vector, deleted-class, unlisted-class and
  duplicate-member cases.
- The Python checker catches 15 of 15 planted defects in both normal and -O
  modes, plus the same pack-level cases.

regeneration.md is consolidated into one accurate entry, which corrects the
earlier claim that no draft vector was passable. VERSIONS.md rule 4 and the
README now scope supersession to frozen packs.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

7dba0f3db9

• View at this point in history

2026-09-30 21:34:19 -04:00

Transaction Science

smart-byte-keri-witness: reject rec by default, as spec v1 requires ...

Identity spec §3.2 reserves `rec`, and a v1 verifier MUST reject it as
malformed. `LogVerifier::new()` accepted it, because the strict flag defaulted
to false. Strict is now the default. Accepting recoveries is an explicit
opt-in (`strict(false)`), documented as an extension outside v1; the recovery
test already opts in.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

b7df79083a

• View at this point in history

2026-09-30 21:33:24 -04:00

Transaction Science

smart-byte-libp2p: validate gossip before it is forwarded ...

The previous fix checked envelopes on local delivery only. Gossipsub forwards
every message before application code runs, so a forged envelope still crossed
every honest node. A three-node test (A — B — C, A and C unconnected)
reproduced it: C received the forgery, relayed by B.

The default gossipsub config now sets `validate_messages()`, so a received
message is held until the application rules on it. `settle_gossip` accepts and
forwards a genuine envelope and rejects anything else, which also scores the
sender down. Every loop that drives the swarm — startup, `run`, `subscribe`,
`next_envelope_on`, `fetch_by_said`, the provider lookup — settles each message
it sees, so nothing stalls and nothing forged is relayed. `next_envelope_on` now
skips a bad message instead of ending the wait.

`tests/gossip_validation.rs` runs A — B — C with B on the crate's own event
loop. A publishes a forgery, a padded copy of a genuine envelope, and the
genuine one; only the last reaches C. Falsified both ways: without
`validate_messages()`, B relays the forgery; without `from_cbor`'s
own-encoding check, B relays the padded copy.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

58d1efd353

• View at this point in history

2026-09-30 21:33:24 -04:00

Transaction Science

smart-byte-lockstep: a node's own view of a frame ...

`node_view` tallies a node's peers' reported hashes plus the hash the node
computed itself. For a node that reports what it computes, that is the shared
tally. For treatise §13.4's transport-level liar — honest state, garbage on
the wire — it is not: the liar sees unanimity and commits, while its peers log
it as a dissenter. The previous model used one global view, so it could not
express the case the treatise calls the substrate's real misbehaviour signal.

Tests pin both shapes: the liar logged by its peers at n = 4, and at n = 3
the liar alone believing the frame committed while the honest pair halts.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

a36226a594

• View at this point in history

2026-09-30 21:33:24 -04:00

Transaction Science

smart-byte-core: harden the codec, key decoding, and receiving ...

From an independent review of the previous commits.

- canonical: the decoder no longer sizes any allocation from a declared
  count. Bounding each `with_capacity` by the remaining input did not bound
  nesting: 128 nested containers multiplied it, so a 1 MiB input held 4–8 GiB.
  Containers now grow as their items decode. An input with several defects
  now reports the first in stream order.
- canonical: simple values other than false, true and null are refused
  (`simple_value_forbidden`). The data model has nothing to decode them to;
  identity spec §2.3 now says so.
- sign: a key whose encoded y is not below p is refused as malformed (RFC 8032
  §5.1.3). ed25519-dalek alone accepts y = p + 3, a second spelling of a
  valid large-order point; a test pins that the library accepts it and the
  check refuses it.
- inbox: receiving SAID-addressed objects (identity spec §2.6) in a fixed
  order: canonical bytes, then the SAID they state, then the SAID they arrived
  under, and only then deduplication. Verification comes first, so a forgery
  under a held SAID is reported as a mismatch rather than absorbed as a
  duplicate.
- envelope: `from_cbor` accepts only an envelope's own encoding. The
  underlying decoder tolerated extra members and indefinite lengths, so one
  envelope had unboundedly many byte strings — each a fresh gossip message id
  under the same SAID.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

7cc81a72dd

• View at this point in history

2026-09-30 21:33:24 -04:00

Transaction Science

smart-byte: generate the conformance pack from the reference implementation ...

The v1 pack was filler. Its SAIDs were runs of one repeated character, every
`canonical_cbor_hex` was the same 7-byte stub, and its signatures were RFC 8032
test signatures over the empty message. All five canonicalisation encodings
were wrong; two were odd-length hex that could not be decoded at all. The gate
only ran `jq` over it, and the spec said implementers MUST pass it.

Now every value is computed, and checked twice:

- New crate `smart-byte-conformance`. It is the `gen-conformance-vectors` /
  `verify-conformance-vectors` pair that regeneration.md always promised and
  that never existed. The generator computes every expected value with
  smart-byte-core and smart-byte-lockstep, and fails if a negative vector is
  refused for any reason other than the one it names. Its `cargo test` (run by
  standards-build on every push) checks:
    * every published vector verifies;
    * the published files equal the generator's output byte for byte;
    * a defect planted in each file is caught on the vector it was planted in;
    * identity spec §2.4 prints the worked example's real values.
- `conformance/crosscheck/verify_v1.py`: a second implementation, in Python,
  stdlib only, with BLAKE3 and Ed25519 written from their specs and checked
  against known answers first. It reproduces all 47 vectors, and the
  conformance gate now runs it.

The pack now tests what the spec says:

- canonicalisation: each §2.3 rule, accepted and refused (19 vectors).
- SAIDs: the suite §2.5 names, which the pack never contained — the
  empty-map case, the one-field case, the worked example, five
  envelope-shaped objects, and two adversarial non-canonical encodings (one
  aimed at re-encoding verifiers, one at verifiers that hash bytes as
  received).
- signatures: over real SAIDs, plus a non-canonical S.
- lockstep: §13.3 outcomes and §14.1 dissent entries.
- gossip: content-addressed intake.

Vectors that contradicted the spec are corrected:

- canon-004 encoded floats, which §2.3 forbids;
- lockstep-003 logged dissent on a frame that never committed;
- gossip-002 settled a "collision" by arrival order;
- gossip-003 asserted backfill limits no spec defines.

regeneration.md records each change and why the draft pack was regenerated in
place, not by supersession.

Spec: §2.4 printed an invented digest and SAID, and miscounted the encoding as
62 bytes (it is 61); it now prints the real values. §2.5 names the vectors. §6
pointed at modules that did not exist, and now says where each part lives —
including that the key-event log does not derive §2 SAIDs yet.

Replaces this branch's placeholder pins in smart-byte-core/tests/pack.rs, which
were written to fail once the pack was regenerated, with the assertions they
asked for.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

becbfa8466

• View at this point in history

2026-09-30 20:30:04 -04:00

Transaction Science

smart-byte-libp2p: check what arrives against the SAID it claims ...

No receive path verified anything. `fetch_by_said` returned whatever envelope
the peer sent, so a peer could answer a request for one envelope with another.
The three gossip receive paths decoded envelopes and delivered them under
whatever id they stated, so an envelope whose content did not hash to its id
propagated exactly like a genuine one.

Every gossip path now goes through `accept_envelope` (decode, then the content
must hash to its stated id — treatise §10.2 step 1). A fetch answer must state
the requested SAID and hash to it, and a different envelope is refused with
`Error::Substituted`. Unit tests cover a forged body under a genuine id, a
different envelope, and a relabelled one.

This uses the envelope's existing identity derivation; moving envelope ids to
the identity spec's SAIDs is a separate, wider change.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

a4b899a48c

• View at this point in history

2026-09-30 20:30:04 -04:00

Transaction Science

smart-byte-lockstep: the commit rule and dissent log as a pure function ...

`tally` takes the state hashes a frame's nodes reported and returns what the
treatise says follows: the floor(2n/3)+1 threshold (§13.1), each node's frame
outcome — clean commit, commit with dissent, minority divergence, cluster split
(§13.3) — and the dissent-log entry the committing nodes write (§14.1). A split
frame records no dissent: dissent is divergence from a committed hash (§14.2).
Missing reports still count against n; duplicate reports are refused.

`Cluster::step` now goes through it, so the simulation and the conformance
vectors share one implementation of the rule. It also keeps the dissent log the
treatise calls the substrate's real misbehaviour signal (§13.4): a node that
lies about its hash is now named on every frame it lied, where before the
commit simply succeeded and the lie left no trace.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

49b719b90b

• View at this point in history

2026-09-30 20:30:04 -04:00

Transaction Science

smart-byte-core: canonical CBOR and the identity spec's SAIDs ...

Identity spec §2 defines a SAID precisely: hash the canonical CBOR (§2.3) of
the object with its SAID field holding 53 `#`, and write the BLAKE3-256 digest
as `E` + lowercase unpadded base32. Nothing in smart-byte-rs did that. Envelope
ids and the key-event log hash `serde_cbor` output — struct-declaration key
order, `[u8; 32]` written as arrays of small integers — over a zeroed id, and
print upper-case base32 with no derivation code. The conformance pack could not
be generated from any of it.

- `canonical`: a CBOR data model with no float or tag variant, an encoder that
  sorts map keys by their encoded bytes (not their text), and a STRICT decoder
  that refuses every non-canonical form with its own error class instead of
  re-encoding it. Re-encoding is exactly what §2.5's adversarial vector exists
  to catch: it would let two byte strings verify under one SAID.
- `said`: the §2.2 text form (one spelling only: lowercase, `E`, zero trailing
  bits), `derive`, `verify` and the §3.4 per-key SAID. The module docs now say
  plainly which identifiers in this crate follow §2 and which do not yet, and
  the two contradictory doc comments about the text form's case are gone.
- `sign`: `sign_said` / `verify_said_signature`, verifying strictly (S in
  [0, L), no small-order points) and reporting malformed input before any
  cryptography runs.

Existing APIs are untouched; `Said` additionally derives `Ord`.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

464a40777b

• View at this point in history

2026-09-30 20:30:04 -04:00

2026-09-26

Transaction Science

smart-byte: record the editorial reason for the spec reference fix ...

CI caught this correctly and I had missed it. `conformance.yml`'s coupling gate
requires a change under `smart-byte/spec/` to be accompanied by a change under
`smart-byte/conformance/`, and the previous commit fixed three broken references
in the spec without touching the pack. The gate's own message gives the route
for an editorial change: record the reason in `conformance/v<N>/regeneration.md`
and include that file. This does that.

The wire format, the vectors and every `expected_*` value remain untouched. The
pack was not edited to satisfy the gate — only this log was added, which is what
the gate asks for.

Two further findings went into the same entry, since regeneration.md is where
whoever regenerates will look, and neither is fixed here because both are
normative:

- **The regeneration procedure is not runnable as written.** It names
  `cargo run --bin gen-conformance-vectors` to produce the pack and
  `verify-conformance-vectors` to check it. NEITHER binary exists — the
  workspace ships one binary, `smart-byte`. So the mechanical pass this document
  describes cannot be run in either direction, which is stronger than "it has
  not been run yet". I had previously found only the first of the two.

- **The precondition for regenerating has been met.** The status section says
  the placeholders stand until `smart-byte-rs` lands and calls it a private repo
  to be open-sourced at spec v1.0 freeze. It has landed and is in the
  `standards-build` matrix, so the blocker recorded there is stale.

The entry also carries forward the missing adversarial non-canonical CBOR
vector, so that a regeneration which replaces placeholder values without adding
it does not quietly leave the spec describing a suite the pack lacks.

`check-no-vanity-metrics.sh` passes; pack tests green. The coupling gate was
simulated locally against its own logic before pushing (spec_touched=1,
conformance_touched=1).

Co-Authored-By: Claude Opus 5 <[email protected]>

5391e895d5

• View at this point in history

2026-09-26 05:10:26 -04:00

2026-09-22

Transaction Science

smart-byte: the spec mandates a vector suite that does not exist ...

Follow-up to the pack audit in this branch, from asking why the vectors could
not simply be regenerated. The answer is that the spec is in worse shape than
the pack.

`spec/identity_and_key_rotation.md` says implementers **"MUST pass every
vector"** in the SAID pack. Every `expected_said` in that pack is a
repeated-character placeholder rather than a digest, so the requirement is
unsatisfiable as written — a conforming implementation cannot be conforming.

The same sentence describes the suite as covering "the empty-map case, the
one-field case, the worked example above, several envelope-shaped objects from
treatise §8, and one adversarial case where a non-canonical CBOR encoding
produces a different digest (to verify that implementations reject non-canonical
input rather than silently re-encoding)". The published file holds five vectors,
all envelope-shaped. **The adversarial case is the only negative test named, it
is the security-relevant one, and it is absent** — an implementation that
silently re-encodes non-canonical input passes this pack completely.

Pinned by `the_spec_mandates_a_suite_the_published_pack_does_not_contain`,
written to fail once the pack is regenerated, which is the signal to replace it
with a real assertion that the adversarial vector exists and is rejected.
Falsified both ways: adding an adversarial vector fails it, and softening the
spec's MUST fails it. Pack and spec restored and verified clean afterwards.

Also fixes a broken reference, which is a correction rather than a normative
change. The spec pointed three times at `../conformance/said_vectors.json`,
which does not exist — the file is at `../conformance/v1/said_vectors.json`.
Line 266 states "all paths are relative to this file", so the breakage was
unambiguous and so is the intent. An implementer following the normative
reference reached nothing.

Nothing else in the spec or the pack is touched. Regenerating the vectors, and
deciding whether the pack should gain the missing cases, stays a maintainer's
call. Workspace 622 passed, 0 failures; clippy clean;
`check-no-vanity-metrics.sh` passes.

Co-Authored-By: Claude Opus 5 <[email protected]>

fb8552a72c

• View at this point in history

2026-09-22 01:19:05 -04:00

Transaction Science

smart-byte: verify what the pack really contains, and pin what it does not ...

`smart-byte/conformance/v1` is published for third parties to certify against.
Its gate — `.github/workflows/conformance.yml`, named "smart-byte conformance
gate" — runs `jq` over it: valid JSON, top-level array, `name` and `description`
present, names unique. Those are well-formedness checks. No value in the pack
has ever been compared against the implementation, and `smart-byte-rs` has no
conformance crate that reads it.

`conformance/v1/regeneration.md` is candid that the `expected_*` fields are
placeholders "not yet cryptographically computed". So this is a known state,
openly documented, and not a defect anyone hid. What has changed is its stated
precondition: the doc says regenerate "when `smart-byte-rs` lands", describing
it as a "private repo, to be open-sourced alongside spec v1.0 freeze".
`smart-byte-rs` HAS landed — 22 crates, in the `standards-build` matrix — so
the one mechanical pass that doc describes is now possible, and overdue. Its
named entry point, `cargo run --bin gen-conformance-vectors`, does not exist.

What the pack actually contains, established by running it rather than reading
it:

REAL — every positive vector's `signing_key_seed` derives exactly the
`verifying_key` it publishes. That is asserted properly here, along with
`sig-004-wrong-key`'s deliberate mismatch, so the negative case cannot quietly
become a positive one.

NOT REAL — the `expected_*` values:
  - `sig-001` says it signs "a known envelope SAID". It does not. The value is
    RFC 8032's signature over the EMPTY message, proven by verifying it against
    `b""`. `sig-002`/`sig-003` carry another RFC 8032 signature, over neither
    their SAID nor the empty message.
  - Every SAID envelope carries the literal `<PLACEHOLDER_SAID>`; every
    `expected_said` is one repeated character rather than a digest; every
    `canonical_cbor_hex` is the same 7-byte truncated prefix.
  - `canon-002` and `canon-003` publish `expected_cbor_hex` of ODD length (109
    and 37 characters), which cannot be decoded to bytes at all. A third party
    fails before reaching any comparison.
  - The doc's own consistency claim does not hold in the direction that matters:
    `said-004` and `said-005` are different envelopes published with the SAME
    expected SAID, so even as filler the pack asserts a collision.

The pinning tests are written to FAIL once the pack is regenerated, which is
the intended signal to replace each with a real assertion — the same pattern
`map-conformance` uses for its JSON round-trip defect. Each was falsified by
simulating the fix: substituting a real-looking SAID, giving `said-005` a
distinct value, making `canon-002`'s hex even-length, corrupting a verifying
key, and replacing `sig-001`'s signature. All five fail as intended, and the
pack was restored and verified byte-identical to main.

Regenerating a published pack is a normative change to a public artifact and a
maintainer's call, so no vector is touched here. This only makes the state
visible in test output instead of in a doc nobody opens.

Co-Authored-By: Claude Opus 5 <[email protected]>

02c98eac84

• View at this point in history

2026-09-22 01:19:05 -04:00