open-standards/smart-byte/conformance/v1/regeneration.md

# Regenerating v1 Conformance Vectors

Every value in this directory is computed by the Rust reference implementation
and checked by a second, independent implementation. Nothing in the pack is a
placeholder.

- **Generator and verifier:** the `smart-byte-conformance` crate in
  `../../smart-byte-rs`. Names, descriptions and inputs are written in its
  `src/pack.rs`, `src/logs.rs` and `src/states.rs`, and the parties the
  vectors are built from in `src/world.rs`. Every expected value is computed
  at generation time by:
  - `smart-byte-core`: canonical CBOR, SAIDs, signatures, envelopes;
  - `smart-byte-keri-witness`: key-event logs;
  - `smart-byte-state`: the cluster state machine;
  - `smart-byte-lockstep`: the commit rule and dissent log.
- **Second implementation:** `../crosscheck/verify_v1.py`, Python standard
  library only, written from the spec text in another language. It implements
  BLAKE3 and Ed25519 itself and checks both against known answers before it
  checks any vector. It then replays key-event logs, verifies envelopes and
  runs the cluster state machine, as the specs describe.

## Procedure

From `smart-byte/smart-byte-rs`:

```bash
# Write the pack. --output-dir defaults to this directory.
cargo run --release -p smart-byte-conformance --bin gen-conformance-vectors -- \
    --spec-version v1 --output-dir ../conformance/v1

# Check a pack on disk against the reference implementation, vector by vector.
cargo run --release -p smart-byte-conformance --bin verify-conformance-vectors -- \
    --pack ../conformance/v1

# Check it against the second implementation.
python3 ../conformance/crosscheck/verify_v1.py ../conformance/v1
```

`gen-conformance-vectors --check` writes nothing and exits non-zero if any file
differs from the generator's output.

To change a vector, edit `smart-byte-conformance/src/pack.rs` (or `logs.rs`,
`world.rs`) and regenerate.
Editing the JSON by hand fails CI: `cargo test` in `smart-byte-rs` requires the
published files to equal the generator's output byte for byte, verifies every
vector, and plants a defect in each file to prove the verifier would catch it.
The conformance gate runs the second implementation on every pack change.

## What the generator computes

1. **Canonicalisation vectors.** A positive vector's `input` is encoded to
   `expected_cbor_hex`, and the strict decoder must accept those bytes
   unchanged. A negative vector's `input_cbor_hex` must be refused with its
   `error` class; generation fails if the reference refuses it for any other
   reason, so a negative vector cannot pass by accident.
2. **SAID vectors.** `input` holds the 53-`#` placeholder in its SAID field
   `d`. Its canonical encoding is `canonical_cbor_hex` (the bytes that are
   hashed), their BLAKE3-256 is `blake3_hex`, that digest as `E` + lowercase
   base32 is `expected_said`, and the object with its SAID in place is
   `said_cbor_hex`, which must verify. A negative vector's `input_cbor_hex` is
   refused by SAID verification with its `error` class.
3. **Signature vectors.** The message is the canonical encoding of what is
   signed (`message_hex`): an issuer's attestation, a key event. A positive
   vector's `signature` is the deterministic Ed25519 signature by
   `signing_key_seed`; a negative vector is refused with its `error` class.
4. **Key-event-log vectors.** Each log is replayed by the reference's spec-v1
   verifier, from its bytes. An accepted log's `expected` is what the replay
   establishes; a refused log names the class and the event (`error_at`)
   refused, and generation fails if the reference refuses it anywhere else
   or for any other reason. Every signature with a `seed` is the one that
   seed's key makes.
5. **Envelope vectors.** Each envelope is decoded and verified by the
   reference against the key states that kel-001 to kel-003 establish. An
   accepted envelope's `expected` is what verification establishes; a refused
   one names its class.
6. **State vectors.** Each frame is applied by the reference's state machine,
   from the vector's genesis state. Every input's result, and the state and
   hash the frame reaches, are what the reference computes. Generation fails
   if the reference applies or drops any input other than as the vector says.
7. **Lockstep vectors.** The hashes nodes reported are tallied against the
   threshold floor(2n/3)+1 (treatise §13.1) for the frame's public record.
   Each node's outcome (§13.3) and dissent log (§14.1) come from its own
   view: its peers' reported hashes plus the hash it computed, so a node that
   lies on the wire (§13.4) is modelled exactly as the treatise describes.
8. **Gossip vectors.** Each delivery goes through `smart-byte-core`'s inbox,
   which implements identity spec §2.6: canonical bytes, then the SAID they
   state, then the SAID they arrived under, and only then deduplication.

## Invariant: cross-vector consistency

The files describe one set of parties — an issuer, Alice and a custodian —
and the byte that passes between them.

- said-001 to said-003 are the SAIDs of env-001's snapshot, its identity and
  env-002's snapshot; said-004 and said-005 are kel-002's rotation and
  interaction. said-007's key is the issuer's.
- sig-001 is the issuer's signature in env-001, and sig-002 is the signature
  on kel-002's inception, each without its algorithm identifier.
- kel-002's interaction anchors env-001's identity.
- state-001's founding frame holds kel-001 to kel-003's inceptions and
  env-001's mint, so the byte the cluster mints is env-001's.
- state-019 and state-020 are two clusters exchanging that byte. The export
  each records is the statement the other imports, under its nodes'
  certificate.
- Every envelope vector is checked against the key states the accepted logs
  it names establish. Both checkers enforce this: a log that fails takes the
  envelope vectors that rely on it down with it.
- Gossip deliveries carry SAID vectors' `said_cbor_hex` bytes, except where a
  vector alters them on purpose and says so.

`cargo test` checks the first three in `tests/pack.rs`.

## Cross-language check

This document has always required a second implementation to reproduce the
`expected_*` values before a regeneration merges. `../crosscheck/verify_v1.py`
is that implementation, and it reproduces every vector. Be clear about its
limits: it was written by the same author as the generator, from the spec text
rather than from the Rust. An implementation written by someone else is still
the stronger test. The Go and TypeScript implementations an earlier draft of
this document planned do not exist.

## Editorial change log

### 2026-10-01 — governance, burns, and transfers between clusters

`state_vectors.json` grows to 23 vectors. The cluster state machine now
covers everything its spec had left open. Every check the new rules add has
a vector that fails an implementation leaving it out. That includes the
quorum and certificate thresholds, which an electorate of three and a peer of
three nodes tell apart from a simple majority. It also includes the class
order, wherever two classes can touch one byte in one frame.

**Governance** (state-013 to state-017, and state-022). Registry and peer changes are
proposals, approved by a supermajority of the issuers entitled to vote. They
are numbered against replay, and take effect after the cluster's time-lock
delay (treatise §15.1, §15.5). An issuer being removed has no vote, so it
cannot block its own removal, even as its type's only issuer, when every
other issuer votes. A registry with no issuers can never change.

**Burns** (state-018). The owner retires a byte under its own signature, and
its identity stays held, so no transfer or replayed mint revives it. Redemption is a return to the
issuer and the issuer's burn, in one frame.

**Transfers between clusters** (state-019 to state-021, and state-023):

- **Export.** A transition may name a destination cluster. Its signature and
  history hash cover it.
- **Certificate.** The source cluster's nodes certify the byte's state as it
  left.
- **Import.** The destination checks the certificate against a supermajority
  of its peer's node keys. It then verifies the byte's full history itself,
  from its envelope and its controllers' logs. Each certificate is imported
  once.
- **The vectors** run clusters A and B, with four nodes each, through a round
  trip A → B → A, and every refusal along the way. Each member of a
  certificate's export has its own mismatch case. An import meets the byte
  live, then burned, in state-023.

**Also changed:**

- **The envelope's frame rule.** Frames rise within a cluster, and the frame
  after a move to another cluster is in that cluster's own count. Requiring
  frames to rise along the whole chain would have refused every byte that
  came back to a cluster with a lower frame count.
- **The state** gains the cluster's identifier, its peers, governance, and
  the retired, exported and imported records. Every state vector's state
  and hash change with it; v1 is DRAFT, so they are regenerated in place.
- **Lockstep nodes** sign the certificates for a committed frame's exports,
  as the committing nodes applied the frame.
- **state-011** names `melt` as the class there is not, now that `burn` is
  one. It adds malformed burn, governance and import inputs.

### 2026-10-01 — the cluster state machine, and its vectors

The pack gains `state_vectors.json`: 12 vectors, each a genesis state and
frames of inputs. Every input's result, and the state and state hash every
frame reaches, are pinned. The state hash had been a stand-in. The lockstep
crate hashed a rolling chain over each frame's transitions, while treatise
§9.4 commits a frame at the BLAKE3 of the canonical whole state. The state
behind that hash is now built, and specified in `spec/cluster_state.md`.

**What the vectors pin:**

- **The state's encoding:** the frame number, the issuer registry, each
  controller's key state, and each byte's owner and history.
- **The order inputs apply in:** key events, then mints, then transfers, each
  class by ascending digest; identical inputs are one.
- **Every check, with its error class:**
  - a replayed mint, which is refused in every later frame;
  - an unregistered issuer, or an unregistered cargo type;
  - a rotation retiring the old key in the frame it lands in;
  - a mint and its transfer sharing a frame;
  - a double spend within one frame;
  - a wrong frame, a non-owner, a broken history, an unknown byte;
  - signature-rule failures, and a signer with no log in the cluster;
  - key-event failures and malformed inputs.

**Departures from the treatise, recorded in `cluster_state.md` §7:**

- **A byte's identity carries no frame number.** Treatise §15.2 folded the
  mint frame into it, and so a mint replayed in a later frame would issue a
  second byte (§19.3 concedes "a distinct (and possibly duplicate) byte").
- **Parties are AIDs, so issuers rotate keys through their logs.** That is
  the issuer key rotation §15.5 defers.
- **Key events are a third input class.**
- **Joule cost stays out of the hashed state.** Measured energy differs node
  by node.

**Also changed:**

- `smart-byte-lockstep`'s cluster now runs these state machines, in place of
  the rolling hash.
- The CLI's `cluster simulate` runs a workload of real mints and transfers.
- `smart-byte-keri-witness` exposes a controller's state and resumes a replay
  from it, which is how the cluster extends a log one event at a time.
- The completeness checks count the error classes inside frames.

**Falsified:**

- **Rust checker:** six planted defects in the state vectors, each caught on
  its own vector.
- **Python checker:** 12 planted defects caught the same way, with and without
  `-O`. A frame deleted from the only vector exercising a class is caught by
  the completeness gate.
- **Python rules:** 18 state-machine rule mutations, each caught by a state
  vector.
- **Rust rules:** 16 of the reference's state-machine rules mutated one at a
  time, each compiling and failing named tests.

### 2026-09-30 — treatise §8.6: a counter counts only while it is live; no wire-format change

§8.6 now says when a node may read `measured_microjoules` from a hardware
energy counter: only while that counter is demonstrably live for the work,
meaning it has been seen advancing under load for the domain the
measurement claims. A counter that is present but frozen, or that is
republished in lumps, gives the same honest zero as an absent counter.

**The wire format, the vectors and every `expected_*` value are unchanged.**
The change narrows when a node may report a non-zero measurement. It does
not change the field, its encoding, or any SAID, signature, lockstep or
gossip computation. The envelope and SAID vectors carry
`measured_microjoules` as a fixed input (1200, 0 and 1), and their expected
values depend on that number, not on how a node came to report it, so none
of them changes. This entry is here because the conformance gate requires a
spec change to come with a pack change, and directs an editorial change to
record its reason in this file.

### 2026-09-30 — key-event logs and envelopes, on the v1 wire format

The pack gains `kel_vectors.json` (25 vectors) and `envelope_vectors.json`
(18), and the SAID and signature vectors that described the draft envelope
are rebuilt on the real one. v1 is DRAFT, so this is a regeneration in place.

**What the new files pin.** Key-event logs replayed per identity spec §4, and
envelopes per the new envelope wire format. Each negative vector carries one
defect, and the reference refuses it for that defect alone. The defects:

- **Takeovers:**
  - a rotation to uncommitted keys;
  - a rotation signed by the current keys rather than the keys it reveals;
  - a rotation that lowers the committed threshold;
  - a rotation after a log committed to no next keys;
  - an interaction that changes keys;
  - an inception claiming another controller's AID;
  - a `rec` that reveals keys of its own.
- **Thresholds and keys:** a threshold of 0, which no signature needs to
  meet; a key listed twice, which one holder could use to meet a two-key
  threshold.
- **Log structure:** a sequence gap, a wrong prior event, another
  controller's event, a changed event, and the `rec` and `dip` types v1 does
  not define.
- **Decoding:**
  - non-canonical bytes;
  - an undefined member;
  - a member written in another form;
  - an unsupported version;
  - a reserved algorithm;
  - a witness pool.
- **Envelopes:**
  - cargo changed after signing, with and without re-stamping;
  - a stale snapshot SAID;
  - a non-owner's transfer;
  - a grafted transition;
  - two transfers in one frame;
  - a signature naming a non-establishment event;
  - an old key under a new establishment;
  - below-threshold, duplicate-index, out-of-range and wrong-algorithm
    signatures;
  - non-canonical bytes;
  - an undefined member;
  - cargo bytes written as an array of integers.

**Rebuilt on the v1 wire format:**

- said-001 to said-005 were objects shaped like the draft envelope, which
  had `alg`, `issuer_pubkey` and a top-level `cargo`. That shape no longer
  exists. They are now the issued envelope, its identity, the transferred
  envelope, and a rotation and an interaction event, and they carry those
  objects' real SAIDs.
- Signature vectors signed the 32 digest bytes of a SAID. Nothing in v1 signs
  that: an issuer signs its attested fields, an owner its transition, a
  controller its event, each as canonical CBOR. The vectors now sign real
  messages (`message_hex`), and `envelope_said` and `signed_message_hex` are
  gone. sig-003 is renamed `message-swapped`.

**Found and fixed in the reference while building the pack:**

- `wire::from_slice` accepted canonical CBOR that was not the decoded
  object's own encoding. Serde takes an array of integers for a byte string,
  a byte string for text, and an array for a struct, and it skips a member
  written where it is omitted. Each gave one object a second byte string its
  SAID does not cover. The decoder now re-encodes and refuses anything but
  its input, which covers every type at once; kel-022 and env-018 pin it.
- The key-event-log verifier accepted all of the following:
  - a `dip` event at any sequence, with no pre-rotation check, which is a
    takeover path;
  - thresholds of 0;
  - duplicate keys;
  - rotations after an empty `n`;
  - every extension member, in spec-v1 mode.

  It now replays incrementally and reports which event it refused. Its
  error classes match the README's.
- The extension mode is fixed as well:
  - **Witness receipts** carried their own public key, so anyone could
    receipt in any witness's name. A witness is now named by its key. A
    receipt signs the controller, sequence and event, and counts only if
    its witness is in the log's pool.
  - **Duplicity evidence** was gathered from unverified receipts.
  - **Recovery** accepted any keys a `rec` revealed. It now counts only with
    its witnesses' receipts.
  - **OOBIs** were signed by a key embedded in the OOBI itself. They are
    now controller signatures, checked against the controller's log.
  - **Configuration traits** (`c`) were never enforced, so they are gone.
- A controller signature with a key state threshold of 0 needed no
  signatures. At least one is now always required.
- `pre_rotation_mismatch_is_rejected` asserted nothing: its `matches!`
  result was discarded.

**Spec text changed with it:**

- `envelope_wire_format.md` is new: the envelope, its signatures and its
  verification, with the error classes the pack uses.
- Identity spec:
  - §3.1 says an inception placeholders `i` with `d`.
  - §3.3 says an event has exactly its members, gives the threshold and
    distinct-key rules, and drops the extension note.
  - §3.5 adopts KERI's rule: a rotation is signed by the keys it reveals,
    under both the committed and its own threshold.
  - §4 steps 1, 5 and 6 match.
  - §2.5 and §6 name the new vectors.

**Falsified:**

- **Rust checker:** 28 planted defects across the seven files, each caught
  on its own vector. A defect in a log fails that log and every envelope
  that relies on it.
- **Python checker:** 20 planted defects caught the same way, with and
  without `-O`.
- **Python rules:** 30 rule mutations, each caught by the vector built for
  it.
- **Rust rules:** 12 of the reference's rules mutated one at a time,
  among them the rotation rules, the own-encoding check, duplicate keys and
  indexes, frame order, the inception's AID, the version, and `rec`/`dip`.
  Each mutant compiles, and each fails named tests. All but one break the
  pack itself: generation refuses to emit a vector the reference no longer
  refuses for its stated reason. The exception is a key state with
  threshold 0, which no valid log can produce; a unit test holds that rule.

### 2026-09-30 — the pack regenerated from the reference implementation

Every `expected_*` value was a placeholder. The SAIDs were runs of one repeated
character, every `canonical_cbor_hex` was the same 7-byte stub, and the
signatures were RFC 8032 test signatures over the empty message and over 0x72.
None of them signed any SAID. All five canonicalisation encodings were wrong,
and two of them were odd-length hex that could not be decoded at all. Every
value is now computed, and checked twice (see above).

**Regenerated in place.** v1 is DRAFT, and a DRAFT pack is regenerated in
place; supersession governs a frozen pack (`../VERSIONS.md` rule 4, now
explicit about this). A few draft vectors did hold values an implementation
could pass — lockstep-001's tally over identical hashes, and the sig-004 and
sig-005 negatives. They keep their names and subjects here.

**Inputs that had to change before anything could be computed:**

- JSON has no byte-string type, and the draft wrote byte values as hex text,
  which is indistinguishable from text. Byte strings are now written
  `{"$bytes": "<lowercase hex>"}`, and `$bytes` is reserved (`../README.md`).
- SAID vectors: the SAID field is `d`, holding the 53-`#` placeholder (identity
  spec §2.1, §2.2). The draft used a `said` member holding `<PLACEHOLDER_SAID>`.
  The object member is now `input` rather than `envelope`, and the `version`
  member is gone.
- `said-005`'s `previous_history_hash` values were SAID-shaped filler. They
  are now computed, and said-005's SAID is its own. The draft gave it
  said-004's SAID, but a SAID covers the whole object it is derived over.
- `canon-003`'s byte string was given as base64 text; it is now bytes.

**Vectors whose meaning contradicted the spec, corrected (and renamed where the
name stated the wrong rule):**

- `canon-004-deterministic-float-encoding` → `canon-004-floats-forbidden`. The
  draft encoded floats in their shortest exact width; identity spec §2.3
  forbids floats. canon-016/017 cover the other widths.
- `lockstep-003` logged dissent for all four nodes of a frame that did not
  commit. Treatise §14.2 writes the dissent log only on a commit with dissent.
- Lockstep vectors now separate the hash a node broadcast from the one it
  computed, and give each node's outcome and dissent log from its own view.
  The draft's single view called a node a "liar" while modelling a node whose
  state diverged; treatise §13.4's liar commits in its own view while its
  peers log it, which lockstep-006 and lockstep-007 now pin. lockstep-004 is
  renamed to say what it models: a diverged node.
- Lockstep vectors no longer carry `ordered_transitions`. They were never
  inputs to the commit rule, and their "signatures" were filler.
- `gossip-002-said-conflict-rejected` keeps its name but not its rule. The
  draft kept whichever object arrived first. Two different canonical objects
  cannot share a SAID short of a BLAKE3 collision, so the newcomer is refused
  because it does not hash to the SAID it states, in either arrival order
  (gossip-006). Verification precedes deduplication (identity spec §2.6).
- `gossip-003-sequence-gap-detection` → `gossip-003-backfill-answer-verified`.
  The draft asserted a 256-frame backfill limit, request targets and alert
  classes that no spec text defines. The replacement tests what the spec
  fixes: an answer to a request by SAID is checked against that SAID.
- Gossip vectors carry generic SAID-addressed objects, not envelope-shaped
  ones. A node that also checks a byte's provenance (treatise §8.3) must
  refuse an envelope with null or filler provenance, so such deliveries could
  not be MUST-pass. §2.6 applies to every SAID-addressed object.
- `gossip-001` no longer asserts frames or round counts — gossip is not
  frame-synchronous — and `expected_held` now carries the bytes held.
- Signature vectors: sig-003 and sig-004 share the class `signature_invalid`.
  Both are the same failed equation, and no verifier can tell them apart.
  sig-004's key is derived from its seed, and sig-005's class is
  `malformed_signature`.

**Vectors added:**

- canon-006 to canon-020: each §2.3 rule, accepted and refused.
- said-006 to said-011: the suite identity spec §2.5 names.
- sig-006 (a non-canonical S) and sig-007 (a key spelled with y ≥ p).
- lockstep-004 to lockstep-007.
- gossip-004 to gossip-006.

**Spec text changed with it (identity spec):**

- §2.4 printed an invented digest and SAID, and counted 62 bytes; it now
  prints said-008's values for its 61 bytes.
- §2.1, §2.3, §2.4 and §4 step 1 told a verifier to re-encode an object
  canonically, which is exactly what §2.5's adversarial vector forbids. They
  now require received bytes to be canonical already.
- §2.3 now forbids simple values other than false, true and null.
- §2.6 is new: receiving SAID-addressed objects.
- §3.3 and §3.4 now say a key is a byte string.
- §2.5 and §6 now name the vectors and the modules.

**Reference implementation changed with it:**

- `smart-byte-core`:
  - canonical CBOR, with a strict decoder that never allocates from a
    declared count and reports the first defect in stream order;
  - §2 SAIDs and the §2.6 inbox;
  - SAID signature verification, with RFC 8032-strict key decoding;
  - `Envelope::from_cbor`, which accepts only an envelope's own encoding.
- `smart-byte-lockstep`: the §13/§14 tally and each node's own view.
  `Cluster::step` keeps a dissent log.
- `smart-byte-libp2p`:
  - gossip is validated before it is forwarded, so a forged or padded
    envelope stops at the first honest node;
  - a fetch's answer must be the envelope requested;
  - one bad message no longer ends a wait.
- `smart-byte-keri-witness`: the verifier rejects `rec` by default, as spec
  v1 requires. Recovery is an explicit opt-in.

**Checked whole.** Both checkers require every file to be non-empty, every
vector the identity spec cites to be present, and every README error class
to be exercised — and no other. A pack cannot pass by losing vectors. The
Python checker raises its own failures rather than using `assert`, so
`python -O` cannot switch it off. Both refuse JSON that names a member twice.

**Next:** envelope and key-event-log identifiers move to §2 SAIDs. The
envelope's wire format and signing preimages get a spec of their own, and an
inception event placeholders `i` with `d`, because it derives its controller
AID from its own SAID.

### 2026-09-21 — spec reference corrected; no wire-format change

`spec/identity_and_key_rotation.md` pointed three times at
`../conformance/said_vectors.json`. That path does not resolve: the pack is at
`../conformance/v1/said_vectors.json`. Line 266 of that spec states "all paths
are relative to this file", so both the breakage and the intent were
unambiguous, and an implementer following the normative reference reached
nothing. The three references now carry the `v1/` segment.

**The wire format, the vectors and every `expected_*` value are unchanged.**
This entry exists because the conformance gate requires a spec change to be
accompanied by a pack change, and directs an editorial change to record its
reason here. Recording it is the point; nothing in the pack was edited to
satisfy the gate.

Two things found while making that correction, neither of them fixed here
because both are normative decisions:

1. **The regeneration procedure above is not runnable as written.** It names
   `cargo run --bin gen-conformance-vectors` to produce the pack and
   `verify-conformance-vectors` to check it. **Neither binary exists** in
   `smart-byte-rs` — the workspace ships one binary, `smart-byte`. So the pass
   this document describes cannot be run in either direction, which is a
   stronger statement than "it has not been run yet".

2. **The precondition this document gives for regenerating has been met.** The
   status section says the placeholders stand until `smart-byte-rs` lands, and
   describes it as a private repo to be open-sourced at spec v1.0 freeze. It has
   landed, it is in this repository, and it is built and tested by the
   `standards-build` CI matrix. The blocker recorded here is stale.

Also worth carrying into whoever does the regeneration:
`spec/identity_and_key_rotation.md` says implementers **MUST pass every vector**,
and describes the suite as including "one adversarial case where a non-canonical
CBOR encoding produces a different digest (to verify that implementations reject
non-canonical input rather than silently re-encoding)". **That vector is not in
this pack** — every SAID vector here is envelope-shaped, and the adversarial one
is the only negative case the spec names. A regeneration that replaces the placeholder
values without adding it would still leave the spec describing a suite this pack
does not contain.