open-standards/smart-byte/conformance/v1/regeneration.md

Regenerating v1 Conformance Vectors

Every value in this directory is computed by the Rust reference implementation and checked by a second, independent implementation. Nothing in the pack is a placeholder.

• Generator and verifier: the smart-byte-conformance crate in ../../smart-byte-rs. Names, descriptions and inputs are written in its src/pack.rs, src/logs.rs and src/states.rs, and the parties the vectors are built from in src/world.rs. Every expected value is computed at generation time by:
  • smart-byte-core: canonical CBOR, SAIDs, signatures, envelopes;
  • smart-byte-keri-witness: key-event logs;
  • smart-byte-state: the cluster state machine;
  • smart-byte-lockstep: the commit rule and dissent log.
• Second implementation: ../crosscheck/verify_v1.py, Python standard library only, written from the spec text in another language. It implements BLAKE3 and Ed25519 itself and checks both against known answers before it checks any vector. It then replays key-event logs, verifies envelopes and runs the cluster state machine, as the specs describe.

Procedure

From smart-byte/smart-byte-rs:

# Write the pack. --output-dir defaults to this directory.
cargo run --release -p smart-byte-conformance --bin gen-conformance-vectors -- \
    --spec-version v1 --output-dir ../conformance/v1

# Check a pack on disk against the reference implementation, vector by vector.
cargo run --release -p smart-byte-conformance --bin verify-conformance-vectors -- \
    --pack ../conformance/v1

# Check it against the second implementation.
python3 ../conformance/crosscheck/verify_v1.py ../conformance/v1

gen-conformance-vectors --check writes nothing and exits non-zero if any file differs from the generator's output.

To change a vector, edit smart-byte-conformance/src/pack.rs (or logs.rs, world.rs) and regenerate. Editing the JSON by hand fails CI: cargo test in smart-byte-rs requires the published files to equal the generator's output byte for byte, verifies every vector, and plants a defect in each file to prove the verifier would catch it. The conformance gate runs the second implementation on every pack change.

What the generator computes

1. Canonicalisation vectors. A positive vector's input is encoded to expected_cbor_hex, and the strict decoder must accept those bytes unchanged. A negative vector's input_cbor_hex must be refused with its error class; generation fails if the reference refuses it for any other reason, so a negative vector cannot pass by accident.
2. SAID vectors. input holds the 53-# placeholder in its SAID field d. Its canonical encoding is canonical_cbor_hex (the bytes that are hashed), their BLAKE3-256 is blake3_hex, that digest as E + lowercase base32 is expected_said, and the object with its SAID in place is said_cbor_hex, which must verify. A negative vector's input_cbor_hex is refused by SAID verification with its error class.
3. Signature vectors. The message is the canonical encoding of what is signed (message_hex): an issuer's attestation, a key event. A positive vector's signature is the deterministic Ed25519 signature by signing_key_seed; a negative vector is refused with its error class.
4. Key-event-log vectors. Each log is replayed by the reference's spec-v1 verifier, from its bytes. An accepted log's expected is what the replay establishes; a refused log names the class and the event (error_at) refused, and generation fails if the reference refuses it anywhere else or for any other reason. Every signature with a seed is the one that seed's key makes.
5. Envelope vectors. Each envelope is decoded and verified by the reference against the key states that kel-001 to kel-003 establish. An accepted envelope's expected is what verification establishes; a refused one names its class.
6. State vectors. Each frame is applied by the reference's state machine, from the vector's genesis state. Every input's result, and the state and hash the frame reaches, are what the reference computes. Generation fails if the reference applies or drops any input other than as the vector says.
7. Lockstep vectors. The hashes nodes reported are tallied against the threshold floor(2n/3)+1 (treatise §13.1) for the frame's public record. Each node's outcome (§13.3) and dissent log (§14.1) come from its own view: its peers' reported hashes plus the hash it computed, so a node that lies on the wire (§13.4) is modelled exactly as the treatise describes.
8. Gossip vectors. Each delivery goes through smart-byte-core's inbox, which implements identity spec §2.6: canonical bytes, then the SAID they state, then the SAID they arrived under, and only then deduplication.

Invariant: cross-vector consistency

The files describe one set of parties — an issuer, Alice and a custodian — and the byte that passes between them.

• said-001 to said-003 are the SAIDs of env-001's snapshot, its identity and env-002's snapshot; said-004 and said-005 are kel-002's rotation and interaction. said-007's key is the issuer's.
• sig-001 is the issuer's signature in env-001, and sig-002 is the signature on kel-002's inception, each without its algorithm identifier.
• kel-002's interaction anchors env-001's identity.
• state-001's founding frame holds kel-001 to kel-003's inceptions and env-001's mint, so the byte the cluster mints is env-001's.
• state-019 and state-020 are two clusters exchanging that byte. The export each records is the statement the other imports, under its nodes' certificate.
• Every envelope vector is checked against the key states the accepted logs it names establish. Both checkers enforce this: a log that fails takes the envelope vectors that rely on it down with it.
• Gossip deliveries carry SAID vectors' said_cbor_hex bytes, except where a vector alters them on purpose and says so.

cargo test checks the first three in tests/pack.rs.

Cross-language check

This document has always required a second implementation to reproduce the expected_* values before a regeneration merges. ../crosscheck/verify_v1.py is that implementation, and it reproduces every vector. Be clear about its limits: it was written by the same author as the generator, from the spec text rather than from the Rust. An implementation written by someone else is still the stronger test. The Go and TypeScript implementations an earlier draft of this document planned do not exist.

Editorial change log

2026-10-01 — governance, burns, and transfers between clusters

state_vectors.json grows to 23 vectors. The cluster state machine now covers everything its spec had left open. Every check the new rules add has a vector that fails an implementation leaving it out. That includes the quorum and certificate thresholds, which an electorate of three and a peer of three nodes tell apart from a simple majority. It also includes the class order, wherever two classes can touch one byte in one frame.

Governance (state-013 to state-017, and state-022). Registry and peer changes are proposals, approved by a supermajority of the issuers entitled to vote. They are numbered against replay, and take effect after the cluster's time-lock delay (treatise §15.1, §15.5). An issuer being removed has no vote, so it cannot block its own removal, even as its type's only issuer, when every other issuer votes. A registry with no issuers can never change.

Burns (state-018). The owner retires a byte under its own signature, and its identity stays held, so no transfer or replayed mint revives it. Redemption is a return to the issuer and the issuer's burn, in one frame.

Transfers between clusters (state-019 to state-021, and state-023):

• Export. A transition may name a destination cluster. Its signature and history hash cover it.
• Certificate. The source cluster's nodes certify the byte's state as it left.
• Import. The destination checks the certificate against a supermajority of its peer's node keys. It then verifies the byte's full history itself, from its envelope and its controllers' logs. Each certificate is imported once.
• The vectors run clusters A and B, with four nodes each, through a round trip A → B → A, and every refusal along the way. Each member of a certificate's export has its own mismatch case. An import meets the byte live, then burned, in state-023.

Also changed:

• The envelope's frame rule. Frames rise within a cluster, and the frame after a move to another cluster is in that cluster's own count. Requiring frames to rise along the whole chain would have refused every byte that came back to a cluster with a lower frame count.
• The state gains the cluster's identifier, its peers, governance, and the retired, exported and imported records. Every state vector's state and hash change with it; v1 is DRAFT, so they are regenerated in place.
• Lockstep nodes sign the certificates for a committed frame's exports, as the committing nodes applied the frame.
• state-011 names melt as the class there is not, now that burn is one. It adds malformed burn, governance and import inputs.

2026-10-01 — the cluster state machine, and its vectors

The pack gains state_vectors.json: 12 vectors, each a genesis state and frames of inputs. Every input's result, and the state and state hash every frame reaches, are pinned. The state hash had been a stand-in. The lockstep crate hashed a rolling chain over each frame's transitions, while treatise §9.4 commits a frame at the BLAKE3 of the canonical whole state. The state behind that hash is now built, and specified in spec/cluster_state.md.

What the vectors pin:

• The state's encoding: the frame number, the issuer registry, each controller's key state, and each byte's owner and history.
• The order inputs apply in: key events, then mints, then transfers, each class by ascending digest; identical inputs are one.
• Every check, with its error class:
  • a replayed mint, which is refused in every later frame;
  • an unregistered issuer, or an unregistered cargo type;
  • a rotation retiring the old key in the frame it lands in;
  • a mint and its transfer sharing a frame;
  • a double spend within one frame;
  • a wrong frame, a non-owner, a broken history, an unknown byte;
  • signature-rule failures, and a signer with no log in the cluster;
  • key-event failures and malformed inputs.

Departures from the treatise, recorded in cluster_state.md §7:

• A byte's identity carries no frame number. Treatise §15.2 folded the mint frame into it, and so a mint replayed in a later frame would issue a second byte (§19.3 concedes "a distinct (and possibly duplicate) byte").
• Parties are AIDs, so issuers rotate keys through their logs. That is the issuer key rotation §15.5 defers.
• Key events are a third input class.
• Joule cost stays out of the hashed state. Measured energy differs node by node.

Also changed:

• smart-byte-lockstep's cluster now runs these state machines, in place of the rolling hash.
• The CLI's cluster simulate runs a workload of real mints and transfers.
• smart-byte-keri-witness exposes a controller's state and resumes a replay from it, which is how the cluster extends a log one event at a time.
• The completeness checks count the error classes inside frames.

Falsified:

• Rust checker: six planted defects in the state vectors, each caught on its own vector.
• Python checker: 12 planted defects caught the same way, with and without -O. A frame deleted from the only vector exercising a class is caught by the completeness gate.
• Python rules: 18 state-machine rule mutations, each caught by a state vector.
• Rust rules: 16 of the reference's state-machine rules mutated one at a time, each compiling and failing named tests.

2026-09-30 — treatise §8.6: a counter counts only while it is live; no wire-format change

§8.6 now says when a node may read measured_microjoules from a hardware energy counter: only while that counter is demonstrably live for the work, meaning it has been seen advancing under load for the domain the measurement claims. A counter that is present but frozen, or that is republished in lumps, gives the same honest zero as an absent counter.

The wire format, the vectors and every expected_* value are unchanged. The change narrows when a node may report a non-zero measurement. It does not change the field, its encoding, or any SAID, signature, lockstep or gossip computation. The envelope and SAID vectors carry measured_microjoules as a fixed input (1200, 0 and 1), and their expected values depend on that number, not on how a node came to report it, so none of them changes. This entry is here because the conformance gate requires a spec change to come with a pack change, and directs an editorial change to record its reason in this file.

2026-09-30 — key-event logs and envelopes, on the v1 wire format

The pack gains kel_vectors.json (25 vectors) and envelope_vectors.json (18), and the SAID and signature vectors that described the draft envelope are rebuilt on the real one. v1 is DRAFT, so this is a regeneration in place.

What the new files pin. Key-event logs replayed per identity spec §4, and envelopes per the new envelope wire format. Each negative vector carries one defect, and the reference refuses it for that defect alone. The defects:

• Takeovers:
  • a rotation to uncommitted keys;
  • a rotation signed by the current keys rather than the keys it reveals;
  • a rotation that lowers the committed threshold;
  • a rotation after a log committed to no next keys;
  • an interaction that changes keys;
  • an inception claiming another controller's AID;
  • a rec that reveals keys of its own.
• Thresholds and keys: a threshold of 0, which no signature needs to meet; a key listed twice, which one holder could use to meet a two-key threshold.
• Log structure: a sequence gap, a wrong prior event, another controller's event, a changed event, and the rec and dip types v1 does not define.
• Decoding:
  • non-canonical bytes;
  • an undefined member;
  • a member written in another form;
  • an unsupported version;
  • a reserved algorithm;
  • a witness pool.
• Envelopes:
  • cargo changed after signing, with and without re-stamping;
  • a stale snapshot SAID;
  • a non-owner's transfer;
  • a grafted transition;
  • two transfers in one frame;
  • a signature naming a non-establishment event;
  • an old key under a new establishment;
  • below-threshold, duplicate-index, out-of-range and wrong-algorithm signatures;
  • non-canonical bytes;
  • an undefined member;
  • cargo bytes written as an array of integers.

Rebuilt on the v1 wire format:

• said-001 to said-005 were objects shaped like the draft envelope, which had alg, issuer_pubkey and a top-level cargo. That shape no longer exists. They are now the issued envelope, its identity, the transferred envelope, and a rotation and an interaction event, and they carry those objects' real SAIDs.
• Signature vectors signed the 32 digest bytes of a SAID. Nothing in v1 signs that: an issuer signs its attested fields, an owner its transition, a controller its event, each as canonical CBOR. The vectors now sign real messages (message_hex), and envelope_said and signed_message_hex are gone. sig-003 is renamed message-swapped.

Found and fixed in the reference while building the pack:

• wire::from_slice accepted canonical CBOR that was not the decoded object's own encoding. Serde takes an array of integers for a byte string, a byte string for text, and an array for a struct, and it skips a member written where it is omitted. Each gave one object a second byte string its SAID does not cover. The decoder now re-encodes and refuses anything but its input, which covers every type at once; kel-022 and env-018 pin it.

• The key-event-log verifier accepted all of the following:

  • a dip event at any sequence, with no pre-rotation check, which is a takeover path;
  • thresholds of 0;
  • duplicate keys;
  • rotations after an empty n;
  • every extension member, in spec-v1 mode.

  It now replays incrementally and reports which event it refused. Its error classes match the README's.

• The extension mode is fixed as well:

  • Witness receipts carried their own public key, so anyone could receipt in any witness's name. A witness is now named by its key. A receipt signs the controller, sequence and event, and counts only if its witness is in the log's pool.
  • Duplicity evidence was gathered from unverified receipts.
  • Recovery accepted any keys a rec revealed. It now counts only with its witnesses' receipts.
  • OOBIs were signed by a key embedded in the OOBI itself. They are now controller signatures, checked against the controller's log.
  • Configuration traits (c) were never enforced, so they are gone.

• A controller signature with a key state threshold of 0 needed no signatures. At least one is now always required.

• pre_rotation_mismatch_is_rejected asserted nothing: its matches! result was discarded.

Spec text changed with it:

• envelope_wire_format.md is new: the envelope, its signatures and its verification, with the error classes the pack uses.
• Identity spec:
  • §3.1 says an inception placeholders i with d.
  • §3.3 says an event has exactly its members, gives the threshold and distinct-key rules, and drops the extension note.
  • §3.5 adopts KERI's rule: a rotation is signed by the keys it reveals, under both the committed and its own threshold.
  • §4 steps 1, 5 and 6 match.
  • §2.5 and §6 name the new vectors.

Falsified:

• Rust checker: 28 planted defects across the seven files, each caught on its own vector. A defect in a log fails that log and every envelope that relies on it.
• Python checker: 20 planted defects caught the same way, with and without -O.
• Python rules: 30 rule mutations, each caught by the vector built for it.
• Rust rules: 12 of the reference's rules mutated one at a time, among them the rotation rules, the own-encoding check, duplicate keys and indexes, frame order, the inception's AID, the version, and rec/dip. Each mutant compiles, and each fails named tests. All but one break the pack itself: generation refuses to emit a vector the reference no longer refuses for its stated reason. The exception is a key state with threshold 0, which no valid log can produce; a unit test holds that rule.

2026-09-30 — the pack regenerated from the reference implementation

Every expected_* value was a placeholder. The SAIDs were runs of one repeated character, every canonical_cbor_hex was the same 7-byte stub, and the signatures were RFC 8032 test signatures over the empty message and over 0x72. None of them signed any SAID. All five canonicalisation encodings were wrong, and two of them were odd-length hex that could not be decoded at all. Every value is now computed, and checked twice (see above).

Regenerated in place. v1 is DRAFT, and a DRAFT pack is regenerated in place; supersession governs a frozen pack (../VERSIONS.md rule 4, now explicit about this). A few draft vectors did hold values an implementation could pass — lockstep-001's tally over identical hashes, and the sig-004 and sig-005 negatives. They keep their names and subjects here.

Inputs that had to change before anything could be computed:

• JSON has no byte-string type, and the draft wrote byte values as hex text, which is indistinguishable from text. Byte strings are now written {"$bytes": "<lowercase hex>"}, and $bytes is reserved (../README.md).
• SAID vectors: the SAID field is d, holding the 53-# placeholder (identity spec §2.1, §2.2). The draft used a said member holding <PLACEHOLDER_SAID>. The object member is now input rather than envelope, and the version member is gone.
• said-005's previous_history_hash values were SAID-shaped filler. They are now computed, and said-005's SAID is its own. The draft gave it said-004's SAID, but a SAID covers the whole object it is derived over.
• canon-003's byte string was given as base64 text; it is now bytes.

Vectors whose meaning contradicted the spec, corrected (and renamed where the name stated the wrong rule):

• canon-004-deterministic-float-encoding → canon-004-floats-forbidden. The draft encoded floats in their shortest exact width; identity spec §2.3 forbids floats. canon-016/017 cover the other widths.
• lockstep-003 logged dissent for all four nodes of a frame that did not commit. Treatise §14.2 writes the dissent log only on a commit with dissent.
• Lockstep vectors now separate the hash a node broadcast from the one it computed, and give each node's outcome and dissent log from its own view. The draft's single view called a node a "liar" while modelling a node whose state diverged; treatise §13.4's liar commits in its own view while its peers log it, which lockstep-006 and lockstep-007 now pin. lockstep-004 is renamed to say what it models: a diverged node.
• Lockstep vectors no longer carry ordered_transitions. They were never inputs to the commit rule, and their "signatures" were filler.
• gossip-002-said-conflict-rejected keeps its name but not its rule. The draft kept whichever object arrived first. Two different canonical objects cannot share a SAID short of a BLAKE3 collision, so the newcomer is refused because it does not hash to the SAID it states, in either arrival order (gossip-006). Verification precedes deduplication (identity spec §2.6).
• gossip-003-sequence-gap-detection → gossip-003-backfill-answer-verified. The draft asserted a 256-frame backfill limit, request targets and alert classes that no spec text defines. The replacement tests what the spec fixes: an answer to a request by SAID is checked against that SAID.
• Gossip vectors carry generic SAID-addressed objects, not envelope-shaped ones. A node that also checks a byte's provenance (treatise §8.3) must refuse an envelope with null or filler provenance, so such deliveries could not be MUST-pass. §2.6 applies to every SAID-addressed object.
• gossip-001 no longer asserts frames or round counts — gossip is not frame-synchronous — and expected_held now carries the bytes held.
• Signature vectors: sig-003 and sig-004 share the class signature_invalid. Both are the same failed equation, and no verifier can tell them apart. sig-004's key is derived from its seed, and sig-005's class is malformed_signature.

Vectors added:

• canon-006 to canon-020: each §2.3 rule, accepted and refused.
• said-006 to said-011: the suite identity spec §2.5 names.
• sig-006 (a non-canonical S) and sig-007 (a key spelled with y ≥ p).
• lockstep-004 to lockstep-007.
• gossip-004 to gossip-006.

Spec text changed with it (identity spec):

• §2.4 printed an invented digest and SAID, and counted 62 bytes; it now prints said-008's values for its 61 bytes.
• §2.1, §2.3, §2.4 and §4 step 1 told a verifier to re-encode an object canonically, which is exactly what §2.5's adversarial vector forbids. They now require received bytes to be canonical already.
• §2.3 now forbids simple values other than false, true and null.
• §2.6 is new: receiving SAID-addressed objects.
• §3.3 and §3.4 now say a key is a byte string.
• §2.5 and §6 now name the vectors and the modules.

Reference implementation changed with it:

• smart-byte-core:
  • canonical CBOR, with a strict decoder that never allocates from a declared count and reports the first defect in stream order;
  • §2 SAIDs and the §2.6 inbox;
  • SAID signature verification, with RFC 8032-strict key decoding;
  • Envelope::from_cbor, which accepts only an envelope's own encoding.
• smart-byte-lockstep: the §13/§14 tally and each node's own view. Cluster::step keeps a dissent log.
• smart-byte-libp2p:
  • gossip is validated before it is forwarded, so a forged or padded envelope stops at the first honest node;
  • a fetch's answer must be the envelope requested;
  • one bad message no longer ends a wait.
• smart-byte-keri-witness: the verifier rejects rec by default, as spec v1 requires. Recovery is an explicit opt-in.

Checked whole. Both checkers require every file to be non-empty, every vector the identity spec cites to be present, and every README error class to be exercised — and no other. A pack cannot pass by losing vectors. The Python checker raises its own failures rather than using assert, so python -O cannot switch it off. Both refuse JSON that names a member twice.

Next: envelope and key-event-log identifiers move to §2 SAIDs. The envelope's wire format and signing preimages get a spec of their own, and an inception event placeholders i with d, because it derives its controller AID from its own SAID.

2026-09-21 — spec reference corrected; no wire-format change

spec/identity_and_key_rotation.md pointed three times at ../conformance/said_vectors.json. That path does not resolve: the pack is at ../conformance/v1/said_vectors.json. Line 266 of that spec states "all paths are relative to this file", so both the breakage and the intent were unambiguous, and an implementer following the normative reference reached nothing. The three references now carry the v1/ segment.

The wire format, the vectors and every expected_* value are unchanged. This entry exists because the conformance gate requires a spec change to be accompanied by a pack change, and directs an editorial change to record its reason here. Recording it is the point; nothing in the pack was edited to satisfy the gate.

Two things found while making that correction, neither of them fixed here because both are normative decisions:

1. The regeneration procedure above is not runnable as written. It names cargo run --bin gen-conformance-vectors to produce the pack and verify-conformance-vectors to check it. Neither binary exists in smart-byte-rs — the workspace ships one binary, smart-byte. So the pass this document describes cannot be run in either direction, which is a stronger statement than "it has not been run yet".

2. The precondition this document gives for regenerating has been met. The status section says the placeholders stand until smart-byte-rs lands, and describes it as a private repo to be open-sourced at spec v1.0 freeze. It has landed, it is in this repository, and it is built and tested by the standards-build CI matrix. The blocker recorded here is stale.

Also worth carrying into whoever does the regeneration: spec/identity_and_key_rotation.md says implementers MUST pass every vector, and describes the suite as including "one adversarial case where a non-canonical CBOR encoding produces a different digest (to verify that implementations reject non-canonical input rather than silently re-encoding)". That vector is not in this pack — every SAID vector here is envelope-shaped, and the adversarial one is the only negative case the spec names. A regeneration that replaces the placeholder values without adding it would still leave the spec describing a suite this pack does not contain.