open-standards/smart-byte/spec/cluster_state.md

Cluster State

Smart Byte Substrate — Spec, Section: Cluster State. v1.0-draft.

---

1. What this section fixes

Treatise §9.4 describes a frame as six steps. A node collects inputs, exchanges them with its peers, orders them, applies them to its state, hashes the result, and commits when a supermajority reports the same hash. Other sections add to this:

• §15 adds live issuance, and issuer governance.
• §11.3.3 adds transfers between clusters.
• §16 adds persistence.

This section fixes what those sections leave open:

• what the state is, byte for byte;
• what each kind of input is;
• the order inputs apply in, and every check each one must pass;
• how the registry and the peers change;
• how a byte is burned, and how it moves between clusters;
• the hash a frame commits at.

The supermajority commit itself is treatise §13. Encodings are canonical CBOR as identity spec §2.3 defines it. Envelopes, provenances and transitions are as the envelope wire format defines them.

---

2. The state

{
  "frame":       uint: the next frame to apply,
  "cluster":     SAID: this cluster's identifier,
  "peers":       {cluster: {"keys": [bytes]}},
  "registry":    {issuer AID: [cargo type, ...]},
  "governance":  {"delay": uint, "sequence": uint, "pending": [{"change", "effective"}, ...]},
  "controllers": {AID: controller record},
  "bytes":       {identity: byte record},
  "retired":     {identity: retired record},
  "exported":    {identity: export record},
  "imported":    [32 bytes, ...]
}

• frame is 0 at genesis. Applying frame n leaves n + 1.

• cluster is the cluster's identifier, chosen at genesis: conventionally the SAID of its founding membership.

• peers maps each cluster this one exchanges bytes with to the keys of that cluster's nodes, each with its algorithm identifier (01 for Ed25519). A cluster is never its own peer.

• registry maps each issuer, by controller AID, to the cargo types it may mint, in ascending order with each listed once. This is treatise §15.1's issuer registry.

• governance holds:

  • the frames a change waits before it takes effect (delay);
  • the sequence number the next proposal must carry;
  • the accepted changes not yet in effect, in the order they were accepted (§4.5).

• controllers holds a record for every controller whose key-event log the cluster has applied:

  {
    "establishment":  uint: the sequence number of the establishment event in force,
    "keys":           [bytes]: the signing keys in force (identity spec §3.3 `k`),
    "threshold":      uint: the signing threshold in force (`kt`),
    "next_keys":      [SAID]: the next-key digests committed (`n`),
    "next_threshold": uint: the next-key threshold committed (`nt`),
    "sequence":       uint: the last applied event's `s`,
    "last_event":     SAID: the last applied event's `d`
  }
  

• bytes holds a record for every byte the cluster holds, keyed by the byte's identity:

  {
    "issuer":       AID: the issuer that minted it,
    "cargo":        cargo, as the issuer attested it (envelope wire format §6),
    "minted":       uint: the frame it was minted in, in the cluster that minted it,
    "owner":        AID: its current owner,
    "history_hash": 32 bytes: its current history hash (envelope wire format §5)
  }
  

• retired holds every byte burned here (§4.7): the byte record as it was, plus "retired", the frame it was burned in.

• exported holds every byte that left for another cluster and has not come back (§5). The record holds:

  • the byte's issuer, cargo and mint frame;
  • its owner and history hash as it left;
  • "destination", the cluster it left for;
  • "exported", the frame it left in.

• imported lists, in ascending order, the digests of the export certificates this cluster has imported (§5).

Every map above has exactly the members shown. The encoding is canonical: map keys sort by their encoded bytes. So two nodes holding the same state write the same bytes, whatever order they hold it in.

A burned or exported byte's identity stays in the state. So the cluster never mints that identity again, and never moves it, except to take back an exported byte that returns (§5).

The state holds a byte's current owner and history hash, not its envelope. Holders keep their own envelopes, and the cluster vouches for the current state (treatise §10.3). Two other things are not consensus state:

• The dissent log is kept by each node for itself (treatise §14).
• Joule cost is reported beside the state, not in it (§8).

---

3. Inputs

An input is the canonical encoding of a map with one member, which names the input's class:

{"key_event":  {"event": key event, "signatures": [{"index": uint, "signature": bytes}, ...]}}
{"governance": {"proposal": proposal, "approvals": [{"approver": AID, "signature": controller signature}, ...]}}
{"mint":       provenance}
{"import":     {"certificate": certificate, "envelope": envelope, "logs": [[signed event, ...], ...]}}
{"transfer":   {"byte": identity, "transition": transition}}
{"burn":       {"byte": identity, "previous_history_hash": 32 bytes, "owner": AID, "frame": uint, "signature": controller signature}}

• key_event carries one event of a controller's key-event log (identity spec §3.3), and the signatures over its canonical encoding (§3.5).
• governance carries a proposed change and the issuers' approvals of it (§4.5).
• mint is an issuer's provenance (envelope wire format §3). Its d is the new byte's identity.
• import brings a byte from another cluster. It carries:
  • the source cluster's certificate;
  • the byte's envelope;
  • the key-event logs that verify its history: each log is the signed events {"event", "signatures"} of one controller, from its inception (§5).
• transfer is a transition (envelope wire format §5) of the byte that byte names. A transition with a destination moves the byte to another cluster (§5).
• burn is an owner's retirement of a byte (§4.7).

An input's digest is the BLAKE3-256 of its bytes.

---

4. A frame

A node applies a frame's inputs to a state whose frame is F:

1. Governance. Pending changes whose effective frame is F take effect, in the order they were accepted (§4.5), and leave pending.
2. Duplicates. Identical inputs, byte for byte, are one input.
3. Order. Inputs apply by class: key events, governance, mints, imports, transfers, burns. Inputs that name no class come last. Within a class, inputs apply in ascending bytewise order of their digests (treatise §9.4 step 4).
4. Application. Each input is checked against the state as the inputs before it left it. An input that fails a check is dropped. It changes nothing, and every honest node drops the same inputs. The checks, in order, are given in the subsections below.
5. Advance. frame becomes F + 1.

The class order has reasons:

• Key events apply first, so that a rotation retires its old keys before anything signed with them is checked.
• Mints and imports apply before transfers, so that a byte can arrive and move in one frame (treatise §15.3).
• Burns apply last, so that a byte can be returned to its issuer and burned in one frame.

4.1 Decoding

For every input:

• The bytes are canonical (identity spec §2.3). The canonical-encoding classes apply.

• They are a map with one text member, naming one of the six classes. Otherwise: malformed_input.

• The member's value is exactly its class's form, with each part in its own encoding:

  • A key_event holds exactly event and signatures. The signatures are an array of {index, signature}. The event is then decoded as identity spec §4 step 1 decodes one, with that step's error classes: rec and dip are refused before anything else is read.
  • An import holds exactly certificate, envelope and logs. The certificate must have exactly its form, and the logs must be arrays of {event, signatures}. The envelope and the logged events are decoded when the import is applied (§5).
  • A governance, mint, transfer or burn input holds exactly its body.

  Any other shape: malformed_input.

4.2 Key events

• An inception starts a log. If the cluster already holds its controller: controller_exists. Otherwise it is checked as the first event of a log (identity spec §4).
• A rotation or interaction extends a log. If the cluster holds no log for its controller: unknown_controller. Otherwise it is checked as the next event of that log (identity spec §4 steps 2–6), against the controller's record.

An accepted event replaces the controller's record with the key state it establishes.

4.3 Mints

1. The provenance's d is its SAID. Otherwise: said_mismatch.
2. The registry lists the issuer for the cargo's type. Otherwise: issuer_not_registered.
3. The issuer's signature passes the signature check of §4.8, over the attested fields (envelope wire format §3).
4. The cluster has never held a byte with this identity — live, retired or exported. Otherwise: duplicate_byte.

The byte is then held with these values:

Member	Value
issuer	the issuer
cargo	the cargo
minted	F
owner	the recipient
history_hash	the identity's 32 digest bytes

4.4 Transfers

1. The cluster holds the byte. Otherwise:
   • byte_retired, if it was burned;
   • byte_exported, if it left;
   • unknown_byte, if neither.
2. The transition's frame is F. Otherwise: wrong_frame.
3. Its previous_history_hash is the byte's history hash. Otherwise: broken_history_chain.
4. Its from is the byte's owner. Otherwise: not_current_owner.
5. A destination, if the transition has one, is a peer. Otherwise: unknown_cluster.
6. The owner's signature passes the signature check of §4.8, over the transition's signed fields (envelope wire format §5).

A transition with no destination makes to the byte's owner and advances its history hash. One with a destination exports the byte (§5).

A byte's history hash moves with every transfer. Two transfers of one byte in one frame therefore cannot both apply: the second continues a history that has moved on (broken_history_chain). So a byte moves at most once a frame, as treatise §8.4 requires.

4.5 Governance

A proposal is:

{"cluster": SAID, "sequence": uint, "change": change}

The change is one of:

{"action": "add_issuer",    "cargo_type": uint, "issuer": AID}
{"action": "remove_issuer", "cargo_type": uint, "issuer": AID}
{"action": "set_peer",      "peer": SAID, "keys": [bytes]}
{"action": "remove_peer",   "peer": SAID}

Each approval is an issuer's controller signature over the proposal's canonical encoding. The checks, in order:

1. The proposal's cluster is this one. Otherwise: wrong_cluster.

2. Its sequence is the governance sequence number. Otherwise: stale_sequence.

3. The change makes sense:

   • add_issuer: the issuer is not registered for the type. Otherwise: already_registered.
   • remove_issuer: it is registered. Otherwise: not_registered.
   • set_peer: the peer is not this cluster (wrong_cluster). Its keys are at least one (invalid_threshold), none twice (duplicate_key), each an Ed25519 key (malformed_verifying_key).
   • remove_peer: the cluster is a peer. Otherwise: not_a_peer.

4. The change has an electorate: the issuers who may approve it.

   • Registry changes: the issuers registered for its cargo type. When no issuer is registered for the type, the electorate is every issuer.
   • Peer changes: every issuer.
   • Removals: an issuer being removed is never in the electorate, so it cannot block its own removal.

   If the electorate is empty: no_electorate.

5. Each approval, in order, is by a member of the electorate (not_an_elector). No member approves twice (duplicate_approver). The approval's signature passes the signature check of §4.8.

6. The approvals number at least ⌊2m/3⌋+1, for an electorate of m. Otherwise: below_quorum.

An accepted proposal advances the governance sequence number, so it can never be applied twice. Its change joins pending with effective F + 1 + delay. The change takes effect at the start of that frame (§4 step 1):

• an issuer is added to or removed from its cargo type, and an issuer left with no types leaves the registry;
• a peer is set or removed.

The delay is the time-lock treatise §15.5 asks for. Everyone can see an accepted change before it takes effect.

4.6 Imports

See §5.

4.7 Burns

1. The cluster holds the byte, exactly as for a transfer (§4.4 step 1).
2. The burn's frame is F. Otherwise: wrong_frame.
3. Its previous_history_hash is the byte's history hash. Otherwise: broken_history_chain.
4. Its owner is the byte's owner. Otherwise: not_current_owner.
5. The owner's signature passes the signature check of §4.8, over the canonical encoding of {"burn": identity, "previous_history_hash", "owner", "frame"}.

The byte moves from bytes to retired, with the frame it was burned in.

Redemption (treatise §15.4) is a transfer to the issuer followed by the issuer's burn. Both can happen in one frame.

4.8 Signatures

A signature on a mint, transfer, burn or approval passes when all of the following hold:

1. The cluster holds the signer's key-event log. Otherwise: unknown_controller.
2. The signature names the establishment event in force, the record's establishment. Otherwise: establishment_not_in_force.
3. It is a valid controller signature under that event's key state (envelope wire format §7, with its error classes).

A signature made under keys since rotated away still verifies offline (envelope wire format §8). The cluster accepts new signatures only under the keys in force, which is what retires stolen keys.

---

5. Transfers between clusters

A byte moves from cluster A to cluster B in three steps (treatise §11.3.3).

Export. In A, the owner signs a transition naming B as its destination, and A applies it as a transfer (§4.4). The byte leaves bytes for exported, with its new owner and history hash.

Certificate. Once the frame commits, A's committing nodes sign the canonical encoding of the statement:

{"source": A, "byte": identity, "export": export record}

A certificate is that statement and the nodes' signatures:

{"statement": statement, "signatures": [{"index": uint, "signature": bytes}, ...]}

Each signature is by node index in B's peer entry for A: the algorithm identifier, then the signature. A certificate's digest is the BLAKE3-256 of its statement.

Import. In B, the import checks, in order:

1. The export's destination is B. Otherwise: wrong_cluster.

2. The source is a peer. Otherwise: unknown_cluster.

3. The signatures are a valid controller signature over the statement (envelope wire format §7). The key state is the source's node keys, with a threshold of ⌊2n/3⌋+1 for n keys, the supermajority treatise §13.1 commits at.

4. B has not imported this certificate before. Otherwise: duplicate_import.

5. B does not hold the byte. Otherwise:

   • duplicate_byte, if B holds it live;
   • byte_retired, if B burned it.

   A byte B exported, now returning, is allowed.

6. The envelope decodes, with the envelope classes. Its identity, issuer, cargo, current owner and history hash must be the export's. Otherwise: certificate_mismatch.

7. Each log replays as identity spec §4 describes, with that section's classes. Together they give the key states the envelope's signatures name.

8. The envelope verifies against those key states, end to end (envelope wire format §8), with that section's classes.

The byte is then held in B, with the export's issuer, cargo, mint frame, owner and history hash. It leaves exported if it had left B before, and the certificate's digest joins imported.

So B relies on A for one thing: the byte's state at the moment it left, which A's supermajority certifies. B verifies the byte's history itself, from its envelope and its controllers' logs. That includes the issuer's attestation. Whether the issuer was registered when it minted is A's testimony, carried by the certificate (treatise §15.5).

---

6. The state hash

After a frame, each node computes the state hash: BLAKE3-256 of the state's canonical encoding, new frame number included (treatise §9.4 step 6). The frame commits when ⌊2n/3⌋+1 of the cluster's n nodes report the same hash (treatise §13).

---

7. Genesis and persistence

Genesis. A cluster's genesis state has:

• frame 0;
• its identifier;
• its peers, with their nodes' keys;
• its registry;
• its governance delay, with sequence 0 and nothing pending;
• no controllers, no bytes, nothing retired, exported or imported.

Genesis controllers and genesis bytes arrive as frame 0's inputs: genesis issuance is minting at frame 0 (treatise §15.4). A node seeds itself with its cluster's genesis state before it applies anything (treatise §16.3).

Checkpoint. A checkpoint is the state's canonical encoding, so its digest is the state hash at its frame (treatise §16.1).

Log. The reference implementation's log holds one record per frame committed since the checkpoint. Each record is a four-byte big-endian length, then the canonical encoding of {"frame", "inputs", "state_hash"}: the frame's distinct inputs in the order they applied, and the hash the frame committed at. Recovery works as follows:

1. Load the checkpoint.
2. Skip log records the checkpoint already covers.
3. Replay the rest in order.
4. Refuse a log that skips a frame, or a replay that reaches any hash but the one recorded.

The log format is not normative. The checkpoint is, because its digest is the state hash.

---

8. Where this departs from the treatise

A byte's identity is its provenance's SAID, with no frame number in it. Treatise §15.2 step 3 folds the mint frame into the identity. A mint replayed in a later frame then gets a new identity and issues a second byte: §19.3 concedes it becomes "a distinct (and possibly duplicate) byte". Here a replayed mint has an identity the cluster already holds, and is refused in every frame. The issuer's nonce keeps distinct mints distinct. A burned or exported identity is held as well, so a burn never reopens it.

Parties are controller AIDs. The registry names issuers by AID, and every signature is checked against the key state its controller's log has put in force. An issuer that rotates its keys keeps its registration. So the issuer key rotation that treatise §15.5 defers is a key-event log's ordinary rotation.

Key events and governance are inputs. They are classes of their own, applied before mints and transfers. Treatise §9.1 lists "attestation requests" among a frame's inputs, and §15.1 asks for issuer-set changes to be inputs.

Governance covers the peers as well as the registry. Both are voted on by issuers, the cluster's governing body in §15.1.

An import certificate certifies the byte's state; the history travels as its envelope. Treatise §11.3.3 has the source cluster sign the byte's full history. Here the source signs the byte's state as it left, and the history hash in that state commits to the full history, which the destination verifies from the envelope.

Joule cost is not state. Measured energy differs from node to node (treatise §8.6), so no hash can agree on it, and the estimate function §8.6 calls for is not yet specified. A byte's joule cost travels in its envelope and in nodes' reports.

Not specified here: cluster membership. This section does not cover how nodes join and leave a cluster, or how clusters split and merge (treatise §11.2, §11.3.2). The state machine starts where membership is settled: a cluster's own nodes are outside its state, and its peers' nodes are in it, changed by governance.

---

9. Error classes

A dropped input reports one of these classes:

• Decoding: the canonical-encoding classes (identity spec §2.3) and malformed_input.
• Key events: identity spec §4's classes, plus controller_exists and unknown_controller.
• Governance:
  • wrong_cluster, stale_sequence, already_registered, not_registered and not_a_peer;
  • no_electorate, not_an_elector, duplicate_approver and below_quorum;
  • for a peer's keys, invalid_threshold, duplicate_key and malformed_verifying_key.
• Mints: said_mismatch, issuer_not_registered and duplicate_byte.
• Imports:
  • wrong_cluster, unknown_cluster, duplicate_import, duplicate_byte, byte_retired and certificate_mismatch;
  • the envelope classes (envelope wire format §8);
  • the key-event-log classes.
• Transfers and burns: unknown_byte, byte_retired, byte_exported, wrong_frame, broken_history_chain, not_current_owner and unknown_cluster.
• Signatures: unknown_controller, establishment_not_in_force, and the controller-signature classes of envelope wire format §7.

---

10. Implementation and vectors

smart-byte-rs/crates/smart-byte-state implements this section:

• state.rs: the state and its hash;
• input.rs: inputs;
• frame.rs: the frame transition;
• certificate.rs: export certificates;
• store.rs: checkpoint and log.

smart-byte-rs/crates/smart-byte-lockstep runs a cluster of these state machines under the supermajority commit, and has the committing nodes sign each frame's export certificates.

The conformance vectors are ../conformance/v1/state_vectors.json. Each one starts from a genesis state and gives every frame's inputs, the result of each input, and the state and state hash each frame reaches. The vectors on transfers between clusters are state-019 to state-021 and state-023. They run clusters A and B, of four nodes each, and in state-021 a peer D of three.