open-standards/mesh/mesh-rs/crates/mesh-conformance/tests/pack.rs
Transaction Science 62a81b3472
All checks were successful
standards workspaces build + test / which workspaces this change touches (pull_request) Successful in 6s
repository checks / lockfiles, tests in CI, workflow hygiene, spec and section references, public text (pull_request) Successful in 44s
standards workspaces build + test / joule-ui-rs (joule-ui/joule-ui-rs, joule-ui-rs, linux) (pull_request) Successful in 1m22s
standards workspaces build + test / jcp-rs (jcp-conformance, jcp/jcp-rs, -p jcp-energy --features nvml, jcp-rs, linux) (pull_request) Successful in 1m41s
standards workspaces build + test / mesh-rs (mesh/mesh-rs, -p mesh-engine-gguf -p mesh-openai --features mesh-engine-gguf/gguf,mesh-openai/server, mesh-rs, linux-arm64) (pull_request) Successful in 1m23s
standards workspaces build + test / joulehook-rs (joulehook/joulehook-rs, joulehook-rs, linux) (pull_request) Successful in 1m11s
standards workspaces build + test / jouleclaw-rs (-nodebug, jouleclaw/jouleclaw-rs, -p jouleclaw-energy --features rapl,nvml, jouleclaw-rs, linux-arm64, -C target-feature=+fp16) (pull_request) Successful in 3m7s
standards workspaces build + test / joule-code-rs (joule-code/joule-code-rs, joule-code-rs, linux-arm64, true) (pull_request) Successful in 3m54s
standards workspaces build + test / sandbox-rs (sandbox-conformance, sandbox/sandbox-rs, sandbox-rs, linux) (pull_request) Successful in 2m16s
standards workspaces build + test / macOS energy backends (pull_request) Successful in 7m37s
standards workspaces build + test / jouledesk-rs (jouledesk/jouledesk-rs, jouledesk-rs, macos) (pull_request) Successful in 2m42s
mesh/1 + AR-1: receipts commit to the output; deterministic serves replay
A mesh receipt bound what was asked (request_id) and what it cost, and
nothing about what came back, so no one could later show which answer a
receipt was for. AR-1 had the same gap beside input_hash.

mesh/1 (SPEC §6.4):
- output_commitment: a salted BLAKE3 commitment over the request id, the
  digest of the prompt payload the engine was handed, and the output tokens.
  It is a commitment, not a hash, because a short output is reversible from
  its hash. The opening is derived per request from a key drawn from the
  steward's signing key, so a restarted steward can still disclose it.
  The field is absent for a typed decision, whose answer the engine
  interface does not carry, and omitted when absent, so older receipts keep
  their bytes.
- The payload is bound because prompt_hash addresses the prompt's bytes
  while an engine is handed token ids, so request_id alone cannot confirm
  what a replay runs on.
- ServingEngine::deterministic (default false; the reference engine true).
- mesh_core::replay re-executes a sealed serve. Its verdicts:
  - Verified: the re-run opens the commitment.
  - Diverged { first_difference }: only when a disclosed output opens the
    commitment on the same payload and opening, so the mismatch is the
    serve's and not the replayer's input.
  - NotReplayable(why): the engine samples; another backend, recipe or
    weights; another request; no commitment; or a mismatch nothing
    attributes.
- The conformance pack is regenerated (the reference receipt gains the
  field; the envelope grows from 663 to 747 bytes).

AR-1: output_hash, an optional field after input_hash, sealed only when
bound (§2.3). It recommends a commitment over a plain hash and names
mesh's output_commitment as one. The published AR-1 vectors regenerate
byte-identical.

Also: ar-verify's `len() % 2 == 0` becomes `is_multiple_of(2)`, a clippy
error that predates this change, so the crate is clippy-clean.

Tests:
- mesh: the commitment binds request, opening, payload and tokens; a
  generation commits and a decision does not; deterministic replay
  verifies and locates a drift; replay refuses what it cannot settle,
  including a wrong payload with the honest output disclosed; a restarted
  steward reopens.
- AR-1: output_hash sealed when bound and absent otherwise.
- Mutation testing: 20 mutants, all killed (13, then 7 more after
  binding the payload; one of the 13 targets code since replaced).

Verified on the build host:
- mesh-rs: 94 tests plus 19 behind features; clippy and rustdoc clean.
- jcp-rs: 565 tests and the conformance binary; jcp-action-receipt
  clippy-clean.
- Repo checks pass.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-21 04:35:32 -04:00

217 lines
8.6 KiB
Rust

//! Consume the published `mesh/1` conformance pack.
//!
//! `gen_vectors` writes `conformance/v1/` and nothing read it back: the other tests in
//! this crate re-run `reference_scenario()`, so they check the implementation against
//! itself. A pack that says a conforming implementation "MUST reproduce" its bytes
//! needs a test that holds the reference implementation to those bytes too.
//!
//! Two different questions, answered separately:
//!
//! 1. **Drift** — does the reference implementation still emit exactly the published
//! files? This is the failure a pack silently develops when the receipt shape
//! changes and nobody regenerates.
//! 2. **Cross-language check** — the README's own claim, checked the way a third
//! party would: from the pack files and RFC 9052 alone, with a hand-built CBOR
//! encoder and no mesh COSE code, reproduce `envelope.hex` byte for byte.
use ed25519_dalek::{Signature, Signer, SigningKey, Verifier, VerifyingKey};
use mesh_conformance::{MAX_TOKENS, SPAN_UJ, STEWARD_SEED, reference_scenario};
use serde_json::Value;
use std::path::PathBuf;
fn pack_dir() -> PathBuf {
let d = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../../conformance/v1");
d.canonicalize().expect("the conformance pack must exist next to the spec")
}
fn text(rel: &str) -> String {
std::fs::read_to_string(pack_dir().join(rel)).unwrap_or_else(|e| panic!("read {rel}: {e}"))
}
fn hexfile(rel: &str) -> Vec<u8> {
hex::decode(text(rel).trim()).unwrap_or_else(|e| panic!("{rel} is not hex: {e}"))
}
fn json(rel: &str) -> Value {
serde_json::from_str(&text(rel)).unwrap_or_else(|e| panic!("{rel} must parse: {e}"))
}
// --- A deliberately minimal CBOR encoder (RFC 8949), independent of mesh ---------
fn head(major: u8, len: u64) -> Vec<u8> {
let m = major << 5;
match len {
0..=23 => vec![m | len as u8],
24..=0xff => vec![m | 24, len as u8],
0x100..=0xffff => {
let mut v = vec![m | 25];
v.extend_from_slice(&(len as u16).to_be_bytes());
v
}
_ => {
let mut v = vec![m | 26];
v.extend_from_slice(&(len as u32).to_be_bytes());
v
}
}
}
fn bstr(b: &[u8]) -> Vec<u8> {
let mut v = head(2, b.len() as u64);
v.extend_from_slice(b);
v
}
fn tstr(s: &str) -> Vec<u8> {
let mut v = head(3, s.len() as u64);
v.extend_from_slice(s.as_bytes());
v
}
/// The protected header the README specifies: `{1: -8, 4: <steward bytes>}`, keys in
/// deterministic (bytewise) order.
fn protected_header(kid: &str) -> Vec<u8> {
let mut v = head(5, 2); // map(2)
v.push(0x01); // 1 (alg)
v.push(0x27); // -8 (EdDSA): major 1, value 7
v.push(0x04); // 4 (kid)
v.extend(bstr(kid.as_bytes()));
v
}
/// RFC 9052 §4.4: `Sig_structure = ["Signature1", protected, external_aad, payload]`.
fn sig_structure(protected: &[u8], payload: &[u8]) -> Vec<u8> {
let mut v = head(4, 4);
v.extend(tstr("Signature1"));
v.extend(bstr(protected));
v.extend(bstr(&[])); // empty external AAD
v.extend(bstr(payload));
v
}
/// RFC 9052 §4.2: tag 18 over `[protected, unprotected, payload, signature]`.
fn cose_sign1(protected: &[u8], payload: &[u8], sig: &[u8]) -> Vec<u8> {
let mut v = vec![0xd2]; // tag 18
v.extend(head(4, 4));
v.extend(bstr(protected));
v.extend(head(5, 0)); // empty unprotected map
v.extend(bstr(payload));
v.extend(bstr(sig));
v
}
// --- 1. Drift ---------------------------------------------------------------------
/// The reference implementation still emits exactly the published files. Compared in
/// the form `gen_vectors` writes them, so any change to the receipt — a new field, a
/// renamed one, a different route — shows up here instead of in a third party's
/// failed verification.
#[test]
fn the_reference_implementation_still_emits_the_published_pack() {
let s = reference_scenario();
let core = s.receipt.signed_core(&s.steward).expect("reference receipt signs");
assert_eq!(
format!("{}\n", hex::encode(&s.envelope)),
text("reference/envelope.hex"),
"envelope.hex is stale against the reference implementation"
);
assert_eq!(
format!("{}\n", hex::encode(&core)),
text("reference/signed_core.hex"),
"signed_core.hex is stale against the reference implementation"
);
assert_eq!(
format!("{}\n", serde_json::to_string_pretty(&s.receipt.to_jcr1_wire(&s.steward)).unwrap()),
text("reference/receipt.json"),
"receipt.json is stale against the reference implementation"
);
let meta = json("meta.json");
assert_eq!(meta["steward"], s.steward.as_str());
assert_eq!(meta["pubkey_hex"], hex::encode(s.verifying_key).as_str());
}
// --- 2. Cross-language check ------------------------------------------------------
/// The README's claim, verified from the pack and the RFC alone: the published
/// envelope is exactly `signed_core` wrapped per RFC 9052 and signed by the key
/// `meta.json` publishes.
#[test]
fn envelope_is_reproduced_from_signed_core_and_rfc_9052_alone() {
let meta = json("meta.json");
let kid = meta["steward"].as_str().expect("meta names the steward");
let pubkey: [u8; 32] = hex::decode(meta["pubkey_hex"].as_str().unwrap())
.unwrap()
.try_into()
.expect("a 32-byte Ed25519 key");
let core = hexfile("reference/signed_core.hex");
let envelope = hexfile("reference/envelope.hex");
let protected = protected_header(kid);
let to_sign = sig_structure(&protected, &core);
// Verifier direction — needs only what is published, not the seed.
let sig_bytes = &envelope[envelope.len() - 64..];
let sig = Signature::from_slice(sig_bytes).expect("64-byte signature");
VerifyingKey::from_bytes(&pubkey)
.unwrap()
.verify(&to_sign, &sig)
.expect("the published envelope must verify under the published key");
// Signer direction — Ed25519 is deterministic, so re-signing with the documented
// steward seed must rebuild the envelope exactly.
let sk = SigningKey::from_bytes(&STEWARD_SEED);
assert_eq!(sk.verifying_key().to_bytes(), pubkey, "meta.json pubkey is not the seed's");
let rebuilt = cose_sign1(&protected, &core, &sk.sign(&to_sign).to_bytes());
assert_eq!(rebuilt, envelope, "hand-built COSE_Sign1 does not match envelope.hex");
// 663 before the receipt committed to its output (SPEC §6.2), whose field
// the reference generation now carries.
assert_eq!(envelope.len(), 747);
}
/// The verifier direction is not vacuous: a single changed payload byte, or the right
/// bytes under the wrong key, must both fail.
#[test]
fn a_tampered_core_or_a_wrong_key_does_not_verify() {
let meta = json("meta.json");
let kid = meta["steward"].as_str().unwrap();
let pubkey: [u8; 32] =
hex::decode(meta["pubkey_hex"].as_str().unwrap()).unwrap().try_into().unwrap();
let envelope = hexfile("reference/envelope.hex");
let sig = Signature::from_slice(&envelope[envelope.len() - 64..]).unwrap();
let protected = protected_header(kid);
let mut core = hexfile("reference/signed_core.hex");
let mid = core.len() / 2;
core[mid] ^= 0x01;
assert!(
VerifyingKey::from_bytes(&pubkey).unwrap().verify(&sig_structure(&protected, &core), &sig).is_err(),
"a tampered signed core verified"
);
let core = hexfile("reference/signed_core.hex");
let stranger = SigningKey::from_bytes(&[0x42; 32]).verifying_key();
assert!(
stranger.verify(&sig_structure(&protected, &core), &sig).is_err(),
"the envelope verified under a key that did not sign it"
);
}
/// The outcome the README pins, read from the published receipt. The wedge metric is
/// recomputed from the receipt's own fields rather than trusted as a number.
#[test]
fn published_receipt_carries_the_pinned_outcome() {
let r = json("reference/receipt.json");
assert_eq!(r["mesh_receipt"], "mesh/1");
assert_eq!(r["backend"], "tiny", "cache is below the F1 floor; merge/frontier exceed 8 J");
assert_eq!(r["energy_provenance"], "HwShunt");
assert_eq!(r["measured_uj"], SPAN_UJ);
assert_eq!(r["tokens_accepted"], MAX_TOKENS);
assert_eq!(r["sequence"], 0, "the reference receipt is the chain's genesis");
assert!(r["fidelity_met"].as_u64() >= r["fidelity_floor"].as_u64(), "fidelity floor not met");
let per_token = r["measured_uj"].as_u64().unwrap() / r["tokens_accepted"].as_u64().unwrap();
assert_eq!(per_token, 50_000);
assert_eq!(r["uj_per_accepted_token"], per_token, "wedge metric disagrees with its inputs");
assert_eq!(r["steward"], json("meta.json")["steward"]);
}