Some checks failed
standards workspaces build + test / aac-rs (push) Has been cancelled
standards workspaces build + test / arl-rs (push) Has been cancelled
standards workspaces build + test / eoc-rs (push) Has been cancelled
standards workspaces build + test / jcp-rs (push) Has been cancelled
standards workspaces build + test / joule-code-rs (push) Has been cancelled
standards workspaces build + test / joule-ui-rs (push) Has been cancelled
standards workspaces build + test / jouleclaw-rs (push) Has been cancelled
standards workspaces build + test / joulecontract-rs (push) Has been cancelled
standards workspaces build + test / jouledesk-rs (push) Has been cancelled
standards workspaces build + test / joulehook-rs (push) Has been cancelled
standards workspaces build + test / map-rs (push) Has been cancelled
standards workspaces build + test / mesh-rs (push) Has been cancelled
standards workspaces build + test / openpay (push) Has been cancelled
standards workspaces build + test / proof-rs (push) Has been cancelled
standards workspaces build + test / sandbox-rs (push) Has been cancelled
standards workspaces build + test / sense-rs (push) Has been cancelled
standards workspaces build + test / smart-byte-rs (push) Has been cancelled
conformance/v1 was written by gen_vectors and read back by nothing. The in-crate
tests call the fixture functions, so they check the library against itself; a
third party told to reproduce these facts "from the published vectors alone" had
no evidence the reference implementation still did.
tests/pack.rs works from the files on disk. Where the pack carries an input the
input comes from the pack, and where it records an output the whole recorded
output is compared:
- every file matches its MANIFEST content address;
- C1: registry and both surfaces deserialized from the pack, validate()
reproducing the exhaustive error list exactly and in order;
- C2/C3: interactions rebuilt from the pack's surface, action, inputs, meter
step and budget, compared whole; C2's surface_id checked against C1's valid
surface so the two classes agree on one published surface;
- C4: the sealed chain deserialized and verified, excision breaking at index 1,
and tampering with content, a signature bit, or link order each caught at
the right link.
The pack is current -- unlike joulehook's, nothing here was stale.
One sharp edge the test has to cover, because verify_chain does not:
verify_chain checks each link's signature against the key that link CARRIES, and
never compares that key to a trusted one. Re-sealing the same interactions under
an unrelated key produces a chain verify_chain accepts in full. The pack
publishes steward_verifying_key_hex for exactly this reason, so the test binds
every link's signer to it and includes the forged chain as a negative. That is
arguably correct for an integrity primitive -- binding to a trusted key is the
caller's job -- so verify_chain's semantics are left alone; but a verifier that
calls it alone is not checking authenticity, and nothing tested that before.
Falsified by disabling the DisallowedChild rule. The pack test fails on exactly
the C1 case and nowhere else. Stated plainly so coverage is not overclaimed: the
existing conformance lib test (which asserts only invalid.len() >= 3 against a
recorded list of 4) stays green, but joule-ui-core's own unit tests already catch
that rule. The pack test's unique contribution is binding the published bytes and
the steward check, not that rule.
Co-Authored-By: Claude Opus 5 <[email protected]>
227 lines
9.4 KiB
Rust
227 lines
9.4 KiB
Rust
//! Consume the published `joule-ui-conformance/v1` pack.
|
|
//!
|
|
//! The in-crate tests call the fixture functions (`c1_errors()`, `c4_chain()`, …),
|
|
//! so they check the library against itself and never touch the published bytes.
|
|
//! The pack was written by `gen_vectors` and read back by nothing — a third party
|
|
//! told to reproduce these facts "from the published vectors alone" had no evidence
|
|
//! that the reference implementation still did.
|
|
//!
|
|
//! This test works from the files on disk. Where the pack carries an input, the
|
|
//! input comes from the pack; where it records an output, the *whole* recorded
|
|
//! output is compared, not a count or an `is_ok()`.
|
|
|
|
use joule_ui_conformance::{Budget, STEWARD_SEED, StepMeter};
|
|
use joule_ui_core::{Registry, ValidationError, Widget, validate};
|
|
use joule_ui_interaction::{
|
|
ContentId, Interaction, InteractionLog, SealedInteraction, run_interaction, verify_chain,
|
|
};
|
|
use serde::de::DeserializeOwned;
|
|
use serde_json::Value;
|
|
use std::path::PathBuf;
|
|
|
|
fn pack_dir() -> PathBuf {
|
|
let d = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../../conformance/v1");
|
|
d.canonicalize().expect("the conformance pack must exist next to the spec")
|
|
}
|
|
|
|
fn raw(name: &str) -> Vec<u8> {
|
|
std::fs::read(pack_dir().join(name)).unwrap_or_else(|e| panic!("read {name}: {e}"))
|
|
}
|
|
|
|
fn doc(name: &str) -> Value {
|
|
serde_json::from_slice(&raw(name)).unwrap_or_else(|e| panic!("{name} must parse: {e}"))
|
|
}
|
|
|
|
fn field<T: DeserializeOwned>(v: &Value, key: &str) -> T {
|
|
serde_json::from_value(v[key].clone()).unwrap_or_else(|e| panic!("field `{key}`: {e}"))
|
|
}
|
|
|
|
fn unhex(s: &str) -> Vec<u8> {
|
|
assert!(s.len() % 2 == 0, "odd-length hex");
|
|
(0..s.len())
|
|
.step_by(2)
|
|
.map(|i| u8::from_str_radix(&s[i..i + 2], 16).expect("hex digit"))
|
|
.collect()
|
|
}
|
|
|
|
/// Every file the manifest lists is present and matches its content address. A pack
|
|
/// edited without regenerating the manifest fails here rather than being trusted.
|
|
#[test]
|
|
fn manifest_addresses_every_file_in_the_pack() {
|
|
let m = doc("MANIFEST.json");
|
|
assert_eq!(m["pack"], "joule-ui-conformance/v1");
|
|
let classes: Vec<String> = field(&m, "classes");
|
|
assert_eq!(classes, ["C1", "C2", "C3", "C4"]);
|
|
|
|
let files = m["files"].as_object().expect("manifest lists files");
|
|
assert_eq!(files.len(), 4, "one file per class — a wrong path must not check nothing");
|
|
for (name, want) in files {
|
|
let got = ContentId::of_bytes(&raw(name)).0;
|
|
assert_eq!(&got, want.as_str().unwrap(), "{name} does not match its manifest address");
|
|
}
|
|
}
|
|
|
|
/// C1 — the registry and both surfaces come from the pack; `validate` must reproduce
|
|
/// the *exhaustive* recorded error list exactly, in order.
|
|
#[test]
|
|
fn c1_validation_reproduces_the_exhaustive_error_list() {
|
|
let c1 = doc("c1-validation.json");
|
|
let registry: Registry = field(&c1, "registry");
|
|
let valid: Widget = field(&c1, "valid_surface");
|
|
let invalid: Widget = field(&c1, "invalid_surface");
|
|
let want_valid: Vec<ValidationError> = field(&c1, "expected_valid_errors");
|
|
let want_invalid: Vec<ValidationError> = field(&c1, "expected_invalid_errors");
|
|
|
|
// Guard the pack itself, so a vacuous regeneration cannot pass as conformance.
|
|
assert!(want_valid.is_empty(), "the valid surface must record no errors");
|
|
assert!(want_invalid.len() >= 3, "the invalid surface plants three faults");
|
|
|
|
assert_eq!(validate(&valid, ®istry), want_valid);
|
|
assert_eq!(validate(&invalid, ®istry), want_invalid);
|
|
}
|
|
|
|
/// C2 — a metered, allowed interaction, rebuilt from the pack's surface, action,
|
|
/// inputs and meter step.
|
|
///
|
|
/// Two things the pack does not carry are taken from where they must come from: the
|
|
/// actor from the recorded interaction itself, and the handler from the spec — the
|
|
/// pack records only its output's content address, which cannot be inverted. Because
|
|
/// that address is compared, a different handler output still fails.
|
|
#[test]
|
|
fn c2_interaction_is_reproduced_from_the_pack() {
|
|
let c2 = doc("c2-interaction.json");
|
|
let c1 = doc("c1-validation.json");
|
|
let surface_id: ContentId = field(&c2, "surface_id");
|
|
let action: String = field(&c2, "action");
|
|
let inputs: String = field(&c2, "inputs_utf8");
|
|
let step: u64 = field(&c2, "meter_step_uj");
|
|
let want: Interaction = field(&c2, "expected_interaction");
|
|
|
|
// Cross-file: C2's surface id is the address of C1's valid surface, so the two
|
|
// classes are checked against the same published surface rather than each alone.
|
|
let valid: Widget = field(&c1, "valid_surface");
|
|
assert_eq!(
|
|
surface_id,
|
|
ContentId::of_bytes(serde_json::to_string(&valid).unwrap().as_bytes()),
|
|
"C2 surface_id must address C1's valid surface"
|
|
);
|
|
|
|
let meter = StepMeter::new(step);
|
|
let got = run_interaction(
|
|
&surface_id,
|
|
&action,
|
|
&want.actor,
|
|
inputs.as_bytes(),
|
|
&meter,
|
|
None,
|
|
|| serde_json::to_vec(&Widget::leaf("Image").prop("intensity", 7.0)).unwrap(),
|
|
);
|
|
assert_eq!(got, want);
|
|
assert_eq!(got.energy.uj(), step, "one interaction reads exactly one meter step");
|
|
}
|
|
|
|
/// C3 — a zero budget from the pack denies the action before the handler runs.
|
|
#[test]
|
|
fn c3_denial_happens_before_the_handler() {
|
|
let c3 = doc("c3-authorization.json");
|
|
let surface_id: ContentId = field(&c3, "surface_id");
|
|
let action: String = field(&c3, "action");
|
|
let budget_uj: u64 = field(&c3, "budget_uj");
|
|
let want_ran: bool = field(&c3, "expected_handler_ran");
|
|
let want: Interaction = field(&c3, "expected_interaction");
|
|
|
|
assert!(!want_ran, "the pack must record that the handler did not run");
|
|
assert_eq!(want.outputs, ContentId::genesis(), "a denial's outputs are the genesis sentinel");
|
|
|
|
let meter = StepMeter::new(4200);
|
|
let mut budget = Budget { remaining: budget_uj };
|
|
let mut ran = false;
|
|
let got = run_interaction(
|
|
&surface_id,
|
|
&action,
|
|
&want.actor,
|
|
br#"{"intensity":7}"#,
|
|
&meter,
|
|
Some(&mut budget),
|
|
|| {
|
|
ran = true;
|
|
b"never".to_vec()
|
|
},
|
|
);
|
|
assert_eq!(ran, want_ran, "the handler ran despite the denial");
|
|
assert_eq!(got, want);
|
|
}
|
|
|
|
/// C4 — the sealed chain, deserialized from the pack, verified end to end.
|
|
#[test]
|
|
fn c4_published_chain_verifies_under_the_published_steward() {
|
|
let c4 = doc("c4-chain.json");
|
|
let chain: Vec<SealedInteraction> = field(&c4, "chain");
|
|
let steward_hex: String = field(&c4, "steward_verifying_key_hex");
|
|
let want_count: usize = field(&c4, "expected_verify_ok_count");
|
|
let breaks_at: usize = field(&c4, "expected_excise_index_1_breaks_at");
|
|
|
|
assert_eq!(chain.len(), 3, "the pack records a three-link chain");
|
|
assert_eq!(verify_chain(&chain), Ok(want_count));
|
|
|
|
// The published key is the documented fixed seed (it attests nothing, by design).
|
|
let steward = unhex(&steward_hex);
|
|
let seed_vk = ed25519_dalek::SigningKey::from_bytes(&STEWARD_SEED).verifying_key().to_bytes();
|
|
assert_eq!(steward, seed_vk, "published steward key must be the documented seed's");
|
|
|
|
// `verify_chain` checks each link against the key the link *carries*. Authenticity
|
|
// needs the carried key to be the steward's — see the forged-chain test below.
|
|
for (i, link) in chain.iter().enumerate() {
|
|
assert_eq!(link.signer, steward, "link {i} is not signed by the published steward");
|
|
}
|
|
|
|
// allow, allow, deny — as the pack's note states.
|
|
let verdicts: Vec<bool> = chain.iter().map(|l| l.interaction.verdict.is_allow()).collect();
|
|
assert_eq!(verdicts, [true, true, false]);
|
|
|
|
let mut excised = chain.clone();
|
|
excised.remove(1);
|
|
assert_eq!(verify_chain(&excised), Err(breaks_at));
|
|
}
|
|
|
|
/// Tampering with the published bytes breaks the chain at the tampered link.
|
|
#[test]
|
|
fn c4_tampering_with_a_published_link_is_detected_at_that_link() {
|
|
let chain: Vec<SealedInteraction> = field(&doc("c4-chain.json"), "chain");
|
|
|
|
let mut content = chain.clone();
|
|
content[1].interaction.action = "exfiltrate".into();
|
|
assert_eq!(verify_chain(&content), Err(1), "a rewritten action must not verify");
|
|
|
|
let mut sig = chain.clone();
|
|
sig[2].signature[0] ^= 0x01;
|
|
assert_eq!(verify_chain(&sig), Err(2), "a flipped signature bit must not verify");
|
|
|
|
let mut order = chain.clone();
|
|
order.swap(0, 1);
|
|
assert_eq!(verify_chain(&order), Err(0), "reordered links must not verify");
|
|
}
|
|
|
|
/// Why the steward check above is not optional.
|
|
///
|
|
/// Re-sealing the same interactions under an unrelated key yields a chain that
|
|
/// `verify_chain` accepts in full: every signature is valid for the key its link
|
|
/// carries, `seq` counts from zero, and each `prev` links correctly. Only comparing
|
|
/// the signer against the published steward rejects it.
|
|
#[test]
|
|
fn c4_a_re_signed_chain_passes_verify_chain_but_not_the_steward_binding() {
|
|
let chain: Vec<SealedInteraction> = field(&doc("c4-chain.json"), "chain");
|
|
let steward = unhex(field::<String>(&doc("c4-chain.json"), "steward_verifying_key_hex").as_str());
|
|
|
|
let mut forger = InteractionLog::new(ed25519_dalek::SigningKey::from_bytes(&[0x42; 32]));
|
|
for link in &chain {
|
|
forger.record(link.interaction.clone());
|
|
}
|
|
let forged = forger.entries();
|
|
|
|
assert_eq!(verify_chain(forged), Ok(3), "integrity alone accepts the forgery");
|
|
assert!(
|
|
forged.iter().all(|l| l.signer != steward),
|
|
"the forgery must be distinguishable only by its signer"
|
|
);
|
|
}
|