mesh/1: a signed weights change record, and a gate that holds a backend to it #382
4 changed files with 1021 additions and 5 deletions
72
mesh/SPEC.md
72
mesh/SPEC.md
|
|
@ -570,6 +570,76 @@ These hold a serve to its record. Whether such a record carries any weight in a
|
|||
given dispute is not this specification's to say. It is evidence anyone can
|
||||
produce and check without the steward's help, opening apart.
|
||||
|
||||
### §6.5 The weights change record
|
||||
|
||||
A receipt's `weights_digest` (§3.6) names the weights that answered. Without a
|
||||
further record, nothing shows when a backend's weights changed, from what, or
|
||||
whether the change was one its steward had planned for. A backend's **change
|
||||
record** is that record. It is a hash-chained sequence of entries, each sealed
|
||||
as a JCR-1 envelope under the steward key that seals its receipts, with the
|
||||
profile tag `mesh_change: "mesh/1"`.
|
||||
|
||||
Every entry carries `backend`, `recipe`, `seq` (its position, from 0), `prev`
|
||||
(the content id of the envelope before it, absent for the first), `made_at`
|
||||
(Unix seconds) and `kind`. Optional fields are omitted when absent. There are
|
||||
two kinds:
|
||||
|
||||
| `kind` | Fields | Records |
|
||||
|---|---|---|
|
||||
| `plan` | `precisions`, `fidelity_floor`, `validation`, `foresees`, `risk_assessment`? | the changes the steward foresees for the backend, before them |
|
||||
| `weights` | `from`?, `to`, `precision`?, `fidelity`, `plan`?, `validation_result`?, `effective_from`, `reason` | one change of the backend's weights |
|
||||
|
||||
The digests are hex content addresses. For a `weights` entry:
|
||||
- `from` and `to` are the weights before and after the change; `from` is absent
|
||||
for the backend's first entry of this kind.
|
||||
- `plan` is the content id of the `plan` entry the change falls under, and is
|
||||
absent for a change no plan foresaw.
|
||||
- `effective_from` is the first sequence of the steward's receipt log (§6.3)
|
||||
served on the new weights.
|
||||
|
||||
An entry MUST be refused when:
|
||||
|
||||
- it is for another backend, or does not chain: its `seq` is not the next, or
|
||||
its `prev` is not the content id of the envelope before it;
|
||||
- its `made_at` is earlier than the entry before it;
|
||||
- it is a `weights` entry and:
|
||||
- its `from` is not the `to` of the last `weights` entry before it;
|
||||
- its `to` equals its `from`;
|
||||
- its `effective_from` is earlier than the last change's;
|
||||
- it names a `plan`, and that is not a `plan` entry earlier in the same record,
|
||||
or the change is outside it. A change is outside its plan when it has:
|
||||
- another `recipe`;
|
||||
- a `precision` the plan's non-empty `precisions` does not list, or no
|
||||
precision at all;
|
||||
- a `fidelity` below the plan's `fidelity_floor`;
|
||||
- no `validation_result`.
|
||||
|
||||
A plan therefore covers only the changes recorded after it. A change recorded
|
||||
as planned was foreseen before it was made, by the record's own order. A plan
|
||||
recorded later does not reach back over it. A change no plan covers is
|
||||
recorded too, without a `plan`. The record shows what changed, and makes no
|
||||
judgement of it.
|
||||
|
||||
**The gate.** Once a backend has a change record, the steward MUST serve on it
|
||||
only the weights its last `weights` entry has in effect. It MUST refuse a serve
|
||||
before any grant is decided or work runs where the engine names other weights,
|
||||
or none. It MUST refuse to seal a serve whose engine reports other weights, or
|
||||
none. A weights change takes effect only by being recorded, so every receipt
|
||||
from `effective_from` on was served on the weights the change put in effect.
|
||||
A steward that restarts holds a backend to the record it persisted, once every
|
||||
entry verifies under its key, the record holds, and its weights in effect are
|
||||
the ones the engine runs.
|
||||
|
||||
**Coverage.** A party holding a receipt and the record for its backend reads the
|
||||
`weights` entry in effect at the receipt's `sequence`: the last one whose
|
||||
`effective_from` is at or before it. The receipt is covered when that entry is
|
||||
for its backend and recipe, its `to` is the receipt's `weights_digest`, and,
|
||||
where both name one, its precision is the receipt's. Otherwise it is not
|
||||
covered. The entry then says which change put the weights in effect, when the
|
||||
steward made it, under which plan, and with which validation result. A position
|
||||
in the receipt log orders a change against receipts without trusting a clock;
|
||||
`made_at` is the steward's statement of the time.
|
||||
|
||||
---
|
||||
|
||||
## §7 Conformance
|
||||
|
|
@ -594,6 +664,8 @@ A conforming implementation MUST:
|
|||
5. Seal the receipt as a JCR-1 `mesh/1` envelope and compute
|
||||
`uj_per_accepted_token` (§6).
|
||||
6. Maintain a tamper-evident hash chain (§6.3).
|
||||
7. Where a backend has a change record, refuse its entries and its serves as
|
||||
§6.5 sets out, so that its weights change only by being recorded.
|
||||
|
||||
The published vectors in [`../conformance/v1/`](../conformance/v1/) pin a
|
||||
deterministic reference serve. `reference/signed_core.hex` MUST canonical-CBOR
|
||||
|
|
|
|||
|
|
@ -43,9 +43,10 @@ pub use journal::{FileJournal, Journal, JournalEntry, JournalRecord, MemoryJourn
|
|||
pub use presentation::Presentation;
|
||||
use mesh_engine::ServingEngine;
|
||||
use mesh_meter::{Meter, Session};
|
||||
use mesh_receipt::change::{ChangeBody, ChangeEntry, ChangeLog, ChangePlan, WeightsChange};
|
||||
use mesh_receipt::{output_commitment, payload_digest, MeshReceipt, ReceiptError, ReceiptLog};
|
||||
use mesh_router::{BackendSpec, Router};
|
||||
use mesh_schema::{Energy, RouteDecision, ServeOutcome, ServeRequest};
|
||||
use mesh_schema::{BackendId, Energy, RouteDecision, ServeOutcome, ServeRequest};
|
||||
|
||||
/// The JCP capability action a Mesh serve request exercises.
|
||||
pub const INFER_ACTION: &str = "infer";
|
||||
|
|
@ -112,6 +113,13 @@ pub enum MeshError {
|
|||
/// The chosen backend has no registered engine (internal invariant).
|
||||
#[error("internal: no engine registered for backend `{0}`")]
|
||||
NoEngine(String),
|
||||
/// The backend has a change record and runs weights the record does not
|
||||
/// have in effect (SPEC §6.5): its weights were changed some other way.
|
||||
#[error("backend `{backend}` runs weights {running}, and its change record has {recorded} in effect")]
|
||||
WeightsUnrecorded { backend: String, running: String, recorded: String },
|
||||
/// A change record that does not hold, or a change it refuses (SPEC §6.5).
|
||||
#[error("change record: {0}")]
|
||||
ChangeRecord(String),
|
||||
/// A backend reported delivering above the level it is rated for. The
|
||||
/// router admitted it on the rating, so it cannot claim a higher basis
|
||||
/// after the fact.
|
||||
|
|
@ -180,6 +188,11 @@ pub struct Mesh<'m> {
|
|||
/// steward that restarts can disclose the opening of any receipt it
|
||||
/// sealed, without having stored one.
|
||||
output_key: [u8; 32],
|
||||
/// Each backend's weights change record, for the backends that have one
|
||||
/// (SPEC §6.5).
|
||||
changes: HashMap<String, ChangeLog>,
|
||||
/// The steward key, which change records are sealed under as receipts are.
|
||||
change_key: SigningKey,
|
||||
}
|
||||
|
||||
impl<'m> Mesh<'m> {
|
||||
|
|
@ -190,6 +203,7 @@ impl<'m> Mesh<'m> {
|
|||
pub fn new(meter: &'m dyn Meter, steward_sk: SigningKey, steward: impl Into<String>) -> Self {
|
||||
let steward_did = Did::from_verifying_key(&steward_sk.verifying_key());
|
||||
let output_key = ContentId::of_bytes(&[b"mesh/1 output opening key\0".as_slice(), &steward_sk.to_bytes()].concat()).0;
|
||||
let change_key = steward_sk.clone();
|
||||
Self {
|
||||
router: Router::new(),
|
||||
engines: HashMap::new(),
|
||||
|
|
@ -205,6 +219,8 @@ impl<'m> Mesh<'m> {
|
|||
unresolved: Vec::new(),
|
||||
clock: system_clock,
|
||||
output_key,
|
||||
changes: HashMap::new(),
|
||||
change_key,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -362,12 +378,114 @@ impl<'m> Mesh<'m> {
|
|||
}
|
||||
|
||||
/// Register a backend: its routing spec and the engine that serves it.
|
||||
///
|
||||
/// Registering another engine for a backend that has a change record does
|
||||
/// not change the weights it may serve: until the change is recorded
|
||||
/// ([`record_weights`](Self::record_weights)), it is refused.
|
||||
pub fn register(&mut self, spec: BackendSpec, engine: Box<dyn ServingEngine>) -> &mut Self {
|
||||
self.engines.insert(spec.id.0.clone(), engine);
|
||||
self.router.register(spec);
|
||||
self
|
||||
}
|
||||
|
||||
/// Change a registered backend's weights to `engine`'s, recording the
|
||||
/// change in the backend's change record first (SPEC §6.5), and return
|
||||
/// the change's id. The first call for a backend records the weights it
|
||||
/// starts on and gives it a change record. From then on the backend serves
|
||||
/// only the weights its record has in effect.
|
||||
///
|
||||
/// The change applies from the next receipt this steward seals.
|
||||
/// `precision` is the precision the new weights run at, `plan` the plan
|
||||
/// the change falls under ([`plan_changes`](Self::plan_changes)) or `None`
|
||||
/// for a change no plan foresaw, and `validation_result` the digest of
|
||||
/// what its validation produced. A change outside its plan is refused, and
|
||||
/// the backend keeps the weights it had.
|
||||
pub fn record_weights(
|
||||
&mut self,
|
||||
engine: Box<dyn ServingEngine>,
|
||||
precision: Option<String>,
|
||||
plan: Option<ContentId>,
|
||||
validation_result: Option<ContentId>,
|
||||
reason: impl Into<String>,
|
||||
) -> Result<ContentId, MeshError> {
|
||||
let backend = engine.backend_id();
|
||||
let spec = self.router.spec(&backend).ok_or_else(|| MeshError::NoEngine(backend.0.clone()))?;
|
||||
let (recipe, fidelity) = (spec.recipe.clone(), spec.fidelity);
|
||||
let to = engine
|
||||
.weights_digest()
|
||||
.ok_or_else(|| MeshError::ChangeRecord("an engine that names no weights cannot be recorded".into()))?;
|
||||
let mut log = self.changes.get(&backend.0).cloned().unwrap_or_else(|| ChangeLog::new(backend.clone()));
|
||||
let body = ChangeBody::Weights(WeightsChange {
|
||||
from: log.current().and_then(ChangeEntry::weights).map(|w| w.to),
|
||||
to,
|
||||
precision,
|
||||
fidelity,
|
||||
plan,
|
||||
validation_result,
|
||||
effective_from: self.log.next_sequence(),
|
||||
reason: reason.into(),
|
||||
});
|
||||
let id = log
|
||||
.append(recipe, (self.clock)(), body, &self.change_key, self.log.steward())
|
||||
.map_err(|e| MeshError::ChangeRecord(e.to_string()))?;
|
||||
self.changes.insert(backend.0.clone(), log);
|
||||
self.engines.insert(backend.0, engine);
|
||||
Ok(id)
|
||||
}
|
||||
|
||||
/// Record, in a registered backend's change record, a plan for the
|
||||
/// changes its steward foresees (SPEC §6.5), and return the plan's id,
|
||||
/// which a change made under it names. A plan covers only the changes
|
||||
/// recorded after it.
|
||||
///
|
||||
/// A backend whose record holds a plan and no weights yet serves nothing
|
||||
/// until its weights are recorded.
|
||||
pub fn plan_changes(&mut self, backend: &str, plan: ChangePlan) -> Result<ContentId, MeshError> {
|
||||
let id = BackendId::new(backend);
|
||||
let recipe = self.router.spec(&id).ok_or_else(|| MeshError::NoEngine(backend.into()))?.recipe.clone();
|
||||
let mut log = self.changes.get(backend).cloned().unwrap_or_else(|| ChangeLog::new(id));
|
||||
let plan_id = log
|
||||
.append(recipe, (self.clock)(), ChangeBody::Plan(plan), &self.change_key, self.log.steward())
|
||||
.map_err(|e| MeshError::ChangeRecord(e.to_string()))?;
|
||||
self.changes.insert(backend.into(), log);
|
||||
Ok(plan_id)
|
||||
}
|
||||
|
||||
/// Hold a registered backend to the change record `sealed`, as the
|
||||
/// steward persisted it from [`change_log`](Self::change_log): how a
|
||||
/// restarted steward picks its records back up. Call it after
|
||||
/// [`with_journal`](Self::with_journal), so the receipt log has its
|
||||
/// position back.
|
||||
///
|
||||
/// Every entry must be sealed under this steward's key and the record must
|
||||
/// hold (SPEC §6.5). Its weights in effect must be the ones the backend's
|
||||
/// engine runs, and must apply from a receipt the log has reached.
|
||||
pub fn govern(&mut self, backend: &str, sealed: Vec<(ChangeEntry, Vec<u8>)>) -> Result<(), MeshError> {
|
||||
let record = |reason: &str| MeshError::ChangeRecord(reason.into());
|
||||
let log = ChangeLog::from_sealed(BackendId::new(backend), sealed, self.log.steward(), &self.log.verifying_key())
|
||||
.map_err(|e| MeshError::ChangeRecord(e.to_string()))?;
|
||||
let current = log.current().and_then(ChangeEntry::weights).ok_or_else(|| record("it records no weights"))?;
|
||||
if current.effective_from > self.log.next_sequence() {
|
||||
return Err(record("its weights apply from a receipt the log has not reached"));
|
||||
}
|
||||
let running = self.engines.get(backend).ok_or_else(|| MeshError::NoEngine(backend.into()))?.weights_digest();
|
||||
if running != Some(current.to) {
|
||||
return Err(MeshError::WeightsUnrecorded {
|
||||
backend: backend.into(),
|
||||
running: running.map_or_else(|| "none".into(), |d| d.to_hex()),
|
||||
recorded: current.to.to_hex(),
|
||||
});
|
||||
}
|
||||
self.changes.insert(backend.into(), log);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// A backend's change record (SPEC §6.5), if it has one: what the steward
|
||||
/// persists, and what it discloses beside the receipts it covers.
|
||||
pub fn change_log(&self, backend: &str) -> Option<&ChangeLog> {
|
||||
self.changes.get(backend)
|
||||
}
|
||||
|
||||
/// The steward identity receipts are sealed under.
|
||||
pub fn steward(&self) -> &str {
|
||||
self.log.steward()
|
||||
|
|
@ -474,6 +592,20 @@ impl<'m> Mesh<'m> {
|
|||
.ok_or_else(|| MeshError::NoRoute(route.justification.clone()))?;
|
||||
let model = self.engines.get(&backend.0).and_then(|e| e.weights_digest());
|
||||
|
||||
// 1a. A backend with a change record serves only the weights the
|
||||
// record has in effect (SPEC §6.5). Weights changed any other way
|
||||
// are refused here, before any grant is decided or work runs.
|
||||
if let Some(log) = self.changes.get(&backend.0) {
|
||||
let recorded = log.current().and_then(ChangeEntry::weights).map(|w| w.to);
|
||||
if recorded.is_none() || model != recorded {
|
||||
return Err(MeshError::WeightsUnrecorded {
|
||||
backend: backend.0.clone(),
|
||||
running: model.map_or_else(|| "none".into(), |d| d.to_hex()),
|
||||
recorded: recorded.map_or_else(|| "none".into(), |d| d.to_hex()),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Authorise, and record the decision, before any work runs: a
|
||||
// decision recorded after the work would lose its debit if the
|
||||
// steward stopped in between (JCP §4.11).
|
||||
|
|
@ -528,13 +660,17 @@ impl<'m> Mesh<'m> {
|
|||
};
|
||||
|
||||
// 3b. The engine must have run the weights it named before serving:
|
||||
// those are the ones the grant was decided over.
|
||||
if let (Some(named), Some(reported)) = (model, served.outcome.weights_digest) {
|
||||
if named != reported {
|
||||
// those are the ones the grant was decided over. A backend with a
|
||||
// change record must also report them, or its receipt would name
|
||||
// no weights for the record to cover.
|
||||
let reported = served.outcome.weights_digest;
|
||||
let unreported = reported.is_none() && self.changes.contains_key(&backend.0);
|
||||
if let Some(named) = model {
|
||||
if reported.is_some_and(|r| r != named) || unreported {
|
||||
return Err(MeshError::ModelMismatch {
|
||||
backend: backend.0.clone(),
|
||||
named: named.to_hex(),
|
||||
reported: reported.to_hex(),
|
||||
reported: reported.map_or_else(|| "none".into(), |d| d.to_hex()),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
|
@ -1857,4 +1993,171 @@ mod tests {
|
|||
let again = mesh_with(&meter);
|
||||
assert!(resp.receipt.opens(&again.output_opening(&req.id()), &payload_digest(&[]), &resp.tokens));
|
||||
}
|
||||
|
||||
// ── the weights, changed only on the record (SPEC §6.5) ─────────────────
|
||||
|
||||
/// The reference engine for `tiny`, naming the weights numbered `n`.
|
||||
fn weighed(n: u8) -> Box<dyn ServingEngine> {
|
||||
let digest = ContentId::of_bytes(&[b"tiny weights ".as_slice(), &[n]].concat());
|
||||
Box::new(tiny().reporting_model_identity(digest, "int8"))
|
||||
}
|
||||
|
||||
fn tiny_spec() -> BackendSpec {
|
||||
BackendSpec::new("tiny", "ssm-1b", Fidelity::F1, 50_000)
|
||||
}
|
||||
|
||||
/// Once a backend has a change record it serves only the weights the
|
||||
/// record has in effect. Weights swapped in without a record are refused
|
||||
/// before any work, and each receipt is covered by the change in effect
|
||||
/// when it was sealed.
|
||||
#[test]
|
||||
fn a_recorded_backend_serves_only_the_weights_its_record_has_in_effect() {
|
||||
let meter = SteppingMeter::new("test-rapl", Provenance::HwShunt, 1_000);
|
||||
let mut mesh = mesh_with(&meter);
|
||||
let req = request(16, Fidelity::F1, 5_000_000);
|
||||
mesh.record_weights(weighed(1), Some("int8".into()), None, None, "the weights it starts on").unwrap();
|
||||
let first = mesh.serve(&req, &[], None).unwrap();
|
||||
|
||||
mesh.register(tiny_spec(), weighed(2));
|
||||
let (head, before) = (mesh.head(), meter.snapshot().uj);
|
||||
assert!(matches!(mesh.serve(&req, &[], None), Err(MeshError::WeightsUnrecorded { .. })));
|
||||
assert_eq!(meter.snapshot().uj, before + 1_000, "refused before the meter was read: no work ran");
|
||||
assert_eq!(mesh.head(), head, "and nothing was sealed");
|
||||
|
||||
mesh.record_weights(weighed(2), Some("int8".into()), None, None, "retrained").unwrap();
|
||||
let second = mesh.serve(&req, &[], None).unwrap();
|
||||
let log = mesh.change_log("tiny").unwrap();
|
||||
assert_eq!(log.covers(&first.receipt).unwrap().seq, 0);
|
||||
assert_eq!(log.covers(&second.receipt).unwrap().seq, 1);
|
||||
assert_eq!(log.current().unwrap().weights().unwrap().from, first.receipt.weights_digest);
|
||||
|
||||
// An engine that names no weights is refused under a record, and
|
||||
// cannot be recorded.
|
||||
mesh.register(tiny_spec(), Box::new(tiny()));
|
||||
assert!(matches!(mesh.serve(&req, &[], None), Err(MeshError::WeightsUnrecorded { .. })));
|
||||
assert!(mesh.record_weights(Box::new(tiny()), None, None, None, "no weights").is_err());
|
||||
|
||||
// A record that holds only a plan has no weights in effect, so it
|
||||
// serves nothing, not even an engine that names none.
|
||||
let mut planned = mesh_with(&meter);
|
||||
planned.plan_changes("tiny", plan_for_int8()).unwrap();
|
||||
assert!(matches!(planned.serve(&req, &[], None), Err(MeshError::WeightsUnrecorded { .. })));
|
||||
}
|
||||
|
||||
/// An engine that names its weights before serving and reports none after
|
||||
/// would seal a receipt naming no weights, which no record covers.
|
||||
#[test]
|
||||
fn a_recorded_backend_must_report_its_weights() {
|
||||
struct Forgetting(Box<dyn ServingEngine>);
|
||||
impl ServingEngine for Forgetting {
|
||||
fn backend_id(&self) -> BackendId {
|
||||
self.0.backend_id()
|
||||
}
|
||||
fn recipe_id(&self) -> String {
|
||||
self.0.recipe_id()
|
||||
}
|
||||
fn fidelity(&self) -> Fidelity {
|
||||
self.0.fidelity()
|
||||
}
|
||||
fn est_uj_per_token(&self) -> u64 {
|
||||
self.0.est_uj_per_token()
|
||||
}
|
||||
fn weights_digest(&self) -> Option<ContentId> {
|
||||
self.0.weights_digest()
|
||||
}
|
||||
fn serve(&mut self, req: &ServeRequest, prompt: &[u32]) -> mesh_engine::Served {
|
||||
let mut s = self.0.serve(req, prompt);
|
||||
s.outcome.weights_digest = None;
|
||||
s
|
||||
}
|
||||
}
|
||||
let meter = SteppingMeter::new("test-rapl", Provenance::HwShunt, 1_000);
|
||||
let req = request(16, Fidelity::F1, 5_000_000);
|
||||
let mut unrecorded = mesh_with(&meter);
|
||||
unrecorded.register(tiny_spec(), Box::new(Forgetting(weighed(1))));
|
||||
unrecorded.serve(&req, &[], None).unwrap();
|
||||
|
||||
let mut recorded = mesh_with(&meter);
|
||||
recorded.record_weights(Box::new(Forgetting(weighed(1))), None, None, None, "forgets").unwrap();
|
||||
let head = recorded.head();
|
||||
assert!(matches!(recorded.serve(&req, &[], None), Err(MeshError::ModelMismatch { .. })));
|
||||
assert_eq!(recorded.head(), head, "nothing was sealed");
|
||||
}
|
||||
|
||||
fn plan_for_int8() -> ChangePlan {
|
||||
ChangePlan {
|
||||
precisions: vec!["int8".into()],
|
||||
fidelity_floor: Fidelity::F1,
|
||||
validation: "the mesh/1 conformance pack".into(),
|
||||
foresees: "requantization of the same checkpoint".into(),
|
||||
risk_assessment: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// A change made under a plan is held to it. One outside it is refused,
|
||||
/// and the backend keeps serving the weights it had.
|
||||
#[test]
|
||||
fn a_planned_weights_change_is_held_to_its_plan() {
|
||||
let meter = SteppingMeter::new("test-rapl", Provenance::HwShunt, 1_000);
|
||||
let mut mesh = mesh_with(&meter);
|
||||
let req = request(16, Fidelity::F1, 5_000_000);
|
||||
mesh.record_weights(weighed(1), Some("int8".into()), None, None, "the weights it starts on").unwrap();
|
||||
let plan = mesh.plan_changes("tiny", plan_for_int8()).unwrap();
|
||||
|
||||
let outside = mesh.record_weights(weighed(2), Some("int8".into()), Some(plan), None, "requantized");
|
||||
assert!(matches!(outside, Err(MeshError::ChangeRecord(r)) if r.contains("no validation result")));
|
||||
let kept = mesh.serve(&req, &[], None).unwrap();
|
||||
assert_eq!(mesh.change_log("tiny").unwrap().covers(&kept.receipt).unwrap().seq, 0);
|
||||
|
||||
let passed = Some(ContentId::of_bytes(b"conformance result"));
|
||||
mesh.record_weights(weighed(2), Some("int8".into()), Some(plan), passed, "requantized").unwrap();
|
||||
let resp = mesh.serve(&req, &[], None).unwrap();
|
||||
let log = mesh.change_log("tiny").unwrap();
|
||||
let change = log.covers(&resp.receipt).unwrap().weights().unwrap();
|
||||
assert_eq!(log.plan_of(change).unwrap().foresees, "requantization of the same checkpoint");
|
||||
assert_eq!(change.validation_result, passed);
|
||||
}
|
||||
|
||||
/// A restarted steward picks its change record back up from the entries
|
||||
/// it persisted, if they hold and its engine runs the weights they have in
|
||||
/// effect, and not otherwise.
|
||||
#[test]
|
||||
fn a_restarted_steward_governs_from_its_persisted_record() {
|
||||
let meter = SteppingMeter::new("test-rapl", Provenance::HwShunt, 1_000);
|
||||
let req = request(16, Fidelity::F1, 5_000_000);
|
||||
let mut mesh = mesh_with(&meter);
|
||||
mesh.record_weights(weighed(1), Some("int8".into()), None, None, "the weights it starts on").unwrap();
|
||||
let sealed = mesh.change_log("tiny").unwrap().entries().to_vec();
|
||||
|
||||
let restarted = |engine: Box<dyn ServingEngine>| {
|
||||
let mut m = mesh_with(&meter);
|
||||
m.register(tiny_spec(), engine);
|
||||
m
|
||||
};
|
||||
let mut again = restarted(weighed(1));
|
||||
again.govern("tiny", sealed.clone()).unwrap();
|
||||
again.serve(&req, &[], None).unwrap();
|
||||
again.register(tiny_spec(), weighed(2));
|
||||
assert!(matches!(again.serve(&req, &[], None), Err(MeshError::WeightsUnrecorded { .. })), "still held to it");
|
||||
|
||||
assert!(matches!(restarted(weighed(2)).govern("tiny", sealed.clone()), Err(MeshError::WeightsUnrecorded { .. })));
|
||||
// A change that applies from a receipt the restarted log has not
|
||||
// reached is not one it can hold a backend to.
|
||||
mesh.serve(&req, &[], None).unwrap();
|
||||
mesh.record_weights(weighed(2), Some("int8".into()), None, None, "retrained").unwrap();
|
||||
let ahead = mesh.change_log("tiny").unwrap().entries().to_vec();
|
||||
let refused = restarted(weighed(2)).govern("tiny", ahead);
|
||||
assert!(matches!(refused, Err(MeshError::ChangeRecord(r)) if r.contains("not reached")));
|
||||
let mut tampered = sealed.clone();
|
||||
tampered[0].0.made_at += 1;
|
||||
assert!(matches!(restarted(weighed(1)).govern("tiny", tampered), Err(MeshError::ChangeRecord(_))));
|
||||
let (other_sk, other) = (SigningKey::from_bytes(&[42u8; 32]), "another-steward");
|
||||
let mut foreign = ChangeLog::new(BackendId::new("tiny"));
|
||||
let body = sealed[0].0.body.clone();
|
||||
foreign.append("ssm-1b", 1, body, &other_sk, other).unwrap();
|
||||
assert!(matches!(
|
||||
restarted(weighed(1)).govern("tiny", foreign.entries().to_vec()),
|
||||
Err(MeshError::ChangeRecord(_))
|
||||
));
|
||||
}
|
||||
}
|
||||
|
|
|
|||
640
mesh/mesh-rs/crates/mesh-receipt/src/change.rs
Normal file
640
mesh/mesh-rs/crates/mesh-receipt/src/change.rs
Normal file
|
|
@ -0,0 +1,640 @@
|
|||
//! The weights change record (SPEC §6.5).
|
||||
//!
|
||||
//! A receipt's `weights_digest` names the weights that answered. Nothing
|
||||
//! recorded when a backend's weights changed, from what, or whether the change
|
||||
//! was one its steward had planned for. A [`ChangeLog`] is that record, one per
|
||||
//! backend: a hash-chained sequence of entries, each sealed in the JCR-1
|
||||
//! envelope receipts use, under the same steward key. An entry is one of two
|
||||
//! kinds:
|
||||
//!
|
||||
//! - A [`ChangePlan`] records, before the changes it covers, which changes the
|
||||
//! steward foresees for the backend: the precisions it may run at, the
|
||||
//! fidelity it must keep, and the validation each change must pass.
|
||||
//! - A [`WeightsChange`] records one change: the weights before and after, the
|
||||
//! plan it falls under or none, the digest of its validation result, and the
|
||||
//! position in the steward's receipt log from which it applies.
|
||||
//!
|
||||
//! A plan covers only the changes sealed after it, so a change recorded as
|
||||
//! planned was foreseen before it was made, by the chain's own order. A change
|
||||
//! no plan covers is recorded too, as unplanned: the log records what changed,
|
||||
//! and does not judge it.
|
||||
//!
|
||||
//! A change applies from a receipt-log position rather than from a time, so
|
||||
//! [`ChangeLog::covers`] can say which recorded change put a receipt's weights
|
||||
//! in effect without trusting a clock. Each entry also carries the time its
|
||||
//! steward made it.
|
||||
|
||||
use ed25519_dalek::SigningKey;
|
||||
use jcp_schema::ContentId;
|
||||
use mesh_schema::{BackendId, Fidelity};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::json;
|
||||
|
||||
use crate::{jcr1, MeshReceipt};
|
||||
|
||||
/// The changes a steward foresees for a backend, recorded before them.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct ChangePlan {
|
||||
/// The precisions a planned change may run at. Empty: the plan does not
|
||||
/// constrain precision.
|
||||
pub precisions: Vec<String>,
|
||||
/// The fidelity the backend must still be rated at after a planned change.
|
||||
pub fidelity_floor: Fidelity,
|
||||
/// The validation every planned change must pass, as the steward names it
|
||||
/// (a conformance pack, an evaluation suite). A change under the plan
|
||||
/// carries the digest of its result.
|
||||
pub validation: String,
|
||||
/// What the plan foresees, in the steward's words.
|
||||
pub foresees: String,
|
||||
/// The digest of the risk assessment the plan rests on, where there is one.
|
||||
pub risk_assessment: Option<ContentId>,
|
||||
}
|
||||
|
||||
/// One change of a backend's weights.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct WeightsChange {
|
||||
/// The weights before the change; `None` for the backend's first record.
|
||||
pub from: Option<ContentId>,
|
||||
/// The weights after it.
|
||||
pub to: ContentId,
|
||||
/// The precision the new weights run at, where the steward names it.
|
||||
pub precision: Option<String>,
|
||||
/// The backend's fidelity rating after the change.
|
||||
pub fidelity: Fidelity,
|
||||
/// The plan entry this change falls under; `None` for a change no plan
|
||||
/// foresaw.
|
||||
pub plan: Option<ContentId>,
|
||||
/// The digest of the change's validation result.
|
||||
pub validation_result: Option<ContentId>,
|
||||
/// The first sequence in the steward's receipt log served on the new
|
||||
/// weights.
|
||||
pub effective_from: u64,
|
||||
/// Why the weights changed, in the steward's words.
|
||||
pub reason: String,
|
||||
}
|
||||
|
||||
/// What an entry records.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(tag = "kind", rename_all = "snake_case")]
|
||||
pub enum ChangeBody {
|
||||
Plan(ChangePlan),
|
||||
Weights(WeightsChange),
|
||||
}
|
||||
|
||||
/// One sealed entry of a backend's change log.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct ChangeEntry {
|
||||
pub backend: BackendId,
|
||||
pub recipe: String,
|
||||
/// The entry's position in its log.
|
||||
pub seq: u64,
|
||||
/// The content id of the envelope before it; `None` for the first.
|
||||
pub prev: Option<ContentId>,
|
||||
/// Unix seconds at which the steward made the entry.
|
||||
pub made_at: u64,
|
||||
pub body: ChangeBody,
|
||||
}
|
||||
|
||||
impl ChangeEntry {
|
||||
/// The weights change this entry records, if it records one.
|
||||
pub fn weights(&self) -> Option<&WeightsChange> {
|
||||
match &self.body {
|
||||
ChangeBody::Weights(w) => Some(w),
|
||||
ChangeBody::Plan(_) => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// The plan this entry records, if it records one.
|
||||
pub fn plan(&self) -> Option<&ChangePlan> {
|
||||
match &self.body {
|
||||
ChangeBody::Plan(p) => Some(p),
|
||||
ChangeBody::Weights(_) => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// The JCR-1 wire projection. Optional fields are omitted when absent.
|
||||
pub fn to_jcr1_wire(&self, steward: &str) -> serde_json::Value {
|
||||
let mut obj = serde_json::Map::new();
|
||||
obj.insert("mesh_change".into(), json!("mesh/1"));
|
||||
obj.insert("backend".into(), json!(self.backend.0));
|
||||
obj.insert("recipe".into(), json!(self.recipe));
|
||||
obj.insert("seq".into(), json!(self.seq));
|
||||
if let Some(p) = &self.prev {
|
||||
obj.insert("prev".into(), json!(p.to_hex()));
|
||||
}
|
||||
obj.insert("made_at".into(), json!(self.made_at));
|
||||
match &self.body {
|
||||
ChangeBody::Plan(p) => {
|
||||
obj.insert("kind".into(), json!("plan"));
|
||||
obj.insert("precisions".into(), json!(p.precisions));
|
||||
obj.insert("fidelity_floor".into(), json!(p.fidelity_floor.0));
|
||||
obj.insert("validation".into(), json!(p.validation));
|
||||
obj.insert("foresees".into(), json!(p.foresees));
|
||||
if let Some(r) = &p.risk_assessment {
|
||||
obj.insert("risk_assessment".into(), json!(r.to_hex()));
|
||||
}
|
||||
}
|
||||
ChangeBody::Weights(w) => {
|
||||
obj.insert("kind".into(), json!("weights"));
|
||||
if let Some(f) = &w.from {
|
||||
obj.insert("from".into(), json!(f.to_hex()));
|
||||
}
|
||||
obj.insert("to".into(), json!(w.to.to_hex()));
|
||||
if let Some(p) = &w.precision {
|
||||
obj.insert("precision".into(), json!(p));
|
||||
}
|
||||
obj.insert("fidelity".into(), json!(w.fidelity.0));
|
||||
if let Some(p) = &w.plan {
|
||||
obj.insert("plan".into(), json!(p.to_hex()));
|
||||
}
|
||||
if let Some(v) = &w.validation_result {
|
||||
obj.insert("validation_result".into(), json!(v.to_hex()));
|
||||
}
|
||||
obj.insert("effective_from".into(), json!(w.effective_from));
|
||||
obj.insert("reason".into(), json!(w.reason));
|
||||
}
|
||||
}
|
||||
obj.insert("steward".into(), json!(steward));
|
||||
serde_json::Value::Object(obj)
|
||||
}
|
||||
|
||||
/// The canonical signed-core bytes (JCR-1 det-CBOR).
|
||||
pub fn signed_core(&self, steward: &str) -> Result<Vec<u8>, jcr1::Jcr1Error> {
|
||||
jcr1::signed_core_bytes(&self.to_jcr1_wire(steward))
|
||||
}
|
||||
}
|
||||
|
||||
/// Why a change log, or an entry offered to one, does not hold, or why it
|
||||
/// does not cover a receipt.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
|
||||
pub enum ChangeError {
|
||||
/// An entry breaks the log's rules (SPEC §6.5).
|
||||
#[error("change entry {seq}: {reason}")]
|
||||
Entry { seq: u64, reason: &'static str },
|
||||
/// An entry's envelope does not verify, or seals another entry.
|
||||
#[error("change entry {seq}: envelope: {detail}")]
|
||||
Envelope { seq: u64, detail: String },
|
||||
/// The log says nothing that puts the receipt's weights in effect.
|
||||
#[error("not covered by the change log: {0}")]
|
||||
NotCovered(&'static str),
|
||||
}
|
||||
|
||||
/// A backend's weights change record (SPEC §6.5).
|
||||
///
|
||||
/// Every entry is checked as it joins, by [`append`](Self::append) when the
|
||||
/// steward makes it and by [`from_sealed`](Self::from_sealed) when a party
|
||||
/// reads a log it was handed, so a log that exists holds.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct ChangeLog {
|
||||
backend: BackendId,
|
||||
entries: Vec<(ChangeEntry, Vec<u8>)>,
|
||||
}
|
||||
|
||||
impl ChangeLog {
|
||||
/// An empty log for `backend`.
|
||||
pub fn new(backend: BackendId) -> Self {
|
||||
Self { backend, entries: Vec::new() }
|
||||
}
|
||||
|
||||
/// Read a log from its sealed entries, checking each envelope against the
|
||||
/// steward's key and each entry against the ones before it.
|
||||
pub fn from_sealed(
|
||||
backend: BackendId,
|
||||
sealed: Vec<(ChangeEntry, Vec<u8>)>,
|
||||
steward: &str,
|
||||
verifying_key: &[u8; 32],
|
||||
) -> Result<Self, ChangeError> {
|
||||
let mut log = Self::new(backend);
|
||||
for (entry, envelope) in sealed {
|
||||
let envelope_err = |detail: String| ChangeError::Envelope { seq: entry.seq, detail };
|
||||
let payload = jcr1::verify_cose_sign1(&envelope, steward, verifying_key).map_err(|e| envelope_err(e.to_string()))?;
|
||||
let core = entry.signed_core(steward).map_err(|e| envelope_err(e.to_string()))?;
|
||||
if payload != core {
|
||||
return Err(envelope_err("it seals another entry".into()));
|
||||
}
|
||||
log.check_next(&entry)?;
|
||||
log.entries.push((entry, envelope));
|
||||
}
|
||||
Ok(log)
|
||||
}
|
||||
|
||||
/// Make, check and seal the next entry, returning its id.
|
||||
pub fn append(
|
||||
&mut self,
|
||||
recipe: impl Into<String>,
|
||||
made_at: u64,
|
||||
body: ChangeBody,
|
||||
key: &SigningKey,
|
||||
steward: &str,
|
||||
) -> Result<ContentId, ChangeError> {
|
||||
let entry = ChangeEntry {
|
||||
backend: self.backend.clone(),
|
||||
recipe: recipe.into(),
|
||||
seq: self.entries.len() as u64,
|
||||
prev: self.head(),
|
||||
made_at,
|
||||
body,
|
||||
};
|
||||
self.check_next(&entry)?;
|
||||
let core = entry
|
||||
.signed_core(steward)
|
||||
.map_err(|e| ChangeError::Envelope { seq: entry.seq, detail: e.to_string() })?;
|
||||
let envelope = jcr1::cose_sign1_envelope(&core, key, steward);
|
||||
let id = ContentId::of_bytes(&envelope);
|
||||
self.entries.push((entry, envelope));
|
||||
Ok(id)
|
||||
}
|
||||
|
||||
/// The backend this log records.
|
||||
pub fn backend(&self) -> &BackendId {
|
||||
&self.backend
|
||||
}
|
||||
|
||||
/// Every entry with its envelope, in order: what a steward persists, and
|
||||
/// what [`from_sealed`](Self::from_sealed) reads back.
|
||||
pub fn entries(&self) -> &[(ChangeEntry, Vec<u8>)] {
|
||||
&self.entries
|
||||
}
|
||||
|
||||
/// The content id of the last entry's envelope.
|
||||
pub fn head(&self) -> Option<ContentId> {
|
||||
self.entries.last().map(|(_, env)| ContentId::of_bytes(env))
|
||||
}
|
||||
|
||||
/// The last weights change: the weights the backend runs now.
|
||||
pub fn current(&self) -> Option<&ChangeEntry> {
|
||||
self.entries.iter().rev().map(|(e, _)| e).find(|e| e.weights().is_some())
|
||||
}
|
||||
|
||||
/// The weights change in effect at `sequence` of the steward's receipt
|
||||
/// log: the last one that applies from that position or before it.
|
||||
pub fn in_effect_at(&self, sequence: u64) -> Option<&ChangeEntry> {
|
||||
self.entries
|
||||
.iter()
|
||||
.rev()
|
||||
.map(|(e, _)| e)
|
||||
.find(|e| e.weights().is_some_and(|w| w.effective_from <= sequence))
|
||||
}
|
||||
|
||||
/// The plan `change` falls under, or `None` for an unplanned change.
|
||||
pub fn plan_of(&self, change: &WeightsChange) -> Option<&ChangePlan> {
|
||||
change.plan.and_then(|id| self.plan_by_id(&id))
|
||||
}
|
||||
|
||||
/// Which recorded change put `receipt`'s weights in effect.
|
||||
///
|
||||
/// The receipt must be one this steward sealed, which its envelope shows
|
||||
/// ([`MeshReceipt::verify_envelope`]): its position is in the same
|
||||
/// receipt log the changes apply from.
|
||||
pub fn covers(&self, receipt: &MeshReceipt) -> Result<&ChangeEntry, ChangeError> {
|
||||
let not = ChangeError::NotCovered;
|
||||
if receipt.backend != self.backend {
|
||||
return Err(not("the receipt is for another backend"));
|
||||
}
|
||||
let Some(weights) = receipt.weights_digest else {
|
||||
return Err(not("the receipt names no weights"));
|
||||
};
|
||||
let entry = self
|
||||
.in_effect_at(receipt.sequence)
|
||||
.ok_or(not("no recorded change applies at the receipt's position"))?;
|
||||
let change = entry.weights().expect("in_effect_at yields weights changes");
|
||||
if change.to != weights {
|
||||
return Err(not("the receipt's weights are not the ones recorded in effect"));
|
||||
}
|
||||
if entry.recipe != receipt.recipe {
|
||||
return Err(not("the receipt's recipe is not the one recorded in effect"));
|
||||
}
|
||||
if let (Some(recorded), Some(sealed)) = (&change.precision, &receipt.precision) {
|
||||
if recorded != sealed {
|
||||
return Err(not("the receipt's precision is not the one recorded in effect"));
|
||||
}
|
||||
}
|
||||
Ok(entry)
|
||||
}
|
||||
|
||||
fn plan_by_id(&self, id: &ContentId) -> Option<&ChangePlan> {
|
||||
self.entries
|
||||
.iter()
|
||||
.find(|(_, env)| ContentId::of_bytes(env) == *id)
|
||||
.and_then(|(e, _)| e.plan())
|
||||
}
|
||||
|
||||
/// The rules an entry must meet to follow the ones already in the log.
|
||||
fn check_next(&self, entry: &ChangeEntry) -> Result<(), ChangeError> {
|
||||
let bad = |reason: &'static str| Err(ChangeError::Entry { seq: entry.seq, reason });
|
||||
if entry.backend != self.backend {
|
||||
return bad("it is for another backend");
|
||||
}
|
||||
if entry.seq != self.entries.len() as u64 || entry.prev != self.head() {
|
||||
return bad("it does not chain to the entry before it");
|
||||
}
|
||||
if self.entries.last().is_some_and(|(last, _)| entry.made_at < last.made_at) {
|
||||
return bad("it was made before the entry it follows");
|
||||
}
|
||||
let ChangeBody::Weights(w) = &entry.body else {
|
||||
return Ok(());
|
||||
};
|
||||
let current = self.current();
|
||||
if w.from != current.and_then(|e| e.weights()).map(|c| c.to) {
|
||||
return bad("it does not change from the weights in effect");
|
||||
}
|
||||
if w.from == Some(w.to) {
|
||||
return bad("it changes nothing");
|
||||
}
|
||||
if current.is_some_and(|e| e.weights().is_some_and(|c| w.effective_from < c.effective_from)) {
|
||||
return bad("it applies before the change it follows");
|
||||
}
|
||||
let Some(plan_id) = &w.plan else {
|
||||
return Ok(());
|
||||
};
|
||||
let Some((plan_entry, _)) = self
|
||||
.entries
|
||||
.iter()
|
||||
.find(|(e, env)| e.plan().is_some() && ContentId::of_bytes(env) == *plan_id)
|
||||
else {
|
||||
return bad("it names no plan recorded before it");
|
||||
};
|
||||
let plan = plan_entry.plan().expect("found as a plan");
|
||||
if plan_entry.recipe != entry.recipe {
|
||||
return bad("outside its plan: another recipe");
|
||||
}
|
||||
if !plan.precisions.is_empty() && !w.precision.as_ref().is_some_and(|p| plan.precisions.contains(p)) {
|
||||
return bad("outside its plan: a precision the plan does not foresee");
|
||||
}
|
||||
if w.fidelity < plan.fidelity_floor {
|
||||
return bad("outside its plan: below the plan's fidelity floor");
|
||||
}
|
||||
if w.validation_result.is_none() {
|
||||
return bad("outside its plan: no validation result");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use mesh_schema::{Energy, ServeOutcome};
|
||||
|
||||
const STEWARD: &str = "steward-1";
|
||||
|
||||
fn key() -> SigningKey {
|
||||
SigningKey::from_bytes(&[5u8; 32])
|
||||
}
|
||||
|
||||
fn vk() -> [u8; 32] {
|
||||
key().verifying_key().to_bytes()
|
||||
}
|
||||
|
||||
fn weights(n: u8) -> ContentId {
|
||||
ContentId::of_bytes(&[b"weights ".as_slice(), &[n]].concat())
|
||||
}
|
||||
|
||||
fn plan() -> ChangeBody {
|
||||
ChangeBody::Plan(ChangePlan {
|
||||
precisions: vec!["int8".into(), "bf16".into()],
|
||||
fidelity_floor: Fidelity::F2,
|
||||
validation: "mesh/1 conformance pack + held-out suite".into(),
|
||||
foresees: "requantization of the same checkpoint".into(),
|
||||
risk_assessment: Some(ContentId::of_bytes(b"risk assessment v1")),
|
||||
})
|
||||
}
|
||||
|
||||
fn change(from: Option<u8>, to: u8, plan: Option<ContentId>, effective_from: u64) -> ChangeBody {
|
||||
ChangeBody::Weights(WeightsChange {
|
||||
from: from.map(weights),
|
||||
to: weights(to),
|
||||
precision: Some("int8".into()),
|
||||
fidelity: Fidelity::F2,
|
||||
plan,
|
||||
validation_result: plan.map(|_| ContentId::of_bytes(b"validation passed")),
|
||||
effective_from,
|
||||
reason: "requantized".into(),
|
||||
})
|
||||
}
|
||||
|
||||
/// A plan, the backend's first weights, and a change under the plan.
|
||||
fn three() -> (ChangeLog, ContentId) {
|
||||
let mut log = ChangeLog::new(BackendId::new("tiny"));
|
||||
let p = log.append("ssm-1b", 100, plan(), &key(), STEWARD).unwrap();
|
||||
log.append("ssm-1b", 110, change(None, 1, None, 0), &key(), STEWARD).unwrap();
|
||||
log.append("ssm-1b", 120, change(Some(1), 2, Some(p), 7), &key(), STEWARD).unwrap();
|
||||
(log, p)
|
||||
}
|
||||
|
||||
fn receipt(sequence: u64, weights_digest: Option<ContentId>) -> MeshReceipt {
|
||||
MeshReceipt {
|
||||
request_id: ContentId::of_bytes(b"request"),
|
||||
grant_id: None,
|
||||
backend: BackendId::new("tiny"),
|
||||
recipe: "ssm-1b".into(),
|
||||
weights_digest,
|
||||
precision: Some("int8".into()),
|
||||
route_justification: "test".into(),
|
||||
outcome: ServeOutcome { tokens_in: 1, tokens_out: 1, tokens_accepted: 1, weights_digest, ..Default::default() },
|
||||
fidelity_floor: Fidelity::F1,
|
||||
fidelity_advertised: None,
|
||||
fidelity_delivered: None,
|
||||
fidelity_met: true,
|
||||
energy: Energy { estimated_uj: 1, measured_uj: None, provenance: crate::Provenance::Estimator },
|
||||
grant_breach: None,
|
||||
output_commitment: None,
|
||||
sequence,
|
||||
prev: None,
|
||||
nonce: [0; 8],
|
||||
}
|
||||
}
|
||||
|
||||
/// A log made by its steward reads back, whole, from its sealed entries,
|
||||
/// and says which plan a change fell under.
|
||||
#[test]
|
||||
fn a_change_log_reads_back_from_its_sealed_entries() {
|
||||
let (log, p) = three();
|
||||
let read = ChangeLog::from_sealed(log.backend().clone(), log.entries().to_vec(), STEWARD, &vk()).unwrap();
|
||||
assert_eq!(read.head(), log.head());
|
||||
let current = read.current().unwrap().weights().unwrap();
|
||||
assert_eq!(current.to, weights(2));
|
||||
assert_eq!(current.plan, Some(p));
|
||||
assert_eq!(read.plan_of(current).unwrap().validation, "mesh/1 conformance pack + held-out suite");
|
||||
let first = read.in_effect_at(0).unwrap().weights().unwrap();
|
||||
assert_eq!(read.plan_of(first), None, "the first weights fell under no plan");
|
||||
}
|
||||
|
||||
/// Altered, reordered, truncated at the front or sealed by another key,
|
||||
/// a log does not read.
|
||||
#[test]
|
||||
fn a_tampered_log_does_not_read() {
|
||||
let (log, _) = three();
|
||||
let read = |sealed: Vec<(ChangeEntry, Vec<u8>)>, vk: &[u8; 32]| {
|
||||
ChangeLog::from_sealed(BackendId::new("tiny"), sealed, STEWARD, vk)
|
||||
};
|
||||
|
||||
let mut altered = log.entries().to_vec();
|
||||
if let ChangeBody::Weights(w) = &mut altered[2].0.body {
|
||||
w.reason = "something else".into();
|
||||
}
|
||||
assert!(matches!(read(altered, &vk()), Err(ChangeError::Envelope { seq: 2, .. })));
|
||||
|
||||
let mut swapped = log.entries().to_vec();
|
||||
swapped.swap(1, 2);
|
||||
assert!(matches!(read(swapped, &vk()), Err(ChangeError::Entry { seq: 2, .. })));
|
||||
|
||||
let dropped = log.entries()[1..].to_vec();
|
||||
assert!(matches!(read(dropped, &vk()), Err(ChangeError::Entry { seq: 1, .. })));
|
||||
|
||||
let other = SigningKey::from_bytes(&[6u8; 32]).verifying_key().to_bytes();
|
||||
assert!(matches!(read(log.entries().to_vec(), &other), Err(ChangeError::Envelope { seq: 0, .. })));
|
||||
}
|
||||
|
||||
/// Each chaining rule holds by itself. An entry sealed by the steward that
|
||||
/// breaks only one of them (its backend, its position, or its link to the
|
||||
/// entry before it) does not join the log.
|
||||
#[test]
|
||||
fn an_entry_joins_only_where_it_chains() {
|
||||
let (log, _) = three();
|
||||
let sealed = |e: ChangeEntry| {
|
||||
let env = jcr1::cose_sign1_envelope(&e.signed_core(STEWARD).unwrap(), &key(), STEWARD);
|
||||
let mut all = log.entries().to_vec();
|
||||
all.push((e, env));
|
||||
ChangeLog::from_sealed(BackendId::new("tiny"), all, STEWARD, &vk())
|
||||
};
|
||||
let next = ChangeEntry {
|
||||
backend: BackendId::new("tiny"),
|
||||
recipe: "ssm-1b".into(),
|
||||
seq: 3,
|
||||
prev: log.head(),
|
||||
made_at: 130,
|
||||
body: change(Some(2), 3, None, 9),
|
||||
};
|
||||
let reason = |r: Result<ChangeLog, ChangeError>| match r {
|
||||
Err(ChangeError::Entry { reason, .. }) => reason,
|
||||
other => panic!("joined: {other:?}"),
|
||||
};
|
||||
assert_eq!(reason(sealed(ChangeEntry { backend: BackendId::new("frontier"), ..next.clone() })), "it is for another backend");
|
||||
assert_eq!(reason(sealed(ChangeEntry { seq: 4, ..next.clone() })), "it does not chain to the entry before it");
|
||||
let elsewhere = Some(ContentId::of_bytes(b"another envelope"));
|
||||
assert_eq!(reason(sealed(ChangeEntry { prev: elsewhere, ..next.clone() })), "it does not chain to the entry before it");
|
||||
sealed(next).unwrap();
|
||||
}
|
||||
|
||||
/// A change starts from the weights in effect, changes something, and
|
||||
/// applies no earlier than the change before it; entries keep time order.
|
||||
#[test]
|
||||
fn a_change_follows_the_weights_in_effect() {
|
||||
let (mut log, _) = three();
|
||||
let reason = |r: Result<ContentId, ChangeError>| match r {
|
||||
Err(ChangeError::Entry { reason, .. }) => reason,
|
||||
other => panic!("accepted: {other:?}"),
|
||||
};
|
||||
assert_eq!(
|
||||
reason(log.append("ssm-1b", 130, change(Some(1), 3, None, 9), &key(), STEWARD)),
|
||||
"it does not change from the weights in effect"
|
||||
);
|
||||
assert_eq!(reason(log.append("ssm-1b", 130, change(Some(2), 2, None, 9), &key(), STEWARD)), "it changes nothing");
|
||||
assert_eq!(
|
||||
reason(log.append("ssm-1b", 130, change(Some(2), 3, None, 6), &key(), STEWARD)),
|
||||
"it applies before the change it follows"
|
||||
);
|
||||
assert_eq!(
|
||||
reason(log.append("ssm-1b", 119, change(Some(2), 3, None, 9), &key(), STEWARD)),
|
||||
"it was made before the entry it follows"
|
||||
);
|
||||
// Unplanned is recorded, as unplanned.
|
||||
log.append("ssm-1b", 130, change(Some(2), 3, None, 9), &key(), STEWARD).unwrap();
|
||||
assert_eq!(log.plan_of(log.current().unwrap().weights().unwrap()), None);
|
||||
}
|
||||
|
||||
/// A change recorded under a plan is one the plan foresaw: recorded
|
||||
/// before it, for its recipe, at a precision it lists, at or above its
|
||||
/// fidelity floor, with a validation result.
|
||||
#[test]
|
||||
fn a_planned_change_stays_inside_its_plan() {
|
||||
let (mut log, p) = three();
|
||||
let reason = |r: Result<ContentId, ChangeError>| match r {
|
||||
Err(ChangeError::Entry { reason, .. }) => reason,
|
||||
other => panic!("accepted: {other:?}"),
|
||||
};
|
||||
let with = |f: &dyn Fn(&mut WeightsChange)| {
|
||||
let mut body = change(Some(2), 3, Some(p), 9);
|
||||
if let ChangeBody::Weights(w) = &mut body {
|
||||
f(w);
|
||||
}
|
||||
body
|
||||
};
|
||||
assert_eq!(
|
||||
reason(log.append("ssm-1b", 130, with(&|w| w.precision = Some("fp16".into())), &key(), STEWARD)),
|
||||
"outside its plan: a precision the plan does not foresee"
|
||||
);
|
||||
assert_eq!(
|
||||
reason(log.append("ssm-1b", 130, with(&|w| w.precision = None), &key(), STEWARD)),
|
||||
"outside its plan: a precision the plan does not foresee"
|
||||
);
|
||||
assert_eq!(
|
||||
reason(log.append("ssm-1b", 130, with(&|w| w.fidelity = Fidelity::F1), &key(), STEWARD)),
|
||||
"outside its plan: below the plan's fidelity floor"
|
||||
);
|
||||
assert_eq!(
|
||||
reason(log.append("ssm-1b", 130, with(&|w| w.validation_result = None), &key(), STEWARD)),
|
||||
"outside its plan: no validation result"
|
||||
);
|
||||
assert_eq!(reason(log.append("ssm-2b", 130, with(&|_| ()), &key(), STEWARD)), "outside its plan: another recipe");
|
||||
// A weights entry's id is not a plan, and neither is an unknown id.
|
||||
let not_a_plan = log.head().unwrap();
|
||||
assert_eq!(
|
||||
reason(log.append("ssm-1b", 130, with(&|w| w.plan = Some(not_a_plan)), &key(), STEWARD)),
|
||||
"it names no plan recorded before it"
|
||||
);
|
||||
log.append("ssm-1b", 130, with(&|_| ()), &key(), STEWARD).unwrap();
|
||||
}
|
||||
|
||||
/// A plan covers only the changes sealed after it. One sealed later does
|
||||
/// not reach back over an earlier change, and a change can name only a
|
||||
/// plan already in its own log.
|
||||
#[test]
|
||||
fn a_plan_covers_only_what_comes_after_it() {
|
||||
let mut log = ChangeLog::new(BackendId::new("tiny"));
|
||||
log.append("ssm-1b", 100, change(None, 1, None, 0), &key(), STEWARD).unwrap();
|
||||
let p = log.append("ssm-1b", 101, plan(), &key(), STEWARD).unwrap();
|
||||
let first = log.in_effect_at(0).unwrap().weights().unwrap();
|
||||
assert_eq!(log.plan_of(first), None, "a plan sealed later does not cover it");
|
||||
|
||||
// The same plan sealed in another log is another entry, with another id.
|
||||
let mut other = ChangeLog::new(BackendId::new("tiny"));
|
||||
let elsewhere = other.append("ssm-1b", 100, plan(), &key(), STEWARD).unwrap();
|
||||
assert_ne!(elsewhere, p);
|
||||
let r = log.append("ssm-1b", 102, change(Some(1), 2, Some(elsewhere), 3), &key(), STEWARD);
|
||||
assert!(matches!(r, Err(ChangeError::Entry { reason: "it names no plan recorded before it", .. })));
|
||||
log.append("ssm-1b", 102, change(Some(1), 2, Some(p), 3), &key(), STEWARD).unwrap();
|
||||
}
|
||||
|
||||
/// A receipt is covered by the change in effect at its position, for its
|
||||
/// backend, recipe, weights and precision, and by nothing else.
|
||||
#[test]
|
||||
fn a_receipt_is_covered_by_the_change_in_effect_at_its_position() {
|
||||
let (log, _) = three();
|
||||
assert_eq!(log.covers(&receipt(0, Some(weights(1)))).unwrap().seq, 1);
|
||||
assert_eq!(log.covers(&receipt(6, Some(weights(1)))).unwrap().seq, 1);
|
||||
assert_eq!(log.covers(&receipt(7, Some(weights(2)))).unwrap().seq, 2);
|
||||
|
||||
let not = |r: Result<&ChangeEntry, ChangeError>| match r {
|
||||
Err(ChangeError::NotCovered(why)) => why,
|
||||
other => panic!("covered: {other:?}"),
|
||||
};
|
||||
assert!(not(log.covers(&receipt(7, Some(weights(1))))).contains("weights"));
|
||||
assert!(not(log.covers(&receipt(6, Some(weights(2))))).contains("weights"));
|
||||
assert!(not(log.covers(&receipt(7, None))).contains("names no weights"));
|
||||
let mut other_backend = receipt(7, Some(weights(2)));
|
||||
other_backend.backend = BackendId::new("frontier");
|
||||
assert!(not(log.covers(&other_backend)).contains("another backend"));
|
||||
let mut other_recipe = receipt(7, Some(weights(2)));
|
||||
other_recipe.recipe = "ssm-2b".into();
|
||||
assert!(not(log.covers(&other_recipe)).contains("recipe"));
|
||||
let mut other_precision = receipt(7, Some(weights(2)));
|
||||
other_precision.precision = Some("bf16".into());
|
||||
assert!(not(log.covers(&other_precision)).contains("precision"));
|
||||
|
||||
let mut late = ChangeLog::new(BackendId::new("tiny"));
|
||||
late.append("ssm-1b", 100, change(None, 1, None, 5), &key(), STEWARD).unwrap();
|
||||
assert!(not(late.covers(&receipt(4, Some(weights(1))))).contains("no recorded change"));
|
||||
}
|
||||
}
|
||||
|
|
@ -14,6 +14,7 @@ use mesh_schema::{BackendId, Energy, Fidelity, SchemaError, ServeOutcome};
|
|||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::json;
|
||||
|
||||
pub mod change;
|
||||
pub mod jcr1;
|
||||
|
||||
pub use mesh_schema::Provenance;
|
||||
|
|
|
|||
Loading…
Reference in a new issue