mesh §3.2: the fidelity floor was checked against the route table, not the serve #183

Merged
dcharlot merged 2 commits from feat/mesh-fidelity-provenance into main 2026-10-10 09:56:29 -04:00
6 changed files with 362 additions and 5 deletions

View file

@ -92,6 +92,51 @@ Fidelity is an ordered scale `F0 … F4` — `F0` the coarsest sufficient answer
normative: a backend is admissible only when its level ≥ the request's floor
(§4).
#### §3.2.1 Advertised and delivered
A Mesh request carries two bounds, and until this revision only one of them was
held to a standard of honesty. Energy has a provenance model whose coupling MUST
be enforced before sealing (§3.3) — the rule that keeps "measured" from meaning
"estimated". Fidelity had none: a backend's level is an **advertisement**, read
out of the route table *before* the engine runs, and `fidelity_met` was derived
from it and never reconciled with what came back. A backend admitted at `F3`
that degraded at serve time still sealed `fidelity_met: 1` against an `F3`
floor.
Fidelity therefore has two figures, and they are different claims:
| figure | who says it | when |
|---|---|---|
| **advertised** — `ServingEngine::fidelity` | the route table | before the work; the basis the router admits on (§4) |
| **delivered** — `ServeOutcome.delivered_fidelity` | the backend that served | after the work |
The coupling is normative:
- Where the backend reports a delivered level, `fidelity_met` **MUST** be
derived from it, not from the advertisement.
- A backend **MUST NOT** report a delivered level **above** its advertised
rating. The router admitted it on the rating; it cannot raise that basis after
the fact. An implementation **MUST** refuse such a serve rather than clamp the
figure — clamping hides a misconfigured or dishonest engine behind a receipt
that looks ordinary.
- Where the backend reports nothing, `fidelity_met` rests on the advertisement.
This is the older and weaker basis and is retained, because an engine that
cannot report what it served is still conformant. A consumer that requires the
stronger basis **MUST** check for `fidelity_delivered` rather than reading
`fidelity_met` alone.
- The receipt **MUST** carry `fidelity_advertised` and `fidelity_delivered`
together or carry neither. Their purpose is the comparison; the advertisement
alone adds nothing a verifier does not already have from `backend`. Carrying
neither is what a receipt sealed before these fields existed means, so such a
receipt is unchanged and `mesh/1` stays byte-stable.
**What this does not claim.** A delivered figure is the account of the party that
did the work. It is better grounded than a table entry written before the work,
and it is not an independent assessment of the output — Mesh does not score
responses. What the pair buys is that a divergence between promise and delivery
is *visible in the receipt* and attributable to a named backend, where before it
was invisible.
### §3.3 Energy and provenance
Energy is a two-part integer-microjoule record with an honest provenance tag:
@ -218,10 +263,17 @@ per-token energy cost, and serve a request producing tokens and a `ServeOutcome`
```
ServeOutcome {
tokens_in, tokens_out, tokens_accepted,
cache_hit_tokens, spec_proposed, spec_accepted
cache_hit_tokens, spec_proposed, spec_accepted,
delivered_fidelity: Option<Fidelity> // §3.2.1
}
```
The advertised level is a promise made before the work; `delivered_fidelity` is
what the backend says it actually served. An engine that cannot degrade may
report its rating; an engine that can — a fallback, a truncation, a timeout that
drops to a cheaper path — reports what it served. Omitting it is conformant and
means `fidelity_met` rests on the advertisement (§3.2.1).
The input token payload travels **with** the serve call, alongside the request —
it is never part of the request's sealed identity and never enters the receipt,
which records only the content-addressed `prompt_hash` (§3.1). A receipt is
@ -261,7 +313,9 @@ and nulls; booleans are encoded as `0|1` uints and absent options dropped.
| `route` | the router's one-line justification |
| `tokens_in/out/accepted`, `cache_hit_tokens`, `spec_proposed/accepted` | the outcome counters |
| `fidelity_floor` | the floor the request demanded (uint) |
| `fidelity_met` | `0|1` — did the served backend meet the floor |
| `fidelity_advertised` | the level the backend is rated for (uint, OPTIONAL — paired with the next) |
| `fidelity_delivered` | the level the backend says it served (uint, OPTIONAL — §3.2.1) |
| `fidelity_met` | `0|1` — was the floor met, on the delivered figure where there is one |
| `estimated_uj` | always present |
| `measured_uj` | present **iff** provenance is `HwShunt` |
| `uj_per_accepted_token` | **the wedge metric** for a generation — `energy.uj() / tokens_accepted`; omitted when no tokens were accepted |

View file

@ -55,6 +55,14 @@ pub enum MeshError {
/// The chosen backend has no registered engine (internal invariant).
#[error("internal: no engine registered for backend `{0}`")]
NoEngine(String),
/// A backend reported delivering above the level it is rated for. The
/// router admitted it on the rating, so it cannot claim a higher basis
/// after the fact.
#[error(
"backend `{backend}` is rated F{advertised} but reported delivering \
F{delivered}: a backend cannot raise the basis it was admitted on"
)]
FidelityAboveRating { backend: String, advertised: u8, delivered: u8 },
/// Sealing the receipt failed.
#[error(transparent)]
Receipt(#[from] ReceiptError),
@ -169,7 +177,6 @@ impl<'m> Mesh<'m> {
.chosen_fidelity
.expect("a chosen route carries its fidelity");
let recipe = route.recipe.clone().unwrap_or_default();
let fidelity_met = chosen_fidelity >= req.fidelity_floor;
// 3. Meter the served span. Copy the meter reference out so the engines
// borrow and the log borrow stay disjoint from it.
@ -178,10 +185,33 @@ impl<'m> Mesh<'m> {
.engines
.get_mut(&backend.0)
.ok_or_else(|| MeshError::NoEngine(backend.0.clone()))?;
let advertised = engine.fidelity();
let session = Session::start(meter);
let served = engine.serve(req, prompt);
let energy = session.stop();
// 3a. Decide whether the floor was met, from what came back rather than
// from the route table. The old code computed this at step 2, before
// the engine ran, so a backend that degraded at serve time still
// sealed `fidelity_met: 1`.
//
// A backend may not report a level above the one the router admitted
// it on: it would be claiming, after the fact, a basis the route
// decision never rested on. Refused rather than clamped — clamping
// would hide a misconfigured or lying engine behind a plausible
// receipt, which is the failure this field exists to make visible.
if let Some(d) = served.outcome.delivered_fidelity {
if d > advertised {
return Err(MeshError::FidelityAboveRating {
backend: backend.0.clone(),
advertised: advertised.0,
delivered: d.0,
});
}
}
let basis = served.outcome.delivered_fidelity.unwrap_or(chosen_fidelity);
let fidelity_met = basis >= req.fidelity_floor;
// 4. Build, chain, and seal the receipt.
let nonce = nonce_from(&req.id());
let receipt = MeshReceipt {
@ -192,6 +222,13 @@ impl<'m> Mesh<'m> {
route_justification: route.justification.clone(),
outcome: served.outcome,
fidelity_floor: req.fidelity_floor,
// The pair travels together: their purpose is to make a divergence
// between promise and delivery checkable, and the advertisement
// alone adds nothing a verifier does not already have from
// `backend`. Both absent is the historical shape, which seals to
// the bytes the published pack shipped with.
fidelity_advertised: served.outcome.delivered_fidelity.map(|_| advertised),
fidelity_delivered: served.outcome.delivered_fidelity,
fidelity_met,
energy,
sequence: 0, // stamped by the log
@ -307,4 +344,177 @@ mod tests {
let err = mesh.serve(&request(64, Fidelity::F4, 1_000), &[], None).unwrap_err();
assert!(matches!(err, MeshError::NoRoute(_)));
}
// ─── the fidelity floor, measured against what came back ──────────
/// An engine rated at `rated` that actually serves at `serves`.
///
/// This is the shape the old code could not see: a backend admitted on its
/// rating that degrades at serve time — a fallback, a truncation, a timeout
/// that drops to a cheaper path.
struct DegradingEngine {
rated: Fidelity,
serves: Option<Fidelity>,
}
impl mesh_engine::ServingEngine for DegradingEngine {
fn backend_id(&self) -> BackendId {
BackendId::new("degrading")
}
fn recipe_id(&self) -> String {
"degrading-recipe".to_string()
}
fn fidelity(&self) -> Fidelity {
self.rated
}
fn est_uj_per_token(&self) -> u64 {
50_000
}
fn serve(&mut self, req: &ServeRequest, _prompt: &[u32]) -> mesh_engine::Served {
let n = req.max_tokens;
mesh_engine::Served {
tokens: vec![7; n as usize],
outcome: mesh_schema::ServeOutcome {
tokens_in: req.prompt_tokens,
tokens_out: n,
tokens_accepted: n,
cache_hit_tokens: 0,
spec_proposed: 0,
spec_accepted: 0,
decision: None,
delivered_fidelity: self.serves,
},
}
}
}
fn mesh_with_degrading<'m>(
meter: &'m dyn Meter,
rated: Fidelity,
serves: Option<Fidelity>,
) -> Mesh<'m> {
let (sk, steward) = steward_key();
let mut m = Mesh::new(meter, sk, steward);
m.register(
BackendSpec::new("degrading", "degrading-recipe", rated, 50_000),
Box::new(DegradingEngine { rated, serves }),
);
m
}
#[test]
fn a_backend_that_degrades_at_serve_time_does_not_meet_the_floor() {
// The hole. `fidelity_met` was computed before the engine ran, from the
// route table, so a backend admitted at F3 that actually served F1
// sealed `fidelity_met: 1` against an F3 floor.
let meter = SteppingMeter::new("test-rapl", Provenance::HwShunt, 1_000_000);
let mut mesh = mesh_with_degrading(&meter, Fidelity::F3, Some(Fidelity::F1));
let resp = mesh
.serve(&request(8, Fidelity::F3, 10_000_000), &[], None)
.expect("serve");
assert!(
!resp.receipt.fidelity_met,
"a backend that served F1 against an F3 floor met it"
);
// And the divergence is in the receipt, so it is attributable to the
// backend rather than only visible as a failed floor.
assert_eq!(resp.receipt.fidelity_advertised, Some(Fidelity::F3));
assert_eq!(resp.receipt.fidelity_delivered, Some(Fidelity::F1));
assert_eq!(resp.receipt.fidelity_floor, Fidelity::F3);
}
#[test]
fn a_backend_that_delivers_its_rating_still_meets_the_floor() {
let meter = SteppingMeter::new("test-rapl", Provenance::HwShunt, 1_000_000);
let mut mesh = mesh_with_degrading(&meter, Fidelity::F3, Some(Fidelity::F3));
let resp = mesh
.serve(&request(8, Fidelity::F3, 10_000_000), &[], None)
.expect("serve");
assert!(resp.receipt.fidelity_met);
assert_eq!(resp.receipt.fidelity_delivered, Some(Fidelity::F3));
// Delivering above the floor is fine; it is the rating it may not exceed.
let mut mesh = mesh_with_degrading(&meter, Fidelity::F4, Some(Fidelity::F3));
let resp = mesh
.serve(&request(8, Fidelity::F2, 10_000_000), &[], None)
.expect("serve");
assert!(resp.receipt.fidelity_met);
}
#[test]
fn a_backend_may_not_claim_a_level_above_the_one_it_was_admitted_on() {
// Refused rather than clamped: clamping would hide a misconfigured or
// lying engine behind a receipt that looks ordinary.
let meter = SteppingMeter::new("test-rapl", Provenance::HwShunt, 1_000_000);
let mut mesh = mesh_with_degrading(&meter, Fidelity::F1, Some(Fidelity::F4));
let err = mesh
.serve(&request(8, Fidelity::F1, 10_000_000), &[], None)
.expect_err("a backend claiming above its rating must be refused");
assert!(
matches!(
err,
MeshError::FidelityAboveRating { advertised: 1, delivered: 4, .. }
),
"unexpected error: {err:?}"
);
}
#[test]
fn a_backend_that_reports_nothing_keeps_the_advertised_basis() {
// Unknown is not a refusal: an engine that cannot report what it served
// is still conformant, and `fidelity_met` then rests on the
// advertisement — which the receipt says by carrying neither field.
let meter = SteppingMeter::new("test-rapl", Provenance::HwShunt, 1_000_000);
let mut mesh = mesh_with_degrading(&meter, Fidelity::F3, None);
let resp = mesh
.serve(&request(8, Fidelity::F3, 10_000_000), &[], None)
.expect("serve");
assert!(resp.receipt.fidelity_met, "the advertisement met the floor");
assert_eq!(resp.receipt.fidelity_delivered, None);
assert_eq!(
resp.receipt.fidelity_advertised, None,
"the pair travels together: the advertisement alone says nothing new"
);
// And the signed core carries neither key, so such a receipt seals to
// the bytes it sealed to before the fields existed.
let wire = resp.receipt.to_jcr1_wire(mesh.steward());
let obj = wire.as_object().expect("object");
assert!(!obj.contains_key("fidelity_delivered"));
assert!(!obj.contains_key("fidelity_advertised"));
assert_eq!(obj.get("fidelity_met").unwrap(), 1);
}
#[test]
fn a_reported_delivery_changes_the_signed_core_and_a_silent_one_does_not() {
// The additivity claim, made non-vacuous: the reference engine with the
// switch on seals different bytes from the same engine with it off, and
// the difference is exactly the two keys.
let meter = SteppingMeter::new("test-rapl", Provenance::HwShunt, 1_000_000);
let (sk, steward) = steward_key();
let mut silent = Mesh::new(&meter, sk.clone(), steward.clone());
silent.register(
BackendSpec::new("tiny", "ssm-1b", Fidelity::F1, 50_000),
Box::new(ReferenceEngine::new("tiny", "ssm-1b", Fidelity::F1, 50_000, 256)),
);
let mut reporting = Mesh::new(&meter, sk, steward);
reporting.register(
BackendSpec::new("tiny", "ssm-1b", Fidelity::F1, 50_000),
Box::new(
ReferenceEngine::new("tiny", "ssm-1b", Fidelity::F1, 50_000, 256)
.reporting_delivered_fidelity(),
),
);
let req = request(8, Fidelity::F1, 10_000_000);
let a = silent.serve(&req, &[], None).expect("serve");
let b = reporting.serve(&req, &[], None).expect("serve");
assert!(a.receipt.fidelity_met && b.receipt.fidelity_met);
assert_ne!(a.envelope, b.envelope, "the field must be signed, not advisory");
let (wa, wb) = (a.receipt.to_jcr1_wire("s"), b.receipt.to_jcr1_wire("s"));
let (oa, ob) = (wa.as_object().unwrap(), wb.as_object().unwrap());
let mut added: Vec<&String> = ob.keys().filter(|k| !oa.contains_key(*k)).collect();
added.sort();
assert_eq!(added, vec!["fidelity_advertised", "fidelity_delivered"]);
}
}

View file

@ -118,6 +118,11 @@ mod engine {
spec_accepted: 0,
// A token-generating engine answers no typed decision.
decision: None,
// This engine decodes greedily from whatever model was loaded
// and forms no after-the-fact view of what it delivered, so it
// makes no claim: the gate falls back to the advertised
// rating, which is where these engines already stood.
delivered_fidelity: None,
};
Served { tokens, outcome }
}
@ -251,6 +256,11 @@ mod falcon {
spec_accepted: 0,
// A token-generating engine answers no typed decision.
decision: None,
// This engine decodes greedily from whatever model was loaded
// and forms no after-the-fact view of what it delivered, so it
// makes no claim: the gate falls back to the advertised
// rating, which is where these engines already stood.
delivered_fidelity: None,
};
Served { tokens, outcome }
}

View file

@ -29,7 +29,13 @@ pub trait ServingEngine {
fn backend_id(&self) -> BackendId;
/// The recipe (composed model) this engine serves.
fn recipe_id(&self) -> String;
/// The fidelity level this engine is rated to deliver.
/// The fidelity level this engine is rated to deliver — an advertisement,
/// which the router admits the backend on (§4).
///
/// This is a promise made *before* the work. What the backend actually
/// served belongs in [`ServeOutcome::delivered_fidelity`], and a conforming
/// engine MUST NOT report a delivered level above this rating: the router
/// admitted it on this basis, so it cannot upgrade the basis afterwards.
fn fidelity(&self) -> Fidelity;
/// Modeled microjoules per token (the router's a-priori cost estimate).
fn est_uj_per_token(&self) -> u64;
@ -57,6 +63,12 @@ pub struct ReferenceEngine {
work_per_token: u32,
/// Permille of proposed speculative tokens accepted (0 = no speculation).
spec_accept_permille: u32,
/// Whether to report `delivered_fidelity`. Off by default: the published
/// `mesh/1` vectors are generated from this engine, and a receipt that
/// carries the field seals to different bytes, so an engine that reported
/// it unconditionally would change every published vector and break every
/// verifier that byte-compares them.
report_delivered: bool,
/// Seeds served before — a repeat counts as a cache hit.
seen: std::collections::HashSet<u64>,
}
@ -82,11 +94,25 @@ impl ReferenceEngine {
uj_per_token,
work_per_token,
spec_accept_permille: 0,
report_delivered: false,
seen: std::collections::HashSet::new(),
}
}
/// Enable modeled speculative decoding at the given accept rate (permille).
/// Report `delivered_fidelity` on every serve.
///
/// This engine serves at its rating by construction — its output is a pure
/// function of the request — so what it reports is the rating. The point of
/// the switch is that it makes the *field* exercisable: a conformance vector
/// built with it carries the delivered figure, and one built without it
/// keeps the bytes the pack shipped with. A real engine that can degrade
/// does not need a switch; it reports what it served.
pub fn reporting_delivered_fidelity(mut self) -> Self {
self.report_delivered = true;
self
}
pub fn with_speculation(mut self, accept_permille: u32) -> Self {
self.spec_accept_permille = accept_permille.min(1000);
self
@ -174,6 +200,7 @@ impl ServingEngine for ReferenceEngine {
spec_proposed,
spec_accepted,
decision: None,
delivered_fidelity: self.report_delivered.then_some(self.fidelity),
};
Served { tokens, outcome }
}

View file

@ -62,7 +62,32 @@ pub struct MeshReceipt {
pub outcome: ServeOutcome,
/// The fidelity floor the request demanded.
pub fidelity_floor: Fidelity,
/// Whether the served backend met that floor.
/// The level the backend is **rated** to deliver — the advertisement the
/// router admitted it on (§4).
///
/// Sealed beside `fidelity_delivered` so a backend that serves below its
/// rating is attributable from the receipt alone. The two travel as a pair —
/// their point is the comparison, and the advertisement alone adds nothing a
/// verifier does not already have from `backend`. `None` is the historical
/// shape and skipped on the wire.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub fidelity_advertised: Option<Fidelity>,
/// The level the backend says it actually **delivered**, where it reports
/// one (`ServeOutcome::delivered_fidelity`).
///
/// `None` is the historical shape and skipped on the wire; `fidelity_met`
/// then rests on the advertisement alone. Presence is the basis tag: no
/// separate field says which figure `fidelity_met` came from, because
/// presence already does, and a receipt predating the field means exactly
/// what an absent value means — the figure is an advertisement.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub fidelity_delivered: Option<Fidelity>,
/// Whether the floor was met.
///
/// Derived from `fidelity_delivered` when the backend reported one, and from
/// the advertisement otherwise. Read it together with
/// `fidelity_delivered`: without that field it says the route table promised
/// the floor, not that the backend delivered it.
pub fidelity_met: bool,
/// The honest, provenance-tagged energy span.
pub energy: Energy,
@ -123,6 +148,14 @@ impl MeshReceipt {
obj.insert("spec_proposed".into(), json!(self.outcome.spec_proposed));
obj.insert("spec_accepted".into(), json!(self.outcome.spec_accepted));
obj.insert("fidelity_floor".into(), json!(self.fidelity_floor.0));
// Emitted only when known, so a receipt that rests on the advertisement
// alone seals to the bytes it sealed to before these fields existed.
if let Some(f) = self.fidelity_advertised {
obj.insert("fidelity_advertised".into(), json!(f.0));
}
if let Some(f) = self.fidelity_delivered {
obj.insert("fidelity_delivered".into(), json!(f.0));
}
obj.insert("fidelity_met".into(), json!(u8::from(self.fidelity_met)));
obj.insert("estimated_uj".into(), json!(self.energy.estimated_uj));
if let Some(m) = self.energy.measured_uj {
@ -276,6 +309,8 @@ mod tests {
route_justification: "cheapest-sufficient".into(),
outcome: outcome(accepted),
fidelity_floor: Fidelity::F1,
fidelity_advertised: Some(Fidelity::F2),
fidelity_delivered: Some(Fidelity::F2),
fidelity_met: true,
energy,
sequence: 0,

View file

@ -293,6 +293,27 @@ pub struct ServeOutcome {
/// the same bytes.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub decision: Option<DecisionOutcome>,
/// The fidelity the backend says it actually delivered (SPEC §3.2).
///
/// Energy carries a provenance model and fidelity did not, which left the
/// two bounds of a Mesh request held to different standards. `fidelity_met`
/// was derived from [`ServingEngine::fidelity`] — the level the backend is
/// *rated* to deliver, read out of the route table **before** the engine
/// ran — and never reconciled with what came back. A backend that degraded
/// at serve time still sealed `fidelity_met: 1`.
///
/// Reported here by the party that did the work, so it is a claim made
/// after the fact rather than a table entry made before it. That is
/// strictly better grounded and no more than that: it is still the
/// backend's own account, not an independent assessment of the output. What
/// it buys is that a divergence from the rating is *visible* — see
/// `fidelity_delivered` on the receipt.
///
/// `None` is the historical shape: skipped on the wire, and `fidelity_met`
/// then falls back to the advertised level, which is the older and weaker
/// basis.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub delivered_fidelity: Option<Fidelity>,
}
/// Per-request decision counters — the inputs to the µJ/decision metric