mesh §3.2: the fidelity floor was checked against the route table, not the serve #183
6 changed files with 362 additions and 5 deletions
58
mesh/SPEC.md
58
mesh/SPEC.md
|
|
@ -92,6 +92,51 @@ Fidelity is an ordered scale `F0 … F4` — `F0` the coarsest sufficient answer
|
||||||
normative: a backend is admissible only when its level ≥ the request's floor
|
normative: a backend is admissible only when its level ≥ the request's floor
|
||||||
(§4).
|
(§4).
|
||||||
|
|
||||||
|
#### §3.2.1 Advertised and delivered
|
||||||
|
|
||||||
|
A Mesh request carries two bounds, and until this revision only one of them was
|
||||||
|
held to a standard of honesty. Energy has a provenance model whose coupling MUST
|
||||||
|
be enforced before sealing (§3.3) — the rule that keeps "measured" from meaning
|
||||||
|
"estimated". Fidelity had none: a backend's level is an **advertisement**, read
|
||||||
|
out of the route table *before* the engine runs, and `fidelity_met` was derived
|
||||||
|
from it and never reconciled with what came back. A backend admitted at `F3`
|
||||||
|
that degraded at serve time still sealed `fidelity_met: 1` against an `F3`
|
||||||
|
floor.
|
||||||
|
|
||||||
|
Fidelity therefore has two figures, and they are different claims:
|
||||||
|
|
||||||
|
| figure | who says it | when |
|
||||||
|
|---|---|---|
|
||||||
|
| **advertised** — `ServingEngine::fidelity` | the route table | before the work; the basis the router admits on (§4) |
|
||||||
|
| **delivered** — `ServeOutcome.delivered_fidelity` | the backend that served | after the work |
|
||||||
|
|
||||||
|
The coupling is normative:
|
||||||
|
|
||||||
|
- Where the backend reports a delivered level, `fidelity_met` **MUST** be
|
||||||
|
derived from it, not from the advertisement.
|
||||||
|
- A backend **MUST NOT** report a delivered level **above** its advertised
|
||||||
|
rating. The router admitted it on the rating; it cannot raise that basis after
|
||||||
|
the fact. An implementation **MUST** refuse such a serve rather than clamp the
|
||||||
|
figure — clamping hides a misconfigured or dishonest engine behind a receipt
|
||||||
|
that looks ordinary.
|
||||||
|
- Where the backend reports nothing, `fidelity_met` rests on the advertisement.
|
||||||
|
This is the older and weaker basis and is retained, because an engine that
|
||||||
|
cannot report what it served is still conformant. A consumer that requires the
|
||||||
|
stronger basis **MUST** check for `fidelity_delivered` rather than reading
|
||||||
|
`fidelity_met` alone.
|
||||||
|
- The receipt **MUST** carry `fidelity_advertised` and `fidelity_delivered`
|
||||||
|
together or carry neither. Their purpose is the comparison; the advertisement
|
||||||
|
alone adds nothing a verifier does not already have from `backend`. Carrying
|
||||||
|
neither is what a receipt sealed before these fields existed means, so such a
|
||||||
|
receipt is unchanged and `mesh/1` stays byte-stable.
|
||||||
|
|
||||||
|
**What this does not claim.** A delivered figure is the account of the party that
|
||||||
|
did the work. It is better grounded than a table entry written before the work,
|
||||||
|
and it is not an independent assessment of the output — Mesh does not score
|
||||||
|
responses. What the pair buys is that a divergence between promise and delivery
|
||||||
|
is *visible in the receipt* and attributable to a named backend, where before it
|
||||||
|
was invisible.
|
||||||
|
|
||||||
### §3.3 Energy and provenance
|
### §3.3 Energy and provenance
|
||||||
|
|
||||||
Energy is a two-part integer-microjoule record with an honest provenance tag:
|
Energy is a two-part integer-microjoule record with an honest provenance tag:
|
||||||
|
|
@ -218,10 +263,17 @@ per-token energy cost, and serve a request producing tokens and a `ServeOutcome`
|
||||||
```
|
```
|
||||||
ServeOutcome {
|
ServeOutcome {
|
||||||
tokens_in, tokens_out, tokens_accepted,
|
tokens_in, tokens_out, tokens_accepted,
|
||||||
cache_hit_tokens, spec_proposed, spec_accepted
|
cache_hit_tokens, spec_proposed, spec_accepted,
|
||||||
|
delivered_fidelity: Option<Fidelity> // §3.2.1
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The advertised level is a promise made before the work; `delivered_fidelity` is
|
||||||
|
what the backend says it actually served. An engine that cannot degrade may
|
||||||
|
report its rating; an engine that can — a fallback, a truncation, a timeout that
|
||||||
|
drops to a cheaper path — reports what it served. Omitting it is conformant and
|
||||||
|
means `fidelity_met` rests on the advertisement (§3.2.1).
|
||||||
|
|
||||||
The input token payload travels **with** the serve call, alongside the request —
|
The input token payload travels **with** the serve call, alongside the request —
|
||||||
it is never part of the request's sealed identity and never enters the receipt,
|
it is never part of the request's sealed identity and never enters the receipt,
|
||||||
which records only the content-addressed `prompt_hash` (§3.1). A receipt is
|
which records only the content-addressed `prompt_hash` (§3.1). A receipt is
|
||||||
|
|
@ -261,7 +313,9 @@ and nulls; booleans are encoded as `0|1` uints and absent options dropped.
|
||||||
| `route` | the router's one-line justification |
|
| `route` | the router's one-line justification |
|
||||||
| `tokens_in/out/accepted`, `cache_hit_tokens`, `spec_proposed/accepted` | the outcome counters |
|
| `tokens_in/out/accepted`, `cache_hit_tokens`, `spec_proposed/accepted` | the outcome counters |
|
||||||
| `fidelity_floor` | the floor the request demanded (uint) |
|
| `fidelity_floor` | the floor the request demanded (uint) |
|
||||||
| `fidelity_met` | `0|1` — did the served backend meet the floor |
|
| `fidelity_advertised` | the level the backend is rated for (uint, OPTIONAL — paired with the next) |
|
||||||
|
| `fidelity_delivered` | the level the backend says it served (uint, OPTIONAL — §3.2.1) |
|
||||||
|
| `fidelity_met` | `0|1` — was the floor met, on the delivered figure where there is one |
|
||||||
| `estimated_uj` | always present |
|
| `estimated_uj` | always present |
|
||||||
| `measured_uj` | present **iff** provenance is `HwShunt` |
|
| `measured_uj` | present **iff** provenance is `HwShunt` |
|
||||||
| `uj_per_accepted_token` | **the wedge metric** for a generation — `energy.uj() / tokens_accepted`; omitted when no tokens were accepted |
|
| `uj_per_accepted_token` | **the wedge metric** for a generation — `energy.uj() / tokens_accepted`; omitted when no tokens were accepted |
|
||||||
|
|
|
||||||
|
|
@ -55,6 +55,14 @@ pub enum MeshError {
|
||||||
/// The chosen backend has no registered engine (internal invariant).
|
/// The chosen backend has no registered engine (internal invariant).
|
||||||
#[error("internal: no engine registered for backend `{0}`")]
|
#[error("internal: no engine registered for backend `{0}`")]
|
||||||
NoEngine(String),
|
NoEngine(String),
|
||||||
|
/// A backend reported delivering above the level it is rated for. The
|
||||||
|
/// router admitted it on the rating, so it cannot claim a higher basis
|
||||||
|
/// after the fact.
|
||||||
|
#[error(
|
||||||
|
"backend `{backend}` is rated F{advertised} but reported delivering \
|
||||||
|
F{delivered}: a backend cannot raise the basis it was admitted on"
|
||||||
|
)]
|
||||||
|
FidelityAboveRating { backend: String, advertised: u8, delivered: u8 },
|
||||||
/// Sealing the receipt failed.
|
/// Sealing the receipt failed.
|
||||||
#[error(transparent)]
|
#[error(transparent)]
|
||||||
Receipt(#[from] ReceiptError),
|
Receipt(#[from] ReceiptError),
|
||||||
|
|
@ -169,7 +177,6 @@ impl<'m> Mesh<'m> {
|
||||||
.chosen_fidelity
|
.chosen_fidelity
|
||||||
.expect("a chosen route carries its fidelity");
|
.expect("a chosen route carries its fidelity");
|
||||||
let recipe = route.recipe.clone().unwrap_or_default();
|
let recipe = route.recipe.clone().unwrap_or_default();
|
||||||
let fidelity_met = chosen_fidelity >= req.fidelity_floor;
|
|
||||||
|
|
||||||
// 3. Meter the served span. Copy the meter reference out so the engines
|
// 3. Meter the served span. Copy the meter reference out so the engines
|
||||||
// borrow and the log borrow stay disjoint from it.
|
// borrow and the log borrow stay disjoint from it.
|
||||||
|
|
@ -178,10 +185,33 @@ impl<'m> Mesh<'m> {
|
||||||
.engines
|
.engines
|
||||||
.get_mut(&backend.0)
|
.get_mut(&backend.0)
|
||||||
.ok_or_else(|| MeshError::NoEngine(backend.0.clone()))?;
|
.ok_or_else(|| MeshError::NoEngine(backend.0.clone()))?;
|
||||||
|
let advertised = engine.fidelity();
|
||||||
let session = Session::start(meter);
|
let session = Session::start(meter);
|
||||||
let served = engine.serve(req, prompt);
|
let served = engine.serve(req, prompt);
|
||||||
let energy = session.stop();
|
let energy = session.stop();
|
||||||
|
|
||||||
|
// 3a. Decide whether the floor was met, from what came back rather than
|
||||||
|
// from the route table. The old code computed this at step 2, before
|
||||||
|
// the engine ran, so a backend that degraded at serve time still
|
||||||
|
// sealed `fidelity_met: 1`.
|
||||||
|
//
|
||||||
|
// A backend may not report a level above the one the router admitted
|
||||||
|
// it on: it would be claiming, after the fact, a basis the route
|
||||||
|
// decision never rested on. Refused rather than clamped — clamping
|
||||||
|
// would hide a misconfigured or lying engine behind a plausible
|
||||||
|
// receipt, which is the failure this field exists to make visible.
|
||||||
|
if let Some(d) = served.outcome.delivered_fidelity {
|
||||||
|
if d > advertised {
|
||||||
|
return Err(MeshError::FidelityAboveRating {
|
||||||
|
backend: backend.0.clone(),
|
||||||
|
advertised: advertised.0,
|
||||||
|
delivered: d.0,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let basis = served.outcome.delivered_fidelity.unwrap_or(chosen_fidelity);
|
||||||
|
let fidelity_met = basis >= req.fidelity_floor;
|
||||||
|
|
||||||
// 4. Build, chain, and seal the receipt.
|
// 4. Build, chain, and seal the receipt.
|
||||||
let nonce = nonce_from(&req.id());
|
let nonce = nonce_from(&req.id());
|
||||||
let receipt = MeshReceipt {
|
let receipt = MeshReceipt {
|
||||||
|
|
@ -192,6 +222,13 @@ impl<'m> Mesh<'m> {
|
||||||
route_justification: route.justification.clone(),
|
route_justification: route.justification.clone(),
|
||||||
outcome: served.outcome,
|
outcome: served.outcome,
|
||||||
fidelity_floor: req.fidelity_floor,
|
fidelity_floor: req.fidelity_floor,
|
||||||
|
// The pair travels together: their purpose is to make a divergence
|
||||||
|
// between promise and delivery checkable, and the advertisement
|
||||||
|
// alone adds nothing a verifier does not already have from
|
||||||
|
// `backend`. Both absent is the historical shape, which seals to
|
||||||
|
// the bytes the published pack shipped with.
|
||||||
|
fidelity_advertised: served.outcome.delivered_fidelity.map(|_| advertised),
|
||||||
|
fidelity_delivered: served.outcome.delivered_fidelity,
|
||||||
fidelity_met,
|
fidelity_met,
|
||||||
energy,
|
energy,
|
||||||
sequence: 0, // stamped by the log
|
sequence: 0, // stamped by the log
|
||||||
|
|
@ -307,4 +344,177 @@ mod tests {
|
||||||
let err = mesh.serve(&request(64, Fidelity::F4, 1_000), &[], None).unwrap_err();
|
let err = mesh.serve(&request(64, Fidelity::F4, 1_000), &[], None).unwrap_err();
|
||||||
assert!(matches!(err, MeshError::NoRoute(_)));
|
assert!(matches!(err, MeshError::NoRoute(_)));
|
||||||
}
|
}
|
||||||
|
// ─── the fidelity floor, measured against what came back ──────────
|
||||||
|
|
||||||
|
/// An engine rated at `rated` that actually serves at `serves`.
|
||||||
|
///
|
||||||
|
/// This is the shape the old code could not see: a backend admitted on its
|
||||||
|
/// rating that degrades at serve time — a fallback, a truncation, a timeout
|
||||||
|
/// that drops to a cheaper path.
|
||||||
|
struct DegradingEngine {
|
||||||
|
rated: Fidelity,
|
||||||
|
serves: Option<Fidelity>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl mesh_engine::ServingEngine for DegradingEngine {
|
||||||
|
fn backend_id(&self) -> BackendId {
|
||||||
|
BackendId::new("degrading")
|
||||||
|
}
|
||||||
|
fn recipe_id(&self) -> String {
|
||||||
|
"degrading-recipe".to_string()
|
||||||
|
}
|
||||||
|
fn fidelity(&self) -> Fidelity {
|
||||||
|
self.rated
|
||||||
|
}
|
||||||
|
fn est_uj_per_token(&self) -> u64 {
|
||||||
|
50_000
|
||||||
|
}
|
||||||
|
fn serve(&mut self, req: &ServeRequest, _prompt: &[u32]) -> mesh_engine::Served {
|
||||||
|
let n = req.max_tokens;
|
||||||
|
mesh_engine::Served {
|
||||||
|
tokens: vec![7; n as usize],
|
||||||
|
outcome: mesh_schema::ServeOutcome {
|
||||||
|
tokens_in: req.prompt_tokens,
|
||||||
|
tokens_out: n,
|
||||||
|
tokens_accepted: n,
|
||||||
|
cache_hit_tokens: 0,
|
||||||
|
spec_proposed: 0,
|
||||||
|
spec_accepted: 0,
|
||||||
|
decision: None,
|
||||||
|
delivered_fidelity: self.serves,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn mesh_with_degrading<'m>(
|
||||||
|
meter: &'m dyn Meter,
|
||||||
|
rated: Fidelity,
|
||||||
|
serves: Option<Fidelity>,
|
||||||
|
) -> Mesh<'m> {
|
||||||
|
let (sk, steward) = steward_key();
|
||||||
|
let mut m = Mesh::new(meter, sk, steward);
|
||||||
|
m.register(
|
||||||
|
BackendSpec::new("degrading", "degrading-recipe", rated, 50_000),
|
||||||
|
Box::new(DegradingEngine { rated, serves }),
|
||||||
|
);
|
||||||
|
m
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_backend_that_degrades_at_serve_time_does_not_meet_the_floor() {
|
||||||
|
// The hole. `fidelity_met` was computed before the engine ran, from the
|
||||||
|
// route table, so a backend admitted at F3 that actually served F1
|
||||||
|
// sealed `fidelity_met: 1` against an F3 floor.
|
||||||
|
let meter = SteppingMeter::new("test-rapl", Provenance::HwShunt, 1_000_000);
|
||||||
|
let mut mesh = mesh_with_degrading(&meter, Fidelity::F3, Some(Fidelity::F1));
|
||||||
|
let resp = mesh
|
||||||
|
.serve(&request(8, Fidelity::F3, 10_000_000), &[], None)
|
||||||
|
.expect("serve");
|
||||||
|
|
||||||
|
assert!(
|
||||||
|
!resp.receipt.fidelity_met,
|
||||||
|
"a backend that served F1 against an F3 floor met it"
|
||||||
|
);
|
||||||
|
// And the divergence is in the receipt, so it is attributable to the
|
||||||
|
// backend rather than only visible as a failed floor.
|
||||||
|
assert_eq!(resp.receipt.fidelity_advertised, Some(Fidelity::F3));
|
||||||
|
assert_eq!(resp.receipt.fidelity_delivered, Some(Fidelity::F1));
|
||||||
|
assert_eq!(resp.receipt.fidelity_floor, Fidelity::F3);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_backend_that_delivers_its_rating_still_meets_the_floor() {
|
||||||
|
let meter = SteppingMeter::new("test-rapl", Provenance::HwShunt, 1_000_000);
|
||||||
|
let mut mesh = mesh_with_degrading(&meter, Fidelity::F3, Some(Fidelity::F3));
|
||||||
|
let resp = mesh
|
||||||
|
.serve(&request(8, Fidelity::F3, 10_000_000), &[], None)
|
||||||
|
.expect("serve");
|
||||||
|
assert!(resp.receipt.fidelity_met);
|
||||||
|
assert_eq!(resp.receipt.fidelity_delivered, Some(Fidelity::F3));
|
||||||
|
// Delivering above the floor is fine; it is the rating it may not exceed.
|
||||||
|
let mut mesh = mesh_with_degrading(&meter, Fidelity::F4, Some(Fidelity::F3));
|
||||||
|
let resp = mesh
|
||||||
|
.serve(&request(8, Fidelity::F2, 10_000_000), &[], None)
|
||||||
|
.expect("serve");
|
||||||
|
assert!(resp.receipt.fidelity_met);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_backend_may_not_claim_a_level_above_the_one_it_was_admitted_on() {
|
||||||
|
// Refused rather than clamped: clamping would hide a misconfigured or
|
||||||
|
// lying engine behind a receipt that looks ordinary.
|
||||||
|
let meter = SteppingMeter::new("test-rapl", Provenance::HwShunt, 1_000_000);
|
||||||
|
let mut mesh = mesh_with_degrading(&meter, Fidelity::F1, Some(Fidelity::F4));
|
||||||
|
let err = mesh
|
||||||
|
.serve(&request(8, Fidelity::F1, 10_000_000), &[], None)
|
||||||
|
.expect_err("a backend claiming above its rating must be refused");
|
||||||
|
assert!(
|
||||||
|
matches!(
|
||||||
|
err,
|
||||||
|
MeshError::FidelityAboveRating { advertised: 1, delivered: 4, .. }
|
||||||
|
),
|
||||||
|
"unexpected error: {err:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_backend_that_reports_nothing_keeps_the_advertised_basis() {
|
||||||
|
// Unknown is not a refusal: an engine that cannot report what it served
|
||||||
|
// is still conformant, and `fidelity_met` then rests on the
|
||||||
|
// advertisement — which the receipt says by carrying neither field.
|
||||||
|
let meter = SteppingMeter::new("test-rapl", Provenance::HwShunt, 1_000_000);
|
||||||
|
let mut mesh = mesh_with_degrading(&meter, Fidelity::F3, None);
|
||||||
|
let resp = mesh
|
||||||
|
.serve(&request(8, Fidelity::F3, 10_000_000), &[], None)
|
||||||
|
.expect("serve");
|
||||||
|
assert!(resp.receipt.fidelity_met, "the advertisement met the floor");
|
||||||
|
assert_eq!(resp.receipt.fidelity_delivered, None);
|
||||||
|
assert_eq!(
|
||||||
|
resp.receipt.fidelity_advertised, None,
|
||||||
|
"the pair travels together: the advertisement alone says nothing new"
|
||||||
|
);
|
||||||
|
// And the signed core carries neither key, so such a receipt seals to
|
||||||
|
// the bytes it sealed to before the fields existed.
|
||||||
|
let wire = resp.receipt.to_jcr1_wire(mesh.steward());
|
||||||
|
let obj = wire.as_object().expect("object");
|
||||||
|
assert!(!obj.contains_key("fidelity_delivered"));
|
||||||
|
assert!(!obj.contains_key("fidelity_advertised"));
|
||||||
|
assert_eq!(obj.get("fidelity_met").unwrap(), 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_reported_delivery_changes_the_signed_core_and_a_silent_one_does_not() {
|
||||||
|
// The additivity claim, made non-vacuous: the reference engine with the
|
||||||
|
// switch on seals different bytes from the same engine with it off, and
|
||||||
|
// the difference is exactly the two keys.
|
||||||
|
let meter = SteppingMeter::new("test-rapl", Provenance::HwShunt, 1_000_000);
|
||||||
|
let (sk, steward) = steward_key();
|
||||||
|
|
||||||
|
let mut silent = Mesh::new(&meter, sk.clone(), steward.clone());
|
||||||
|
silent.register(
|
||||||
|
BackendSpec::new("tiny", "ssm-1b", Fidelity::F1, 50_000),
|
||||||
|
Box::new(ReferenceEngine::new("tiny", "ssm-1b", Fidelity::F1, 50_000, 256)),
|
||||||
|
);
|
||||||
|
let mut reporting = Mesh::new(&meter, sk, steward);
|
||||||
|
reporting.register(
|
||||||
|
BackendSpec::new("tiny", "ssm-1b", Fidelity::F1, 50_000),
|
||||||
|
Box::new(
|
||||||
|
ReferenceEngine::new("tiny", "ssm-1b", Fidelity::F1, 50_000, 256)
|
||||||
|
.reporting_delivered_fidelity(),
|
||||||
|
),
|
||||||
|
);
|
||||||
|
|
||||||
|
let req = request(8, Fidelity::F1, 10_000_000);
|
||||||
|
let a = silent.serve(&req, &[], None).expect("serve");
|
||||||
|
let b = reporting.serve(&req, &[], None).expect("serve");
|
||||||
|
assert!(a.receipt.fidelity_met && b.receipt.fidelity_met);
|
||||||
|
assert_ne!(a.envelope, b.envelope, "the field must be signed, not advisory");
|
||||||
|
|
||||||
|
let (wa, wb) = (a.receipt.to_jcr1_wire("s"), b.receipt.to_jcr1_wire("s"));
|
||||||
|
let (oa, ob) = (wa.as_object().unwrap(), wb.as_object().unwrap());
|
||||||
|
let mut added: Vec<&String> = ob.keys().filter(|k| !oa.contains_key(*k)).collect();
|
||||||
|
added.sort();
|
||||||
|
assert_eq!(added, vec!["fidelity_advertised", "fidelity_delivered"]);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -118,6 +118,11 @@ mod engine {
|
||||||
spec_accepted: 0,
|
spec_accepted: 0,
|
||||||
// A token-generating engine answers no typed decision.
|
// A token-generating engine answers no typed decision.
|
||||||
decision: None,
|
decision: None,
|
||||||
|
// This engine decodes greedily from whatever model was loaded
|
||||||
|
// and forms no after-the-fact view of what it delivered, so it
|
||||||
|
// makes no claim: the gate falls back to the advertised
|
||||||
|
// rating, which is where these engines already stood.
|
||||||
|
delivered_fidelity: None,
|
||||||
};
|
};
|
||||||
Served { tokens, outcome }
|
Served { tokens, outcome }
|
||||||
}
|
}
|
||||||
|
|
@ -251,6 +256,11 @@ mod falcon {
|
||||||
spec_accepted: 0,
|
spec_accepted: 0,
|
||||||
// A token-generating engine answers no typed decision.
|
// A token-generating engine answers no typed decision.
|
||||||
decision: None,
|
decision: None,
|
||||||
|
// This engine decodes greedily from whatever model was loaded
|
||||||
|
// and forms no after-the-fact view of what it delivered, so it
|
||||||
|
// makes no claim: the gate falls back to the advertised
|
||||||
|
// rating, which is where these engines already stood.
|
||||||
|
delivered_fidelity: None,
|
||||||
};
|
};
|
||||||
Served { tokens, outcome }
|
Served { tokens, outcome }
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -29,7 +29,13 @@ pub trait ServingEngine {
|
||||||
fn backend_id(&self) -> BackendId;
|
fn backend_id(&self) -> BackendId;
|
||||||
/// The recipe (composed model) this engine serves.
|
/// The recipe (composed model) this engine serves.
|
||||||
fn recipe_id(&self) -> String;
|
fn recipe_id(&self) -> String;
|
||||||
/// The fidelity level this engine is rated to deliver.
|
/// The fidelity level this engine is rated to deliver — an advertisement,
|
||||||
|
/// which the router admits the backend on (§4).
|
||||||
|
///
|
||||||
|
/// This is a promise made *before* the work. What the backend actually
|
||||||
|
/// served belongs in [`ServeOutcome::delivered_fidelity`], and a conforming
|
||||||
|
/// engine MUST NOT report a delivered level above this rating: the router
|
||||||
|
/// admitted it on this basis, so it cannot upgrade the basis afterwards.
|
||||||
fn fidelity(&self) -> Fidelity;
|
fn fidelity(&self) -> Fidelity;
|
||||||
/// Modeled microjoules per token (the router's a-priori cost estimate).
|
/// Modeled microjoules per token (the router's a-priori cost estimate).
|
||||||
fn est_uj_per_token(&self) -> u64;
|
fn est_uj_per_token(&self) -> u64;
|
||||||
|
|
@ -57,6 +63,12 @@ pub struct ReferenceEngine {
|
||||||
work_per_token: u32,
|
work_per_token: u32,
|
||||||
/// Permille of proposed speculative tokens accepted (0 = no speculation).
|
/// Permille of proposed speculative tokens accepted (0 = no speculation).
|
||||||
spec_accept_permille: u32,
|
spec_accept_permille: u32,
|
||||||
|
/// Whether to report `delivered_fidelity`. Off by default: the published
|
||||||
|
/// `mesh/1` vectors are generated from this engine, and a receipt that
|
||||||
|
/// carries the field seals to different bytes, so an engine that reported
|
||||||
|
/// it unconditionally would change every published vector and break every
|
||||||
|
/// verifier that byte-compares them.
|
||||||
|
report_delivered: bool,
|
||||||
/// Seeds served before — a repeat counts as a cache hit.
|
/// Seeds served before — a repeat counts as a cache hit.
|
||||||
seen: std::collections::HashSet<u64>,
|
seen: std::collections::HashSet<u64>,
|
||||||
}
|
}
|
||||||
|
|
@ -82,11 +94,25 @@ impl ReferenceEngine {
|
||||||
uj_per_token,
|
uj_per_token,
|
||||||
work_per_token,
|
work_per_token,
|
||||||
spec_accept_permille: 0,
|
spec_accept_permille: 0,
|
||||||
|
report_delivered: false,
|
||||||
seen: std::collections::HashSet::new(),
|
seen: std::collections::HashSet::new(),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Enable modeled speculative decoding at the given accept rate (permille).
|
/// Enable modeled speculative decoding at the given accept rate (permille).
|
||||||
|
/// Report `delivered_fidelity` on every serve.
|
||||||
|
///
|
||||||
|
/// This engine serves at its rating by construction — its output is a pure
|
||||||
|
/// function of the request — so what it reports is the rating. The point of
|
||||||
|
/// the switch is that it makes the *field* exercisable: a conformance vector
|
||||||
|
/// built with it carries the delivered figure, and one built without it
|
||||||
|
/// keeps the bytes the pack shipped with. A real engine that can degrade
|
||||||
|
/// does not need a switch; it reports what it served.
|
||||||
|
pub fn reporting_delivered_fidelity(mut self) -> Self {
|
||||||
|
self.report_delivered = true;
|
||||||
|
self
|
||||||
|
}
|
||||||
|
|
||||||
pub fn with_speculation(mut self, accept_permille: u32) -> Self {
|
pub fn with_speculation(mut self, accept_permille: u32) -> Self {
|
||||||
self.spec_accept_permille = accept_permille.min(1000);
|
self.spec_accept_permille = accept_permille.min(1000);
|
||||||
self
|
self
|
||||||
|
|
@ -174,6 +200,7 @@ impl ServingEngine for ReferenceEngine {
|
||||||
spec_proposed,
|
spec_proposed,
|
||||||
spec_accepted,
|
spec_accepted,
|
||||||
decision: None,
|
decision: None,
|
||||||
|
delivered_fidelity: self.report_delivered.then_some(self.fidelity),
|
||||||
};
|
};
|
||||||
Served { tokens, outcome }
|
Served { tokens, outcome }
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -62,7 +62,32 @@ pub struct MeshReceipt {
|
||||||
pub outcome: ServeOutcome,
|
pub outcome: ServeOutcome,
|
||||||
/// The fidelity floor the request demanded.
|
/// The fidelity floor the request demanded.
|
||||||
pub fidelity_floor: Fidelity,
|
pub fidelity_floor: Fidelity,
|
||||||
/// Whether the served backend met that floor.
|
/// The level the backend is **rated** to deliver — the advertisement the
|
||||||
|
/// router admitted it on (§4).
|
||||||
|
///
|
||||||
|
/// Sealed beside `fidelity_delivered` so a backend that serves below its
|
||||||
|
/// rating is attributable from the receipt alone. The two travel as a pair —
|
||||||
|
/// their point is the comparison, and the advertisement alone adds nothing a
|
||||||
|
/// verifier does not already have from `backend`. `None` is the historical
|
||||||
|
/// shape and skipped on the wire.
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub fidelity_advertised: Option<Fidelity>,
|
||||||
|
/// The level the backend says it actually **delivered**, where it reports
|
||||||
|
/// one (`ServeOutcome::delivered_fidelity`).
|
||||||
|
///
|
||||||
|
/// `None` is the historical shape and skipped on the wire; `fidelity_met`
|
||||||
|
/// then rests on the advertisement alone. Presence is the basis tag: no
|
||||||
|
/// separate field says which figure `fidelity_met` came from, because
|
||||||
|
/// presence already does, and a receipt predating the field means exactly
|
||||||
|
/// what an absent value means — the figure is an advertisement.
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub fidelity_delivered: Option<Fidelity>,
|
||||||
|
/// Whether the floor was met.
|
||||||
|
///
|
||||||
|
/// Derived from `fidelity_delivered` when the backend reported one, and from
|
||||||
|
/// the advertisement otherwise. Read it together with
|
||||||
|
/// `fidelity_delivered`: without that field it says the route table promised
|
||||||
|
/// the floor, not that the backend delivered it.
|
||||||
pub fidelity_met: bool,
|
pub fidelity_met: bool,
|
||||||
/// The honest, provenance-tagged energy span.
|
/// The honest, provenance-tagged energy span.
|
||||||
pub energy: Energy,
|
pub energy: Energy,
|
||||||
|
|
@ -123,6 +148,14 @@ impl MeshReceipt {
|
||||||
obj.insert("spec_proposed".into(), json!(self.outcome.spec_proposed));
|
obj.insert("spec_proposed".into(), json!(self.outcome.spec_proposed));
|
||||||
obj.insert("spec_accepted".into(), json!(self.outcome.spec_accepted));
|
obj.insert("spec_accepted".into(), json!(self.outcome.spec_accepted));
|
||||||
obj.insert("fidelity_floor".into(), json!(self.fidelity_floor.0));
|
obj.insert("fidelity_floor".into(), json!(self.fidelity_floor.0));
|
||||||
|
// Emitted only when known, so a receipt that rests on the advertisement
|
||||||
|
// alone seals to the bytes it sealed to before these fields existed.
|
||||||
|
if let Some(f) = self.fidelity_advertised {
|
||||||
|
obj.insert("fidelity_advertised".into(), json!(f.0));
|
||||||
|
}
|
||||||
|
if let Some(f) = self.fidelity_delivered {
|
||||||
|
obj.insert("fidelity_delivered".into(), json!(f.0));
|
||||||
|
}
|
||||||
obj.insert("fidelity_met".into(), json!(u8::from(self.fidelity_met)));
|
obj.insert("fidelity_met".into(), json!(u8::from(self.fidelity_met)));
|
||||||
obj.insert("estimated_uj".into(), json!(self.energy.estimated_uj));
|
obj.insert("estimated_uj".into(), json!(self.energy.estimated_uj));
|
||||||
if let Some(m) = self.energy.measured_uj {
|
if let Some(m) = self.energy.measured_uj {
|
||||||
|
|
@ -276,6 +309,8 @@ mod tests {
|
||||||
route_justification: "cheapest-sufficient".into(),
|
route_justification: "cheapest-sufficient".into(),
|
||||||
outcome: outcome(accepted),
|
outcome: outcome(accepted),
|
||||||
fidelity_floor: Fidelity::F1,
|
fidelity_floor: Fidelity::F1,
|
||||||
|
fidelity_advertised: Some(Fidelity::F2),
|
||||||
|
fidelity_delivered: Some(Fidelity::F2),
|
||||||
fidelity_met: true,
|
fidelity_met: true,
|
||||||
energy,
|
energy,
|
||||||
sequence: 0,
|
sequence: 0,
|
||||||
|
|
|
||||||
|
|
@ -293,6 +293,27 @@ pub struct ServeOutcome {
|
||||||
/// the same bytes.
|
/// the same bytes.
|
||||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
pub decision: Option<DecisionOutcome>,
|
pub decision: Option<DecisionOutcome>,
|
||||||
|
/// The fidelity the backend says it actually delivered (SPEC §3.2).
|
||||||
|
///
|
||||||
|
/// Energy carries a provenance model and fidelity did not, which left the
|
||||||
|
/// two bounds of a Mesh request held to different standards. `fidelity_met`
|
||||||
|
/// was derived from [`ServingEngine::fidelity`] — the level the backend is
|
||||||
|
/// *rated* to deliver, read out of the route table **before** the engine
|
||||||
|
/// ran — and never reconciled with what came back. A backend that degraded
|
||||||
|
/// at serve time still sealed `fidelity_met: 1`.
|
||||||
|
///
|
||||||
|
/// Reported here by the party that did the work, so it is a claim made
|
||||||
|
/// after the fact rather than a table entry made before it. That is
|
||||||
|
/// strictly better grounded and no more than that: it is still the
|
||||||
|
/// backend's own account, not an independent assessment of the output. What
|
||||||
|
/// it buys is that a divergence from the rating is *visible* — see
|
||||||
|
/// `fidelity_delivered` on the receipt.
|
||||||
|
///
|
||||||
|
/// `None` is the historical shape: skipped on the wire, and `fidelity_met`
|
||||||
|
/// then falls back to the advertised level, which is the older and weaker
|
||||||
|
/// basis.
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub delivered_fidelity: Option<Fidelity>,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Per-request decision counters — the inputs to the µJ/decision metric
|
/// Per-request decision counters — the inputs to the µJ/decision metric
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue