open-standards/jouleclaw/jouleclaw-rs/crates/jouleclaw-cascade/tests/r37_closed_calibration_loop.rs
Transaction Science 8f9a42e4d4 jouleclaw Phase 2: cascade + loader-gguf + pack + history + wire
Five crates ported / designed. All 235 tests pass; 0 errors;
0 warnings outside the loader-gguf NEON-intrinsics module (suppressed
with a documented #![allow(unsafe_op_in_unsafe_fn)] — proper fix is
wrapping each intrinsic call in unsafe { }, tracked as kernel-quality
maintenance, not standards work).

jouleclaw-cascade
  Ported from pattern-lang/joule-cascade (~7044 LOC, 17 files).
  L0 (Cache), tier dispatch, 7-axis Synthesis coordinate router, cost
  model, calibration ledger (in-memory + disk), history, verification,
  body + body_safety policy gates, active-entity tracking, l2_service
  trait surface. Two tests that depended on joule-l1's lawful-primitive
  impl (which requires pattern-lang's pattern-core + cortex-ir +
  synthesizer — OpenIE IP) were removed; the L1 trait surface lives
  here and consumers plug their own lawful lexicon in.

jouleclaw-loader-gguf
  Ported from pattern-lang/joule-loader-gguf (~11605 LOC, 17 files).
  GGUF v3 parser with mmap, Llama and Gemma / Gemma 4 decoders,
  Q4_0 / Q5_K / Q8_0 / Q8_0 grouped quants, in-place KV cache,
  matmul_q8_0 with NEON intrinsics on Apple Silicon, safetensors
  reader, tokenizer + Gemma tokenizer, sample, dequant.

jouleclaw-pack (NEW)
  The .jc.toml sidecar — JouleClaw's standard-defining contribution.
  A model on disk (GGUF / safetensors / MLX) tells you tensor shapes
  and quant schemes. It does NOT tell you what one forward pass
  actually costs in joules on real hardware. The .jc.toml is a
  declared-cost contract that travels with the model:

    - per-tensor precision (FP16/BF16/FP8/INT8/NVFP4/NF4/Q4_K/...)
    - page-alignment status (mmap-zero-copy possible? on which UMA?)
    - measured J/op on a reference hardware matrix with explicit
      Provenance (HwShunt / ModelBased / Estimator)
    - drift_factor tolerance per measurement; the runtime trips
      PackError::CostDrift when measured > declared * drift_factor
      and down-weights the backend in subsequent auction rounds

  Without a declared-cost contract, "energy-optimized inference" is
  marketing. With one, it is an engineering claim any third party
  can verify against the published reference-hardware corpus.

  5 lib tests pass. ModelFormat, Precision, Alignment, MeasurementOp,
  Pack, TensorContract, Measurement, hash_model(), verify_measurement().

jouleclaw-history
  Ported from pattern-lang/joule-history (~2026 LOC, 6 files).
  Persistent resolution history: disk + in-memory + tiered
  (hot/warm/cold) + semantic. One Rust-2024 edition fix in
  tiered.rs (binding-mode pattern: |(_, &seq)| seq -> |(_, seq)| *seq).

jouleclaw-wire
  Ported from pattern-lang/joule-wire (~1129 LOC, 2 files).
  L4 wire tier — the remote frontier escape hatch. RPC client +
  transport, invoked only when the local cascade returns Unresolvable.

Phase 3 next: HAL from efficient-genai, backend-{apple,reference,
cuda,metal,rocm,vulkan}, runtime (port joule-runtime — chat/embed/
generate/streaming/PLD/drafter — needs backends to compile).

Co-Authored-By: Claude Opus 4.7 <[email protected]>
2026-05-26 11:55:46 -04:00

135 lines
4.5 KiB
Rust
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

//! R37 — the calibration loop is now *closed*.
//!
//! Before R37 the Runtime recorded estimate-vs-actual per coord cell
//! but never used it: budget/routing decisions ran on the tier's raw
//! `estimate_cost`. R37 scales the estimate by the learned μ-correction
//! before the budget check, so a tier that consistently under-reports
//! gets its future estimates corrected from observed reality.
//!
//! This test proves the behavioural difference: a tier that estimates
//! 5× too low is, after enough samples, correctly skipped when the
//! budget only covers the *honest* (corrected) cost — something the
//! open loop could never do.
use jouleclaw_cascade::coord::*;
use jouleclaw_cascade::*;
use std::time::Duration;
/// A tier that always claims 1 nJ but actually spends 5 nJ — a
/// consistent 5× under-estimator. Distinct coord cell so per-cell
/// calibration applies.
struct UnderEstimator;
impl Tier for UnderEstimator {
fn id(&self) -> TierId {
TierId::L1(L1Primitive::Execute)
}
fn estimate_cost(&self, q: &Query) -> Option<TierEstimate> {
match &q.input {
QueryInput::Text(_) => Some(TierEstimate {
joules: 1e-9, // claims 1 nJ
latency: Duration::from_nanos(1),
confidence_floor: 1.0,
}),
_ => None,
}
}
fn try_answer(&mut self, _q: &Query, _b: f64) -> Result<Answer, AnswerError> {
let actual = 5e-9; // truth: 5 nJ — 5× the estimate
let mut trace = ExecutionTrace::default();
trace.attempts.push(TraceEntry {
tier: self.id(),
outcome: TraceOutcome::Hit,
joules: actual,
});
Ok(Answer {
output: AnswerOutput::Text("ok".into()),
tier_used: self.id(),
joules_spent: actual,
confidence: 1.0,
trace,
verification: jouleclaw_cascade::verification::VerificationStatus::Resolved,
})
}
fn coord(&self) -> Option<Coord> {
Some(Coord::new(
Zone::Z1,
Entity::Reactive,
Thermo::L1_Measure,
Interface::Tokens,
Verify::Full,
Encoding::Facts,
))
}
}
fn ask(rt: &mut Runtime, budget: JouleBudget) -> Result<Answer, AnswerError> {
rt.answer(Query {
input: QueryInput::Text("x".into()),
budget,
quality: QualityFloor::any(),
context: ContextRef::fresh(),
deadline: None,
})
}
#[test]
fn learned_mu_converges_to_the_true_ratio() {
let mut cascade = Cascade::new();
cascade.register(Box::new(UnderEstimator));
let mut rt = Runtime::new_without_l0(cascade);
let coord = UnderEstimator.coord().unwrap();
// Cold start: no samples → μ = 1.0 (loop must not perturb startup).
assert_eq!(rt.calibration().learned_mu(&coord), 1.0);
// Warm it up with a generous budget so it always dispatches.
for _ in 0..6 {
let _ = ask(&mut rt, JouleBudget::expensive());
}
let mu = rt.calibration().learned_mu(&coord);
// actual/estimate = 5e-9 / 1e-9 = 5.0.
assert!(
(mu - 5.0).abs() < 0.5,
"learned μ should converge to ~5, got {}",
mu
);
}
#[test]
fn closed_loop_skips_a_tier_the_open_loop_would_have_run() {
let mut cascade = Cascade::new();
cascade.register(Box::new(UnderEstimator));
let mut rt = Runtime::new_without_l0(cascade);
// Train calibration (6 dispatches under a generous budget).
for _ in 0..6 {
let _ = ask(&mut rt, JouleBudget::expensive());
}
assert!(rt.calibration().learned_mu(&UnderEstimator.coord().unwrap()) > 4.0);
// Budget = 2 nJ. Raw estimate (1 nJ) fits → the OPEN loop would
// dispatch and overspend. Corrected estimate (1 nJ × ~5 = ~5 nJ)
// exceeds 2 nJ → the CLOSED loop must skip and surface no-tier.
let r = ask(
&mut rt,
JouleBudget { hard_limit: 2e-9, soft_target: 2e-9 },
);
match r {
Err(AnswerError::BudgetExhausted { .. })
| Err(AnswerError::NoTierSatisfied { .. }) => { /* correct: skipped */ }
Ok(a) => panic!(
"closed loop must skip the under-estimator at a 2 nJ budget; \
instead it ran and spent {:.3e} J",
a.joules_spent
),
Err(e) => panic!("unexpected error: {:?}", e),
}
// Sanity: with a budget above the *corrected* cost it still runs.
let ok = ask(&mut rt, JouleBudget::expensive());
assert!(ok.is_ok());
}