cbcc27f1f7
The docs said a host with no per-process counter gets the token-count
estimate off macOS, but there ProcessSpanMeter falls back to detect_meter()
(the 28.6 W x wall estimator in this build), which it marks Whole, and grade()
kept that figure. A fallback span is the machine's figure, not the engine's,
so grade() now treats it as no reading and uses the rest (host_counter, else
the token estimate): one way of reading the host whichever engine ran, and
the docs hold. That is simpler than documenting a second host figure that
only the in-process engine would get.
JOULE_CODE_L3_SELF_CONTAINED could never take effect: the llama engine is
read at its exit, which the tree meter always marks (TreeGap::Exited), so no
reading was ever whole. The setting and its plumbing are gone and the docs
say the reading is a lower bound, always graded as an estimate.
A missing llama binary had become LlmError::Upstream("... exited 127"): the
binary is exec'd by the gating sh, whose failed exec reads as the model's own
failure. The binary is now resolved first (a path as given, else PATH; a
regular executable file) and one that cannot run is Unavailable, as a failed
spawn was, with nothing spawned.
The README says the oracle tests run only locally on Apple Silicon (CI tests
this workspace on Linux, where they take the fallback), and lines this branch
added name the capability, not a third-party engine.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
663 lines
33 KiB
Rust
663 lines
33 KiB
Rust
//! Run the Joule Code L3 (Model) tier against a real local GGUF model and print
|
||
//! the resulting coding operation + receipt.
|
||
//!
|
||
//! Usage:
|
||
//! cargo run --bin run -- <model.gguf> "<coding request>"
|
||
//!
|
||
//! By default it runs the model **in-process on JouleClaw's own from-scratch GGUF
|
||
//! tensor runtime** (`jouleclaw-runtime`); set `JOULE_CODE_L3_ENGINE=llama` to use
|
||
//! the `llama.cpp` shell-out backend instead. Either way it content-addresses the
|
||
//! result as a `CodingOp` and puts the energy of the inference on the receipt,
|
||
//! read as close to the work as this host allows and graded by how much of it the
|
||
//! reading covers ([`grade`]):
|
||
//!
|
||
//! - **The in-process engine** is metered on this process's own CPU and GPU
|
||
//! (`jouleclaw_energy::span::ProcessSpanMeter`): the CPU energy the kernel
|
||
//! counts for this process's threads, plus its GPU energy — an exact zero where
|
||
//! it spent no GPU time, the GPU rail where it had the GPU to itself. A span
|
||
//! that covers the whole inference keeps that grade (`HwShunt` for the two
|
||
//! counters). One that covers part of it — the GPU shared with another
|
||
//! process, so the CPU alone — is an estimate of the whole, never a reading.
|
||
//! - **The llama engine** runs in a subprocess, which this process's counters do
|
||
//! not see; its own process is read instead (`sandbox_meter::TreeMeter`), where
|
||
//! the host counts each process's CPU energy. That reading is never whole: it is
|
||
//! the process's CPU only, not the GPU the binary may offload to, and the
|
||
//! process exits in its window, so its record lacks its last time on a core (a
|
||
//! lower bound). It is always an estimate of the whole.
|
||
//!
|
||
//! An estimate of the whole is the reading plus the host meter's figure over the
|
||
//! same window (or, with no host counter, the token-count estimate): never less
|
||
//! than was read. The receipt has no coverage field, so a partial reading is
|
||
//! never sealed as a hardware reading. Where the engine's own process gives no
|
||
//! figure — no per-process counter, or this process's counters not live (the
|
||
//! span then falls back to a host meter, whose figure is the machine's, not the
|
||
//! engine's) — the host meter's figure (a measurement where it is a live
|
||
//! counter, an estimate from host load where it is not) or, with no host
|
||
//! counter, the token-count estimate stands. The printed receipt says which.
|
||
|
||
use jouleclaw_energy::meter::EnergyMeter;
|
||
use jouleclaw_energy::span::{ProcessSpanMeter, SpanEnergy, SpanSource, SpanUses};
|
||
use jouleclaw_energy::{Coverage, EnergyError, Provenance as JcProvenance};
|
||
use jouleclaw_llm::{LlmBackend, LlmRequest, LlmResponse};
|
||
use sandbox_meter::{TreeError, TreeReading};
|
||
|
||
use joule_code_cargo::{Anchor, CodingOp, Edit};
|
||
use joule_code_receipt::{Energy, Provenance, Receipt};
|
||
use joule_code_schema::{ContentId, Tier, ALG_ED25519};
|
||
|
||
use l3_jouleclaw_binding::llama_backend::LlamaCompletionBackend;
|
||
use l3_jouleclaw_binding::runtime_backend::JouleclawRuntimeBackend;
|
||
|
||
/// The host's best energy counter, or None where unavailable (e.g. CI/Linux):
|
||
/// the meter of the rest of the host over the inference, which a partial reading
|
||
/// of the engine's own process is graded with, and the figure that stands where
|
||
/// the engine's own process gives none ([`grade`]).
|
||
///
|
||
/// Not, on its own, the inference's meter: this process's CPU counter alone
|
||
/// would bill the in-process engine's GPU kernels (Metal and MPS matmuls, the
|
||
/// quantized GEMVs that serve decode) nothing, and the llama engine's subprocess
|
||
/// all of nothing. Those are read by [`ProcessSpanMeter`] and the llama
|
||
/// backend's own tree meter.
|
||
fn host_counter() -> Option<Box<dyn EnergyMeter>> {
|
||
#[cfg(all(target_os = "macos", target_arch = "aarch64"))]
|
||
{
|
||
// The IOReport energy counter (HwShunt, a live vendor counter) where it can
|
||
// meter CPU work: it is unavailable unless IOReport opens AND its CPU energy
|
||
// rail advances under load (`jouleclaw_energy::ioreport_status()` says why
|
||
// not). Otherwise the load-average estimator (Estimator).
|
||
let ioreport = jouleclaw_energy::apple_ioreport::AppleIoReportCounter::new();
|
||
if ioreport.is_available() {
|
||
return Some(Box::new(ioreport));
|
||
}
|
||
let meter = jouleclaw_energy::apple::AppleSiliconMeter::new();
|
||
if meter.is_available() {
|
||
return Some(Box::new(meter));
|
||
}
|
||
}
|
||
None
|
||
}
|
||
|
||
/// The receipt energy for an inference: the host counter's delta across it, with
|
||
/// the counter's provenance, or — without a counter, or without two readings of
|
||
/// it — the token-count estimate. `true` when the figure came from the counter.
|
||
/// A counter that went backwards, one that registered nothing across the
|
||
/// inference (no inference runs on zero energy), or readings that disagree about
|
||
/// provenance give no counter figure: never a zero.
|
||
fn receipt_energy(
|
||
before: Option<(u64, JcProvenance)>,
|
||
after: Option<(u64, JcProvenance)>,
|
||
input_tokens: u64,
|
||
output_tokens: u64,
|
||
) -> (Energy, bool) {
|
||
if let (Some((b, before_prov)), Some((a, prov))) = (before, after) {
|
||
if let Some(uj) = a.checked_sub(b).filter(|uj| *uj > 0 && before_prov == prov) {
|
||
let prov = map_provenance(prov);
|
||
let measured_uj = (prov == Provenance::HwShunt).then_some(uj);
|
||
return (Energy { measured_uj, estimated_uj: uj, provenance: prov }, true);
|
||
}
|
||
}
|
||
(Energy::from_tokens(input_tokens, output_tokens), false)
|
||
}
|
||
|
||
/// How the receipt's energy was obtained, for the printout: by which counter, or
|
||
/// from token counts. An estimate is never called metered.
|
||
fn energy_note(energy: &Energy, counter: &str, from_counter: bool) -> String {
|
||
match (from_counter, energy.provenance) {
|
||
(false, _) => "estimated from token counts".to_string(),
|
||
(true, Provenance::HwShunt) => format!("measured by {counter}"),
|
||
(true, Provenance::ModelBased) => format!("host-metered by {counter} (energy model)"),
|
||
(true, Provenance::Estimator) => format!("estimated by {counter} from host load"),
|
||
}
|
||
}
|
||
|
||
fn map_provenance(p: JcProvenance) -> Provenance {
|
||
match p {
|
||
JcProvenance::HwShunt => Provenance::HwShunt,
|
||
JcProvenance::ModelBased => Provenance::ModelBased,
|
||
JcProvenance::Estimator => Provenance::Estimator,
|
||
}
|
||
}
|
||
|
||
/// What the engine's own process read over the inference.
|
||
enum Own {
|
||
/// The in-process engine: this process's CPU and GPU over the call
|
||
/// ([`ProcessSpanMeter`], [`SpanUses::CpuGpu`]).
|
||
Process(Result<SpanEnergy, EnergyError>),
|
||
/// The llama engine's subprocess: its own process tree, `None` where the
|
||
/// host has no per-process counter.
|
||
Child { reading: Option<Result<TreeReading, TreeError>> },
|
||
}
|
||
|
||
/// Why a reading of the llama engine's process tree is partial although the
|
||
/// tree meter saw no gap in it: the process had not exited when it was read
|
||
/// (the wait for its exit failed). One read at its exit always carries the
|
||
/// exit's own gap instead.
|
||
const NOT_READ_AT_EXIT: &str = "the model had not exited when its process tree was read";
|
||
|
||
/// The receipt's energy for the inference, and how it was obtained: the one
|
||
/// place it is graded. `rest` is the host meter's figure over the same window,
|
||
/// or the token-count estimate ([`receipt_energy`]), with its note.
|
||
///
|
||
/// - **Whole** — the in-process span covers the whole inference on this
|
||
/// process's own counters (this process the GPU's effective sole user, or no
|
||
/// GPU time at all): the reading at its own grade, `HwShunt` with
|
||
/// `measured_uj` for a hardware reading.
|
||
/// - **Partial** — the CPU alone (the GPU shared, a client vanished), or the
|
||
/// child's tree, which is never read whole (it exits in its window): an
|
||
/// estimate of the whole, `Estimator`, the reading plus `rest` — never less
|
||
/// than was read, nothing outside the reading left out; it errs high by at most
|
||
/// the reading, `rest` covering the host. The rule mesh's in-process meter
|
||
/// bills a partial span by.
|
||
/// - **None** — no figure from the engine's own process, including a span that
|
||
/// fell back to a host meter because this process's counters gave none (that
|
||
/// figure is the machine's, not the engine's): `rest` as it is, so the host
|
||
/// figure is read the one way, [`host_counter`], whichever engine ran.
|
||
fn grade(own: &Own, rest: (Energy, String)) -> (Energy, String) {
|
||
let reading = |uj: u64, provenance: Provenance| Energy {
|
||
measured_uj: (provenance == Provenance::HwShunt).then_some(uj),
|
||
estimated_uj: uj,
|
||
provenance,
|
||
};
|
||
let estimate = |uj: u64| Energy { measured_uj: None, estimated_uj: uj, provenance: Provenance::Estimator };
|
||
let (rest, rest_note) = rest;
|
||
match own {
|
||
Own::Process(Ok(SpanEnergy { source: SpanSource::Fallback { why, .. }, .. })) => {
|
||
(rest, format!("{rest_note} (this process's counters gave no figure: {why})"))
|
||
}
|
||
Own::Process(Ok(s)) if s.coverage == Coverage::Whole => {
|
||
let note = match &s.source {
|
||
SpanSource::Process { cpu_uj, gpu } => format!(
|
||
"measured on this process's CPU ({cpu_uj} µJ) and GPU ({} µJ), the whole inference",
|
||
gpu.uj
|
||
),
|
||
// (A fallback is graded as no reading, above.)
|
||
SpanSource::CpuOnly { why, .. } | SpanSource::Fallback { why, .. } => {
|
||
format!("this process's CPU ({why})")
|
||
}
|
||
};
|
||
(reading(s.uj, map_provenance(s.provenance)), note)
|
||
}
|
||
Own::Process(Ok(s)) => {
|
||
let why = match &s.source {
|
||
SpanSource::CpuOnly { why, .. } => why.clone(),
|
||
_ => "a span that covers part of the inference".to_string(),
|
||
};
|
||
let note = format!(
|
||
"estimated: this process's CPU read {} µJ, not all of the inference ({why}), plus {} µJ {rest_note}",
|
||
s.uj, rest.estimated_uj
|
||
);
|
||
(estimate(s.uj.saturating_add(rest.estimated_uj)), note)
|
||
}
|
||
Own::Process(Err(e)) => (rest, format!("{rest_note} (this process's counters: {e})")),
|
||
Own::Child { reading: Some(Ok(r)) } => {
|
||
let cpu_uj = r.cpu_nj.div_ceil(1_000);
|
||
let mut because: Vec<String> = r.gaps.iter().map(ToString::to_string).collect();
|
||
if because.is_empty() {
|
||
because.push(NOT_READ_AT_EXIT.to_string());
|
||
}
|
||
let note = format!(
|
||
"estimated: the model's own process read {cpu_uj} µJ of CPU, not all of the inference ({}), plus {} \
|
||
µJ {rest_note}",
|
||
because.join("; "),
|
||
rest.estimated_uj
|
||
);
|
||
(estimate(cpu_uj.saturating_add(rest.estimated_uj)), note)
|
||
}
|
||
Own::Child { reading: Some(Err(e)) } => (rest, format!("{rest_note} (the model's own process: {e})")),
|
||
Own::Child { reading: None } => (rest, rest_note),
|
||
}
|
||
}
|
||
|
||
/// The engine that runs the model.
|
||
enum Engine {
|
||
Runtime(Box<JouleclawRuntimeBackend>),
|
||
Llama(LlamaCompletionBackend),
|
||
}
|
||
|
||
impl Engine {
|
||
fn model_name(&self) -> &str {
|
||
match self {
|
||
Engine::Runtime(b) => b.model_name(),
|
||
Engine::Llama(b) => b.model_name(),
|
||
}
|
||
}
|
||
}
|
||
|
||
/// Run in-process `work` — the engine's decode, its CPU threads and the Metal
|
||
/// kernels it dispatches and waits on — under `meter`, as running on this
|
||
/// process's CPU and GPU and nothing else.
|
||
fn in_process<T>(meter: &ProcessSpanMeter, work: impl FnOnce() -> T) -> (T, Own) {
|
||
let (out, energy) = meter.measure(SpanUses::CpuGpu, work);
|
||
(out, Own::Process(energy))
|
||
}
|
||
|
||
/// Run `request` on `engine`, read on the engine's own process: the in-process
|
||
/// engine by `span` (built before any window opens: its first construction
|
||
/// probes the counters), the llama engine by its subprocess's tree.
|
||
fn infer(
|
||
engine: &Engine,
|
||
span: Option<&ProcessSpanMeter>,
|
||
request: &LlmRequest,
|
||
) -> (Result<LlmResponse, jouleclaw_llm::LlmError>, Own) {
|
||
match (engine, span) {
|
||
(Engine::Runtime(b), Some(meter)) => in_process(meter, || b.complete(request)),
|
||
(Engine::Runtime(b), None) => (b.complete(request), Own::Process(Err(EnergyError::NotCurrent))),
|
||
(Engine::Llama(b), _) => {
|
||
let (response, reading) = b.complete_metered(request);
|
||
(response, Own::Child { reading })
|
||
}
|
||
}
|
||
}
|
||
|
||
fn main() {
|
||
let mut args = std::env::args().skip(1);
|
||
// Model path: first arg, else $JOULE_CODE_L3_MODEL.
|
||
let model = match args.next().or_else(|| std::env::var("JOULE_CODE_L3_MODEL").ok()) {
|
||
Some(m) => m,
|
||
None => {
|
||
eprintln!(
|
||
"usage: run <model.gguf> [\"<coding request>\"] (or set JOULE_CODE_L3_MODEL)"
|
||
);
|
||
std::process::exit(2);
|
||
}
|
||
};
|
||
let intent = args.next().unwrap_or_else(|| {
|
||
"Write a Rust function `fib(n: u64) -> u64` returning the nth Fibonacci number.".to_string()
|
||
});
|
||
|
||
// Backend: in-process from-scratch runtime by default; llama.cpp on request.
|
||
let engine_name = std::env::var("JOULE_CODE_L3_ENGINE").unwrap_or_else(|_| "runtime".to_string());
|
||
let engine = match engine_name.as_str() {
|
||
"llama" => Engine::Llama(LlamaCompletionBackend::new(&model)),
|
||
_ => match JouleclawRuntimeBackend::from_path(&model) {
|
||
Ok(b) => Engine::Runtime(Box::new(b)),
|
||
Err(e) => {
|
||
eprintln!("failed to load model on jouleclaw-runtime: {e}");
|
||
std::process::exit(1);
|
||
}
|
||
},
|
||
};
|
||
// The in-process engine's meter, built (and its counters probed) before any
|
||
// window opens.
|
||
let span = matches!(engine, Engine::Runtime(_))
|
||
.then(|| ProcessSpanMeter::new(jouleclaw_backend_apple::gpu_liveness_load));
|
||
let counter = host_counter();
|
||
let counter_name = counter.as_ref().map_or("none", |c| c.name());
|
||
|
||
// Meter host energy across the inference. Prime the counter once so the
|
||
// first-sample init delta is excluded from the measured window.
|
||
let read = |c: &Option<Box<dyn EnergyMeter>>| {
|
||
c.as_ref()
|
||
.and_then(|c| c.read().ok())
|
||
.map(|r| (r.uj, r.provenance))
|
||
};
|
||
let _ = read(&counter); // prime
|
||
let before = read(&counter);
|
||
|
||
let (response, own) = infer(&engine, span.as_ref(), &LlmRequest::from_prompt(&intent, 256));
|
||
let response = match response {
|
||
Ok(r) => r,
|
||
Err(e) => {
|
||
eprintln!("L3 inference failed: {e}");
|
||
std::process::exit(1);
|
||
}
|
||
};
|
||
|
||
let after = read(&counter);
|
||
|
||
// Materialize the model output as a coding operation.
|
||
let op = CodingOp {
|
||
alg: ALG_ED25519,
|
||
prior: ContentId::of_bytes(b"scratch-tree"),
|
||
edits: vec![Edit {
|
||
path: "GENERATED.rs".into(),
|
||
anchor: Anchor::ByteRange { start: 0, end: 0 },
|
||
replacement: response.text.clone().into_bytes(),
|
||
}],
|
||
author: [0u8; 32],
|
||
intent: intent.clone(),
|
||
tier: Tier::L3,
|
||
};
|
||
let op_id = op.op_id().expect("op_id");
|
||
|
||
// Receipt energy: the engine's own reading, graded by how much of the
|
||
// inference it covers, with the host counter's figure (else the token-based
|
||
// estimate) for the rest.
|
||
let (rest, from_counter) = receipt_energy(
|
||
before,
|
||
after,
|
||
response.input_tokens as u64,
|
||
response.output_tokens as u64,
|
||
);
|
||
let rest_note = energy_note(&rest, counter_name, from_counter);
|
||
let (energy, note) = grade(&own, (rest, rest_note));
|
||
let receipt = Receipt {
|
||
op_id: op_id.clone(),
|
||
tier: Tier::L3,
|
||
energy,
|
||
sequence: 0,
|
||
nonce: [0u8; 8],
|
||
};
|
||
|
||
println!("== Joule Code L3 (local model) ==");
|
||
println!("model : {}", engine.model_name());
|
||
println!("engine : {engine_name}");
|
||
println!("intent : {intent}");
|
||
println!("--- generated ---");
|
||
println!("{}", response.text);
|
||
println!("--- coding op ---");
|
||
println!("op_id : {op_id}");
|
||
println!("edits : {} (GENERATED.rs)", op.edits.len());
|
||
println!("tier : {}", receipt.tier);
|
||
println!(
|
||
"host meter : {counter_name} (IOReport: {:?})",
|
||
jouleclaw_energy::ioreport_status()
|
||
);
|
||
println!(
|
||
"tokens : in={} out={}",
|
||
response.input_tokens, response.output_tokens
|
||
);
|
||
println!(
|
||
"energy : {:.3} J ({:?}, {note})",
|
||
receipt.energy.estimated_uj as f64 / 1_000_000.0,
|
||
receipt.energy.provenance,
|
||
);
|
||
}
|
||
|
||
#[cfg(test)]
|
||
mod tests {
|
||
use super::*;
|
||
|
||
#[test]
|
||
fn a_counter_reading_keeps_its_provenance() {
|
||
let (e, from_counter) =
|
||
receipt_energy(Some((100, JcProvenance::HwShunt)), Some((400, JcProvenance::HwShunt)), 7, 9);
|
||
assert!(from_counter);
|
||
assert_eq!((e.measured_uj, e.estimated_uj, e.provenance), (Some(300), 300, Provenance::HwShunt));
|
||
for (jc, p) in [(JcProvenance::ModelBased, Provenance::ModelBased), (JcProvenance::Estimator, Provenance::Estimator)] {
|
||
let (e, from_counter) = receipt_energy(Some((100, jc)), Some((400, jc)), 7, 9);
|
||
assert!(from_counter);
|
||
assert_eq!((e.measured_uj, e.estimated_uj, e.provenance), (None, 300, p));
|
||
assert!(e.check_provenance().is_ok());
|
||
}
|
||
}
|
||
|
||
#[test]
|
||
fn without_two_readings_the_receipt_is_the_token_estimate() {
|
||
let tokens = Energy::from_tokens(7, 9);
|
||
let est = JcProvenance::Estimator;
|
||
for (before, after) in [
|
||
(None, None),
|
||
(None, Some((400, est))),
|
||
(Some((100, est)), None),
|
||
(Some((400, est)), Some((100, est))), // went backwards
|
||
(Some((400, est)), Some((400, est))), // registered nothing
|
||
(Some((9, JcProvenance::ModelBased)), Some((9, JcProvenance::ModelBased))), // a held rail
|
||
(Some((100, JcProvenance::ModelBased)), Some((400, est))), // disagree
|
||
] {
|
||
let (e, from_counter) = receipt_energy(before, after, 7, 9);
|
||
assert!(!from_counter, "{before:?} → {after:?}");
|
||
assert_eq!(e, tokens);
|
||
}
|
||
}
|
||
|
||
#[test]
|
||
fn an_estimate_is_never_printed_as_metered() {
|
||
let est = Energy { measured_uj: None, estimated_uj: 5, provenance: Provenance::Estimator };
|
||
assert_eq!(energy_note(&est, "apple-silicon", true), "estimated by apple-silicon from host load");
|
||
assert_eq!(energy_note(&est, "none", false), "estimated from token counts");
|
||
let modeled = Energy { measured_uj: None, estimated_uj: 5, provenance: Provenance::ModelBased };
|
||
assert_eq!(energy_note(&modeled, "apple-ioreport", true), "host-metered by apple-ioreport (energy model)");
|
||
let measured = Energy { measured_uj: Some(5), estimated_uj: 5, provenance: Provenance::HwShunt };
|
||
assert_eq!(energy_note(&measured, "rapl", true), "measured by rapl");
|
||
}
|
||
|
||
fn est(uj: u64) -> (Energy, String) {
|
||
(Energy { measured_uj: None, estimated_uj: uj, provenance: Provenance::Estimator }, "the rest".into())
|
||
}
|
||
|
||
fn span(uj: u64, provenance: JcProvenance, coverage: Coverage, source: SpanSource) -> Own {
|
||
Own::Process(Ok(SpanEnergy { uj, provenance, coverage, source }))
|
||
}
|
||
|
||
fn tree(cpu_nj: u64, gaps: Vec<sandbox_meter::TreeGap>) -> Option<Result<TreeReading, TreeError>> {
|
||
Some(Ok(TreeReading {
|
||
cpu_uj: cpu_nj.div_ceil(1_000),
|
||
cpu_nj,
|
||
processes: 1,
|
||
pids: vec![7],
|
||
wall: std::time::Duration::from_secs(1),
|
||
gaps,
|
||
}))
|
||
}
|
||
|
||
/// The grading, pure: a reading keeps its hardware grade only when it covers
|
||
/// the whole inference; a partial one is an estimate, the reading plus the
|
||
/// rest; no reading is the rest as it is. The receipt never seals a partial
|
||
/// reading as `HwShunt`.
|
||
#[test]
|
||
fn a_reading_keeps_its_hardware_grade_only_when_it_covers_the_whole_inference() {
|
||
use jouleclaw_energy::process_gpu::GpuShare;
|
||
let gpu = GpuShare { uj: 700, self_ns: 9, rail_uj: Some(710), all_ns: Some(9) };
|
||
let whole = span(1_000, JcProvenance::HwShunt, Coverage::Whole, SpanSource::Process { cpu_uj: 300, gpu });
|
||
let (e, note) = grade(&whole, est(40_000));
|
||
assert_eq!((e.measured_uj, e.estimated_uj, e.provenance), (Some(1_000), 1_000, Provenance::HwShunt));
|
||
assert!(note.starts_with("measured on this process's CPU (300 µJ) and GPU (700 µJ)"), "{note}");
|
||
|
||
let cpu_only = SpanSource::CpuOnly { cpu_uj: 300, why: "process GPU: shared".into() };
|
||
let (e, note) = grade(&span(300, JcProvenance::HwShunt, Coverage::Partial, cpu_only), est(40_000));
|
||
assert_eq!((e.measured_uj, e.estimated_uj, e.provenance), (None, 40_300, Provenance::Estimator));
|
||
assert!(note.contains("shared") && note.starts_with("estimated"), "{note}");
|
||
e.check_provenance().unwrap();
|
||
|
||
// A span that fell back to a host meter (off macOS, the 28.6 W × wall
|
||
// estimator) is no reading of the engine: the rest stands, as it is.
|
||
let fallback = SpanSource::Fallback { meter: "estimation", why: "process CPU: not live".into() };
|
||
let (e, note) = grade(&span(5_000, JcProvenance::Estimator, Coverage::Whole, fallback), est(40_000));
|
||
assert_eq!(e, est(40_000).0, "the fallback's figure stood in for the rest");
|
||
assert!(note.starts_with("the rest") && note.contains("process CPU: not live"), "{note}");
|
||
let hw_fallback = SpanSource::Fallback { meter: "rapl", why: "process CPU: not live".into() };
|
||
let (e, _) = grade(&span(5_000, JcProvenance::HwShunt, Coverage::Whole, hw_fallback), est(40_000));
|
||
assert_eq!(e, est(40_000).0, "a machine-wide reading is not the engine's own");
|
||
|
||
let (e, note) = grade(&Own::Process(Err(EnergyError::NotAdvanced)), est(40_000));
|
||
assert_eq!(e, est(40_000).0);
|
||
assert!(note.starts_with("the rest"), "{note}");
|
||
|
||
// The llama engine's subprocess: never read whole.
|
||
let exited = vec![sandbox_meter::TreeGap::Exited { pid: 7 }];
|
||
let child = |reading| Own::Child { reading };
|
||
let (e, note) = grade(&child(tree(9_000_001, exited.clone())), est(40_000));
|
||
assert_eq!((e.measured_uj, e.estimated_uj, e.provenance), (None, 49_001, Provenance::Estimator), "{note}");
|
||
assert!(note.contains("not all of the inference") && note.contains("pid 7 exited"), "{note}");
|
||
let (e, note) = grade(&child(tree(9_000_000, vec![])), est(40_000));
|
||
assert_eq!((e.measured_uj, e.estimated_uj, e.provenance), (None, 49_000, Provenance::Estimator), "{note}");
|
||
assert!(note.contains(NOT_READ_AT_EXIT), "{note}");
|
||
// A hardware host figure does not make the sum a reading.
|
||
let hw = (Energy { measured_uj: Some(40_000), estimated_uj: 40_000, provenance: Provenance::HwShunt }, "r".into());
|
||
let (e, _) = grade(&child(tree(9_000_000, exited)), hw.clone());
|
||
assert_eq!((e.measured_uj, e.provenance), (None, Provenance::Estimator));
|
||
for reading in [None, Some(Err(TreeError::RootGone { pid: 7 }))] {
|
||
let (e, _) = grade(&child(reading), hw.clone());
|
||
assert_eq!(e, hw.0, "no reading: the host meter's figure as it is");
|
||
}
|
||
}
|
||
|
||
/// This process's own CPU energy count, nJ: `proc_pid_rusage(getpid(),
|
||
/// RUSAGE_INFO_V6)`'s `ri_energy_nj`, read with a bare buffer of the struct's
|
||
/// 464 bytes at its SDK offset, 336. Written out here, sharing no code with
|
||
/// the meters it checks.
|
||
#[cfg(target_os = "macos")]
|
||
fn own_energy_nj() -> u64 {
|
||
extern "C" {
|
||
fn proc_pid_rusage(pid: i32, flavor: i32, buffer: *mut std::ffi::c_void) -> i32;
|
||
}
|
||
let mut buf = [0u64; 58];
|
||
// SAFETY: `buf` is 464 writable bytes, the size of `rusage_info_v6`.
|
||
let rc = unsafe { proc_pid_rusage(std::process::id() as i32, 6, buf.as_mut_ptr().cast()) };
|
||
assert_eq!(rc, 0, "proc_pid_rusage on this process");
|
||
buf[336 / 8]
|
||
}
|
||
|
||
#[cfg(not(target_os = "macos"))]
|
||
fn own_energy_nj() -> u64 {
|
||
0
|
||
}
|
||
|
||
/// A fixed amount of CPU work: about a second of one core, unoptimized.
|
||
fn burn() -> u64 {
|
||
let mut x = 1u64;
|
||
for i in 0..150_000_000u64 {
|
||
x = std::hint::black_box(x.wrapping_mul(6_364_136_223_846_793_005).wrapping_add(i));
|
||
}
|
||
x
|
||
}
|
||
|
||
/// The in-process engine's meter ([`in_process`], as `main` runs the engine) on
|
||
/// CPU work, against an oracle sharing no code with it: this process's own
|
||
/// count, read around the same work. The work uses
|
||
/// no GPU, so its GPU share is an exact zero and the span covers it whole, a
|
||
/// hardware reading of this process — the oracle's window holds the span's, so
|
||
/// the reading is at most the oracle and short of it only by the meter's own
|
||
/// snapshots. Twice. Where the counters are not live it is the fallback's.
|
||
#[test]
|
||
fn the_in_process_engine_is_read_on_this_process_as_its_own_count_says() {
|
||
let meter = ProcessSpanMeter::new(jouleclaw_backend_apple::gpu_liveness_load);
|
||
let metered = meter.is_metered();
|
||
for round in 1..=2 {
|
||
let before = own_energy_nj();
|
||
let (x, own) = in_process(&meter, burn);
|
||
std::thread::sleep(std::time::Duration::from_millis(20));
|
||
let oracle_uj = (own_energy_nj() - before) / 1_000;
|
||
std::hint::black_box(x);
|
||
let (e, note) = grade(&own, est(1));
|
||
let Own::Process(energy) = own else { unreachable!() };
|
||
eprintln!("in-process round {round}: oracle {oracle_uj} µJ; span {energy:?}; receipt {e:?} ({note})");
|
||
if metered {
|
||
let s = energy.expect("a metered span has a figure");
|
||
let (cpu_uj, gpu) = s.parts().expect("this process's own counters");
|
||
assert_eq!((gpu.uj, gpu.self_ns), (0, 0), "CPU work spent no GPU time");
|
||
assert_eq!((s.coverage, s.provenance), (Coverage::Whole, JcProvenance::HwShunt));
|
||
let ratio = cpu_uj as f64 / oracle_uj as f64;
|
||
assert!(oracle_uj > 100_000, "setup: the burn read {oracle_uj} µJ");
|
||
assert!((0.8..=1.02).contains(&ratio), "the span read {cpu_uj} µJ of {oracle_uj} µJ");
|
||
assert_eq!((e.measured_uj, e.estimated_uj, e.provenance), (Some(cpu_uj), cpu_uj, Provenance::HwShunt));
|
||
} else {
|
||
assert_ne!(e.provenance, Provenance::HwShunt, "no live counter, yet a reading: {e:?}");
|
||
}
|
||
}
|
||
}
|
||
|
||
/// The stand-in model's marker and report, in its working directory.
|
||
const STAND_IN_MARKER: &str = ".llama-stand-in";
|
||
const STAND_IN_REPORT: &str = ".llama-stand-in-report";
|
||
|
||
/// Not a test of its own: the stand-in llama engine, run as a subprocess by
|
||
/// [`the_llama_engine_is_read_on_its_own_process_as_its_own_count_says`] (this
|
||
/// test binary, filtered to this test, behind a script that takes the engine's
|
||
/// arguments). It does a fixed amount of CPU work, leaves the core (which folds
|
||
/// its count), and writes its own count to its working directory. Anywhere
|
||
/// without its marker it returns at once.
|
||
#[test]
|
||
#[ignore = "a subprocess stand-in, run by the llama engine's energy test"]
|
||
fn llama_stand_in() {
|
||
if !std::path::Path::new(STAND_IN_MARKER).exists() {
|
||
return;
|
||
}
|
||
let x = burn();
|
||
std::thread::sleep(std::time::Duration::from_millis(20));
|
||
std::fs::write(STAND_IN_REPORT, format!("{} {x}", own_energy_nj())).expect("the report");
|
||
}
|
||
|
||
/// The llama engine's subprocess, read on its own process: the stand-in burns a
|
||
/// fixed amount of CPU and reports its own count from inside before it exits —
|
||
/// the oracle, sharing no code with the tree meter. Where this host reads the
|
||
/// process (macOS, the per-process counter live) the reading agrees with that
|
||
/// count to within the shells' start-up and the stand-in's tear-down, and the
|
||
/// receipt is an estimate (the process exited in its window), never `HwShunt`.
|
||
#[test]
|
||
fn the_llama_engine_is_read_on_its_own_process_as_its_own_count_says() {
|
||
let dir = std::env::temp_dir().join(format!("l3-llama-stand-in-{}", std::process::id()));
|
||
let _ = std::fs::remove_dir_all(&dir);
|
||
std::fs::create_dir_all(&dir).unwrap();
|
||
std::fs::write(dir.join(STAND_IN_MARKER), "").unwrap();
|
||
let exe = std::env::current_exe().expect("this test binary");
|
||
let script = dir.join("stand-in-llama");
|
||
std::fs::write(
|
||
&script,
|
||
format!("#!/bin/sh\ncd '{}' || exit 1\nexec '{}' tests::llama_stand_in --exact --ignored\n", dir.display(), exe.display()),
|
||
)
|
||
.unwrap();
|
||
assert!(std::process::Command::new("chmod").arg("+x").arg(&script).status().unwrap().success());
|
||
let reads_processes =
|
||
cfg!(target_os = "macos") && sandbox_meter::process_energy_status() == sandbox_meter::CounterStatus::Live;
|
||
let engine = Engine::Llama(LlamaCompletionBackend::new(dir.join("model.gguf")).with_binary(script.display().to_string()));
|
||
|
||
for round in 1..=2 {
|
||
let (response, own) = infer(&engine, None, &LlmRequest::from_prompt("fib", 16));
|
||
let response = response.expect("the stand-in ran");
|
||
assert!(response.text.contains("1 passed"), "{}", response.text);
|
||
let report = std::fs::read_to_string(dir.join(STAND_IN_REPORT)).expect("the stand-in's report");
|
||
let _ = std::fs::remove_file(dir.join(STAND_IN_REPORT));
|
||
let oracle_uj = report.split_whitespace().next().unwrap().parse::<u64>().unwrap() / 1_000;
|
||
let (e, note) = grade(&own, est(40_000));
|
||
eprintln!("llama engine round {round}: oracle {oracle_uj} µJ; receipt {e:?} ({note})");
|
||
let Own::Child { reading } = own else { panic!("the llama engine is read as a child") };
|
||
if reads_processes {
|
||
let r = reading.expect("a tree meter on this host").expect("a reading of the model's process");
|
||
let read = r.cpu_nj.div_ceil(1_000);
|
||
assert!(oracle_uj > 100_000, "setup: the stand-in's work read {oracle_uj} µJ");
|
||
let ratio = read as f64 / oracle_uj as f64;
|
||
assert!((0.8..=1.25).contains(&ratio), "read {read} µJ of a process that counted {oracle_uj} µJ");
|
||
assert!(!r.is_whole(), "an exited process is never read whole: {r:?}");
|
||
assert_eq!((e.measured_uj, e.estimated_uj, e.provenance), (None, read + 40_000, Provenance::Estimator));
|
||
} else {
|
||
assert!(reading.is_none());
|
||
assert_eq!(e, est(40_000).0);
|
||
}
|
||
}
|
||
let _ = std::fs::remove_dir_all(&dir);
|
||
}
|
||
|
||
/// On this host: the IOReport counter is used only where its CPU rail is live.
|
||
/// Elsewhere on Apple Silicon (macOS 27.0.1 on an M5 Max and an M3 Ultra) the
|
||
/// load estimator runs and meters the burn, and the receipt says Estimator; with
|
||
/// no counter at all (CI's Linux runner), the token estimate does.
|
||
#[test]
|
||
fn on_this_host_the_estimator_fallback_runs_unless_ioreport_is_live() {
|
||
let status = jouleclaw_energy::ioreport_status();
|
||
let counter = host_counter();
|
||
let name = counter.as_ref().map_or("none", |c| c.name());
|
||
let read = || counter.as_ref().and_then(|c| c.read().ok()).map(|r| (r.uj, r.provenance));
|
||
let _ = read(); // prime, as main does
|
||
let before = read();
|
||
let t = std::time::Instant::now();
|
||
let mut x = 1u64;
|
||
while t.elapsed() < std::time::Duration::from_millis(20) {
|
||
x = x.wrapping_mul(6_364_136_223_846_793_005).wrapping_add(1);
|
||
}
|
||
std::hint::black_box(x);
|
||
let after = read();
|
||
let (e, from_counter) = receipt_energy(before, after, 7, 9);
|
||
let note = energy_note(&e, name, from_counter);
|
||
println!("l3 on this host: ioreport={status:?} meter={name} energy={e:?} ({note})");
|
||
if status == jouleclaw_energy::IoReportStatus::Live {
|
||
assert_eq!(name, "apple-ioreport");
|
||
assert_eq!(e.provenance, Provenance::HwShunt, "a live IOReport reading is a hardware reading");
|
||
assert!(from_counter && e.measured_uj.is_some_and(|uj| uj > 0), "{e:?}");
|
||
} else {
|
||
assert_ne!(name, "apple-ioreport", "IOReport is used only when its CPU rail is live");
|
||
assert_eq!(e.provenance, Provenance::Estimator);
|
||
assert_eq!(e.measured_uj, None);
|
||
assert!(note.starts_with("estimated"), "{note}");
|
||
if cfg!(all(target_os = "macos", target_arch = "aarch64")) {
|
||
assert_eq!(name, "apple-silicon");
|
||
assert!(from_counter && e.estimated_uj > 0, "the estimator metered 20 ms of burn: {e:?}");
|
||
assert_eq!(note, "estimated by apple-silicon from host load");
|
||
} else {
|
||
assert_eq!((name, from_counter), ("none", false));
|
||
assert_eq!(e, Energy::from_tokens(7, 9));
|
||
}
|
||
}
|
||
}
|
||
}
|