jouleclaw/SPEC.md

@@ -94,6 +94,27 @@ unmetered, and **SHOULD** record why. A figure **MUST NOT** claim
 coverage of a domain its counter does not read; where the format has
 a coverage field, such a figure is Partial.
 
+**The bound is two-sided (MUST).** Where an implementation judges a vendor
+energy counter against an independent yardstick — the board's own power
+telemetry, a datasheet envelope — it **MUST** apply the bound in **both**
+directions. A counter that accrues far *less* than the yardstick says the
+device drew is as refuted as one that accrues far more, and it is the more
+dangerous of the two: **under-reporting is the unsafe direction at an energy
+ceiling, because it admits work the budget cannot cover**, while a counter
+running fast fails safe by spending a budget sooner. A counter republished in
+lumps, sampled across its republish period, reads low rather than frozen — it
+advances, so a liveness probe alone calls it live.
+
+Such a judgement **refutes, never verifies**. The yardstick is itself usually
+`ModelBased` (a vendor power estimate), so a counter inside both bounds is *not
+refuted*, which is not a finding that it is correct. Where the yardstick is
+absent or itself unusable the result is an **indeterminate**, and an
+indeterminate **MUST NOT** be graded as a hardware reading either: erring
+toward declining is the safe direction. A bound **MUST NOT** fire where the
+device is near idle and the yardstick leaves no room for a finding — at idle
+draw a correctly reporting counter accrues almost nothing, and refuting it
+would be noise, not a defect.
+
 Apple IOReport's GPU rail is not a partial reading of CPU work. It is
 machine-wide — it meters every client of the GPU on the host, display
 compositing among them, and moves while nothing of the metered work

jouleclaw/jouleclaw-rs/crates/jouleclaw-energy/src/liveness.rs

@@ -337,6 +337,24 @@ pub(crate) const GPU_DRAW_FACTOR: u128 = 2;
 #[cfg_attr(not(any(test, feature = "nvml")), allow(dead_code))] // used by `nvml`
 pub(crate) const GPU_DRAW_SLACK_MW: u128 = 5_000;
 
+/// How far **below** its board's mean draw a counter may accrue before it is
+/// refuted: a shortfall of more than this factor, after [`GPU_DRAW_SLACK_MW`]
+/// is taken off the draw, is no count of the board's energy.
+///
+/// The bound has to exist in this direction at all because **under-reporting
+/// is the unsafe direction at an energy ceiling** — it admits work the budget
+/// cannot cover — while the ceiling bounds only catch a counter running *fast*,
+/// which is the direction that fails safe. SPEC §1.3 already refuses a counter
+/// "republished in lumps", and a counter sampled across its republish period
+/// reads low rather than frozen, so the liveness probe alone does not see it.
+///
+/// It is deliberately looser than [`GPU_DRAW_FACTOR`]. The draw readings are
+/// one-second averages, so a bursty load that is idle between samples really
+/// does integrate to less than its sampled mean, and the slack subtraction
+/// means the floor collapses to zero at or below [`GPU_DRAW_SLACK_MW`]: at
+/// idle draw there is no finding to make, only noise to avoid.
+pub(crate) const GPU_UNDER_FACTOR: u128 = 4;
+
 /// Whether one GPU's energy counter, which counted `energy_mj` over `window`,
 /// plausibly counted its board's energy, beside the board's power draw
 /// `draws_mw` read with the samples and its maximum power limit `ceiling_mw`.
@@ -387,6 +405,24 @@ pub(crate) enum GpuPlausibility {
         /// Which bound that was.
         bound: GpuBound,
     },
+    /// The counter accrued far less energy than the board's own telemetry says
+    /// it drew.
+    ///
+    /// This is the finding the ceiling bounds cannot make, and the dangerous
+    /// one: a counter reading low is graded `HwShunt` and then spends a budget
+    /// it is not measuring. A counter republished in lumps and sampled across
+    /// the republish period looks exactly like this — it advances, so the
+    /// liveness probe calls it live, while accruing a fraction of the energy.
+    CounterUnderRate {
+        /// The rate the counter implies over the window, in milliwatts.
+        rate_mw: u64,
+        /// The floor it fell below, in milliwatts.
+        floor_mw: u64,
+        /// The board's mean usable draw the floor was taken from, in
+        /// milliwatts — the yardstick, so the judgement can be re-made against
+        /// a different factor.
+        mean_draw_mw: u64,
+    },
     /// The board's own telemetry read past the board's own power limit, so
     /// there is no yardstick left to judge the counter against. An
     /// indeterminate, not a finding about the counter — see
@@ -421,6 +457,11 @@ impl GpuPlausibility {
             GpuPlausibility::Plausible { .. } => 0,
             GpuPlausibility::BoardTelemetryUnusable { .. } => 1,
             GpuPlausibility::CounterOverRate { .. } => 2,
+            // Above a proven overrun: both are refutations, but only this one
+            // is the direction that admits work a budget cannot cover, so where
+            // one GPU of a probe over-reports and another under-reports, the
+            // under-report is the verdict an operator has to act on.
+            GpuPlausibility::CounterUnderRate { .. } => 3,
         }
     }
 
@@ -429,6 +470,12 @@ impl GpuPlausibility {
         match self {
             GpuPlausibility::Plausible { rate_mw } => *rate_mw,
             GpuPlausibility::CounterOverRate { rate_mw, .. } => *rate_mw,
+            // The shortfall, so that among two under-reporting GPUs the worse
+            // one speaks: a lower rate is a bigger shortfall, so it is ordered
+            // by how far the floor was missed rather than by the rate itself.
+            GpuPlausibility::CounterUnderRate { rate_mw, floor_mw, .. } => {
+                floor_mw.saturating_sub(*rate_mw)
+            }
             GpuPlausibility::BoardTelemetryUnusable { draw_mw, .. } => *draw_mw,
         }
     }
@@ -493,6 +540,17 @@ pub(crate) fn plausible_gpu_counter(
                 bound: GpuBound::MeanDraw,
             };
         }
+        // And the same yardstick from below. The slack comes off the draw
+        // before the division, so the floor is zero at or under
+        // `GPU_DRAW_SLACK_MW` of mean draw: an idle board makes no finding.
+        let floor = mean.saturating_sub(GPU_DRAW_SLACK_MW) / GPU_UNDER_FACTOR;
+        if floor > 0 && rate_mw < floor {
+            return GpuPlausibility::CounterUnderRate {
+                rate_mw: clamp(rate_mw),
+                floor_mw: clamp(floor),
+                mean_draw_mw: clamp(mean),
+            };
+        }
     }
     GpuPlausibility::Plausible { rate_mw: clamp(rate_mw) }
 }
@@ -529,8 +587,10 @@ impl GpuWatch {
     /// except that a live counter is re-judged on whether its GPUs counted
     /// their boards' energy over the probe, beside each GPU's maximum power
     /// limit `ceilings_mw`: a counter proven to run faster than its board can
-    /// draw is [`CounterLiveness::Implausible`], and a board whose own
+    /// draw — **or far slower than it** — is [`CounterLiveness::Implausible`],
-    /// telemetry is unusable leaves it [`CounterLiveness::Unjudgeable`].
+    /// and a board whose own telemetry is unusable leaves it
+    /// [`CounterLiveness::Unjudgeable`]. Both directions refute, because a
+    /// counter reading low spends an energy ceiling it is not measuring.
     pub(crate) fn liveness(&self, verdict: CounterLiveness, ceilings_mw: &[Option<u32>]) -> CounterLiveness {
         let (Some((first, t0)), Some((last, t1))) = (&self.first, &self.last) else {
             return verdict;
@@ -557,10 +617,16 @@ impl GpuWatch {
                 )
             })
             .max_by_key(|v| (v.rank(), v.magnitude()));
+        // Exhaustive by intent. This was `_ => Live`, which would have made a
+        // newly added verdict silently admissible — and "unrecognised verdict
+        // becomes live" is the unsafe direction. A new arm must now be given a
+        // liveness deliberately, or the crate does not compile.
         match worst {
-            Some(GpuPlausibility::CounterOverRate { .. }) => CounterLiveness::Implausible,
+            Some(
+                GpuPlausibility::CounterOverRate { .. } | GpuPlausibility::CounterUnderRate { .. },
+            ) => CounterLiveness::Implausible,
             Some(GpuPlausibility::BoardTelemetryUnusable { .. }) => CounterLiveness::Unjudgeable,
-            _ => CounterLiveness::Live,
+            Some(GpuPlausibility::Plausible { .. }) | None => CounterLiveness::Live,
         }
     }
 }
@@ -578,6 +644,99 @@ pub(crate) mod tests {
     const TEST_MIN_ADVANCES: u32 = 5;
     const TEST_MIN_WINDOW: Duration = Duration::from_millis(300);
 
+    /// **The bound that was missing: a counter reading LOW.**
+    ///
+    /// The ceiling bounds catch a counter running faster than its board can
+    /// draw, which is the direction that fails safe — an over-reported figure
+    /// spends a budget sooner. A counter reading *low* is the dangerous one: it
+    /// is graded `HwShunt` and then spends a ceiling it is not measuring, which
+    /// SPEC §1.3 refuses ("republished in lumps"). Such a counter still
+    /// *advances*, so the liveness probe calls it live.
+    ///
+    /// Figures are the jetson-hub RTX 4050's own: a board drawing 14.8 W while
+    /// the counter accrues as if it were drawing a fraction of that.
+    #[test]
+    fn a_gpu_counter_must_count_no_slower_than_its_board_draws() {
+        let w = Duration::from_millis(500);
+        // Over a 500 ms window a counter's rate in mW is twice its mJ.
+        let draws = &[14_800u32, 14_800];
+        // floor = (14_800 - 5_000) / 4 = 2_450 mW.
+
+        // A counter accruing 0.5 W against a 14.8 W board: refuted, with the
+        // yardstick carried so the judgement can be re-made.
+        assert_eq!(
+            plausible_gpu_counter(250, w, draws, Some(65_000)),
+            GpuPlausibility::CounterUnderRate {
+                rate_mw: 500,
+                floor_mw: 2_450,
+                mean_draw_mw: 14_800
+            },
+            "0.5 W counted against a 14.8 W draw is no count of the board's energy"
+        );
+
+        // A counter that merely lags its board still passes: the draws are
+        // one-second averages, so a bursty load really does integrate to less
+        // than its sampled mean.
+        assert!(
+            plausible_gpu_counter(1_300, w, draws, Some(65_000)).plausible(),
+            "2.6 W against a 14.8 W mean is within the factor and must pass"
+        );
+        // Exactly at the floor is not below it.
+        assert!(plausible_gpu_counter(1_225, w, draws, Some(65_000)).plausible());
+    }
+
+    /// An idle board makes no finding. The slack comes off the draw before the
+    /// division, so the floor collapses to zero at or below it — otherwise a
+    /// correctly-reporting counter on an idle GPU would be refuted for
+    /// accruing almost nothing, which is exactly what it should accrue.
+    #[test]
+    fn an_idle_board_yields_no_under_rate_finding() {
+        let w = Duration::from_millis(500);
+        for mean in [0u32, 1_000, 4_000, 5_000] {
+            assert!(
+                plausible_gpu_counter(0, w, &[mean, mean], Some(65_000)).plausible(),
+                "a counter accruing nothing beside a {mean} mW board must not be refuted"
+            );
+        }
+        // Just above the slack the floor is barely open, and only a near-dead
+        // counter falls through it.
+        let just_above = plausible_gpu_counter(0, w, &[9_000, 9_000], Some(65_000));
+        assert_eq!(
+            just_above,
+            GpuPlausibility::CounterUnderRate { rate_mw: 0, floor_mw: 1_000, mean_draw_mw: 9_000 }
+        );
+    }
+
+    /// With no usable board telemetry there is no yardstick, so there is no
+    /// finding in either direction — an indeterminate is not a refutation.
+    #[test]
+    fn no_telemetry_means_no_under_rate_finding() {
+        let w = Duration::from_millis(500);
+        assert!(
+            plausible_gpu_counter(0, w, &[], Some(65_000)).plausible(),
+            "no draw readings: the bound is not applied"
+        );
+        // Every reading past the board's own limit: unjudgeable, not refuted.
+        assert!(matches!(
+            plausible_gpu_counter(0, w, &[590_000], Some(65_000)),
+            GpuPlausibility::BoardTelemetryUnusable { .. }
+        ));
+    }
+
+    /// Under-reporting outranks over-reporting, so across a probe's GPUs the
+    /// verdict an operator must act on is the one that admits unbudgeted work.
+    #[test]
+    fn an_under_reporting_gpu_speaks_for_a_probe_before_an_over_reporting_one() {
+        let w = Duration::from_millis(500);
+        let under = plausible_gpu_counter(250, w, &[14_800], Some(65_000));
+        let over = plausible_gpu_counter(152_250, w, &[5_550], Some(65_000));
+        assert!(matches!(under, GpuPlausibility::CounterUnderRate { .. }));
+        assert!(matches!(over, GpuPlausibility::CounterOverRate { .. }));
+        let worst = [over, under].into_iter().max_by_key(|v| (v.rank(), v.magnitude()));
+        assert_eq!(worst, Some(under), "the unsafe direction is the one that speaks");
+        assert!(!under.plausible(), "and it is not admissible");
+    }
+
     /// The plausibility rule on one GPU counter, with the readings of the RTX
     /// 4050 Laptop GPU on jetson-hub (driver 595.84, 65 W maximum limit):
     /// polled at the NVML probe's 100 ms tick its counter ran at 58-305 W while
@@ -702,6 +861,50 @@ pub(crate) mod tests {
         assert_eq!(GpuWatch::default().liveness(Live, &ceilings), Live, "no sample");
     }
 
+    /// **The wiring, not just the verdict.** A refutation that never reaches
+    /// [`CounterLiveness`] changes nothing: the counter would still be graded
+    /// `HwShunt` and still spend a ceiling it is not measuring. So this pins
+    /// the path from an under-reporting GPU through `GpuWatch::liveness`.
+    ///
+    /// GPU 0's counter rises 0.05 J over the 500 ms probe — 100 mW — while its
+    /// board reports drawing 14.8 W. It advances in every interval, so the
+    /// sustained-advance rule calls it live; only the lower bound catches it.
+    #[test]
+    fn an_under_reporting_gpu_makes_the_probe_implausible() {
+        let t0 = Instant::now();
+        let watch = |draws: [Option<u32>; 2]| {
+            let mut g = GpuWatch::default();
+            for k in 0..6u64 {
+                // GPU 0: +10 mJ per 100 ms interval (100 mW). GPU 1: 1 J (10 W).
+                let uj = [8_000_000_000 + 10_000 * k, 9_000_000_000 + 1_000_000 * k];
+                g.saw(Some(&uj), &draws, t0 + Duration::from_millis(100) * k as u32);
+            }
+            g
+        };
+        let ceilings = [Some(65_000), Some(65_000)];
+        use CounterLiveness::*;
+
+        assert_eq!(
+            watch([Some(14_800), Some(10_000)]).liveness(Live, &ceilings),
+            Implausible,
+            "GPU 0 counted 0.1 W while its board drew 14.8 W: live, and not measuring it"
+        );
+
+        // The discriminating half: the same slow counter beside an *idle*
+        // board is exactly what a correct counter looks like, and must stay
+        // live. Nothing about the counter changed — only the yardstick.
+        assert_eq!(
+            watch([Some(2_000), Some(10_000)]).liveness(Live, &ceilings),
+            Live,
+            "0.1 W counted beside a 2 W board is a counter doing its job"
+        );
+
+        // And a verdict that was not `Live` still stands, whatever the draws.
+        for verdict in [Frozen, Unavailable, Unjudgeable] {
+            assert_eq!(watch([Some(14_800), Some(10_000)]).liveness(verdict, &ceilings), verdict);
+        }
+    }
+
     /// Feed a fresh rule `samples`, the i-th at `i * every`; return it and
     /// whether it was live after each interval.
     fn feed(channels: usize, samples: &[Vec<u64>], every: Duration) -> (AdvanceWatch, Vec<bool>) {

jouleclaw/jouleclaw-rs/crates/jouleclaw-energy/src/nvml.rs

@@ -12,11 +12,15 @@
 //! 4050 Laptop GPU, driver 595.84, republishes the counter every ~100 ms, so a
 //! live counter has moved in nearly every interval; even at a 2 W idle that is
 //! ~0.2 J an interval, against a 1 mJ quantum). And every GPU's counter must
-//! plausibly count its board's energy over the probe: no faster than 1.5
+//! plausibly count its board's energy over the probe, bounded in **both**
-//! times the board's maximum power limit, or than twice the board's own power
+//! directions: no faster than 1.5 times the board's maximum power limit, or
-//! draw plus 5 W ([`crate::CounterLiveness::Implausible`]). That RTX 4050's
+//! than twice the board's own power draw plus 5 W, and no slower than a
-//! counter runs at 4 to 30 times its board's draw while it is polled, so it
+//! quarter of that draw once 5 W of slack is taken off it
-//! is declined there. Probed once per process and cached.
+//! ([`crate::CounterLiveness::Implausible`]). The lower bound is the one that
+//! matters at a ceiling — a counter reading low is graded as hardware and then
+//! spends a budget it is not measuring — while a counter running fast fails
+//! safe. That RTX 4050's counter runs at 4 to 30 times its board's draw while
+//! it is polled, so it is declined there. Probed once per process and cached.
 //!
 //! Where that counter is unsupported (pre-Volta, some consumer parts) or not
 //! live, the counter integrates `power_usage()` over wall time instead. That
@@ -194,7 +198,7 @@ impl NvmlCounter {
 /// The real liveness probe of every GPU's cumulative-energy counter: the
 /// sustained-advance rule at [`PROBE_TICK`], with every GPU's power draw read
 /// at each sample and its maximum power limit once, so that a live counter
-/// which ran faster than its board can draw is
+/// which ran faster than its board can draw, or far slower than it, is
 /// [`CounterLiveness::Implausible`] ([`crate::liveness::GpuWatch`]).
 fn probe_liveness(nvml: &nvml_wrapper::Nvml, device_count: u32) -> CounterLiveness {
     let ceilings: Vec<Option<u32>> = (0..device_count)