1fb670a512
All checks were successful
standards workspaces build + test / jouleclaw-rs (pull_request) Successful in 7m16s
standards workspaces build + test / aac-rs (pull_request) Successful in 6m34s
standards workspaces build + test / joule-code-rs (pull_request) Successful in 7m20s
standards workspaces build + test / joulecontract-rs (pull_request) Successful in 7m37s
standards workspaces build + test / map-rs (pull_request) Successful in 5m17s
standards workspaces build + test / mesh-rs (pull_request) Successful in 2m8s
standards workspaces build + test / openpay (pull_request) Successful in 13m7s
no-vanity-metrics / Scan committed Markdown for vanity metrics (pull_request) Successful in 5s
no-vanity-metrics / WAI public text names no comparison (pull_request) Successful in 5s
no-vanity-metrics / WAI public text names no company or product (pull_request) Successful in 5s
standards workspaces build + test / arl-rs (pull_request) Successful in 1m25s
standards workspaces build + test / eoc-rs (pull_request) Successful in 2m32s
standards workspaces build + test / jcp-rs (pull_request) Successful in 1m55s
standards workspaces build + test / joule-ui-rs (pull_request) Successful in 1m27s
standards workspaces build + test / joulehook-rs (pull_request) Successful in 1m27s
standards workspaces build + test / proof-rs (pull_request) Successful in 1m33s
standards workspaces build + test / sandbox-rs (pull_request) Successful in 3m4s
standards workspaces build + test / sense-rs (pull_request) Successful in 1m24s
standards workspaces build + test / smart-byte-rs (pull_request) Successful in 5m30s
workflow hygiene / every workflow supersedes its own queue (pull_request) Successful in 5s
standards workspaces build + test / jouledesk-rs (pull_request) Successful in 2m45s
standards workspaces build + test / macOS energy backends (pull_request) Successful in 4m51s
Mesh's headline figure, microjoules per accepted token, is to be measured
at the meter. Yet every real serve through mesh was estimated: the GGUF
serving binaries were hardwired to EstimatorMeter, and HwMeter bridges
the host meter, which on macOS 27 is the estimator (IOReport's CPU rails
are frozen).
- mesh-meter `ProcessMeter`, behind `hw-meter`, for an engine that runs
in this process. Each serve is metered by jouleclaw's ProcessSpanMeter:
this process's own CPU energy plus its own GPU energy.
- A span that covers the whole request is graded as its counters
grade it: HwShunt for a live reading. Whole means the process was
effectively the GPU's sole user and used no Neural Engine; a
process with no GPU time spent exactly 0 J on it, so CPU-only
engines qualify.
- Any other span is an estimate of the whole request, Estimator: the
reading plus EstimatorMeter for the rest of the host, as HwMeter
bills a counter that reads part of a request. `mesh/1` has no
coverage field.
- `Meter::open_span` and `Window`. The default is None, so existing
meters keep snapshot semantics. ProcessMeter grades over the whole
span, because its GPU grade depends on the process's share over it.
Session uses a span window where a meter keeps one.
- mesh-engine-gguf, with `metal`: serve_falcon and serve_openai_falcon
meter with ProcessMeter (the Metal liveness load comes from
jouleclaw-backend-apple). New `serve_gemma_metered` serves Gemma 4
through mesh with ProcessMeter.
- Fixes mesh-engine-gguf under `gguf`: it no longer compiled on main.
ServeOutcome and ServeRequest gained a `decision` field and three
initializers lacked it, unseen because CI never built the feature.
standards-build now runs `cargo test -p mesh-engine-gguf --features
gguf` on the mesh-rs leg.
Measured on an M5 Max:
- mesh-core tests/process_meter.rs: a ReferenceEngine serve sealed
HwShunt 1,051,668 µJ (4,108 µJ per accepted token), within the
1,090,946 µJ of this process's CPU energy the test read around the
serve.
- serve_gemma_metered, Gemma 4 E2B on CPU, 3 x 16 tokens: HwShunt
12.52 J, 12.16 J and 12.54 J (0.78, 0.76 and 0.78 J per accepted
token). Every envelope verifies.
Mutation caught: billing a whole span's figure plus the rest-of-host
estimate fails the unit test and the end-to-end bracket ("the receipt
read 55691805 µJ, more than this process's 966177 µJ").
Co-Authored-By: Claude Opus 5.5 <[email protected]>
67 lines
2.5 KiB
TOML
67 lines
2.5 KiB
TOML
[package]
|
|
name = "mesh-engine-gguf"
|
|
version.workspace = true
|
|
edition.workspace = true
|
|
license.workspace = true
|
|
repository.workspace = true
|
|
rust-version.workspace = true
|
|
description = "Mesh — a real serving backend: wraps the family's pure-Rust GGUF/Metal inference substrate as a ServingEngine, so the mesh routes and meters actual model tokens. Behind the `gguf` feature so the default workspace build stays light."
|
|
|
|
[dependencies]
|
|
mesh-schema = { workspace = true }
|
|
mesh-engine = { workspace = true }
|
|
jouleclaw-loader-gguf = { workspace = true, optional = true }
|
|
thiserror = { workspace = true }
|
|
# Used only by the `serve_falcon` bin (required-features = ["gguf"]).
|
|
mesh-core = { workspace = true, optional = true }
|
|
mesh-meter = { workspace = true, optional = true }
|
|
mesh-router = { workspace = true, optional = true }
|
|
jcp-schema = { workspace = true, optional = true }
|
|
jcp-grant = { workspace = true, optional = true }
|
|
ed25519-dalek = { workspace = true, optional = true }
|
|
serde_json = { workspace = true, optional = true }
|
|
mesh-openai = { path = "../mesh-openai", optional = true }
|
|
# The GPU load for `ProcessMeter`'s liveness probe, with the `metal` feature.
|
|
jouleclaw-backend-apple = { workspace = true, optional = true }
|
|
|
|
[features]
|
|
default = []
|
|
# Pull the GGUF/safetensors loader and compile the real engines. Off by default:
|
|
# the loader is a heavier dependency (mmap, tokenizer, per-arch decode), and the
|
|
# default `cargo test --workspace` should not need model weights on disk.
|
|
gguf = [
|
|
"dep:jouleclaw-loader-gguf",
|
|
"dep:mesh-core",
|
|
"dep:mesh-meter",
|
|
"dep:mesh-router",
|
|
"dep:jcp-schema",
|
|
"dep:jcp-grant",
|
|
"dep:ed25519-dalek",
|
|
"dep:serde_json",
|
|
"dep:mesh-openai",
|
|
"mesh-openai/server",
|
|
]
|
|
# Apple Silicon UMA Metal overlay (zero-copy GGUF → GPU). Forwarded to the loader.
|
|
# Decode on the GPU, in process, and meter each serve by this process's own CPU
|
|
# and GPU energy (`mesh_meter::ProcessMeter`) rather than the load estimator.
|
|
metal = ["gguf", "jouleclaw-loader-gguf/metal", "mesh-meter/hw-meter-apple", "dep:jouleclaw-backend-apple"]
|
|
# Data-parallel matmul in the loader (faster CPU decode).
|
|
rayon = ["gguf", "jouleclaw-loader-gguf/rayon"]
|
|
|
|
[[bin]]
|
|
name = "serve_falcon"
|
|
path = "src/bin/serve_falcon.rs"
|
|
required-features = ["gguf"]
|
|
|
|
[[bin]]
|
|
name = "serve_openai_falcon"
|
|
path = "src/bin/serve_openai_falcon.rs"
|
|
required-features = ["gguf"]
|
|
|
|
[[bin]]
|
|
name = "serve_gemma_metered"
|
|
path = "src/bin/serve_gemma_metered.rs"
|
|
required-features = ["metal"]
|
|
|
|
[lints]
|
|
workspace = true
|