77f480c064
Adds JouleclawRuntimeBackend to examples/l3-jouleclaw: a third LlmBackend that
runs a GGUF model in-process on jouleclaw-runtime (the family's own kernel-graph
tensor engine over jouleclaw-loader-gguf + the reference/Apple backends) — no
llama.cpp, no FFI. It drops into the same FnStage(L3) slot as EchoBackend and
LlamaCompletionBackend. The run binary now defaults to this in-process engine
(JOULE_CODE_L3_ENGINE=llama selects the shell-out path); host energy is metered
over the call exactly as before.
Verified live against ternary-bonsai-1.7b-q2_0 (qwen3, Q2_0): the runtime loads
the real GGUF, runs the forward pass on its own reference kernels, and the Joule
Code cascade closes at L3 with a content-addressed CodingOp
(b3:Rtqw6jFwDKCtrvplkbLwmKUtXh3xeHwGsaaRtaBnHJM). tests/runtime_l3.rs (opt-in
via --ignored) covers both the bare backend and the full cascade path.
So the L3 tier now has a fully in-house local path end to end. Only JouleClaw's
own heavy GGUF tensor runtime *internals* remain its in-dev concern; as a Joule
Code L3 backend it works today. README + CHANGELOG updated; example lockfiles
tracked (incl. the transport-moqt lock missed in 52d929e).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
2.7 KiB
L3 binding — JouleClaw model tier → Joule Code cascade
A consumer binding example: how to wire JouleClaw's L3 (Model) tier into the Joule Code cascade. It is a standalone workspace, so the standard's reference library keeps no JouleClaw dependency.
A coding request that misses the cheaper tiers (L0 cache, L1 lawful, L2 embed) is
generated by a local model at L3, producing a content-addressed CodingOp and a
joules-class Receipt.
Three backends, same slot
All implement JouleClaw's jouleclaw_llm_cheap::LlmBackend, so they drop into
the same cascade stage:
EchoBackend(from JouleClaw) — deterministic, model-free; used by the unit tests so the wiring is verifiable with no model present.JouleclawRuntimeBackend(here) — runs the model in-process on JouleClaw's own from-scratch GGUF tensor runtime (jouleclaw-runtime): no external binary, no FFI. Tokenize → build the decode-step graph → compile against the registered kernels (jouleclaw-backend-reference/-apple) → autoregressive decode → detokenize, all in Rust the family wrote. This is the default backend for therunbinary.LlamaCompletionBackend(here) — drives a real on-device GGUF viallama-completion(llama.cpp). SetJOULE_CODE_L3_ENGINE=llamato use it.
Either local path: no datacenter, no network. In production the cheapest capable
backend is chosen by jouleclaw_dispatch::ModelBank's cost table.
Run it against a local model
# Default: in-process on jouleclaw-runtime (the from-scratch tensor engine).
cargo run --release --bin run -- /path/to/model.gguf "Write a Rust function that reverses a string."
# or: JOULE_CODE_L3_MODEL=/path/to/model.gguf cargo run --release --bin run
# Use the llama.cpp shell-out backend instead:
JOULE_CODE_L3_ENGINE=llama cargo run --release --bin run -- /path/to/model.gguf "..."
The run binary generates with the local model, content-addresses the result as
a CodingOp (tier L3), and meters host energy over the inference with
JouleClaw's energy counter — so the receipt's joule cost is a real local
measurement, honestly tagged with the counter's provenance (HwShunt on
RAPL/NVML hosts; Estimator on Apple Silicon until IOReport is wired), not a
datacenter token estimate.
Verified live: the llama.cpp path against a local 350M GGUF, and the in-process
from-scratch runtime against ternary-bonsai-1.7b-q2_0 (qwen3, Q2_0) — real text
generated, a real op_id, the cascade closing at L3.
Tests
cargo test # EchoBackend path — no model required
# In-process from-scratch runtime against a real GGUF (opt-in):
JOULE_BONSAI=/path/to/ternary-bonsai-1.7b-q2_0.gguf \
cargo test --release --test runtime_l3 -- --ignored --nocapture