77f480c064
Adds JouleclawRuntimeBackend to examples/l3-jouleclaw: a third LlmBackend that
runs a GGUF model in-process on jouleclaw-runtime (the family's own kernel-graph
tensor engine over jouleclaw-loader-gguf + the reference/Apple backends) — no
llama.cpp, no FFI. It drops into the same FnStage(L3) slot as EchoBackend and
LlamaCompletionBackend. The run binary now defaults to this in-process engine
(JOULE_CODE_L3_ENGINE=llama selects the shell-out path); host energy is metered
over the call exactly as before.
Verified live against ternary-bonsai-1.7b-q2_0 (qwen3, Q2_0): the runtime loads
the real GGUF, runs the forward pass on its own reference kernels, and the Joule
Code cascade closes at L3 with a content-addressed CodingOp
(b3:Rtqw6jFwDKCtrvplkbLwmKUtXh3xeHwGsaaRtaBnHJM). tests/runtime_l3.rs (opt-in
via --ignored) covers both the bare backend and the full cascade path.
So the L3 tier now has a fully in-house local path end to end. Only JouleClaw's
own heavy GGUF tensor runtime *internals* remain its in-dev concern; as a Joule
Code L3 backend it works today. README + CHANGELOG updated; example lockfiles
tracked (incl. the transport-moqt lock missed in 52d929e).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
60 lines
2.7 KiB
Markdown
60 lines
2.7 KiB
Markdown
# L3 binding — JouleClaw model tier → Joule Code cascade
|
|
|
|
A consumer **binding example**: how to wire JouleClaw's **L3 (Model)** tier into
|
|
the Joule Code cascade. It is a standalone workspace, so the standard's reference
|
|
library keeps no JouleClaw dependency.
|
|
|
|
A coding request that misses the cheaper tiers (L0 cache, L1 lawful, L2 embed) is
|
|
generated by a local model at L3, producing a content-addressed `CodingOp` and a
|
|
joules-class `Receipt`.
|
|
|
|
## Three backends, same slot
|
|
|
|
All implement JouleClaw's `jouleclaw_llm_cheap::LlmBackend`, so they drop into
|
|
the same cascade stage:
|
|
|
|
- **`EchoBackend`** (from JouleClaw) — deterministic, model-free; used by the
|
|
unit tests so the wiring is verifiable with no model present.
|
|
- **`JouleclawRuntimeBackend`** (here) — runs the model **in-process on
|
|
JouleClaw's own from-scratch GGUF tensor runtime** (`jouleclaw-runtime`): no
|
|
external binary, no FFI. Tokenize → build the decode-step graph → compile
|
|
against the registered kernels (`jouleclaw-backend-reference` /
|
|
`-apple`) → autoregressive decode → detokenize, all in Rust the family wrote.
|
|
This is the **default** backend for the `run` binary.
|
|
- **`LlamaCompletionBackend`** (here) — drives a real on-device GGUF via
|
|
`llama-completion` (llama.cpp). Set `JOULE_CODE_L3_ENGINE=llama` to use it.
|
|
|
|
Either local path: no datacenter, no network. In production the cheapest capable
|
|
backend is chosen by `jouleclaw_dispatch::ModelBank`'s cost table.
|
|
|
|
## Run it against a local model
|
|
|
|
```bash
|
|
# Default: in-process on jouleclaw-runtime (the from-scratch tensor engine).
|
|
cargo run --release --bin run -- /path/to/model.gguf "Write a Rust function that reverses a string."
|
|
# or: JOULE_CODE_L3_MODEL=/path/to/model.gguf cargo run --release --bin run
|
|
|
|
# Use the llama.cpp shell-out backend instead:
|
|
JOULE_CODE_L3_ENGINE=llama cargo run --release --bin run -- /path/to/model.gguf "..."
|
|
```
|
|
|
|
The `run` binary generates with the local model, content-addresses the result as
|
|
a `CodingOp` (tier L3), and **meters host energy over the inference** with
|
|
JouleClaw's energy counter — so the receipt's joule cost is a real local
|
|
measurement, honestly tagged with the counter's provenance (`HwShunt` on
|
|
RAPL/NVML hosts; `Estimator` on Apple Silicon until IOReport is wired), not a
|
|
datacenter token estimate.
|
|
|
|
Verified live: the llama.cpp path against a local 350M GGUF, and the in-process
|
|
from-scratch runtime against `ternary-bonsai-1.7b-q2_0` (qwen3, Q2_0) — real text
|
|
generated, a real `op_id`, the cascade closing at L3.
|
|
|
|
## Tests
|
|
|
|
```bash
|
|
cargo test # EchoBackend path — no model required
|
|
|
|
# In-process from-scratch runtime against a real GGUF (opt-in):
|
|
JOULE_BONSAI=/path/to/ternary-bonsai-1.7b-q2_0.gguf \
|
|
cargo test --release --test runtime_l3 -- --ignored --nocapture
|
|
```
|