joule_core: LFM2 on-device, semantic code graph, and codebase-health analysis #1

Open
dcharlot wants to merge 0 commits from joule-semantic-graph-fusion-forge into main
Owner

Six commits across three threads of work. All deterministic/on-device; no cloud dependency added.

1. Semantic code graph + embedding-fused auto-context

  • code_graph — a queryable symbol/reference map (where is X defined, who calls it): the LogicLens-style navigation that ranking (repo_map) and retrieval don't provide. Control method: symbol.
  • Embedding signal fused into auto-context — the L2 index was built in the GUI but never queried at runtime. run_agent_loop now takes a retrieve_semantic closure; the GUI embeds the intent and fuses cosine-nearest chunks with the lexical + structural hits. The daemon passes a no-op and degrades cleanly.

2. Run Liquid AI LFM2 on-device

generate() decodes via a step-graph builder that assumes every block is attention, so LFM2 — which pairs gated short-convolution blocks with a few GQA blocks — died on the first conv layer hunting blk.0.attn_q.weight. Routed the L3 GGUF backend through Conversation instead, which builds the per-layer-aware graph (wiring the recurrent conv state) and caches the compiled step across tokens — so LFM2 runs and every arch decodes faster.

Also made on-device instruct models usable for structured output: prompts are wrapped in the model's declared chat template (detected from GGUF metadata) and stopped at the turn-end marker the configured eos often omits (FIM stays raw); l3_generate_budgeted gives a decomposition plan room to finish as one JSON object; parse_plan errors now carry the model's actual output.

Verified: LFM2-1.2B/2.6B load, generate coherent text, and run the full agent path (~12–18 tok/s release).

3. Codebase-health analysis (CodeFlow port)

New joule_core::codehealth — a Rust port of the deterministic analysis core of CodeFlow (MIT), rebuilt on code_graph rather than re-parsing: every symbol's definers + call sites already are the file→file dependency graph these metrics run over. Energy-free (graph walks + string scans, no model, no per-language parser).

  • blast_radius"if I change this file, what breaks?" (direct + depth-decayed transitive dependents)
  • circular_dependencies — strongly-connected components (iterative Tarjan; naive path enumeration was exponential and timed out on the real graph)
  • coupling, security_scan, health (A–F grade)
  • clone_groups — structural fingerprint + LCS-confirmed duplicate detection
  • cyclomatic_complexity — ranked hotspots
  • suggestions — prioritized, actionable recommendations synthesized from the findings

Control methods: health, clones, complexity.

Precision work, because a tool that cries wolf gets ignored: only single-definer symbols create dependency edges (generic names like new/run otherwise wire every file to every other — cut joule_core's edges 2553→1118); the secret rule requires the secret word to be the assigned identifier; dead-code excludes public API (a pub symbol's callers live outside the tree — 608→359); and test detection reads content, since Rust keeps tests inline in #[cfg(test)] and a path-only check reported 0% coverage for a fully-tested crate.

Found real duplication on first run — in CodeFlow's own card/lib (receipt-md.js vs receipt.js, 100%) and in joule_core itself (sandbox_bridge vs mcp_client, 98%).

Verification

joule_core suite green (176 passed), daemon + GUI panel build clean.

Release Notes:

  • Added on-device support for Liquid AI's LFM2 models, a symbol-navigation code graph, and codebase-health analysis (blast radius, duplicate detection, complexity, and an A–F grade); auto-context retrieval now fuses the embedding signal with lexical and structural search.
Six commits across three threads of work. All deterministic/on-device; no cloud dependency added. ## 1. Semantic code graph + embedding-fused auto-context - **`code_graph`** — a queryable symbol/reference map (where is `X` defined, who calls it): the LogicLens-style navigation that ranking (`repo_map`) and retrieval don't provide. Control method: `symbol`. - **Embedding signal fused into auto-context** — the L2 index was built in the GUI but never queried at runtime. `run_agent_loop` now takes a `retrieve_semantic` closure; the GUI embeds the intent and fuses cosine-nearest chunks with the lexical + structural hits. The daemon passes a no-op and degrades cleanly. ## 2. Run Liquid AI LFM2 on-device `generate()` decodes via a step-graph builder that assumes **every block is attention**, so LFM2 — which pairs gated short-convolution blocks with a few GQA blocks — died on the first conv layer hunting `blk.0.attn_q.weight`. Routed the L3 GGUF backend through `Conversation` instead, which builds the per-layer-aware graph (wiring the recurrent conv state) **and** caches the compiled step across tokens — so LFM2 runs and every arch decodes faster. Also made on-device instruct models usable for structured output: prompts are wrapped in the model's declared chat template (detected from GGUF metadata) and stopped at the turn-end marker the configured `eos` often omits (FIM stays raw); `l3_generate_budgeted` gives a decomposition plan room to finish as one JSON object; `parse_plan` errors now carry the model's actual output. Verified: LFM2-1.2B/2.6B load, generate coherent text, and run the full agent path (~12–18 tok/s release). ## 3. Codebase-health analysis (CodeFlow port) New **`joule_core::codehealth`** — a Rust port of the deterministic analysis core of [CodeFlow](https://github.com/braedonsaunders/codeflow) (MIT), rebuilt on `code_graph` rather than re-parsing: every symbol's definers + call sites already *are* the file→file dependency graph these metrics run over. Energy-free (graph walks + string scans, no model, no per-language parser). - **`blast_radius`** — *"if I change this file, what breaks?"* (direct + depth-decayed transitive dependents) - **`circular_dependencies`** — strongly-connected components (iterative Tarjan; naive path enumeration was exponential and timed out on the real graph) - **`coupling`**, **`security_scan`**, **`health`** (A–F grade) - **`clone_groups`** — structural fingerprint + LCS-confirmed duplicate detection - **`cyclomatic_complexity`** — ranked hotspots - **`suggestions`** — prioritized, actionable recommendations synthesized from the findings Control methods: `health`, `clones`, `complexity`. **Precision work, because a tool that cries wolf gets ignored:** only single-definer symbols create dependency edges (generic names like `new`/`run` otherwise wire every file to every other — cut joule_core's edges 2553→1118); the secret rule requires the secret word to be the *assigned identifier*; dead-code excludes **public API** (a `pub` symbol's callers live outside the tree — 608→359); and test detection reads content, since Rust keeps tests inline in `#[cfg(test)]` and a path-only check reported 0% coverage for a fully-tested crate. Found real duplication on first run — in CodeFlow's own `card/lib` (`receipt-md.js` vs `receipt.js`, 100%) and in `joule_core` itself (`sandbox_bridge` vs `mcp_client`, 98%). ## Verification `joule_core` suite green (**176 passed**), daemon + GUI panel build clean. Release Notes: - Added on-device support for Liquid AI's LFM2 models, a symbol-navigation code graph, and codebase-health analysis (blast radius, duplicate detection, complexity, and an A–F grade); auto-context retrieval now fuses the embedding signal with lexical and structural search.
Add code_graph, a queryable symbol/reference map over a repo: where a
symbol is defined and who calls it. This is the LogicLens-style repo
understanding the July SOTA scan flagged as the last open gap -- it
complements repo_map (ranks files by PageRank) and retrieval (fuses search
signals) by answering precise navigation questions neither does.

Built from the same plain-tokenizer symbol extraction repo_map already uses
(no per-language parser -- best-effort, deterministic, energy-free). Exposed
over the control protocol as a `symbol` method: `{"symbol":"name"}` returns
its definition + call sites; no symbol returns the graph's symbol count.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
joule_core: fuse the semantic embedding signal into auto-context
Some checks failed
deploy_docs.yml / joule_core: fuse the semantic embedding signal into auto-context (push) Failing after 0s
extension_bump.yml / joule_core: fuse the semantic embedding signal into auto-context (push) Failing after 0s
deploy_docs.yml / joule_core: fuse the semantic embedding signal into auto-context (pull_request) Failing after 0s
extension_bump.yml / joule_core: fuse the semantic embedding signal into auto-context (pull_request) Failing after 0s
Assign Reviewers / assign-reviewers (pull_request_target) Has been cancelled
PR Issue Labeler / check-authorship-and-label (pull_request_target) Has been cancelled
danger / danger (pull_request) Has been cancelled
run_tests / orchestrate (pull_request) Has been cancelled
run_tests / check_style (pull_request) Has been cancelled
run_tests / clippy_windows (pull_request) Has been cancelled
run_tests / clippy_linux (pull_request) Has been cancelled
run_tests / clippy_mac (pull_request) Has been cancelled
run_tests / clippy_mac_x86_64 (pull_request) Has been cancelled
run_tests / run_tests_windows (pull_request) Has been cancelled
run_tests / run_tests_linux (pull_request) Has been cancelled
run_tests / run_tests_mac (pull_request) Has been cancelled
run_tests / miri_scheduler (pull_request) Has been cancelled
run_tests / doctests (pull_request) Has been cancelled
run_tests / check_workspace_binaries (pull_request) Has been cancelled
run_tests / build_visual_tests_binary (pull_request) Has been cancelled
run_tests / check_wasm (pull_request) Has been cancelled
run_tests / check_dependencies (pull_request) Has been cancelled
run_tests / check_docs (pull_request) Has been cancelled
run_tests / check_licenses (pull_request) Has been cancelled
run_tests / check_scripts (pull_request) Has been cancelled
run_tests / check_postgres_and_protobuf_migrations (pull_request) Has been cancelled
run_tests / tests_pass (pull_request) Has been cancelled
4d08df595f
The L2 embedding index (CodebaseIndex::search) was built in the GUI but
never queried at runtime -- auto-context fused only the lexical + structural
signals. Thread the embedding signal in so retrieval is truly hybrid: a file
all three signals agree on now ranks highest.

- retrieval::semantic_candidates maps a cosine search into chunk-precise
  fusion candidates (caller owns the model, keeping retrieval model-free).
- run_agent_loop takes a `retrieve_semantic` closure; its auto-context fuses
  the returned candidates. `|_| Vec::new()` opts out and degrades cleanly to
  lexical + structural (the headless/daemon path, which builds no index).
- The GUI passes a real closure: embed the intent with the same on-device
  model, search the built index, fuse the nearest chunks. One embed per run.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Route the L3 GGUF backend through the runtime's `Conversation` API instead of
the one-shot `generate()`. `generate()` decodes via a step-graph builder that
assumes every block is attention, so LFM2 — which pairs gated short-convolution
blocks with a few GQA blocks — died on the first conv layer hunting for
`blk.0.attn_q.weight`. `Conversation` builds the per-layer-aware graph (wiring
the recurrent conv state) and caches the compiled step across tokens, so LFM2
runs and every arch decodes faster (first-token graph build amortized).

Also make on-device instruct models usable for structured output:
- Wrap prompts in the model's declared chat template (detected from GGUF
  metadata) and stop at the turn-end marker the configured eos often omits; FIM
  infill stays raw. Fed the bare prompt, an instruct model just continues text.
- `l3_generate_budgeted` so a decomposition plan gets room to finish as one JSON
  object rather than truncating mid-object against the single-edit default.
- Decomposition prompt uses a concrete filled example, not `<placeholder>`
  slots — small models echo angle brackets into `<...>` token soup.
- parse_plan errors carry the model's actual output, so a plan that came back as
  prose or truncated is debuggable rather than an opaque "no JSON found".

Verified: LFM2-1.2B/2.6B load, generate coherent text, and run the full agent
path in joule-coded (release, ~12-18 tok/s). Strict-JSON AOrchestra decomposition
still exceeds these small dense LFM2s' structured-output ability (they ramble/
degenerate); a real OrchestrationReport needs a more capable model.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add joule_core::codehealth — a deterministic, energy-free codebase-health layer
built on the existing symbol graph. A Rust port of the analysis core of CodeFlow
(github.com/braedonsaunders/codeflow, MIT), rebuilt on code_graph rather than
re-parsing: every symbol's definers + call sites already ARE the file->file
dependency graph these metrics run over.

- blast_radius: "if I change this file, what breaks?" — direct + depth-decayed
  transitive dependents (reverse reachability), the flagship query.
- circular_dependencies: strongly-connected components (iterative Tarjan, linear)
  — an SCC of >=2 files is a dependency cycle.
- coupling: fan-in/fan-out, average, and the high-fan-in hubs.
- security_scan: string rules (no regex dep) for hardcoded secrets, string-built
  SQL, eval/dynamic exec, shell exec, and left-in debug logging.
- health: the CodeFlow 100-minus-capped-penalties formula -> an A-F grade.
- analyze(): whole-repo report; exposed over the control protocol as `health`
  ({"file":"path"} returns just that file's blast radius).

Precision: only symbols with a SINGLE definer produce edges — the token-based
graph can't attribute a name defined in many files (`new`, `run`, `build`) to
one place, and counting those wires every file to every other. The secret rule
requires the secret word to be the assigned identifier (not any mention) with a
long literal, so a codebase that legitimately deals in tokens/auth isn't drowned
in false positives.

Verified on two real repos: joule_core (36 files) and CodeFlow's own card/lib
(27 files, coupling avg 5.1). 5 unit tests; full joule_core suite green (171).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extend joule_core::codehealth with the rest of CodeFlow's quality analysis:

- code_fingerprint: structural signature (identifiers/strings/numbers erased,
  summarized as L<loops>C<conds>F<calls>R<returns>S<len>) for grouping clones.
- code_similarity / lcs_length: normalized longest-common-subsequence ratio to
  confirm a fingerprint match is a real clone (capped at 500 chars, cheap).
- clone_groups: extract function-sized brace-balanced blocks across the repo,
  group by fingerprint, confirm each group by minimum pairwise similarity (>=85%).
- cyclomatic_complexity / complexity_level: 1 + control-flow branch points,
  bucketed low/medium/high/critical.

Exposed over the control protocol as `clones` and `complexity` (the latter
ranks per-file hotspots, or scores one file with {"file":"path"}).

Found real duplication on first run: in CodeFlow's own card/lib (receipt-md.js
vs receipt.js, 100%) and in joule_core itself (sandbox_bridge vs mcp_client 98%,
an intra-file dup in sessions.rs 92%). 8 codehealth tests; suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
joule_core: actionable suggestions + honest dead-code and test detection
Some checks failed
extension_bump.yml / joule_core: actionable suggestions + honest dead-code and test detection (pull_request) Failing after 0s
danger / danger (pull_request) Has been cancelled
nix_build / build_nix_linux_x86_64 (pull_request) Has been cancelled
nix_build / build_nix_mac_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_linux_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_linux_x86_64 (pull_request) Has been cancelled
run_bundling / bundle_mac_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_mac_x86_64 (pull_request) Has been cancelled
run_bundling / bundle_windows_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_windows_x86_64 (pull_request) Has been cancelled
run_tests / orchestrate (pull_request) Has been cancelled
run_tests / check_style (pull_request) Has been cancelled
run_tests / clippy_windows (pull_request) Has been cancelled
run_tests / clippy_linux (pull_request) Has been cancelled
run_tests / clippy_mac (pull_request) Has been cancelled
run_tests / clippy_mac_x86_64 (pull_request) Has been cancelled
run_tests / run_tests_windows (pull_request) Has been cancelled
run_tests / run_tests_linux (pull_request) Has been cancelled
run_tests / run_tests_mac (pull_request) Has been cancelled
run_tests / miri_scheduler (pull_request) Has been cancelled
run_tests / doctests (pull_request) Has been cancelled
run_tests / check_workspace_binaries (pull_request) Has been cancelled
run_tests / build_visual_tests_binary (pull_request) Has been cancelled
run_tests / check_wasm (pull_request) Has been cancelled
run_tests / check_dependencies (pull_request) Has been cancelled
run_tests / check_docs (pull_request) Has been cancelled
run_tests / check_licenses (pull_request) Has been cancelled
run_tests / check_scripts (pull_request) Has been cancelled
run_tests / check_postgres_and_protobuf_migrations (pull_request) Has been cancelled
run_tests / tests_pass (pull_request) Has been cancelled
f4182ddca9
Complete the CodeFlow port with the layer that makes the analysis actionable, and
fix two false signals the first run exposed.

- suggestions(): synthesize prioritized recommendations from the findings
  (security, cycles, clones, dead code, god objects, coupling, test coverage),
  sorted critical -> low. Folded into HealthReport alongside clones, dead_symbols
  and god_objects, so `health` returns one comprehensive artifact.

Precision, because a tool that cries wolf gets ignored:
- Dead code no longer counts PUBLIC API. A `pub`/`export` symbol with no call site
  inside the analyzed tree is used by consumers of the crate, not dead — counting
  it made every library look rotten (joule_core reported 608 dead; now 359).
- Test detection reads content, not just paths. Rust keeps tests inline in
  `#[cfg(test)] mod tests`, so a path-only check reported 0% coverage for a crate
  with a full passing suite and emitted a bogus "add tests" suggestion. Gone.

10 codehealth tests (incl. regression tests for both fixes); suite green (176).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
dcharlot changed title from joule_core: semantic code graph and embedding-fused auto-context to joule_core: LFM2 on-device, semantic code graph, and codebase-health analysis 2026-07-12 14:30:25 -04:00
joule_core: layer-violation + pattern detection, and data-file precision
Some checks failed
extension_bump.yml / joule_core: layer-violation + pattern detection, and data-file precision (pull_request) Failing after 0s
danger / danger (pull_request) Has been cancelled
nix_build / build_nix_linux_x86_64 (pull_request) Has been cancelled
nix_build / build_nix_mac_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_linux_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_linux_x86_64 (pull_request) Has been cancelled
run_bundling / bundle_mac_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_mac_x86_64 (pull_request) Has been cancelled
run_bundling / bundle_windows_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_windows_x86_64 (pull_request) Has been cancelled
run_tests / orchestrate (pull_request) Has been cancelled
run_tests / check_style (pull_request) Has been cancelled
run_tests / clippy_windows (pull_request) Has been cancelled
run_tests / clippy_linux (pull_request) Has been cancelled
run_tests / clippy_mac (pull_request) Has been cancelled
run_tests / clippy_mac_x86_64 (pull_request) Has been cancelled
run_tests / run_tests_windows (pull_request) Has been cancelled
run_tests / run_tests_linux (pull_request) Has been cancelled
run_tests / run_tests_mac (pull_request) Has been cancelled
run_tests / miri_scheduler (pull_request) Has been cancelled
run_tests / doctests (pull_request) Has been cancelled
run_tests / check_workspace_binaries (pull_request) Has been cancelled
run_tests / build_visual_tests_binary (pull_request) Has been cancelled
run_tests / check_wasm (pull_request) Has been cancelled
run_tests / check_dependencies (pull_request) Has been cancelled
run_tests / check_docs (pull_request) Has been cancelled
run_tests / check_licenses (pull_request) Has been cancelled
run_tests / check_scripts (pull_request) Has been cancelled
run_tests / check_postgres_and_protobuf_migrations (pull_request) Has been cancelled
run_tests / tests_pass (pull_request) Has been cancelled
7f61909d80
Finish the CodeFlow port with the last two analyses, plus a precision pass that
keeps them from crying wolf.

- layer_violations: architecture conformance — a file in a more foundational layer
  (utils/data) that depends on a less foundational one (ui/services) is reaching up
  the stack. layer_of classifies by path convention and returns None for the
  unclassifiable (a flat crate has no layers to violate), unlike CodeFlow which
  defaults everything to `utils` and invents violations. Semantics fixed too:
  CodeFlow's own suggestion text contradicts its data model; this uses the importer
  = connection target throughout. Feeds a "Fix Architecture Violations" suggestion.
- detect_patterns: the language-agnostic subset of CodeFlow's detector — Singleton,
  Factory, Observer/Event, and the Long File anti-pattern — made Rust-aware
  (OnceCell/OnceLock/lazy_static, mpsc). Its framework-specific rules (React hooks,
  Django, VBA) are dropped. Control method `patterns`.

Precision, since a graph is token-based (no import resolution):
- Data/markup files (.json, .yaml, .md, .lock, ...) are excluded from the
  dependency graph and symbol counts — a JSON key matching a symbol name is not an
  import (was inventing "data imports UI" violations; pattern-lang-web 86 -> 26).
- A layer violation now requires the pair to share >=2 distinct symbols — one
  coincidental name match isn't a cross-layer dependency (26 -> 10). The residual
  floor is inherent to token analysis; real import parsing is a separate effort.

12 codehealth tests; suite green (178).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
joule_core: BM25 lexical ranking + RRF top-rank bonus (qmd ingestion)
Some checks failed
extension_bump.yml / joule_core: BM25 lexical ranking + RRF top-rank bonus (qmd ingestion) (pull_request) Failing after 0s
danger / danger (pull_request) Has been cancelled
nix_build / build_nix_linux_x86_64 (pull_request) Has been cancelled
nix_build / build_nix_mac_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_linux_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_linux_x86_64 (pull_request) Has been cancelled
run_bundling / bundle_mac_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_mac_x86_64 (pull_request) Has been cancelled
run_bundling / bundle_windows_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_windows_x86_64 (pull_request) Has been cancelled
run_tests / orchestrate (pull_request) Has been cancelled
run_tests / check_style (pull_request) Has been cancelled
run_tests / clippy_windows (pull_request) Has been cancelled
run_tests / clippy_linux (pull_request) Has been cancelled
run_tests / clippy_mac (pull_request) Has been cancelled
run_tests / clippy_mac_x86_64 (pull_request) Has been cancelled
run_tests / run_tests_windows (pull_request) Has been cancelled
run_tests / run_tests_linux (pull_request) Has been cancelled
run_tests / run_tests_mac (pull_request) Has been cancelled
run_tests / miri_scheduler (pull_request) Has been cancelled
run_tests / doctests (pull_request) Has been cancelled
run_tests / check_workspace_binaries (pull_request) Has been cancelled
run_tests / build_visual_tests_binary (pull_request) Has been cancelled
run_tests / check_wasm (pull_request) Has been cancelled
run_tests / check_dependencies (pull_request) Has been cancelled
run_tests / check_docs (pull_request) Has been cancelled
run_tests / check_licenses (pull_request) Has been cancelled
run_tests / check_scripts (pull_request) Has been cancelled
run_tests / check_postgres_and_protobuf_migrations (pull_request) Has been cancelled
run_tests / tests_pass (pull_request) Has been cancelled
f57ec9597a
Ingest the two ranking ideas agent-joule's retrieval was missing, from qmd
(tobi/qmd, an on-device markdown search engine: BM25 + vector + HyDE fused by RRF
then reranked).

- bm25_candidates: Okapi BM25 (k1=1.5, b=0.75) as the LEXICAL signal, replacing
  the raw grep hit count. TF-IDF with document-length normalization — a rare term
  in a short file now outranks a common term buried in a long one, which a hit
  count could not distinguish. Deterministic, energy-free (no model, no index);
  qmd gets this from SQLite FTS5's bm25(), rebuilt here over the file walk.
- fuse(): a top-rank bonus (from qmd's RRF) — a file that placed first in ANY
  single signal gets +0.05 (+0.02 for near-top). Additive, so it only tips
  otherwise-comparable scores toward a strong single-signal leader.
- retrieve() now fuses BM25 + semantic + structural.

2 new retrieval tests (BM25 length-normalization; the top-rank bonus); full
joule_core suite green (180).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
joule_core: HyDE hypothetical-document retrieval (qmd ingestion)
Some checks failed
extension_bump.yml / joule_core: HyDE hypothetical-document retrieval (qmd ingestion) (pull_request) Failing after 0s
danger / danger (pull_request) Has been cancelled
nix_build / build_nix_linux_x86_64 (pull_request) Has been cancelled
nix_build / build_nix_mac_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_linux_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_linux_x86_64 (pull_request) Has been cancelled
run_bundling / bundle_mac_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_mac_x86_64 (pull_request) Has been cancelled
run_bundling / bundle_windows_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_windows_x86_64 (pull_request) Has been cancelled
run_tests / orchestrate (pull_request) Has been cancelled
run_tests / check_style (pull_request) Has been cancelled
run_tests / clippy_windows (pull_request) Has been cancelled
run_tests / clippy_linux (pull_request) Has been cancelled
run_tests / clippy_mac (pull_request) Has been cancelled
run_tests / clippy_mac_x86_64 (pull_request) Has been cancelled
run_tests / run_tests_windows (pull_request) Has been cancelled
run_tests / run_tests_linux (pull_request) Has been cancelled
run_tests / run_tests_mac (pull_request) Has been cancelled
run_tests / miri_scheduler (pull_request) Has been cancelled
run_tests / doctests (pull_request) Has been cancelled
run_tests / check_workspace_binaries (pull_request) Has been cancelled
run_tests / build_visual_tests_binary (pull_request) Has been cancelled
run_tests / check_wasm (pull_request) Has been cancelled
run_tests / check_dependencies (pull_request) Has been cancelled
run_tests / check_docs (pull_request) Has been cancelled
run_tests / check_licenses (pull_request) Has been cancelled
run_tests / check_scripts (pull_request) Has been cancelled
run_tests / check_postgres_and_protobuf_migrations (pull_request) Has been cancelled
run_tests / tests_pass (pull_request) Has been cancelled
e64b7fcd11
Ingest qmd's `hyde` sub-query: instead of embedding the bare query, have the
on-device model write the code that would answer it and embed THAT. A question
("how is the cache invalidated?") and its implementation share few literal
tokens; a hypothetical answer is phrased in the code's own vocabulary, so its
embedding lands nearer the real implementation.

- retrieval::hyde_prompt(query): the model-free prompt (the caller runs the
  generation + embed, so joule_core stays model-free).
- The GUI's auto-context semantic closure uses it when JOULE_CODE_HYDE is set:
  generate a short hypothetical, embed that, fuse as the semantic signal. Off by
  default (it costs one generation); falls back to the raw query on empty/error.

Rounds out the qmd retrieval ingestion (BM25 lexical + RRF top-rank bonus + HyDE)
against the RRF fusion + L2 index agent-joule already had. 10 retrieval tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
joule_core: context tree on retrieval hits (qmd ingestion)
Some checks failed
extension_bump.yml / joule_core: context tree on retrieval hits (qmd ingestion) (pull_request) Failing after 0s
danger / danger (pull_request) Has been cancelled
nix_build / build_nix_linux_x86_64 (pull_request) Has been cancelled
nix_build / build_nix_mac_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_linux_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_linux_x86_64 (pull_request) Has been cancelled
run_bundling / bundle_mac_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_mac_x86_64 (pull_request) Has been cancelled
run_bundling / bundle_windows_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_windows_x86_64 (pull_request) Has been cancelled
run_tests / orchestrate (pull_request) Has been cancelled
run_tests / check_style (pull_request) Has been cancelled
run_tests / clippy_windows (pull_request) Has been cancelled
run_tests / clippy_linux (pull_request) Has been cancelled
run_tests / clippy_mac (pull_request) Has been cancelled
run_tests / clippy_mac_x86_64 (pull_request) Has been cancelled
run_tests / run_tests_windows (pull_request) Has been cancelled
run_tests / run_tests_linux (pull_request) Has been cancelled
run_tests / run_tests_mac (pull_request) Has been cancelled
run_tests / miri_scheduler (pull_request) Has been cancelled
run_tests / doctests (pull_request) Has been cancelled
run_tests / check_workspace_binaries (pull_request) Has been cancelled
run_tests / build_visual_tests_binary (pull_request) Has been cancelled
run_tests / check_wasm (pull_request) Has been cancelled
run_tests / check_dependencies (pull_request) Has been cancelled
run_tests / check_docs (pull_request) Has been cancelled
run_tests / check_licenses (pull_request) Has been cancelled
run_tests / check_scripts (pull_request) Has been cancelled
run_tests / check_postgres_and_protobuf_migrations (pull_request) Has been cancelled
run_tests / tests_pass (pull_request) Has been cancelled
a3ae5ce3e2
Ingest qmd's context tree — its self-described key feature: attach context to a
document's place in the tree so an LLM chooses better. Adapted for code, the
context is derived from the code itself, so there is nothing to configure.

- retrieval::file_context: a file's one-line purpose — its leading doc-comment
  header (`//!` / `///` / `#` / block comments), falling back to the nearest
  README title walking up the directory tree. Deterministic, model-free.
- render_hits_with_context: each retrieval hit now carries that context, so the
  agent sees WHY a file is relevant, not just where.
- Wired into local_agent auto-context, so every run's retrieved files arrive with
  their module's purpose.

Completes the qmd retrieval ingestion: BM25 + RRF top-rank bonus + HyDE + context
tree, layered onto the RRF fusion + L2 index agent-joule already had. 11 retrieval
tests; full joule_core suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Update to Rust 1.97 and latest dependencies, and wire cross-encoder reranking
Some checks failed
extension_bump.yml / Update to Rust 1.97 and latest dependencies, and wire cross-encoder reranking (pull_request) Failing after 0s
danger / danger (pull_request) Has been cancelled
nix_build / build_nix_linux_x86_64 (pull_request) Has been cancelled
nix_build / build_nix_mac_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_linux_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_linux_x86_64 (pull_request) Has been cancelled
run_bundling / bundle_mac_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_mac_x86_64 (pull_request) Has been cancelled
run_bundling / bundle_windows_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_windows_x86_64 (pull_request) Has been cancelled
run_tests / orchestrate (pull_request) Has been cancelled
run_tests / check_style (pull_request) Has been cancelled
run_tests / clippy_windows (pull_request) Has been cancelled
run_tests / clippy_linux (pull_request) Has been cancelled
run_tests / clippy_mac (pull_request) Has been cancelled
run_tests / clippy_mac_x86_64 (pull_request) Has been cancelled
run_tests / run_tests_windows (pull_request) Has been cancelled
run_tests / run_tests_linux (pull_request) Has been cancelled
run_tests / run_tests_mac (pull_request) Has been cancelled
run_tests / miri_scheduler (pull_request) Has been cancelled
run_tests / doctests (pull_request) Has been cancelled
run_tests / check_workspace_binaries (pull_request) Has been cancelled
run_tests / build_visual_tests_binary (pull_request) Has been cancelled
run_tests / check_wasm (pull_request) Has been cancelled
run_tests / check_dependencies (pull_request) Has been cancelled
run_tests / check_docs (pull_request) Has been cancelled
run_tests / check_licenses (pull_request) Has been cancelled
run_tests / check_scripts (pull_request) Has been cancelled
run_tests / check_postgres_and_protobuf_migrations (pull_request) Has been cancelled
run_tests / tests_pass (pull_request) Has been cancelled
4b69bc197b
Toolchain and dependencies:
- rust-toolchain.toml 1.96.0 -> 1.97.0 (current stable).
- cargo update across the workspace: latest semver-compatible versions.
- All open-standards git deps (jouleclaw-*, joule-code-*, sandbox-*, joule-moq)
  move together 320fa1e -> c4b536cf, and off github.com onto the sovereign forge
  (git.transaction.science) — the flagged org is 404 to anonymous clients, which
  breaks clean-container CI even though a warm dev cache hides it.

  c4b536cf is 320fa1e plus the additive score_binary commit, deliberately NOT the
  newer fix/omni-build branch: that line has dropped crates this workspace still
  depends on (jcp-a2a), so bumping onto it would mean reconciling against a
  restructured open-standards rather than taking a runtime addition.

Cross-encoder reranking (the last stage of qmd's pipeline):
- l3::l3_rerank_score — relevance of a document to a query in [0,1], read as
  P(yes) after ONE forward pass over a yes/no reranker prompt. Resolves the
  yes/no ids by exact token lookup and encodes the chat markers atomically; the
  prompt omits the reasoning <think> block, which derails the 0.6B reranker.
- retrieval::rerank — reorders fused hits by an injected scorer (model-free
  here); an unscored hit keeps its fused rank rather than mis-ordering.
- `search` control method: fuse BM25 + structural, then rerank the top with the
  loaded model ({"rerank": false} to skip).

Fusion alone ranks cascade.rs first for "invalidate the embedding cache"; with
Qwen3-Reranker-0.6B the model re-reads each candidate against the query and
demotes fabric.rs (compute fabric, not caches) from 3rd to last. 182 joule_core
tests green on 1.97; joule_coded builds release.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Some checks failed
deploy_docs.yml / Update to Rust 1.97 and latest dependencies, and wire cross-encoder reranking (push) Failing after 0s
extension_bump.yml / Update to Rust 1.97 and latest dependencies, and wire cross-encoder reranking (push) Failing after 0s
deploy_docs.yml / Update to Rust 1.97 and latest dependencies, and wire cross-encoder reranking (pull_request) Failing after 0s
extension_bump.yml / Update to Rust 1.97 and latest dependencies, and wire cross-encoder reranking (pull_request) Failing after 0s
danger / danger (pull_request) Has been cancelled
nix_build / build_nix_linux_x86_64 (pull_request) Has been cancelled
nix_build / build_nix_mac_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_linux_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_linux_x86_64 (pull_request) Has been cancelled
run_bundling / bundle_mac_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_mac_x86_64 (pull_request) Has been cancelled
run_bundling / bundle_windows_aarch64 (pull_request) Has been cancelled
run_bundling / bundle_windows_x86_64 (pull_request) Has been cancelled
run_tests / orchestrate (pull_request) Has been cancelled
run_tests / check_style (pull_request) Has been cancelled
run_tests / clippy_windows (pull_request) Has been cancelled
run_tests / clippy_linux (pull_request) Has been cancelled
run_tests / clippy_mac (pull_request) Has been cancelled
run_tests / clippy_mac_x86_64 (pull_request) Has been cancelled
run_tests / run_tests_windows (pull_request) Has been cancelled
run_tests / run_tests_linux (pull_request) Has been cancelled
run_tests / run_tests_mac (pull_request) Has been cancelled
run_tests / miri_scheduler (pull_request) Has been cancelled
run_tests / doctests (pull_request) Has been cancelled
run_tests / check_workspace_binaries (pull_request) Has been cancelled
run_tests / build_visual_tests_binary (pull_request) Has been cancelled
run_tests / check_wasm (pull_request) Has been cancelled
run_tests / check_dependencies (pull_request) Has been cancelled
run_tests / check_docs (pull_request) Has been cancelled
run_tests / check_licenses (pull_request) Has been cancelled
run_tests / check_scripts (pull_request) Has been cancelled
run_tests / check_postgres_and_protobuf_migrations (pull_request) Has been cancelled
run_tests / tests_pass (pull_request) Has been cancelled
This pull request has changes conflicting with the target branch.
  • Cargo.lock
  • crates/agent_joule_panel/Cargo.toml
  • crates/joule_coded/Cargo.toml
  • crates/joule_core/Cargo.toml
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin joule-semantic-graph-fusion-forge:joule-semantic-graph-fusion-forge
git switch joule-semantic-graph-fusion-forge

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main
git merge --no-ff joule-semantic-graph-fusion-forge
git switch joule-semantic-graph-fusion-forge
git rebase main
git switch main
git merge --ff-only joule-semantic-graph-fusion-forge
git switch joule-semantic-graph-fusion-forge
git rebase main
git switch main
git merge --no-ff joule-semantic-graph-fusion-forge
git switch main
git merge --squash joule-semantic-graph-fusion-forge
git switch main
git merge --ff-only joule-semantic-graph-fusion-forge
git switch main
git merge joule-semantic-graph-fusion-forge
git push origin main
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Transaction-Science/agent-joule!1
No description provided.