Add score_binary / score_binary_tokens: prefill a prompt and return the softmax
probability of one token vs another as the next token — the mechanism of a
cross-encoder reranker (Qwen3-Reranker and friends format query+document into a
yes/no question and read P(yes)). One forward pass, no generation.
- Uses the lfm2-aware cached step builder (run_inplace_step_cached), so it scores
hybrid conv+attention models too (the plain run_inplace_step assumes every block
is attention and dies on a conv layer).
- score_binary_tokens takes pre-tokenized input so a reranker prompt's chat
special tokens (<|im_start|>/<|im_end|>) can be encoded atomically — plain BPE
would shatter them and the model would score garbage.
- Reads the last-position logits as the final vocab window, robust to whether the
step returns per-position or last-only logits.
Verified with Qwen3-Reranker-0.6B (real_reranker.rs, ignored): relevant document
0.300 vs irrelevant 0.120. Note: the reranker template must omit the reasoning
<think> block for the 0.6B GGUF.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
92de7f86a0