open-standards/jouleclaw/jouleclaw-rs/crates/jouleclaw-cache
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Transaction Science d2a9c4ee1f jouleclaw-cache: 4-vendor 0.1x cache-read convention as a normative cost model
Wave-6 item 6. Per the wave-6 brief: "CAG" as a named research
artifact is a 4-page short paper; the deployed reality is
**prefix KV-cache reuse exposed as a billing line item by every
major LLM provider**. We ingest the deployed shape — the
0.1× input-token cache-read multiplier convention that converged
across Anthropic, Google, OpenAI, and DeepSeek between Oct 2024
and mid-2025.

Surface:

- `CachePolicy { provider, write_multiplier, read_multiplier,
  ttl_kind: { Explicit | Implicit }, ttl, min_tokens }` —
  every shipping cache provider fills this struct. Multipliers
  clamped to [0, 100] so transposed values can't blow up cost
  arithmetic. `matches_converged_read_multiplier()` checks the
  ≤0.15 (≥85% off) ceiling.
- `vendor::*` presets verified against the brief's table:
  anthropic_5m (write=1.25, read=0.1, 5min explicit),
  anthropic_1h (write=2.0, read=0.1, 1h explicit),
  google_gemini_flash (write=1.0, read=0.1, 1h explicit,
  storage billed separately), openai_automatic (write=1.0,
  read=0.1, implicit, 5min typical), deepseek_disk_cache
  (write=1.0, read=0.1, implicit, 1h typical).
- `CacheEvidence { cache_read_input_tokens,
  cache_creation_input_tokens, fresh_input_tokens, output_tokens,
  provider }` — the converged usage-struct shape Anthropic +
  DeepSeek already return per call.
- `hit_rate()` — `cache_read / (cache_read + fresh)`,
  deliberately excludes creation tokens (those are one-off write
  cost, not part of the hit denominator).
- `CostEstimate::from_policy_and_evidence(...)` — realised cost
  breakdown: `effective_input_cost`, `output_cost`, `total_cost`,
  `uncached_total_cost`, `savings_ratio` (clamped to [0, 1]).
  Rejects provider mismatch between policy and evidence (apples
  and oranges guard).

Honest scope (v1): types + computation only. We do NOT call any
provider's cache API. Cost units are arbitrary; the consumer
scales to $ / joules / quota. TTL for implicit providers is a
typical observed value, NOT a guaranteed lower bound — the
TtlKind::Implicit tag makes that explicit.

13 tests cover: all five vendor presets satisfy the converged
read-multiplier test; explicit vs implicit TTL classified
correctly; multipliers clamp into [0, 100]; total_input_tokens
sums three categories; hit_rate excludes creation tokens +
zero-input case; cost arithmetic applies read_multiplier
correctly + provider-mismatch rejected + write_multiplier
applied to creation tokens; zero-savings edge case; CachePolicy
+ CacheEvidence JSON round-trip; non-converged read multiplier
fails the predicate.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
2026-05-31 09:37:07 -04:00
..
src jouleclaw-cache: 4-vendor 0.1x cache-read convention as a normative cost model 2026-05-31 09:37:07 -04:00
Cargo.toml jouleclaw-cache: 4-vendor 0.1x cache-read convention as a normative cost model 2026-05-31 09:37:07 -04:00