joule-bench/tasks/real/interval-merge/task.toml
David Charlot 0c0cde789f Set the targets I omitted; 0/6 holds, and amortisation does not
Two corrections, the first to my own work.

The `real` tasks shipped without a `target` field, so the cascade ran
with an empty document path and its single-file tiers had nothing to act
on. The 0/6 I published was therefore partly a handicap I had introduced.
`target` is now set on the four tasks whose fix really is single-file;
the other two span two files, which is why they are in the suite. The
result is unchanged — still 0/6 — so the finding was right for a reason
I had not actually established, and now is.

Second: an L4-enabled two-pass run over this suite does not amortise. All
six resolved at L4 on pass one and all six paid full price again on pass
two; nothing closed at L0. At $0.3274 per resolved task the cascade is
not cheaper than raw claude at $0.2809 — slightly worse, for the local
overhead on top of the same cloud call. That contradicts the `mix`
result, where the cache fired perfectly.

The likely cause is the same missing target: with no single file to
identify a close by, the L0 key has nothing stable to hang on. That run
predates the fix and re-running it costs about $4, so it is stated as the
likely cause rather than the confirmed one.

What no fix repairs is the two genuinely multi-file tasks. They have no
single target by construction, so if L0 keys on one file's content a
multi-file change cannot be cached at all — and multi-file is 53-66% of
real commits. That limit is stated now rather than waiting, because `mix`
never tested the case.

Cost note: this run was estimated at ~$1.80 on the assumption pass two
would be free. It was not, and came to ~$3.93.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-15 11:14:19 -04:00

5 lines
331 B
TOML

name = "interval-merge"
prompt = "merge() only joins spans that are already adjacent in the input and assumes they arrive sorted. Make it merge any overlapping or touching spans regardless of input order, including fully contained ones."
test_cmd = "python3 -m pytest -q"
test_files = ["test_intervals.py"]
target = "intervals.py"