From the tool log to the bill
Suppose a coding agent starts an issue with a packet of excerpts from relevant files. Its tool log ought to show fewer searches. In the main experiments in Zerui Yang and colleagues' preprint, read and search commands fell 18–34 per cent. With cached input billed at a tenth of the fresh input rate, estimated costs for the same tasks rose 33–42 per cent. The authors measured no improvement in the share of tasks solved.[1]
A developer assembles that packet once. The code remains part of every subsequent model call as long as it stays in the prompt. Prefix caching makes it cheaper to reuse an opening stretch that stays the same, and even cached input has a price in this experiment. Removing searches therefore saves one kind of work while recurring input still incurs a charge. A packet can save a substantial amount of exploration and still cost more.[1]
Trimming old tool output also depended on the pricing terms. With zero cache discount the researchers estimated savings of 19–36 per cent. When cached input cost one tenth as much, costs increased by 38–116 per cent. The team observed agents repeating some searches and file reads after old outputs were hidden.[1]
Price the solved task
A team tuning an agent's context can compare two runs on the same tasks with the same model. One leaves the agent to search the repository. The other supplies relevant code upfront or prunes old outputs. For each solved task I'd count fresh and cached tokens, model output and repeated searches. When both patches pass the same tests, I'd measure the maintainer's review time separately.[1]
There is a trap in running the comparisons side by side. The authors found that concurrent experiments could share cached prompt prefixes. One run can warm the cache for another and change the apparent cost of a configuration. The comparison needs to track which run got to reuse which input.[1]
I'd keep context management configurable. If the resolution rate holds steady, whether the agent needs a code packet depends on what caching costs and how often the agent goes back to the same files. A team can repeat that comparison when provider prices or its mix of tasks change. The goal is a patch ready for review at a lower cost.[1]