RESEARCHAUDIO.IO

Microsoft Kept Five Tool Calls and Gained 20 Points

Full history scored 71.0%. Pruning plus a summary hit 91.6% on 62.7% fewer tokens.

arXiv:2606.10209  ·  Microsoft  ·  17 pages, 8 tables

A team at Microsoft pointed a GPT-5 agent at a live Dynamics 365 expense system and asked it to split hotel receipts into line items until the unallocated balance hit zero. Giving the agent its complete conversation history produced 71.0% clean completions. Throwing away everything except the last five tool calls produced 79.0%. Adding a short running summary of what got thrown away produced 91.6%.

The best configuration used 62.7% fewer tokens and 60.2% less wall-clock time than the worst one. That is the part worth sitting with. Accuracy and cost usually trade against each other. Here they moved together, in the same direction, because the extra context was not neutral ballast. It was actively wrong.

Bars show complete itemization, the paper's primary metric. Source: Table 2.

Why enterprise tools break agents

The task sounds trivial and is not. The agent opens an expense report with a known total, creates one line per item on the receipt, maps each item to a subcategory from a fixed 23-entry catalog, enters the amount, and continues until the remaining balance is exactly zero. Anything above zero counts as failure, because a partially itemized report cannot be finalized in the accounting system. Tasks in the benchmark run from 4 to 23 lines, median 8.

The agent drives the system through an MCP server that exposes form navigation, field reads, field writes, and button clicks as tools. Those tool responses are the problem. Each one returns a full form-state snapshot: field values, metadata, navigation breadcrumbs, system state. The authors measured 500 to 3,000 tokens per response, with 15 to 30 tool interactions per task, so a single receipt accumulates 50,000 to 150,000 tokens of transcript.

Anyone who has wired an agent to a real internal system recognizes this shape. The tool was designed for a program that reads one field and discards the rest. The agent reads all of it, forever. In the full-history run, input tokens outnumbered output tokens 594.7 to 1. Across every configuration tested, input accounted for between 99.75% and 99.87% of total token spend. Whatever you do to reduce an agent's spend, do it to the input.

The policy is about forty lines of code

Before every inference call, count the tool messages in the history. If there are more than N, evict the oldest ones along with the assistant message that requested them, so a call and its response always leave together. Then summarize the last W evicted messages into one short assistant message and slot it in where the evicted block used to start. That is the whole method. It runs at inference time, touches no weights, and costs one extra model call per eviction event.

Two design choices matter more than they look. First, eviction happens at the level of whole tool call and response pairs, not tokens. Token-level compressors such as LLMLingua can shred the exact strings an ERP agent has to read back verbatim: control names, numeric balances. Second, N=5 was picked from task structure, not tuning. One itemization line costs 2 to 3 tool calls, so five calls hold roughly two complete cycles of working memory.

Illustrative step count. Policy from Algorithm 1.

Pruning does not remove failures, it swaps them

This is the most useful table in the paper and the reason the result is not a fluke. The authors hand-labeled every task that failed to reach zero across all three configurations, pooled over five runs, into six failure modes. The taxonomy makes a prediction before it reports a count, which is rare.

Full history should over-produce stale-state references, where the agent acts on a form snapshot that has since been superseded. Pruning should crush those and introduce premature termination, because once the older responses are gone the agent can no longer see the running balance and stops early believing it is done. Summarization should kill the second problem without reviving the first. All three predictions land.

Source: Table 5. Counts pooled over 5 runs of the 50-task hotel benchmark.

Read the two bold rows across. Stale-state references fall from 34 of 73 non-completions under full history (47%) to 6 of 53 under pruning (11%). Premature termination doubles in the other direction, 9 to 18, and becomes the dominant failure once the history is gone. Summarization cuts that back to 3 without letting stale state return, which is what takes total non-completions from 73 to 21, a 71% reduction.

The four remaining modes barely move. Wrong subcategory mapping stays between 6 and 9 across every configuration, because the hotel catalog contains near-synonyms like Room tax against Non-Room tax, and Restaurant against Room service against Loungebar. That ambiguity is a labeling problem, not a memory problem, and no context policy touches it. Useful ceiling to know before you go tuning windows.

Where the curve flattens

A recency window is a hyperparameter, so the obvious objection is that 5 and 3 were tuned into existence. The authors swept both while holding the other fixed. The sweep is the most portable thing in the paper, because it tells you the shape of the curve rather than a single operating point.

Source: Table 7. Each sweep holds the other parameter fixed.

Both curves have a knee and both knees are early. Tightening to N=3 costs 5 points. Loosening to N=10 recovers 1 point at 53% more spend. Removing the bound entirely is worse than every bounded setting on both axes at once. On the summary side, W=1 gives up 5.2 points, while W=5 and full-history summarization land in the same place as W=3 for 4% and 11% more tokens.

The practical read: if you are going to prune, a small fixed window and a short summary get you most of what is available. The tuning surface is flat enough that a rough guess based on how many tool calls one unit of work takes will land close to optimal.

Three things this result is not

It is not an 8% to 91.6% improvement. The paper reports a fourth configuration, C1, at 8.0% complete itemization, and that number is going to get quoted as the before. It should not be. C1 removes two things at once: the context policy and a second model that plays the user and answers the agent's clarifying questions. In a harness with no human present, GPT-5 stalls mid-task waiting for an answer that never arrives. The authors flag this themselves and rest every context claim on the C2 to C4 comparison, where the user model is held constant. The honest headline is 71.0 to 91.6.

It is not equally dramatic on every model. Repeating the benchmark with Claude Sonnet 4.5 and no user model at all, the unpruned baseline already reached 88.0%, against GPT-5's 8.0% in the comparable slot. Sonnet does not stall. Pruning still moved it to 92.0% and pruning with summarization to 94.5%, so the ranking holds, but the ceiling was much closer to begin with. Worth noting that Sonnet's wall-clock time went the wrong way under pruning, 6.20 hours to 10.70, which the paper does not explain.

It is not a universal claim, by the authors' own framing. They scope it to structured, single-session, form-driven tasks with verbose tool responses and a hard completion criterion. They did test beyond hotels: travel receipts (n=30) went 76.0 to 95.0, and the simplest group, discretionary social spend (n=32), went 75.6 to 96.1, so the same ranking held in all three categories with a consistent 19 to 21 point gain. Still one vendor's ERP, one benchmark family, five runs.

On statistics, they did the reasonable thing. Pooling 250 task-runs, the Wilson interval for pruning-plus-summary is [87.5, 94.4] against [73.5, 83.6] for pruning alone, cleanly separated. The full-history and pruning intervals overlap slightly, so the 8-point step is directionally supported rather than proven. One detail worth borrowing: run-to-run spread was 4.4 points for full history, 8.2 for pruning alone, and 1.7 for pruning with summary. The summary did not merely raise the mean, it absorbed the variance that aggressive pruning exposed.

The industry spent two years treating context length as headroom, something you grow into. This paper is a small, careful argument that past a point it is a liability you inherit: an agent handed its own history will read a form state that stopped being true nine steps ago and act on it. The fix was not a bigger window or a better model. It was deciding, before every call, what the agent is allowed to still believe.

Worth remembering that the best run still left 8.4% of reports unfinished, and the authors say plainly that production deployments should keep human review for flagged cases.

Sources

Lodha, Varnosfaderani, Chakraborty, Mithal (Microsoft). Less Context, Better Agents, arXiv:2606.10209. Full text.

Related: ACON on learned compression, and Chroma's context rot report.

ResearchAudio.io  ·  frontier AI research for people shipping it