An agent stopped compacting its own conversation and posted a warning: it was over the
compression threshold, and compression was now blocked. Not an error. Not a crash. It just
stopped being able to continue the thread.
The symptom looked like a model problem. It was arithmetic.
The documented number was wrong
Our internal notes said the compaction trigger was 75% of the context window — about 55,000
tokens. Reading the actual code path, the ratio result is passed through a floor, and that floor
is a hard-coded 64,000. The floor raises, never lowers. At the window we were running, the
percentage calculation produced about 52,000, the floor won, and the real trigger was a flat
64,000 every time.
That leaves under 10,000 tokens of headroom between the trigger and the hard limit, most of it
reserved for output. The earlier, comfortable conclusion that truncation was “unreachable by
construction” had been arithmetic performed on the wrong number.
Why it tipped over on one particular day
Compaction can only shrink messages. The system prompt and the tool schemas are an
incompressible floor — they are re-sent every single turn and no amount of summarising touches
them. When that floor plus the protected recent messages already exceed the threshold, every
compaction pass shrinks the messages honestly and still lands over the line, scoring a failure.
Two failures latch a breaker that turns compaction off for the session.
Two changes the same day raised the floor. One added two more tool servers, taking the agent
from roughly 32 tools to 82. Measured against the live schemas, that is about 220 tokens per tool,
permanent — roughly 11,000 tokens of unreclaimable prompt added in a single commit.
The other raised a file-inclusion limit, and the agent’s instruction document grew by about 6,000
tokens to fill it.
Neither change was wrong on its own. Neither had a budget attached. Together they put the floor
at 35,000–40,000 tokens of a 64,000 budget, and the agent went quiet.
The intuitive fix is strictly worse
The obvious move is to lower the trigger so compaction fires earlier. That is backwards, and
seeing why is the whole lesson.
Post-compaction size is floor plus protected tail. Lowering the trigger does not change either
term. It just makes the same comparison fail sooner, so the breaker latches faster. The
absolute cap was deliberately left unset.
What actually worked was attacking the two terms:
- Shrink the protected tail. Twenty verbatim recent messages down to eight,
with the token budget cut alongside it. On an agent whose “messages” are routinely hundred-row
query results, twenty of them is enormous. - Keep the question. A minimum number of recent user messages is now
protected, because one bulky tool result could otherwise evict the request that caused it — which
reads, from the outside, as the agent forgetting what you asked mid-task. - Prune stale tool results deterministically. A no-model pass that drops large,
old tool outputs. This is the only lever that attacks the bloat rather than rationing around
it.
A compacted prompt now lands around 45,000 — roughly 19,000 clear of the trigger — so a pass
scores as effective and the failure counter resets instead of latching.
The rule that came out of it
Every tool you add costs about 220 tokens on every turn, forever. A forty-tool
server is roughly 9,000 tokens off the conversation, paid whether or not any of those tools is
ever called, and the failure mode is not an error message — it is the agent going quiet.
So tool surfaces get budgeted now, the same way memory or disk would be. An integration that
adds 43 tools to answer one question is a bad trade, and it is only visibly bad if someone is
keeping score.
Two things I had to write down rather than fix
A wedged session stays wedged. The failure count persists on disk and survives a restart, so
the deployment came back healthy and the affected conversation was still stuck. It needs a fresh
thread; an existing one does not recover on its own.
And the automated post-deploy check that should have caught this was inspecting a class of
workload that had been retired weeks earlier. It was passing because it was looking at nothing.
That is the kind of failure that is invisible right up until the day you needed it, and it is why
“the check is green” is worth less than “I know what the check looked at.”