Pattern · last reviewed 2026-11-03

Compaction forgets on purpose: window, summary or server

Context & problem

The decision this page improves is the one you make the first time a conversation gets long: do I trim it, summarize it, hand it to the provider, or keep notes on the side? Teams usually make that call by reaching for whichever mechanism is closest. In our reading, the vendor documentation compares its own mechanisms against each other and does not argue for compacting nothing.

Two things make the choice harder than it looks. First, quality drops as a conversation grows, not only cost, so "it still fits in the window" is not a reason to leave it alone. Anthropic's compaction documentation makes the same point: longer conversations get worse answers, not just pricier ones, and compaction is how it keeps the active context small. Second, a summary is a new object. It is a derived copy of what the user said, written by a model, and it has to be stored, exported and erased like any other copy.

Forces

• Exactness vs size. A summary keeps the gist and loses the literal. An order number, a file path or a quoted error message survives a window if it is recent, and survives a summary only if the summarizer chose to keep it.
• The opening goal. A sliding window drops the oldest turns first, and the oldest turn is usually where the task was stated. A long session can end up answering the latest message with no memory of what it was for.
• Control vs effort. Writing your own summarizer gives you the prompt, the timing and the storage. It also gives you the code to maintain. Server-side compaction removes that code and removes some of the control.
• Cache behavior. Per Anthropic's threshold documentation, put a cache breakpoint after the system prompt so that compaction invalidates only the conversation part. Compaction is a cost event as well as a size event; see cache-aware prompt layout.
• History is user data. A stored summary of a conversation is personal data derived from that conversation. Compaction creates a copy that retention, export and erasure must reach.

The pattern

Pick the mechanism from three questions, in this order.

1. Must exact strings survive? If yes, a summary alone is the wrong tool. Keep those strings as structured facts outside the history, or keep the turns that hold them verbatim.
2. Must the opening goal survive? If yes, a pure sliding window is the wrong tool. Pin the goal where truncation cannot reach it.
3. Is the history user data you must be able to export and erase? If yes, count every derived copy. A summary you store is one more thing to delete.

The fourth row is the one that rarely appears in vendor comparisons, because it is not a compaction feature. It is a design decision: the facts that matter are too important to leave to a summarizer, so you decide what they are and store them separately. It is the same split that agent memory describes as working context versus durable state. This page covers only the choice of what to do when the working context gets too big.

Reference implementation notes

Anthropic: two server modes, one client option removed

Anthropic's compaction documentation describes compaction as the API swapping early history for a model-written summary, produced server-side. It lists three ways to get it, and they differ in who decides when to compact.

• Compaction on demand (beta header compact-2026-09-04, with a top-level compaction parameter): you decide when, by sending a request for the summary. The documentation says recent turns can stay word for word, and that compaction can run in the background so the conversation does not have to pause.
• Compaction at a token threshold (beta header compact-2026-01-12, the compact_20260112 edit under context_management.edits): the API decides, once input tokens reach a trigger. The default trigger is 150,000 input tokens and the value must be at least 50,000. The documentation says the API drops the content blocks before the compaction block on later requests. You can set pause_after_compaction, and a custom instructions string completely replaces the default summarization prompt.
• Your own summarizer: the documentation lists it for teams whose existing summarizer already rewrites the full history.

The documentation's guidance is "Use on-demand compaction wherever it is available." Both server modes are in beta, so pin the header and expect the shape to move.

The 2026-10-21 SDK major versions changed one thing that matters here. The migration guide says the TypeScript and Ruby SDKs dropped the tool runner's own compaction and now defer to the API's. In TypeScript the compactionControl option is gone. The guide's replacement for compacting automatically at a threshold is to send the beta header and a compact_20260112 edit with an input_tokens trigger, and to call runner.compactBeforeNextTurn() if you want to decide the moment yourself.

// Threshold compaction, the shape the migration guide gives (TypeScript)
betas: ["compact-2026-01-12"],
context_management: { edits: [{
  type: "compact_20260112",
  trigger: { type: "input_tokens", value: 100_000 }   // minimum 50,000
}] }

Context editing is a separate, smaller tool for a separate problem. It deletes stale tool output or thinking according to rules you set, with no summary step. If most of your tokens are stale tool output, clear it before you reach for a summary.

On cost, Anthropic's own measurement is worth reading with its limits in mind. On Claude Haiku 5.5, in a synthetic 45-request conversation with filler text, a session cost $0.272 with the default 150,000 trigger, $0.082 with a 90,000 trigger, $0.075 with a 50,000 trigger and $0.729 with no compaction. The conversation was filler, so the summaries were short. Treat it as evidence that the trigger value matters for price, not as a number your workload will reproduce.

AWS and Cloudflare

Neither platform changes the choice. On Amazon Bedrock, Anthropic lists threshold compaction as available, so the same decision applies. On Cloudflare, our coach keeps a window of recent turns and a pinned-facts table in its Durable Object's SQLite (this.sql in src/coach/CoachAgent.ts), which holds both comfortably. What differs between platforms is who owns the code that rewrites the history: you, in both cases, unless you hand it to the model provider.

Trade-offs

• Every option is lossy and the loss is silent. A truncated turn or an omitted detail produces no error. You find out when the agent contradicts something it was told an hour ago. Test compaction with a conversation whose early turns contain a fact you will ask about later.
• A model-written summary can be wrong. It can merge two things the user kept separate or state a guess as a fact. A bad summary is then treated as ground truth by every later turn.
• A summary of untrusted content is untrusted. If the history contains tool results or retrieved documents, instructions inside them can be carried into the summary and read back as if they were yours. Fence it the way you fence the originals.
• Compaction moves the cost, it does not remove it. Per the threshold documentation, the summary is itself a billed model call that also draws on your rate limits, and its output lands as a fresh cache write. In our reading, if the conversation ends soon after, you may have paid for nothing.
• Pinned memory shifts the work to write time. You decide up front what is worth keeping, and you carry the validation, caps and deletion logic for it.

When not to use this

Do not compact when the conversation is short, or when each session ends before the window matters. A summarization step on a ten-turn exchange adds latency, a cost event and a lossy copy to protect against a problem you do not have.

Do not compact when the content is the record. If the exact wording of the turns is what the user or an auditor needs back, such as a contract negotiation or a debugging log, a summary is the wrong thing to keep. Keep the full history in storage and let the model see a window of it.

And do not reach for compaction when a smaller change fixes it. If the growth is stale tool output, clear it. If the growth is a large static prefix, cache it. If the same facts keep needing to be re-stated, pin them.

As-built evidence

The aiArch coach does not compact. This is a choice, and the code shows it.

On each learner turn, the coach reads the last eight messages of the session from its Durable Object's SQLite and sends only those as history (src/coach/CoachAgent.ts:210, with the limit of 8 set at :301-307). Older turns stay in storage and are never summarized or sent. That is a sliding window with no summarizer, so what falls out of the window is simply not seen by the model on later turns.

What must survive does not depend on the window. The coach keeps a small memory table the model can write to through a tool. The caps live in code, not in the prompt: at most 12 entries of at most 200 characters each (src/lib/memory.ts:7-8), with keys restricted to a goal entry or a preference, struggle or context namespace. The snapshot is formatted with invisible characters stripped (formatMemorySnapshot in src/lib/memory.ts) and passed into the prompt as fenced untrusted content (src/lib/coach.ts:236). So the opening goal has a pinned home, and the window is free to forget.

That is the fourth option in the table, and the reason we chose it over a summarizer is the third question. The history is learner data. Both the transcript and the memory rows live in the learner's own agent, they are returned together by the export hook (exportTranscript in src/coach/CoachAgent.ts), and the agent schedules a wipe after 365 days of inactivity (rescheduleRetentionWipe in src/coach/CoachAgent.ts, window in src/lib/retention.ts). A stored model-written summary would be another derived copy of what a learner said, in a store we would have to cover with the same export and erasure paths. We did not need that copy, so we did not create it.

What this does not do. It does not compress. A learner who refers back to something said twelve turns ago, and did not trigger a memory write, will find the coach has lost it. We accept that because we expect coaching sessions to be short. Nothing in the code caps session length, and we would revisit this if sessions got long. The 8-row limit is a constant, not a measured optimum.

Changelog

• 2026-11-03 — Initial publication. Verified against Anthropic's compaction overview, threshold compaction page and the 2026-10-21 migration guide, accessed 2026-11-03.