Pattern · last reviewed 2026-07-13

Agent memory and state: what to remember, where, and for how long

Context & problem

A demo agent has no memory problem: one prompt, one answer, the context window is empty when it starts and discarded when it ends. Production is the opposite. A support agent needs the last twenty turns of the conversation, the customer's plan and history, and what it learned about this account three weeks ago — three different things that people casually call "memory" and that must not share a store.

Conflating them is how you end up with the two failures at once. Stuff everything into the prompt and the context window grows until latency, cost, and the token limit all bite — and a model re-reads every token on every turn, so a 40k-token history is a 40k-token tax per turn, not once. Persist nothing and the agent is an amnesiac that re-asks the customer's name each morning. The pattern exists because these pull in opposite directions: the fix for one is the cause of the other unless you separate the tiers and give each its own budget, store, and retention rule.

Forces

• Context economics vs recall. Every remembered token is re-processed each turn. More recall in-context means more cost and latency and a nearer token ceiling.
• Persist vs recompute. Some state is cheap to recompute from source of truth (the user's current plan — query it); some is expensive or lost if not written down (what the agent concluded across a long session).
• Freshness vs stability. Durable user state must reflect the real record now; a cached copy in memory drifts. Derived summaries are stable but can preserve a fact that has since changed.
• Autonomy vs safety. Letting the model write its own memory is powerful and is also a prompt-injection surface — untrusted content can plant instructions that a later session reads back as fact.
• Retention vs privacy. Memory that outlives its usefulness is liability, not asset. Storing forever is a GDPR and security problem; deletion has to be a designed path, not an afterthought.

The pattern

Model memory as three tiers, each with one owner and one store, and route every "remember this" question to exactly one of them. The decision at each tier is the same three-way choice you already make for any cache: persist, recompute, or forget.

Tier 1 — working context (forget by default). The conversation window: the current turns the model needs to stay coherent. Its correct lifetime is the task, not forever. You bound it — a turn cap, a rolling window, or server-side clearing of old tool results — and you accept that anything not promoted to a lower tier is gone when the window closes. Trying to make Tier 1 permanent by never trimming it is the unbounded-growth bug.

Tier 2 — durable user state (recompute from source of truth). Facts you already own in a system of record: the user's plan, their progress, their preferences, the session transcript. This does not belong in the prompt as standing text; it belongs in a store you query on demand and inject only the slice a turn needs. Its lifetime is the user relationship, governed by your retention policy. The key discipline: read it, never let the model's copy become the truth.

Tier 3 — derived knowledge (persist deliberately). What the agent concluded that isn't written anywhere else: a summary of a 50-turn session, a learned fact ("this customer prefers email"), a running progress log for a multi-session task. This is the tier worth the write cost, because it is expensive or impossible to recompute. It lives in a small, curated store — memory files or a vector index — retrieved just in time rather than loaded up front. It is also the only tier the model writes to, which makes it the tier that needs validation and a retention clock.

The whole pattern is one routing rule: for each thing you want to remember, ask which tier owns it, then choose persist / recompute / forget for that tier. Most "the agent should remember X" requests resolve to Tier 2 (you already have it — query it) or Tier 1 (you don't need it after this task — let it go). Tier 3 is the smallest tier by design; if everything is landing there, you are hoarding.

Reference implementation notes

The three tiers map onto concrete features across the stacks we teach. Each platform's primitives differ; the taxonomy does not.

Anthropic

Two current API features sit squarely in this pattern. Context editing (beta header context-management-2025-06-27) is Tier-1 hygiene: it clears old tool-use/result pairs and thinking blocks server-side, before token counting and after the prompt-cache lookup, so you shrink the window without destroying the cache prefix. On a 100-turn benchmark Anthropic reports it cut token use ~84% while improving the outcome. The memory tool (memory_20250818, now GA on the Messages API — no beta header) is Tier 3: the model issues view/create/str_replace/delete commands against a /memories directory, and your application executes them against storage you control.

# Tier 3: the model requests, you store. Nothing is server-side.
tools: [{ "type": "memory_20250818", "name": "memory" }]

# The model emits a tool_use; your handler maps /memories onto real storage
{ "command": "create", "path": "/memories/acme.md",
  "file_text": "Acme Corp prefers email follow-ups\n" }

Because it is client-side, the store is yours to place, cap, and expire — and the security burden is yours too: reject any path outside /memories (a /memories/../../secrets.env traversal is the documented attack), cap file growth, and periodically delete stale files. Pair it with server-side compaction when a conversation nears the window limit and you want the whole history summarized rather than specific results cleared.

AWS

Amazon Bedrock AgentCore Memory is a managed service that implements Tiers 2 and 3 directly. Short-term memory stores raw interactions for a session so the agent can reload the exact conversation — Tier 2, the session transcript. Long-term memory is an asynchronous background process that extracts insights from those raw events after they land, consolidating them into retrievable records — Tier 3, derived knowledge, without you writing the extraction loop. Long-term records now carry metadata, so you can tag and filter them alongside semantic search rather than relying on similarity alone. The managed split is the pattern's short-term/long-term boundary drawn for you; the design decision it does not make is what is worth promoting to long-term — that judgement stays yours.

Cloudflare

The Agents SDK gives each agent its own Durable Object, and each Durable Object its own colocated SQLite database reachable at this.sql — zero-latency reads, no cross-region round-trip. State written with this.setState() is serialized into the cf_agents_state table and survives evictions, redeploys, and hibernation, so Tier 2 session state persists across the agent going to sleep between turns.

// Tier 1/2 boundary inside one agent: transient turn buffer vs persisted state
this.setState({ ...this.state, lastSummary });   // persisted, survives eviction
this.sql`INSERT INTO messages (session_id, role, content, ts)
         VALUES (${sid}, ${role}, ${text}, ${Date.now()})`;

For Tier 3 across users, D1 holds shared relational state and Vectorize holds embeddings for semantic recall of past knowledge. The division is clean: per-agent SQLite for one learner's live session, D1 for durable cross-session records, Vectorize when retrieval has to be by meaning rather than by key.

Trade-offs

• Three stores, three consistency stories. Splitting memory across a window, a DB, and a vector index means three retention clocks and the risk that a Tier-3 summary preserves a fact Tier 2 has since changed. You trade a single blob for correctness that has to be maintained per tier.
• Write-back is a new failure surface. The moment the model can write memory, a poisoned tool result or a prompt-injected document can plant a "fact" that a future session trusts. Tier 3 needs validation on write and an audit trail, which is engineering you would not need if the agent only read.
• Just-in-time retrieval adds latency per turn. Not loading everything up front means fetching the right slice at the right moment — an extra query or vector lookup on the critical path, in exchange for a smaller, cheaper context.
• Retention is now your problem. Deliberate memory means deliberate deletion: inactivity windows, per-user erasure, export. That is real code and a real obligation, not a checkbox.

When not to use this

Skip the tiered pattern when the agent is genuinely single-turn or single-session and stateless is correct — a classifier, a one-shot extraction, a search-and-answer with no follow-up. Building three memory tiers for a task that ends when the response is sent is pure overhead; the whole point of Tier 1 being "forget by default" is that some agents never need anything else.

Skip Tier 3 specifically — the model-written memory — when everything the agent needs already lives in a system of record you can query. If the answer to "what should it remember?" is always "look it up," you want Tier 2 alone: recompute from source, and let the window forget. Adding a write-back store there just invents a second, staler copy of data you already own, plus a prompt-injection surface you did not have to expose. And if long-context is cheap enough for your corpus and it is small and stable, stuffing it into the window can beat any external memory — retrieval earns its complexity only past the point where the context economics stop favoring "just include it."

As-built evidence

aiArch runs a modest version of this pattern in production. The coach is a Cloudflare Agents SDK CoachAgent — one Durable Object per learner, addressed by the authenticated user id, WebSocket transport. Its own colocated SQLite holds the session transcript as rows in a messages table (Tier 2, keyed by session_id), and a separate memory table holds coach-curated per-learner facts written through a validated update_memory tool (Tier 3 — the model writes, but only through a formatter that checks the entry). Durable learning state — mastery, spaced-repetition reviews, attempts — lives in D1, queried on demand rather than carried in the prompt.

Retention is designed, not deferred. The agent schedules a 365-day inactivity wipe of its transcript via the Agents SDK schedule() callback, re-armed on each user turn. A single USER_TABLES registry drives both GDPR erasure (delete every row for a user across those tables in one D1 batch) and export (the same registry read out as a "give me my data" copy) — and a schema test fails the build if a new user-scoped table is added without registering it there. We are deliberately not claiming a semantic-recall tier: there is no Vectorize-backed long-term memory of past conversations in the coach today. The tiers that exist are Tier 1 (bounded coach loop), Tier 2 (session + mastery state), and a small validated Tier 3.

Changelog

• 2026-07-13 — Initial publication. Verified against the memory tool (memory_20250818, GA), context editing (context-management-2025-06-27, beta), Bedrock AgentCore Memory (short-/long-term), and Cloudflare Agents SDK state.