Agent Memory Design Patterns That Hold Up in Production

Most agent failures that look like reasoning problems are memory problems. The agent forgets a constraint it agreed to three steps ago, re-reads a file it already summarized, or carries a stale fact into a decision long after it stopped being true. Agent memory design patterns are the rules that decide what an agent keeps in front of the model, what it compresses, and what it drops, so behavior stays stable as a task runs long.

The conceptual side is well covered. What's missing is the part you can reuse: a prompt that turns those patterns into a repeatable decision, with a memory state you can read and check every turn.

Why agent memory is a design problem, not a storage problem

Storing text is easy. Deciding which text earns a slot in the next model call is the actual job, and it's where most setups break.

A working agent juggles three pressures at once. The context window has a hard token ceiling. The model weights recent tokens more heavily than early ones, so a constraint stated at the top of a long session quietly loses force. And every token you carry forward costs money on each call. Memory design is how you spend a fixed budget on the facts that change the next decision.

The three memory tiers, and what each one is for

Agent memory splits into three tiers that serve different jobs. You rarely need fewer than all three for a task that runs more than a handful of steps.

Working memory is the live context for the current step: the goal, the last few actions, and their observations. It's fast to read and small by design. When it grows past a threshold, the oldest entries get summarized or dropped, not kept verbatim.

Episodic summary is the compressed history of the session so far. Instead of replaying twelve tool calls, the agent carries a short, structured recap: what's been decided, what's still open, what failed and why. This is the tier that keeps a long task coherent.

Long-term store is everything that should outlive the session, kept outside the prompt and pulled back by relevance when a query needs it. Codebase facts, prior decisions, user preferences. The agent retrieves a handful of entries, not the whole store.

The skill is the boundary between them. A fact that matters for the next two steps lives in working memory. A decision that constrains the rest of the task gets promoted to the episodic summary. A fact that should survive a restart goes to the long-term store. A prompt can make that call consistently if you give it a contract to fill.

What you can do with a memory-aware agent prompt

• Decide each turn what to keep verbatim, what to summarize, and what to evict
• Promote durable decisions into a rolling summary so they don't decay with token position
• Cap working memory at a token budget and compress past it instead of overflowing
• Emit a structured memory state you can log, diff, and audit between turns
• Pull only the relevant long-term entries for the current query, not the whole store
• Catch the moment a stale fact contradicts a newer one and flag it

Anatomy of the memory contract

The reusable piece is a prompt that reads the current state plus the latest turn and emits an updated state in a fixed shape. Structure beats prose here, because a fixed shape is what lets you check the output.

Variables → {{current_memory_state}}, {{latest_turn}}, {{token_budget}}, {{task_goal}}

Prompt → Role: you maintain the agent's memory under a token budget.
         Read the current state and the latest turn.
         Decide: keep / summarize / evict for each item.
         Promote any decision that constrains future steps.

Output contract (locked):
  working_memory:   [ up to N items, each ≤ one line ]
  episodic_summary: [ decisions, open questions, failures ]
  evicted:          [ what was dropped and why ]
  promoted:         [ what moved to long-term, with a key ]

Because the output is the same four blocks every run, you can store it, feed it back as {{current_memory_state}} next turn, and diff two states to see exactly what changed. That's the difference between a memory system and a context window you hope behaves.

Where models differ

Memory contracts aren't model-agnostic, and pretending they are is how output drifts. Claude honors a ## Output format heading and tends to keep the four blocks intact across long sessions. GPT-4o follows the same contract but benefits from the schema restated on the final line of the prompt, since it weights the most recent instruction heavily. Gemini holds structure well on short turns but is more likely to merge evicted and promoted unless they're shown as clearly separate sections. Name the model you're targeting and test the contract against it before you depend on it.

An opinionated take: most teams over-invest in vector databases and under-invest in the summary tier. A clean episodic summary that the model actually reads beats a sprawling vector store it queries badly. The promotion rule earns more reliability per hour than swapping embedding models. The Context Engineering Harness leans on this: a {{token_budget}} variable drives a keep-summarize-evict pass with a locked memory-state contract, so the same policy runs every turn.

Step-by-step: wiring memory into an agent

1. Set the budget

Pick a token ceiling for working memory well under the model's window. Leave room for the task instructions and the next observation. {{token_budget}} is the lever; smaller forces earlier compression.

2. Define the promotion rule

Write one sentence: "Promote anything that constrains a future step." That sentence is the whole memory policy. Everything else is bookkeeping.

3. Run the contract each turn

Feed {{current_memory_state}} and {{latest_turn}} in, get the four blocks back, store them. Next turn, the output becomes the input. The loop is the system.

4. Diff between turns

Log each state. When the agent does something surprising, the diff usually shows the cause: a constraint that got evicted too early, or a stale fact that was never flagged.

5. Retrieve sparingly

When a query needs long-term context, pull a handful of entries by relevance, not the whole store. Cramming the store back into the window recreates the overflow you were avoiding.

Variables you'll set

Memory's quiet failure mode

The dangerous failure isn't overflow, which is loud. It's the stale fact that survives a promotion it shouldn't have and then biases every later decision. A model that confidently recorded "the build passes" will keep acting on it long after the build broke, because nothing told it to re-check. Build a recency check into the contract: when a new observation contradicts a promoted fact, flag it rather than silently keeping both. And pin your model version for anything you depend on, since a model update can shift how aggressively it compresses.

This connects directly to context-window work. The budgeting side, how much to load before a run even starts, is covered in context window budgeting for AI agents, and the broader practice of shaping what the model sees lives in the context-engineering pack below.

Getting started

1. Pick the model you're targeting and note how it handles a ## Output format heading.
2. Set {{token_budget}} to roughly a third of the window.
3. Write the one-sentence promotion rule.
4. Run the four-block contract on a real task for ten turns.
5. Diff the states and tune what gets promoted.
6. Add a long-term store only once the summary tier is clean.
7. Reach for the Context Engineering Harness when you want the contract pre-built.

For the planning side of the same agent, see ReAct vs plan-and-execute agent prompts, and pair this with the Context Window Budget Harness when a single task keeps blowing its window.