How Agent Memory Works Under the Hood

Study Guide · Chapter 1: State & Memory · 7 min read

Memory sounds abstract until you see the moving parts. In practice it comes down to four loops: reading memory before each turn, writing results after each turn, compressing what grows stale, and isolating what must stay private.

Read before every turn

Before the model answers, the system assembles context from all three tiers: working state from a fast store like Redis, recent episodes summarized from a database, and semantically related facts found via a vector index such as pgvector, Pinecone, or Qdrant.

Only this assembled package is sent to the model. The model never touches raw storage — it sees a clean, filtered view of everything the system decided is relevant right now.

Write after every turn

Each turn produces artifacts worth keeping: the objective, actions taken, outcomes, and confidence levels. These are appended to episodic storage with metadata — who, when, which tenant, what sensitivity level.

Writes are never fire-and-forget in serious systems. Every record carries a retention policy and an audit trail so that months later you can explain exactly why the agent remembered something.

Compact and isolate

Episodic logs grow forever unless pruned. Compaction jobs summarize old interactions into durable takeaways and drop stale detail, keeping sessions fast and preventing outdated facts from contradicting fresh ones.

Tenant isolation is enforced at the retrieval layer itself: every search is filtered by customer, project, and policy scope, so one customer's documents can never surface in another's context.

Key Points

Memory is a pipeline: assemble context → run the model → commit new memories → compact periodically.
Vector search finds related facts; metadata filters enforce who may see them.
Summarization keeps long sessions affordable without losing important history.
Tenant isolation must live inside retrieval — it is a compliance requirement, not a feature.

All study guides for this chapter: Agent Memory, Explained Simply · How Agent Memory Works Under the Hood · Agent Memory in the Real World