The cognitive science taxonomy of memory types — working, episodic, semantic, procedural — is a useful frame and a poor implementation guide. Naming the types does not tell you which storage primitive preserves the retrieval characteristics each type requires. "Episodic memory" does not specify whether to use a vector store, a relational table, a key-value store, or a structured document. The mapping from cognitive abstraction to storage primitive is not given by the taxonomy — it has to be worked out, and working it out is where the research actually lives.
The four-type taxonomy was introduced in RES/AGT as it relates to agent control topology. This page starts where that section ends: the taxonomy is assumed, the storage primitive question is the subject. The research questions here are operational — what breaks when the wrong primitive is used, what interference arises when memory types share infrastructure, what forgetting strategies are required rather than optional, and what procedural memory versioning demands from a production system.
Most agent memory bugs are retrieval bugs. The agent is not failing to remember — it is retrieving the wrong thing, or retrieving from the wrong store, or retrieving content that was correct when it was written and stale when it was retrieved. Understanding the failure modes by memory type is the prerequisite for designing systems that are reliable over time rather than over a single session.
Cognitive science and information systems research both have deep bodies of knowledge about memory. The question is which concepts transfer to LLM-backed agent memory, which require adaptation, and which produce misleading intuitions because the underlying mechanism is different.
| Concept | Transfers? | Adaptation Required | Confidence |
|---|---|---|---|
| Working memory = context window | Yes | The analogy is clean structurally. The operational consequence is different: biological working memory degrades gracefully under load; context window degradation is abrupt and asymmetric — early content loses attention weight before it is evicted. Explicit management of context composition is required; passive accumulation is not equivalent to working memory use. | High |
| Episodic memory via vector store | Partially | Vector stores can hold episodic records. They cannot natively distinguish episodic from semantic content — a query that should retrieve domain knowledge surfaces past task records that happen to be semantically adjacent. Episodic retrieval requires either store isolation or a retrieval routing layer that classifies query intent before execution. | Medium |
| Semantic-episodic retrieval interference | No | No established pattern exists for preventing episodic records from contaminating semantic retrievals in a shared vector store. The interference is not a bug in the store — it is the correct behavior of a similarity search that cannot distinguish record provenance. Store separation is the structural fix; the routing decision that determines which store to query is an unsolved classification problem. | Low |
| Forgetting as architectural primitive | No | Production memory systems treat forgetting as cleanup — TTL expiry, manual pruning, storage limits. Forgetting in biological memory is active and adaptive: what is not rehearsed decays, freeing capacity for new encoding. Architectural forgetting — compression, summarization, intentional eviction before capacity pressure — is not a standard pattern in LLM memory systems. It is treated as optional rather than as a required design decision. | Low |
| Procedural memory versioning | Partially | System prompts and few-shot examples encode procedural knowledge. They are updated but rarely versioned — old procedures are overwritten without preserving the change history. A procedure that was correct at deployment and wrong after fine-tuning produces incorrect behavior that is difficult to diagnose without the prior version for comparison. | Medium |
| Compression as active memory management | Partially | Summarization of intermediate results is used in long-context pipelines. It is not treated as a memory management discipline — compression decisions are ad-hoc rather than governed by a retention policy. What to compress, when, and at what fidelity loss is a design question that is currently answered by intuition. | Medium |
| RAG as working memory extension | Yes | Retrieval-augmented generation functionally extends working memory by loading relevant content at inference time. The constraint is that retrieved content competes with in-context content for the attention budget. A context window filled with retrieved semantic content leaves less room for task instructions and intermediate reasoning. RAG is a working memory extension with a fixed budget — budget allocation is a design decision, not a default. | High |
| Cross-session continuity | Contested | Continuity across sessions requires that episodic records from prior sessions be retrievable and contextually relevant in the current session. Whether vector store retrieval produces usable continuity depends on how episodic records are structured. Records written as raw transcripts retrieve differently than records written as structured summaries of outcomes. The right encoding for continuity has not been characterized empirically. | Low |
State of Understanding — RES/MEM · Memory System Primitives
The context window is working memory. The analogy is structurally accurate and operationally misleading. Accurate: both are bounded, fast, and hold the active contents of current processing. Misleading: biological working memory degrades gracefully as capacity is approached. Context windows degrade asymmetrically — content near the beginning and end of the window receives more attention than content in the middle, and the degradation is not proportional to how recently the content was added.
This means passive accumulation — adding content to context without managing what is there — is not equivalent to working memory use. An agent that adds every retrieved document, every tool response, and every intermediate reasoning step to the context has not extended its working memory. It has filled a fixed-size buffer unevenly, with the most recently added content competing against earlier task framing for the attention that reasoning needs to produce correct outputs.
Active context management treats working memory as a resource that requires allocation decisions: what must be in context for the current step, what from prior steps is still relevant, and what can be summarized or evicted without losing the information it carried. These decisions are not made by the model by default — they must be enforced by the orchestration layer. The relationship between context management and orchestration topology is explored further in RES/ORC.
Episodic memory is the record of what happened — specific past task executions, what was tried, what succeeded, what failed. It is the most directly useful memory type for improving agent performance across sessions and the hardest to implement correctly. The storage problem is secondary to the retrieval problem. An episodic record that cannot be retrieved when it is relevant — or that is retrieved when it is not — provides no continuity benefit and may actively degrade performance by surfacing past failures as candidate approaches.
The diagram below shows the retention characteristics of all four memory types over time — specifically how accessibility evolves without intervention, and how each type responds differently to retrieval events and compression operations.
Fig. RES/MEM — Memory Retention Topology · Accessibility over time by type · retrieval events extend episodic availability · procedural near-flat without versioned update
The vector store retrieval pattern that underlies most episodic memory implementations searches by semantic similarity against the query. This is appropriate for semantic memory — finding domain knowledge adjacent to a question. It is inappropriate as the primary retrieval mechanism for episodic memory, because semantic similarity does not encode temporal recency, task context, or outcome polarity. A past failure that is semantically adjacent to a current task will retrieve as strongly as a past success — and will be presented to the agent as a relevant memory without distinguishing which outcome it represents.
Episodic records need structure that semantic retrieval alone cannot provide: outcome field (success / failure / partial), task context, recency weight, and a mechanism for explicitly suppressing records that represent deprecated approaches. The retrieval system in RES/RAG treats retrieval quality as a function of indexing and query design — those principles apply here, but episodic retrieval adds polarity and recency as additional required dimensions.
Semantic memory is general domain knowledge — facts, relationships, and concepts that do not belong to any specific task execution. Vector stores are the dominant implementation. The retrieval quality problem for semantic memory is well-studied: chunking strategy, embedding model choice, query formulation, and re-ranking all affect what is returned. The interference problem between semantic and episodic memory is less studied and more structurally problematic.
When episodic records and semantic knowledge share a single vector store, a query for domain knowledge surfaces episodic records that are semantically adjacent but contextually wrong. A query for the standard approach to a task retrieves records of past attempts at that task alongside authoritative documentation — without distinguishing which is domain knowledge and which is execution history. The agent receives both and has no reliable mechanism for identifying which is which unless the records were structured to encode their provenance at write time.
Store separation is the structural fix: episodic and semantic content in distinct stores, with a routing layer that classifies query intent before executing retrieval. The second brain documented in SYS-01 uses a single store with provenance metadata as an interim approach. Whether provenance filtering at query time produces equivalent isolation to store separation has not been measured. The routing classification problem — determining at query time whether a query is episodic or semantic — remains open.
Procedural memory is how to do things — the steps, the sequence, the decision points. In agent systems it encodes most directly into system prompts, few-shot examples, and workflow templates. It is the slowest-changing memory type and the most operationally consequential when it is wrong.
A wrong procedure that is encoded in a system prompt is wrong on every call until it is corrected. There is no decay mechanism, no retrieval that produces a correct alternative. The agent executes the wrong procedure reliably — which means the failure is systematic rather than intermittent, and systematic failures are harder to detect because they look like consistent behavior rather than anomalies. The evaluation system at SYS-03 catches procedure regressions only if a fixture exists that represents the correct behavior — which requires knowing what correct looks like independently of the procedure being tested.
Procedural memory requires version control, not just storage. The version history is what makes behavioral changes diagnosable: an agent that behaves differently after a system prompt update can be compared against the prior version to distinguish intended change from regression. Without version history, diagnosing a procedural regression requires reconstructing the prior procedure from behavioral evidence — which is expensive and often impossible if the change was subtle. The relationship between durable execution and procedural memory version consistency during replay is a direct open question from RES/ORC.
Forgetting is not cleanup. In biological memory, forgetting is active and adaptive — what is not rehearsed decays, freeing capacity and reducing interference from outdated information. In agent memory systems, forgetting is treated as a secondary concern: TTL expiry for storage cost management, manual pruning when retrieval quality degrades, capacity limits enforced by the infrastructure rather than by a retention policy.
Three forgetting strategies are implementable now. Compression summarizes an episodic record or a set of working memory contents into a shorter representation, preserving the essential outcome while reducing token count. Compression is lossy — the specificity of the original is not recoverable from the summary. Which records to compress and at what fidelity loss is a policy question that must be answered before the system encounters storage pressure, not under it.
Eviction removes records entirely based on a criterion: age, retrieval frequency, outcome polarity (past failures that have been superseded), or explicit deprecation by a human reviewer. Eviction is irreversible without a backup mechanism. The criterion must be specified such that eviction removes content that genuinely degrades retrieval quality rather than content that is infrequently retrieved but still correct.
TTL-based expiry applies a time-to-live to records at write time. It is the most operationally simple strategy and the least semantically informed — an episodic record written two years ago may be more relevant to a current task than one written last week, and TTL does not distinguish. TTL is appropriate for records whose validity is genuinely time-bound: API responses, market data, session state. It is inappropriate as the primary forgetting strategy for episodic knowledge whose relevance is task-dependent rather than time-dependent.
These are the questions where current understanding runs out. They are first-class content, not an appendix.
What is the correct routing primitive for distinguishing episodic from semantic queries before retrieval executes?
The interference problem requires a routing layer that classifies query intent before deciding which store to search. The classification is itself a language understanding problem — the same surface query can be episodic ('what did I try last time') or semantic ('what is the standard approach') depending on context that the routing layer may not have. A routing layer that misclassifies will retrieve from the wrong store without surfacing an error. No production pattern exists for this classification.
At what context depth does passive accumulation reliably degrade instruction-following, and is the threshold stable across model families?
Empirical evidence suggests instruction-following degrades as context deepens, with the degradation accelerating past roughly 60–70% of context capacity. The threshold varies by model and is not stable across prompt structures — task instructions positioned at the end of a long context retrieve differently than those at the beginning. Whether context management strategies can reliably counteract this degradation or only delay it is not established.
How should procedural memory be structured so that a wrong procedure is diagnosable without the prior version?
A system prompt that was updated produces different agent behavior. If the prior version is not retained, diagnosing whether the behavioral change is a regression or an intended improvement requires reconstructing the prior prompt from behavioral evidence. Version control for procedural memory — structured diffs, rollback paths, behavioral regression tests tied to procedure versions — does not have an established implementation pattern for production agent systems.
What retention policy governs when to compress episodic records vs. retain them at full fidelity?
Compression reduces storage and retrieval cost. It also reduces the specificity of episodic records — a summarized failure record may not preserve the exact inputs that caused the failure. The retention policy question is: which episodic records require full-fidelity preservation, which are safe to compress, and what triggers the decision. No formal criterion exists. The current practice is to retain everything until storage pressure forces a decision, at which point the decision is made under pressure rather than by policy.
How does durable execution interact with procedural memory when a procedure is updated mid-replay?
Durable execution replays steps from the last checkpoint using cached results. If a procedure encoded in a system prompt is updated between the original execution and the replay, the replay runs against the new procedure with cached intermediate results from the old procedure. The checkpoint semantics guarantee step result preservation but not procedure consistency. Whether the replay produces correct behavior depends on whether the new procedure is compatible with the prior intermediate results — a compatibility question that has no verification mechanism.
The dependency order runs from retrieval routing outward. The interference problem cannot be resolved without first solving the routing classification problem — store separation requires a classifier that can distinguish episodic from semantic query intent, and that classifier does not yet exist in a reliable form. The compression retention policy cannot be specified without empirical data on which episodic records are retrieved under which task conditions — which requires operating the memory system long enough to accumulate a retrieval pattern that is representative of production use. The procedural memory versioning problem and the durable execution compatibility question both depend on building the version control infrastructure first — without a version history, the replay compatibility question cannot even be observed, let alone answered. Context degradation thresholds are the most immediately measurable open question and the one that requires the least infrastructure to investigate. The rest require the system to exist and run. That is what makes this research rather than engineering.
Continue reading