Physical Address
304 North Cardinal St.
Dorchester Center, MA 02124
When an enterprise LLM retrieves a product name, technical specification, or standard contract clause, it uses expensive GPU computing designed for complex reasoning, just to access static information. This happens millions of times a day. Each search wastes cycles and inflates infrastructure costs.
DeepSeek recently published research on "conditional memory" directly addresses this architectural limitation. The work presents Engram, a module that separates static model retrieval from dynamic reasoning. It provides results that challenge assumptions about the actual usefulness of memory in neural networks. The document was co-authored by Deep search founder Liang Wenfeng.
Through systematic experiments, DeepSeek found the optimal balance between computation and memory with 75% of the sparse model capacity allocated to dynamic reasoning and 25% to static searches. This memory system improved reasoning more than knowledge retrieval.
Complex reasoning tests jumped from 70% to 74% accuracy, while knowledge-focused tests improved from 57% to 61%. These improvements come from tests such as Big-Bench Hard, ARC-Challenge and MMLU.
The research comes as companies face increasing pressure to deploy more capable AI systems while dealing with GPU memory constraints and infrastructure costs. DeepSeek’s approach offers a potential path forward by fundamentally rethinking how models should be structured.
How conditional memory solves a different problem than agentic memory and RAG
Agent memory systems, sometimes called contextual memory, such as Hindsight, MemoryOSOr Memp — focus on episodic memory. They store records of past conversations, user preferences and interaction history. These systems help agents maintain context throughout sessions and learn from their experience. But they are external to the direct transmission of the model and do not optimize how the model internally handles static linguistic patterns.
For Chris Latimer, founder and CEO of Vectorize, which developed Hindsight, the conditional memory approach used in Engram solves a different problem than AI agentic memory.
"This does not solve the problem of connecting agents to external memory like conversation histories and knowledge stores," Latimer told VentureBeat. "It is more focused on optimizing the performance of smaller models and making more use of scarce GPU resources."
Conditional memory solves a fundamental problem: transformers do not have a native knowledge search primitive. When processing text, they must simulate the retrieval of static patterns through expensive neural computation across multiple layers. These templates include named entities, technical terminology, and common expressions.
The DeepSeek article illustrates this with a concrete example. Recognize "Diana, Princess of Wales" requires consuming multiple layers of attention and feedback networks to gradually compose features. The model essentially uses deep, dynamic logic circuits to perform what should be a simple lookup in a hash table. It’s like using a calculator to remember your phone number rather than just looking it up.
"The problem is that Transformer does not have a “native knowledge search” capability," write the researchers. "Many tasks that should be solved in O(1) time, such as retrieval, must be “simulated for retrieval” via a large amount of computation, which is very inefficient."
How conditional memory works
Engram presents "conditional memory" to work alongside the conditional MoE calculation.
The mechanism is simple. The module takes sequences of two to three tokens and uses hash functions to look them up in a massive embedding table. Retrieval takes place at constant time, regardless of the size of the table.
But the retrieved models need to be filtered. A hash search for "Apple" could collide with unrelated content, or the word could mean the fruit rather than the business. Engram solves this problem with a trigger mechanism. The model’s current understanding of the context (accumulated through previous layers of attention) acts as a filter. If the retrieved memory contradicts the current context, the gate deletes it. If he comes in, the door lets him through.
The module is not applied to each layer. Strategic placement balances performance gains against system latency.
This dual-system design raises a crucial question: How much capacity should each have? DeepSeek’s main conclusion: the optimal split is 75-80% compute and 20-25% memory. Testing revealed that pure MoE (100% calculation) was suboptimal. Too many calculations waste the depth of static model reconstruction; too much memory makes you lose the ability to reason.
Infrastructure Efficiency: GPU Memory Bypass
Perhaps Engram’s most pragmatic contribution is his infrastructure-aware design. Unlike dynamic routing in MoE, which depends on runtime hidden states, Engram’s recovery indices depend only on input token sequences. This deterministic nature allows for a prefetching and overlapping strategy.
"The challenge is that GPU memory is limited and expensive, so using larger models becomes expensive and more difficult to deploy." Latimer said. "The clever idea behind Engram is to keep the main model on the GPU, but offload much of the model’s stored information into separate memory on regular RAM, which the model can use just in time."
During inference, the system can asynchronously retrieve embeddings from the host processor’s memory via PCIe. This happens while the GPU calculates previous transformer blocks. Strategic layer placement leverages the computation of early layers as a buffer to hide communication latency.
The researchers demonstrated this with a fully unloaded 100 B parameter integration table to accommodate the DRAM. They obtained debit penalties of less than 3%. This decoupling of storage and compute addresses a critical business constraint, as high-bandwidth GPU memory remains expensive and rare.
What this means for enterprise AI deployment
For companies evaluating AI infrastructure strategies, DeepSeek results suggest several actionable insights:
1. Hybrid architectures outperform pure approaches. The 75/25 allocation law states that optimal models should distribute sparse capacity between compute and memory.
2. Infrastructure costs can shift from GPU to memory. If Engram-type architectures prove viable in production, infrastructure investment models could change. The ability to store more than 100B of parameters in CPU memory with minimal overhead suggests that memory-rich, compute-moderate configurations can deliver better performance per dollar than pure GPU scaling.
3. Improvements in reasoning exceed gains in knowledge. The surprising finding that reasoning is more beneficial than knowledge retrieval suggests that the value of memory extends beyond obvious use cases.
For companies leading AI adoption, Engram demonstrates that the next frontier may not simply be bigger models. These are smarter architectural choices that respect the fundamental distinction between static knowledge and dynamic reasoning. Research suggests that optimal AI systems will increasingly resemble hybrid architectures.
Organizations waiting to adopt AI later in the cycle should check whether major model vendors are integrating conditional memory principles into their architectures. If the 75/25 allocation law applies across scales and domains, the next generation of baseline models could deliver significantly better reasoning performance at lower infrastructure costs.
