Context Engineering: Entropy Reduction Through Memory — From Theory to Practice

Github：https://github.com/Tencent/TencentDB-Agent-Memory

Introduction: Why Is Talking to AI So Exhausting?

Have you ever noticed that a conversation with an old friend requires nothing more than a glance, while a conversation with AI demands an entire essay?

The gap comes down to one word: Entropy. In information theory, entropy represents uncertainty. An old friend shares memories with you and can automatically fill in the blanks of your speech (low entropy); AI lacks this layer of context, causing enormous ambiguity to lurk behind every simple word (high entropy).

The essence of Context Engineering is to help machines complete this “fill-in-the-blank” exercise, bridging the vast cognitive gap between machines and human cognition.

We are no longer merely storing conversations mechanically. Instead, we actively capture lost contextual information by building layered Memory systems. This article combines cases from TencentDB Agent Memory to explore how we cantechnically “reduce entropy” for machines — transforming your AI applications from awkward repeated confirmations into intuitive, precise execution.

1. What Is “Entropy Reduction” in Context Engineering: From Formula to Intuition

1.1 What Is Entropy? Let’s Start with a “Bill” Example

Before diving into the technical definition, let’s look at an extremely simple everyday scenario.

You’re busy working and, without even turning around, you say to your long-time partner:

“Get Bill, and let’s meet at the usual place.”

Your partner nods and immediately gets it done — even though you gave zero details.

But if you said the same thing to an AI assistant, it would most likely hallucinate (or start talking nonsense):

• Who is “Bill”? Bill from the contact list? Bill from the next department? Or the new intern?
• When is “then”? 9 PM tonight? 10 PM? Or tomorrow morning?
• “The usual place” — where is that? A restaurant? The park pond? The office lounge?

This is the difference in Entropy.

Entropy represents uncertainty. The more possibilities and the harder the outcome is to predict, the higher the entropy. The more ordered and clear something is, the lower the entropy.

In the tacit understanding between humans, the specific referents of “that thing” and “Old Wang” are instantly collapsed into a single definite fact by shared memories, current context, and past habits — this is an extremely low-entropy interaction.

But from the machine’s perspective, a single sentence corresponds to thousands of possible combinations — this is an extremely high-entropy instruction.

Context Engineering is essentially us acting as translators, converting humans’ habitual high-entropy expressions into low-entropy instructions that machines can understand.

1.2 Context Engineering: Before Changing the “Model,” First Change the “Input”

If we reduce a large language model to its most abstract mathematical core, it is actually something very cold and mechanical:

y = f(x)

• f: The pre-trained LLM itself (parameters fixed, “IQ” determined);
• x: Everything fed to it — broadly speaking, this is the Context;
• y: The model’s response.

In application development, we often cannot easily change f (for most applications, the cost and risk are higher). All the “magic” we can control is largely about refining this x:

• The user’s current question / instruction;
• System Prompt: telling the model “who you are, what you’re doing”;
• Memory (cross-session memory): what happened before;
• Available tool list (Tools) and invocation methods;
• Retrieved snippets from external knowledge bases (RAG).

How to select, arrange, trade off, and compress these elements — that is Context Engineering.

In the paper *Context Engineering 2.0: The Context of Context Engineering*, the authors provide a more philosophically-flavored definition:

Context Engineering is the systematic process of designing and optimizing the collection, storage, management, and usage of context, aimed at bridging the Cognitive Gap between humans (carbon-based intelligence) and machines (silicon-based intelligence).

When you say something casually, can the AI understand “what you actually want”? When a business system feeds a pile of logs to the model, can it grasp the key points? This is not just a question of “model size” — it’s a question of “have we prepared the context properly.”

1.3 The Entropy Reduction Perspective: Bringing “Disorder” Down

Context Engineering 2.0 proposes a core viewpoint: context engineering is fundamentally an “entropy reduction process.”

The “entropy” here is not the thermodynamics formula from physics class, but refers to information “uncertainty”:

• Humans are high-entropy: emotions, implications, omissions, background knowledge, pauses in tone… much information goes unsaid because you assume the other party will “fill in the gaps.”
• Machines have low-entropy requirements: they cannot automatically “fill in gaps” — they only see a sequence of tokens. If we dump the real world into them as-is, there’s too much information, too much noise, and too poor a structure for them to find “the thing that actually matters for this task.”

Therefore, what Context Engineering does can be roughly summarized as three steps of entropy reduction:

1. Collect: Capture all potential information from humans and the environment without missing key clues.
2. Organize/Abstract: Structure, compress, and filter out noise from this information.
3. Use: Before a specific task, pick out only the small subset most relevant to this task from the above materials, and precisely feed it into the model.

From this perspective, that simple function can be drawn as a longer pipeline:

y = f(g(x))

• The outer f is the LLM algorithm;
• The inner g represents all your Context Engineering strategies: including knowledge base retrieval, tool invocation, and — what we’ll focus on today — Memory storage and recall strategies.

1.4 Core Formulas in CE 2.0:

To describe this process more scientifically, Context Engineering 2.0(<<Context Engineering 2.0: The Contextof Context Engineering>>) provides a set of formal definitions, which we can understand intuitively:

1. View the world as a collection of Entities: people, applications, tools, environments, Memory modules, etc.
2. Each entity has a set of characterizing information

such as “who the user is,” “what time it is,” “what the current configuration is.”

3. In a specific interaction, we select only the relevant entity set

aggregate their characterizing information, and obtain the Context for this interaction:

A simple explanation of this formula:

The Context for one task = all “relevant roles” + the union of their “current states.”

And Memory is the most easily missing and hardest to maintain part in this formula:

1.5 Memory’s Role: The “Entropy Reducer” on the Human Side

Within this vast CE framework, Context contains many things: system instructions, tool outputs, knowledge base documents… So what exactly does Memory do?

If we say a Knowledge Base helps the model “understand the world” (general world knowledge entropy reduction), then Memory is more like helping the model “understand this specific person, this specific long-term task.”

Memory is the entropy reduction tool in the Context Engineering pipeline that specifically faces “person + time.” Its core mission is to compress the massive high-entropy fragments about “this person, this relationship, this task thread” into a small amount of ordered, reusable, structured context across the long dimension of time.

• Without Memory, AI lives forever in the “present” — every conversation is a high-entropy cold start.
• With Memory, AI gains a “past” — enabling it to understand, with extremely low entropy, what that simple “OK” from you actually means.

2. What Is Memory: An “Intent Recognition + Information Supplementation” Entropy Reduction Tool

If we say LLMs are super-brains with extraordinary IQs but only 7-second memories, then Memory is the external “notebook” we attach to them. But within the precision pipeline of Context Engineering, Memory is far more than simply “storing a few chat records.”

2.1 Redefining Memory from a CE Perspective: Not Just Storage, But “Prediction”

In traditional understanding, Memory is like a warehouse — stuffing what users said in and digging it out when needed. But in the Context Engineering pipeline, Memory is a dynamic context layer specifically oriented toward “person + long-term tasks.”

Its core purpose is quite simple — doing two things to reduce interactional entropy:

① Intent Reduction: Helping the model understand you faster

This is like the convenience store owner downstairs:

• Without Memory (high-entropy state): Every time you go, you have to say: “Boss, I want a pack of soft reds, not the hard ones, and a lighter.”
• With Memory (entropy-reduced state): You just walk in and nod (minimal token input), and the boss hands over the cigarettes and lighter.

Memory’s role here is to use historical data to instantly collapse an ambiguous “nod” into a precise “buy cigarettes” intent.

② Information Supplementation: Restrained background filling

When you’re discussing “how to modify this version of code,” Memory doesn’t need to stuff all 100,000 lines of the project into the Prompt (that would be context pollution). It only needs to hand over a note at the critical moment, like a seasoned colleague: “Note — the previous authentication logic in this module uses OAuth 2.0. Don’t break it.”

The key is “restraint”: only supplement the one piece of high-value background that the current decision lacks most. One extra bit is noise.

2.2 From the Demand Side: What Kind of Memory Do We Actually Want?

If we ask users or business stakeholders “what memory features do you want,” their answers are usually scattered. When we peel these needs open like an onion, we find they correspond to 5 different dimensions of entropy reduction desire:

① Consistent role-playing and instructions that don’t get forgotten

• Pain point: Many agents forget whether they’re “Socrates” or a “catgirl” mid-conversation, or forget “please respond only in JSON format” after 20 rounds.
• Essential need: Users want “Persona” and “Key Constraints” to become long-held, low-entropy context. Users shouldn’t have to manually restart “hey, don’t forget your identity” every few turns.

② Personalization and user profiles: Getting to know you better over time

• Pain point: One-size-fits-all AI, like an assembly-line worker, without warmth.
• Essential need:

Education: The AI teacher doesn’t just see “this question is wrong” but recalls — “Xiao Ming, you’ve gotten this wrong three times now, always because of sign confusion.” That’s personalized teaching.

Marketing/Service: Not pushy sales, but memory-based — “The cat food you bought last time should be running out, right?” But this requires extreme explainability — being understanding without making users feel “over-surveilled.”

③ Context continuity: Long tasks without fragmentation

• Pain point: In hours-long coding sessions or complex after-sales processes, users most fear the “switching agents means explaining everything from scratch” fragmentation.
• Essential need: This is a relay race of “collect → compress/abstract → reuse based on summaries.” Memory needs to carry the momentum and State of the previous task round like a baton, passing it to the next round, so users feel no “gap.”

④ Information storage and mining: Turning interactions into “reusable assets”

• Pain point: Conversations flow away — once the chat ends, it’s gone.
• Essential need: B2B clients value this greatly. Conversations, file modifications, click behaviors — these are all high-value assets.
• Memory’s role is similar to “tagging” in recommendation systems, but more advanced. Recommendation systems process structured click streams; Memory processes unstructured text.
• We not only leverage existing memories to serve users, but also mine new profile information from the service process, forming a data closed loop.

⑤ Making AI more “human”: Remembering relationships, not just facts

• Pain point: AI remembers “your name is Bob” but can’t remember “Bob has been under a lot of work pressure lately and needs encouragement.”
• Essential need: Human relationship building depends on accumulating emotional resonance. Memory needs to record not just Facts, but Relationships. It must remember your stance, your emotional curve, your unexpressed concerns, upgrading interactions from “Q&A” to “ongoing interpersonal relationships.”

2.3 Summary: Memory’s Role in CE

One-sentence summary: Memory is the “human-side entropy reducer” in the Context Engineering pipeline. Its mission is to compress the massive high-entropy fragments about “this person, this relationship, this task thread” into a small amount of ordered, reusable, structured context in real-time — paving the road for AI to understand the user before it even speaks.

3. Claude’s Secret: “Memory Entropy Reduction Focus” Across Chat & Code

Why do many developers and users feel that Claude seems “smarter” when handling complex contexts, even though model parameters across vendors are similar?

The secret, beyond the model’s reasoning ability (IQ), lies in Context Engineering strategy. If we carefully deconstruct Anthropic’s designs for Chatbot and Claude Code, we discover two distinctly different “entropy reduction movements.” This provides an excellent reference for our Memory design.

3.1 Chat Mode: Entropy Reduction Centered on “Person and Conversation”

In the Chatbot scenario, Claude’s memory design clearly revolves around entropy reduction for “person + long-term conversation.”

From official documentation and recent upgrades, Claude’s Memory by default records not scattered sentences, but summarizes around several categories of stable information: user identity and role, preferred language/writing style, ongoing projects, client needs, and team workflows.

These memories are organized into different “spaces”: personal space, workspace, project space — each maintaining its own memories separately to avoid carrying Project A’s context into Project B. Users can toggle memory on/off globally in settings, explicitly ask Claude to remember/forget specific information, or even enable “incognito mode” sessions without memory.

From the “entropy reduction” perspective, Claude Chat memory primarily reduces three types of uncertainty:

• Who is speaking: Which user am I, what role am I in (student / PM / developer / client), what’s my approximate knowledge background.
• Which long-term topic thread are we on: Which project, course, OKR set, what consensus was previously reached.
• What style and stance should this response adopt: Lists or not, code or not, popular science tone or formal proposal.

In practice: project spaces + historical conversation summaries + editable memory together form a relatively stable “conversation shell.” Even if you come back days later asking “where did we leave off?” or “continue writing in the previous style,” the model can continue the same “you” and narrative thread within this shell without requiring you to re-introduce yourself.

In other words, memory in the Chat scenario is more like performing entropy reduction on “characters and storylines” — compressing “who this person is, what’s the history of this relationship” into a few persistent tags and summaries, lowering the cost of intent recognition each time.

3.2 Code Mode: Entropy Reduction Centered on “Repository and Task”

With Claude Code, the focus of memory shifts entirely: from “remembering you as a person” to “remembering this task thread.”

Claude Code’s documentation and practical experience can be summarized into several layers of memory sources:

• Persistent instruction files within the project: The model’s “notebook” — e.g., CLAUDE.md, specifying project structure, development rules, "don't touch" directories, quality standards, etc., serving as the long-term memory skeleton.
• Claude Code’s own cross-session memory: Remembering frequently used commands, preferred code style, fixed workflow prompts — but these are more “work environment habits” rather than Chat’s “entire life profile.”

In the Coding scenario, the entropy being reduced by memory is primarily three things:

1. Current repo structure and boundaries: Where important modules are, dependency relationships, which directories are generated artifacts that can’t be modified.
2. Current task progress and context: What steps have been completed in this refactoring/debugging, what approaches were tried, what the current hypothesis is.
3. Constraints and red lines: Architectural principles that must be followed, testing requirements, security/compliance restrictions — typically written in CLAUDE.md or project config.

To prevent “context pollution” and “increasingly muddled writing,” Claude Code also emphasizes context isolation: through MCP configuration, sub-agents, and slash commands, pulled into the main session only when needed -avoiding stuffing everything into a single 20k-token mega-conversation. This is essentially alsoentropy reduction: letting each sub-task face only its own small “world” rather than the entire repository’s chaotic state.

3.3 Implications for Our Memory Design

Through this comparison, we can see that Memory is by no means a universal “just store it and you’re done” feature, but rather a highly scenario-specific proposition:

• Chatbot scenario: Needs Intent Memory, combating entropy increase in persona and topic continuity.
• Coding scenario: Needs State Memory, combating entropy increase in system complexity and task progress.

The golden rule of Memory design is not “more memory = smarter” but rather precisely identifying “which type of uncertainty (entropy) is most lethal in the current scenario.”

Is the fear that it forgets “who I am”? Or that it messes up “code dependencies”?

Only based on this “scenario-specific, layer-specific” entropy reduction thinking can we build a more efficient Memory framework.

4. “Attention before Attention”: Two Major Entropy Reduction Challenges in Memory Design

This is a hands-on summary brought back from the “trenches.”

During Memory product development, we found that theoretical Context Engineering 2.0 hits countless concrete “walls” at the engineering implementation stage. These walls forced us to acknowledge a fact: Memory is not just database reads and writes — it’s an “attention defense battle” under limited computing power and latency budgets.

The core mechanism of LLMs is Self-Attention, which determines where the model focuses its computing power within the context. But what the Memory system must do is perform a round of filtering before tokens enter the model.

These challenges are called “Attention before Attention.” This process is extremely dangerous: done well, it’s a brilliant assist; done poorly, Memory itself becomes the biggest source of noise.

4.1 Storage Layer Challenges: Data Integrity vs. Structural Constraints

In Memory system design, we pursue ultimate personalization — theoretically storing all user-related traces. However, engineering practice proves that if the storage stage lacks structure (Schema), simply increasing storage volume is itself introducing high entropy — manufacturing system uncertainty.

We must strike a balance between preserving raw details (avoiding sparse signals) and preventing critical information loss (avoiding brute-force compression). Three critical storage traps exist in practice:

Trap 1: Data Fragmentation (Over-Sparsity)

• Symptom: The system mechanically records every conversation round’s raw text (Event) but loses the high-level task structure (Task Schema).
• System Impact: Memory becomes an unstructured collection of short texts. During Retrieval, the search engine must process thousands of low-relevance short texts. Due to lacking a complete context chain, the LLM cannot piece together the full picture.
• Design Principle: Storage must include structured Schema. Storage without “structure” is merely data hoarding, not usable memory sediment.

Trap 2: Over-Abstraction

• Symptom: To save tokens and storage space, the Memory system implements brute-force summarization, compressing massive interactions (e.g., 100 conversation rounds) into conclusions.
• System Impact: Critical intermediate state information is discarded. In long-timeline tasks like coding or teaching, “the user tried this approach but it failed” process data is often more important than final conclusions. Information loss causes AI to suggest previously-failed approaches, leading to capability degradation.
• Design Principle: Summarization is entropy reduction, but over-summarization is information loss. Summary granularity must ensure preservation of task intermediate-process granularity.

Trap 3: Memory Island Effect (Isolation)

• Symptom: Individual memory Events lack contextual background and are not attached to specific Scene IDs or Task IDs.
• System Impact: Causes Context Mismatch. For example, incorrectly recalling “coding” preferences during a “cooking” scenario. This typical entropy increase directly causes Agent persona drift and user experience interruption.
• Design Principle: Memory must have “anchors.” Memory isolation is the bottom line for recall accuracy. Scattered conversations should be aggregated into “blocks with contextual boundaries” to combat “memory islands.”

4.2 Recall Challenges: Balancing “Rejecting Pollution” and “Rejecting Stupidity”

If storage is laying the foundation, recall is walking a tightrope. The most intense business feedback we received during implementation focuses here — practical recall often faces an “impossible triangle” of effectiveness, speed, and generalization ability.

Challenge 1: Rejecting Context Pollution

• Key Conflict: Business stakeholders, especially B2B clients, universally emphasize: “Better to under-recall than to pollute the entire context.”
• Consequence: If incorrect historical information is recalled (e.g., bringing an old incorrect price into a new quotation), this doesn’t reduce entropy — it becomes serious misguidance. This is a classic entropy-increasing effect.
• Design Conclusion: High-availability Memory recall results must not only be relevant but also ensure low ambiguity (Unambiguous). Recall strategy must prioritize pollution prevention above all.

Challenge 2: Limitations of RAG-Style Retrieval

• Mechanism Trap: If Memory recall over-relies on vector similarity — focusing only on the current query’s surface-level similarity — the system easily falls into a local optimum.
• Phenomenon: When users express vague intent (e.g., “it’s that old problem again”), traditional retrieval only matches “old problem” literally, without understanding it refers to “the database deadlock discussed last Tuesday.”
• Engineering Assessment: This reflects Memory’s lack of multi-hop logical reasoning capability — it remembers much data but lacks the ability to judge based on context and deep associations.

Challenge 3: Engineering Difficulty of Multi-Hop Recall

• Engineering Difficulty: Truly valuable memories are often multi-step indirectly related (similar to multi-hop QA).
• Latency Challenge: To avoid slowing down the overall conversational UX, we must complete complex recall pipelines within an extremely tight 500ms latency budget: coarse ranking → fine ranking → even small-scale Agentic Search. This is not just an algorithmic challenge but an extreme test of engineering architecture.

Challenge 4: Low-Entropy Processing of Recalled Fragments

• Core Requirement: The recall task isn’t just about bringing historical data out — it must ensure that what’s brought out is in a form the model can easily digest.
• Practical Requirement: Recalled fragments must be spliced into context in a clear, concise, context-annotated form. What the model receives shouldn’t be “a few inexplicably appearing old sentences” but must be structured prompts with background.

Core Summary: The recall stage’s task is not to prove “I remembered” but to “select, within an extremely short time, those few memories that can produce real entropy reduction effects on the current decision.”

4.3 The Deeper Meaning of “Attention before Attention”

The LLM’s internal Attention mechanism, while powerful, is expensive (O(n²) complexity) and limited (Context Window). It performs secondary information allocation after tokens have entered the model.

The Memory system needs to perform a round of “pre-attention allocation” before tokens enter the model.

• LLM’s Attention: Deciding who to stare at in a room full of people.
• Memory’s Attention: Deciding which people to let into the room.

A good Memory product’s core job is playing this “gatekeeper” well: preparing what the model truly needs within 500ms, blocking noise, and letting signals through. This is the most extreme entropy reduction manifestation of Context Engineering on the engineering side.

5. Memory’s Layered Architecture: Engineering Entropy Reduction in Practice

In the previous chapter, we outlined Memory design’s “impossible triangle”: storing comprehensively yet recalling precisely, understanding deeply yet operating quickly. Facing these contradictions, simply stacking Vector DBs cannot solve the problem.

Our recent explored solution leverages information’s “hierarchical nature.” Drawing from how the human brain operates, we designed a “three-layer storage + three-layer recall” architecture that balances information retention and usage efficiency, engineering the entropy reduction process of Context Engineering.

5.1 Three-Layer Storage: From Event → Scene → Record

Rather than simply “dumping conversations into a database,” we act like librarians, performing three-level “purification” of information. Events ensure no detail is lost, MemoryBooks ensure tasks have context, and Records distill long-term cognition into high-density tags. The three layers collaborate to progressively clean “originally chaotic data streams” into low-entropy context.

Layer 1: Event (Raw Layer) — Reducing “Original Context Loss” Entropy

• Role: The lowest-level “black box recorder.” It records in chronological order who (Source) said/did what, and when (Time).
• Entropy reduction logic: Combating “over-abstraction”: As mentioned previously, excessive summarization causes information loss. The Event layer performs no lossy compression, only ensuring “nothing is lost.” It’s the Source of Truth for all higher-level memories, ensuring detailed traceability when the model needs it.

Layer 2: Scene (Context Layer) — Reducing “Task Fragmentation” Entropy

• Role: Our core design for combating “memory islands.” We aggregate Events scattered across the timeline into MemoryBooks organized by Task or Scene.
• Structure: Each MemoryBook contains multiple Scenes (e.g., “2025–03–01 Math Class: Adding and Subtracting Fractions”). Each Scene includes key original text fragments and stage-level conclusions extracted from them.
• Entropy reduction logic: Combating “context mismatch”: It compresses scattered conversations into “blocks with contextual boundaries.” When recalled in the future, the model receives not a sentence out of nowhere, but the background of “which class, which project discussion” — completely solving the “cooking advice suddenly appearing during coding” cross-contamination problem.

Layer 3: Record (Archive Layer) — Reducing “Long-Term Cognitive Ambiguity” Entropy

• Role: The highest information-density layer, similar to “archive cards.” We extract high-value information from massive MemoryBooks across scenes: user profiles, stable preferences, key timeline events.
• Flexibility: Schema supports business customization (education version stores learning progress; e-commerce version stores brand preferences).
• Entropy reduction logic: Combating “high entropy in intent recognition”: Compresses a user’s lifetime of chaotic interactions into a structured card of just a few dozen KB. Within milliseconds at the start of a conversation, the model can achieve “intent calibration” through this card.

5.2 Three-Layer Recall: Completing “Attention Routing” Within 500ms

Good storage exists for good usage. On the recall side, we designed a routing mechanism from near to far, fast to deep, addressing the tension between “latency” and “accuracy.”

Layer 1: Current Conversation — Direct Event Context Usage

• Strategy: Prioritize splicing the most recent few rounds of Event raw text directly into the Context window.
• Solving: The “goldfish memory” problem. Ensures the model has absolutely precise short-term memory of “what was just said,” preventing users from repeatedly restating needs within a dozen rounds.

Layer 2: Task Context — Awakening MemoryBooks, Cross-Time “Relay”

• Strategy: When conversations cross rounds or even days, the system awakens dormant MemoryBooks based on Session ID or Task ID.
• Solving: The “long task fragmentation” problem.
• Key engineering innovation — Async Pre-fetch: To combat the 500ms latency deadline, MemoryBooks have “predictive” capability. They continuously “watch the current task” — before the user even asks, the backend may have already pre-fetched relevant error records based on the current scene (e.g., “entering Xiao Ming’s final review mode”). This is true Attention before Attention: trading space for time, making real-time recall lighter while the model remains “prepared.”

Layer 3: Long-Term Archive — Quick Retrieval + Agentic Search Deep Dive

• Strategy A (Fast path): Within extremely short time, use vector + keyword dual-path retrieval to extract long-term memories most relevant to the current Query from the Record layer (e.g., user’s preferred name, dietary restrictions).
• Strategy B (Deep thinking path — Agentic Search): For complex questions like “What was I doing this day last year?” or “Summarize my coding style over the past month,” pure RAG falls into an “information cocoon.” This triggers Agentic Search: a small internal Agent performs multiple rounds of retrieval + logical reasoning. Though slightly slower, it solves multi-hop logic problems, serving as the MemoryBook’s “behind-the-scenes strategist.”

5.3 Metrics and Observations: Real-World Response to Entropy Reduction Design

This “three-layer storage + three-layer usage” architecture brought significant quality improvements in actual internal evaluations:

① Temporal Memory Breakthrough

When facing recall tasks heavily dependent on temporal cues like “What was I doing this day last year?”, the accuracy using only the base model was approximately 45% (prone to hallucination or forgetting); after integrating layered Memory, accuracy increased to 88%. This proves structured storage effectively suppresses “temporal entropy.”

② “Load Reduction” for Long-Chain Tasks

In complex Coding or teaching scenarios, the number of times users were forced to “repeatedly explain requirements and constraints” dropped significantly. Memory successfully served as the “task skeleton maintainer,” keeping AI behavior logically consistent across sessions spanning days.

③ Persona Stability

“Persona confusion” and “cross-session fragmentation” issues were significantly alleviated. Memory is no longer scattered recall fragments but the backbone supporting Agents in maintaining long-term “personas.”

6. Conclusion: From Entropy-Reducing Memory to Our “Digital Presence”

When we step back from the details of code and architecture to re-examine the evolution of Context Engineering, we find that what we’re building is not merely a smarter database — it’s the beginning of deep cognitive coupling between humans and machines.

6.1 The Future of CE: The Entropy Reducer Will Migrate from “Human” to “Machine”

The Context Engineering 2.0 paper predicted that as machine intelligence improves, the main force of entropy reduction will gradually shift.

In the current CE 1.0 and 2.0 stages, Memory, Knowledge Bases, and Tools are essentially “scaffolding” built by human engineers for models. Through precise Prompt design and layered storage strategies, we manually help models that don’t yet possess complete minds filter noise and extract intent.

In the future CE 3.0 and 4.0 eras, as models’ own reasoning capabilities and long-window technologies break through, machines will possess the ability to autonomously perceive, infer, and construct context. By then, Memory systems will no longer be merely passive storage containers but will evolve into the model’s active “attention organs” — capable of autonomously performing entropy reduction in massive data, capturing implicit associations that humans haven’t even noticed.

6.2 Digital Presence: Memory Shapes a “Low-Entropy You”

Some commentators call the ultimate Context Engineering vision a form of “Digital Presence.”

When a Memory system undergoes sufficient engineering design, what it carries is no longer cold logs, but a complete mapping of the user’s knowledge structure, behavioral habits, decision-making style, and emotional preferences. In this sense, Memory is gradually shaping a “low-entropy model” of the user.

This model is both the cornerstone of AI understanding users and the user’s projection in the digital world. It not only helps AI provide precise service “here and now” but also accumulates the user’s “presence” across time. Memory transforms AI interaction from discrete computation into continuous companionship; it makes users feel not the coldness of a tool, but the resonance of being understood.

6.3 Some Thoughts Beyond Work

Today, as we rigorously design Memory layering strategies and work to “reduce entropy” for machines, we are on one hand making AI interactions smoother and more efficient. But from a deeper scale, we may be building a computable “Self-Narrative.”

Perhaps years from now, when we look back at today’s Context Engineering, we’ll find that we weren’t just providing context to models — we were also building a stable characterization of “who I am” outward.

How to find balance between “making machines more useful” and “protecting human openness as subjects” may be the true philosophical proposition of the next-generation CE 3.0 era.

7. Key Takeaways: Three Questions to Answer Before Building Your Own Memory System

The following checklist is designed to help you evaluate the rationality and necessity of your system design from the “entropy reduction” perspective.

7.1 Identify Your Core Goal: Which Type of “Entropy” Do You Most Want to Reduce?

The essence of Context Engineering is eliminating uncertainty. But with limited resources, we cannot solve all problems simultaneously. First, clarify which category your business pain point primarily falls into:

① Intent Entropy

• Pain point: The model struggles to remember user identity or cannot accurately understand the implicit needs behind instructions, leading to repeated confirmation interactions.
• Optimization target: Help the model understand “who the user is,” “what their preferences are,” and “the current subtext” faster and with less ambiguity.
• Applicable scenarios: Emotional companion assistants, personal assistants, personalized recommendation Agents.

② Knowledge Entropy

• Pain point: Business data is vast (documents, logs, databases), and AI easily gets lost during retrieval or recalls large amounts of irrelevant noise.
• Optimization target: Efficiently locate “the fragment this task truly needs” within massive information stacks, achieving high-precision information supply.
• Applicable scenarios: Enterprise search, legal and medical document Q&A, after-sales knowledge bases.

③ Process Entropy

• Pain point: In long-chain tasks, AI easily forgets initial constraints or collapses logically after multiple rounds, preventing task closure.
• Optimization target: Prevent mid-stream “state confusion” and “conclusion oscillation,” ensuring task progress continues forward like an arrow.
• Applicable scenarios: Coding copilots, AI tutors, complex ticket processing systems.

7.2 Product & Engineering Design Checklist

During the system architecture design phase, check whether your solution addresses these key questions to avoid engineering-level entropy increase:

Memory Hierarchy

• Consideration: Does the system distinguish between Event (raw facts), Session/Scene (current task context), and Profile (long-term portrait)?
• Recommendation: Avoid flat storage of all information. Memory stores lacking hierarchy easily lead to retrieval chaos — establish layered storage and recall mechanisms.

Namespace & Isolation

• Consideration: Are memories from different projects, users, or business lines strictly isolated?
• Recommendation: Memory isolation is the bottom line for data security and recall accuracy — cross-project or cross-user memory pollution must be prevented.

Storage vs. Usage Strategy

• Consideration: Is there a distinction between “hot memory” and “cold memory”?
• Recommendation: High-priority information (core persona, task red lines) should be placed in the System Prompt or context head for instant access; low-frequency information (historical casual chat details) should be stored in RAG or MemoryBooks for on-demand retrieval.

Entropy Monitoring & Feedback

• Consideration: When AI produces false memories causing “hallucination” or “persona collapse,” is there a mechanism to capture and correct it?
• Recommendation: Design closed-loop feedback mechanisms. If users indicate memory errors, the system must capture the signal and trigger corrections, preventing erroneous information from accumulating.

7.3 Business Value Translation: How to Communicate Memory’s Value

When thinking about the value of Memory systems, we shouldn’t stop at “developing a cool feature” but should translate it into concrete scenario value:

• Instead of saying: “We’re launching a vector storage feature.”
• Better to say: “We’re reducing uncertainty in specific scenarios.” For example:
• Teachers no longer need to repeatedly document each student’s learning history;
• Customer service agents receiving transferred tickets no longer need to re-read lengthy historical records;
• Users no longer need to re-state their identity and needs every three conversation rounds.

Of course, building an ideal “low-entropy” system from scratch is no easy task. If you’re exploring how to efficiently implement the layered design described above, Tencent Cloud Database Memory may serve as a mature practical reference.

We are using a “short-medium-long” three-layer architecture to deconstruct complex memory: leveraging medium-term MemoryBooks to maintain task continuity (addressing process entropy), accumulating user preferences through long-term profiles (alleviating intent entropy), and pairing logic-capable Agentic Search with fast-recall paths for ultra-fast response. We aim to provide users with more precise retrieval support in knowledge entropy governance.

If you wish to learn more about the product, you can click on the product’s GitHub page for further details：https://github.com/Tencent/TencentDB-Agent-Memory