Building Intelligent Applications needs Memory

TL:DR;

Memory is quickly becoming the differentiator in intelligent application development. Today’s leading AI applications track what’s happened (awareness), learn from interactions (introspection), and adapt to each user (personability). Each capability depends on the application’s ability to remember and use past information, but building these features requires distinct technical approaches.

Three Types of Memory in Intelligent Applications

Developers are focusing on building intelligent applications that can adapt and grow with their users. This trend is observed in both enterprise and consumer software developers, with companies like Cursor, Microsoft, OpenAI, and Anthropic leading the charge.

Three types of memory capabilities — awareness, introspection, and personability — drive this development. Let’s explore how each works.

Awareness

Awareness centers on the application’s ability to track the state of processes as they run and actions to take post process. This process-centric memory enhances workflow management and helps optimize future actions.

In-process awareness tracks what actions have been taken and maintains state (e.g., a workflow system knowing steps 1 and 2 of a 5-step process are complete). This state management underpins reliable automation and context continuity.

Post-process awareness enables intelligent applications to determine appropriate follow up after a workflow is finished. For example, Orin, an AI-powered tutoring app, observes a student’s learning journey and determines optimal outreach timing based on previous interactions¹.

Introspection:

Introspection refers to an application’s ability to learn from past interactions, enriching its own knowledge base with experiential and factual insights learned from interactions with users or workflow runs.

Introspective knowledge can be user supplied or learned through pattern recognition from past experiences (e.g. workflow runs). For example, Cursor, a popular AI coding app, learns from user supplied facts such as “don’t restart dev servers as they hot-reload”². Alternatively, introspection logic can be implemented in previous workflow runs and extract experiential knowledge such as “password reset workflows need to call tool ABC when app XYZ is involved.”

Personability:

Personability aims to make AI interactions more tailored and human-like, adapting to user preferences and context.

Enterprise applications lean towards leveraging semantic relationships across established organizational entities such as roles, departments, teams, artifacts, and tools³.

Commercial applications often look to leverage demographics data such as age, gender, interests⁴, and marital status⁵.

Different Solutions in the Wild

Given the variety of use cases, there’s no one-size-fits-all solution. AI application developers are developing bespoke approaches tailored to their specific use cases.

Awareness:

In-process memory is often managed by tracking the state of ongoing tasks such as “planning mode” in Windsurf⁶.

Developers often implement custom logic to determine what information is worth retaining regarding post-process memory. For example, Orin, an AI tutoring tool, developed decaying resolution memory that changes over time to power proactive outreach feature⁷.

Introspection:

Most introspection solutions are highly bespoke, given the complexity of supporting different AI interaction modes, extracting salient facts, and managing the right level of information grain.

Different Human-AI Interaction Modes

Most AI applications today rely on a human-in-the-loop interaction model. For example, coding assistants pause for developers to review and approve generated code. ChatGPT produces text designed for human review and input. This approach allows for human intervention, such as when Cursor notifies “hot reloading of dev servers” as a memory.

Enterprise AI applications are increasingly adopting a human-on-the-loop interaction model. In high-volume settings like service desks, it’s impractical for managers to review every AI-customer interaction. Instead, human-on-the-loop systems need to distill and summarize large volumes of interactions⁸, and highlight only the most relevant insights for human review⁹ and subsequent decision-making.

Extracting Salient Facts

Identifying which details are truly salient in a conversation remains a major challenge. When an engineering manager interacts with financial data as a part of an annual planning exercise, should the AI application remember if the manager knows different amortization methods? When a consumer uses ChatGPT to shop for a family member should they remember their shopping preference?

Determining what’s salient is nuanced and techniques are still evolving.

Choosing the Right Information Grain

Memory grows over time. LLM context window does not. In addition, stuffing more information can actually reduce output relevance. Deciding the right level of detail for storing and retrieving memories is a key challenge. Orin, an online tutoring app, decided to summarize over different time windows. This solution approach is a good fit for their process-centric use case. However, time isn’t always the best dimension. For instance, when a new school semester begins, information about previous semester classes often becomes irrelevant.

Personability:

Two distinctive approaches between enterprise and consumer applications exist today. Enterprise AI focuses on learning information based on role and organizational context. For example, Glean, a Work AI company, maintains a personal graph in their overall enterprise knowledge graph solution¹⁰. The personal graph is built on role, system access, etc. that models real work situations.

In contrast, consumer AI focuses on the whole person, adapting responses using personal preferences and insights drawn from behaviors like dining choices or social media activity.

Conclusion:

Building applications that can track actions, learn from experience, and deliver truly personalized interactions is shaping the future of AI. While there’s no universal solution, the most innovative advancements are coming from teams willing to experiment with memory models tailored to their users and workflows. As this field rapidly evolves, organizations that invest in effective memory strategies will be best positioned to deliver smarter, more engaging products.

1. “Building Proactive AI Agents,” Orin, https://bryanhoulton1.substack.com/p/building-proactive-ai-agents
2. “Memory,” Cursor, https://www.youtube.com/watch?v=s5kX-UHgMLo
3. “How Knowledge Graphs Work and Why They Are the Key to Context for Enterprise AI,” Glean, https://www.glean.com/blog/knowledge-graph-agentic-engine
4. “Remix a prompt on Meta AI,” Meta, https://www.meta.com/help/artificial-intelligence/3980468722223023/
5. “From Dot to Dots,” New Computer, https://drive.google.com/file/d/1vM6QOHJfntyFu4gO9ROpN3_ME4EWspqR/view
6. “Planning Mode,” Windsurf Documentation, https://docs.windsurf.com/windsurf/cascade/planning-mode
7. “Building Proactive AI Agents,” Orin, https://bryanhoulton1.substack.com/p/building-proactive-ai-agents
8. “Glean Agent Design Pattern — Collect and Create,” Glean Community, https://community.glean.com/discussion/184/glean-agent-design-pattern-collect-and-create
9. “Topics Explorer,” Fin by Intercom, https://bfy.fin.ai/#:~:text=Find%20the%20hidden,specific%2C%20actionable%20Subtopics
10. “How Knowledge Graphs Work and Why They Are the Key to Context for Enterprise AI,” Glean, https://www.glean.com/blog/knowledge-graph-agentic-engine