Why Your AI Agent’s Memory Is Broken and How Pydantic Schemas Fix It

There is a pattern that shows up repeatedly across AI agent deployments, and it quietly undermines everything built on top of it. Most agent memory pipelines take raw text, hand it to a language model, and ask that model to decide what entities exist, what relationships matter, and what attributes are worth tracking.

The model does what it was trained to do with underspecified tasks: it generalizes. Every node becomes “Topic” or “Object.” Every edge becomes “RELATES_TO.” The graph fills up with facts that technically exist in the data structure but cannot be reached with any useful precision. Queries return either everything or nothing. The agent appears to have memory but cannot use it reliably.

The result is a knowledge graph that behaves like an expensive vector store. The semantic similarity retrieval works, but the structured, relational reasoning that a graph was supposed to enable is absent. Asking the agent what changed between last week and today produces a hallucination. Asking it whether a competitor product overlaps with a feature the team is building returns a generic summary that could apply to anything.

The source of the problem is not the language model. It is the absence of schema discipline before extraction begins.

The Pattern That Already Solves This Problem

The fix is not new. It is the same pattern that already works for reliable function calling: constrain the output space before generation happens, not after.

When a language model is given a function schema with typed parameters and descriptive field names, it produces structured, usable output because the schema tells it exactly what to look for. The same principle applies to knowledge graph extraction.

Define entity types as Pydantic models before any data is ingested. Define edge types with explicit source and target constraints. The extraction model now operates with a map of the domain rather than an open-ended instruction to find whatever seems relevant.

This shift transforms extraction from a generative task into a guided classification task. The model stops inventing ontology and starts filling in a structure that already exists.

What Graphiti Is and Why It Exists

Graphiti is a fully open-source framework from Zep AI for building and querying temporal context graphs for AI agents. It is purpose-built for the problem described above, and it goes substantially further than most retrieval-augmented generation approaches.

The core distinction is temporal. Traditional knowledge graphs and most RAG pipelines treat data as static. A fact is either in the index or it is not. When reality changes, the old fact gets overwritten or becomes noise. There is no way to ask what was true three months ago or to automatically invalidate a claim that has since been superseded.

Graphiti tracks how facts change over time. Every relationship and every entity has a validity window recording when it became true and when, if ever, it was no longer true. Old facts are not deleted. They are invalidated and preserved. This means the agent can answer questions about the present state of the world and questions about its history with the same query interface.

The framework also handles provenance. Every derived fact traces back to the raw episode that produced it, giving a full lineage from source data to structured knowledge.

The Three Pillars of Schema-Driven Memory

Entities Control What the Agent Is Allowed to Remember

When entity types are defined as Pydantic models, two things happen simultaneously. The typed fields constrain what attributes the extraction model will populate, preventing it from inventing fields that have no meaning in the domain. The descriptive docstrings carry domain vocabulary the model was never trained on.

A generic extraction prompt produces nodes of type “Person” with a name field. A Pydantic entity model for a sales context might produce nodes of type “Prospect” with fields for company size, deal stage, decision-maker status, and last engagement date. The graph that results from the latter is queryable in ways that directly serve the application’s actual needs.

Here is a minimal example of how Pydantic models define the ontology in Graphiti:

from pydantic import BaseModel, Field
from graphiti_core.nodes import EntityNode
class Competitor(EntityNode):
    """A company that competes in the same product category."""
    name: str
    primary_market: str = Field(description="The main market segment this competitor targets")
    pricing_model: str = Field(description="How the competitor charges: per-seat, usage-based, flat-rate")
class Feature(EntityNode):
    """A product capability or planned development item."""
    name: str
    status: str = Field(description="One of: planned, in-progress, shipped, deprecated")
    owner_team: str

With these types registered, the extraction model will recognize competitors and features as distinct categories, populate their fields from source text, and reject entity shapes that do not match the defined schema.

Edges Control How Things Connect

The relationship layer is where most generic knowledge graphs lose their value. If the extraction model can invent any edge type connecting any two nodes, the graph becomes a web of weakly typed associations that is difficult to traverse meaningfully.

In Graphiti, edge types are defined with explicit source and target constraints. If the schema has no edge type linking a “Project” node to a “Competitor” node, that relationship cannot be formed in the graph. This is not a limitation. It is a reasoning boundary.

A well-designed edge schema forces clarity about the domain model. Defining which relationships are possible requires thinking through the actual structure of the problem before any data is ingested. The resulting graph reflects that structure rather than the language model’s generic associations.

This also means that what the agent cannot represent, it cannot hallucinate connections about. The schema acts as a constraint on the invention of spurious relationships.

Temporal Resolution Separates What Was True From What Is True

The third pillar is the one that most distinguishes Graphiti from a standard knowledge graph or vector store.

Real-world data changes. A competitor’s pricing model changes. A feature moves from planned to shipped. A person’s role in an organization changes. A policy that was in effect last quarter is no longer in effect.

Without temporal tracking, every update to a fact either overwrites the previous value or creates a duplicate that pollutes retrieval. The agent cannot answer historical questions. It cannot detect when its stored knowledge conflicts with new information.

Graphiti handles this with explicit validity windows on every edge in the graph. When new information arrives that contradicts an existing fact, the old edge is not deleted. Its validity window is closed, recording when it became untrue. The new fact gets its own edge with a validity window starting from the point of ingestion.

Querying for the current state of an entity returns only facts with open validity windows. Querying for a historical state returns facts whose windows covered the requested point in time. Both queries use the same interface.

Hybrid Retrieval Across Three Dimensions

Query precision in Graphiti comes from combining three retrieval methods rather than relying on any single one.

Semantic search using vector embeddings handles conceptual similarity, the same mechanism that powers most RAG pipelines. Keyword search using BM25 handles exact term matching, which is often more reliable for proper nouns, product names, and technical identifiers. Graph traversal handles relational queries, following edges through the graph to find entities connected through chains of relationships.

This combination produces results that semantic search alone cannot. Finding all entities related to a specific competitor through a chain of feature overlap edges requires graph traversal. Finding mentions of an exact product version number requires keyword search. Finding conceptually related strategies requires semantic search. Graphiti runs all three and merges the results.

The practical effect is sub-second query latency for retrieval tasks that would take seconds to tens of seconds with approaches that rely on sequential LLM summarization.

Installation and Setup

Getting Graphiti running requires Python 3.10 or higher and a graph database backend. Neo4j and FalkorDB are the two primary supported options. FalkorDB can be started immediately with a single Docker command:

docker run -p 6379:6379 -p 3000:3000 -it --rm falkordb/falkordb:latest

The core package installs with pip or uv:

pip install graphiti-core

For FalkorDB support:

pip install graphiti-core[falkordb]

For Anthropic, Gemini, or Groq as the LLM backend instead of OpenAI:

pip install graphiti-core[anthropic]
pip install graphiti-core[google-genai]
pip install graphiti-core[groq]

Multiple extras can be combined in a single install command when needed.

Graphiti defaults to OpenAI for both LLM inference and embedding generation. An OPENAI_API_KEY environment variable needs to be set before initialization. Providers that support structured output, including OpenAI, Anthropic, and Gemini, are strongly recommended. Smaller or local models that do not reliably produce schema-constrained JSON output will produce extraction failures.

Using Local and Alternative LLM Providers

For deployments that require privacy, cost control, or specific model characteristics, Graphiti supports any OpenAI-compatible endpoint through the OpenAIGenericClient. This covers local servers running Ollama, vLLM, llama.cpp, and LM Studio, as well as hosted providers like DeepSeek and OpenRouter.

The following example connects Graphiti to a locally running Ollama instance:

from graphiti_core import Graphiti
from graphiti_core.llm_client.config import LLMConfig
from graphiti_core.llm_client.openai_generic_client import OpenAIGenericClient
from graphiti_core.embedder.openai import OpenAIEmbedder, OpenAIEmbedderConfig
llm_config = LLMConfig(
    api_key="ollama",
    model="deepseek-r1:7b",
    small_model="deepseek-r1:7b",
    base_url="http://localhost:11434/v1",
)
graphiti = Graphiti(
    "bolt://localhost:7687",
    "neo4j",
    "password",
    llm_client=OpenAIGenericClient(config=llm_config),
    embedder=OpenAIEmbedder(
        config=OpenAIEmbedderConfig(
            api_key="ollama",
            embedding_model="nomic-embed-text",
            embedding_dim=768,
            base_url="http://localhost:11434/v1",
        )
    ),
)

When using local models, it is worth noting that smaller models frequently produce JSON that does not match the requested schema. Using the most capable model available locally and keeping the SEMAPHORE_LIMIT environment variable low helps reduce extraction failures.

Connecting Graphiti to AI Assistants via MCP

Graphiti includes a Model Context Protocol server implementation that enables AI assistants like Claude and Cursor to interact with context graphs directly. The MCP server exposes episode management, entity and relationship handling, semantic and hybrid search, and group management through a standardized protocol.

This makes it possible to give any MCP-compatible assistant persistent, temporally aware memory backed by a structured knowledge graph, without any custom integration work beyond pointing the client at the MCP server endpoint.

The server deploys using Docker Compose alongside Neo4j, and the repository includes a complete setup guide with usage examples.

Concurrency and Rate Limit Management

Graphiti’s ingestion pipelines are designed for high concurrency. By default, the SEMAPHORE_LIMIT environment variable is set to 10 concurrent operations to avoid triggering rate limits from LLM providers. For providers with higher throughput allowances, increasing this value directly improves episode ingestion speed.

For deployments hitting rate limits despite a low semaphore setting, reducing the value further or switching to a provider with more generous limits resolves the issue without changes to the application code.

Privacy and Telemetry

Graphiti collects anonymous usage statistics by default to help the development team understand which configurations and providers are most commonly used. The telemetry system records the operating system, Python version, LLM provider type, database backend, and Graphiti version. It never records API keys, actual data, query content, graph content, IP addresses, or any personally identifiable information.

Telemetry can be disabled at any point with a single environment variable:

export GRAPHITI_TELEMETRY_ENABLED=false

This can also be set permanently in shell profiles or at the start of a Python session before initializing Graphiti. Test runs with pytest disable telemetry automatically.

Conclusion

The core insight behind Graphiti is straightforward: agent memory without schema discipline is storage without structure. Handing an LLM untyped extraction tasks produces generic graphs that cannot support precise reasoning. Defining the ontology upfront with Pydantic models constrains extraction to what the domain actually contains, prevents the invention of spurious relationships, and gives queries a reliable structure to traverse.

The temporal layer adds the dimension that most agent memory systems completely ignore. Real-world knowledge changes, and an agent that cannot track those changes cannot reason reliably about the current state of anything it has learned. Graphiti handles this by preserving the full history of every fact rather than overwriting it, making historical and current queries equally precise.

For any team building agents that need to operate on evolving, real-world data, the combination of schema-guided extraction, temporal validity tracking, and hybrid retrieval that Graphiti provides represents a substantially more reliable foundation than the untyped, batch-oriented alternatives. The schema is not overhead. It is the thing that turns a collection of facts into a model of the domain the agent actually needs to reason about.

The repository is available at: https://github.com/getzep/graphiti