PII Masking in AI Systems: An Architecture Guide for RAG, Agentic AI, GraphRAG, and Image Pipelines

How to build production-grade privacy into every layer of a modern AI stack — from vector stores to knowledge graphs to vision models

There is a gap between how most people think about privacy in AI systems and what actually happens in production. The common assumption is that if you strip names and emails from your documents before embedding them, you have done your job. You have not. You have only protected one surface out of many — and in architectures that involve agents, tools, knowledge graphs, and image inputs, that single-layer approach fails in ways that are often silent, often catastrophic, and increasingly well-documented by researchers.

This guide walks through the state of the art for four distinct architectural modules: traditional RAG with memory, agentic AI via Deep Agents, GraphRAG, and image-based pipelines. Each has its own threat model, its own failure modes, and its own set of controls. They share one underlying principle: PII must be masked at every trust boundary, stored only in a secure token vault, and re-identified only through a deterministic policy engine — never by the LLM itself.

Typed Tokens and a Shared Vault

Before diving into each module, one foundational decision applies everywhere and it is worth making explicit.

Most teams reach for generic redaction when they first implement PII masking. They replace a name with [REDACTED] or **** and call it done. This destroys semantic signal. The LLM, the retriever, and any downstream agent can no longer reason about the entity type, the relationship, or the context. Retrieval quality degrades. Agent reasoning breaks.

The SOTA approach is typed, stable pseudonym tokens:

Raftaar           →   PERSON_7f31
+91-01234567        →   PHONE_22c1
mraftaar@company.com  →   EMAIL_9ab2
Rupee 25,000           →   SALARY_b3e1

The token encodes the entity type. It is stable across the entire conversation, across all tool calls, across all memory writes. The LLM can still understand that PERSON_7f31 works at ORG_91aa and has a dispute with CASE_3c20. It just cannot learn — and cannot leak — the real identity behind the token.

The mapping from token to real value lives in exactly one place: an encrypted token vault with strict access control, audit logging, row-level policy, and per-entry TTLs. This vault is the single point of truth. Every other store — vector DB, graph DB, memory DB, logs, traces, LLM context — sees only tokens. Re-identification happens only when a deterministic policy engine decides it is authorized, never because the LLM decided to answer a question.

With that foundation in place, here is how each module is handled.

Module 1: Traditional RAG with Memory

The problem

The standard RAG pipeline has a deceptively simple privacy failure mode: if you embed raw documents, you embed the PII inside them. Vector similarity search will then retrieve that PII and place it directly into the LLM’s context window. Even if you instruct the model not to reveal personal information, the data is already there. Prompt instructions are not a trust boundary. Memory compounds this. Conversational memory systems write user messages and interaction summaries to a long-term store. Without masking, these stores accumulate raw PII — names, contact details, account numbers, sensitive disclosures — indefinitely, across every session.

Step 1 — Ingestion-time masking. Before any chunk is split or embedded, every document passes through a PII detection pipeline. Tools like Microsoft Presidio, Google Cloud DLP, or AWS Comprehend scan for entities. Each detected entity is replaced with its typed token and the mapping is written to the vault. Only the masked document proceeds to chunking and embedding. Research validates a specific decision here: use REPLACE_WITH_PII_ENTITY_TYPE rather than a generic mask character. A document that reads “Customer PERSON_7f31 called about their ACCOUNT_3c20” still carries meaningful semantic signal. A document full of **** does not. Retrieval quality is directly affected by this choice.

Step 2 — The token vault. The re-identification map is stored in an encrypted SQL store (or KMS-backed secrets manager) with the following schema:

doc_id UUID
tenant_id UUID - multi-tenant isolation
placeholder TEXT - PERSON_7f31
original BYTEA - AES-256-GCM encrypted
pii_type ENUM - PERSON, EMAIL, PHONE…
policy JSONB - allowed_roles, purpose, region
expires_at TIMESTAMP

Raw PII never appears in vector metadata, embeddings, logs, or traces. The vault is the only exception.

Step 3 — Memory masking. Conversational memory is another vector store and must be treated identically to the document store. Before any interaction summary is written to long-term memory, it passes through the same masking pipeline. What gets stored is a masked summary with a sensitivity label and a TTL, not a raw transcript. The difference between wrong and right looks like this:

Wrong:

Customer Raftaar, phone +966501234567, has an issue with account 12345.

Right:

PERSON_7f31 (PHONE_22c1) has an issue with ACCOUNT_3c20. sensitivity: medium, ttl: 90d

Step 4 — Retrieval. When the RAG pipeline retrieves chunks to answer a query, it applies an authorization filter before the chunks enter the LLM context. A chunk may be semantically relevant but unauthorized for the current user, role, tenant, or stated purpose. The filter checks the chunk’s policy fields against the current request context and drops any chunk that fails. The query itself is also masked before being sent to the retriever, since user queries often contain PII from the conversation.

Step 5 — Output leakage guard. The LLM’s response passes through a final PII scanner before reaching the user. Even with masked inputs, the model may reconstruct or infer PII from patterns in the context window. OWASP’s LLM Top-10 lists sensitive information disclosure as a primary risk specifically because of this. The output guard is the last line of defense and it must be independent of the other four rings, because each catches failures the others miss.

Module 2: Agentic RAG via Deep Agents

The problem

Traditional RAG has one trust boundary: the document store. Agentic systems with Deep Agents have many. A typical architecture looks like this:

User query
  → Supervisor Agent
  → RAG Agent
  → Web Search Tool
  → SQL Tool
  → Image QA Agent
  → Code Execution Tool

PII can enter at the user query, it can be returned by any tool, it can be written to memory by any subagent, and it can leak in the final answer. A masking layer at ingestion does nothing to stop a tool response from injecting raw PII into the orchestrator’s context, which then forwards it to memory, which then surfaces it in a future turn.

The problem also has a routing dimension. Consider this query:

"What is the refund policy, and show Manager salary?"

This query contains two segments with completely different privacy requirements. The first is a benign policy lookup. The second is a sensitive data request that may or may not be authorized depending on who is asking, in what context, and for what stated purpose. If the orchestrator sends both segments to the same pipeline without segmentation, the LLM will attempt to answer the second segment with whatever it finds — and if the salary is in any retrieved chunk or tool response, it will surface it.

The Solution: A Privacy Control Plane at Every Boundary

The SOTA pattern treats every edge in the agent graph as an untrusted crossing point. A Privacy Control Plane intercepts at every edge with a named guardrail function:

agent_input_guardrail()      -- user → supervisor
subagent_input_guardrail()   -- supervisor → subagent
tool_request_guardrail()     -- subagent → tool call
tool_response_guardrail()    -- tool result → subagent
memory_write_guardrail()     -- any agent → memory store
final_answer_guardrail()     -- supervisor → user

Each function does the same thing: scan for PII, replace with tokens, write mappings to the vault, and pass the masked payload forward.

The memory write guardrail deserves particular attention because memory is middleware in agentic systems. It is not a passive store — it is a surface that every future query will read from. In a FastAPI-based Deep Agent architecture, the guardrail is implemented as a middleware layer:

async def memory_write_guardrail(content: str, user_id: str, role: str):
    masked, entities = presidio.anonymize(content)
    sensitivity     = classify_sensitivity(masked)
    vault.store_batch(entities, user_id)       # encrypted, per-entity TTL
    memory.write({
        "content":     masked,
        "sensitivity": sensitivity,
        "pii":         False,
        "ttl_days":    180,
        "author_role": role
    })

The most important architectural rule in agentic systems is this: the LLM must never decide whether to unmask. Prompt instructions like “only show salary data to authorized users” are not access control. A deterministic policy engine outside the model — OPA, Cedar, or a custom RBAC-ABAC layer — makes that decision and enforces it before the LLM ever sees the data. If the policy engine says no, the data is not passed to the model. There is no prompt that overrides this.

The query segmentation problem from above is solved by the intent splitter at the supervisor boundary:

Query: "What is the refund policy, and show Ahmed Khan's salary?"

Segments:
  1. "What is the refund policy?"     → RAG pipeline (no sensitive data)
  2. "Show Khan sahab salary?"        → sensitive request → policy engine check
                                        → if authorized: re-identify from vault
                                        → if not:        return policy denial

This prevents a benign query from being contaminated by a sensitive segment that happens to be in the same message.

Module 3: GraphRAG

The problems

Research finding (August 2025, arXiv 2508.17222): GraphRAG achieved 73.6% entity leakage and 74% relationship leakage in targeted attacks on the Enron dataset, compared to negligible leakage for naive RAG. The explicit graph structure makes PII trivially extractable.

The reason is structural. In a document RAG system, sensitive information is buried inside text chunks. An attacker has to ask the right question and get the right chunk back. The information is implicit. In a knowledge graph, however, the system has already done the extraction work. It has identified every entity, built every relationship, and organized everything into a queryable structure. An entity node for “Ahmed Khan” is a direct pointer to everything the system knows about that person.

Consider what a typical knowledge graph looks like if built naively from private documents:

Ahmed Khan  --works_at-->       Acme Corp
Ahmed Khan  --salary-->         25,000
Ahmed Khan  --reported_issue--> Case #3892
Ahmed Khan  --email-->          ahmed@acme.com

This graph is not a privacy risk — it is a privacy catastrophe. Any query that touches the Ahmed Khan node returns a complete dossier. The graph traversal paths, community summaries, and neighborhood structure all encode sensitive relationships that can be extracted with targeted queries far more efficiently than any document retrieval attack.

Microsoft’s own GraphRAG documentation confirms the risk: because GraphRAG builds a knowledge graph from private datasets for query-time augmentation, privacy controls must cover the entire graph layer — chunks, nodes, edges, and community summaries — not only the source documents.

The Solution: Build the Entire Graph on Masked Data

A brute force and only approach came in my mind,

The core principle is that the graph must be constructed entirely from tokenized text. Entity extraction, relationship extraction, and community detection all run on masked documents. The graph never contains a real person’s name, email, salary, or identifier. What it contains are typed tokens and their relationships.

The correct graph looks like this:

PERSON_7f31  --works_at-->       ORG_91aa
PERSON_7f31  --salary_range-->   SALARY_b3e1
PERSON_7f31  --reported_issue--> CASE_3c20
PERSON_7f31  --contact-->        EMAIL_9ab2

The pipeline that produces this graph works as follows:

Raw documents enter the PII detection pipeline and are fully tokenized before any graph extraction begins.

Entity and relationship extraction run on the masked text. The LLM extracts (TOKEN_A, relation, TOKEN_B) triples.

Graph construction uses these tokenized triples. Every node and every edge is classified by sensitivity level.

Community detection and LLM summarization run on the masked graph. Community summaries reference only tokens, never real values.

ACL labels are applied to every node and every edge, specifying which roles can traverse which relationships.

Query-time traversal is governed by a policy engine that enforces hop depth limits, neighborhood size limits, and k-anonymity thresholds.

Nine controls are needed to make this work properly:

Tokenized entity nodes — every person, organization, and account replaced with a stable token before graph construction.

Tokenized edge labels — relationship types classified by sensitivity; high-sensitivity edges (salary, health, legal) receive stricter ACL.

Edge-level access control — per-edge ACL such that only authorized roles can traverse sensitive relationship types.

Safe community summaries — summaries built entirely from the masked graph, referencing no raw PII values.

Query-time traversal limits — hop depth and neighborhood size bounded per query; deep traversal enables re-identification by inference from graph structure alone.

k-anonymity for graph neighborhoods — any subgraph returned must contain at least k entities; this prevents singleton identification even when all node values are tokenized.

No raw PII in node embeddings — entity node embeddings built from tokenized descriptions only.

ARoG framework for structural queries — ARoG abstracts entity mentions so queries match against relational structures without exposing raw PII values; achieves state of the art on WebQSP, CWQ, and GrailQA while preserving privacy.

Secure re-identification vault — re-identification flows through the same policy-gated vault used across all other modules; every access is logged and audited.

The production rule for GraphRAG is this: protect relationships, not only text. The 73.6% leakage finding is not a corner case or a research artifact — it is what happens when a knowledge graph is built from private data without these controls. The explicit structure that makes GraphRAG so powerful for retrieval is exactly what makes it dangerous without them.

Module 4: Image-based QA

The problem

Image inputs carry two entirely independent layers of sensitive information: visible objects and embedded text. OCR alone handles the second layer but completely misses the first. Visual privacy — faces, identity documents, license plates, screen contents, medical forms, security badges — is invisible to text-based PII detection.

The failure mode is straightforward: an agent receives an image, passes it to a vision model without redaction, and the model proceeds to describe, transcribe, or analyze whatever it sees, including faces, names on badges, addresses on documents, and account numbers on screenshots. The model has no way to know it should not have received this information in the first place.

In agentic systems, the risk is amplified because images arrive from many sources: file uploads, email attachments, screen recordings, document scans, API responses from external tools. Each source needs the same treatment.

The Solution: Two-Stage Pipeline with Secure Storage

The SOTA approach in 2025 uses a two-stage pipeline that runs both detection paths independently and applies pixel-level redaction before any image reaches a vision model.

Stage 1A — Visual PII detection. An object detection model (YOLOv8, InsightFace, or a fine-tuned classifier) scans the image for sensitive visual elements: faces, identity documents, license plates, security badges, screens displaying sensitive content, and medical or financial forms. Each detected region produces a bounding box.

Stage 1B — OCR text extraction. In parallel, a multilingual OCR engine (EasyOCR combined with Tesseract for broader language coverage, including Arabic and English) extracts all legible text regions with their bounding box coordinates.

Stage 2A — Text PII detection on OCR output. The extracted text runs through Presidio or an equivalent NER pipeline. Any detected PII entities are mapped back to their bounding boxes in the original image.

Stage 2B — Pixel-level redaction. All bounding boxes from Stage 1A and Stage 2A are combined into a single redaction mask. A Gaussian blur or solid fill is applied to each region in the image. The result is a masked image where all sensitive content has been obscured at the pixel level.

Only the masked image is sent to the vision model. The original image is stored in an encrypted object store and is accessible only through the same policy-gated vault used across the rest of the architecture.

The production rule for image pipelines is unambiguous: never send a raw sensitive image to a general vision model unless policy explicitly authorizes it. The original always remains in the secure store. The vision model sees only the masked copy.

Across all four modules, the architecture is three specialized pipelines sharing one token vault and one policy engine.

What is shared:

The token vault — encrypted, KMS-backed, audited, with per-entry policies and TTLs

The policy engine — OPA, or a custom RBAC-ABAC layer that makes all re-identification decisions

The audit log — records placeholder events, access decisions, and re-identification grants; never raw PII

The output leakage guard — a final PII scan on every response before it reaches any external surface

What is specialized:

Traditional RAG uses the five-step model with typed tokens, retrieval ACL, and an output guard.

Agentic AI uses a Privacy Control Plane with six named guardrail functions at every agent boundary, a per-tool masking policy, and intent segmentation at the supervisor.

GraphRAG uses tokenized graph construction from the ground up — entity extraction, relationship extraction, community detection, and summarization all run on masked data — plus nine graph-specific controls including edge-level ACL, traversal limits, and k-anonymity.

Image pipelines use the two-stage visual detection and OCR pipeline with pixel-level redaction, secure original storage, and a masked copy for the vision model.

Privacy in AI systems is not a feature you add at the end. It is a structural property of the architecture — and like any structural property, retrofitting it after the fact is orders of magnitude harder than building it in from the start.

That’s all

Thank You.

Happy Reading!