Stop Manually Building RAG Agents — Let an AI Agent Do It Using PRDs

Problem

Building a Retrieval-Augmented Generation (RAG) system appears straightforward at first. In practice, it quickly becomes complex and fragile.

A typical pipeline involves multiple moving parts: document ingestion, chunking, embedding generation, vector indexing, retrieval tuning, and orchestration with a large language model. Each step introduces configuration choices that are easy to get wrong and difficult to debug.

Even small mistakes such as selecting an incorrect embedding model or poorly chosen chunk size can silently degrade performance. Over time, the system becomes fragmented across environments, with ingestion handled in one place, querying in another, and deployment managed separately. Iteration slows down, and maintaining consistency becomes increasingly difficult.

This is not primarily a tooling problem. It is a process problem.

A Different Approach: Use an Agent to Build the Agent

Instead of manually implementing the RAG system using Google ADK, the approach taken here is to define the system using a structured Product Requirements Document (PRD) and then use an AI coding agent (OpenAI Codex) to generate and refine the implementation.

The workflow is simple but powerful:

• Define the system clearly in a PRD
• Use an AI coding agent (OpenAI Codex) to generate the implementation
• Iteratively refine based on constraints and feedback
• Deploy programmatically to Vertex AI Agent Engine using the AI coding agent (OpenAI Codex)

This shifts the focus from writing code to specifying specification, behavior and constraints.

What Is RAG

Retrieval-Augmented Generation combines two tightly coupled components that work together at query time:

• A retriever that searches over a corpus of documents using vector similarity
• A generator (LLM) that produces responses using the retrieved context

The retriever converts both the query and documents into embeddings and finds the most semantically similar chunks. These chunks are then passed to the LLM as additional context. The LLM does not generate answers in isolation. It grounds its response in the retrieved information.

Instead of relying solely on model memory, the system dynamically pulls in relevant knowledge at query time. This has two major advantages. First, it allows the system to work with up-to-date or domain-specific data without retraining the model. Second, it improves factual accuracy because the model is conditioned on real documents rather than relying only on learned patterns.

A useful way to think about RAG is as a “search + reasoning” system. The retriever handles recall by finding the right pieces of information, while the LLM handles synthesis by combining those pieces into a coherent answer. If retrieval is poor, even the best model will produce weak answers. If retrieval is strong, even a smaller model can perform surprisingly well.

Google’s Vertex AI RAG Engine

Vertex AI RAG Engine abstracts much of the underlying complexity involved in building retrieval systems.

https://cloud.google.com/vertex-ai/generative-ai/docs/rag-engine/rag-overview

It handles several important responsibilities that map directly to the stages shown in the pipeline.

• Source: the raw documents (PDFs, Docs, files, or cloud data) that enter the system.
• Parsing: extracting and cleaning text from the source while enriching it with structure if needed.
• Transformation: splitting the parsed content into smaller chunks so they can be retrieved effectively.
• Indexing: converting chunks into embeddings and organizing them for efficient lookup.
• Vector DB: storing these embeddings in a vector database that supports fast similarity search.

At query time, the system follows another structured path:

• Preparing: the user query is processed and converted into an embedding.
• Retrieval: the system searches the Vector DB to find the most relevant chunks.
• Ranking: retrieved results are ordered so the most relevant context is prioritized.
• Serving: the selected context is formatted and sent to the LLM.

The full pipeline can be understood in two phases.

Ingestion phase:

1. Source → Parsing → Transformation → Indexing → Vector DB

Query phase:

2. Preparing → Retrieval → Ranking → Serving → LLM

This alignment makes it easier to map conceptual understanding directly to the actual system behavior shown in the diagram.

This removes the need to manually integrate multiple independent components.

System Architecture

https://googlecloudplatform.github.io/agent-starter-pack/

The system is built using the following components:

• Google ADK for agent orchestration
• Vertex AI RAG Engine for retrieval
• Vertex AI for LLM inference
• Vertex AI Agent Engine for deployment

The agent manages the complete lifecycle:

create → ingest → query → inspect → delete

Each operation is implemented as a tool, and the agent coordinates these tools based on user intent.

Key Configuration Parameters

The following parameters control the behavior of the RAG system:

EMBEDDING_MODEL = "text-embedding-005"

RAG_CHUNK_SIZE = 500
RAG_CHUNK_OVERLAP = 50
RAG_TOP_K = 5

The embedding model defines the semantic space of the corpus. Once a corpus is created with a specific embedding model, it cannot be changed without recreating the corpus.

Chunk size determines how documents are split. Smaller chunks improve retrieval precision but may lose context. Larger chunks preserve context but reduce retrieval accuracy.

Overlap ensures continuity between chunks, preventing important information from being split across boundaries.

Top-K controls how many results are retrieved. A higher value increases recall but may introduce noise.

Careful tuning of these parameters is essential for effective retrieval, balancing tradeoffs with respect to specific use cases.

The PRD

Below, each key PRD excerpt is shown as a snippet, followed by what it enforces in the generated system. Refer the entire PRD.md file in the Github repo, link in reference section.

Objective

Build a production-grade RAG agent using Google ADK and Vertex AI that can:
- create and manage Vertex AI RAG corpora
- ingest documents
- retrieve context with Vertex AI RAG
- keep an active corpus in session state
- run locally with `adk web`
- deploy programmatically to Vertex Agent Engine

This anchors scope. The agent must implement end-to-end lifecycle + local run + deployment. It prevents partial implementations (for example, ingestion without deployment).

Primary Implementation Rules

- Reuse ADK and Vertex AI SDK components
- Do not introduce custom abstractions
- The implementation must explicitly reuse:
  - google.adk.agents.Agent
  - google.adk.tools.ToolContext
  - vertexai.rag
- All corpus operations must go through tools

This constrains architecture. The generated code stays thin and SDK-aligned, and forces the agent to act via tools (deterministic) rather than free-form reasoning.

System Scope

create -> ingest -> query -> inspect -> delete

- list corpora
- create a corpus
- import data
- query a corpus
- inspect contents
- delete document / corpus (with confirmation)

Defines the exact feature surface. This becomes the tool set and prevents feature creep.

Runtime and Platform Baseline

- Framework: Google ADK
- RAG backend: Vertex AI RAG Engine
- LLM backend: Vertex AI
- Local UI: adk web
- Deployment target: Vertex Agent Engine

Pins the stack. Also implies a hard rule (stated elsewhere): initialize Vertex AI before any tool call.

Environment Configuration

All runtime and deployment configuration must come from `.env`.

- GOOGLE_GENAI_USE_VERTEXAI
- GOOGLE_CLOUD_PROJECT
- GOOGLE_CLOUD_LOCATION
- DEFAULT_EMBEDDING_MODEL
- GOOGLE_CLOUD_STORAGE_BUCKET

Forces an environment-driven system with fail-fast validation. Eliminates hidden defaults and config drift.

Embedding Model Requirements

- text-embedding-005
- text-embedding-004
- text-multilingual-embedding-002

Disallowed:
- textembedding-gecko@001
- textembedding-gecko@002
- ...

Embedding model is effectively fixed for that corpus lifecycle.

Three critical behaviors:

1. allow only current publisher models,
2. reject retired models early,
3. enforce immutability per corpus (changing .env does not fix existing corpora).

Tooling Requirements

- rag_query
- list_corpora
- create_corpus
- add_data
- get_corpus_info
- delete_document
- delete_corpus

- Tools must call `vertexai.rag` directly
- Return { status, message, data }
- Validate inputs strictly
- Do not expose internal resource names

Defines the contract for every tool: direct SDK calls + structured outputs + strict validation.

create_corpus

- If exists, reuse as active corpus
- Use RagVectorDbConfig
- Use approved embedding model
- Fail clearly on invalid model

Implements idempotent creation + state setting. Also enforces embedding correctness at creation time.

add_data

Supported:
- GCS paths
- Google Drive URLs
- Google Docs URLs
- Normalize URLs (HTTPS → gs://)
- Reject empty/unsupported
- Report imported/failed/skipped

This block encodes input normalization + ingestion reporting. Most ingestion bugs are handled here, not in networking.

rag_query

- Use selected or active corpus
- Use configurable top_k
- Apply distance-threshold filtering
- Return document name, text, distance

Defines retrieval behavior + output schema. This is where quality is controlled (top_k + filtering).

get_corpus_info

- Return summary
- Return numbered documents

Provides a stable handle for follow-up actions (for example, delete by index instead of opaque IDs).

Delete Tools

- require confirm=True
- never delete when confirmation is absent

Hard safety gate. Prevents destructive actions by default.

Ingestion Rules

Allowed:
- gs://...
- https://storage.googleapis.com/...
- https://drive.google.com/...
- https://docs.google.com/...

If message contains supported paths → route to add_data

Encodes routing + normalization so the agent does not ask users to reformat already-valid inputs.

State Management

Track:
- current_corpus
- current_corpus_resource
- corpus_exists_<slug>
- corpus_resource_<slug>

- creating/resolving sets active corpus
- operations fall back to current corpus
- keep resource names internal

Defines a simple, serializable state model that enables multi-step workflows without leaking internals.

Agent Behavior

- knowledge questions must use rag_query
- corpus actions must use matching tool
- must mention which corpus it used
- must explain actions after tool calls
- must not invent backend causes

Turns the agent into a tool-orchestrator with guardrails, not a free-form chatbot.

Local Execution

adk web

1. load .env
2. validate
3. initialize Vertex AI
4. construct agent
5. serve

Defines a reproducible local loop.

Deployment Requirements

Provide:
- deploy.py
- invoke.py
- requirements.txt
- README.md

Flow:
1. load env
2. validate
3. initialize Vertex
4. package
5. deploy to Agent Engine
6. print details
7. support invocation

Encodes one-command deployment with no manual UI steps and verifiable output.

Invocation

- accept resource name
- create session if missing
- allow session reuse
- stream/print responses

Ensures the deployed agent is actually usable from CLI/SDK.

Acceptance Criteria

- local testing works
- corpus lifecycle works end-to-end
- queries return structured results
- deletion requires confirmation
- deployment works in one command
- deployed agent can be invoked
- retired models are rejected

Defines “done”. The system is only valid if all constraints hold together.

This snippet-driven structure is what lets an AI coding agent translate specification into a consistent, production-ready system.

Implementation Overview

The system is implemented as a set of tools that directly interact with Vertex AI RAG APIs.

create_corpus

Creates a new corpus or reuses an existing one with the same name. It also initializes state by setting the active corpus.

add_data

Handles ingestion of documents from supported sources such as Google Drive, Google Docs, and Cloud Storage. Inputs are normalized before ingestion, and the tool reports successes and failures explicitly.

rag_query

Performs semantic retrieval using the active corpus. It applies the configured Top-K value and returns structured results including matched text and metadata.

Other tools include listing corpora, inspecting corpus contents, and deleting documents or corpora with explicit confirmation.

The implementation avoids unnecessary abstraction and directly uses the Vertex AI RAG SDK, as enforced by the PRD.

Example Interaction Flow

A typical interaction follows these steps:

1. List available corpora
2. Create a new corpus named “Literature”
3. Add a document using a Google Drive link
4. Query the corpus with a natural language question

Behind the scenes, the agent invokes the appropriate tools, updates session state, and retrieves relevant context before generating a response.

Corpus creation, document addition and RAG Query

Deployment with Agent Engine

https://docs.cloud.google.com/agent-builder/agent-engine/overview

Deployment is handled programmatically.

The AI coding agent generates scripts that:

• Validate environment variables
• Initialize Vertex AI
• Package the agent
• Deploy to Vertex AI Agent Engine
• Provide an invocation interface

This ensures that deployment is reproducible and does not depend on manual steps. After local testing, the deployment script is also implemented using OpenAI Codex coding Agent.

Why This Approach Works

The PRD defines the system. The OpenAI Codex coding agent implements it.

This separation provides several advantages:

• Consistent system behavior
• Faster iteration
• Reduced implementation errors
• Easier debugging and maintenance

Instead of writing code directly, effort is focused on defining correct behavior.

Common Mistakes to Avoid

RAG systems often fail due to predictable issues.

Changing the embedding model after corpus creation leads to inconsistent representations and degraded retrieval.

Improper chunking reduces retrieval effectiveness. Both chunk size and overlap must be chosen carefully.

Unnormalized input paths can cause ingestion failures, especially when working with different data sources.

Allowing the AI agent to generate code without a structured PRD results in inconsistent implementations.

Finally, manually stitching together local and cloud workflows leads to fragile systems that are difficult to maintain.

Lessons Learned

Building RAG systems manually introduces unnecessary complexity.

A well-defined PRD enables an AI coding agent to generate a complete and correct system, including deployment.

Vertex AI RAG Engine significantly reduces the effort required to build retrieval pipelines.

Agent Engine simplifies deployment and turns the system into a production-ready service.

Most importantly, the effectiveness of the system depends more on the clarity of specification than on the amount of code written.

Final Thought

The primary challenge in building RAG systems is not implementation but design.

When the system is clearly specified, an AI agent can generate, refine, and deploy the entire pipeline end-to-end.

This shifts the role of the developer from writing code to defining systems.

That shift is where the real productivity gain comes from.

It no more feels like coding but more like cooking, giving the appropriate ingredients while OpenAI Codex does the cooking.

References

• Github repo: https://github.com/kgaero/RAG_agent_adk