Building a RAG Platform on AWS- Offline Ingestion + GenAI Deployment 🚀

In this article, I’ll walk through a production-oriented AWS architecture for building an end-to-end RAG ingestion and retrieval platform using:

• Amazon Web Services S3
• Amazon Web Services Lambda
• Amazon Web Services Step Functions
• Amazon Web Services ECS Fargate
• Amazon Web Services ECR
• Amazon Web Services Bedrock
• FastAPI
• Vector Databases
• Hybrid Retrieval

High-Level Architecture

The architecture below separates:

• ingestion
• orchestration
• processing
• retrieval
• online inference

into independently scalable components.

Document Upload
      ↓
Amazon S3
      ↓
Lambda Trigger
      ↓
AWS Step Functions
      ↓
ECS/Fargate GenAI Workers
      ↓
Chunking + Embeddings
      ↓
Vector Database
      ↓
FastAPI Runtime APIs
      ↓
Amazon Bedrock LLM

Step 1 — Document Upload to S3

S3 acts as the durable ingestion layer.

Typical uploads:

• PDFs
• DOCX
• PPT
• Excel
• images

Example:

s3://enterprise-rag-documents/hr/policy.pdf

S3 provides:

• durability
• versioning
• lifecycle policies
• scalable object storage

This becomes the central source for ingestion workflows.

Step 2 — Event-Driven Ingestion with Lambda

Whenever a document is uploaded, S3 emits an event notification.

Lambda functions receive:

• bucket name
• file path
• metadata

Example payload:

{
  "bucket": "enterprise-rag-documents",
  "key": "hr/policy.pdf"
}

Lambda is intentionally lightweight.

Its responsibilities:

• validate uploads
• extract metadata
• trigger orchestration
• initiate processing pipelines

Why Heavy Processing Should NOT Run in Lambda

Many teams initially attempt to:

• parse PDFs
• run OCR
• generate embeddings

inside Lambda.

This quickly becomes problematic because:

• OCR is compute-heavy
• embeddings are latency-intensive
• large PDFs exceed memory limits
• execution timeouts occur frequently

Lambda works best as: ✅ a trigger layer ✅ lightweight routing layer

— not as the primary GenAI execution engine.

Step 3 — Orchestration with Step Functions

Step Functions manage workflow orchestration.

This layer coordinates:

• execution sequencing
• retries
• state tracking
• failure handling
• parallel task execution

Typical orchestration stages:

Validate File
      ↓
Extract Metadata
      ↓
Chunk Document
      ↓
Generate Embeddings
      ↓
Store in Vector DB
      ↓
Update Metadata

One major advantage of Step Functions is observability.

Instead of debugging logs across multiple services, teams gain:

• visual workflow tracking
• retry visibility
• execution history
• operational transparency

This becomes critical in enterprise environments.

Step 4 — ECS/Fargate for GenAI Processing

The actual GenAI processing happens inside ECS workers.

This includes:

• PDF parsing
• OCR
• structure-aware chunking
• embedding generation
• metadata enrichment
• vector DB ingestion

ECS/Fargate is ideal because it supports:

• long-running jobs
• scalable containers
• heavy compute workloads
• autoscaling
• Docker-based deployment

Why ECS Works Better for Enterprise GenAI

GenAI workloads are fundamentally different from lightweight APIs.

A single ingestion task may require:

• OCR models
• PDF parsers
• embedding pipelines
• NLP preprocessing
• image extraction

These workloads demand:

• higher memory
• longer execution windows
• scalable compute

This is exactly where ECS excels.

Chunking Strategy Is Extremely Important

One of the most underestimated parts of RAG systems is chunking.

Naive chunking:

500 characters per chunk

often performs poorly in enterprise environments.

Production systems require:

• section-aware chunking
• title-aware chunking
• semantic chunk boundaries
• table-aware extraction
• metadata lineage
• parent-child chunk relationships

Why?

Because retrieval quality directly impacts hallucination rates.

Bad chunks → poor retrieval → hallucinated answers.

Metadata Is Critical

Every chunk should carry metadata such as:

{
  "doc_id": "policy_001",
  "page": 12,
  "section": "Leave Policy",
  "department": "HR"
}

Metadata enables:

• filtering
• traceability
• citations
• RBAC access control
• source attribution

Without metadata, enterprise RAG systems quickly become unmanageable.

Embedding Generation

Once chunking completes, embeddings are generated.

Typical models:

• Titan Embeddings
• OpenAI embeddings
• MiniLM
• Bedrock embedding models

Each chunk becomes a high-dimensional vector representation stored in a vector database.

Vector Database Layer

The vector database powers semantic retrieval.

Common choices include:

• Pinecone
• OpenSearch
• pgvector
• Weaviate
• FAISS

This layer stores:

• embeddings
• metadata
• chunk references

and enables efficient similarity search.

Why Hybrid Retrieval Is Better Than Pure Vector Search

A common misconception:

“Semantic search alone is enough.”

In production systems, hybrid retrieval usually performs significantly better.

Typical architecture:

BM25 + Vector Search + Reranking

Why?

Semantic search struggles with:

• exact invoice numbers
• policy IDs
• compliance references
• transaction IDs
• exact keyword matching

Hybrid retrieval combines:

• semantic similarity
• keyword precision

to improve grounding accuracy.

FastAPI as the Online Runtime Layer

FastAPI exposes GenAI APIs such as:

POST /chat
POST /search
POST /summarize

Responsibilities include:

• query embedding
• retrieval orchestration
• prompt construction
• Bedrock invocation
• response formatting

Runtime RAG Query Flow

The online inference flow looks like this:

User Query
      ↓
FastAPI
      ↓
Hybrid Retrieval
      ↓
Top-K Chunks
      ↓
Prompt Construction
      ↓
Amazon Bedrock
      ↓
Grounded Response

This ensures responses remain grounded in enterprise knowledge.

Why Docker + ECS + ECR Matter

Production GenAI systems require reproducible deployments.

Containerization enables:

• consistent environments
• scalable deployments
• CI/CD automation
• dependency isolation
• autoscaling

Typical deployment flow:

Git Push
     ↓
CI/CD Pipeline
     ↓
Docker Build
     ↓
Push to ECR
     ↓
Deploy to ECS

This provides enterprise-grade deployment maturity.

Observability Is Essential

Production RAG systems require deep monitoring.

Important metrics include:

• p95 latency
• embedding failures
• OCR failures
• queue depth
• retrieval latency
• token usage
• hallucination rates

Typical tooling:

• CloudWatch
• LangSmith
• tracing systems
• structured logging

Without observability, debugging GenAI systems becomes extremely difficult.

Final Thoughts

Modern enterprise RAG systems are no longer just about prompt engineering.

They require expertise across:

• AI engineering
• cloud architecture
• retrieval systems
• distributed systems
• orchestration
• infrastructure scalability

The future of GenAI belongs to engineers who can combine:

• LLM systems
• scalable cloud platforms
• retrieval engineering
• platform engineering
• operational reliability

into production-ready AI architectures.