Advanced RAG Course: Build a Mini RAG Pipeline | Structured Dataset for Enterprise AI Apps

Building Your First RAG Pipeline: A Step-by-Step Guide in Python (Part 5/8)

This is Part 5 of our 8-part series on Retrieval Augmented Generation. In the previous parts, we covered the theory, architecture, and common pitfalls of RAG. Now, it’s time to write some code.

In this guide, we will build a Simple RAG Pipeline from scratch.

We will take a small financial dataset, load it into a vector database, and use an LLM to answer questions about it. To demonstrate why RAG is necessary, we’ll first attempt to ask the LLM the same questions without providing it with the data — and observe it hallucinate.

Watch the Video Walkthrough

[embed]Advanced RAG Course: Build a Mini RAG Pipeline | Structured Dataset for Enterprise AI Apps

Let’s get started.

The Setup: What We Are Building

We are going to use a simple tech stack for this demo:

• Data: A CSV file containing financial Q&A pairs (sourced from Huggingface).
• Embeddings: The all-MiniLM-L6-v2 model from Hugging Face (a lightweight, efficient embedding model).
• Vector Database: Qdrant (running in-memory for simplicity).
• LLM: OpenAI’s GPT-3.5 Turbo.

Step 1: Loading the Data

First, we need to load our knowledge base. We are using a dataset of financial questions and answers.

import pandas as pd

data = (
    pd
    .read_csv('data/finance-dataset-for-RAG-testing.csv')
    .query('pdf_content.notna()')
    .reset_index(drop=True)
    .to_dict('records')
)
console.print(data[:2])

Dataset Output

This dataset contains questions, answers, and the source PDF content. We will be using the PDF content column as the source of truth for our RAG system.

Step 2: The “Control” Test (Watch the AI Hallucinate)

Before we build the RAG system, let’s ask GPT-3.5 a specific question from our dataset: “What was the comprehensive income attributable to 3M in 2018?”

Because GPT-3.5 has a training cutoff and doesn’t have access to this specific internal document, it either says “I don’t know” or makes up a plausible-sounding but incorrect number.

from openai import OpenAI
from rich.panel import Panel

client = OpenAI(api_key=OPENAI_API_KEY)
completion = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[
        {"role": "system", "content": "You are chatbot, A Financial Analyst."},
        {"role": "user", "content": user_prompt},
        {"role": "assistant", "content": "Here is the analyst answer:"}
    ]
)

response_text = Text(completion.choices[0].message.content)
styled_panel = Panel(
    response_text,
    title="Answers without Retrieval",
    expand=False,
    border_style="bold green",
    padding=(1, 1)
)

console.print(styled_panel)

In my test, the model hallucinated a completely wrong figure.

LLM hallucination

This proves exactly why we need RAG.

Step 3: Creating Embeddings and Indexing

Now, let’s fix this. We need to convert our text data into numbers (vectors) that the Language Model can understand and search.

We use the sentence-transformers library to turn our PDF content into embeddings and store them in Qdrant.

from qdrant_client import models, QdrantClient
from sentence_transformers import SentenceTransformer

# create the vector database client
qdrant = QdrantClient(":memory:") # Create in-memory Qdrant instance

# Create the embedding encoder
encoder = SentenceTransformer('all-MiniLM-L6-v2') # Model to create embeddings

# Create collection to store the wine rating data
collection_name="FinanceQA"

qdrant.recreate_collection(
    collection_name=collection_name,
    vectors_config=models.VectorParams(
        size=encoder.get_sentence_embedding_dimension(), # Vector size is defined by used model
        distance=models.Distance.COSINE
    )
)

Step 4: The Retrieval Step

When a user asks a question, we don’t send it straight to the LLM. First, we convert the question into a vector and search our Qdrant database for the most similar documents.

hits = qdrant.search(
    collection_name=collection_name,
    query_vector=query_vector,
    limit=3
)

LLM retrieval

Step 5: Augmentation and Generation

We found the right data! Now we construct a new prompt that includes this retrieved context.

The prompt looks something like this:

System: You are a helpful financial assistant. Use the following context to answer the user’s question.

Context: [Insert Retrieved Document Text Here]

User: What was the comprehensive income attributable to 3M in 2018?

When we send this prompt to GPT-3.5, it answers perfectly, citing the exact numbers from our dataset.

Conclusion

We just built a working RAG pipeline in a few dozen lines of Python. We went from an AI that hallucinates financial data to one that answers with precision based on your own documents.

This is a “Simple RAG” implementation. In the next part of this series, we will look at how to improve this further by handling longer documents, messy data, and more complex queries.