Indirect Prompt Injection (Using Vector Embedding and RAG) — Episode 1

Gemini helped me with this stimulation. Based on the User’s Query, the retrieved context will differ.

I am starting my joruney to learn more about Vector embedding security and RAG to upskill myself as per the market standards. I will try to include as much information as I can from my learnings in this post so that it can help people like me in unfolding new areas of knowledge. Gaining a strong mental model of how data moves geometrically is essential because AI exploitation targets mathematical relationships, not traditional syntax. Let’s start!

What is Vector Embedding?

We all know that in our traditional Applications, the database looks for exact or regex matching and sometimes pattern matching to locate a text. An LLM however, can not read words as we do. Instead, it relies on numbers to dig out the meaning of a word. The mathematical representation of a text’s meaning is what we call “Vector Embedding”.

An embedding model will take a word, phrase, or even entire paragraph and convert it into a long list of numbers. These numbers or coordinate represent a different abstract feature of the language. When these numbers are placed in a mathematical space, phrases with similar meanings will be placed close to each other, even if there is no common word in them.

Let’s consider one example for better understanding:

• “where is the password reset link?” may map to 0.85, 0.90 coordinates in a mathematical space.
• “Password change instructions” may maps to 0.83, 0.92 coordinates.
• “What is for lunch today” may have a mapping of -0.40, -0.75.

If you notice, the first two phrases have a similar meaning and are related to account recovery so the mathematical distance between them is really small. This is usually referred to as “Cosine Similarity” and they will be considered as “neighbors” in the vector space.

What is RAG?

Once an LLM is trained, it becomes staticl; meaning that it will only have as much information as was included in it’s training model. However, in an internal company setup, it’s usual for companies to refer their internal documents and get something upon it. Now, instead of retraining the LLM model, companies use RAG architecture. RAG can be thought of as an automated Open-Book Exam for the AI. Let’s see how RAG works:

1. The Knowledge Base (Ingestion): All of companies documents will be chopped into small chunks of data. Then each chunk is passed through an embedding model to get the coordinate vector. Both the raw text and coordinates are then stored in a specialised Vector Database.
2. The Query (Retrieval): Whenever a user asks a question, the system immediately converts the question into a vector using the same embedding model as in the Ingestion step.
3. The Search: The system then runs a similarity check inside the vector database to find the closest text chunks that matches the user’s query vector.
4. The Augmentation: The system will retrieve the closest text chunks and glue them into the hidden system prompt alongside with the user’s original question and hands all of this information to the LLM.
5. The Generation: The LLM will read the contextual information from the system and generate a response.

Indirect Prompt Injection (Using Vector Embedding and RAG)

If a business allows users to upload external data like (resumes, support tickets, emails, PDFs) and later ingests these into the vector database; then an attacker can perform Indirect Prompt Injection.

The attacker will simply embed a malicious instruction inside a normal document and the embedding model will map it to a specific coordinate. When an employee or any user asks a seemingly benign question that happens to be semantically close to the poisoned intruction text chunk, the RAG system will fetch the payload and blindly hand it to the LLM.

Because the LLM cannot distinguish between “system instructions” and “retrieved context,” it executes the payload or does as specified.