Spring AI & Vector Search: A Developer’s Guide to Building Smarter AI Applications

From embeddings to pgvector — everything you need to know to build production-ready AI features with Spring AI

Introduction

If you’ve been exploring how to add AI capabilities to your Java applications, Spring AI is one of the most elegant ways to do it. In this post, I’ll walk through the core concepts — from ChatClient and embeddings to vector databases and semantic search — based on my learning journey with Spring AI.

1. ChatClient — The Abstraction Layer

The ChatClient is Spring AI’s abstraction layer that acts as a component between your application and AI models. It helps you talk to AI models (like OpenAI) without worrying about low-level API details.

The flow looks like this:

User Input → ChatClient → AI Model (OpenAI) → Response → User

Builder Pattern

Spring AI uses the Builder pattern to configure the ChatClient. The Builder helps you create objects — it allows you to configure complex properties like system prompts, memory advisors, and other settings before creating the final ChatClient object. This makes it readable and flexible.

Builder pattern: Used to create complex objects in a step-by-step and readable way.

ChatResponse

When the AI responds, you don’t just get plain text. You get a ChatResponse object that contains:

• The actual response text
• Metadata — including token details and model details

Memory Advisor

The Memory Advisor is a powerful feature that keeps track of previous conversations, enabling context-aware, multi-turn conversations without you having to manually manage conversation history.

2. Ollama — Run LLMs Locally

One of the most exciting features in the Spring AI ecosystem is Ollama integration.

• Ollama allows you to run Large Language Models (LLMs) locally, without depending on external APIs.
• Spring AI sends prompts to a locally running Ollama server through configured endpoints.

This is a game-changer for privacy-sensitive applications or when you want to avoid API costs during development.

3. Prompt Templates

Prompt Templates help you create dynamic and reusable prompts. Instead of hardcoding prompt strings, you define templates with placeholders that get filled at runtime — making your prompts maintainable and adaptable.

4. Embeddings — Text to Numbers

This is where things get really interesting.

Embedding = Text → Numbers

AI models only understand numbers. Embeddings convert text into numerical vectors while preserving semantic meaning. So two sentences that mean the same thing will have vectors that are close to each other in vector space.

Text → Numerical Vector (preserving semantic meaning)

Without Spring AI:

Text → Request Object (manually created) → Embedding API → Vector

With Spring AI:

Text → Spring AI → API/Model → Vector

Spring AI abstracts away the boilerplate of building request objects manually, making embedding generation clean and simple.

5. Cosine Similarity — The Secret Sauce

When you have two vectors, how do you know how “similar” they are? That’s where Cosine Similarity comes in.

• It measures semantic similarity between vectors by comparing their direction (not magnitude).
• It’s used internally by vector stores to find the closest matching documents to a query.

This is the actual AI-powered comparison happening under the hood when you do a semantic search.

6. Vector DB — Why Normal Databases Fall Short

If you try to do semantic text search in a traditional relational database, it becomes painfully slow at scale. That’s why we use Vector Databases.

The pipeline for storing and retrieving semantically meaningful content looks like this:

Text → Embedding → Vector DB → Similarity Search → Relevant Results

pgvector

pgvector is a PostgreSQL extension that adds vector support:

PostgreSQL + Vector Support = pgvector

It lets you store and query high-dimensional vectors directly inside your existing PostgreSQL database — no need for a completely separate vector database infrastructure.

Spring AI’s In-Memory Vector Store

Spring AI also offers an in-memory vector store, which is great for development and testing. The catch? It’s ephemeral — when the app stops, the data is gone. For production, use pgvector or Chroma.

7. The Full RAG Flow (Retrieval-Augmented Generation)

Here’s how all of this comes together in a RAG (Retrieval-Augmented Generation) pipeline:

User Query
    ↓
Embedding Model
    ↓
Query Vector Generated
    ↓
Vector Store (pgvector / Chroma)
    ↓
Cosine Similarity (internally)
    ↓
Find Closest Vectors
    ↓
Return Relevant Documents
    ↓
LLM-Generated Response

The user asks a question → it gets converted to a vector → the vector store finds the most semantically similar stored documents → those documents are passed to the LLM → the LLM generates a grounded, accurate answer.

8. Token Text Splitter — Handling Large Documents

What happens when you have a 1000-page PDF? You can’t embed the whole thing at once. That’s where the Token Text Splitter comes in.

It splits large documents into smaller chunks that can each be embedded and stored individually:

Large Text
    ↓
Split into Chunks
    ↓
Embed each chunk
    ↓
Store in Vector DB

This is critical for building knowledge bases, document Q&A systems, and any RAG application that works with large corpora.

Summary

ConceptWhat it DoesChatClientAbstraction to talk to AI modelsBuilder PatternConfigures ChatClient step-by-stepMemory AdvisorMaintains conversation historyOllamaRun LLMs locallyEmbeddingsConvert text to semantic vectorsCosine SimilarityMeasure semantic closenessVector DB / pgvectorStore & search vectors efficientlyToken Text SplitterChunk large documents for embedding

Final Thoughts

Spring AI elegantly brings the power of modern AI into the Java ecosystem. Whether you’re building a chatbot, a document Q&A system, or a semantic search engine, these building blocks — ChatClient, embeddings, vector stores, and RAG — are your foundation.

Start local with Ollama, experiment with in-memory vector stores, and when you’re ready for production, switch to pgvector. The abstraction Spring AI provides means your code barely changes.

Happy building! 🚀