Day 2 — Choosing the Stack Under Constraints

After breaking the documents into chunks on Day 1, the next question was obvious:

Day 2 — Choosing tools isn’t about what’s best, it’s about what works within your constraints.

How do I turn this into something an AI system can actually understand and search?

At a high level, I needed three things:

• A way to convert text into embeddings
• A way to store and retrieve those embeddings
• A model that can generate answers from retrieved context

Sounds straightforward, but the choices weren’t.

Because I was working entirely with browser-based tools and free-tier APIs.

The constraint

I wanted to build this using mostly free or low-cost tools.

That immediately ruled out several options.

Some APIs have strict quotas. Some vector databases require setup or paid tiers. Some models are powerful but expensive for repeated usage.

So every choice had to balance:

• cost
• ease of setup
• performance

The environment I used

To keep things simple and lightweight, I used Google Colab as my development environment.

Why:

• no local setup required
• runs entirely in the browser
• easy to experiment with libraries like LangChain and vector databases

Tradeoff:

• session resets can wipe temporary data
• not ideal for persistent systems
• limited control compared to a full local or cloud setup

What I chose (and why)

Embeddings — Gemini embedding model

I used Gemini’s embedding model to convert text into vectors.

Why:

• simple API
• no separate infrastructure
• works well enough for small to medium datasets

Tradeoff:

• strict rate limits
• not as configurable as some open-source alternatives

Vector Store — ChromaDB

I chose ChromaDB as the vector database.

Why:

• lightweight and easy to run
• no external server needed
• integrates well with LangChain

Tradeoff:

• not ideal for large-scale production
• persistence and scaling need more care

LLM — Gemini 2.5 Flash

For generating answers, I used a fast Gemini model.

Why:

• low latency
• good enough for Q&A tasks
• fits within free-tier usage

Tradeoff:

• not as strong as larger models
• sometimes needs better prompt control for accuracy

Framework — LangChain

To tie everything together, I used LangChain.

Why:

• simplifies chaining retrieval + generation
• reduces boilerplate code
• good for rapid prototyping

Tradeoff:

• abstraction can hide what’s actually happening
• debugging can get tricky

What I didn’t choose (and why)

I could have gone with:

• more powerful models
• managed vector databases
• custom embedding pipelines

But that would have:

• increased cost
• added setup complexity
• slowed down iteration

Right now, speed of building mattered more than perfection.

What I’m noticing already

Even before completing the pipeline, some challenges are becoming clear:

• API limits are real and easy to hit
• performance depends heavily on chunking and retrieval
• small decisions here affect everything later

Where this is going

Now that the stack is in place, the next step is to actually connect everything:

• store embeddings
• retrieve relevant chunks
• generate answers from context

This is where things usually start breaking.

And I expect errors.

I’ll share what happens when I try to run the first full pipeline.

Stay tuned.