PageIndex: Vectorless, Reasoning-based RAG

Vectorless RAG · No Vector DB · No Chunking · Human-like Retrieval

Vector databases have been the backbone of RAG systems for years — but a fundamentally different paradigm is gaining serious traction. What if you let an LLM reason its way to the right answer, the same way a human expert navigates a technical document?

That’s the bet PageIndex is making. Built by VectifyAI and open-sourced on GitHub (5.7k stars and climbing), PageIndex replaces cosine similarity with genuine reasoning — trading raw retrieval speed for precision that, on real-world benchmarks, leaves classical RAG in the dust.

This post breaks down both approaches side by side: what they do, how they differ architecturally, and — most importantly — when you should reach for one over the other.

Classical RAG: The Chunking Machine

Classical Retrieval-Augmented Generation follows a well-worn playbook. You take a document, slice it into chunks, embed each chunk into a high-dimensional vector, and store those vectors in a database. At query time, you embed the question, run a cosine similarity search, pull the top-K chunks, and hand them to an LLM to synthesize an answer.

Document
    ↓
Chunk into flat pieces
    ↓
Embed each chunk → Vector DB
    ↓
Query embed + cosine similarity
    ↓
Top-K chunks → LLM → Answer

Embedding models are cheap, vector lookups are fast, and it scales to millions of documents. Frameworks like LangChain, LlamaIndex, and dozens of vector databases (Pinecone, Weaviate, Chroma) have made this the default RAG architecture for good reason.

But classical RAG has one quiet, persistent flaw: similarity is not relevance.

A chunk can be semantically close to your query and still not answer it. Multi-hop questions need reasoning across sections, not the nearest vector. And chunking silently destroys the one thing that makes complex documents navigable — their structure.

Vectorless RAG (PageIndex): The Reasoning Machine

PageIndex takes conceptual inspiration from AlphaGo: don’t brute-force every possibility — reason strategically over a structured representation of the problem space.

Applied to document retrieval, this means building a hierarchical tree index of the document first, then asking an LLM to navigate that tree to find what’s relevant.

Document
    ↓
Parse into header tree
    ↓
Summarize nodes bottom-up → JSON
    ↓
Query: LLM reasons over tree
    ↓
Selected nodes → LLM → Answer

The tree index is best understood as an enriched, LLM-optimized table of contents. Each node corresponds to a section of the document and carries an LLM-generated summary that captures what that section is about. The result is a JSON structure an LLM can scan, reason over, and navigate — without ever reading the full source document.

Here’s what a real PageIndex tree node looks like:

{
  "title": "Financial Stability",
  "node_id": "0006",
  "start_index": 21,
  "end_index": 22,
  "summary": "The Federal Reserve ...",
  "nodes": [
    {
      "title": "Monitoring Financial Vulnerabilities",
      "node_id": "0007",
      "start_index": 22,
      "end_index": 28,
      "summary": "The Federal Reserve's monitoring ..."
    },
    {
      "title": "Domestic and International Cooperation",
      "node_id": "0008",
      "start_index": 28,
      "end_index": 31,
      "summary": "In 2023, the Federal Reserve collaborated ..."
    }
  ]
}

No vector database. No embedding model at query time. Just a JSON file on disk and an LLM that can reason about it.

Pros and Cons of Vectorless RAG

“Similarity is not relevance. What we need in retrieval is reasoning — and that changes everything.” — PageIndex Framework, VectifyAI

The Trade-off — Precision Comes at a Retrieval Cost

The core tension is between retrieval precision and retrieval efficiency.

PageIndex wins on precision — especially for structured, professional documents with meaningful section hierarchy. Classical RAG wins on efficiency — especially for large corpora where speed and scale are non-negotiable.

Think of it this way: a human expert navigating a legal brief doesn’t read it word by word or jump to random paragraphs. They scan the table of contents, identify the relevant section, then drill down. PageIndex replicates this behavior. But ask that same expert to search across 10,000 legal briefs and the strategy breaks down — you need a different tool for that job.

When to Use Which One

Neither approach is universally superior. The right choice depends on your corpus, your query patterns, and your latency budget. Here’s a direct comparison across the dimensions that matter most.

Use PageIndex when you’re working with structured professional documents — SEC filings, legal briefs, academic papers, technical manuals — and your users are asking analytical, multi-hop questions that require understanding the document’s logical structure. It shines brightest for single-document deep dives.

Use Classical RAG when you have a large, heterogeneous corpus and need fast, scalable retrieval across thousands of documents. Keyword-based factual queries, product search, customer support knowledge bases — these are the domains where vector similarity earns its keep.

There’s also a compelling middle path: use PageIndex for deep analysis of individual important documents, while relying on classical RAG for the initial document discovery. The approaches aren’t mutually exclusive.

Getting Started with PageIndex

Self-hosted (open source):

# Install dependencies
pip3 install --upgrade -r requirements.txt

# Set your OpenAI API key in .env
CHATGPT_API_KEY=your_openai_key_here

# Run PageIndex on your PDF
python3 run_pageindex.py --pdf_path /path/to/your/document.pdf

# Optional: customize chunking behavior
python3 run_pageindex.py \
  --pdf_path /path/to/document.pdf \
  --model gpt-4o-2024-11-20 \
  --max-pages-per-node 10 \
  --if-add-node-summary yes

Python SDK (cloud):

from pageindex import PageIndexClient

pi_client = PageIndexClient(api_key="YOUR_API_KEY")

# Submit document
result = pi_client.submit_document("./annual-report.pdf")
doc_id = result["doc_id"]

# Query with agentic, reasoning-based RAG
response = pi_client.chat_completions(
    messages=[{"role": "user", "content": "What are the key risk factors?"}],
    doc_id=doc_id
)

print(response["choices"][0]["message"]["content"])

The SDK also supports streaming responses and multi-document queries — pass a list of doc_id values to compare or synthesize across documents.

Closing Thoughts

PageIndex represents something genuinely new in the RAG landscape: the recognition that how you retrieve matters as much as what you retrieve.

By encoding document structure into a navigable tree index and delegating the retrieval decision to an LLM that can reason about it, VectifyAI has built a system that handles the kinds of complex, hierarchical queries that have always been the Achilles’ heel of classical RAG.

It’s not a replacement for vector databases — it’s a complement, and in some cases, a superior alternative. The 98.7% accuracy on FinanceBench (via Mafin 2.5, powered by PageIndex) isn’t just a benchmark number; it’s evidence that reasoning-based retrieval can unlock real value for teams working with structured, high-stakes documents.

Resources

• GitHub: github.com/VectifyAI/PageIndex
• SDK Docs: docs.pageindex.ai/sdk
• Cookbook: docs.pageindex.ai/cookbook/vectorless-rag-pageindex