Build Your Own AI Knowledge Base: A Hands-On Guide

Andrej Karpathy recently tweeted about using LLMs to build a “personal second brain,” and it struck a chord. The appeal is obvious: instead of letting notes languish across notebooks, bookmarking apps and chat logs, you can have an AI systematically organize everything you feed it.

In this post you’ll learn how to set up such a system yourself, from folder structure to automation, and how to make your knowledge base smarter each time you use it.

The pipeline

Why a Wiki beats RAG

Most people jump straight to RAG (Retrieval-Augmented Generation). You ask a question, the model hunts for a few relevant sentences in your PDFs, and it summarizes them. It works, but it’s ephemeral. The model “forgets” as soon as the session ends.

Karpathy’s approach is different: you’re building a persistent wiki.

Instead of searching for facts on the fly, the AI reads your sources once and integrates them into a structured set of Markdown files. It creates concept pages, links related ideas, and flags contradictions. You aren’t just chatting with your data; you’re building a compounding asset.

Step 1: The Three-Folder Setup

Don’t over-engineer the database. You just need three folders in a local directory:

1. sources/: The “read-only” zone. This is for your raw PDFs, transcripts, and notes. The AI reads these but never touches them. You can save images as well, just make sure it synchs with the raw paper / files.
2. wiki/: The “brain.” This is where the AI writes its organized pages and summaries.
3. reports/: The “output” zone. This is for deep-dive analyses or answers to specific questions you’ve asked.

This is basically a Git repository of text files. It’s future-proof, portable, and requires zero subscriptions.

Step 2: Collecting the Raw Material

Dump everything into sources/. Don’t worry about naming conventions or subfolders, that’s exactly what we’re hiring the AI to handle.

If you want to automate the “gathering” part, you can use a tool like agent-browser. It’s a CLI that lets an LLM actually drive a browser, wait for pages to load, and extract the clean text.

# Example of grabbing a paper via the CLI
agent-browser open https://medium.com/my-paper
agent-browser wait --load networkidle
agent-browser get text "body" > sources/research_paper.txt

You can also use Obsidian Web Clipper. This browser extension allows you to convert web pages into Markdown files and save them either in your Obsidian Vault or as a file.

Step 3: The “Rules of the Road” (SCHEMA.md)

You need to tell the AI how you want your brain to work. Create a SCHEMA.md in your root folder. This is the instruction manual for the “Knowledge Engineer” (the AI).

Minimal Template:

• Structure: One .md file per major topic in wiki/.
• Linking: Use [[brackets]] for cross-links (Obsidian style).
• Index: Maintain an INDEX.md at the root of the wiki.
• Format: Every page starts with a TL;DR paragraph.

Example:

# Stock Research Schema

## Purpose
A personal knowledge base about value investing in stocks.

## Directories
- **sources/**: unprocessed materials — annual reports, earnings call transcripts, investor letters, market research, and valuation notes.
- **wiki/**: AI-generated pages. The AI maintains this directory; do not edit it manually.
- **reports/**: generated analyses, stock briefs, comparison notes, and investment memos.

## Wiki rules
- Each topic has its own `.md` file in `wiki/`.
- Begin each page with a one-paragraph summary (TLDR).
- Use `[[brackets]]` to link related pages.
- Maintain an `INDEX.md` listing all pages with one-line descriptions.
- When new sources are added, update the relevant pages and the index.

## Interests
- Undervalued companies
- Business quality and moats
- Earnings, cash flow, and valuation

Step 4: Let the AI Build

Now, point a coding-capable model (like Claude 4.5 Sonnet or GPT-5.4) at your directory. Give it a clear mission:

"Read the sources/ folder. Following the rules in SCHEMA.md, build out the wiki/. 
Create concept pages for recurring themes and link them together. 
If a new source contradicts an old one, flag it in a 'Contradictions' section."

As the AI ingests data, it doesn’t just summarize the data, it synthesizes. During ingestion the model reads the source, writes a summary page, updates the index and revises existing concept pages. One source might touch multiple pages. In Essence, it sees that a podcast transcript from Tuesday connects to a research paper you saved six months ago.

You can stay involved by reviewing the summaries and steering the emphasis if needed.

Step 5: Stop Chatting, Start Interacting

Once the wiki has some meat on it, you stop asking questions about individual files. You ask questions about the wiki.

• “Based on my wiki, compare Apple and Microsoft as long-term investments. Summarize their business models, growth drivers, valuation, and key risks.”
• “Identify three recurring themes across the shareholder letters, earnings transcripts, and company filings I’ve collected, and explain how they connect.”

When the AI gives you a great answer, don’t let it sit in the chat window. Have the AI save it as a new report or a new wiki page. This ensures your own curiosity feeds back into the system.

Step 6: Fighting Entropy (The Health Check)

Knowledge bases get messy, and that’s due to data accumluation. Errors and contradictions will creep in, especially if you ask the AI to extend or reinterpret its own writing. So, every few weeks, run a “Lint Pass.” Ask the AI to scan the wiki/ folder for:

1. Orphan pages: Pages with no inbound links.
2. Stale info: Claims that have been superseded by newer sources.
3. Missing links: Concepts mentioned in one page that don’t have their own page yet.

Example:

"Review the wiki/ directory. List any facts that conflict with more recent sources; identify pages that lack inbound links; and suggest three topics that deserve their own pages based on mentions across the wiki."

Two Ways to Level Up

As you build out your knowledge base, consider these improvements (borrowed from community experiments):

• Extraction Strategies: Tell the AI to treat 100-page PDFs differently than 2-minute voice memos. You want deep section-by-section analysis for the former and high-level bullet points for the latter.
• The “Devil’s Advocate” Section: Require every major wiki page to have a “Counter-Arguments” or “Open Questions” section. It forces the AI to look for gaps and prevents your knowledge base from becoming an echo chamber of your own biases.

The Bottom Line

You don’t need a heavy-duty app. You need folders, Markdown files, and a disciplined AI agent. By moving from “disposable chat” to a “persistent wiki,” you turn your research into something that actually grows over time.

Start with three folders and a handful of files. Let the system evolve as you do.