7-Day AI Agent Crash Course: Building a GitHub Repository Chatbot

Photo by Mohamed Nohassi on Unsplash

Course Overview: Building a Conversational Agent

The course’s goal is ambitious yet practical: create a conversational agent capable of answering questions about any (preferable your) GitHub repository.

Over seven days, we’ll:

• Download and process data from GitHub repositories
• Index the data in a search engine
• Connect the search engine to our conversational agent

Day 1, described below focuses on data ingestion — downloading data from two GitHub repositories and saving it to our file system.

Why I’m Taking on This Challenge

If you ask who loves a challenge that is going to generate impact, I’ll be the first to raise my hand. The idea of analysing data through conversation fascinates me. Especially working on the health sector where you see the first hand the impact and the stories, so question in the back of my mind for quite some time now is: what if instead of looking at dashboards telling us a story, we could simply talk to our data and get meaningful insights?

Working in the health sector, I’ve witnessed firsthand how critical data accessibility has become. With USAID closed and funding looking at maximising every dollar there is a need to make sure resources are directly impacting beneficiaries, to achieve it we urgently need systems that can effectively track program performance. What if this knowledge could be more accessible to everyone who needs it? This question drives my curiosity to learn and master these new tools through Alexey Grigorev’s 7-day challenge.

Project Setup

Folder Structure

ai_crash_course/
├── course/
│   ├── example.md
│   ├── course.ipynb
│   └── ...

Environment Setup

I’m using GitHub Codespaces for this project, coding in Visual Studio Desktop. Here’s how to set up your environment:

Prerequisites:

• Python 3.10 or higher

Step 1: Install UV

UV is a package manager that creates isolated virtual environments, preventing conflicts with other Python projects:

pip install uv

Step 2: Initialise Your Environment

uv init

Step 3: Install Dependencies

We need python-frontmatter and requests for the core functionality, plus Jupyter for development:

uv pip install python-frontmatter requests
uv pip install --dev jupyter ipykernel

uv add requests python-frontmatter
uv add --dev jupyter

Note: We’re intentionally installing Jupyter in dev mode to simulate a development environment rather than production.

Step 4: Connect Jupyter to Your Environment

uv run python -m ipykernel install --user --name=uv-env --display-name "Python (uv)"

Important: When running your Jupyter notebook, ensure you’ve selected the correct kernel — it should be “Python (uv)”.

Testing Our Setup

Let’s verify everything works correctly.

Create a Test Document

Create a file called example.md with the following content:

---
title: "Getting Started with AI"
author: "John Doe"
date: "2024-01-15"
tags: ["ai", "machine-learning", "tutorial"]
difficulty: "beginner"
---
# Getting Started with AI
This is the main content of the document written in **Markdown**.
You can include code blocks, links, and other formatting here.

Read the Test Document

In your Jupyter notebook, run this code:

import frontmatter
with open('example.md', 'r', encoding='utf-8') as f:
    post = frontmatter.load(f)
# Access metadata
print(post.metadata['title'])  # "Getting Started with AI"
print(post.metadata['tags'])   # ["ai", "machine-learning", "tutorial"]
# Access content
print(post.content)  # The markdown content without frontmatter

You should see this output:

Getting Started with AI
['ai', 'machine-learning', 'tutorial']
# Getting Started with AI
This is the main content of the document written in **Markdown**.
You can include code blocks, links, and other formatting here.

Excellent! We’ve successfully read our first input file.

Reading Data from GitHub Repositories

Now let’s scale up to read FAQ documents from two repositories, preparing them for ingestion into our chatbot system:

import io
import zipfile
import requests
import frontmatter

def read_repo_data(repo_owner, repo_name):
    """
    Download and parse all markdown files from a GitHub repository.

    Args:
        repo_owner: GitHub username or organization
        repo_name: Repository name

    Returns:
        List of dictionaries containing file content and metadata
    """
    prefix = 'https://codeload.github.com' 
    url = f'{prefix}/{repo_owner}/{repo_name}/zip/refs/heads/main'
    resp = requests.get(url)

    if resp.status_code != 200:
        raise Exception(f"Failed to download repository: {resp.status_code}")

    repository_data = []
    zf = zipfile.ZipFile(io.BytesIO(resp.content))

    for file_info in zf.infolist():
        filename = file_info.filename
        filename_lower = filename.lower()

        if not (filename_lower.endswith('.md') 
            or filename_lower.endswith('.mdx')):
            continue

        try:
            with zf.open(file_info) as f_in:
                content = f_in.read().decode('utf-8', errors='ignore')
                post = frontmatter.loads(content)
                data = post.to_dict()
                data['filename'] = filename
                repository_data.append(data)
        except Exception as e:
            print(f"Error processing {filename}: {e}")
            continue

    zf.close()
    return repository_data

dtc_faq = read_repo_data('DataTalksClub', 'faq')
evidently_docs = read_repo_data('evidentlyai', 'docs')

print(f"FAQ documents: {len(dtc_faq)}")
print(f"Evidently documents: {len(evidently_docs)}")

What’s Next?

With Day 1 complete, we’ve successfully:

• Set up our development environment
• Tested our ability to read markdown files with frontmatter
• Downloaded and parsed documentation from GitHub repositories

This foundation sets us up perfectly for the next steps: indexing this data in a search engine and building our conversational interface.

Key Takeaways

• UV provides a clean, isolated Python environment for your projects
• Frontmatter allows us to extract both metadata and content from markdown files
• GitHub’s codeload API makes it easy to download entire repositories programmatically
• Building AI agents starts with solid data ingestion. Once we get this right, we hope everything else follows.

Ready to build your own GitHub chatbot?

Follow along with this series as we continue through the 7-day challenge! Join the challenge: https://alexeygrigorev.com/aihero/