Building a Local AI Code Assistant with Code Llama(No API, No Cost)

Introduction

What if you could generate high-quality Python code without paying for APIs, relying on the cloud or needing an internet connection?

In this blog, I’ll walk you through how I built a fully local AI code assistant using Meta’s Code Llama and the blazing-fast llama.cpp framework. This assistant reads natural language prompts and generates complete, executable Python scripts all on your Mac, running directly on the M-series chip.

Whether you’re a developer looking to experiment with open-source LLMs or just want your own offline “Copilot”, this project is a perfect starting point. Let’s dive in.

What is Code Llama?

Code Llama is Meta’s powerful open-source LLM designed for programming tasks. It understands structured prompts and can generate complete functions, modules, or even scripts similar to GitHub Copilot, but free and offline. Code Llama comes in multiple sizes (7B, 13B, 34B), and we’ll use the codellama-7b-instruct GGUF variant for this project.

📁 Project Overview

We built a Python code generation assistant that:

• Reads natural language prompts.
• Uses Code Llama via llama.cpp to generate code.
• Outputs the code to a .py file.
• Runs entirely locally using Apple Silicon (tested on M3).

Example Output Task: Scraping a product page with requests, BeautifulSoup, and saving the result to a CSV.

💻 How to Set It Up Locally

Prerequisites

• macOS with M1/M2/M3 chip (Metal backend)
• Python 3.11+ via pyenv
• Git, CMake, Clang, wget, make

Clone the Repo

git clone https://github.com/thatjetman/code-llama-.git
cd code-llama-

Download Code Llama Model

mkdir -p ../models/codellama
cd ../models/codellama
wget https://huggingface.co/TheBloke/CodeLlama-7B-Instruct-GGUF/resolve/main/codellama-7b-instruct.Q4_K_M.gguf

Build llama.cpp

cd ../../llama.cpp
mkdir build && cd build
cmake ..
make -j

Run Code Llama with a Prompt

Prompt Used

Here’s the exact instruction prompt from prompts/scrape_agent_prompt.txt:

You are a senior Python engineer. Write a complete Python script that:

• Uses requests and BeautifulSoup to scrape the first page of product listings from [https://example.com/laptops](https://example.com/laptops`)
• Extracts product name, price, and rating
• Saves the data to a CSV file
• Includes error handling
• Uses functions and includes inline comments

Output only valid Python code. Do not include explanations or markdown.

Sample Output Generated

The model returned a fully functional Python script, which can be viewed here: ai_agent.py

Here’s a snippet:

import csv
import requests
from bs4 import BeautifulSoup

def get_data(url):
    # ...

🔗 GitHub Repository

All code, prompts, and setup instructions: 👉 **thatjetman/code-llama-**

Feel free to fork, star, or raise an issue!

🔍 Why This Matters

• Free & Local: No API calls, no cloud costs.
• Full Control: Customize prompts, context size, or model size.
• Great for Learning: See exactly how LLMs generate code and integrate with tooling.

🙌 Final Thoughts

This is just the beginning. You can extend this setup to generate Flask APIs, scraping bots, test scripts — even pair it with LangChain or AutoGen for multi-step workflows.