How to create your own local AI coding agent using MLX

This tutorial was designed for Macs with Apple Silicon, such as M1+ chip models. MLX is optimized for this type of machine, so the commands and performance expectations in this article are based on that environment.

Create local AI Agents with local LLM (MachineLearningMastery.com)

In the world of artificial intelligence, agents have gained a lot of attention when it comes to software development. With easy and practical access to folders, files, and the structure of large projects, interacting directly with AI becomes a natural and efficient part of the development workflow.

With great power comes great responsibility… and cost. Since most agents are paid, even small requests can quickly consume thousands of tokens. Having access to an agent can become a barrier for some people, especially independent developers who do not have access to this type of infrastructure.

The goal of this article is to teach you how to create your own local agent using MLX, running directly on your own machine, with no API costs, so you can use it whenever you want.

Prerequisites

Before we start building our agent, we need to prepare the environment where it will run. To do that, we need to install a few tools that will help us throughout the process.

1) Homebrew

Homebrew is a package manager for macOS. It allows you to install, update, and remove tools directly from the Terminal, without having to search for installers, manually configure paths, or manage different versions of each program.

During the process of building our agent, we will use Homebrew to install some of the required dependencies, such as Python, OpenCode, and other command-line tools.

To check whether Homebrew is already installed on your machine, run the following command in the Terminal:

brew --version

If the command shows the installed Homebrew version, as in the example below, it means Homebrew is already configured correctly:

Homebrew 6.0.6

If the Terminal says that the brew command was not found, you will need to install Homebrew. To do that, run the official installation command:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

During the installation process, you may also be asked to enter your macOS user password. Type the same password you use to log in to your computer and press Return.

Finally, run the command below again to validate that Homebrew was installed correctly on your device:

brew --version

If the installed version is displayed, Homebrew is configured correctly, and we can move on to the next prerequisite.

2) OpenCode

To interact with our agent directly from the Terminal, we will use OpenCode. It will be the interface responsible for receiving our requests, displaying the model’s responses, and allowing the agent to interact with the project files.

It is important to understand that OpenCode is not the AI model itself. It works as a communication layer between the user and the local model we will build throughout this article.

Before installing OpenCode, it is worth checking whether it is already available on your machine. To do that, run the command below:

opencode --version

If the command shows the installed version, as in the example below, it means OpenCode is already configured correctly:

1.17.11

If the Terminal says that the opencode command was not found, you will need to install it. To do that, we will use Homebrew with the following command:

brew install anomalyco/tap/opencode

After the installation, we can check whether everything was configured correctly by running the following command again in the Terminal:

opencode --version

If the installed version is displayed, OpenCode is configured correctly, and we can move on to the next prerequisite.

3) Python

Python is one of the most widely used programming languages for building artificial intelligence solutions. Many of the libraries, tools, and models available today offer native support for Python, making its ecosystem a key part of this project.

In our case, we will use Python to install the necessary libraries, load the language model, and implement the logic responsible for running the agent.

Before performing a new installation, check whether Python is already available on your machine. To do that, run the following command in the Terminal:

python3 --version

If the command shows a Python version, check whether it is equal to or higher than the version required by the project.

Python 3.12.12

If no version is displayed, it’s outdated or the Terminal does not recognize the python3 command, you will need to install Python.

For this article, we will use Python 3.12 to ensure compatibility with the libraries and tools used to build our local agent.

To install this version of Python, we will use Homebrew with the command below:

brew install python@3.12

Homebrew will download Python and also install the tools needed for package management, including pip, which is used to install and manage Python packages. We will use it later to install tools such as MLX.

After the installation, validate the installed version by running:

python3 --version

If the installed version is displayed, Python is configured correctly. With that, we have finished the prerequisites stage of our project.

Virtual Environment

With Python installed, the next step is to create a virtual environment for our project. A virtual environment works as an isolated space inside your machine, where we can install libraries and dependencies without affecting the global Python installation on the system.

This is important because different projects may depend on different versions of the same libraries. By using a virtual environment, we avoid version conflicts, keep the system Python clean, and ensure that everything installed for our agent stays restricted to this project.

In this article, we will use a virtual environment called mlx, where we will install the required dependencies and run the local server for our agent.

To create the virtual environment, run the command below in the Terminal:

/opt/homebrew/bin/python3.12 -m venv ~/venvs/mlx

This command creates a new virtual environment inside the ~/venvs/mlx folder, using the Python 3.12 version installed earlier.

After creating the environment, we need to activate it. To do that, run:

source ~/venvs/mlx/bin/activate

If the virtual environment is activated correctly, the Terminal will start showing the environment name at the beginning of the command line, as in the example below:

(mlx) academys-MacBook-Pro ...

From this moment on, all Python commands executed in the Terminal will be related to this virtual environment. This means that libraries installed with pip, executed commands, and the agent’s local server itself will remain isolated from the rest of the system.

To validate that Python is available inside the virtual environment, run:

python --version

If the Python version is displayed correctly, the virtual environment is configured and ready to receive the project dependencies.

Installing MLX

With the virtual environment created and activated, we can install MLX, which will be used to run the language model locally on our machine.

Before installing MLX, it is recommended to make sure that pip is up to date. pip is the Python package manager, responsible for installing and managing libraries inside the virtual environment.

With the mlx virtual environment active, run the command below:

python -m pip install --upgrade pip

This command updates pip to the latest available version, reducing the chance of problems during dependency installation.

Now we can install MLX LM, the library we will use to load and run language models with MLX:

python -m pip install --upgrade mlx-lm

The installation may take a few minutes, as Python will download the package and the dependencies required for the library to work.

After the installation, we can validate that everything was configured correctly using the command below:

python -m pip show mlx-lm

If the installation was completed successfully, the Terminal will display information about the installed package, such as its name, version, location, and dependencies.

With that, MLX LM will be installed in the project’s virtual environment, and we will be ready to use it to run our local model.

Choosing the Language Model

With MLX LM installed, the next step is to choose which language model will run locally on our machine.

For that, we have several options available through the Hugging Face platform. In simple terms, you can think of Hugging Face as a kind of GitHub for artificial intelligence models: a large repository where companies, researchers, and the community publish ready-to-use models.

Inside this platform, the MLX community contains more than 5,000 models already converted and optimized to run on Apple Silicon machines, as shown in the image below. This makes the process much easier, because we can choose an MLX-compatible model and run it locally without having to perform manual conversions.

Some language models available in the MLX community.

At this stage, we will choose the model that will be used by our agent. For this article, we chose Qwen3.5-4B-8bit, provided by the mlx-community.

This model has 4 billion parameters and uses 8-bit quantization.

Parameters: In simple terms, parameters are the internal values the model uses to understand and generate text. A model with more parameters can usually handle more complex tasks, but it also requires more memory to run.

Quantization: a technique that makes the model lighter by storing those values with less numerical precision. In this case, 8-bit quantization means the model uses fewer bits to represent each parameter, reducing memory usage and making it easier to run locally.

There is a trade-off: the model may lose a small amount of precision compared to its full-size version. But for many day-to-day coding tasks, this is a reasonable compromise.

Although this model is not necessarily the best option for very complex agents, it is a good choice for a local environment with 24 GB of RAM, since it can run without exceeding the available memory limit.

To download and run the model locally, use the command below:

python -m mlx_lm.server --model mlx-community/Qwen3.5-4B-8bit --host 127.0.0.1 --port 8080

This command starts a local server using MLX LM and loads the model specified in the --model parameter.

On the first run, if the model is not yet available on your machine, MLX LM will download it automatically. Since the file may take up a few gigabytes, this process can take a few minutes or even longer, depending on your internet speed.

After the download is complete, the model will be available locally at the following address:

127.0.0.1:8080

This address indicates that the server is running on your own machine, on port 8080. For now, you do not need to worry too much about this detail. We will come back to it later when we integrate our agent with Xcode.

While the server is running, the Terminal will remain occupied, displaying the MLX LM logs. This is expected. For the agent to communicate with the model, this Terminal process needs to stay active, and the tab must remain open while you are using the agent.

Configuring OpenCode

With the local model running through MLX LM, we need to configure OpenCode so it can communicate with it.

Up to this point, we have two important pieces working: the language model running locally on our machine, and OpenCode installed as the Terminal interface. Now, we will create a configuration file to tell OpenCode which model it should use and where that model is available.

Since the MLX LM server is occupying the previous Terminal tab, open a new Terminal tab or window. You can do this by pressing Command + T or by clicking the + button in the top-right corner of the Terminal.

Then, create the directory where OpenCode will store its configuration:

mkdir -p ~/.config/opencode

This command creates the ~/.config/opencode folder if it does not already exist. The -p option ensures that the command does not throw an error if the directory has already been created.

Now, let’s create and edit the OpenCode configuration file using nano:

nano ~/.config/opencode/opencode.json

Inside the editor, paste the content below:

{
  "$schema": "https://opencode.ai/config.json",
  "model": "mlx/default_model",
  "small_model": "mlx/default_model",
  "provider": {
    "mlx": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "MLX Local",
      "options": {
        "baseURL": "http://127.0.0.1:8080/v1",
        "apiKey": "local"
      },
      "models": {
        "default_model": {
          "id": "mlx-community/Qwen3.5-4B-8bit",
          "name": "Qwen3.5 4B 8-bit (local)"
        }
      }
    }
  }
}

After pasting the content into nano, save and close the file by following this sequence of commands:

Ctrl + O
Return
Ctrl + X

It's important to use Ctrl, not Command, to run shortcuts inside a nano editor.

cat ~/.config/opencode/opencode.json

If everything is correct, the Terminal will display the same JSON content that was pasted into the file in the previous step.

With that, OpenCode will be configured to use our local model running with MLX LM.

Running the Agent

After finishing the OpenCode configuration, the next step is to open the project folder where we want to use the agent.

Before that, it is important to make sure the MLX LM server is still running in the other Terminal tab. It needs to remain active so OpenCode can communicate with the local model.

Now, in a new Terminal tab or window, access the project folder using the cd command, which is used to enter a directory.

For example, if the project is inside the Documents folder and is called my-project, the command to access it could be written like this:

cd Documents/my-project

A simpler way to avoid typing errors in the path is to type cd, add a space at the end, and drag the project folder into the Terminal. The full path will be filled in automatically.

After accessing the correct folder, run OpenCode with the command below:

opencode

When you run this command, OpenCode will start inside the current project folder. This is important because it is from this directory that it will be able to read files, understand the code structure, and suggest changes.

Inside the OpenCode interface, type:

/models

This command allows you to view and select the available models. Choose the model configured earlier, which in this case is the local model hosted by MLX.

After that, you can start interacting with the agent. Just write what you want it to do, such as reviewing files, explaining code snippets, suggesting improvements, or helping implement new features.

Since the model is running locally, the requests are sent to the MLX LM server running on your own machine, without relying on a paid external API.

Day-to-Day Usage

Once the initial setup is complete, using the agent day to day becomes very simple.

Whenever you want to use the local agent with MLX, you will need to keep the model server running in a Terminal tab. This server is responsible for loading the language model and receiving requests sent by OpenCode, Xcode, or any other tool configured to communicate with it.

To start using it, open the Terminal and activate the virtual environment created earlier:

source ~/venvs/mlx/bin/activate

This command makes the Terminal use the mlx virtual environment, where we installed mlx-lm and all the dependencies required to run the model locally.

After activating the virtual environment, start the MLX server with the command below:

python -m mlx_lm.server --model mlx-community/Qwen3.5-4B-8bit --host 127.0.0.1 --port 8080

With the MLX server running, open a new Terminal tab or window and access the project folder you want to work on. For example:

cd Documents/my-project

Inside the project folder, run:

opencode

From that moment on, OpenCode will start using the configuration created earlier, pointing to the local model running through MLX.

Xcode Integration

In addition to using the local model through OpenCode, we can also integrate it directly with Xcode. This way, the model can be used inside Apple’s own development environment, helping with reading, explaining, and editing code.

Before configuring Xcode, make sure the MLX server is running in the Terminal. To run the server again, use the same command shown in the “Day-to-Day Usage” section.

After that, open Xcode and follow the steps below:

1. Press Command + , or click Xcode in the top-left corner and then Settings.

2. Find the Intelligence tab on the left side and click the button at the bottom of the screen called Add a Chat Provider.

3. Select the Locally Hosted option and, in the port field, add the port of our service, which in this case is 8080.

4. After that, click Add, and that is it. We now have our local model running inside Xcode.

To use it, start a new conversation in Xcode, select MLX, and then select our Qwen model. That is it. Now you can ask whatever you want, and the model will respond and help you apply changes directly to your code.