Self-Host Your Own Local AI Platform — Open WebUI + Ollama

Ever wanted to run multiple LLMs from a single interface on your own machine? Yes, it’s totally possible with Open WebUI and Ollama. With this setup, you can select any supported LLM from a dropdown menu and interact with it directly, all without relying on external APIs or the cloud.

Let’s walk through how to set this up using Docker.

What Is Open WebUI?

Open WebUI is an extensible, self-hosted AI interface that adapts to your workflow — and best of all, it works entirely offline. It gives you a sleek web-based interface to interact with local LLMs and supports user management, chat history, role-based access, and more.

Install Open WebUI Using Docker

First, let’s spin up Open WebUI:

docker run -d -p 3000:8080 -v open-webui:/app/backend/data - name open-webui ghcr.io/open-webui/open-webui:main

Now, open your browser and navigate to:

http://localhost:3000

openweb UI landing page

On your first visit, you’ll be prompted to register an admin account. After registration, you can add multiple users with different roles.

At this point, you’ll notice there are no models available — let’s fix that by integrating Ollama.

OpenWebUI — UI page

What Is Ollama?

Ollama makes it easy to run and manage LLMs locally. It supports a variety of models that you can browse in the Ollama model library, including LLaMA 3, Mistral, Gemma, and more.

Install Ollama Using Docker

Run the following Docker command:

docker run -d -p 11434:11434 --name ollama ollama/ollama

You can verify both containers are running:

docker ps

CONTAINER ID   IMAGE                                COMMAND               STATUS         PORTS                      NAMES
fe9c119c9ba2   ollama/ollama                        "/bin/ollama serve"   Up             0.0.0.0:11434->11434/tcp   ollama
106f0ec7314f   ghcr.io/open-webui/open-webui:main   "bash start.sh"       Up             0.0.0.0:3000->8080/tcp     open-webui

Pull and Run a Model with Ollama

To load a model, such as llama3:latest, run:

docker exec -it ollama ollama run llama3

This will download and run the model. You’ll then enter an interactive prompt where you can chat directly:

docker exec -it ollama ollama run llama3.1
>>>Hello
>>>
Use Ctrl + d or /bye to exit.

To exit, use Ctrl + D or type /bye.

Using Open WebUI with Ollama

Open WebUI automatically detects Ollama running on port 11434, so once you have models pulled via Ollama, they’ll appear in the Open WebUI interface. From here, you can:

• Select any available model from the dropdown
• Start chatting in a web interface
• View chat history
• Manage users and sessions

llama3.1 in OpenWebUi

You can run multiple models in ollama. For example to pull mistral or gemma —

docker exec -it ollama ollama run mistral
docker exec -it ollama ollama run gemma

All pulled models will appear in Open WebUI, ready to switch with a click.

With Open WebUI and Ollama, setting up a powerful local AI platform is easier than ever. This stack offers full control over your LLM.