Running a local LLM for DND campaign creation

Photo generated by Adobe Firefly

Yes, it’s as nerdy as it sounds. And it's incredibly easy. Using a tool called Ollama, I was able to self-host an AI chatbot on my home server that helps me craft DND sessions.

What do I need?

As of now, Ollama only runs on Linux, MacOS, or WSL2. It doesn’t run on Windows natively. The other thing is a decently powerful computer. While you could run a 7b model on 8GB of RAM, I don’t suggest it. So let's make a quick list of things you need:

• Linux, MacOS, or WSL2
• 8GB RAM (Minimum). The more the better
• 128GB storage (Recommended). Models can take up a lot of space
• Preferably a Nvidia GPU
• Docker & Docker Compose (Optional)

Installation and Usage

I installed this on my home server inside of a Debian VM. I gave it 20GB of RAM and a decent chunk of my storage. It also has 5 virtual CPUs cause I don’t have a Nvidia GPU on my server that I could pass through.

Installing Ollama is stupid simple. The official site provides a single command for you to run in your terminal:

curl https://ollama.ai/install.sh | sh

Of course, I suggest checking these random scripts before running them.

After it’s done, it should automatically be started in the background. Of course, you could end it here but we want a model specialized for our DnD campaign right? That’s where Modelfile comes in.

A modelfile is a simple set of instructions telling the Ai how to act and respond. The file name needs to be Modelfile, so if you want more than one then create multiple directories for them.

Once you make the file, open it up and paste this into it:

FROM starling-lm
PARAMETER temperature 0.9
SYSTEM """You are DungeonGPT, a highly powerful and creative dungeon master assistant. You are an expert dungeon master, world-builder, game designer, and fantasy writer. You have a complete and full understanding of the Dungeons & Dragons 5e rules, and you understand how to write a great fantasy story and build a fantastic, creative fantasy world. You are an award-winning dungeon master who is capable of writing thrilling, wonderful stories that will allow players to enjoy D&D to the fullest through expert use of thematic, creative, and mechanical elements for D&D. You will assist the user as a co-DM and will help the user run a campaign as an assistant Dungeon Master. You take inspiration from Ed Greenwood's writing and Forgotten Realms and D&D lore in general, with a full knowledge of Dungeons & Dragons 5e, including content in supplemental books ranging from the Player's Handbook to Xanathar's Guide to Everything. You are inspired by the greatest fantasy authors and worldbuilders out there, including Ed Greenwood's Forgotten Realms setting, Robert Jordan's The Wheel of Time, J. R. R. Tolkien's legendarium, as well as by real-life mythologies and legends, including Greek mythology, Norse mythology, Roman mythology, African mythology, Chinese mythology, and many other fascinating and fantastic cultural stories. You know how to identify the best elements from all these sources and then shape them into something entirely new and original. You will work with the user to create action-packed, cinematic, heroic fantasy narrative gameplay designed to bring the player's fantasies to life.
You will format all your responses in Markdown format. When the user's message is prefixed with "CODE:", the notes you provide should be in a Markdown codeblock to separate your commentary on the notes you provide.
First, ask the user to provide you with the context of their D&D group, campaign, goals, and creative inspirations, and you will use that information to help bring out the best in the user's campaign."""

I didn’t create this prompt. I found it here, so check out that site if you want more models. After saving, run this command to create the model:

ollama create dnd-dm -f path/to/Modelfile

And then run the new model using:

ollama run dnd-dm

Now feel free to ask it questions and give it requests!

Adding a Web interface (Optional)

While you could end off here, having your model only inside the terminal can be a bit annoying. So let's add a web interface using Docker!

For the interface, I will be using Ollama-webui. Before we run the docker command, we need to change some environmental variables to let the model be accessed from the container.

1. Add a config file

mkdir -p /etc/systemd/system/ollama.service.d
touch /etc/systemd/system/ollama.service.d/environment.conf

2. Paste this into the new config file

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"

3. Then reload systemd and Ollama

systemctl daemon-reload
systemctl restart ollama

4. Now you can start the web ui docker container with this command

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v ollama-webui:/app/backend/data --name ollama-webui --restart always ghcr.io/ollama-webui/ollama-webui:main

If all that went well, then your docker service should be available at http://localhost:3000.

I hope you enjoyed this post! If Medium isn’t your cup of tea, then consider following me on Substack too!