Local LoRA Training for Specialised Domain Data

Like any software engineer, I love local control. While cloud APIs limit customisation, local fine-tuning allows us to update model weights directly on consumer hardware like a MacBook Air. I wanted to get my hands dirty with local fine-tuning to see exactly how much control we can claw back right from a laptop.

While Retrieval-Augmented Generation (RAG) is the standard architecture for factual retrieval, this personal weekend project explores using LoRA to override the base model default refusals and force adherence to niche domain terminology.

Rather than retraining an entire massive AI model (which requires immense memory and computing power), LoRA isolates a small set of adapter parameters. It updates these tiny side-matrices while leaving the original base instruct model frozen. Thus, we can customise model behaviour, tone, or response formatting entirely offline without sending sensitive business data to the cloud or paying expensive server fees.

MLX on Mac and LoRA

Angelos Katharopoulos from the MLX team shows how to build and run agentic AI workflows entirely on Mac using MLX. (Source: Apple Developer)

Recent announcements at WWDC 2026 highlight how Apple unified memory architecture allows local hardware to act as an accessible sandbox for testing low-resource fine-tuning. This can be achieved using [mlx-lm, a Python package designed for generating text and fine-tuning LLMs directly on Apple silicon](https://github.com/ml-explore/mlx-lm).

In this article, we will go through the journey of building a LoRA on a 24GB MacBook Air M5. The LoRA we will be training is a Honkai: Star Rail (HSR) game lore LoRA. We choose HSR because forcing a model to adopt highly specific domain terminology and structured conversational constraints is exactly where LoRA shines.

Honkai: Star Rail featuring the character, Silver Wolf. (Image Credit: HSR YouTube)

Simple Q&A tasks do not require the model to invent new logic or solve deep mathematical equations. The base instruct model already possesses conversational alignment, native grammatical fluency, and core reasoning capabilities.

The base instruct model lacks the specific terminology dataset. Hence, fine-tuning the adapter provides a lightweight method to override defensive refusal tokens for these niche queries.

However, fine-tuning a small model on a hyper-specific dataset like HSR lore involves a clear trade-off, i.e. even though we gain high alignment on localised domain terminology, we risk degrading the model generalised reasoning capabilities outside of that specific domain.

Data Preparation

Since the base instruct model already possesses basic conversational fluency, the training data must focus purely on structural consistency and accurate domain mapping.

In our HSR example here, we will be using the JSONL format that explicitly defines the roles (user for the user, and assistant for our ideal chatbot response) in a message array, as shown below. This format is also fully supported by mlx-lm.

{"messages":[{"role":"user","content":"Is Silver Wolf from Preservation?"},{"role":"assistant","content":"No. Silver Wolf is listed under Elation."}]}
{"messages":[{"role":"user","content":"Silver Wolf belongs to Elation or Nihility?"},{"role":"assistant","content":"Elation."}]}
{"messages":[{"role":"user","content":"Correct this statement: Silver Wolf is a Nihility Emanator."},{"role":"assistant","content":"Incorrect. Silver Wolf is listed under Elation."}]}

To make our semantic mapping LoRA successful, the user fields must include diverse ways real humans ask questions, incorporating slang, typos, and short fragments. This teaches the model how to map erratic real-world language to our clean chatbot answers.

In addition, quality over quantity. 1,000 meticulously cleaned, perfectly formatted JSONL rows will yield a significantly better LoRA than 100,000 messy, unverified rows.

The mlx-lm framework requires a very specific file structure and naming convention to auto-detect our dataset. A train.jsonl file is mandatory to run the training script. We must organise our files inside a data folder (e.g., data/) exactly like this:

data/
├── train.jsonl  <── 80% of our data (used to train the LoRA weights)
└── valid.jsonl  <── 20% of our data (used to test accuracy during training)

We shall not dump an entire HSR wiki page entry into a single assistant response because LLMs train on token predictions. If a single prompt contains 15 different facts about Silver Wolf, the character in HSR, the model will struggle to associate the specific question with the specific answer matrix. We shall break complex character lore down into granular Q&A slices.

However, writing 100 JSON lines by hand is tedious. Instead, we can use a larger local model in LM Studio (or an online LLM) to bootstrap our dataset synthetically. For example, after we have copied and pasted raw text from accurate HSR lore sources, we can provide the LLM with this specific system prompt to format the output for us:

Act as a data engineering pipeline. I will provide you with a raw text block of video game lore. Your task is to extract exact, factually dense information and output it strictly as a JSONL format matching this schema: *{"messages": [{"role": "user", "content": "..."} , {"role": "assistant", "content": "..."}]}*. Ensure questions are diverse and precise, and answers are concise and factually solid. Do not use markdown codeblocks in your output, just plain text lines.

CRITICAL DIRECTIONS:

1. For every core character fact, generate 5 distinct permutations (different phrasing, alternative questions).

2. Include explicit negative examples (e.g., “Is [Character] from [Wrong Path]? No, they are from [Right Path]”).

3. Output raw text lines only. No markdown fences.

Once we have our train.jsonl and valid.jsonl generated and cleaned up, we can proceed to set up the MLX.

The MLX Ecosystem Installation

Let’s install the MLX ecosystem, which provides the framework and CLI tools optimised for training and running models on Apple silicon.

While MLX handles raw tensor math, mlx-lm knows how a Transformer works. It contains the pre-written code for tokenisation, text generation pipelines, model quantisation, and the entire LoRA training loops.

In short, we will install the MLX ecosystem with the following script.

# 1. Create a dedicated workspace directory for your lore project
mkdir -p ~/repos/honkai-star-rail-lore-lora
cd ~/repos/honkai-star-rail-lore-lora

# 2. Create a virtual environment named '.venv' inside the directory
python3 -m venv .venv

# 3. Activate the virtual environment
source .venv/bin/activate

# 4. Upgrade pip within the isolated environment to prevent caching bugs
pip install --upgrade pip

# 5. Safely install the MLX tooling
pip install mlx-lm

# 6. Install the base instruct model download client
pip install huggingface_hub

I have already downloaded a base instruct model Qwen2.5–1.5B-Instruct in my previous exploration, so I have skipped the sixth step above.

Training LoRA

After the installation is done, we can fire off the mlx_lm lora command on our Mac, and it will update the adapter weights to align with the structural constraints of our HSR dataset.

When we install the mlx-lm, the setup configuration registers a native entry point directly in our virtual environment execution directory (.venv/bin/mlx_lm). Thus, we can train our HSR lore adapter on the base instruct model Qwen2.5–1.5B-Instruct with the following command.

mlx_lm lora \
  --model ../models/Qwen2.5-1.5B \
  --data ./data/ \
  --train \
  --batch-size 2 \
  --num-layers 16 \
  --iters 1000 \
  --learning-rate 2e-5

Here are some flags worth mentioning:

• **--iters 1000:** Setting the iteration count to 1,000 to allow sufficient gradient steps for the low-rank matrices to converge on our specific target vocabulary;
• **--batch-size 2:** Instead of calculating adjustments based on a single line of text at a time, the engine looks at two examples simultaneously;
• **--num-layers 16:** Restricting the LoRA adapters to the final 16 layers of the transformer, reducing the number of trainable parameters and backpropagation depth to keep compute overhead manageable on MacBook Air fanless hardware configurations;
• **--learning-rate 2e-5:** Applying a stable, standard learning rate baseline to ensure the adapter weights converge smoothly without overshooting or destabilising the pre-trained base features.

Our LoRA training run completed with 100% success, finished processing all 1,000 iterations.

Once our LoRA training is done, we will get the final production-ready adapter. This adapter acts as a modular, lightweight skin that dynamically snaps onto a base instruct model at runtime to alter its behavior without changing the core engine. It contains only the newly trained mathematical weights (usually just a few megabytes) that alter the behaviour of the base instruct model. This is the file we will use to serve our chatbot.

Adapters are generated.

Testing with Adapter

Now we can run this command in our terminal to talk to our custom HSR lore chatbot:

mlx_lm generate \
  --model ../models/Qwen2.5-1.5B \
  --adapter-path adapters \
  --prompt "Under which path is Silver Wolf listed as an Emanator entry?"

It should reply “elation”, as shown in the screenshot below.

The output confirms that the model now identifies Silver Wolf as an Elation Emanator.

So, you may ask, how do we know it is our LoRA that injects that knowledge of Silver Wolf? Well, let’s test the model with the same question but without the LoRA. As shown in the following screenshot, it apologised and gave up answering!

The base instruct model refused to answer our Silver Wolf question.

When we pass this prompt asking about Silver Wolf’s Emanator path to the base Qwen2.5–1.5B-Instruct model without our adapter, the model politely apologises and refuses to answer. Attaching the adapter shifts the token probability weights to output the specific target terms.

Lore Drop! Silver Wolf is only officially revealed to be the Emanator of Elation by HSR dev in April 2026. So many current LLMs may not have this latest piece of information yet. (Source: HSR YouTube)

When we attached our trained adapters.safetensors with HSR data, we integrated a low-rank adapter matrix layer that scales the weight updates during the forward pass. This mathematical addition directly alters the final token probability distribution, shifting the attention head outputs toward our specific game lore terms rather than the default, defensive refusal tokens of the base model. In short, our custom HSR data effectively overrode the model instinct to give up answering.

Wrap-Up: The Mac Edge-Finetuning Sandbox

Running 1,000 iterations at a batch size of 2 on a Qwen2.5–1.5B-Instruct model demonstrates the efficiency of the MLX ecosystem on unified memory, allowing fast iteration cycles without relying on cloud resources.

This local execution demonstrates that specialised fine-tuning can be effectively sandboxed on consumer hardware like MacBook Air using precise data engineering. Mastering data preparation, formatting, and structural permutation on a localised machine provides a reliable foundation for scaling domain-specific models without immediate reliance on cloud infrastructure.