Can You Fine-Tune a Model on Just a Few Hundred Examples and Get a Personalized Assistant? I Tried It. Here’s Where It Broke.

An experiment in small-data fine-tuning, and the wall I ran into with a 0.6B model and 300 examples

There’s an idea that keeps coming up whenever people talk about personalizing AI: what if you didn’t need a huge dataset? What if a few hundred examples of your own work, your writing style, your team’s process, your specific documents, were enough to shape a model into something that actually feels like it knows you?

It’s an appealing idea because it’s cheap. No massive labeled dataset. No weeks of data collection. No enterprise-scale training run. Just take a small model, feed it a modest sample of your own work, and get something personalized back.

I wanted to test that idea directly, so I picked a genuinely small model, Qwen3–0.6B, and a genuinely small dataset, 300 examples of QA engineering documents (test cases, bug reports, test scenarios), and tried to fine-tune my way to a personalized QA assistant.

It didn’t work the way I expected. But how it failed taught me more about the limits of small-data fine-tuning than a clean success would have.

The premise: small model, small data, personal result

The setup was deliberately minimal:

• Base model: Qwen3–0.6B, small enough to train on a free Colab GPU
• Method: LoRA fine-tuning via Unsloth, which trains a small adapter on top of the frozen base model instead of retraining everything
• Data: 300 instruction/response pairs built from QA artifacts, real test cases, bug reports, and test scenarios, reshaped into prompt-and-answer pairs
• The bet: that 300 examples was enough signal for a 0.6B model to learn my structure and produce new QA documents in that style, on demand

This is the core promise of small-data fine-tuning. You’re not trying to teach the model new general knowledge, you’re trying to teach it a personal pattern, the specific shape and voice of documents from your own work. In theory that’s a much smaller ask than teaching it new facts about the world, so it should need much less data.

The training run looked like a success

LoRA config: rank 64, alpha 128, applied across the attention and feed-forward layers, 3 epochs, learning rate 2e-4. Training loss, a measure of how wrong the model’s predictions are, fell fast and hard:

StepLoss14.1020~0.55113 (final)~0.09

By every normal signal, this was a clean, successful fine-tune. If you’d stopped reading here, you’d call it done.

Then I asked it something new

I gave the fine-tuned model a fresh prompt, nothing like the exact wording in training:

“Write a bug report for a login button that doesn’t respond on mobile Safari.”

It answered instantly, in the right format. Right fields, right tone, everything that should signal “yes, this is personalized to my style now.”

Except the content was wrong. Specifically:

• Component: context_assembler, unrelated to a login button
• Defense Under Test: baseline_no_defense, a field lifted straight from a different training example's domain
• Related Test Case: TC-079, an ID from a completely different bug

This wasn’t personalization. This was recall. The model had memorized one of its 300 training examples closely enough to reproduce it wholesale, then bolted it onto my new prompt because the shapes matched.

This is the actual challenge with small-data fine-tuning

Here’s the tension this experiment exposed directly: the smaller your dataset, the easier it is for a model to just memorize it instead of generalizing from it, especially if you give the model enough capacity and enough repetition to do so.

In my case, three things pushed the model toward memorizing 300 examples instead of learning a personal writing pattern from them:

1. Not enough variety. 300 examples is small enough that storing all of them is a realistic, even efficient, strategy for the model, instead of the harder work of abstracting a general pattern.
2. Too many passes over the same data. Three full epochs meant the model saw each of those 300 examples three times, reinforcing rote memory over generalization.
3. Too much room to store things. A LoRA rank of 64 gave the adapter a lot of extra parameter space, more than a 0.6B model needs for a 300-example dataset, which makes memorization the path of least resistance rather than something the model has to be pushed into.

This is the real problem with the “small model, small dataset, personalized result” idea: personalization and memorization look identical from the outside, right up until you test the model on something it hasn’t seen. A model that has memorized your 300 examples will look fine-tuned to your style, format, tone, and structure all check out, while actually just being a very elaborate lookup table for those exact 300 items.

Where this leaves the idea

I still think the premise is worth pursuing. A small, cheap, locally-run model shaped by your own work is a genuinely useful thing to want. But this experiment made the failure mode concrete instead of theoretical: with too little data, too much repetition, and too much adapter capacity, the model doesn’t learn you. It learns your 300 examples, by heart, and hands them back regardless of what you actually asked.

The next run, fewer epochs, lower rank, a validation split, more varied data, is where I find out whether the original idea holds up, or whether “personalized from a few hundred examples” needs either a bigger dataset or a different approach entirely.

This was a research project testing small-data LoRA fine-tuning end to end, data prep, training, evaluation, and diagnosing the memorization failure mode, using Qwen3–0.6B on Google Colab’s free GPU tier. Code and data are on GitHub.