I Beat Claude Sonnet With a 6B Model I Built in 15 Days for $300

Not in every benchmark. Not at creative writing or complex reasoning. But in my specific niche — the domain I trained it on — my 6B dense model outperforms Sonnet 4.6 and Gemini Flash.

And someone else built a Siri alternative that runs 100% locally, is trained on 950+ macOS tools, and beats Sonnet 4.6 in macOS workflows. Also a 6B model.

This is what’s happening right now in AI, and most people are missing it.

The shift from LLMs to VLMs

Not “Vision Language Models.” Vertical Language Models.

Small, niche-focused models in the 7B-15B range that are beating state-of-the-art general models in their specific domain. The benchmarks that matter aren’t the general ones — they’re the niche ones. And on those, a well-trained small model consistently beats a large general model.

The reason makes intuitive sense. A 6B model trained specifically on macOS workflows knows more about macOS workflows than a 200B model that knows a little about everything. Depth beats breadth when the task is narrow enough.

How I built a dataset for $300 that beat frontier models

The workflow uses three AI tools in specific roles:

Codex 5.5 (Extra High) → orchestrator + planner
DeepSeek V4 Pro        → data generator (executor)
Kimi 2.6               → data generator (executor)

Step 1: Plan the SFT dataset scope with Codex

Prompt to Codex 5.5 Extra High:

"We're training a 6B dense model for [specific domain]. 
Plan the supervised fine-tuning dataset scope:
- What categories of examples do we need?
- What's the distribution across categories?
- What makes a high-quality example in this domain?
- What are the quality gates for filtering weak data?
- How many examples per category for good coverage?
Output a complete dataset specification with quality criteria."

Codex produces a detailed spec: categories, distributions, quality gates, edge cases to cover. This is the blueprint everything else follows.

Step 2: Generate examples with DeepSeek and Kimi

No synthetic templated datasets. Handwritten examples — varied phrasing, varied complexity, varied context.

# Rough pattern for batch generation
import anthropic  # or equivalent for DeepSeek/Kimi

def generate_batch(category: str, count: int, spec: str) -> list:
    """Generate handwritten examples for a category."""

    prompt = f"""
    Dataset spec: {spec}
    Category: {category}

    Generate {count} high-quality training examples for this category.
    Each example should:
    - Feel naturally written, not templated
    - Cover different complexity levels
    - Include realistic edge cases
    - Vary the phrasing and structure

    Format: JSONL with 'prompt' and 'completion' fields.
    """

    # Send to DeepSeek V4 or Kimi API
    response = client.generate(prompt)
    return parse_jsonl(response)
# Generate across all categories
all_examples = []
for category in dataset_spec.categories:
    batch = generate_batch(
        category=category,
        count=dataset_spec.target_count[category],
        spec=dataset_spec.quality_criteria
    )
    all_examples.extend(batch)

Step 3: Quality gates — Codex filters the weak data

This is where most people skip and pay for it later. Bad data destroys fine-tuning quality.

Prompt to Codex for each batch:

"Review these [N] training examples against our quality criteria:
[paste quality gates from spec]
For each example, output:
- PASS or FAIL
- If FAIL: specific reason
- If borderline: improvement suggestion
Only PASS examples that genuinely meet all criteria. 
Be strict. We'd rather have 10,000 excellent examples 
than 50,000 mediocre ones."

Codex runs every batch through this filter. Weak data gets removed or flagged for improvement. What remains is clean.

The result:

Dataset size:     350M parameters worth of examples
Time to build:    15 days
Cost:             ~$300
Performance:      Beats Sonnet 4.6 on domain benchmarks

Fine-tuning a 7B model on a MacBook Air

Someone in this space fine-tuned a 7B base model on an M3 MacBook Air with 16GB RAM. 6,000 rows of domain-specific data. 14% improvement in their benchmark performance.

The tools that make this possible:

MLX for Apple Silicon:

pip install mlx-lm

# Fine-tune on local hardware
python -m mlx_lm.lora \
  --model mlx-community/Qwen2.5-7B-Instruct-4bit \
  --train \
  --data ./your_dataset \
  --iters 1000 \
  --batch-size 4 \
  --lora-layers 16

Unsloth for speed on CUDA (Google Colab works):

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Qwen2.5-7B-Instruct",
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=True,
)
# Add LoRA adapters
model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
    lora_alpha=16,
    lora_dropout=0,
    bias="none",
    use_gradient_checkpointing="unsloth",
)

Dataset format (JSONL):

{"prompt": "How do I [domain task]?", "completion": "Here's the correct approach: [detailed answer]"}
{"prompt": "[domain scenario]", "completion": "[expert response]"}
{"prompt": "[edge case]", "completion": "[handling strategy]"}

6,000 high-quality examples in your domain. That’s enough. Quality over quantity.

The base models worth using

Qwen3.5 (7B-14B)  → strong reasoning, excellent for instruction following
Gemma 4 (9B-12B)  → Google's latest, great multilingual support
Llama 3.1 (8B)    → widely supported, huge fine-tuning community
Mistral (7B)      → efficient, fast inference on consumer hardware

All open source. All available on Hugging Face. All fine-tuneable on hardware you already own.

The business model hiding in plain sight

The agency math:

Client charges:   $10,000 - $20,000 per fine-tuned model
Model training:   Qwen3.5 or Gemma 4 as base (free)
Dataset creation: Codex + DeepSeek + Kimi (~$300-500)
Compute:          Colab Pro or RunPod (~$100-300)
────────────────────────────────────────────────────────
Total cost:       ~$500-800
Margin:           ~95%

Enterprises that pay $100K/year for API access to a general model that’s mediocre at their specific task will pay $15K once for a model that’s excellent at that task, runs locally, and never sends data to a third party.

The three problems VLMs solve for enterprises:

1. Cost:    10x cheaper per token than frontier model APIs
2. Privacy: data never leaves their infrastructure  
3. Control: they own the weights, can update, customize, redeploy

What domains are worth targeting

The hardware angle is where it gets obvious. Devices that can’t run a 200B model but need intelligence:

Cars             → driving assistants, navigation, diagnostics
Drones           → autonomous navigation, object recognition
Home assistants  → local voice commands, home automation
Smart glasses    → real-time translation, context awareness
Smartwatches     → health monitoring, quick responses
Cameras          → scene understanding, tagging
Industrial IoT   → anomaly detection, process optimization

None of these ship with GPT-4. All of them need a specialized model that runs locally, responds in milliseconds, and costs almost nothing per inference.

The macOS assistant example is a preview of this. A 6B model, 950 macOS tools in the training data, runs entirely locally, beats Sonnet on macOS-specific tasks. That’s the pattern.

The actual opportunity

General LLMs are getting commoditized. The companies building them will compete on price until margins collapse.

Vertical intelligence is different. A fine-tuned model for a specific industry is a specialized tool that solves a specific problem better than anything else available. That’s defensible.

The barrier to entry is lower than it’s ever been:

• Open source base models are genuinely competitive with frontier models
• Training hardware is accessible (M3 MacBook Air, Colab, RunPod)
• Dataset creation costs are in the hundreds, not millions
• The AI orchestration stack (Codex + DeepSeek + Kimi) makes dataset creation fast

Six months from now, agencies doing exactly this will be everywhere. The question is whether you’re building one or hiring one.

The playbook exists. The tools exist. The models exist. All that’s missing is someone picking a domain and executing.