Qwen3-Coder-Next: Running an 80B Coding Model Locally on 46GB RAM

Qwen recently launched Qwen3-Coder-Next, which stands out as one of the most useful large coding models we’ve encountered in some time.

On paper, it sounds wild:

• 80B parameter MoE model
• Only ~3B active parameters
• 256K context window
• Runs locally on ~46GB RAM
• Comparable to models with 10–20× more active parameters

In practice, it’s even more interesting — especially if you care about agentic coding, tool use, and running serious models on your own machine.

This article explains how to run Qwen3-Coder-Next.

Why Qwen3-Coder-Next Matters

Most large coding models today fall into one of two buckets:

1. Small and fast, but shallow reasoning
2. Huge and capable, but cloud-only and expensive

Qwen3-Coder-Next sits right in the uncomfortable middle — and that’s a good thing.

Because it’s a Mixture-of-Experts (MoE) model, only a small subset of parameters is active per token. That’s how an 80B model behaves like a much smaller one at inference time while still retaining the capacity of a much larger model.

What this means:

• Fast responses for code
• Strong long-horizon reasoning
• Excellent recovery from execution failures
• Local deployment without a datacenter

Hardware Requirements

You don’t need a monster setup, but you do need to be honest about memory.

• 4-bit quant: ~46GB RAM / VRAM / unified memory
• 8-bit quant: ~85GB
• 3-bit quants: viable on smaller machines

A good rule of thumb:

Disk + RAM + VRAM ≥ quant size

If the entire model fits in memory, you can expect 20+ tokens/sec. If not, it will still work by offloading, just more slowly.

Recommended Generation Settings

Qwen suggests the following, and they work well in practice:

• Temperature = 1.0
• Top-P = 0.95
• Top-K = 40
• Min-P = 0.01

The model supports up to 262,144 tokens of context, but if you’re memory-constrained, dropping to 32K or 16K is perfectly reasonable.

Also important:

This model is non-thinking only. It does not emit <think></think> blocks.

So you no longer need enable_thinking=False.

Running Qwen3-Coder-Next with llama.cpp

We’ll start with llama.cpp, since it’s the most common local setup.

Build llama.cpp

apt-get update
apt-get install pciutils build-essential cmake curl libcurl4-openssl-dev -y
git clone https://github.com/ggml-org/llama.cpp
cmake llama.cpp -B llama.cpp/build \
    -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-mtmd-cli llama-server llama-gguf-split
cp llama.cpp/build/bin/llama-* llama.cpp

(Change -DGGML_CUDA=ON to OFF if you’re CPU-only.)

Run Directly from Hugging Face

./llama.cpp/llama-cli \
    -hf unsloth/Qwen3-Coder-Next-GGUF:UD-Q4_K_XL \
    --jinja --ctx-size 16384 \
    --temp 1.0 --top-p 0.95 --min-p 0.01 --top-k 40 --fit on

Download the Model Manually

pip install -U huggingface_hub
hf download unsloth/Qwen3-Coder-Next-GGUF \
    --local-dir unsloth/Qwen3-Coder-Next-GGUF \
    --include "*UD-Q4_K_XL*"

Then run it:

./llama.cpp/llama-cli \
    --model unsloth/Qwen3-Coder-Next-GGUF/Qwen3-Coder-Next-UD-Q4_K_XL.gguf \
    --fit on \
    --seed 3407 \
    --temp 1.0 \
    --top-p 0.95 \
    --min-p 0.01 \
    --top-k 40 \
    --jinja

You can push context up to 262K if memory allows.

Serving Qwen3-Coder-Next as an API

For agentic workflows, you’ll want llama-server.

./llama.cpp/llama-server \
    --model unsloth/Qwen3-Coder-Next-GGUF/Qwen3-Coder-Next-UD-Q4_K_XL.gguf \
    --alias "unsloth/Qwen3-Coder-Next" \
    --fit on \
    --seed 3407 \
    --temp 1.0 \
    --top-p 0.95 \
    --min-p 0.01 \
    --top-k 40 \
    --port 8001 \
    --jinja

Then connect using the OpenAI-compatible API:

from openai import OpenAI
import json
openai_client = OpenAI(
    base_url = "http://127.0.0.1:8001/v1",
    api_key = "sk-no-key-required",
)
completion = openai_client.chat.completions.create(
    model = "unsloth/Qwen3-Coder-Next",
    messages = [{"role": "user", "content": "Create a Flappy Bird game in HTML"},],
)
print(completion.choices[0].message.content)

What the Model Actually Produces

The generated Flappy Bird example wasn’t a toy snippet. It was a fully working HTML5 Canvas game, complete with:

• Physics
• Collision detection
• Scoring
• Persistent high scores
• Keyboard, mouse, and touch support

The entire code executed successfully upon extraction and testing — no manual corrections were needed.

That’s a strong signal for agentic coding reliability.

Tool Calling with Qwen3-Coder-Next

Qwen3-Coder-Next supports structured tool calling for:

• Python execution
• Terminal commands
• Math functions
• Custom logic

Here’s the full tool setup:

import json, subprocess, random
from typing import Any
def add_number(a: float | str, b: float | str) -> float:
    return float(a) + float(b)
def multiply_number(a: float | str, b: float | str) -> float:
    return float(a) * float(b)
def substract_number(a: float | str, b: float | str) -> float:
    return float(a) - float(b)
def write_a_story() -> str:
    return random.choice([
        "A long time ago in a galaxy far far away...",
        "There were 2 friends who loved sloths and code...",
        "The world was ending because every sloth evolved to have superhuman intelligence...",
        "Unbeknownst to one friend, the other accidentally coded a program to evolve sloths...",
    ])
def terminal(command: str) -> str:
    if "rm" in command or "sudo" in command or "dd" in command or "chmod" in command:
        msg = "Cannot execute 'rm, sudo, dd, chmod' commands since they are dangerous"
        print(msg); return msg
    print(f"Executing terminal command `{command}`")
    try:
        return str(subprocess.run(command, capture_output = True, text = True, shell = True, check = True).stdout)
    except subprocess.CalledProcessError as e:
        return f"Command failed: {e.stderr}"
def python(code: str) -> str:
    data = {}
    exec(code, data)
    del data["__builtins__"]
    return str(data)
MAP_FN = {
    "add_number": add_number,
    "multiply_number": multiply_number,
    "substract_number": substract_number,
    "write_a_story": write_a_story,
    "terminal": terminal,
    "python": python,
}
tools = [
    {
        "type": "function",
        "function": {
            "name": "add_number",
            "description": "Add two numbers.",
            "parameters": {
                "type": "object",
                "properties": {
                    "a": {
                        "type": "string",
                        "description": "The first number.",
                    },
                    "b": {
                        "type": "string",
                        "description": "The second number.",
                    },
                },
                "required": ["a", "b"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "multiply_number",
            "description": "Multiply two numbers.",
            "parameters": {
                "type": "object",
                "properties": {
                    "a": {
                        "type": "string",
                        "description": "The first number.",
                    },
                    "b": {
                        "type": "string",
                        "description": "The second number.",
                    },
                },
                "required": ["a", "b"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "substract_number",
            "description": "Substract two numbers.",
            "parameters": {
                "type": "object",
                "properties": {
                    "a": {
                        "type": "string",
                        "description": "The first number.",
                    },
                    "b": {
                        "type": "string",
                        "description": "The second number.",
                    },
                },
                "required": ["a", "b"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "write_a_story",
            "description": "Writes a random story.",
            "parameters": {
                "type": "object",
                "properties": {},
                "required": [],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "terminal",
            "description": "Perform operations from the terminal.",
            "parameters": {
                "type": "object",
                "properties": {
                    "command": {
                        "type": "string",
                        "description": "The command you wish to launch, e.g `ls`, `rm`, ...",
                    },
                },
                "required": ["command"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "python",
            "description": "Call a Python interpreter with some Python code that will be ran.",
            "parameters": {
                "type": "object",
                "properties": {
                    "code": {
                        "type": "string",
                        "description": "The Python code to run",
                    },
                },
                "required": ["code"],
            },
        },
    },
]

And the inference loop:

from openai import OpenAI
def unsloth_inference(
    messages,
    temperature = 1.0,
    top_p = 0.95,
    top_k = 40,
    min_p = 0.01,
    repetition_penalty = 1.0,
):
    messages = messages.copy()
    openai_client = OpenAI(
        base_url = "http://127.0.0.1:8001/v1",
        api_key = "sk-no-key-required",
    )
    model_name = next(iter(openai_client.models.list())).id
    print(f"Using model = {model_name}")
    has_tool_calls = True
    original_messages_len = len(messages)
    while has_tool_calls:
        print(f"Current messages = {messages}")
        response = openai_client.chat.completions.create(
            model = model_name,
            messages = messages,
            temperature = temperature,
            top_p = top_p,
            tools = tools if tools else None,
            tool_choice = "auto" if tools else None,
            extra_body = {"top_k": top_k, "min_p": min_p, "repetition_penalty" :repetition_penalty,}
        )
        tool_calls = response.choices[0].message.tool_calls or []
        content = response.choices[0].message.content or ""
        tool_calls_dict = [tc.to_dict() for tc in tool_calls] if tool_calls else tool_calls
        messages.append({"role": "assistant", "tool_calls": tool_calls_dict, "content": content,})
        for tool_call in tool_calls:
            fx, args, _id = tool_call.function.name, tool_call.function.arguments, tool_call.id
            out = MAP_FN[fx](**json.loads(args))
            messages.append({"role": "tool", "tool_call_id": _id, "name": fx, "content": str(out),})
        else:
            has_tool_calls = False
    return messages

Benchmarks (Context Matters)

Qwen3-Coder-Next performs surprisingly well given its active parameter count:

| Benchmark              | Score|
| ---------------------- | ---- |
| SWE-Bench Verified     | 70.6 |
| SWE-Bench Multilingual | 62.8 |
| SWE-Bench Pro          | 44.3 |
| Terminal-Bench 2.0     | 36.2 |
| Aider                  | 66.2 |

The key takeaway isn’t that it beats everything — it’s that it matches or exceeds much larger models while being deployable locally.

Final Thoughts

Qwen3-Coder-Next isn’t just another model release.

This indicates that local, high-calibre agentic coding is now feasible not only for labs and corporations but also for autonomous developers handling complex workflows.

If you care about:

• Running models locally
• Long-context coding
• Tool-calling agents
• Cost-efficient inference

This model is worth your time.

And probably your RAM.