Squeeze the most out of Qwen 3.6

Introduction

Running large language models (LLMs) locally offers significant advantages in data privacy, inference performance, and system customization. With an NVIDIA GPU featuring 24 GB of VRAM, you can deploy powerful local models capable of web search, application development, code refactoring, translation, and document summarization.

This guide provides step-by-step instructions for installing and configuring the following components on Ubuntu 24.04:

• llama.cpp — the inference engine, with support for Google TurboQuant and Multi-Token Prediction
• Qwen 3.6 27B — a dense 27-billion-parameter LLM
• OpenCode — an AI coding agent
• Graphify, Playwright, Context, and PDF-Reader — MCP tools for web automation, document parsing, documentation lookup, and knowledge graphing

Prerequisites

• Ubuntu 24.04
• A GPU with 24GB VRAM like NVIDIA 3090, 4090 or 5090.

On an NVIDIA RTX 4090, the model typically achieves ~70 tokens/second while maintaining a 90K context window, with the vision model loaded into GPU.

Llama.cpp

llama.cpp serves as the primary inference engine and includes a built-in web interface. To maximize local model performance, this guide leverages Google DeepMind’s TurboQuant compression and Multi-Token Prediction (MTP).

• TurboQuant is a KV cache compression algorithm that significantly reduces LLM memory usage with negligible accuracy loss, enabling larger context windows without additional VRAM.
• Multi-Token Prediction (MTP) allows the model to predict multiple future tokens simultaneously during a single inference step. This technique can nearly double generation throughput without increasing memory overhead.

As of this writing, the official llama.cpp repository supports MTP but lacks native TurboQuant integration. For this reason, we will use a maintained community fork that includes both features.

Open a terminal and execute the following commands:

git clone https://github.com/TheTom/llama-cpp-turboquant.git
cd llama-cpp-turboquant
cmake -B build -DGGML_CUDA=ON && cmake --build build -j

Qwen 3.6 27B

Qwen 3.6 27B, fine-tuned by Unsloth, is highly optimized for agentic and coding tasks. The following command automatically downloads the model with MTP support from Hugging Face and launches the inference server:

./build/bin/llama-server \
  --hf-repo unsloth/Qwen3.6-27B-MTP-GGUF:UD-Q4_K_XL \
  --jinja \
  --chat-template-kwargs '{"preserve_thinking": true}' \
  --alias "Qwen3.6-27B" \
  --n-gpu-layers 999 \
  --ctx-size 90000 \
  --cache-type-k q8_0 \
  --cache-type-v turbo4 \
  --host 0.0.0.0 \
  --port 8084 \
  --reasoning-budget -1 \
  --temp 0.6 \
  --top-p 0.95 \
  --top-k 20 \
  --min-p 0.0 \
  --presence-penalty 0.0 \
  --repeat-penalty 1.0 \
  --flash-attn on \
  --parallel 2 \
  --spec-type draft-mtp \
  --spec-draft-n-max 2

Vision Model

If you use the vision capabilities infrequently, you can offload the multimodal projector to system RAM (~1 GB) to reclaim VRAM for a larger context window:

--ctx-size 131072
--no-mmproj-offload

The vision model remains highly valuable during application development, particularly when an agent needs to capture and analyze screenshots of running interfaces for testing or debugging.

Turbo Quant

Enable TurboQuant for the value cache using the following flag, as recommended by the fork’s documentation:

--cache-type-v turbo4

Multi-Token Prediction

The following parameters activate MTP inference:

--spec-type draft-mtp
--spec-draft-n-max 2

Creativity and thinking mode

For analytical and precise coding tasks, use the following parameters to prioritize deterministic, structured outputs:

--temp 0.6
--top_p 0.95
--top_k=20
--min_p=0.0
--presence_penalty=0.0
--repeat_penalty=1.0

For open-ended or creative tasks, increase the temperature to encourage more diverse generation:

--temp 1.0
--top-p 0.95
--top_k=20
--min_p=0.0
--presence_penalty=0.0
--repeat_penalty=1.0

OpenCode

OpenCode is an open-source, terminal-native AI coding agent designed to assist developers with code generation, refactoring, debugging, and project automation. Unlike traditional AI assistants that are embedded within IDEs, OpenCode is built as a standalone, command-line-first tool with optional web and API interfaces, making it highly adaptable to local and privacy-focused workflows.

Installation on Ubuntu is straightforward:

curl -fsSL https://opencode.ai/install | bash

OpenCode supports multiple execution modes:

• CLI Interface: Launch the terminal-based agent

opencode

• Web Interface: Access the graphical dashboard

opencode web --hostname 0.0.0.0

• API-Only Mode: Run a headless service for integration with scripts or external tools

opencode serve

Following is a typical configuration file ~/.config/opencode/opencode.json. The different MCP components are explained further down the document.

{
  "$schema": "https://opencode.ai/config.json",
  "default_agent": "build",
  "compaction": {
    "auto": true,
    "prune": true,
    "reserved": 8192
  },
  "mcp": {
    "playwright": {
      "type": "local",
      "command": ["npx", "@playwright/mcp@latest"],
      "enabled": true
    },
    "pdf-reader": {
      "type": "local",
      "command": ["npx", "@sylphx/pdf-reader-mcp"],
      "enabled": true
    },
    "context": {
      "command": ["context", "serve"],
      "enabled": true,
      "type": "local"
    }
  },
  "provider": {
    "llamacpp": {
      "name": "llama.cpp",
      "npm": "@ai-sdk/openai-compatible",
      "options": {
        "baseURL": "http://127.0.0.1:8084/v1",
        "apiKey": "local"
      },
      "models": {
        "local-model": {
          "name": "llama.cpp local model",
          "tool_call": true
        }
      }
    }
  }
}

MCP servers

Model Context Protocol (MCP) servers extend OpenCode beyond plain text generation by giving the local model access to external tools, data sources, and workflows through a standardized interface. In this setup, MCP acts as the bridge between Qwen and the surrounding development environment: the model can inspect websites, read documents, retrieve live documentation, and build structured project memory instead of relying only on its training data or the files currently loaded into context.

The recommended MCP servers are:

• Graphify / Knowledge Graph — builds a queryable knowledge graph from code, documents, diagrams, and project artifacts. Instead of repeatedly scanning the same files, the agent can query relationships between modules, functions, dependencies, requirements, and architectural concepts. This provides a lightweight memory layer for large or long-running development projects.
• Playwright — provides browser automation capabilities. The agent can open pages, click buttons, fill forms, inspect accessibility trees, and verify application behavior in a real browser. This is especially useful for end-to-end testing, UI debugging, and validating web applications generated or modified by the coding agent.
• Context — gives the agent access to up-to-date, version-specific documentation and code examples for libraries and frameworks. This reduces hallucinated APIs and is particularly useful when working with fast-moving ecosystems where the model’s built-in knowledge may be outdated.
• PDF Reader — allows the agent to extract text, metadata, and structured content from PDF documents. This makes it possible to summarize papers, inspect manuals, analyze reports, or pull implementation details from documentation without manually converting files first.

Together, these MCP servers turn OpenCode into a much more capable local agent.

Playwright gives it eyes and hands in the browser, PDF Reader lets it consume external documents, Context keeps its coding knowledge current, and Graphify gives it structured long-term understanding of the project.

Graphify

https://github.com/safishamsi/graphify

Graphify is an essential tool for navigating large codebases. It analyzes your repository to map the relationships between methods and components, generating a structured knowledge base that enables efficient code querying.

By leveraging this pre-built index, the LLM avoids re-parsing the codebase for every query, resulting in faster and more accurate responses.

To install Graphify, execute the following commands:

curl -LsSf https://astral.sh/uv/install.sh | sh
uv tool install graphifyy

To integrate Graphify with OpenCode in your repository, run:

graphify install --project --platform opencode

Within the OpenCode interface, trigger a full repository scan by running:

/graphify .

Once indexed, you can ask targeted questions such as: “How does this component load data, and how is that data passed to neighboring components?”

Playwright

https://github.com/microsoft/playwright-mcp

Playwright MCP provides browser automation capabilities, allowing the agent to navigate web pages, interact with UI elements, fill forms, capture screenshots, and inspect accessibility trees. Combined with Qwen’s vision model, it enables the agent to observe and validate running web applications — making it ideal for end-to-end testing, UI debugging, and verifying generated code in a real browser environment.

No separate installation is required. Playwright runs through npx and is enabled by adding the following entry to the mcp block in your opencode.json configuration file (~/.config/opencode/opencode.json):

"playwright": {
  "type": "local",
  "command": ["npx", "@playwright/mcp@latest"],
  "enabled": true
}

Once configured, the agent can open URLs, click elements, type into fields, take screenshots, and evaluate JavaScript — all within a headless browser instance.

Context

https://github.com/neuledge/context

Context MCP gives the agent access to up-to-date, version-specific documentation and code examples for libraries and frameworks. Instead of relying on stale training data, the agent can query live documentation for any installed package, reducing hallucinated APIs and improving code accuracy — especially in fast-moving ecosystems.

To install Context, add the following entry to the mcp block in your opencode.json configuration file (~/.config/opencode/opencode.json):

"context": {
  "command": ["context", "serve"],
  "enabled": true,
  "type": "local"
}

Once configured, the agent can search documentation for any library by name and version, or download documentation packages on demand when needed.

PDF Reader

https://github.com/SylphxAI/pdf-reader-mcp

PDF Reader MCP allows the agent to extract text, metadata, tables, and embedded images from PDF documents — both local files and remote URLs. This makes it possible to summarize research papers, inspect technical manuals, analyze reports, or pull implementation details from documentation without manual conversion.

No separate installation is required. PDF Reader runs through npx and is enabled by adding the following entry to the mcp block in your opencode.json configuration file (~/.config/opencode/opencode.json):

"pdf-reader": {
  "type": "local",
  "command": ["npx", "@sylphx/pdf-reader-mcp"],
  "enabled": true
}

Once configured, the agent can read PDFs by file path or URL, optionally extracting specific pages, metadata, tables, or embedded images as needed.