Opencode is capable of doing so much more, but I’ll use it as a chat

A dive into the hidden friction between local LLMs and autonomous agents that leaves us doing the old-school Ctrl+C / Ctrl+V dance.

Since I completely missed the initial LLM hype train back in 2022 and 2023 — honestly, like many Ukrainians at the time, I had far more critical things to worry about than chatting with neural networks — I recently decided I couldn’t let the local AI revolution pass me by. Let’s see what an open-source model is capable of on weak, old hardware in the hands of a guy who is returning to the industry after almost a year, and why open-source stack refused to create files, leaving me to copy pieces of code from the terminal and return them back to the terminal with every iteration.

The Battle Station

I’m running a pretty standard, middle-of-the-road consumer setup. The author possesses the following goods:

OS: Windows 11

RAM: 16GB of RAM

GPU: 8 GB of VRAM, RTX-4060

Backend Engine: llama-server build 9222 (9a532ae4b) with Clang 19.1.5 for Windows x86_64

Agent Engine: opencode 1.15.5

Llama.cpp and qwen2.5-coder-7b-instruct-q6

It all started exactly with this setup. Just in time, I came across an excellent article “Why I Stopped Using Gemma 4 and Switched to Qwen 3.6”. Having 16GB RAM + 8GB VRAM, I thought that things weren’t all that bad; such a configuration would be enough to just run a model and see how it works. Since my home PC has memory limitations, for the practical test I took the younger brother from this lineup — Qwen 2.5 Coder 7B, which fits perfectly into my 8GB VRAM. A few evenings spent reading the documentation later, I was watching my weak hardware generate tokens at a speed of 39 t/s. Well, of course, it’s not Claude Code, but it’s no worse than GPT-4, I thought.

How about we generate the whole project entirely?

In 2024–25, before I left the IT industry to work on my PhD project, I worked on several “AI-based” projects. We built various services, from translating a project from an old stack (like COBOL) to a modern one, a smart assistant for a medical company, to a RAG service for a well-known electronics manufacturer. And in all of these projects, I never delegated task execution directly to the LLM.

*“If you are typing a prompt, getting an answer, and closing the tab, your operational unit economics are broken.”*

Well, here was the perfect opportunity to check out how it feels :)

An LLM is the best prompt designer for another LLM

(I accidentally overheard this thought in an interview with Dr. Roman Yampolskiy.)

Since then, I very often ask a model to prepare a prompt that describes a specific task instead of asking the question directly. In most cases, already at this stage, it’s possible to make a lot of corrections. After some time, I held a certain basic prompt for deploying the project in my hands.

And this is where the most interesting part began.

The issue: progress runs beautifully in the terminal, but the files aren’t there

Opencode (OC) picked up the request to deploy the project and compiled a clear list of actions to be performed. After the final approve, OC rushed to launch agents, perform analysis, and generate code according to the specification.

But the files kept not appearing in the directory.

No, don’t you dare think that I expected all of this to be that simple.

After a multiple examination of the following OC configuration:

{
 "$schema": "https://opencode.ai/config.json",
 "lsp": true,
 "shell": "pwsh",
 "model": "llama.cpp/qwen2.5-coder-7b",
 "provider": {
 "llama.cpp": {
 "npm": "@ai-sdk/openai-compatible",
 "options": {
 "baseURL": "http://localhost:8080/v1",
 "apiKey": "not-needed"
 },
 "models": {
 "name": "qwen2.5-coder-7b"
 }
 }
 },
 "permission": { "bash": "allow", "edit": "allow" }
}

And making sure that "edit": true is explicitly specified, and "shell" points to "pwsh" I thought that probably llama.cpp was doing something wrong. Or maybe the model itself decided to work as it sees fit. Llama.cpp is just a server that creates all the necessary conditions to run the model. Let's leave it alone for now and see how the model behaves.

So, quick thoughts on the issue:

• OC does nothing by itself. Instead it delegates all of the work.
• In that case it has to have a set of tools.
• If no files are being created, then the dedicated tool is probably missing today.

Let’s see if my assumption has anything to do with reality. To do this, we’ll run a simple script:

import json
import pprint

from openai import OpenAI
client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="not-needed" 
)
def get_current_weather(location: str, unit: str = "celsius") -> dict:
    """
      There should be really good docstring, since llm will make a decision based on
      it. Non-relevant for this case.
    """
    supported_units = ["celsius", "fahrenheit"]
    if unit.lower() not in supported_units:
        raise ValueError(f"Invalid unit '{unit}'. Expected one of {supported_units}")

    normalized_location = location.lower()
    if "paris" in normalized_location:
        return {"temperature": "18", "condition": "sunny", "unit": unit}
    else:
        return {"temperature": "15", "condition": "cloudy", "unit": unit}

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "The city and state, e.g. San Francisco, CA"},
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
                },
                "required": ["location"]
            }
        }
    }
]
response = client.chat.completions.create(
    model="qwen2.5-coder-7b", # Target model name from your server configuration
    messages=[{"role": "user", "content": "What's the weather like in Paris?"}],
    tools=tools,
)

message = response.choices[0].message

print("\n=== RESPONSE ANALYSIS ===")
pprint.pp(message)
print("\n")

And the result:

=== RESPONSE ANALYSIS ===ChatCompletionMessage(
content='```json\n{\n "name": "get_current_weather",\n 
"arguments": {\n "location": "Paris, France",\n "unit": "celsius"\n }\n}\n```', 
refusal=None, role='assistant', annotations=None, audio=None, 
function_call=None, tool_calls=None)

And that actually explains almost everything. As we can see, the model didn’t waste time on trifles and left tool_calls completely empty. Meanwhile, content contains everything needed to execute the request. This is exactly what happens in OC. Instead of calling the appropriate tool to create a file, OC received an empty tool_calls key.

But the second thing I would like to check is the log statements on the other side. Let’s run llama.cpp in verbose mode:

llama-server -m qwen2.5-coder-7b-instruct-q6_k-00001-of-00002.gguf -c 16384 -ngl 99 --verbose

And ask OC to execute a primitive request: “list all folders from the current directory”

And the result in the Opencode’s console:

{"name": "glob", "arguments": {"pattern": "**/*", "path": "./"}}

From the server logs, we are interested in this piece:

 \n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{{\"name\": <function-name>, \"arguments\": 
<args-json-object>}}\n</tool_call><|im_end|>\n<|im_start|>user\nlist all folders from the current directory<|im_end|>\n<|im_start|>assistant\n","has_new_line":true,"truncated":false,
"stop_type":"eos","stopping_word":"","tokens_cached":8888,"timings":{"cache_n":0,"prompt_n":8864,"prompt_ms":13526.019,"prompt_per_token_ms":1.525949796931408,"prompt_per_second":655.3295540986597,
"predicted_n":25,"predicted_ms":708.728,"predicted_per_token_ms":28.34912,"predicted_per_second":35.2744635459584}}}

These records show that a schema was forcibly hardcoded by the OC client into the llama.cpp server, requiring the model to generate outputs exclusively within <tool_call> tags. The local model completely ignored this trigger and outputted its usual text JSON instead, causing the client’s parser to fail to recognize the command.

Did I choose the wrong model?

The first thing I thought was that I rushed and picked the wrong model. Like, I should have spent more time on preparation instead of rushing to launch what fits my hardware configuration right away. For example, looking for options to launch Qwen 3.6–35B-A3B. Perhaps you would be right to reason the exact same way.

I don’t have enough experience yet to state with absolute certainty, but I will assume that we would get the same behavior in the case of Qwen 3.6, Gemma 4, Llama, or Phi — with any other open-source model. Apparently opencode, as an open-source tool, was created with an eye toward commercial OpenAI and Anthropic solutions, and therefore expects operations strictly in accordance with the OpenAI API standard.

A Possible Fix

And here, it seems to me, there will be no simple solution. If an open-source model violates what seems to me to be a generally accepted API standard, then someone from llama.cpp or opencode will have to invent crutches in their design. From personal experience, I wouldn’t be glad at all if someone told me, “Oh you know, we need to somehow bolt on an additional middleware that will return the correct response in the case of such and such an LLM.”

And that means that for now… I will be copying pieces, snippets, from the console and copying pieces of code back into the console :)

The next step I would like to check is configuring the behavior of the model. Perhaps there is a way to adjust the LLM’s behavior by explicitly stating: “Hey, look, everything that belongs to tool_call needs to be put into tool_calls.”

Stay tuned. See ya.