Setting up Sarvam 105B as coding agent

A few months back, Sarvam announced its indigenous open source models (Sarvam-105B and Sarvam-30B) at India AI Impact Summit 2026, which I attended in person.

Recently, I became curious about experimenting with these models as coding agents, but I was surprised to find that there are no guides or experiments showing how to use these models for agentic coding.

Both the models are capable of reasoning and agentic tasks, and the Sarvam-105B model does have promising benchmark metrics. So, out of curiousity, I decided to set it up as my coding agent.

One of the reasons it’s not widely used/experimented as coding agent is that it’s currently not available as a built-in provider in popular coding agent tools. Getting them up and running requires a bit of manual setup, which creates friction for users. I expect that to change in future as these models (or its successors), become directly integrated in these popular coding tools.

In this guide, we’ll go through how to set up Sarvam-105B as a coding agent using **OpenCode as the agent harness, [Sarvam API](https://docs.sarvam.ai/api-reference-docs/getting-started/welcome) and [LiteLLM](https://docs.litellm.ai/)** as proxy. after which, I’ll share some results from the experiments that I did with the coding agent.

Sarvam-105B benchmarks

Contents:

1. Setting up your sarvam developer account
2. Getting OpenCode up ready as harness
3. Setting up LiteLLM proxy with sarvam api key
4. opencode configurations for proxy
5. Sending the first prompt to the coding agent!
6. fun experiments with the model as a coding agent

This blog post has detailed guide about each section. So, feel free to skip the content and move to the setting up LiteLLM proxy section if you have already setup your Sarvam account and OpenCode.

Setting up Sarvam developer account (with free credits)

For us to setup the model running with opencode, we’ll first need an API key from the Sarvam platform and process is pretty straight-forward:

1. Login into the **Sarvam dashboard**. (New users get free 100 API credits)
2. From the left sidebar select “Sarvam API”. This will open up the Sarvam API dashboard and sidebar with API related options.

Sarvam Dashboard

3. Select API Key option in the sidebar. If this is first time you’re setting up, you will have an empty dashboard without any api keys. To create one, click on the Create API Key button at the top right.

API Key dashboard (Sarvam Platform)

Make sure to save the key somewhere as you can only copy it once from the dashboard.

Sarvam gives us an OpenAI-compatible API so we can use agent tools to connect to the model by using it’s base url.

Getting OpenCode ready as a harness

For us to use the Sarvam-105B model, we’ll need an agent client, we can connect this model to, for doing some agentic coding. There are lot of popular options (such as **pi, [cline](https://cline.bot/), [opencode](https://opencode.ai/)**, etc) but we’ll be using opencode in this guide.

opencode (https://opencode.ai/)

1. Install opencode using the curl command or using other available options on your machine (You may refer the official docs for various installation methods).

curl -fsSL https://opencode.ai/install | bash

2. Once installed, move to your project directory and run the command opencode in the terminal to start the opencode client. To close opencode, enter /exit in the chatbox of the opencode client.

Feel free to refer the official docs for various available TUI commands https://opencode.ai/docs/tui/

After installing OpenCode, we’re ready to setup a proxy with LiteLLM. We’ll comeback for setting up opencode configuration after that.

Setting up LiteLLM Proxy with Sarvam API

Sarvam exposes an OpenAI compatible chat completion API that we can utilize in our LiteLLM proxy setup.

LiteLLM proxy is a server that sits between a application and model provider (e.g. Gemini, Anthropic, OpenAI, Sarvam, etc). So instead of opencode directly talking to the model provider, it talks to this proxy server via single API. This allows us to add some custom filters to the requests such as custom rate limiting or request logging or centralized API management so that we can use multiple models from the same API.

Sarvam chat completion OpenAI-compatible API

1. Install LiteLLM proxy (using PIP or bash command)

pip install litellm[proxy]

curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/install.sh | sh 

Refer to the quick start docs if you’re using the curl command for the setup.

2. Create a config.yaml file in your project folder with the following fields:

model_list:
  - model_name: sarvam-105b
    litellm_params:
      model: openai/sarvam-105b 
      api_base: "https://api.sarvam.ai/v1"
      api_key: "paste_the_sarvam_api_key_here"
    model_info: 
      max_tokens: 4096

3. Start the proxy server using the following command:

litellm --config config.yaml

This will start the proxy at [*http://localhost:4000/v1](http://localhost:4000/v1)*

we’ll come back to edit config.yaml after the opencode configuration for troubleshooting errors.

OpenCode configuration for proxy

OpenCode needs some configuration so that it can talk to the proxy server that we just set up in the previous steps.

In the project’s root, create a file named opencode.json with the following JSON fields:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "sarvam-local": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Sarvam Local Proxy",
      "options": {
        "baseURL": "http://localhost:4000/v1"
      },
      "models": {
        "sarvam-105b": {
          "name": "Sarvam-105B (Local Proxy)",
          "limit": {
            "context": 128000,
            "output": 1500
          }
        }
      }
    }
  },
  "model": "sarvam-local/sarvam-105b"
}

The sarvam API has a max token limit of 4096 and that’s one of the big limitations for working around with it as a coding agent. That’s why we limit output tokens to 1500 so that we have more space for input.

Sending first prompt to the Coding Agent!

Now let’s start opencode (making sure that the LiteLLM proxy is running as well in different terminal tab).

1. Use the /connect command in the chatbox to connect to the proxy provider. Search for “Sarvam local proxy”, select it and enter a dummy api key as litellm proxy will handle the api key for us.

2. Select the “Sarvam-105B (Local Proxy)” model from the selection.

3. Send a sample prompt like “Write a hello world python program in hello_world.py file”.

Now you might get a “litellm.BadRequestError: OpenAIException” error from LiteLLM proxy as shown in the following image:

This happens because Sarvam expects the the request content to be non empty. When opencode calls a tool, assistant’s message comes back with a tool call field but an empty content, i.e. a message with no text. To fix this we’ll need to modify the config.yaml for the proxy server to add some content to the message so that they don’t get send as empty strings.

For this, we’ll use LiteLLM’s custom logger to modify the request data and add some text if it’s a tool call.

So, let’s just update the config.yaml file in your project’s root that we created for the LiteLLM proxy in earlier steps:

# config.yaml
model_list:
  - model_name: sarvam-105b
    litellm_params:
      model: openai/sarvam-105b # or your specific endpoint
      api_base: "https://api.sarvam.ai/v1"
      api_key: "YOUR_SARVAM_API_KEY"
    model_info:
      max_tokens: 4096

litellm_settings:
  modify_params: true
  callbacks: custom_callbacks.proxy_handler_instance #custom callback

We now have callbacks field which has a custom python script that’ll transform the requests before they reach sarvam. For this, Create a new file named custom_callbacks.py with the following script:

from litellm.integrations.custom_logger import CustomLogger

class SanitizeEmptyToolContent(CustomLogger):
    async def async_pre_call_hook(self, user_api_key_dict, cache, data, call_type):
        messages = data.get("messages", [])

        for msg in messages:
            content = msg.get("content")
            is_empty = (
                (isinstance(content, str) and content.strip() == "")
                or (isinstance(content, list) and len(content) == 0)
                or content is None
            )
            if is_empty:
                msg["content"] = "[Tool Execution, No Content]"

        return data

proxy_handler_instance = SanitizeEmptyToolContent()

This is a simple custom logger script generated by Claude to make sure we’re not sending empty content to sarvam as a request.

For detailed explanation of how this works, refer to this **Notion Document**.

Now let’s retry sending a prompt in opencode. But, before that:

1. Save the config.yaml and restart the proxy using thecommand litellm --config config.yaml

2. Restart opencode by first using the /exit command and opencode again.

3. Let’s try a different prompt for the Agent this time: “Take a look at pwd, create new file “bubble.py”, write bubble sort algorithm for list of integers in it.”

Sarvam-105B Agentic workflow 1

Sarvam-105B Agentic Workflow 2 (re-prompting)

It should call some bash commands, try to create a new file and edit that file. If it does not, try re-prompting it as a followup message. It will sometimes ask you again for command execution permission if it fails. Try creating new session if things don’t work even after re-prompting.

Woohoo! So, finally after lot of work, we successfully set up Sarvam-105B model as a coding agent in opencode. There are definitely some issues like occasional tool call failures but it was super fun setting this up running!

Fun Experiments with Sarvam -105B Coding Agent

1. Building a webpage.

I asked Sarvam 105B coding agent to create a fun website about itself. There were some errors during tool executions for creating and writing a file but it was able to do it after some reattempts and came up with this simple animated website.

Website created by Sarvam-105B as a coding agent in opencode

2. Reading files:

In a new session, I asked the agent to read index.html file in the current directory (the file it just made for the webpage), then find UI/UX issues with the website code. It did pretty well at that explaining the problems in UI/UX.

File reading by Sarvam -105B Agent.

I’ll keep updating this section as I’m really curious and interested in doing some more experiments such as

1. Multi-file edits,
2. Codebase understanding,
3. Fixing bugs in a Github repo,
4. Implementing features,
5. Using more tools and so on.

Current token limits make it very hard to do that but I definitely think there will be better token limits in the future. If you build something cool with this agent setup or just wanna share your thoughts, please feel free share it in the comments!

Thanks for reading! I hope you enjoyed reading it and found it helpful :)