Practical Gemini CLI: Intelligent Model Router

Gemini CLI is amazing!

You don’t need to use it only for coding. I’ve used it to scrape content and make sense of it, process PDFs, create presentation proposals and even generate slides, and I’ve even used it to help build my speaker bio for a recent talk, by scanning through all my public content on the internet!

Some of these tasks are simple, whereas others are complex. Some tasks take a lot of token, while others don’t. Is there an optimal and efficient way to use Gemini CLI?

Gemini CLI

First things first. There are 3 methods of authentication when you use Gemini CLI.

> 1. Login with Google
> 2. Use Gemini API key
> 3. Vertex AI

Let’s start from the bottom.

When you use Vertex AI — and bring in your Application Default Credentials, it will use the billing associated with your Google Cloud Project and you will be billed as per the usage of the Gemini API.

When you use Gemini API key, be it via AI Studio, or via Vertex AI on Google Cloud, the billing depends on the type of key you have. If you have a free key on Google AI Studio, then you will be subject to the free quotas and limits as defined on AI Studio. Similarly, if you have Tier 1/2/3 billing, you get higher quotas but pay for the Gemini API usage depending on the tokens and “thinking” that you consume.

Finally, you can use your personal Gmail account to authenticate and use Gemini CLI. This gives you some very generous free quotas with Gemini CLI, and should you want more quotas on an individual plan (without the hassle of API keys etc.), then you can opt for the Google AI Pro or Ultra plans which gives you enhanced quotas for Gemini CLI and Gemini Code Assist.

That said, you will realise that you have higher quotas on the more mainstream models, whereas the quotas on the advanced models are limited. So in order to use Gemini CLI more efficiently, it is best to use the right model for the task, namely using the light-weight models for simpler tasks and the pro models for more complex “thinking” tasks.

So, how do you do this?

Glad you asked. With the v0.11.2 patch that was recently released, the Intelligent Model Router is enabled by default for all users. As mentioned in the linked page ->

Why Model Routing? 🤔

Until now, Gemini CLI used one primary model (like Gemini 2.5 Pro) for your entire session. While fantastic for complex tasks, using such a heavy-duty model for simpler requests often increased latency and burned through your Pro quota faster than necessary.

Intelligent Model Routing solves this by dynamically directing your requests to the model best suited for the job. Simple queries are handled instantly by faster, lighter models (like Flash), while complex analytical or creative tasks continue to leverage the full power of Pro.

By upgrading, you gain:

• 📉 Reduced latency for a snappier, more responsive command-line experience.
• 💰 Extended quotas by automatically reserving the Pro model only for tasks that truly require it.

Try it out to see it working. You will be able to see that it is active by the “auto” text on the bottom right, where it indicates the model being used. Of course you can use /model to change the model being used, deterministically, for the duration of the session, or until you change it again.

But, how does this work?

You can use this and get a feel for it yourself, however power users would like to know if this actually works, and how does it do the routing. Btw, which model do you think will be used for the actual routing?

OSS for the WIN!!

Let’s start with some analysis and exploration. I mean, what’s the fun in being handed the answer sheet during the exam, huh?

API Stats

Gemini CLI has this cool /stats command to show you details about API usage — token usage and latency. The /stats model command will give you more model-specific details.

When you start a new session, of course no API calls have been made.

Side Note: Have you been reading my blogs? I have 2 GEMINI.md files it says, but shouldn’t that have been sent to the model and counted towards the API calls? How does that work? I’ve written in detail about this in the blog post “Decoding the Context”.

Now after I send my first command — “Hi”, I can then run the /stats command and see the API usage.

Note: Model routing is now enabled.

API stats after a single call

What’s happening here? Calls are being made to 2 models.

1. The gemini-2.5-flash-lite model with 1 API call, and a latency of 1.6s with a prompt length of about 9000 tokens.

2. The gemini-2.5-flash mainstream model with 2 API calls, an avg latency of 2.6s and about 25000 tokens sent.

What’s happening?

Let’s investigate further with another simple prompt.

API stats after a second call

So, the calls have doubled now with 2 more calls to flash and 1 more call to flash-lite. With double the tokens too.

However, there is an interesting bit — the cached tokens. The flash model has a ~50% cache hit which means a prompt was repeated and it used the cache, saving cost and time. We can dig deeper into the prompt where there was a cache hit later if required.

Let’s try something more complex now.

So I did this ->

prompt for a complex task

and it gave me an animation / drawing of the output

Gemini CLI output

I expect that this is a complex enough task that demands some thinking and it might have leveraged the pro model. Let’s check the /stats model.

API stats for a complex task

As expected, there was indeed a call to the pro model now as you can see in the stats.

1. There are 2 more calls made to lite of which 1 errored out, so likely we can count it as 1 call to lite. Net tokens to lite is very similar (1K more) as before, so very little delta tokens in the 2 additional calls
2. There was 1 more call made to flash, and about 10K extra tokens exchanged.
3. Finally 1 new call made to pro which also has a cache hit.

The key here is that the pro model does take time, and hence it makes sense to use it for complex tasks and not for simple tasks. Over and above the cost aspect, the latency and performance is even more important for a simple task.

Note: Theconcept of caching is an interesting one, and I think I will take that up in another blog. For now it is good to see caching working and saving on tokens.

Let’s try a very simple task now and see what model is used.

A simple prompt

Stats after a simple prompt

We see now that flash-lite was called 1 additional time, flash was called 3 more times, and pro wasn’t called at all — likely because this wasn’t that complex a task that it needed gemini-2.5-pro.

Think about what has been happening. Try to interpret some of these results and numbers, and try to figure out how model routing is working. Once you have thought about it, read on below to understand in more detail what is happening.

Hint: I have pasted the output of the simple prompt above. Notice what it tried to do (irrespective of whether you agree to it or not), and interpret the API calls made. I’d love to read your thoughts and interpretations in the comments.

Routing Logic

The most critical piece of the routing logic is the prompt. How exactly does the model decide which model has to be invoked to execute the user’s task?

In the announcement thread on X, there were a bunch of questions on how efficient the model routing is, and whether it optimises for cost, etc.

The answer to all these questions is in the file [classifierStrategy.ts](https://github.com/google-gemini/gemini-cli/blob/main/packages/core/src/routing/strategies/classifierStrategy.ts).

We have the CLASSIFIER_SYSTEM_PROMPT which has the following instructions (not completely replicated here, just the important bits).

You are a specialized Task Routing AI. Your sole function is to analyze the user's request and classify its complexity. Choose between \`${FLASH_MODEL}\` (SIMPLE) or \`${PRO_MODEL}\` (COMPLEX).

<complexity_rubric>
A task is COMPLEX (Choose \`${PRO_MODEL}\`) if it meets ONE OR MORE of the following criteria:
1.  **High Operational Complexity (Est. 4+ Steps/Tool Calls):** Requires dependent actions, significant planning, or multiple coordinated changes.
2.  **Strategic Planning & Conceptual Design:** Asking "how" or "why." Requires advice, architecture, or high-level strategy.
3.  **High Ambiguity or Large Scope (Extensive Investigation):** Broadly defined requests requiring extensive investigation.
4.  **Deep Debugging & Root Cause Analysis:** Diagnosing unknown or complex problems from symptoms.
A task is SIMPLE (Choose \`${FLASH_MODEL}\`) if it is highly specific, bounded, and has Low Operational Complexity (Est. 1-3 tool calls). Operational simplicity overrides strategic phrasing.
</complexity_rubric>

**Output Format:**
Respond *only* in JSON format according to the following schema. Do not include any text outside the JSON structure.
{
  "type": "object",
  "properties": {
    "reasoning": {
      "type": "string",
      "description": "A brief, step-by-step explanation for the model choice, referencing the rubric."
    },
    "model_choice": {
      "type": "string",
      "enum": ["${FLASH_MODEL}", "${PRO_MODEL}"]
    }
  },
  "required": ["reasoning", "model_choice"]
}

Then there are some examples that indicate how to interpret and respond to the prompt.

These are clear and simple instructions to make an educated determination on the type of model that should be used to process the user’s input.

What’s missing here? Where is the actual user’s prompt?

      // Take the last N turns from the *cleaned* history.
      const finalHistory = cleanHistory.slice(-HISTORY_TURNS_FOR_CONTEXT);

      const jsonResponse = await baseLlmClient.generateJson({
        contents: [...finalHistory, createUserContent(context.request)],
        schema: RESPONSE_SCHEMA,
        model: DEFAULT_GEMINI_FLASH_LITE_MODEL,
        systemInstruction: CLASSIFIER_SYSTEM_PROMPT,
        config: CLASSIFIER_GENERATION_CONFIG,
        abortSignal: context.signal,
        promptId,
      });

Further down in the file, we see what all is passed to the model to help it to determine the right model. The important one is the CLASSIFIER_SYSTEM_PROMPT itself which is the instruction to make the determination.

Then you see contents which is composed of the user’s actual input context.request which forms the basis of the decision making, and very interestingly, ...finalHistory which you will see is the last 4 turns from the conversation history. Likely this is to provide more context to the model in order to help it make its determination.

And then, the most important bit — the model that is used to make this decision — gemini-2.5-flash-lite.

Now relate all this back to the screenshots that we were seeing.

1. The calls to gemini-2.5-flash-lite increased by 1 count each time we sent a prompt. This is because the lite model is used for making the routing decision — it is an extremely cheap model, and a very capable model as well.
2. This model increments only once per turn, since the resolved models are the flash and pro models only. So the actual processing of the user’s input is never done by the lite model.
3. Since the prompt sent to the model included not just the classifier system instruction, but also the user’s input and the last 4 turns of the conversation history, clearly the prompt is very different each time it is sent to the router. Hence you don’t see any cache hit.

There are a bunch of things that are still not answered. I am going to leave this as an exercise for your readers. Let me know what you think.

1. In the very first call, the flash model is called 2 times. Why do you think this is?
2. In the 3rd try with the complex prompt, the flash model was called 1 time, whereas the pro model is the one that probably actually did the processing. Why was the flash model likely used?
3. In the final example, the flash model was called 3 times. What do you think happened here?

Note: My answers for these 3 questions, too, are an educated guess. I haven’t debugged the system to exactly trace what happened. However I believe I have sufficient understanding of the way the CLI works, to wager a guess on what might actually be happening. I’d love to discuss this with y’all and reason with your perspectives.

Read my series on Practical Gemini CLI, and Advanced Gemini CLI to understand some of these aspects better.

In Conclusion

The model router is a much needed feature to optimise both cost and latency. It helps your quotas go much further while you don’t notice any degradation of results.

If you haven’t yet, do upgrade to the latest version of Gemini CLI

npm install -g @google/gemini-cli@latest

It is turned on by default, however you can still go into /settings within Gemini CLI and turn on the Use Model Router flag to get going.

At any time within a session, you can force the usage of a specific model using the /model command and select a model. Switch back to auto to use the model router.

Try this feature out and let us know what you feel about it.