AI Agents - To tool or not to tool…

AI Agents with tools and skills can be very powerful, but I’ve been wondering how many tokens it actually cost to instrument your AI Agents with tools. Adding tools is not free. They take up context and the Agent has to figure out what tools to call and when to call them. Those things cost tokens.

Another consideration is time. Each roundtrip to the LLM takes time.

So to find out I’ve done a very simple experiment where I’ve implemented the Evaluator-Optimizer pattern in three different ways:

1. Multi-agents with no tools, orchestrated by a workflow
2. Multi-agents with tools, orchestrated by simple code logic
3. Autonomous multi-agents with tools, orchestrated by a manager agent

The evaluator optimizer pattern. Source: Anthropic

The idea is simple. Run these three implementations 10 times each and calculate the average token consumption and execution time. Then compare the results.

I’m using .NET, the Microsoft Agent Framework and LLMs from Azure Foundry.

The Experiment

To keep it simple I’ll implement a console application that can summarize product reviews by a given product name. Reviews contains a text, a score, a product name and a reviewer username. They are placed in a document database.

Multi-agents with no tools, orchestrated by a workflow

Microsoft Agent Framework comes with a workflow library that can be used to create… workflows. It’s out of scope here to go into details, but it allows you to create executors and edges that can be used to compose pretty complex workflows. The executors can be either custom code or an AI Agent. The edges can be simple connectors of executors or more advanced conditions, switches and fans.

For this experiment, my implementation of the evaluator-optimizer pattern, with the agent workflow looks like this.

Implementation of the evaluator-optimizer pattern using a workflow

The first executor gets the reviews from the database and puts them in the context object. Next a conditional edge edge makes sure that there is more than one review, else it does not make sense to make a summary. After that we get to the generator and evaluator loop that continues until the summary is approved or a limit is reached.

Compared to an AI Agent, a workflow is a predefined sequence of operations, where some of the operations may be calls to an LLM and other operations might be simple logic like fetching data from an API. AI Agents are more autonomous.

The source code for this implementation can be found here https://github.com/simon-k/agent-patterns/tree/main/AgentPatterns.MultiAgentEvaluation.WorkflowOrchestration

Multi-agents with tools, orchestrated by simple code logic

A more autonomous solution would be to create two agents that uses tools to get the product reviews and then add some logic to handle retry if the summary was not approved by the evaluator agent.

The implementation of the evaluator optimizer pattern then looks like this:

Implementation of the evaluator-optimizer pattern using agents with tools

The implementation is a bit simpler than the workflow. An AI Agent handles the summary generation and another AI agent handles the review. Both agents uses a tool to get the reviews. Both agents returns structured output that is then used to control the feedback loop.

The source code for this implementation can be found here https://github.com/simon-k/agent-patterns/tree/main/AgentPatterns.MultiAgentEvaluation.CodeOrchestration

Autonomous multi-agents with tools, orchestrated by a manager agent

The third solution is the fully autonomous manager agent that uses child agents to do everything. Here we really let our hands off the steering wheel.

The implementation of the evaluator optimizer pattern then looks like this:

Implementation of the evaluator-optimizer pattern using a manager agent

The manager agent and the child agents are implemented by letting the manager use the agents as tools. Really simple and powerful for the right scenarios.

The source code for this implementation can be found here https://github.com/simon-k/agent-patterns/tree/main/AgentPatterns.MultiAgentEvaluation.ManagerOrchestration

Result

So I ran each implementation 10 times each, using GPT-5-mini. To have comparable flows, I made sure to exclude executions where the evaluator dismissed the reviews.

So the difference is obvious. The difference between the workflow and the programmatic orchestration is the AI Agent tools. The workflow is not using tools to get the data, and that saves both initial tokens in the context and roundtrips to the LLM This shows both in the cost and execution time.

Having a completely autonomous manager is more than twice as expensive and the execution time is also more than double.

Conclusion

The result shows that it is really important to select the right implementation when dealing with AI. And it all comes down to efficiency vs flexibility. A couple of thousand tokens might not sound like a lot, but when it scales, then I’m sure your manager cares about a double up on the cost. And considering UX, then performance is always important. Would you rather have your response in 12 seconds or 29 seconds? The answer is obvious. But of course, sometimes the fully autonomous solution is preferable. It depends on the use case. When you have a fixed flow then consider using a workflow.

Another thing I haven’t touched here is the reliability of the final result of the solution. Every time you have a roundtrip to the LLM there is a slight chance that it will go wrong. It might hallucinate or call the wrong tool. So minimize the number of roundtrips to the LLM. It will increase your chances of getting a reliable response, and it will save you tokens.

Final notes

In my example above I really think that the workflow is the right approach. But one interesting and potentially very useful feature of the Agent Framework Workflow is that you can convert it to an agent

var myAgent =myWorkflow.AsAIAgent();

So you can combine a workflow and other agents really easily.