Agentic AI: The Model Didn’t Get Smarter. The Pipeline Did

So far, I have been writing about Transfer Learning, adapting models, breaking down LoRA, understanding tokenizers like EnCodec, and fine-tuning MusicGen to a new domain. All of that lives at the foundational model level. You take a model and you change it to fit a task.

But there is another way to extend what a model can do, and it doesn’t touch the weights at all and is all at the applied level and easy to adapt. It changes how the model is used. That is what Agentic AI is about, and it is what I want to break down in this article.

This article is based on my notes from DeepLearning.AI’s Agentic AI course for module 1, refined into the way I actually think about the topic. A big thanks to Andrew NG for making AI education accessible to all.

PC: A screenshot from course page of Agentic AI in DeepLearning.ai website

Non-Agentic vs Agentic

The most easiest way to understand any new concept or application is comparing and contrasting it with the existing ones.

Imagine you ask an LLM to write an essay on a topic. There are two ways this can play out.

The non-agentic way is the direct one. You prompt the model:

Please type out an essay on topic X.

This would mean the LLM gives you the essay from start to finish in one go, without using backspace. The model produces the whole essay in a single shot. No outlining, no looking things up, no revising. Whatever comes out is what you get.

The agentic way looks more like how you would actually write an essay:

Write an outline on topic X. Do you need to do web research? Write a first draft. Consider what parts need revision or more research. Revise your draft.

Same end goal, but the path is broken up. There is a chain of thought like humans actually do, breaking down tasks. The model plans, possibly uses tools, reflects on what it produced, iterates and then gives you the final output.

That is the whole idea in one sentence:

An agentic AI workflow is a process where an LLM-based application executes multiple steps to complete a task.

How autonomous is an Agentic AI?

Once you decide your workflow is going to have multiple steps, the next question is who decides what those steps are. Is it going to be you (the developer), the user or the agent itself, or a mix of every stakeholder???

This is the degrees of autonomy spectrum, and it is more useful to think about than a binary “agentic / not agentic” label.

Degrees of Autonomy in Agentic AI

On one end, you have less autonomous systems. Every step is predetermined. The tool calls are hardcoded. The only thing the LLM is really doing is generating text at fixed points in the pipeline based on the input it gets. For example, you write the essay flow yourself: prompt → LLM outline → hardcoded web search tool→ hardcoded web fetch tool → LLM essay. The LLM never decides what comes next.

In the middle sit semi-autonomous systems. The agent can make some decisions and choose between tools, but the toolset itself is predefined. Think of an LLM that gets to pick whether to call web search, arXiv search, or a Wikipedia lookup, but cannot invent a new tool.

On the other end, highly autonomous systems make many decisions on their own and can even create new tools on the fly, generating code that becomes part of their toolkit.

Where on this spectrum should you build? It depends almost entirely on the task.

What kind of problem are you solving?

Some tasks are easier to make agentic than others. The dividing line is roughly this:

Task Complexity

An invoice processing workflow is on the easier end. The steps are obvious: extract the biller, the biller’s address, the amount due, the due date, and record it in a database. The agentic version replaces the manual extraction with a PDF-to-text tool feeding an LLM that then calls a database update tool. Same recipe, just automated.

A customer service agent is harder. The user might write “Do you have any black jeans or blue jeans?” one moment and “I’d like to return the beach towel I bought” the next. Different intents, different tools, different sub-flows. The agent has to decide on the fly.

A travel agent that browses the web and books tickets is harder still, because the steps are genuinely unknown ahead of time and depend on what the web shows the agent in the moment.

The harder the task, the more autonomy you need. And the more autonomy you need, the more thinking goes into the next piece, which is decomposition.

The actual work of building an agent

Most of building an agentic system is not picking the model. It is figuring out what the sub-tasks are and how to sequence them.

Let me show this with the essay example again, because the progression is striking.

Zero-shot direct generation. Prompt the LLM with “Write an essay on topic X.” Get back something surface-level that covers only the obvious facts.

Three-step generation. Reflect on how you would write an essay, and decompose it: outline, search the web, write the essay using the research. Each step can be done by an LLM, a tool, or a function call. The result is noticeably better, but still not great.

Five-step generation. Refine your reflection a little further: write an outline, search the web, produce a rough draft, check for revision points, revise. Now the result is getting close to something useful.

The model did not get smarter. But the tasks in the pipeline did.

This is exactly the observation that makes agentic workflows so interesting: an agentic approach with GPT-3.5 can match a direct prompt to GPT-4o on many tasks. You get much better performance even with lower-end models.

Building blocks of an Agentic workflow

When you start assembling one of these, the pieces fall into two buckets:

Models are what produce text or interpret modalities. LLMs handle text generation, tool selection, and information extraction. Other AI models handle PDF-to-text, text-to-speech, image analysis, and so on.

Tools are everything the model can call out to. APIs for web search, real-time data, sending email, checking a calendar. Information retrieval through databases or RAG. Code execution for calculators, data analysis, or anything where running a few lines of Python is reliable.

Evaluating an Agentic AI workflow

This is where most agentic projects quietly fail. You can ship a workflow that demos beautifully and falls apart in production, and the only thing that catches it is a disciplined evaluation process.

The starting point is unglamorous: look at low-quality outputs manually. See if the output meets all your requirements, what should and shouldn’t be included. If certain failures keep showing up, pinpoint them and write an eval that just measures how often that specific failure repeats.

From there, evals split into two flavours:

• Objective evals are code-based. You write a script that programmatically checks whether the output is desired or not, and how often the failure appears.
• Subjective evals are LLM-as-judge. When something is too nuanced for code to check (tone, helpfulness, faithfulness to a brief), you ask another LLM to judge. It is not perfect, but it is often the only practical option.

Evals also split by scope:

• End-to-end evals check whether the final output is good.
• Component-level evals check whether each step (each tool call, each intermediate LLM output) did its job.

The component-level traces are where error analysis actually happens. If the final essay is bad, you trace back through every intermediate output to find where the workflow broke. Evals and error analysis are what separate a good agentic system from an average one.

Design Patterns of Agentic AI

Design Patterns of Agentic AI

Reflection. The LLM looks at its own output and critiques it. Then it uses the critique to produce a better version. This can run for multiple iterations, and the critic does not even have to be the same model, you can use a different LLM or a different agent as the reviewer.

Tool use. The LLM is given a set of tools and decides which one to call to make progress. Tools fall into categories like analysis, information gathering, productivity, and image processing. The LLM is essentially being used as a router that picks the right tool for the moment.

Planning. Instead of a developer hardcoding the sequence of steps, the LLM decides the sequence itself. This is the move from “I wrote the recipe” to “the agent writes its own recipe.”

Multi-agent collaboration. Multiple specialised agents take on different sub-tasks and work together to accomplish a larger one. One agent does research, another writes, a third reviews. Each can be tuned for its role.

Takeaway

The shift from non-agentic to agentic AI is not a shift in models. It is a shift in how we use them.

In a non-agentic workflow, the model does the task. In an agentic workflow, the model takes additional responsibility of even orchestrating the task.