I Built an Agentic App Without Writing Code. Here’s What I Learnt

Ever since the whole AI wave kicked off, one thing has been on my mind — experimenting with an agentic flow. I wanted to start with something small enough not to complicate, but real enough to learn from the same.

I landed on the idea to build a small conversational weather app. Ask it about weather in any location, it figures out what you mean and gets you an answer. Simple premise. Turns out, it’s not that simple (to avoid over confidence) to build well.

The app works on the basis of the location a user provides . However, guardrails and validations around the location had to be built. The app also relies on two open-source APIs from Open-Meteo to fetch actual weather data, so handling API failures and model processing issues gracefully was part of the considerations too. The diagram below shows how it all fits together.

Weather App Flow (Created via Napkin AI)

For transparency — I did not write a single line of code. I used Replit throughout.

Here is what the experiment actually taught me.

1. Clear requirements matter just as much when you’re using AI coding tools.

In standard software development, incomplete requirements lead to faulty systems. Same thing happens here — except the feedback loop is faster and the failures may look different.

When I hadn’t defined what would happen if a user provided an ambiguous location — say, “East London” — the system didn’t handle it gracefully. It needed a clear pathway: ask the user to clarify, don’t just guess or fail silently. None of that gets figured out on its own. It has to be specified.

That’s where the PM role stays critical even in an AI-assisted build. The tools can generate the code. They can’t define the experience.

2. In LLM-based systems, you have to decide what the LLM is allowed to figure out on its own — and what it isn’t.

As stated earlier, the app uses two MCP tool calls — get_coordinates for geocoding and get_weather for the forecast. A guardrail was enforced: the LLM must use the API output for further processing and cannot deduce its own interpretation of what the weather might be.

That’s not just a technical constraint. It’s a product decision — about where the LLM’s judgment ends and the system’s facts begin. If that line isn’t drawn explicitly, the model will fill the gap on its own. Sometimes correctly. Often not.

As a PM, I’d frame it this way: decide early what your LLM is responsible for deciding, and what it should just execute. Leaving it ambiguous doesn’t give you flexibility — it gives you unpredictability.

3. Your requirements will have gaps. Testing is how you find them.

I had defined a constraint: the weather app should show a 7-day forecast, including the current day. Seems clear enough.

When I tested it for a 30-day range, it acknowledged it could only fetch 7 days — but then went ahead and returned 30 days of data anyway. Technically responsive. Not what I intended.

The requirement was incomplete. I had defined the output but not the boundary condition — what should happen when a request falls outside scope. Should it cap at 7 days? Return an error? The requirement did not explicitly state what to do, so the system made it on its own.

This is the spec-build-test-discover loop that every PM knows very well. It doesn’t go away with AI-assisted development. If anything, it moves faster — which means the gaps surface sooner, but only if you’re actually testing with intent.

What’s next

This was a single-agent, single-task setup. What I want to understand next is what happens when there are multiple agents, longer workflows, and integrations with tools like Jira or Confluence — where the boundary decisions get harder and the gaps in requirements get more expensive.

More on that as I build it.

Originally published at https://mohitgarg3189.substack.com.