The Messy Middle: Building a Cloud Automation System as a Side Project

There’s a version of this story that sounds a lot cleaner than what actually happened. In that version, I sketched an architecture on a whiteboard, broke it into pieces, and built each piece in order. Every test passed. Every service integrated on the first try. I hit my milestone by Sunday.

That’s not this story.

This week I was deep in what I’ve started calling the “messy middle” — that phase of building where you’ve committed to something, the early excitement has worn off, and every progress update sounds like a list of things that didn’t work yet. It’s not glamorous. But it’s where most of the actual learning happens.

What I Did

The project I’m building is a cloud automation system — a set of services that handle recurring tasks without me having to manually trigger them. Think scheduling, monitoring, retry logic, alerting. The kind of infrastructure that you don’t appreciate until it breaks.

This week I was focused on one specific piece: retry logic for downstream service calls. When one of my automation tasks calls an external endpoint and it’s slow or temporarily unavailable, what should the system do? Fail immediately? Wait and retry? How many times? With what delay?

These questions sound simple until you actually implement them. My first version was too aggressive — I had the system retry every 2 seconds, which under load basically became a small DDoS against my own downstream service until the rate limiter kicked in. Not ideal.

So I slowed it down. Exponential backoff, standard stuff. But then I had a different problem: tasks were taking so long to process that a queue was building up faster than I could drain it. The retry logic was correct in isolation, but it wasn’t accounting for the full system load.

By end of week, I had something that works under the conditions I’ve tested. I wouldn’t call it production-ready. But it’s good enough to move forward.

What I Learned

The biggest lesson from this week wasn’t technical. It was about how I think about “done.”

I’ve been building this as a side project in 90-minute focus windows — usually after my day work, sometimes early morning. Time is the real constraint. And I noticed I was spending a lot of mental energy on this question: is this good enough to move on?

What actually helped was shifting the question. Instead of asking “is this done?”, I started asking: “what’s the simplest thing that would break this in production?” If I can answer that and I’m okay with it, I can move forward. If not, that’s the thing to fix next.

That reframe saved me from two different failure modes: over-engineering something that didn’t need it yet, and shipping something I knew was fragile.

The other thing I re-learned is that documenting where you stopped matters more than documenting what you built. When I come back to a session after 24 hours, I need to know my mental state at exit, not just the last file I edited. I’ve started ending every focus session with a 2-sentence note: what I just finished, and what the first decision I need to make is next time. That alone cuts my re-orientation time roughly in half.

Why This Matters

If you’re an Indonesian developer thinking about building something as a side project — an automation tool, a small SaaS, an internal system — I want to be honest about what the middle of the process looks like.

It’s not a series of breakthroughs. It’s a lot of debugging things you thought you understood. It’s making something work at 11pm and then reading your own code the next day and wondering who wrote that. It’s being proud of fixing a bug that you introduced while fixing a different bug.

This matters because the gap between “I want to build something” and “I’m actually building it” is usually not technical. It’s the expectation that building should feel like progress every day. It often doesn’t. Progress in a side project is less like a smooth gradient and more like a staircase — flat for a while, then a sudden step up when enough small pieces connect.

The cloud automation I’m building isn’t going to change my life overnight. What it’s doing is training a very specific capability: the ability to take something from idea to running system, in real infrastructure, with real constraints. That skill compounds. Every system you finish makes the next one faster.

Next Steps

Next week I’m building the monitoring layer. I need something lightweight that tells me when a task has failed, how many times it’s retried, and whether it’s in a stuck state. I’ve looked at a couple of options and I’m leaning toward building something minimal from scratch rather than integrating a full observability platform — partly because of cost, partly because I want to understand the internals.

If I can get alerting working by the end of the week, even just a basic notification when something breaks, I’ll count it as a win. Small, concrete, shippable.

That’s the system I’m building. Not just the automation — the habit of finishing things.