Your Automation Works. Until It Restarts. Here’s Why That’s a Design Problem.

Most bots are built for the happy path. Production doesn’t care about your happy path.

I used to think a well-tested process was a reliable process.

Then I maintained one long enough to watch it survive a restart.

It didn’t.

Not because the logic was wrong. Not because the code had bugs. Because the process had never been designed for the one thing that production guarantees will happen eventually: interruption.

The Process That Forgot What It Did

Picture any sequential automation. It works through a list of items, one by one. Reads. Processes. Moves on. Clean, linear, predictable.

Now picture it stopping halfway through.

Maybe the application it was talking to stopped responding. Maybe a timeout hit. Maybe something upstream failed and the process crashed.

When it restarts — and it will restart — it has no memory of what it already finished. It goes back to the beginning of the list. Items that were fully processed look identical to items that were never touched. The process tries them again. Some fail again in the same way. The ones that were stuck before are still stuck. And the cycle repeats.

Nobody raises an alarm. There is no error. The process is running. The logs look fine.

But the throughput has collapsed. Work that should take an hour takes four. And somewhere, someone is wondering why things are so slow today.

This is not a bug. It is a design gap. And it is one of the most common gaps in production automation.

The Assumption That Breaks Everything

The gap has a name: stateless design.

The process is treating its environment — the list of items, the folder structure, whatever it reads at startup — as its only source of truth. If an item is still in the source location, it must not have been processed yet. That is the assumption.

But that assumption only holds on the happy path. On the interrupted path, an item can be fully processed and still sitting in the source location — because the final step, the one that moves it out, is what failed. The process cannot tell the difference. So it tries again.

This is a well-known problem in distributed systems. It is called the at-least-once delivery problem. You guarantee that work gets done at least once — but without idempotency checks, you cannot prevent it from being done multiple times.

Script-based automation hits this problem constantly. And because the failure mode is “running slowly” rather than “crashing loudly,” it often goes undiagnosed for a long time.

What the Fix Actually Looks Like

The fix is not clever. It is boring. That is why it works.

Give the process external memory.

Maintain a record — separate from the environment the process operates in — that tracks the status of every item. Not “is this item still in the source folder?” but “has this item been processed, and what was the outcome?”

When the process starts, it reads that record first. If an item is already marked complete, it skips it. If an item has been in an intermediate state for longer than a reasonable timeout, it marks it as failed and requeues it. If an item has no record at all, it processes it fresh.

Now a restart is a non-event. The process picks up exactly where it left off. No repeated work. No compounding failures. No throughput collapse.

The second part of the fix is a cleanup routine. Before the main process begins each run, a lightweight check sweeps for items that got stranded in intermediate states — started but never finished, probably because the last run was interrupted. It resolves their status before new work begins.

Together these two things — external state and a cleanup pass — turn a fragile sequential process into one that can survive interruption without losing its place.

The Deeper Lesson

Here is the shift that happens when you maintain production systems long enough:

You stop designing for what should happen. You start designing for what will happen when it doesn’t.

A process that works is a starting point. A process that recovers is a production system.

The question to ask before anything goes live is not “does this work?” It is: “what does this do when it stops halfway through?”

If the answer is “it starts over,” that is the next thing to build.

Before You Ship Anything Next Time

Run through this checklist:

• If this process crashes mid-run and restarts, does it know what it already did?
• Is the status of each work item tracked somewhere outside the process itself?
• Is there a cleanup step that resolves stuck items before new work begins?
• Does a restart produce the same outcome as an uninterrupted run?

If any of these is a no, the process is not production-ready. It is testing-ready. There is a difference.

The happy path is easy to build. Every engineer gets it right eventually. What separates production engineers from the rest is how they design for the path that breaks.