Designing an Autonomous Error-Resolution Pipeline

Photo by Growtika on Unsplash

Pointing a coding agent at your error tracker and letting it fix bugs sounds like free leverage. While you can hand an LLM a stack trace, let it edit main, and open a pull request, that approach quietly collapses two things you normally keep apart: the part of your workflow that is safe to automate and the part that needs judgment. One solution is to wrap the agent in a pipeline that isolates every change, verifies it before anyone sees it, and leaves a paper trail. This not only makes the agent's output reviewable like any other contributor's, it makes a bad fix cost you a closed PR instead of a production incident.

This article walks through the design of such a pipeline. The running example throughout is a backend service that has accumulated a backlog of production errors in Sentry — a TypeError from a null firstName in a name-formatting helper, a foreign-key violation on an orphaned booking row, a cron job that times out mid-query, a webhook that throws "order not found." The concepts apply to any error tracker and any language, but a single concrete backlog makes the trade-offs easier to see.

The numbers below are from a real implementation. In one pass the pipeline turned 6,217 events across 9 issues into 6 reviewed pull requests; a larger run resolved 55 issues across 11 merged PRs and cleared more than 60,000 events. None of that required a human until review time — which is exactly the property the design is built to earn.

The Six-Phase Pipeline

The agent is not a single prompt. It is a fixed sequence of phases, and each phase has one job:

fetch    → pull unresolved issues from the error tracker
triage   → dedup, group, and rank them deterministically
ticket   → file a tracking ticket per issue (idempotency anchor)
fix      → in an isolated worktree, one issue at a time
verify   → reproduce, fix, re-run tests in that worktree
report   → open a PR + post a verification summary, notify Slack

The phases are deliberately boring. The LLM only gets real latitude inside fix; everything around it is plumbing you can reason about without thinking about probabilities. That boundary — a small creative core wrapped in deterministic phases — is the whole design.

Note: Resist the urge to let one mega-prompt do “fetch, decide, and fix” in a single call. The moment the model is choosing which bugs to work on in the same breath as fixing them, you lose the ability to inspect its decisions. Keep selection out of the model’s hands.

Isolate Every Fix in Its Own Git Worktree

The first structural decision is where the agent does its work. Editing your checkout in place means one bad fix corrupts the workspace for every other fix, and it makes parallelism impossible. Instead, give each issue its own git worktree branched off the latest main:

# One worktree per issue, branched off origin/main
RUN_ID=$(date +%Y%m%d-%H%M%S)
git worktree add "runs/$RUN_ID/$ISSUE_ID" -b "fix/$ISSUE_ID" origin/main

A worktree is a second working directory backed by the same .git object store. Each one has its own branch, its own index, and its own files, so an agent thrashing in fix/null-firstname cannot touch fix/orphaned-booking. That isolation is what lets you run them at once — a single pass might launch a dozen worktrees in parallel, each chewing on a different category: UUID validation, rate limiting, webhook handling, a missing table, an N+1 query.

Note: Worktrees share the object database. That is the point — branching is cheap and you are not re-cloning the repo a dozen times — but it also means a git gc or a force-push from one place can surprise another. Treat each worktree as disposable and never run repo-wide maintenance mid-run.

Tip: Name the branch and the directory after the issue’s stable identifier (fix/$ISSUE_ID), not after a description of the bug. The fix's understanding of the bug changes as it works; the identifier does not, and you will want it again in the ticket and report phases.

Worktrees also give you a natural unit of cleanup, which matters more than it looks — see the idempotency section below.

Make Triage Deterministic Before You Touch Code

Before any model runs, decide what to work on with plain code. Your error tracker already groups events by a fingerprint; lean on it. Pull the unresolved issues, drop anything you have already filed, and sort by something you can defend:

# Deterministic selection — no model involved
issues = tracker.fetch_unresolved(project, since=last_run)
issues = [i for i in issues if i.fingerprint not in already_filed]
issues.sort(key=lambda i: (i.level, i.event_count), reverse=True)
candidates = issues[:MAX_PER_RUN]

level puts fatals ahead of warnings; event_count puts the bug hitting 6,000 users ahead of the one that fired twice. MAX_PER_RUN is a throttle — you do not want a run that opens 80 PRs nobody will review.

Note: A single Sentry issue is not always a single bug, and two issues are sometimes the same bug seen from two endpoints. The fingerprint is a good default key, but expect to merge or split a few by hand the first few runs until you trust the grouping.

Tip: Persist the set of fingerprints you have already filed against and load it at the start of every run. That one file is what turns “a script you babysit” into “a job you can run on a schedule” — it is the difference between idempotent and annoying.

Doing selection deterministically means that when someone asks “why did the agent skip the auth crash?”, the answer is a sort key you can point at, not a model’s mood that afternoon.

File a Ticket Before You Open a PR

It is tempting to go straight from fix to pull request. Filing a tracking ticket first — before the code exists — buys you two things. It gives every fix a stable home that links the original error, the eventual PR, and any human follow-up; and the ticket’s existence becomes the idempotency check that stops a scheduled run from re-fixing what it fixed yesterday.

# The ticket is created from the issue, keyed by fingerprint
ticket = tracker_to_issue_tracker(
    summary=f"{issue.title}",
    body=f"Auto-triaged from error {issue.permalink}\nfingerprint: {issue.fingerprint}",
    labels=["auto-triage"],
)
already_filed.add(issue.fingerprint)   # persisted; see triage

Note: Put the error tracker’s event ID and the fingerprint in the ticket body. The event ID is what a human clicks to see the crash; the fingerprint is what your code matches on. You need both, and you need them written down somewhere outside the agent’s memory.

The ticket also gives reviewers context the diff alone cannot. A PR that says “fix null guard in generateName" is far easier to approve when it links a ticket that says "this fired 6,000 times last week on users with no surname."

Verify Before You Open the PR, Not After

This is the phase that earns the word “autonomous.” The agent does not get to open a pull request because it believes the fix is correct. It opens one only after reproducing the failure, applying the change, and watching the build and tests go green — all inside that issue’s worktree:

# Inside runs/$RUN_ID/$ISSUE_ID — the gate that decides PR or no PR
npm ci
npm run build || exit 1
npm test -- --runInBand || exit 1   # red here means NO pull request

If any step fails, the phase exits without a PR and the failure goes into the report instead. A worktree that cannot prove its fix produces a note for a human, not a green checkmark nobody asked for.

Note: “Reproduce, then fix, then re-run” is not optional. The agent should capture the failure before its change (a failing test, a logged stack trace) and the success after. Without the before-state you have no evidence the fix addressed the reported error rather than some adjacent symptom.

Tip: Have the verify phase write its before/after evidence straight into the PR description — the reproduced error, the passing test names, the count of tests run. Reviewers spend their attention on the diff instead of re-deriving what the agent already proved, and you get a verification trail for free.

The asymmetry here is the safety model in one line: a false “it’s broken” costs you a skipped issue you will catch next run; a false “it’s fixed” merged straight to main costs you an incident. The gate is tuned to fail toward the cheap mistake.

Make Runs Idempotent and Self-Cleaning

A pipeline you run once is a demo. A pipeline you run on a cron needs two properties: running it twice must not double its work, and it must not leave a growing pile of branches and directories behind. The persisted fingerprint set from triage handles the first. A dated run directory handles the second:

# Every run lives under a timestamped dir; prune anything older than a week
find runs/ -maxdepth 1 -type d -mtime +7 | while read -r dir; do
  git worktree remove --force "$dir"
done
git worktree prune

Note: git worktree remove is the right tool, not rm -rf. Deleting the directory by hand leaves the worktree registered in .git/worktrees, and the next worktree add with the same name fails with a confusing "already exists." Always remove through git, then prune.

Tip: Tag the auto-filed tickets and auto-opened PRs with a consistent label. When you want to know what the pipeline has actually been doing — or audit it after a bad week — that label is the only query you need.

Idempotency is what lets you stop treating the agent as an event and start treating it as infrastructure. The run that found nothing new should be a no-op that exits in seconds, not a re-run of last night’s work.

Know Where the Agent Must Stop

The pipeline opens pull requests. It does not merge them. That single boundary is what makes the whole thing safe to deploy, and it should be load-bearing in your design, not an afterthought. The agent’s authority ends at “here is a verified, reviewed-by-tests change with evidence attached.” A human still reads the diff, still owns the merge, still decides whether a fix that passes tests is actually the right fix.

Note: Some failures should never reach the fix phase at all. A crash rooted in a schema migration, a secret rotation, or anything that touches money deserves a ticket and a human — not an auto-generated patch. Encode that as a skip list in triage, not as a hope that the model will be cautious.

There is a related discipline worth stating plainly: when the agent investigates an issue and finds there is nothing to fix, that is a successful outcome, and it should say so on the ticket and move on. One of the most useful real runs concluded that a proposed migration was unnecessary because the columns already existed — and closed the work with that finding rather than inventing a change. An agent that always produces a diff is an agent that will eventually produce a wrong one.

Some of you will point out that you could skip the tickets, skip the worktrees, and just let the model push fixes to a branch and open PRs directly — and for a five-issue backlog on a weekend project, you would be right; the plumbing would cost more than it saves. The trade-off changes with scale and stakes. Once a single run is touching thousands of events across a dozen categories, the deterministic phases around the model are not bureaucracy — they are the only reason you can read the output, schedule the job, and trust that the worst case is a closed pull request rather than a 2am page. The agent is not replacing the engineer who fixes bugs; it is removing the toil of the ones that were never interesting, and handing back the rest with the evidence already attached.