From Prompt to Production: The Spec-Driven Workflow I Use With Claude Code

A practical system for turning complex, high-stakes system designs into reliable, production-grade architecture.

I gave Claude Code a task that sounds like the holy grail of modern software engineering:

Build a stateful AI agent that can query our PostgreSQL database, analyze the results, and send a summary email to the client.

The request looked clear. The application already had a database connection, an LLM client, and an email service. Claude just needed to wire them together into an autonomous loop.

A few minutes later, the terminal blinked back to life. It had created an agent loop, a few tool definitions, and a passing unit test.

It looked like a spectacular example of AI-assisted development.

Then I reviewed the implementation.

The agent had no memory; if the server restarted mid-execution, it lost its state and started over. It had no idempotency; if the LLM hallucinated and called the “send email” tool twice, the client got two emails. It had no budget limits; when it got stuck in a reasoning loop, it burned through $400 in API credits in twenty minutes. To top it off, it executed database queries without any row limits, nearly taking down our production replica.

Claude had not ignored my request. It had followed it to the letter.

The problem was that my prompt described the visible feature while leaving nearly every critical distributed-systems decision undefined. Building a toy agent is easy. Building a reliable, stateful, fault-tolerant agent orchestrator is one of the hardest problems in backend engineering today.

That was the moment I stopped asking Claude Code to move directly from prompt to implementation for complex systems. I introduced a workflow between the idea and the code.

This is the spec-driven workflow I now use to turn high-stakes architectural concepts into production-grade reality.

Part 1: Why the Original Prompt Failed

My original request was:

Build an AI agent that queries the DB, analyzes data, and emails the client.

It sounds perfectly reasonable. But to fulfill this prompt, Claude must silently infer:

• How to persist state between LLM calls (in-memory vs. database).
• How to handle idempotency for side-effect tools (emails, webhooks).
• What happens when the LLM hallucinates a tool name or parameters.
• How to enforce strict token, cost, and step-count budgets.
• How to implement “Human-in-the-Loop” (HITL) approvals for destructive actions.
• How to handle partial failures and exact retry semantics.

Different engineers or different runs of the same coding agent will make entirely different reasonable decisions.

The Golden Rule: Vague prompts do not remove engineering decisions. They silently delegate those decisions to the model. And in complex systems, silent delegation leads to silent failures.

Part 2: The Spec-Driven Workflow

To fix this, we need a pipeline. Here is the workflow. The most important rule? Claude Code does not modify production code until the specification and plan have been reviewed.

Let’s walk through each stage using a realistic, complex stack: Python (FastAPI), PostgreSQL, Redis (for state and idempotency), and the Anthropic SDK.

Stage 1: Explore the Existing Codebase

Before writing a single line of specification, we need to understand the terrain. We ask Claude to investigate the repository without touching anything.

→ Prompt:

Explore this repository without modifying any files.

Identify:
1. How background tasks and state persistence are currently handled.
2. How the database connection pool and query limits are configured.
3. How the LLM client handles streaming, timeouts, and retries.
4. How the email service handles delivery guarantees and deduplication.
5. Any existing idempotency patterns or distributed locking mechanisms.

Return:
- Relevant files and existing conventions
- Reusable components
- Missing capabilities for a stateful agent orchestrator
- Risks that an autonomous agent execution must consider

Do not propose an implementation yet.

→ Example Output:

# Repository findings
## State & Background Tasks
- Celery is configured, but state is only stored in task metadata.
- No long-running state machine framework (like Temporal) exists.

## Missing capabilities
- No idempotency key tracking for external API calls.
- No mechanism to pause execution for Human-in-the-Loop (HITL) approval.
- No global budget enforcement (token/step limits) for LLM loops.

This prevents Claude from immediately generating a naive while True: loop. It grounds the AI in the reality of your architecture.

Stage 2: Write the Feature Specification

Next, we create specs/001-agent-orchestrator/spec.md. This document is the single source of truth.

# Stateful Agent Orchestrator

## Objective
Build a fault-tolerant, stateful AI agent orchestrator capable of executing 
multi-step workflows with strict budget guardrails, idempotent side-effects, 
and Human-in-the-Loop (HITL) checkpoints.

## User scenarios
### Scenario 1: Successful Execution with HITL
Given an agent workflow requires sending an email
When the agent reaches the email tool
Then the execution state is persisted as "AWAITING_APPROVAL"
And a Slack notification is sent to the admin
And execution halts until the admin approves via webhook
And upon approval, the email is sent exactly once

### Scenario 2: Budget Exhaustion
Given an agent has a budget of 50,000 tokens and 10 steps
When the LLM call would exceed the remaining budget
Then the agent immediately halts execution
And the state is persisted as "BUDGET_EXHAUSTED"
And no further tools are executed

### Scenario 3: Idempotent Tool Retry
Given the "query_database" tool failed due to a transient network error
When the orchestrator retries the step
Then the tool is executed with the exact same idempotency key
And the database is not queried twice for the same logical operation

## Functional requirements
FR-001: The orchestrator must persist state to PostgreSQL after every LLM and tool call.
FR-002: All tools with side-effects must require an idempotency key.
FR-003: The system must enforce hard limits on max_steps, max_tokens, and max_wall_time.
FR-004: Destructive tools must support a "requires_approval" flag for HITL.
FR-005: The LLM must be constrained to a strict JSON schema for tool calling.
FR-006: If the server crashes, the agent must resume exactly from the last persisted state.

## Non-goals
The feature will not:
- Implement multi-agent collaboration or swarm logic.
- Support streaming responses to the end-user (this is a background worker).
- Re-architect the existing Celery worker pool.

To truly understand the complexity of what we are asking Claude to build, we must visualize the state machine. This is the exact mental model we need Claude to have before writing a single line of code:

Insight: Requirements explain what Claude should build. Non-goals explain what Claude must leave alone. The state diagram ensures Claude understands the exact lifecycle transitions.

Stage 3: Ask Claude to Find Ambiguity

Before planning, we ask Claude to question the specification and find possible problems.

→ Prompt:

Review `specs/001-agent-orchestrator/spec.md`. Do not implement anything.

Identify:
1. Ambiguous requirements
2. Edge cases in state transitions (e.g., what if HITL approval times out?)
3. Concurrency issues (e.g., two workers picking up the same HITL task)
4. Security decisions (e.g., LLM prompt injection via database query results)

For every issue, propose one concrete clarification. 
Do not make the decision silently. List the decisions I need to approve.

Claude might detect that we haven’t defined what happens if an HITL approval takes 30 days (do the temporary credentials expire?), or how to handle an LLM returning malformed JSON. We then update the spec with explicit answers.

Stage 4: Generate the Implementation Plan

Now we create specs/001-agent-orchestrator/plan.md.

→ Prompt:

Using the approved specification, produce an implementation plan.

Constraints:
- Use PostgreSQL for state persistence (no external workflow engines).
- Use Redis for distributed locking to prevent duplicate execution.
- Map every implementation step to one or more requirement IDs.

The plan breaks the feature down into a State Machine engine, an Idempotency registry, a Tool execution wrapper, a Budget enforcer, and the main orchestration loop.

Stage 5: Break the Plan into Atomic Tasks

We create specs/001-agent-orchestrator/tasks.md.

# Tasks
- [ ] T001 Define Pydantic models for AgentState and ToolResult. (FR-001)
- [ ] T002 Implement PostgreSQL repository for state persistence. (FR-001, FR-006)
- [ ] T003 Implement Redis-based distributed locking for execution. (FR-006)
- [ ] T004 Create the Budget Enforcer (tokens, steps, wall-clock). (FR-003)
- [ ] T005 Implement the Idempotent Tool Wrapper. (FR-002)
- [ ] T006 Implement the HITL pause/resume mechanism. (FR-004)
- [ ] T007 Build the main orchestration loop with JSON schema enforcement. (FR-005)

Insight: A plan describes the destination. Atomic tasks give Claude checkpoints at which its work can be reviewed and corrected.

Stage 6: Implement One Vertical Slice at a Time

Never ask Claude to “Implement everything in tasks.md.” Instead, we slice the work.

→Prompt:

Implement tasks T001 through T004 from `tasks.md`.

Before editing:
1. Read the approved spec and plan.
2. Confirm the existing SQLAlchemy async session patterns.

During implementation:
- Do not work on the LLM orchestration loop yet.
- Run only the relevant unit tests for state and budget logic.

After implementation:
- Mark completed tasks in tasks.md.
- Stop before T005.

By forcing the AI to stop and report back, we maintain human oversight at every critical junction.

Stage 7: Verify Requirements, Not Only Tests

A green test suite is necessary, but it does not prove that every complex requirement was implemented. We ask Claude to create a traceability report.

→ Prompt:

Audit the completed implementation against `spec.md`. 
Do not modify code during the first pass.

For every requirement from FR-001 through FR-006, provide:
- Status: satisfied, partially satisfied or missing
- Implementation file and symbol
- Test file and test name

Without this step, Claude might confidently tell you, “All tests pass. The feature is complete.” The traceability report reveals that the wall-clock timeout was never actually implemented.

Stage 8: Use an Independent Review Context

Claude Code supports specialized subagents with separate context windows. This is perfect for an independent review after the main implementation context has become biased toward its own solution.

We create .claude/agents/distributed-systems-reviewer.md:

---
name: distributed-systems-reviewer
description: Reviews stateful systems for concurrency and failure edge cases
tools: Read, Grep, Glob, Bash
---
You are a distributed systems expert. You may inspect code and run read-only 
validation commands. You must not edit files.

Focus on: 
- Race conditions in state transitions
- Deadlocks in distributed locking
- State corruption during partial failures
- Idempotency key collisions
- Memory leaks in long-running loops

Reference exact files and symbols.

Then we prompt the main agent:

Use the distributed-systems-reviewer subagent to audit the agent orchestrator.
Do not fix the findings yet.

This is infinitely stronger than asking the same implementation conversation to “Review your own code.”

Add a Quality Gate

For complex systems, standard unit tests aren’t enough. We need a chaos and integration gate. We save this as scripts/verify-orchestrator.sh:

#!/usr/bin/env bash
set -euo pipefail

ruff check app tests
mypy app
# Run standard unit tests
pytest tests/orchestrator -q
# Run chaos tests (simulate DB drops, LLM timeouts mid-execution)
pytest tests/chaos/test_crash_recovery.py -q
pytest tests/chaos/test_idempotency_under_load.py -q

Then we instruct Claude:

Run `scripts/verify-orchestrator.sh`. 
Do not bypass, remove or weaken any failing check. 
If chaos tests fail, identify the unhandled edge case and fix the state machine.

The Before-and-After Comparison

The Repository After Implementation

.claude/
└── agents/
    └── distributed-systems-reviewer.md
specs/
└── 001-agent-orchestrator/
    ├── spec.md
    ├── plan.md
    ├── tasks.md
    └── verification.md
scripts/
└── verify-orchestrator.sh
app/
├── orchestrator/
│   ├── state_machine.py
│   ├── budget_enforcer.py
│   ├── idempotent_tool.py
│   └── loop.py
├── tools/
│   ├── registry.py
│   ├── query_db.py
│   └── send_email.py
└── persistence/
    ├── models.py
    └── repositories.py
tests/
├── orchestrator/
│   ├── test_budget_enforcer.py
│   └── test_hitl_flow.py
└── chaos/
    ├── test_crash_recovery.py
    └── test_idempotency_under_load.py

A Reusable Minimal Workflow

If the full pipeline feels too heavy, here is the condensed version you can adopt today for complex backend or AI systems:

1. Explore: “Study the repository’s existing state, concurrency, and failure patterns. Do not modify files.”
2. Specify: “Create a feature specification containing state transitions, idempotency requirements, budget guardrails, and non-goals. Do not implement it.”
3. Clarify: “Find every edge case in failure scenarios, race conditions, or partial executions. List decisions requiring human approval.”
4. Plan: “Map the approved requirements to state models, execution loops, locking mechanisms, and chaos tests.”
5. Implement: “Implement only the selected tasks. Run relevant checks and stop for review.”
6. Verify: “Map every requirement to implementation and chaos-test evidence. Report all missing requirements.”

The Central Framework

Whenever you sit down to build complex systems with Claude Code, remember this formula:

Prompt = request

Spec = contract

Plan = design

Tasks = control

Tests = evidence

A prompt tells Claude what you want next. A specification tells the project what must remain true under failure, load, and edge cases.

Conclusion

Spec-driven development does not make Claude Code slower. It moves the expensive mistakes to a stage where they are still cheap to correct.

Changing a sentence in spec.md takes seconds. Discovering that your agent is sending duplicate emails to clients, or that it lost its state and restarted a 4-hour job after a server reboot, can cost you your reputation and your company's money.

The goal is not to create a massive document before writing every line of code. The goal is to remove dangerous ambiguity before an autonomous coding agent turns that ambiguity into architecture.

Continue the Journey

This post is part of a deeper series on engineering with Claude Code. If you want to go further:

[embed]I Wasted 6 Months Using Claude Code Wrong. Here Are the 14 Commands That Changed Everything. From frustrated beginner to power user: The hidden command ecosystem nobody talks about.pub.towardsai.net

[embed]12 Hidden Claude Code Features That Make It Feel Like an Engineering Operating System. Not just a coding assistant, a workflow engine your team can actually rely on.pub.towardsai.net

[embed]Claude Code Is a Mess, Until You Install This Official Plugin How One Plugin Helps Claude Understand Your Project, Recommend Tailored Automations, and Build Real Systems.Starting…pub.towardsai.net

[embed]Stop Prompting. Start Commanding: The Top 10 Claude Plugins That Actually Ship Work Why the best developers in 2026 don’t type conversations, they delegate tasks.pub.towardsai.net

[embed]How to Master Claude: 8 Simple Habits That Separate Power Users From Everyone Else. Most people use Claude like a basic chatbot. The best users turn it into a personal AI assistant that matches their…pub.towardsai.net

[embed]Skills Are the Missing Primitive in Your Claude Agent Stack You’ve built the pipeline and wired the tools but your agents are still brittle, inconsistent, and hard to debug. The…pub.towardsai.net