The Hidden Danger in Spec-Driven Development: When “Human in the Loop” Is Just a Phrase

Why building AI-generated code without decision provenance is a liability waiting to happen

Accident requiring human assistance AI generated image

Spec-Driven Development (SDD) is gaining momentum as a disciplined approach to AI-assisted coding. The workflow is elegant in its architecture: a Constitution establishes non-negotiable principles, a Specification defines the what and why, a Plan bridges requirements to implementation, tasks are broken into actionable markdown checklists, and an AI agent executes. Developers point to real advantages — consistency, portability, and a “human-in-the-loop” posture that appears to address the wildest fears about autonomous AI coding. The markdown files are editable. The spec is readable. The human is present.

What could go wrong?

Quite a lot, as it turns out — and the failure mode is not where most practitioners are looking.

The core danger in SDD as currently practiced is the conflation of human presence with human accountability.

Editing a markdown file is not the same as auditing a decision.

Reviewing a generated plan against a constitution is not the same as certifying that the reasoning behind each technical choice is traceable, verifiable, and reproducible.

Simply editing a markdown file or examining a generated goal against a constitution isn’t ‘human in the loop’ and does not audit or certify HAG-driven decision provenance. This includes answering questions about security, transparence and accountability for every piece of code written.

Standardizing Drift

When the Specification detaches requirements from the tech stack — one of SDD’s celebrated features — it also detaches requirements from the decision trail that would explain why one technical choice was made over another. The plan stage introduces technical decisions like “use Next.js” or “avoid databases,” but nothing in the standard SDD workflow captures why those decisions were made, what alternatives were considered, and who is accountable for that reasoning.

The tasks.md file is a checklist, not a decision record. When something breaks in production six months later, the checklist cannot answer the question that matters: how do you know?

And who keeps track of all the changes, upgrades, code refactoring, design changes, library updates and dependencies? If developer Igor goes on vacation or gets fired, does developer Raj’s decisions get a new review?

This gap becomes structurally dangerous when AI-generated code is refactored — which, according to a McKinsey report is the developer task most accelerated by AI and most starved for time in legacy enterprises. In refactored code, the delta between the original and the new version is the decision record. SDD produces that delta but does not interrogate it.

A Constitution for All?

The Constitution ensures stylistic consistency — always use Tailwind, always include tests — but it says nothing about the reasoning embedded in the change itself. Decision provenance asks not “what does this do” but “what alternatives would produce the same behavior, and why might this specific choice have been made instead.” A constitution cannot answer that question. A spec cannot answer it. Only a systematic, HAG-anchored quality management system that captures reasoning at each decision point — not just the output of each task — can begin to answer it. Without that, the SDD practitioner and the persons who put trust in any AI sysytem is, to put it bluntly, flying blind.

YOLO, right?

Confidence without comprehension is not intelligence. It is liability.

Liability

The liability dimension is no longer theoretical. The EU AI Act, which begins applying to high-risk AI systems in 2026, requires conformity assessment — an audit-like process — before any AI system is placed on the market or in service. Article 17 of the Act mandates a quality management system that includes record-keeping designed to automatically record events relevant to decision trails throughout a system’s lifecycle. An SDD workflow that produces a tasks.md file and a generated codebase satisfies none of this. The spec is not a QMS. The constitution is not a conformity record. The plan, however carefully written, is not an audit trail. And the human who edited the markdown files is not, under the Act, a “responsible person” who has performed a conformity assessment.

Ghost of Jer Crane

The Railway and Cursor events, the OpenClaw email deletion — illustrate precisely the accountability gap that the Act is designed to close. In each case, a human was nominally present. In each case, the decision trail was absent. The blame, as one observer noted, sits with the developer who delegated decision-making to the AI agent and did not review or possibly understand the actions taken.

SDD, without decision provenance, institutionalizes exactly that delegation at scale insinuating quality control where none in fact exists.

Society deserves better.

YOLO: Is There Better Solution?

The solution is not to abandon SDD. But the current SDD approach isn’t anywhere near ready for prime time, and may actually just be moving the too small bedsheet. Its discipline around specification and planning is genuinely valuable — the Constitution and Spec stages map reasonably well to the Situation and Task stages of a new methodology.

The serious implementation gap exists, where SDD currently produces only code and a completed checklist rather than a governed, legally binding human-verified decision record. What is needed is a HAG layer embedded at the end of each task execution — not a post-hoc audit, but a real-time capture of the reasoning behind each technical choice, stored in a structured format that supports human review, cost tracking, token logging, and iterative learning across the lifetime of the codebase.

The Human in the Loop Fantasy

In SDD systems as currently implemented, the human is not in the loop. The human must be the loop — at the beginning, in the middle, and at the end of every AI operation that produces code with real-world consequences. SDD gives developers a structured way to talk to AI agents. Decision provenance gives them a structured way to be accountable for what those agents build.

In a world where two-thirds of new code is already AI-generated and ninety percent is on the horizon, the question “how do you know?” is not a philosophical one. It is a legal, operational, and ethical requirement — and a markdown checklist embedded in a “constitution” is simply not the answer we need.

© 2026 David Grunwald. All rights reserved.

David Grunwald is a Founder of ASTR AI, a decision provenance company. Before that he worked for 32 years at Oracle and taught at De Anza College.