I had Claude rebuild what Claude built — from documentation it also wrote

Previously I used Claude to build a multi agents AI investigator. The overall experience is great. At the end I have a functional product, documentation, website and Youtube promotion videos.

It took me probably a couple weeks going from brain-storming with Claude, planning, designing, implementing, testing, starting over many times, more planning and repeat this loop until we (Claude and I) nailed down every detail I care to flesh out.

The follow up questions become clear —

• Is it a repeatable process? — can I or someone else rebuild what I built?
• If it is not a repeatable process, what does it take to make it so?
• Which parts are not repeatable?
• Was it worth it?

Photo by 張 峻嘉 on Unsplash

Is It a Repeatable Process?

Can I just ask Claude to create the same solution again any time from scratch?

I “vibe-coded” my way creating a non-trivial solution. The original docs grew alongside the code. Each time we wrapped a phase I asked Claude to write down what we just did and why. This includes: research, planning, multi phases implementation, use cases and operation documents. All these documents serve the purpose of providing context to both human as well as AI readers in the future. If I clear the current session, I can simply ask Claude to “catch up” by reading documents and memory file.

However, none of these documents provide sufficient context for Claude to rebuild it again from scratch.

So, the answer to the question is “No”, it is not a repeatable process.

What Does It Take To Make It Repeatable?

The new problem I am solving becomes figuring out a path forward and a set of specifications for AI (Claude in my case) to follow and expect the result implementation is also a fully functional solution.

Anything that can be written down as contract: API interface, database table schema, external API calls, etc. Claude does a pretty good job following the specification and implements it.

This is especially true if the instructions also include use cases to verify. While unit tests are important, it is the end-to-end functional tests that can keep Claude honest about its implementation. The feedback loop provided by test results is crucial to the AI and offer confidence to human.

The Solution

Specification is the solution I came up with.

Turning documentation into a specification went roughly like this:

1. Have clear understanding of what I had by purpose. Some documents are story (vision, use case, runbook). Some are contract (API, database table schemas, data source). Each serves different purposes.
2. Picked an ontology. This was the surprise turn — see The borrowed idea below.
3. Pinned every contract on its own line. Claude built a single checklist where each binding rule — every column name, every wire shape, every external API call — gets one row with a citation back to the spec doc that owns it. The checklist became one-page view of what is locked down versus what is still prose.
4. Wrote the use cases as scenarios we could verify. End-to-end queries the system must answer. The implementer doesn’t get to decide whether a feature is “done”; the use cases do.
5. Wrote operator prompts. Staged prompts that walk the implementer through the build in order, so it isn’t reading sixteen documents cold.
6. Tested the spec by handing it to a fresh Claude. Different working folder, no source code, only the spec. Whatever it couldn’t reproduce was a gap in the spec — not a failure of the AI.
7. The Surprise Helper. Asking AI itself to evaluate the work it did often results in disappointing answer. We all have heard AI can be too overly confident about itself and gave answers without either facts or self-reflection. I found out that having one Claude session criticize the work other Claudes produce improves the quality of the work significantly.

The borrowed idea

After ColdTrace was already built, I came across how Palantir frames its data — as a typed model of the world: entities with stable identities, attributes, and named relations between them. Aircraft. Operators. Persons. Sanctioned entities. Relations like operated by, flagged by, matches sanctioned. Claude rewrote the ontology doc around this idea.

Suddenly every loose noun in the prior docs had to become a named entity with rules. A lot of ambiguity disappeared in that one pass.

Which Parts Are Not Repeatable?

Two discoveries I made along the way.

1. The devil is in the details

The specification only says what to build and what to verify but it does not say how do you go about doing it. AI, like human, has its way of interpreting the specification and go about implementing it anyway it sees fit. On a couple of occasions AI even found loopholes in the specification and circumvent the use case verification. For example, AI in few cases decided that it is not going to run the end-to-end testing despite the criteria to close the task demands it.

Just like human beings, AI work needs verification.

2. The weak spot you have

To my surprise, the UI/UX part can not simply be described in the specification. If we just let AI decide on the UI, it will most likely come up with something that looks completely different from what the UI looks like in existing ColdTrace. This should not be a surprise but I was not thinking clearly in the beginning. If the requirement is to produce exactly the same look and feel of existing UI, it becomes necessary to provide UX design to AI.

I am very weak in UI/UX design and didn’t take time asking Claude to help me figure this part out. The result is that I had to leave the Frontend part out of the specification and keep using the same UI Claude originally built.

Was It Worth It?

The effort of trying it, absolutely, 100% fun.

Is it practical to create the specification first and expect Claude to implement it to a T without human supervision? I am only 95% there.

It is not the AI hallucination that I ran into which prevent me from closing the last 5% gap but it is the flexibility I intentionally provide (only “what” and “why” but not “how”) in the specification. AI interpreted it differently every time I started from scratch. Not enough instruction and guardrail I put in would definitively prevent this from happening. What I got out of it isn’t a second copy of ColdTrace — it’s a clearer view of the first one. Writing a spec to rebuild the system forced me to see the parts of it I had never bothered to articulate while building it.

The Takeaways

• Having separate AI to verify AI’s work helps.
• Writing documents to recreate an application isn’t something most projects need — but doing it teaches you something about the application you didn’t see while building it.
• No one likes to write documentation. AI can help. lol