My First Real AI Win at a Non-Tech Firm: Turning 4 Hours of Document Work Into 5 Minutes

My colleague used to spend 4+ hours doing something that required zero judgment: reading regulatory documents and reformatting them manually.

Last week, she did it in 5 minutes.

Same task. Same documents. Just… a different approach.

This is the story of how I built an LLM tool that people actually use — inside a non-tech urban planning firm — and what it taught me about making AI real in organizations that don’t speak “AI”.

The moment I knew this was worth building

I’m the only developer in a company of ~130 people.

That means I don’t get to build “cool AI demos.” I build things that either:

• get used
• or quietly die

So when a colleague asked:

“Is there a way to automate this?”

I paid attention.

Because what she described wasn’t analysis. It wasn’t decision-making.

It was:

• reading
• locating information
• copy-pasting
• reformatting

Four hours of mechanical work. Every single time a new mandate comes in.

That’s exactly the kind of problem AI should kill.

The real problem wasn’t the documents

At first glance, this looks simple:

“Just parse the document and map it to a template.”

But here’s the reality:

• Every municipality has its own format
• Headings are inconsistent
• Sections are misplaced or unlabeled
• Structure is… creative

The same information might be:

• on page 2 in one document
• buried in an appendix in another
• split across multiple sections elsewhere

This isn’t structured data.

It’s structured chaos.

Why I didn’t use rules (and didn’t fine-tune anything)

I had 3 options:

1. Rule-based parsing

→ breaks instantly on edge cases

2. Fine-tuning

→ requires dataset + time + maintenance

3. Few-shot prompting

→ fast, flexible, surprisingly powerful

I chose option 3.

Not because it’s trendy. Because it matched the problem perfectly.

The key insight most people miss about LLMs

LLMs don’t need to learn how to read documents.

They already know how.

What they don’t know is:

What you consider a “correct” structure.

That’s the only thing you need to teach them.

And that’s where few-shot comes in.

The entire system (simpler than you think)

The architecture is almost boring:

1. Upload Word document
2. Extract raw text
3. Send to LLM with few-shot examples
4. Generate structured output
5. Export as .docx

That’s it.

No complex pipelines. No heavy ML infra. No fine-tuning.

Just:

• a good prompt
• good examples
• and a clear output format

The part that actually took time (and nobody talks about)

It wasn’t the code.

It was this question:

“What goes where?”

Because the answer is… subjective.

A single section might belong to:

• one place
• two places
• or depend on interpretation

My first instinct:

“I’ll fix this in the prompt.”

That didn’t work.

What did work:

Bringing the domain expert into the loop

The turning point

Instead of guessing, I asked my colleague:

• “Where would you put this?”
• “Why is this wrong?”
• “What’s the rule in your head?”

And that changed everything.

Because:

Prompt engineering is not a technical problem. It’s a knowledge transfer problem.

I didn’t add more examples.

I replaced weak ones with:

• better edge cases
• clearer boundaries
• real expert decisions

The result

• 4 hours → 5 minutes
• ~95% accuracy
• Remaining work = human judgment only

Exactly how it should be.

The tool doesn’t replace the planner.

It removes the part that never needed them.

The unexpected impact

The time savings were great.

But that’s not what mattered most.

What mattered was this:

People started coming to see the tool.

Not because they had to.

Because they were curious.

The real signal of success

In non-tech companies, AI usually lives in:

• presentations
• strategy decks
• vague ideas

This was different.

This was:

“I used this Tuesday morning and it saved me hours.”

That changes how people think.

From:

“Can we do this?”

To:

“What else can we automate?”

The only metric that matters

After testing the MVP, my colleague said:

“We’re going to use this tool to its full potential.”

That’s it.

Not:

• accuracy
• latency
• architecture

Usage.

That’s the real KPI.

What I’d tell any developer building AI in a non-tech company

1. Start with a real user (not an idea)

If no one owns the problem → your tool won’t survive.

2. Your examples matter more than your prompt

Stop over-engineering instructions.

Start curating better examples.

3. Don’t iterate alone

You don’t know what “correct” looks like.

Your user does.

4. Quantify everything

• 4 hours → 5 minutes
• 95% accuracy

This is how you unlock your next project.

Final thought

If you’re a solo developer in a non-tech environment:

You don’t need:

• perfect architecture
• perfect models
• perfect systems

You need:

One tool that actually gets used.

Because that’s how everything starts.

If you’re working on similar problems

LLMs + messy documents LLMs + internal workflows LLMs + “this should be automated but isn’t”

I’d love to hear what you’re building.

These problems look simple.

They’re not.