From Messy Coffee Orders to Clean JSON: Building an LLM Extraction Pipeline

Take a customer’s coffee order and convert it into structured JSON.

Easy, right?

Not really.

A clean order like this is simple:

Can I get a grande latte with oat milk and a butter croissant?

But the real inputs looked more like this:

Lemme get one tall Strawberry Smoothie with caramel drizzle — remove that.
Next, I need a venti drip coffee and extra hot.
Oh, and add three trenta chai lattes with sugar free vanilla...
scratch that vanilla. Add caramel drizzle and make sure no whip.

This is not just entity extraction. The model has to understand the order as a sequence of events: add this, modify that, cancel this modifier, remove that item, update the quantity, and finally return only what survived.

The full notebook is here: Chain-of-Thought + Few-Shot Retrieval

The challenge

The task was to transform messy spoken-style orders into JSON like this:

{
  "items": [
    {
      "name": "Latte",
      "size": "Grande",
      "quantity": 1,
      "modifiers": ["Oat Milk"]
    },
    {
      "name": "Butter Croissant",
      "size": null,
      "quantity": 1,
      "modifiers": []
    }
  ],
  "total_price": 9.3
}

The difficulty came from the edge cases:

• spoken quantities: “a couple”, “double”, “a few”
• filler words: “like”, “uh”, “actually”, “you know”
• item cancellations: “scratch that”, “remove that”, “nevermind”
• modifier cancellations: “scratch that vanilla”
• exact menu names: Frappe (Mocha) is not the same as mocha frappe
• food rules: food has no size and no modifiers
• pricing: every size and modifier has a deterministic cost

This is where a single prompt is usually not enough. The system needs structure around the LLM.

My approach

I built the pipeline around one principle:

Let the LLM handle messy language. Let code handle anything that must be exact.

The final architecture looked like this:

Customer order
   ↓
Retrieve similar examples from training data
   ↓
Build menu-aware prompt
   ↓
LLM extracts structured JSON
   ↓
Parse and clean the JSON
   ↓
Calculate price deterministically in Python
   ↓
Write submission.csv

The LLM was not responsible for everything. It only handled language understanding. The menu rules, validation, and pricing stayed in Python.

That separation made the pipeline much easier to debug.

Step 1: Make the model menu-aware

The first step was converting the menu JSON into a readable prompt section.

The menu included item names, categories, base prices, sizes, modifiers, and pricing rules. I added this context to the prompt so the model could map messy phrases to exact menu items.

For example, the model had to output:

Frappe (Mocha)
Drip Coffee
Caramel Macchiato
Bacon Gouda Sandwich

not random variations like:

Mocha Frappe
large drip
caramel coffee
bacon sandwich

This matters because downstream scoring expects exact names.

Step 2: Use dynamic few-shot retrieval

Instead of giving the model the same examples every time, I used semantic search to retrieve similar past orders.

For each new order:

1. Embed the order using sentence-transformers.
2. Compare it with embeddings from the training examples.
3. Select the top-k most similar examples.
4. Add those examples to the prompt.

Simplified code:

q_emb = model.encode([order], normalize_embeddings=True)
scores = embeddings @ q_emb.T
top_indices = np.argsort(scores.flatten())[::-1][:5]

This helped because different orders fail for different reasons.

Some orders are hard because of cancellations:

Add a latte with oat milk, actually remove that oat milk

Some are hard because of quantity changes:

Make that three grande cappuccinos

Others are hard because the same item appears multiple times with different modifiers.

Dynamic few-shot retrieval gives the model examples that are close to the current problem instead of relying on generic examples.

Step 3: Treat the order as an event stream

The most important prompting idea was this:

Process the order left to right as a sequence of events.

That sounds small, but it changes the task.

The model should not just collect keywords. It has to maintain state.

Example:

Add a latte with oat milk, actually remove that oat milk

Final result:

{"name": "Latte", "modifiers": []}

But with this order:

Add a latte with oat milk, actually remove that

The final result may be no latte at all.

So the prompt focused heavily on corrections and cancellations:

- Process the order left to right.
- “scratch that” removes the most recent item when no modifier is specified.
- “scratch that vanilla” removes only the vanilla modifier.
- Food items always have size = null and modifiers = [].
- Item names must exactly match the menu.
- Return raw JSON only.

This was one of the biggest improvements. The task became less like simple parsing and more like tracking changes to an order cart.

Step 4: Do not let the LLM calculate prices

This was the most important engineering decision.

The model extracted the items, but it did not calculate the total price.

Why?

Because pricing is not a language problem.

If the menu says:

Latte = $4.50
Grande = +$0.50
Oat Milk = +$0.80

then the total should be calculated by code, not guessed by an LLM.

The pricing logic was deterministic:

line_price = base_price
line_price += size_adjustment
line_price += modifier_prices
total += line_price * quantity

This avoided a common LLM failure mode: the model understands the order correctly but returns the wrong arithmetic.

The takeaway here is simple:

Use the LLM for ambiguity. Use code for certainty.

Challenges I ran into

1. Cancellations were ambiguous

The phrase “remove that” can mean different things depending on context.

Add caramel drizzle — remove that

Probably means remove the modifier.

Add a strawberry smoothie — remove that

Probably means remove the item.

The fix was to make the prompt explicitly track state from left to right.

2. Exact names were fragile

The model sometimes produced names that were semantically right but not valid according to the menu.

For example:

MATCHA LATTE

instead of:

Matcha Latte

To reduce this, I gave the menu in the prompt and also added fuzzy matching / cleanup in Python.

3. LLM output was not always clean JSON

Even when asked for JSON only, models can sometimes add extra text or formatting.

So I added a JSON extraction layer that removed markdown fences, ignored scratchpad text, and parsed the first valid JSON object.

4. Long-running inference needed resilience

The test set had thousands of orders. That means rate limits, retries, and interruptions matter.

So I added:

• retry logic
• exponential backoff
• small delays between calls
• incremental writes to submission.csv

This is not the exciting part of LLM work, but it is the part that makes the pipeline usable.

Local benchmark

I ran a small local benchmark on 20 training examples before generating the final submission.

Full match:  20/20
Items match: 20/20
Price match: 20/20

This does not prove the system is perfect. Twenty examples are only a sanity check.

But it was useful for catching obvious issues in extraction, formatting, and pricing before running the full test set.

What made the biggest difference

The strongest parts of the final solution were not just the model choice.

They were the system design choices around the model:

1. Menu-aware prompting The model needed to know the exact item names and modifier names.
2. Dynamic few-shot retrieval Similar examples helped more than generic examples.
3. Stateful cancellation logic Orders had to be processed left to right, not as keywords.
4. Deterministic pricing The LLM extracted meaning; Python calculated the total.
5. Resilient execution Retries and incremental writes made the full run safer.

Final thoughts

This project was a good reminder that useful LLM systems are rarely just one prompt.

The prompt matters, but the surrounding pipeline matters just as much.

For this task, the LLM was useful because the input was messy and ambiguous. But once the order was extracted, everything else became a normal software engineering problem: validation, pricing, formatting, and reliability.

That is the pattern I would reuse:

Use LLMs where language is messy. Use deterministic code where correctness matters.

In this project, that meant Llama 3.3 handled the messy coffee orders, few-shot retrieval gave it better context, and Python made sure the final output followed the rules.