Intelligent Prompt Optimization with GEPA: Using Reflection LLMs to Fix What Manual Engineering Can’t

Turning business outcomes into better LLM instructions — no fine-tuning, no labeled data, no guesswork.

The Problem: Static Prompts in a Dynamic World

Business rules change. User behavior shifts. New product categories appear. Compliance requirements tighten. The prompt that worked last quarter doesn’t work this quarter — because the world moved, but the agent didn’t.

This is the fundamental challenge with LLM-powered agents in production: they don’t learn. An agent makes a mistake on Monday, and on Tuesday it makes the exact same mistake because nothing in its instructions changed. The prompt is static in a world that isn’t.

Failures come from two directions at once:

• Changing business needs — evolving business rules, shifting user behavior, new categories, seasonal patterns, updated compliance requirements
• Edge cases at scale — domain-specific nuances, ambiguous inputs, category-specific rules that no amount of upfront prompt engineering can anticipate

You fix one failure, break two others. The edge cases multiply faster than any human can patch them.

Most teams already have the signals to fix this:

• KPIs and accuracy scores
• Revenue and conversion impact
• Resolution metrics and customer satisfaction
• Binary outcomes (did this work? yes/no) and scalar outcomes (metric improved by 3%)

But the loop isn’t closed. The outcome data sits in a dashboard while the prompt stays frozen.

The problem: how do you make agents learn from production outcomes and continuously improve their own decision logic — automatically, evidence-based, and at scale?

That’s what GEPA solves.

What Is GEPA?

GEPA (Genetic-Pareto Optimization) is a framework for intelligent prompt optimization that comes out of research (GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning).

The core idea:

What if an AI model could learn from its own mistakes — not through brute-force training, but through thoughtful reflection, much like a human?

GEPA treats your LLM prompt as a candidate solution and iteratively improves it using an evaluation loop:

• Instead of you reading failures and rewriting the prompt, a reflection LLM does it for you
• Guided by structured feedback from real production outcomes
• Doesn’t need labeled training data — just a scoring function that says right or wrong

That scoring function can be powered by:

• Business outcomes (revenue, conversion)
• Resolution metrics (time to resolve, customer satisfaction)
• Operational signals (error rates, escalation frequency)
• Any measurable signal from production

Why “Pareto”?

Pareto optimality means you can’t make something better without making something else worse. In prompt optimization, this matters because improving accuracy on one type of input often degrades accuracy on another.

Traditional optimization finds one best prompt — but that prompt might overfit to one pattern while breaking others. GEPA takes a fundamentally different approach:

• Maintains a Pareto front — instead of evolving only the global best prompt, it stochastically explores the top-performing prompts for each problem instance, enabling robust generalization
• Pareto-based candidate sampling — instead of always mutating the single best-performing candidate (which leads to local optima), it filters and samples from the list of best candidates per task, ensuring diversity
• Two-stage evaluation — proposes a new candidate each iteration, first evaluating on a minibatch, and if improved, evaluating on a larger dataset

Think of it as turning multiple knobs simultaneously to find the optimal configuration — it doesn’t get stuck optimizing for one pattern while breaking others.

These design decisions make GEPA:

• Highly sample-efficient — fewer LLM calls needed to find improvements
• Strongly generalizing — improvements hold across diverse inputs, not just the training set

How It Differs from Fine-Tuning

GEPA sits in a sweet spot:

• More powerful than manual prompt engineering
• Lighter than fine-tuning
• Output is a readable, auditable prompt — not an opaque model checkpoint

Not Just Prompts

While this article focuses on prompt optimization, GEPA’s optimize_anything API can optimize any string-representable parameter:

• System prompts
• Few-shot example selection
• Filter criteria and decision thresholds
• Configuration templates
• Any runtime variable that drives agent behavior and can be scored against outcomes

The Workflow

1. Target LLM

The LLM whose prompt you’re optimizing:

• Sees only the input data — never the scores, never the ground truth
• Runs the candidate prompt and returns its prediction
• Doesn’t know it’s being optimized

2. Evaluator (Scoring Function)

This is the piece you write. It compares the LLM’s output against your ground truth and returns:

• Score — binary (1.0 correct, 0.0 wrong) or scalar
• ASI (Actionable Side Information) — a structured payload describing:
• What the LLM saw (input)
• What it predicted (output)
• What the correct answer was (ground truth)
• Why it was wrong (failure diagnosis)

The ASI is what makes GEPA’s optimization intelligent rather than random. The richer your failure descriptions, the better the reflection LLM can diagnose root causes and propose targeted fixes.

3. GEPA Engine + Reflection LLM

GEPA orchestrates the optimization loop:

• After each iteration, feeds all failures (with ASI) to a reflection LLM
• The reflection LLM reads failure patterns and proposes a modified prompt
• If the mutation improves the Pareto front → keep it
• If not → sample a different candidate, try a different direction
• Avoids local optima by maintaining diversity across the solution space

Example ASI payload the reflection LLM might see:

{
  "Input": {
    "message": "Cancel my subscription immediately",
    "active_incidents": ["outage-2024-03"]
  },
  "Ground Truth": {
    "correct_team": "tech_support",
    "resolution": "outage resolved, customer retained"
  },
  "LLM Prediction": {
    "team": "billing",
    "reasoning": "Customer requested cancellation"
  },
  "Evaluation": "WRONG: Routed to billing but customer was venting about outage. Unnecessary churn risk."
}

Reflection LLM’s proposed fix:

“When the message contains cancellation/refund language, check for active service incidents. If an incident is ongoing, route to technical support — the customer is likely expressing frustration, not intent.”

Implementation

Step 1: Define Your Seed Prompt

Start with a reasonable first attempt. It doesn’t need to be perfect — GEPA improves from here.

SEED_PROMPT = """You are a customer support ticket classifier.
Given a customer message and order metadata, route to the correct team.
Rules:
- Analyze the customer's language and intent
- Cross-reference with order status metadata
- Return JSON: {team: string, confidence: float, reasoning: string}
"""

Step 2: Build Your Evaluator

The evaluator encodes your scoring logic and builds ASI for failures.

def evaluate(candidate_prompt: str, example: Ticket) -> tuple[float, dict]:
    # Call the target LLM with candidate prompt
    result = call_llm(system_prompt=candidate_prompt, input=example.to_llm_input())

# Score against ground truth (actual resolution outcome)
    correct = result["team"] == example.correct_team
    score = 1.0 if correct else 0.0
    # Build ASI for GEPA's reflection LLM
    side_info = {
        "Input": {"message": example.message, "order_status": example.order_status},
        "Ground Truth": {"correct_team": example.correct_team},
        "LLM Prediction": result,
        "Evaluation": "CORRECT" if correct else f"WRONG: {explain_failure(result, example)}"
    }
    return score, side_info

Key design decision: what is your ground truth?

• Historical resolution outcomes (which team actually resolved it)
• A/B test results
• Revenue impact
• Customer satisfaction scores
• Any measurable signal from production

The prompt learns not just “what seems right” but “what actually produces good outcomes.”

Step 3: Configure and Run GEPA

from gepa.optimize_anything import optimize_anything, GEPAConfig, EngineConfig, ReflectionConfig

result = optimize_anything(
    seed_candidate=SEED_PROMPT,
    evaluator=evaluate,
    dataset=evaluation_examples,
    objective="Optimize this prompt for routing accuracy. "
              "Do NOT change the output JSON schema. "
              "You may add context-aware rules and escalation guards.",
    background="Common failure modes: keyword-based misrouting, "
               "ignoring service context that contradicts customer's literal words.",
    config=GEPAConfig(
        engine=EngineConfig(max_metric_calls=10, seed=42),
        reflection=ReflectionConfig(reflection_lm="gpt-4o"),
    ),
)
optimized_prompt = result.best_candidate

Key parameters:

• objective — constrains what the reflection LLM can change (locks down schema, safety rules)
• background — describes known failure modes to guide mutation direction
• max_metric_calls — budget for LLM evaluations
• reflection_lm — model used for analyzing failures and proposing mutations

Step 4: Analyze Results

Every run produces structured logs (JSONL) for analysis:

• Accuracy curve — did it improve? How fast? Where did it plateau?
• Per-example tracking — which examples got fixed at which iteration?
• Reasoning evolution — how did the LLM’s reasoning change as the prompt improved?
• Prompt diff — what rules did GEPA add, modify, or remove?

What Happened in Practice

Before (Seed Prompt): Poor Accuracy

The hand-crafted prompt applied generic rules:

• Correctly handled straightforward cases
• Failed when business context mattered
• Relied on surface-level keyword matching

Example: Customer writes "Cancel my subscription immediately"
Seed prompt: Classified as "Cancellation" → routed to billing ← WRONG
Reality: Customer was frustrated about a service outage and vented.
         They didn't actually want to cancel — they wanted the outage fixed.
         Routing to billing instead of technical support caused unnecessary churn.

Iteration 1–2: No Improvement

• First two mutations weren’t aggressive enough
• Prompt still classified based on keyword matching
• GEPA’s Pareto front hadn’t yet discovered the right mutation direction

Iteration 3: Breakthrough

The reflection LLM analyzed recurring failures and added intent-vs-emotion classification guards:

“When the message contains high-emotion action words (‘cancel,’ ‘refund,’ ‘close my account’), cross-check against recent service events. If there’s an active incident or outage, classify as technical support — not the literal action requested. The customer is expressing frustration, not intent.”

Why this matters:

• This rule would never have been written manually
• The reflection LLM discovered from resolution outcome data that cancellation language during active outages was overwhelmingly a frustration signal, not genuine churn intent
• The pattern was invisible in the raw data but emerged from structured failure analysis

After (Optimized Prompt): Significant Accuracy Improvement

Key additions GEPA discovered:

Accuracy per Iteration
  High |                    *---------*
       |                   /
       |                  /
  Low  |  *---------*    /
       |                /
    0% +--+----+----+----+
       Iter 1  Iter 2  Iter 3  Iter 4

The most valuable output wasn’t just the accuracy number — it was reading what GEPA added and understanding domain patterns that weren’t obvious from the data alone.

Guardrails: Keeping Optimization Safe

Unconstrained prompt mutation is risky. GEPA could theoretically:

• Remove safety rules
• Change the output format
• Inject business metrics into the prompt
• Over-specialize for training data

Two layers of protection:

1. Mutation constraints via GEPA’s objective:

objective = (
    "CONSTRAINTS — mutations MUST preserve:\n"
    "1. Output JSON schema — do NOT change field names or types\n"
    "2. The prompt must NOT reference business metrics or scores\n"
    "3. Must default to safe routing when uncertain\n"
    "You may add rules. You may NOT remove existing safety rules."
)

2. Schema validation in the evaluator:

• Validates output types, required fields, value ranges
• Any schema violation → score=0
• GEPA’s Pareto sampling learns to avoid those mutation directions
• Acts as a hard constraint the reflection LLM cannot bypass

The Reusable Tuner Pattern

The real value isn’t just one optimized prompt. It’s a reusable pattern that any agent can adopt:

Any agent that has these three things can plug in:

1. A scoring function — business outcomes, resolution metrics, any measurable signal
2. Runtime variables — prompts, thresholds, configs that drive decisions
3. Outcome data — flowing back from production

The tuner:

• Runs offline (not in the request path)
• Discovers improvements from real outcomes
• Publishes approved changes to the agent’s config registry
• The agent doesn’t need to know GEPA exists — it just reads its latest config and runs

When to Use GEPA

Works best when:

• You have a scoring function tied to real business outcomes
• Your agent needs domain-specific rules that evolve as business needs and user behavior change
• You want interpretable output — readable prompt diffs, not black-box weights
• You need to iterate faster than fine-tuning allows
• You want your agents to adapt continuously to production signals
• You need evidence-based improvement — “this prompt scored 93% on the last 100 real cases, up from 90%”

Less suitable when:

• You need the LLM to learn fundamentally new capabilities (use fine-tuning)
• Your scoring function is noisy or unreliable
• Your prompt is already near-optimal and you need marginal gains

Key Takeaways

1. Your prompt is a hypothesis. GEPA treats it as one and tests it against real outcomes. The optimized prompt is evidence-based, not intuition-based.
2. ASI quality matters more than dataset size. Rich, structured failure descriptions help the reflection LLM make targeted fixes. Small evaluation sets work when the ASI clearly explains why each failure occurred.
3. Pareto optimization prevents whack-a-mole. By maintaining a front of diverse top-performing candidates, GEPA avoids the trap of fixing one pattern while breaking others.
4. The prompt diff is the insight. The most valuable output isn’t just the accuracy number — it’s reading what GEPA added and understanding domain patterns you didn’t know existed.
5. Ground truth doesn’t need to be labeled data. Business outcomes — resolution time, revenue impact, customer satisfaction, conversion rates — work as ground truth. The LLM learns to make decisions that produce good outcomes, not just technically correct ones.
6. Production is not static. Business rules change, user behavior shifts, new scenarios emerge. GEPA enables a continuous optimization loop — re-run with fresh outcome data and the agent evolves to match the current reality.

References

GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning — arxiv.org/pdf/2507.19457

GEPA GitHub Repository — github.com/gepa-ai/gepa