GEPA: Learning from Explanations

This is the final article in a three-part series: DSPy, SIMBA, and GEPA.

A score of zero can describe many different failures.

A support ticket may be routed incorrectly because the model misunderstood the customer’s intent. It may have identified the intent but returned an invalid category. It may have ignored a policy rule, confused two similar products, or been deprived of information that another component should have retrieved.

The number tells us that something went wrong. It throws away the explanation.

GEPA begins with the observation that this is an unnecessary loss of information. Language-model systems operate in language, produce trajectories that can be read, and often fail in ways that can be described. Instead of reducing every rollout to a scalar reward, an optimizer can use natural-language feedback to diagnose the failure and propose a change.

GEPA—Genetic-Pareto—is a reflective prompt optimizer for compound AI systems. It samples execution trajectories, receives scores and textual feedback, uses an LM to propose instruction mutations, and preserves candidates that contain useful lessons.¹

The intuitive loop is:

execute the program
        ↓
observe its trajectory
        ↓
score the result and explain the failure
        ↓
reflect on what instruction should change
        ↓
test the mutated program
        ↓
retain useful candidates and repeat

The algorithm is interesting. The engineering lesson is more important: if we want a system to improve, we must preserve enough information about its failures to make improvement possible.

A score is a verdict; feedback is a lesson

Imagine that our support router predicts account for this ticket:

I updated my card yesterday. The card is correct in settings, but today’s renewal still charged the old card.

The expected route is billing. A binary metric returns 0.

Now compare that with:

The request concerns which payment method was charged, not whether the customer can access or edit the account. Route disputes about completed charges to billing.

The feedback identifies the distinction the model missed. A reflection model can convert it into a more general instruction, perhaps:

Route by the failed outcome. Questions about a completed or duplicated charge are billing issues even when the ticket mentions account settings.

The new rule may still be wrong. GEPA tests it. Reflection proposes; evaluation decides.

This division matters. The reflection LM is not the source of truth. It is a hypothesis generator operating over the trajectory and feedback supplied to it.

GEPA’s three central ideas

Reflective prompt mutation

GEPA runs a candidate program on a small set of examples and captures what happened inside the program: inputs, intermediate predictions, tool calls, tool outputs, final output, score, and feedback.

A reflection LM reads this material and proposes a revised instruction for a particular predictor. In a multi-step system, this enables a form of credit assignment. Instead of rewriting the entire agent prompt after an end-to-end failure, GEPA can focus on the component whose behavior should change.

This is where program structure becomes valuable. A monolithic prompt hides responsibility. Explicit modules give both the engineer and optimizer somewhere local to apply a lesson.

A Pareto frontier across examples

Suppose candidate A is best on billing tickets, candidate B is best on account-security reports, and candidate C has the highest average score.

A greedy search might retain only C. GEPA preserves candidates that are best on at least one validation example. Those candidates form a per-example Pareto frontier: a candidate survives when no other candidate is at least as good everywhere and better where it matters.

This acts like memory. A specialist may contain a rule the current generalist lacks. Keeping the specialist makes that lesson available for future mutation or merging.

One subtle point is easy to misunderstand: the Pareto dimensions are validation examples, not automatically separate business objectives such as quality, cost, and latency. If those objectives matter, they must be represented in the scoring and evaluation design. The frontier broadens exploration; the final returned program is still selected by aggregate validation performance.²

System-aware merging

A compound program may contain several predictors. One candidate may improve query generation while another improves answer validation.

GEPA can attempt to combine complementary component-level improvements. This is more useful for a multi-module program than for a single predictor, where there are fewer independent pieces to recombine.

Merging makes the “genetic” metaphor visible, but GEPA is not simply a conventional genetic algorithm applied to strings. Its mutations are proposed through language-level reflection informed by program traces and task feedback.

How GEPA works

At a high level, optimization proceeds as follows:

1. Begin with the current program as the seed candidate.
2. Evaluate candidates on a validation set and record per-example scores.
3. Select a candidate from the Pareto frontier.
4. Run that candidate on a training mini-batch and capture trajectories.
5. Pass predictor-specific traces and textual feedback to a reflection LM.
6. Ask the reflection LM for a revised instruction.
7. Test the mutation on the mini-batch.
8. If it passes the acceptance check, evaluate it on the validation set.
9. Update the candidate pool and Pareto frontier.
10. Occasionally merge complementary candidates in multi-component programs.
11. Continue until the optimization budget is exhausted.
12. Return the candidate with the best aggregate validation score.

This separation between training and validation has a precise role. The training mini-batch produces and initially tests mutations. The validation set determines what enters and remains useful in the wider search.

Neither is the final test set. A final test set should remain invisible until development choices are finished.

The feedback-shaped metric

GEPA works with an ordinary numeric metric, but that reduces it to a weaker form. In DSPy, its intended metric can return both a score and feedback:

def routing_feedback(
    example,
    prediction,
    trace=None,
    pred_name=None,
    pred_trace=None,
):
    predicted = prediction.route.strip().lower()
    expected = example.route.strip().lower()
    correct = predicted == expected

    if correct:
        feedback = "The route matches the expected destination."
    else:
        feedback = (
            f"Expected {expected}, but predicted {predicted}. "
            f"Relevant distinction: {example.rationale}"
        )

    return dspy.Prediction(
        score=float(correct),
        feedback=feedback,
    )

The extra arguments allow GEPA to provide either the whole-program trajectory or the trace for one predictor. A simple application may use the same feedback for both. A sophisticated multi-module program can grade different stages differently.

The official documentation makes the contract explicit: score is used for comparison, while feedback is inserted into the reflection prompt. Returning a bare float remains valid, but the reflection model receives only generic information about the score.²

Feedback should be:

• Specific: identify the important error.
• Causal when possible: explain which decision produced the failure.
• Local: address the module being optimized.
• Generalizable: expose a rule rather than prescribe one memorized answer.
• Grounded: come from tests, validators, expert labels, policies, or a calibrated judge.

“Wrong answer” is not useful feedback. Neither is an elaborate explanation invented by an unreliable judge.

Running GEPA in DSPy

Using the router from the first article, an illustrative configuration looks like this:

task_lm = dspy.LM("openai/gpt-4o-mini")
reflection_lm = dspy.LM("openai/gpt-4o")

dspy.configure(lm=task_lm)

optimizer = dspy.GEPA(
    metric=routing_feedback,
    auto="light",
    reflection_lm=reflection_lm,
    track_stats=True,
    log_dir="./gepa-routing-logs",
)

optimized_router = optimizer.compile(
    SupportRouter(),
    trainset=trainset,
    valset=valset,
)

The model names are examples, not recommendations. The architectural distinction is what matters:

• The task model runs the program repeatedly and therefore dominates evaluation volume.
• The reflection model is called less often to propose mutations and benefits from stronger reasoning.

A stronger reflection model can optimize instructions for a cheaper task model. This can make economic sense because optimization is occasional while the task program may run thousands of times.

GEPA accepts several ways to define its budget. The current DSPy API requires choosing exactly one of auto, max_full_evals, or max_metric_calls. auto="light" is a useful exploratory start; a cost-sensitive experiment may prefer an explicit metric-call ceiling. Consult the GEPA in-depth documentation and current API before running a large job.

The evaluator is part of the product

GEPA makes evaluator quality impossible to ignore. The optimizer can turn a subtle bias in a metric into a highly effective strategy for exploiting that bias.

An LLM judge should therefore be treated as another predictive model, not as an oracle. It needs:

• A narrow, operationally meaningful rubric.
• Human-labeled examples and rationales.
• Agreement measurements against those labels.
• Precision and recall for consequential classes.
• Inspection of false positives and false negatives.
• Tests for prompt injection inside evaluated trajectories.
• Ongoing calibration as production behavior changes.

The Judge the Judge workshop offers a useful case study. Its initial evaluator tended to approve examples, producing respectable overall accuracy while missing policy violations. Optimizing the rubric improved the important negative-class recall much more than the headline accuracy. The lesson is that the confusion matrix may tell a more important story than the average.

Deterministic checks should be used whenever possible. A support router does not need an LM judge to determine whether its output belongs to the allowed set. Code can check that perfectly. Save model-based evaluation for semantic distinctions that code cannot reliably express.

What the GEPA paper establishes—and what it does not

The GEPA paper reports results across six tasks. In the authors’ experiments, GEPA outperformed GRPO by six percentage points on average and by up to nineteen points, while using up to 35 times fewer rollouts. It also outperformed MIPROv2 by more than ten points in the reported comparisons and showed promising inference-time results for code optimization.¹

These results support a compelling research claim: rich linguistic feedback can be a highly sample-efficient optimization medium for language-model systems.

They do not establish that prompt optimization universally beats reinforcement learning. The two methods modify different surfaces:

• GEPA changes readable instructions and other textual system components.
• Reinforcement learning changes model weights.

Prompt evolution is attractive when failures can be explained, rollouts are expensive, interpretability matters, and the required behavior lies within the model’s existing capabilities. Weight optimization may be needed when instructions cannot reliably elicit the desired capability or when enormous interaction volume is available.

The methods can also be composed. A program can be optimized around a fine-tuned model, and a prompt-optimized program can later supply useful traces for weight training.

SIMBA and GEPA are related, not interchangeable

Both optimizers use LM-generated reflection and maintain multiple candidate programs. Their information and search strategies differ.

Neither is “DSPy versus GEPA.” DSPy is the programming model; SIMBA and GEPA are optimization strategies available for programs expressed within it.

The simplest optimizer that produces a real held-out improvement is often the correct choice. A complex evolutionary search is not intrinsically better engineering.

GEPA outside DSPy

GEPA is also available as a standalone library. If an existing harness is not built from DSPy modules, it can expose its optimizable text and evaluation behavior through an adapter rather than being rewritten wholesale.

This creates three reasonable architectures:

The boundary should reflect ownership. Authentication, permissions, irreversible actions, privacy constraints, and deterministic validation belong in ordinary code. Instructions, rubrics, routing descriptions, and other textual policies may be reasonable optimization targets.

A responsible optimization experiment

A convincing GEPA result should include more than a before-and-after score.

I would record:

Run at least one simple optimizer as a baseline. If four carefully selected demonstrations produce the same gain as GEPA, the simpler artifact may be easier to maintain.

Finally, rerun the evaluation when the underlying model changes. An evolved prompt may encode behavior specific to the model on which it was optimized.

The deeper idea

DSPy separates a language-model program from its learned implementation. SIMBA demonstrates that variation among attempts can be converted into rules and demonstrations. GEPA shows that explanations can become an optimization medium.

Together they suggest a broader way to build AI systems:

specify the task
      ↓
make behavior observable
      ↓
define success and failure precisely
      ↓
preserve informative feedback
      ↓
let optimization propose changes
      ↓
validate those changes outside the loop that created them

The bottleneck moves. Less time is spent guessing prompt wording. More responsibility moves into task definition, dataset construction, evaluation, and judgment.

That is not the elimination of engineering. It is engineering at a more durable level.

Learning resources

• GEPA Explained — Connor Shorten’s visual 30-minute introduction.
• GEPA with Lakshya A. Agrawal — a one-hour conversation with the paper’s lead author.
• DSPy GEPA listwise reranker and its companion notebook — the clearest compact implementation walkthrough.
• GEPA in depth — current DSPy mechanics and metric contract.
• Official GEPA tutorials — examples for DSPy and other systems.
• Judge the Judge — evaluator calibration and GEPA-based rubric optimization.
• Agent Optimization with Pydantic AI — a production-oriented loop connecting evals, traces, GEPA, and controlled configuration delivery.

References

1. Lakshya A. Agrawal et al. “GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning.” International Conference on Learning Representations, 2026. Official proceedings entry.
2. Omar Khattab et al. “DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines.” International Conference on Learning Representations, 2024.
3. DSPy contributors. “GEPA in depth.” DSPy documentation.
4. GEPA contributors. “GEPA documentation and tutorials.” GEPA project.