Papperoni

2026

GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning

Lakshya A. Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl Ong, Arnav Singhvi, Herumb Shandilya, Michael J. Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, Omar Khattab

Open PDF Google Scholar

citations

Cite Score

AI summary

This paper introduces GEPA (Genetic-Pareto), a reflective prompt optimizer that leverages natural language reflection and Pareto-based evolutionary search to learn high-level rules from trial and error, achieving significant performance gains over GRPO and MIPROv2 on six tasks with fewer rollouts and demonstrating promising results for code optimization.

Main Contributions

Introduced GEPA (Genetic-Pareto), a novel prompt optimizer that integrates natural language reflection and multi-objective evolutionary search.
GEPA significantly outperforms GRPO, an RL-based method, achieving up to 19 percentage points higher performance with 35x fewer rollouts.
GEPA surpasses MIPROv2, a leading prompt optimizer, by over 10 percentage points (e.g., +12pp on AIME-2025).
Demonstrates GEPA's effectiveness as an inference-time search strategy for code optimization.
GEPA-learned prompts are sample-efficient, generalize well, and can be used to generate robust, generalizable sets.

Abstract

Large language models (LLMs) are increasingly adapted to downstream tasks via reinforcement learning (RL) methods like Group Relative Policy Optimization (GRPO), which often require thousands of rollouts to learn new tasks. We argue that the interpretable nature of language often provides a much richer learning medium for LLMs, compared to policy gradients derived from sparse, scalar rewards. To test this, we introduce GEPA (Genetic-Pareto), a prompt optimizer that thoroughly incorporates natural language reflection to learn high-level rules from trial and error. Given any AI system containing one or more LLM prompts, GEPA samples trajectories (e.g., reasoning, tool calls, and tool outputs) and reflects on them in natural language to diagnose problems, propose and test prompt updates, and combine complementary lessons from the Pareto frontier of its own attempts. As a result of GEPA's design, it can often turn even just a few rollouts into a large quality gain. Across six tasks, GEPA outperforms GRPO by 6 percentage points on average and by up to 19pp, while using up to 35× fewer rollouts. GEPA also outperforms the leading prompt optimizer, MIPROv2, by over 10 percentage points (e.g., +12pp on AIME-2025), and demonstrates promising results as an inference-time search strategy for code optimization. We release our code at https://github.com/gepa-ai/gepa.

Citation Graph

Loading graph...

References [82]

Sort:

Filter:

[1]Deepseekmath: Pushing the Limits of Mathematical Reasoning in Open Language Models

Zhihong Shao, Peng Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Yiwei Li, Yonghui Wu - 2024

3 papers in library cite