CoChat
Sign Up
PreprintCited by 39Topic ModelingNatural Language Processing TechniquesReinforcement Learning in Robotics

GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning

Open access
Agrawal, Lakshya A, Tan, Shangyin, Soylu, Dilara, Ziems, Noah, Khare, Rishi, Opsahl-Ong, Krista, Singhvi, Arnav, Shandilya, Herumb +9 more·2025·arXiv
DOI 10.48550/arxiv.2507.19457arXiv 2507.19457
Open this paper in CoChat — free accountOpen PDF

Abstract

Large language models (LLMs) are increasingly adapted to downstream tasks via reinforcement learning (RL) methods like Group Relative Policy Optimization (GRPO), which often require thousands of rollouts to learn new tasks. We argue that the interpretable nature of language often provides a much richer learning medium for LLMs, compared to policy gradients derived from sparse, scalar rewards. To test this, we introduce GEPA (Genetic-Pareto), a prompt optimizer that thoroughly incorporates natural language reflection to learn high-level rules from trial and error. Given any AI system containing one or more LLM prompts, GEPA samples trajectories (e.g., reasoning, tool calls, and tool outputs) and reflects on them in natural language to diagnose problems, propose and test prompt updates, and combine complementary lessons from the Pareto frontier of its own attempts. As a result of GEPA's design, it can often turn even just a few rollouts into a large quality gain. Across six tasks, GEPA outperforms GRPO by 6% on average and by up to 20%, while using up to 35x fewer rollouts. GEPA also outperforms the leading prompt optimizer, MIPROv2, by over 10% (e.g., +12% accuracy on AIME-2025), and demonstrates promising results as an inference-time search strategy for code optimization. We release our code at https://github.com/gepa-ai/gepa .

Verified 2026-10-19 · 1 reference not checked yet

AI snapshot

Written from the paper’s full text.

TL;DR

GEPA (Genetic-Pareto) is a prompt optimizer for compound AI systems that uses natural-language reflection on execution and evaluation traces plus Pareto-based evolutionary search to learn high-level rules from trial and error. Across six benchmarks it outperforms GRPO by 6% on average (up to 20%) with up to 35× fewer rollouts, and beats MIPROv2 by over 10%.

Key findings

  • On Qwen3 8B, GEPA beats GRPO (24,000 rollouts) by up to 19% while using up to 35× fewer rollouts, with an average gain of +6% across six tasks; GEPA reaches optimal test performance with 4–35× fewer rollouts and exceeds GRPO on 5 of 6 tasks by 19.0%, 2.73%, 13.66%, 5.19% and 0.7%. GEPA matches GRPO's best validation after only 243, 402, 330, 1143, 1179, and 306 rollouts (up to 78× greater sample efficiency).
  • GEPA outperforms MIPROv2 on all benchmarks and models, with aggregate gains of +13.33% (GEPA+Merge) and +12.19% (GEPA) vs. +5.64% for MIPROv2 on GPT-4.1 Mini; margins reach 11.1% for GPT-4.1 Mini and 10.3% for Qwen3 8B. On IFBench, GEPA found optimal prompts after just 678 rollouts (38.61%) vs. GRPO's 35.88% at 24,000 rollouts.
  • Pareto-based candidate selection matters: GEPA yields +12.44% aggregate improvement vs. +6.05% for SelectBestCandidate and +5.11% for BeamSearch(N=4); GEPA outperforms BeamSearch by up to 11.33% and SelectBestCandidate by up to 8.17%.
  • GEPA-optimized prompts generalize across models: "GEPA-Qwen-Opt" (optimized on Qwen3-8B, evaluated on GPT-4.1-Mini) achieves +9.00% aggregate improvement (up to +27.67% on HotpotQA), beating MIPROv2 (+5.64%), TextGrad (+6.11%), and Trace (+3.27%) which optimized directly on GPT-4.1-Mini. GEPA prompts are also up to 9.2× shorter than MIPROv2's (around 33% shorter in aggregate).
  • Extended applications: as inference-time search on NPUEval, GEPA with GPT-4o reached a mean vector utilization of 30.52% (several kernels up to 70%) vs. 4.25% for Sequential10 alone, 16.33% with RAG, and 19.03% with MIPROv2; a single GEPA prompt enabled 26.85%. On KernelBench, GEPA boosted GPT-4o's close-to-0% fast1 score to above 20% over 35 tasks. For adversarial prompt search, a GEPA-learned instruction reduced AIME-2025 pass@1 from 76% to 10%.

Methods

  • GEPA iteratively proposes new candidates via reflective prompt mutation or system-aware Merge, evaluating each on a minibatch and, if improved, on a validation set (Dpareto); it maintains a Pareto frontier over per-instance best scores and stochastically samples candidates weighted by how many tasks they lead. Only prompts (ΠΦ) are evolved; LLM weights (ΘΦ) stay frozen.
  • Evaluation covers six benchmarks — HotpotQA, IFBench, HoVer, PUPA, AIME-2025, LiveBench-Math — with Qwen3 8B and GPT-4.1 Mini, compared against Baseline, GRPO (24,000 rollouts, LoRA rank 16), MIPROv2, Trace (OptoPrime), and TextGrad. Standard train/validation/test splits; GEPA used minibatch size 3 and merge invoked at most 5 times.
  • Costs: under $500 for all Table 2 experiments with GPT-4.1 Mini (GEPA $86, GEPA-Merge $67, MIPROv2 $76, Trace and TextGrad $172 total). GEPA made 17–92 reflection LM calls per benchmark.

Limitations

  • The paper states that GEPA+Merge led to performance degradation with Qwen3 8B (benefiting only one of four tasks), attributed to fixed hyperparameters for budget allocation between mutation and crossover and the timing of merge invocation; the authors propose adaptive techniques as future work.
  • The paper notes that most of GEPA's rollout budget is spent on validation (scores used only for candidate selection, not learning signals), and proposes smaller or dynamically selected validation subsets as future work.
  • The paper states that the inference-time search and adversarial prompt search results are "early results" that "warrant further systematic study," and leaves generalization to other code-generation and domain-adaptation tasks for future work.
  • Evident from the text: GEPA underperforms GRPO on AIME-2025 (32.00 vs. 38.00 on Qwen3 8B), and GEPA+Merge scores 28.23 on IFBench with Qwen3 8B, below the 36.90 baseline — the paper's aggregate claims mask per-task regressions.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2507.19457

Cites

0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

6 of 10 cited sources checked · 2 supported · 0 flagged · accuracy not yet measured

  • Findingown data
    § 4 EVALUATION

    GEPA's prompts frequently contain detailed declarative instructions for completing the task, in contrast to prior findings where instruction optimization yielded improvements primarily through quasi-exemplars.

    “Notably, in contrast to prior findings where instruction optimization yielded improvements primarily through quasi-exemplars (Wan et al., 2024), GEPA's prompts frequently contain detailed declarative instructions for completing the task, as illustrated in Figure 2.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Wan et al., 2024 source
  • Findingown data
    § 4 EVALUATION

    If restricted to train set rollouts, GEPA requires only 79 to 737 rollouts to reach optimal performance.

    “If we restrict the analysis to train set rollouts, GEPA requires only 79 to 737 rollouts to reach optimal performance.”
    Quote in the paperSubstantial claim
  • Findingown data
    § 4 EVALUATION

    GEPA+Merge outperforms GRPO by 21% at a comparable rollout budget to GEPA.

    “GEPA+Merge widens the gap, outperforming GRPO by 21% at a comparable rollout budget to GEPA.”
    Quote in the paperSubstantial claim
  • Findingnone given
    § 1 INTRODUCTION

    Rollouts sampled from even highly sophisticated LLM systems can be serialized into traces of natural language containing instructions, reasoning chains, tool calls, and reward function internals.

    “We observe that rollouts sampled from even highly sophisticated LLM systems can be serialized into traces of natural language: they contain nothing but the instructions of each LLM module, the resulting LLM reasoning chains, tool calls, and potentially the internal workings of the reward function (e.g., compiler error messages, before they are collapsed into scalar rewards).”
    Quote in the paperNot a substantial claim
  • Findingown data
    § Figure 1

    As more rollouts are sampled, prompt optimizers can learn much more quickly than GRPO.

    “As more rollouts are sampled, the prompt optimizers can learn much more quickly than GRPO.”
    Quote in the paperSubstantial claim
  • May combine two claims. 38
  • Classified as probably not a substantial claim. 22
  • The quote only partly states this claim. 1

  • May combine two claims.

    Premisecitation
    § 4 EVALUATION

    SelectBestCandidate evolves only from the top-scoring candidate, and BeamSearch maintains a pool of top-N candidates but is still prone to local optima.

    “At each step, SelectBestCandidate (used by TextGrad Yuksekgonul et al. (2025)) evolves only from the top-scoring candidate. BeamSearch maintains a pool of the top-N candidates (used by APO Pryzant et al. (2023)), but is still prone to local optima.”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites Yuksekgonul et al. (2025) source
    • cites Pryzant et al. (2023) source

      Section 2.2.2 (page not available in our copy) says APO keeps the b most promising candidates on the beam for the next iteration, supporting the top-N pool description. The Beam Search Ablation reports that beam search outperformed greedy search on the tested tasks, but our copy does not say it is prone to local optima. The SelectBestCandidate clause is attributed to a different source and is not assessed here. (LLM, conf=0.84)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Interpretationown data
    § Figure 16

    Our results suggest that instructions generated by reflective prompt evolution can achieve stronger generalization as well as improved overall performance.

    “While Wan et al. (2024) previously observed that exemplars tend to generalize better, our results suggest that instructions generated by reflective prompt evolution can achieve stronger generalization as well as improved overall performance.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Wan et al. (2024) source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationcitation
    § Observation 4

    Reducing prompt size reduces runtime cost for downstream tasks, decreases latency, and improves the overall efficiency of LLM-serving systems.

    “This reduction in prompt size has a significant impact—not only reducing runtime cost for downstream tasks (as all API-providers meter the input tokens), but also decreasing latency and improving the overall efficiency of LLM-serving systems (Kwon et al., 2023; Zheng et al., 2024; Agrawal et al., 2023; Yu et al., 2025).”
    Quote in the paperNot a substantial claim3 of 3 sources checked
    • cites Kwon et al., 2023 source

      Section 2.2 says serving processes the input prompt tokens, and the Shared prefix section says avoiding repeated prefix processing reduces computation. Section 3 links token-related KV-cache memory to serving throughput. These passages support an efficiency benefit, but the paper does not establish that all API providers meter input tokens or directly show that reducing prompt size lowers billed cost and latency. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites Zheng et al., 2024 source

      Section 5 says repeated input tokens can incur API fees and that avoiding an API call can save input costs and latency. Sections 3 and 6.2 report that reusing cached prompt prefixes reduces computation and improves latency and throughput. The paper does not establish that shortening prompts themselves produces all these benefits, or that all API providers meter input tokens. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites Agrawal et al., 2023 source

      The Introduction says the prefill phase processes input-prompt tokens, and §3.2 says varying numbers of prefill tokens cause pipeline bubbles that reduce serving throughput. This supports a connection between prompt length and serving efficiency, but the paper does not establish that shortening prompts reduces API charges or show a general latency reduction from doing so. (LLM, conf=0.83)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingown data
    § 4 EVALUATION

    GEPA reaches optimal test performance with 4-35× fewer rollouts and exceeds GRPO on 5 out of 6 tasks by 19.0%, 2.73%, 13.66%, 5.19% and 0.7%.

    “It reaches optimal test performance with 4–35× fewer rollouts and exceeds GRPO on 5 out of 6 tasks by 19.0%, 2.73%, 13.66%, 5.19% and 0.7%.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4 EVALUATION

    The next-candidate selection strategy strongly influences the optimization trajectory and final performance, with Pareto-based sampling providing a distinct advantage.

    “The next-candidate selection strategy strongly influences the optimization trajectory and final performance, with Pareto-based sampling providing a distinct advantage.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § Figure 1

    GEPA substantially outperforms both GRPO and MIPROv2 in final score.

    “GEPA substantially outperforms both GRPO and MIPROv2 in final score.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § Abstract

    Across six tasks, GEPA outperforms GRPO by 6% on average and by up to 20%, while using up to 35x fewer rollouts.

    “Across six tasks, GEPA outperforms GRPO by 6% on average and by up to 20%, while using up to 35x fewer rollouts.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § Figure 5

    GEPA's reflective prompt mutation systematically incorporates task-specific nuances, leading to substantial improvements in performance.

    “GEPA's reflective prompt mutation systematically incorporates task-specific nuances, leading to substantial improvements in performance.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4 EVALUATION

    GEPA's Pareto-based selection yields a +12.44% improvement, significantly outperforming the +6.05% and +5.11% gains of greedy and beam-search strategies respectively.

    “In comparison, GEPA's Pareto-based selection yields a +12.44% improvement, significantly outperforming the +6.05% and +5.11% gains of greedy and beam-search strategies respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4 EVALUATION

    GEPA matches GRPO's best validation after only 243, 402, 330, 1143, 1179, and 306 rollouts-up to 78× greater sample efficiency.

    “GEPA matches GRPO's best validation after only 243, 402, 330, 1143, 1179, and 306 rollouts—up to 78× greater sample efficiency.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4 EVALUATION

    GEPA and GEPA+Merge more than double the aggregate gains over baseline seen with MIPROv2 across all benchmarks and models (+13.33% and +12.19% vs +5.64% for MIPROv2).

    “Further, GEPA and GEPA+Merge more than double the aggregate gains over baseline seen with MIPROv2 across all benchmarks and models (+13.33% and +12.19% vs +5.64% for MIPROv2).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § Observation 6

    Prompts optimized using Qwen3-8B transfer effectively to GPT-4.1-Mini, achieving a +9.00% aggregate improvement across 6 benchmarks, with gains as high as +27.67% on HotpotQA.

    “Despite originating from a weaker model in a different family, these prompts transfer effectively, achieving a +9.00% aggregate improvement across 6 benchmarks (with gains as high as +27.67% on HotpotQA).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § main text after Figure 6

    Always choosing the current best candidate gives immediate improvement but quickly stalls, wasting rollouts on a single candidate.

    “always choosing the current best candidate gives immediate improvement but quickly stalls, wasting rollouts on a single candidate.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § E.3 COSTS

    MIPROv2 costs $76.

    “MIPROv2 costs $76”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 5.1 NPU Kernels

    With GPT-4o alone, Sequential10 reaches only 4.25% mean vector utilization; adding RAG improves this to 16.33%, and integrating MIPROv2 raises it to 19.03%.

    “With GPT-4o alone, Sequential10 reaches only 4.25% mean vector utilization. Adding RAG, sourced from technical manuals, improves this to 16.33%, and integrating MIPROv2 further raises it to 19.03%.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given
    § Figure 17

    Most of GEPA's prompt tokens are used for providing instructions, whereas most of MIPROv2's prompt tokens pertain to few-shot examples.

    “Most of GEPA's prompt tokens are used for providing instructions, whereas most of MIPROv2's prompt tokens pertain to few-shot examples.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § E.3 COSTS

    It costs under $500 to run all experiments in Table 2 with GPT-4.1 mini.

    “It costs under $500 to run all experiments in Table 2 with GPT-4.1 mini.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § main text after Figure 6

    GEPA with Pareto-based sampling outperforms the BeamSearch strategy by up to 11.33% and the SelectBestCandidate strategy by up to 8.17%, with aggregate margins of +7.33% and +6.4% across all benchmarks, respectively.

    “GEPA with Pareto-based sampling outperforms the BeamSearch strategy by upto 11.33%, and SelectBestCandidate strategy by up to 8.17%, with an aggregate margin of +7.33% and +6.4% across all benchmarks, respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § Domain-Specific Notes:

    GEPA's generated prompt for NPUEval achieves a 26.85% score with GPT-4o, whereas a simple prompt achieves 4.25% with the same GPT-4o agent.

    “Figure 27 shows the prompt generated by GEPA with GPT-4o for NPU Keernel Generation, that achieves 26.85% score with the same same GPT-4o agent, that achieved just 4.25% with a simple prompt.”
    • Also stated in: § Domain-Specific Notes: · plain
      “Figure 27: GEPA generated prompt for NPUEval that achieves 26.85% score with the same GPT-4o agent, that achieved a 4.25% score.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § I COST VS. PERFORMANCE ANALYSIS FOR OPTIMIZED SYSTEMS

    GEPA's prompts are around 33% shorter than MIPROv2's prompts, while achieving higher performance.

    “Notably, GEPA's prompts are around 33% shorter than MIPROv2's prompts, while achieving higher performance.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given
    § 7 Conclusion

    GEPA shows superior sample efficiency compared to reinforcement learning (GRPO) while outperforming leading prompt optimizers (MIPROv2).

    “showing superior sample efficiency compared to reinforcement learning (GRPO), while outperforming leading prompt optimizers (MIPROv2).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § E.3 COSTS

    GEPA-Merge costs $67.

    “GEPA-Merge costs $67”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given
    § L EXAMPLES OF BEST PROMPTS FOR EVERY BENCHMARK

    GEPA's prompts only consist of the optimized instruction.

    “GEPA's prompts only consist of the optimized instruction”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given
    § L EXAMPLES OF BEST PROMPTS FOR EVERY BENCHMARK

    MIPROv2 optimized prompts contain up to 4 few-shot examples for each task.

    “MIPROv2 optimized prompts contain upto 4 few-shot examples for each task.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § Figure 6 caption

    Using a Pareto-based candidate selection strategy, GEPA generates a balanced search tree and finds a better performing program within the same budget.

    “using pareto-based candidate selection strategy, GEPA was able to generate a balanced search tree, finding a better performing program within the same budget.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § E.3 COSTS

    GEPA costs a total of $86.

    “GEPA costs a total of $86”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 5.2

    The learned adversarial instruction reduced pass@1 on AIME-2025 from 76% (clean prompt) to 10% (adversarial prompt), while preserving the nominal task description and the required answer-format directive.

    “The learned adversarial instruction reduced pass@1 on AIME-2025 from 76% (clean prompt) to 10% (ad- versarial prompt), despite preserving the nominal task description and the required answer-format directive.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § Observation 5

    GEPA+Merge can outperform GEPA by as much as 5%, providing an aggregate 2% additional improvement.

    “GEPA+Merge can outperform GEPA by as much as 5%, providing an aggregate 2% additional improvement over the already strong performance established by GEPA.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 5.1 NPU Kernels

    Applying GEPA to Sequential10 (without RAG) boosts kernel performance, with several generated kernels achieving up to 70% vector utilization and a mean of 30.52%.

    “applying GEPA to Sequential10 (without RAG) dramatically boosts kernel performance, with several generated kernels achieving up to 70% vector utilization and a mean of 30.52%.”
    • Also stated in: § Figure 7 caption · plain
      “GEPA with GPT-4o is able to generate kernels for AMD NPUs that achieve vector utilization rates as high as 70%, with a mean utilization score of 30.52%.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § Observation 6

    This transfer performance outperforms MIPROv2 (+5.64%), TextGrad (+6.11%), and Trace (+3.27%), even though those methods were optimized directly on the target GPT-4.1-Mini model.

    “Remarkably, this transfer performance outperforms strong baselines like MIPROv2 (+5.64%), TextGrad (+6.11%), and Trace (+3.27%), even though those methods were optimized directly on the target GPT-4.1-Mini model.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § Observation 4

    Instruction-optimized prompts are computationally cheaper and generalize better than few-shot demonstration prompts.

    “Instruction-optimized prompts are computationally cheaper and generalize better than few-shot demonstration prompts”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § E.4 OPTIMIZERS

    Overall, MIPROv2's rollouts ranged from a minimum of 2270 (for PUPA) to maximum of 6926 (for HoVer).

    “Overall, MIPROv2's rollouts ranged from a minimum of 2270 (for PUPA) to maximum of 6926 (for HoVer).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § Observation 5

    GEPA+Merge works especially well for GPT-4.1 Mini but leads to performance degradation when used with Qwen3 8B.

    “While in our analysis, we found GEPA+Merge works especially well for GPT-4.1 Mini, it lead to perfor- mance degradation when used with Qwen3 8B.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § E.3 COSTS

    Trace and TextGrad cost $172 in total.

    “Trace and TextGrad cost $172 in total”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 1 INTRODUCTION

    GEPA's Pareto-front diversification enables robust generalization and mitigates getting stuck in local minima.

    “This diversification enables robust generalization and mitigates getting stuck in local minima.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning
    § 3 GEPA: REFLECTIVE PROMPT EVOLUTION

    When paired with the final outcome, natural language traces provide substantial diagnostic value, allowing errors or successes to be traced back to specific module-level decisions.

    “When these traces are paired with the final outcome of the system (e.g., success or failure), they provide substantial diagnostic value, allowing practitioners to trace errors or successes back to specific decisions made at the module level.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § Figure 3

    GEPA's design decisions allow it to be highly sample-efficient while demonstrating strong generalization.

    “Overall, these design decisions allow GEPA to be highly sample-efficient while demonstrating strong generalization.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4 EVALUATION

    The shift is primarily attributed to recent advances in instruction-following and self-reflective abilities of LLMs, as well as GEPA's design choices.

    “We attribute this primarily to recent advances in the instruction-following and self-reflective abilities of LLMs, as well as the design choices in GEPA that capitalize on these improved capabilities.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 1 INTRODUCTION

    Reflective prompt evolution with language feedback enables improved sample efficiency and robust generalization, offering a practical approach to optimizing complex AI workflows in data- or budget-constrained environments.

    “These results highlight that reflective prompt evolution with language feedback enables improved sample efficiency and robust generalization, offering a practical approach to optimizing complex AI workflows in data- or budget-constrained environments.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisereasoning
    § 3 GEPA: REFLECTIVE PROMPT EVOLUTION

    Natural language traces generated during execution of a compound AI system offer rich visibility into the behavior and responsibilities of each module.

    “Natural language traces generated during the execution of a compound AI system offer rich visibility into the behavior and responsibilities of each module, as they capture the intermediate inferences and underlying reasoning steps.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data
    § 4 EVALUATION

    The study used a standard train/validation/test split with optimizers having full access to the train split and restricted access to validation instance content.

    “We adopt a standard train/validation/test split. Optimizers have full access to the train split, including text and labels, for program tuning. Although optimizers may monitor the performance of candidate parameters (like model checkpoints) by tracking scores on the validation set (to implement early stopping, for example), direct access to the content of validation instances is restricted.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning
    § E.4 OPTIMIZERS

    To ensure a fair comparison with MIPROv2, we align the computational budget between GEPA and MIPROv2 on a per-benchmark basis.

    “To ensure a fair comparison with MIPROv2, we align the computational budget between GEPA and MIPROv2 on a per-benchmark basis.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § 6 Related Work

    GEPA additionally uses domain-specific feedback for targeted mutations, achieving higher sample efficiency.

    “GEPA additionally uses domain-specific feedback for targeted mu- tations achieving higher sample efficiency.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data
    § 7 Conclusion

    GEPA shows promise as an inference-time search strategy, showing the ability to write code in challenging domains.

    “GEPA also shows promise as an inference-time search strategy, showing the ability to write code in challenging domains.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 5.2

    By finding universal, task-preserving distractors, adversarial prompt search reveals brittle instruction-following interactions and turns them into reusable stress tests and regression suites for continuous evaluation.

    “By finding universal, task-preserving distractors (e.g., trivia plus strict formatting), it reveals brittle instruction-following interactions and turns them into reusable stress tests and regression suites for continuous evaluation.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given
    § E.4 OPTIMIZERS

    We also provide both optimizer the same metric and feedback functions as GEPA.

    “We also provide both optimizer the same metric and feedback functions as GEPA”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given
    § E.4 OPTIMIZERS

    All optimization experiments were under the same rollout budget as MIPROv2 and GEPA.

    “All optimization experiments were under the same rollout budget as MIPROv2 and GEPA.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning
    § E.4 OPTIMIZERS

    While differences in proposal and validation procedures cause the exact budget usage by the systems to be slightly different, the discrepancy is always within 10.15%.

    “While differences in proposal and validation procedures cause the exact budget usage by the systems to be slightly different, the discrepancy is always within 10.15%.”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Interpretationown data
    § main text after Figure 6

    GEPA's Pareto-based method expands the search by considering all Pareto-optimal candidates, balancing exploration and exploitation and converging to a higher-performing solution within the same rollout budget.

    “the search by considering all Pareto-optimal candidates (all”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 6 Related Work

    Reinforcement learning can be sample-inefficient when rewards are slow to compute.

    “Feedback-driven improvement often uses reinforcement learning, such as majority voting signals (Zuo et al., 2025), but RL can be sample-inefficient when rewards are slow to compute.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zuo et al., 2025

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § D.1 MERGE: SYSTEM-AWARE CROSSOVER STRATEGY FOR COMPOUND AI OPTIMIZATION

    Merge will be helpful when there are candidates in the pool that learn complementary strategies.

    “Intuitively, merge will be helpful when there are candidates in the pool that learn complementary strategies.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given
    § 6 Related Work

    GEPA differs from prior prompt optimization methods by incorporating textual environment feedback, Pareto-aware search over candidates, and evolution strategies per submodule within an AI system.

    “GEPA leverages LLMs, but differs by incorporating textual environment feedback, Pareto-aware search over candidates, and evolution strategies per submodule within an AI system.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 6 Related Work

    Prompt optimization improves LLMs but often needs manual expertise.

    “Prompt optimization improves LLMs but often needs manual expertise; for instance, chain-of-thought prompting Wei et al. (2023).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Wei et al. (2023)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Study design and conductreasoning
    § 5.1

    Operating under a cache eliminates generation stochasticity (temperature based sampling), ensuring that observed improvements tie closely to inference scaling through prompt updates and GEPA's prompt exploration rather than to sampling stochasticity.

    “generation stochasticity (temperature based sampling) is eliminated by operating under a cache; this ensures that ob- served improvements tie closely to inference scaling through prompt updates and GEPA's diverse prompt exploration, rather than stochasticity in the model's sampling process.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § D.1 MERGE: SYSTEM-AWARE CROSSOVER STRATEGY FOR COMPOUND AI OPTIMIZATION

    These strict lineage conditions mean merge occurs sparsely.

    “These strict lineage conditions mean merge occurs sparsely.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning
    § 6 Related Work

    The Pareto-guided evolution lets GEPA explore diverse prompt/code/agent design strategies before converging to a robust, generalizable set.

    “The Pareto-guided evolution lets GEPA explore diverse prompt/code/agent design strategies before converging to a robust, generalizable set.”
    Quote in the paperNot a substantial claim
56 of 56

  • The paper relies on this without support.

    In many real-world scenarios, rollouts (invocations of Φ plus evaluation by µ) are often computationally, monetarily, or timewise expensive.

    “In many real-world scenarios, rollouts—concretely, invocations of Φ plus evaluation by µ—are often computationally, monetarily, or timewise expensive.”

  • The paper relies on this without support.

    Many evaluation metrics produce natural language traces before computing a scalar reward.

    “Many evaluation metrics apply rich strategies (e.g., code evaluation may involve compilation, execution, and profiling), producing natural language traces before computing a scalar reward.”

  • The paper relies on this without support.

    Many downstream LLM applications invoke expensive tool calls, have limited inference budget for sampling from the LLM itself, or cannot finetune the weights of the largest or best-performing LLMs.

    “many downstream LLM applications invoke expensive tool calls, have limited inference budget for sampling from the LLM itself, or simply cannot finetune the weights of the largest or best-performing LLMs.”

  • The paper relies on this without support.

    The prompt size of the optimized system plays an important role in determining the downstream cost of using the optimized system.

    “The prompt size of the optimized system plays an important role in determining the downstream cost of using the optimized system.”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Existing text optimizers are poorly matched to harness engineering because they compress feedback too aggressively: they are memoryless, condition only on scalar scores, or restrict feedback to short templates or summaries.

    “existing text optimizers are poorly matched to this setting because they compress feedback too aggressively: they are memoryless, condition only on scalar scores, or restrict feedback to short templates or summaries.”

    From Meta-Harness: End-to-End Optimization of Model Harnesses (2026) · cites this paper as [1]

    The Related Work section says several prompt optimizers largely rely on global rewards, while GEPA uses textual environment feedback; the discussion of evaluation traces also explains what is lost when diagnostic information is reduced to a scalar reward. But the paper does not establish that existing optimizers are memoryless or restrict feedback to short templates or summaries, and it does not assess their fit for harness engineering. It explicitly describes TextGrad as using textual feedback. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • An agent harness turns a stateless model into a functional agent capable of long-running task execution.

    “An agent harness refers to the software layer that surrounds an LLM with tools, APIs, sandboxes, memory, validators, permission boundaries, execution loops, and feedback channels, thereby turning a stateless model into a functional agent capable of long-running task execution [17, 18, 19, 20, 21, 22, 23].”

    From Code as Agent Harness (2026) · cites this paper as [18]

    The paper’s system formalization (p. 3) describes control-flow logic that orchestrates LLM modules, passes outputs between them, uses tool APIs, and can invoke modules repeatedly. This supports the orchestration part of the sentence, but the paper does not describe an agent harness turning a stateless model into an agent capable of long-running tasks, or establish the full list of harness components. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • Meta-Harness is closely related to methods such as ProTeGi, TextGrad, OPRO, GEPA, AlphaEvolve/OpenEvolve, and Feedback Descent, which iteratively improve prompts or other text artifacts using feedback from prior attempts.

    “Meta-Harness is also closely related to methods such as ProTeGi, TextGrad, OPRO, GEPA, AlphaEvolve/OpenEvolve, and Feedback Descent, which iteratively improve prompts or other text artifacts using feedback from prior attempts [38; 31; 53; 51; 1; 35; 43; 26].”

    From Meta-Harness: End-to-End Optimization of Model Harnesses (2026) · cites this paper as [1]

  • GEPA optimizes textual artifacts by using reflective feedback to guide an evolutionary process.

    “GEPA [1] optimizes textual artifacts by using reflective feedback to guide an evolutionary process.”

    From HarnessOpt-Bench: Evaluating LLMs at Harness Optimization (2026) · cites this paper as [1]

  • Existing approaches to harness optimization differ along two axes: which parts of the harness they may modify, and the role assigned to the LLM during search.

    “Existing approaches to harness optimization differ along two axes: which parts of the harness they may modify, and the role assigned to the LLM during search.”

    From HarnessOpt-Bench: Evaluating LLMs at Harness Optimization (2026) · cites this paper as [1]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Prompt-length reduction is described in two incompatible ways

    Minor

    The appendix says GEPA prompts are around 33% shorter than MIPROv2 prompts, but Figure 17 says they are less than 33% of MIPROv2's size. Being 33% shorter would mean being around 67% of the size.

  • GEPA and GEPA+Merge aggregate gains are assigned to the wrong variants

    Minor

    The observation assigns +13.33% to GEPA and +12.19% to GEPA+Merge, while Table 2 assigns those gains in the opposite order.

  • Table caption overstates GEPA+Merge's performance against GRPO

    Minor

    The caption says GEPA+Merge beats GRPO on every benchmark except AIME. In the table, GEPA+Merge scores 28.23 versus GRPO's 35.88 on IFBench and 86.26 versus 86.66 on PUPA.

3 of 3

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 12

  1. Eshaan Agarwal, Joykirat Singh, Vivek Dani, Raghav Magazine, Tanuja Ganu, and Akshay Nambi. Promptwizard: Task-aware prompt optimization framework, 2024. URL https://arxiv.org/abs/ 2405.18369.
  2. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022.
  3. Omar Khattab, Keshav Santhanam, Xiang Lisa Li, David Hall, Percy Liang, Christopher Potts, and Matei Zaharia. Demonstrate-search-predict: Composing retrieval and language models for knowledge-intensive NLP. arXiv preprint arXiv:2212.14024, 2022. http://arxiv.org/abs/2503.19470 http://arxiv.org/abs/2503.19470 http://arxiv.org/abs/2505.11595 https://arxiv.org/abs/2505.14398 https://arxiv.org/abs/2505.14398 https://arxiv.org/abs/2406.16218 https://arxiv.org/abs/2411.14251 https://arxiv.org/abs/2411.14251 https://openreview.net/forum?id=ZG3RaNIsO8 https://openreview.net/forum?id=ZG3RaNIsO8 http://arxiv.org/abs/2506.20629 https://aclanthology.org/2020.findings-emnlp.309/ http://arxiv.org/abs/2503.09516 https://arxiv.org/abs/2507.14403 · arXiv
  4. Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan A, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. DSPy: Compiling declarative language model calls into state-of-the-art pipelines. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview. net/forum?id=sY5N0zY5Od.
  5. Nathan Lambert. Policy gradient algorithms. In RLHF Book: Reinforcement Learning from Human Feed- back, chapter 11. RLHF Book, 2025. URL https://rlhfbook.com/c/11-policy-gradients.html. Accessed July 16, 2025.
  6. Jean-Baptiste Mouret and Jeff Clune. Illuminating search spaces by mapping elites, 2015. URL https: //arxiv.org/abs/1504.04909. https://openreview.net/forum?id=sY5N0zY5Od https://openreview.net/forum?id=sY5N0zY5Od https://rlhfbook.com/c/11-policy-gradients.html https://aclanthology.org/2025.naacl-long.173/ https://aclanthology.org/2025.naacl-long.173/ http://arxiv.org/abs/2506.03487 http://arxiv.org/abs/2506.03487 http://arxiv.org/abs/2506.05154 https://arxiv.org/abs/2410.16392 https://proceedings.neurips.cc/paper_files/paper/2023/file/91edff07232fb1b55a505a9e9f6c0ff3-Paper-Conference.pdf https://proceedings.neurips.cc/paper_files/paper/2023/file/91edff07232fb1b55a505a9e9f6c0ff3-Paper-Conference.pdf https://arxiv.org/abs/2410.05362 https://arxiv.org/abs/1504.04909 https://arxiv.org/abs/1504.04909 · arXiv
  7. Jon Saad-Falcon, Adrian Gamarra Lafuente, Shlok Natarajan, Nahum Maru, Hristo Todorov, Etash Guha, E. Kelly Buchanan, Mayee Chen, Neel Guha, Christopher Ré, and Azalia Mirhoseini. Archon: An ar- chitecture search framework for inference-time techniques, 2025. URL https://arxiv.org/abs/2409. 15254.
  8. Zeyang Sha, Shiwen Cui, and Weiqiang Wang. SEM: Reinforcement Learning for Search-Efficient Large Language Models, May 2025. URL http://arxiv.org/abs/2505.07903. arXiv:2505.07903 [cs]. https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/AlphaEvolve.pdf https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/AlphaEvolve.pdf https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/AlphaEvolve.pdf https://openai.com/index/gpt-4-1/ https://openai.com/index/gpt-4-1/ https://aclanthology.org/2024.emnlp-main.525/ https://aclanthology.org/2024.emnlp-main.525/ https://openreview.net/forum?id=k6V4jb8jkX http://arxiv.org/abs/2506.09942 http://arxiv.org/abs/2506.09942 https://aclanthology.org/2023.emnlp-main.494/ https://huggingface.co/datasets/allenai/IF_multi_constraints_upto5 https://huggingface.co/datasets/allenai/IF_multi_constraints_upto5 https://arxiv.org/abs/2507.02833 https://arxiv.org/abs/2507.02833 https://arxiv.org/abs/2409.15254 https://arxiv.org/abs/2409.15254 http://arxiv.org/abs/2505.07903 · arXiv
  9. Hongru Wang, Cheng Qian, Wanjun Zhong, Xiusi Chen, Jiahao Qiu, Shijue Huang, Bowen Jin, Mengdi Wang, Kam-Fai Wong, and Heng Ji. Acting Less is Reasoning More! Teaching Model to Act Efficiently, May 2025a. URL http://arxiv.org/abs/2504.14870. arXiv:2504.14870 [cs]. https://arxiv.org/abs/2402.03300 https://github.com/codelion/openevolve https://github.com/codelion/openevolve http://arxiv.org/abs/2505.16483 http://arxiv.org/abs/2403.10704 http://arxiv.org/abs/2403.10704 http://arxiv.org/abs/2503.05592 https://arxiv.org/abs/2407.10930 http://arxiv.org/abs/2505.12886 http://arxiv.org/abs/2505.12886 https://arxiv.org/abs/2504.07952 https://arxiv.org/abs/2502.20315 https://arxiv.org/abs/2502.20315 https://huggingface.co/Qwen/Qwen3-8B http://arxiv.org/abs/2408.11857 https://proceedings.neurips.cc/paper_files/paper/2024/hash/6b031defd145b02bed031093d8797bb3-Abstract-Conference.html https://proceedings.neurips.cc/paper_files/paper/2024/hash/6b031defd145b02bed031093d8797bb3-Abstract-Conference.html http://arxiv.org/abs/2504.14870 · arXiv
  10. Shirley Wu, Parth Sarthi, Shiyu Zhao, Aaron Lee, Herumb Shandilya, Adrian Mladenic Grobelnik, Nurendra Choudhary, Eddie Huang, Karthik Subbian, Linjun Zhang, Diyi Yang, James Zou, and Jure Leskovec. Optimas: Optimizing compound ai systems with globally aligned local rewards, 2025a. URL https: //arxiv.org/abs/2507.03041.
  11. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. Re- act: Synergizing reasoning and acting in language models, 2023. URL https://arxiv.org/abs/2210. 03629. http://arxiv.org/abs/2504.15777 https://arxiv.org/abs/2409.07429 https://arxiv.org/abs/2504.06821 https://arxiv.org/abs/2201.11903 https://arxiv.org/abs/2406.19314 https://arxiv.org/abs/2507.03041 https://arxiv.org/abs/2507.03041 http://arxiv.org/abs/2505.17086 https://arxiv.org/abs/2506.10341 http://arxiv.org/abs/2504.13818 https://arxiv.org/abs/2505.09388 https://arxiv.org/abs/2505.09388 https://arxiv.org/abs/2309.03409 https://arxiv.org/abs/2210.03629 https://arxiv.org/abs/2210.03629 · arXiv
  12. Yuxin Zuo, Kaiyan Zhang, Li Sheng, Shang Qu, Ganqu Cui, Xuekai Zhu, Haozhan Li, Yuchen Zhang, Xinwei Long, Ermo Hua, Biqing Qi, Youbang Sun, Zhiyuan Ma, Lifan Yuan, Ning Ding, and Bowen Zhou. Ttrl: Test-time reinforcement learning, 2025. URL https://arxiv.org/abs/2504.16084. https://doi.org/10.1145/3689031.3696086 http://arxiv.org/abs/2505.18454 http://arxiv.org/abs/2505.18454 http://arxiv.org/abs/2505.17447 https://aclanthology.org/2024.acl-long.292/ http://arxiv.org/abs/2310.11523 http://arxiv.org/abs/2504.12216 https://arxiv.org/abs/2508.04660 https://arxiv.org/abs/2504.16084 · DOI

Cited by

Sorted by citations

Authors
Agrawal, L. A., Tan, S., Soylu, D., Ziems, N., Khare, R., Opsahl-Ong, K., Singhvi, A., Shandilya, H., Ryan, M. J., Jiang, M., Potts, C., Sen, K., Dimakis, A. G., Stoica, I., Klein, D., Zaharia, M., & Khattab, O.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2507.19457
arXiv
2507.19457
OpenAlex
W4416411532
Semantic Scholar
f1c0aad419cd942d5a135b47fcce58d101dbb3bd
Cited by
7
References
0
Citations by year
2026: 7
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
  • Zenodo (CERN European Organization for Nuclear Research) (cc-by)
  • Zenodo (CERN European Organization for Nuclear Research)
Funders
U.S. Department of Energy (DESC0021982)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Keywords
prompt optimization, large language models, genetic algorithm, Pareto front, Group Relative Policy Optimization, rollout, program optimization, interpretability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-19.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-19.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review