The DOI matches its DataCite record; 45 of the 45 references it cites were found in a registry or an index; 10 of the 15 sources its claims cite were checked for support.
ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution
Open accessAbstract
We introduce ShinkaEvolve: a new open-source framework leveraging large language models (LLMs) to advance scientific discovery with state-of-the-art performance and unprecedented efficiency. Recent advances in scaling inference time compute of LLMs have enabled significant progress in generalized scientific discovery. These approaches rely on evolutionary agentic harnesses that leverage LLMs as mutation operators to generate candidate solutions. However, current code evolution methods suffer from critical limitations: they are sample inefficient, requiring thousands of samples to identify effective solutions, and remain closed-source, hindering broad adoption and extension. ShinkaEvolve addresses these limitations, introducing three key innovations: a parent sampling technique balancing exploration and exploitation, code novelty rejection-sampling for efficient search space exploration, and a bandit-based LLM ensemble selection strategy. We evaluate ShinkaEvolve across diverse tasks, demonstrating consistent improvements in sample efficiency and solution quality. ShinkaEvolve discovers a new state-of-the-art circle packing solution using only 150 samples, designs high-performing agentic harnesses for AIME mathematical reasoning tasks, identifies improvements to ALE-Bench competitive programming solutions, and discovers novel mixture-of-expert load balancing loss functions that illuminate the space of optimization strategies. Our results demonstrate that ShinkaEvolve achieves broad applicability with exceptional sample efficiency. By providing open-source accessibility and cost-efficiency, this work democratizes open-ended discovery across diverse computational problems.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
ShinkaEvolve is an open-source LLM-driven evolutionary program search framework with three innovations—weighted parent sampling, code novelty rejection sampling, and bandit-based LLM ensemble selection—designed for sample efficiency. It reports state-of-the-art or improved results across circle packing, AIME agent scaffolds, ALE-Bench, and mixture-of-experts load-balancing loss discovery, including a new circle-packing solution in 150 evaluations.
Key findings
- Circle packing: ShinkaEvolve discovers a new state-of-the-art solution using only 150 samples/evaluations, outperforming AlphaEvolve; the discovered solution has sum of radii 2.635983099011548 under relaxed verification and 2.6359828390115476 after exact adjustment, with an exact-verification replication score of 2.63597770931127.
- AIME 2024: the evolved agent scaffold uses 7 LLM calls (3 generation + 3 review + 1 synthesis) within the 10-call limit, reaches a Pareto frontier between performance and query budget, generalizes to 2025 AIME problems, and transfers to gpt-4.1-mini, gpt-4.1, and o4-mini.
- ALE-Bench LITE: ShinkaEvolve improves ALE-Agent solutions by approximately 2.3% on average across 10 tasks; on ahc039, the combined solution would rank 2nd on the AtCoder leaderboard (3140 performance) versus ALE-Agent’s 5th (2880).
- MoE load balancing: the discovered loss improves over the global-batch LBL across λ ∈ {0.001, 0.01, 0.1} in a 2.7B-parameter MoE; ablations show weighted parent sampling outperforms random search and hill climbing, bandit-based LLM ensembling outperforms single and fixed ensembles, and embedding-based rejection sampling strongly outperforms no rejection sampling.
Methods
- ShinkaEvolve maintains an archive of island subpopulations, samples parents and inspirations, mutates programs via diff edits, full rewrites, and crossover, rejects near-duplicates using embedding cosine similarity above 0.95 with an optional LLM novelty judge, executes programs for scalar fitness plus public metrics and textual feedback, adapts LLM sampling via UCB1, and periodically appends meta-scratchpad insights to mutation prompts.
- It is evaluated on four domains: circle packing (26 circles, 150 generations), AIME 2024 (30 problems, 75 generations, max 10 LLM queries, gpt-4.1-nano, 3 runs per candidate), ALE-Bench LITE (10 AtCoder heuristic tasks, 50 generations, C++), and MoE LBL discovery (556M-parameter/82M-active small MoE with 64 experts and 8 active, 2B fineweb tokens, 30 iterations; 2.7B-parameter/404M-active large MoE, ~30B tokens, seven benchmarks).
- Ablations on circle packing compare parent selection strategies (weighted sampling vs. random search and hill climbing), LLM ensembling (single LLM vs. fixed ensemble vs. bandit-based ensemble), and novelty rejection sampling (none vs. embedding-based vs. embedding plus LLM-as-a-novelty-judge).
Limitations
- Stated in the paper: the implementation uses fixed configurations with limited automatic control over exploration-exploitation balance; task specification requires manual human expertise for objective functions and evaluation; and the framework is constrained to problems with well-defined numerical objectives.
- Evident in the text: ALE-Bench modifications often stayed algorithmically close to the ALE-Agent initialization, indicating potential overfitting to the initialization solution; the MoE architecture used for evolving and testing the LBL was quite similar and the training budget was limited; API costs from large-scale LLM usage could create economic barriers; and the main circle-packing run used relaxed verification with 1×10⁻⁶ slack before exact post-processing.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
31 Own result · 20 Own interpretation · 10 No basis given · 9 Own method · 5 Partly supported · 3 Supported · 2 Sources not found · accuracy not yet measured
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ShinkaEvolve discovers a new stateof-the-art circle packing solution using only 150 samples”
- Also stated in: § 4.1. Circle Packing: Reproducing & Improving AlphaEvolve Results · p. 6 · plain
“ShinkaEvolve outperforms AlphaEvolve's solution within less than 150 program evaluations.”
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4.1. Circle Packing: Reproducing & Improving AlphaEvolve Results · p. 6 · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“on one task, ahc039 , the combination of ShinkaEvolve with ALE-Agent resulted in the second place submission on the AtCoder leaderboard if they had participated.”
- Also stated in: § 4.3. ALE-Bench: Evolving Programs for Combinatorial Optimization · p. 8 · plain
“The base solution by ALE-Agent applies simulated annealing with kd-tree data structure (5th, 2880 performance). ShinkaEvolve further improved the solution (2nd, 3140 performance)”
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4.3. ALE-Bench: Evolving Programs for Combinatorial Optimization · p. 8 · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we observe that the proposed changes by ShinkaEvolve remained algorithmically close to the original ALE-Agent's initialization solution.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“designs high-performing agentic harnesses for AIME mathematical reasoning tasks”
- Also stated in: § 4.2. AIME: Evolving Agent Scaffolds for Math Reasoning · p. 7 · plain
“ShinkaEvolve discovers scaffold designs that significantly outperform hand-designed baselines, including simple single-query agents and sophisticated majority-voting approaches.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4.2. AIME: Evolving Agent Scaffolds for Math Reasoning · p. 7 · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“identifies improvements to ALE-Bench competitive programming solutions”
- Also stated in: § 4.3. ALE-Bench: Evolving Programs for Combinatorial Optimization · p. 8 · plain
“ShinkaEvolve is able to improve the solutions discovered by ALE-Agent by approximately 2.3% across the 10 tasks on average (Figure 7).”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4.3. ALE-Bench: Evolving Programs for Combinatorial Optimization · p. 8 · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Bandit-based LLM ensembling slightly improves the performance over a fixed uniform ensemble distribution.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While the fixed ensemble shows moderate improvements over single LLM usage”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“the adaptive bandit strategy achieves the highest performance”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Consistent with our previous analysis, ShinkaEvolve's LBL appears to improve from the original LBL across both axes.”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, the consistent generalization results across training budgets and coefficients λ provide an optimistic outlook for future extensions to much longer training regimes, where even small efficiency gains could scale to significant cost savings.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 22
- It may state two separate claims as one. 20
- The quote only partly states this claim. 2
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, due to the non-differentiability of the top𝐾 expert selection operation, it is critical to provide the router with an auxiliary load balancing loss (LBL).”
What the sources say
- Partly supportedcites Shazeer et al. (2017) sourceExperimental — accuracy not yet measured
Section 4 (pp. 5–6) introduces auxiliary losses to counter self-reinforcing expert imbalance and unequal loads. Appendix A (p. 13) says discrete expert counts cannot be used directly in backpropagation, so the load loss uses a smooth estimator. But Section 2.1 (p. 4) says selected gate values have nonzero derivatives and that discontinuities have not been a problem in practice; the paper does not say top-K non-differentiability makes an auxiliary load-balancing loss critical. (LLM, conf=0.88)
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“We follow the evaluation protocol of ALE-Agent (Imajuku et al., 2025) and use the score calculated on the public test cases as the fitness function.”
What the sources say
- Supportedcites Imajuku et al., 2025 source
Appendix B.2 (p. 28) describes ALE-Agent selecting candidates using a priority score calculated from their performance on a fixed suite of 50 test cases. That score combines acceptance ratio and problem-specific score; it is not necessarily the raw contest score alone. The paper separately uses private test cases for final evaluation (Appendix A.2.2, p. 26). (LLM, conf=0.87)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The ALE-Bench benchmark (Imajuku et al., 2025) is a collection of heuristic programming problems previously used in competitive programming contests (AtCoder).”
What the sources say
- Supportedcites Imajuku et al., 2025 source
The Introduction (p. 2) says ALE-Bench collects past tasks from the AtCoder Heuristic Contest. Figure 1 (p. 1) likewise describes it as collecting past contest tasks, and Section 3.1 (p. 3) identifies AHC as an AtCoder score-based algorithmic competition. (LLM, conf=0.95)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Minor design variations have been shown to significantly affect both efficiency and specialization ability (Dai et al., 2024; Jiang et al., 2024; Liu et al., 2024b; Qiu et al., 2025; Team, 2024).”
What the sources say
- Partly supportedcites Dai et al., 2024 sourceExperimental — accuracy not yet measured
Sections 3.1 and 3.2 describe how fine-grained expert segmentation and shared expert isolation can improve specialization and parameter efficiency; Section 4.4 reports improved overall performance in ablations. The paper does not characterize these architectural changes as minor or show a significant change in computational efficiency—the comparisons hold computational cost or activated parameters constant. (LLM, conf=0.82)
- Partly supportedcites Jiang et al., 2024 sourceExperimental — accuracy not yet measured
The Size and Efficiency section reports that Mixtral uses fewer active parameters per token than Llama 2 70B while outperforming it across most categories, supporting an efficiency difference between architectures. But the paper does not test minor design variations, and its Routing analysis (§5; page unavailable) finds no obvious topic-based expert specialization. It therefore does not show that minor variations significantly affect both efficiency and specialization ability. (LLM, conf=0.82)
- Not matchedcites Liu et al., 2024b
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“One of the most widely adopted designs is the 'global-batch' LBL introduced by Shazeer et al. (2017), which underpins several state-of-the-art open models such as Qwen 3 (Yang et al., 2025).”
What the sources say
- Partly supportedcites Shazeer et al. (2017) sourceExperimental — accuracy not yet measured
Section 4 (pp. 5–6) introduces batchwise expert-balancing losses, and Section 3.1 (p. 4) describes combining batches across devices for the MoE layer. This supports identifying Shazeer et al. with a batch-based load-balancing design, but the paper does not establish that it is among the most widely adopted designs. The Qwen 3 example has a separate citation and is not attributed to this source here. (LLM, conf=0.82)
- Partly supportedcites Yang et al. (2025) sourceExperimental — accuracy not yet measured
Section 2 (Architecture; page not provided) says the Qwen3 MoE models adopt a global-batch load balancing loss. It does not establish that the design is widely adopted or used by several other models, and it cites Qiu et al. (2025), not Shazeer et al. (2017), for that loss. (LLM, conf=0.88)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Devising an effective load balancing loss that simultaneously encourages efficiency and expert specialization, without hindering expressivity, remains an open design challenge that has driven much of the recent progress in MoEs (Dai et al., 2024; Du et al., 2022; Fedus et al., 2022; Muennighoff et al., 2024; Qiu et al., 2025; Shazeer et al., 2017; Xue et al., 2024; Zoph et al., 2022).”
What the sources say
- Partly supportedcites Dai et al., 2024 sourceExperimental — accuracy not yet measured
Section 3.3, “Load Balance Consideration” (page not provided), says load imbalance can cause routing collapse and computation bottlenecks, while excessive balance constraints compromise model performance. It therefore supports a tradeoff in designing load-balancing losses. The paper does not say that such a loss itself encourages expert specialization or that this specific design challenge has driven much of the recent progress in MoEs; it pursues specialization primarily through architectural strategies. (LLM, conf=0.85)
- Not matchedcites Du et al., 2022
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites Fedus et al., 2022
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The LBL prevents collapse toward uneven token distributions and under-specialized experts.”
What the sources say
- Partly supportedcites Shazeer et al. (2017) sourceExperimental — accuracy not yet measured
Section 4 (pp. 5–6) describes a self-reinforcing imbalance in expert selection and says the load-balancing loss encourages experts to receive similar numbers of examples. It also says experts specialize, but does not establish that the load-balancing loss prevents under-specialization; its balancing constraint is soft rather than a guarantee against collapse. (LLM, conf=0.85)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“To ensure the robustness of our results, we additionally validated our solutions using AlphaEvolve 's (Novikov et al., 2025) exact verification code.”
What the sources say
- Supportedcites Novikov et al., 2025 source
Appendix B, p. 38, states that verification code for its reported mathematical constructions is available in the accompanying Google Colab. This supports the citation’s identification of AlphaEvolve’s verification code; the source does not independently document the citing authors’ validation step. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Cross-LLM model transfer experiments validate robustness, with successful adaptation to gpt-4.1-mini , gpt-4.1 , and o4-mini demonstrating that discovered architectures capture generalizable strategies rather than model-specific optimizations (Figure 6, right).”
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ShinkaEvolve was evaluated over 150 evolutionary generations, demonstrating remarkable sample efficiency compared to existing approaches that typically require thousands of evaluations (Figure 1).”
boostedcorrelationalSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Existing code evolution methods are sample inefficient because they use naive exploration strategies that fail to leverage accumulated knowledge from previous generations.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“discovers novel mixture-of-expert load balancing loss functions that illuminate the space of optimization strategies.”
- Also stated in: § 4.4. LLM Training: Evolving Losses for Balanced and Effective Experts · p. 9 · boosted
“showing that ShinkaEvolve's new loss achieves a consistent edge across our training configurations, growing larger with the value of the 𝜆 coefficient.”
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
- Also stated in: § 4.4. LLM Training: Evolving Losses for Balanced and Effective Experts · p. 9 · boosted
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The search reveals a Pareto frontier between efficiency and performance (Figure 6, left), with 7 LLM queries yielding maximum performance while an alternative scaffold achieves comparable results using the full 10query budget.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The evolved algorithm (Section C.1) combines three key innovations: (1) a sophisticated initialization strategy that places circles in a structured goldenangle spiral pattern with strategic corner and edge positioning, (2) a hybrid optimization approach integrating SLSQP gradient-based refinement with simulated annealing for global exploration, and (3) intelligent perturbation mechanisms that”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Evaluating on 2023 and 2025 AIME problems shows different transfer patterns (Figure 6, middle): smaller improvements on 2023 problems suggest potential saturation due to training data contamination, while larger gains on 2025 problems indicate successful generalization to recent, unseen challenges.”
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Hill climbing shows strong initial performance but plateaus quickly”
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found that our discovered solution can be made trivially exact by reducing each circle's radius by 1 × 10 -8 , demonstrating the high precision of our approach.”
boosteddirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Weighted sampling consistently outperforms both random search and hill climbing across all tasks.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The discovery of the solution requires more samples to be evaluated.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The bandit-based LLM ensemble significantly outperforms both single LLM and fixed ensemble approaches.”
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ShinkaEvolve achieves state-of-the-art results across four domains: circle packing with 150 evaluations (orders of magnitude improvement), sophisticated AIME reasoning scaffolds, ALE-Bench algorithmic improvements, and novel mixture-of-expert load balancing.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Weighted parent sampling outperforms random search and hill climbing.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We evaluate ShinkaEvolve across diverse tasks, demonstrating consistent improvements in sample efficiency and solution quality.”
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Existing code evolution methods are sample inefficient because they use naive exploration strategies that fail to leverage accumulated knowledge from previous generations.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our results demonstrate that ShinkaEvolve achieves broad applicability with exceptional sample efficiency.”
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
- Rests on an assumption: Existing code evolution methods are sample inefficient because they use naive exploration strategies that fail to leverage accumulated knowledge from previous generations.
In the paper
“For the main ShinkaEvolve run presented in the paper, we employed the verification script provided by OpenEvolve (Sharma, 2025), which allows for 1 × 10 -6 numerical slack.”
What the sources say
- Not matchedcites Sharma, 2025
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
In the paper
“After evolution, we evaluate the discovered scaffold designs on 2023 and 2025 AIME problems (AIM, 2023, 2025) to assess generalization as well as robustness to different base agent language models.”
What the sources say
- Not matchedcites AIM, 2023, 2025
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This grounding provides ShinkaEvolve a two-fold search objective: minimize CE while improving load balance.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, ShinkaEvolve's new term can be seen as a safety net that adaptively activates and vanishes once an expert crosses the floor, providing dead experts and avoiding over-regularizing well-balanced layers.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“To avoid local noise affecting the cross-entropy calculations, we average it over the last 10M tokens.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally and to combat stochasticity in LLM queries, we evaluated each candidate evaluated across three independent runs on the complete question set.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Task specification requires manual human expertise for objective functions and evaluation.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Given the expense of pretraining, we run ShinkaEvolve for only 30 iterations, focusing on gpt-4.1 , gemini-2.5-pro , and claude-sonnet-4 .”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Our implementation uses fixed configurations with limited automatic control over exploration-exploitation balance, which may vary across domains.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ShinkaEvolve's open-source release further democratizes advanced evolutionary optimization, making it accessible to researchers and practitioners previously lacking access to proprietary systems.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This formulation encourages token usage across experts to align with the router's average soft assignment probabilities.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, API costs from large-scale LLM usage could create economic barriers, potentially constraining democratization goals.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“However, we also note that the architecture used for evolving and testing the employed LBL was quite similar, and the training budget was still limited.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“However, we also note that the architecture used for evolving and testing the employed LBL was quite similar, and the training budget was still limited.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Intuitively , this term softly affects the MoE router of any layer, with experts getting allocated a fraction of tokens less than 𝜏 . The multiplier 𝑠 ( 𝑃ℓ ) makes this push stronger when the layer's routing entropy 𝐻 ( 𝑃ℓ ) is low and the router is concentrating on fewer dominating experts.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper relies on this without support.
Existing code evolution methods are sample inefficient because they use naive exploration strategies that fail to leverage accumulated knowledge from previous generations.
“This inefficiency stems from naive exploration strategies that fail to effectively leverage accumulated knowledge from previous generations.”
The paper relies on this without support.
Most leading code evolution systems remain closed-source, creating barriers to reproducibility and limiting community-driven improvements.
“Additionally, most leading systems remain closed-source, creating barriers to reproducibility and limiting community-driven improvements.”
The paper relies on this without support.
The performance of different LLMs to propose mutations varies across problem domains and based on the current state of the sampled archive parents and inspiration programs.
“The performance of different LLMs to propose mutations can vary across problem domains and based on the current state of the sampled archive parents and inspiration programs.”
The paper relies on this without support.
The program archive is non-stationary.
“to account for the non-stationarity of the program archive”
The paper relies on this without support.
Similar code variants can waste computational resources without advancing the search frontier.
“Similar code variants can waste computational resources without advancing the search frontier.”
The paper relies on this without support.
LLM-driven scientific discovery has critical limitations in sample efficiency and open-source accessibility.
“addressing critical limitations in LLM-driven scientific discovery through improved sample efficiency and open-source accessibility.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ B.4. Mixture-of-Experts Load Balancing Loss · p. 26
“𝜏 = 0 . 064 / 𝑁𝐸 as a minimum usage threshold”
§ C.4. Mixture-of-Experts Load Balancing Loss · p. 52
“min_threshold = 0.01 * (64.0 / num_experts)”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.4. LLM Training: Evolving Losses for Balanced and Effective Experts · p. 9
“in terms of final perplexity (Figure 8, left) and downstream task performance (Figure 8, center)”
§ 4.4. LLM Training: Evolving Losses for Balanced and Effective Experts · p. 9
“Figure 8 | ShinkaEvolve for discovering Mixture-of-Experts Load Balancing Loss Functions. Left: Downstream task performance across seven benchmarks. Middle: Final perplexity across different missroute fractions.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ C.4. Mixture-of-Experts Load Balancing Loss · p. 52
“attention_mask: Optional[torch.Tensor] = None”
§ C.4. Mixture-of-Experts Load Balancing Loss · p. 52
“bsz, seqlen = attention_mask.shape n_tokens = bsz * seqlen”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.3. ALE-Bench: Evolving Programs for Combinatorial Optimization · p. 8
“ultimately replacing simulated annealing with a more focused optimization combining greedy moves and targeted local search.”
§ C.3.2. ALE-Bench LITE task: ahc025 · p. 49
“// Targeted Simulated Annealing if (D > 1) {”
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 4
- Nan Du, Yanping Huang, Andrew M Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, et al. Glam: Efficient scaling of language models with mixture-of-experts. In International conference on machine learning , pages 5547-5569. PMLR, 2022.
- Joel Lehman, Kenneth O Stanley, et al. Exploiting open-endedness to solve problems through the search for novelty. In ALIFE , pages 329-336, 2008.
- Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. Competition-level code generation with alphacode. Science , 378(6624):1092-1097, 2022.
- Chris Lu, Samuel Holt, Claudio Fanconi, Alex James Chan, Jakob Nicolaus Foerster, Mihaela van der Schaar, and Robert Tjarko Lange. Discovering preference optimization algorithms with and for large language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems , 2024a. URL https://openreview.net/forum?id=erjQDJ0z9L .
Cited by
Sorted by citations
Items 101–196 of 196
Vulcan: Instance-specialized, Verifiable Systems Heuristics Through LLM-driven Search
Dwivedula, R., Saxena, D., Yadalam, S., et al. (2025) · doi:10.1145/3842654.3848582 · Cited by 1
“Vulcan is agnostic to the code-generation approach; for our case studies (§7), we use two approaches (ShinkaEvolve [51], OpenEvolve [87]).”
EvoX: Meta-Evolution for Automated Discovery
Liu, S., Agarwal, S., Maheswaran, M., Cemri, M., Li, Z., Mang, Q., Naren, A., Boneh, E., Cheng, A., Pan, M. Z., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.23413 · Cited by 1
MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution
Wang, Z., Yan, M., Bi, J., Yan, S., Tresp, V., & Ma, Y. (2026) · arXiv · doi:10.48550/arxiv.2607.05297
“Population-based program search makes the search object more executable by pairing behavioural archives with verifiers and recent extensions to coding agents and open-ended evolution (Novikov et al., 2025; Lange et al., 2025; Wang et al., 2025; He et al., 2026; Bi et al., 2026).”
HarnessOpt-Bench: Evaluating LLMs at Harness Optimization
Ursekar, V., Shanker, A., Maurya, Y., Yasser, S., Kalmath, V. S., Chatrath, V., & Xue, Y. (2026) · arXiv · doi:10.48550/arxiv.2608.06301
“ShinkaEvolve [11] and GEPA [1] use it as a mutation operator inside a larger evolutionary procedure, Each candidate is evaluated in isolated, unprivileged sandboxes, and every model call, both the optimizer’s own and the evaluations’, passes through a gateway that enforces model allow-lists and…”
Liu, Z., Wu, J., Peng, R., Ji, Y., Li, D., Jiang, R., & Zhang, Y. (2026) · arXiv · doi:10.48550/arxiv.2608.03501
DeepScientist: Advancing Frontier-Pushing Scientific Findings Progressively
Weng, Y., Zhu, M., Xie, Q., Sun, Q., Lin, Z., Liu, S., & Zhang, Y. (2025) · arXiv · doi:10.48550/arxiv.2509.26603
“…function: Unlike previously studied tasks such as early-stage machine learning (Schmidgall et al., 2025a), algorithmic design (Novikov et al., 2025; Lange et al., 2025), or scientific software development (Aygün et al., 2025), A defining characteristic of frontier scientific discovery is that each…”
AlphaResearch: Accelerating New Algorithm Discovery with Language Models
Yu, Z., Feng, K., Zhao, Y., He, S., Zhang, X.-P., & Cohan, A. (2025) · arXiv · doi:10.48550/arxiv.2511.08522
“ShinkaEvolve (Lange et al., 2025) advances this direction by introducing a more sample-efficient evolutionary framework for open-ended program evolution.”
Automated Discovery Has No Universally Superior Harness
Gupta, A., Lei, J., Lu, A., Anumanchipalli, G., & Choshen, L. (2026) · arXiv · doi:10.48550/arxiv.2607.18235
“A growing set of follow-up systems—including OpenEvolve (Sharma, 2025), ShinkaEvolve (Lange et al., 2025), DeltaEvolve (Jiang et al., 2026), AdaEvolve (Cemri et al., 2026), CodeEvolve (Assumpção et al., 2025), DeepEvolve (Liu et al., 2025), and GigaEvo (Khrulkov et al., 2025)—extend these loops…”
Peng, Z.-Y., Lan, Q.-Y., Wang, Z.-H., Zhang, J., & Piao, Y.-S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.31326
“ShinkaEvolve refines the outer loop through parent sampling, novelty-based candidate rejection and adaptive model selection [21].”
Automating quantum circuit optimization with AI agentsPreprint
Crichigno, M., & Flammia, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.31352 · Cited by 1
“Coding agents also draft evaluators from a description of the problem, through setup procedures distributed with Al-phaEvolve and ShinkaEvolve [26, 27] and in the concurrent Agentic Scientist [28], where a drafted evaluator is checked with smoke tests and a review by a language model before the…”
ADAPT for Science: A Multi-Agent System for Self-Evolving Scientific OptimizationPreprint
Feng, Z., Ni, Y., Liu, J., Liu, L., Dou, D., & Feng, K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.30851
“Related systems extend this idea to scientific equations, computational workflows, and automated optimizer evolution [1, 7, 15, 17, 26, 27, 29].”
Directing to Diversify: Multi-Directional Search for Automatic Heuristic DesignPreprint
Zhu, J., Zhou, J., Liu, F., Zhang, N., Wu, Y., Lin, Z., & Cao, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.31048
“ShinkaEvolve (Lange et al., 2025) further promotes exploration through archive-based parent sampling and code-novelty rejection based on embedding similarity.”
ExpEvolve: Structuring and Reusing Experience for Open-Ended Discovery
Chen, Y.-Q., Liu, C., Zou, D.-Y., et al. (2026) · doi:10.48550/arxiv.2610.29303
“Typically, existing evolutionary coding agents such as ShinkaEvolve directly incorporate successful programs or rely on LLMs to summarize reusable insights and incorporate them as idea-level compares structurally different programs through a shared procedure-level factor, lp = refine_lp(lp) .”
InfluentialInheritance as an Admission Test for LLM Prompt Merge OperatorsPreprint
Chong, Z., Ohsaki, H., & Ng, B. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.29023
SpecEvo: Speculative Evolution with Large Language Models for Cost-Efficient Scientific DiscoveryPreprint
Hieu, H. M., Zhu, T., Long, D. X., & Kan, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.28706
“ShinkaEvolve addresses this setting with a bandit that selects mutation models according to their observed performance (Lange et al., 2026).”
FrontierRefactor: Benchmarking Multi-Scale Performance-Oriented Codebase RefactoringPreprint
Wang, S., Meng, F., Chen, J., Chen, Q., Song, L., Fu, R., Zheng, S., Shao, S., Xie, Q., Zhao, J., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.25834
“We observe an exploration-exploitation trade-off [45]: scalar feedback prompts conservative edits maximizing gate clearance (40.0% strict success), whereas noise-aware telemetry empowers aggressive optimizations, achieving highest speedup (+5.2%, peak +11.6”
InfluentialSelf-Evolving Coding Agents via Harness-Policy OptimizationPreprint
Wei, Z., Yao, W., Peng, B., Wang, D., Wu, Q., Cheng, H., Meng, Y., & Gao, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.22791
“Program evolution also improves solutions and agents (Novikov et al., 2025; Lange et al., 2026; Zelikman et al., 2024; Zhang et al., 2026b;c).”
QD-Rec: Open-Ended Evolution of Recommendations via Quality-DiversityPreprint
Ishida, T., Makino, T., & Sugiura, I. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.23470
“A growing family of systems places a frozen large model inside an evolutionary or QD loop as a proposal operator or mutator, keeping an archive of artifacts that the model conditions on to propose the next: diverse creative text in QDAIF (Bradley et al., 2024), adversarial prompts in Rainbow Teaming (Samvelyan et al.,…”
What Survives Self-Evolution: Provenance Marks in Copied Agent HarnessesPreprint
Li, J., Li, J., Li, S., Zhang, X., & Qian, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.21458
“…et al., 2026); evolvers differ in what they rewrite, how they accept and how many tasks a decision sees (Hu et al., 2025; Agrawal et al., 2026; Lange et al., 2026; Chen et al., 2026), and with small selection sets their gains are hard to tell from noise (Kozyrev et al., 2026; Wang et al.,…”
InfluentialMLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning EngineeringPreprint
Du, S., Shi, J., Zhang, S., Wu, Y., Zhao, J., Feng, S., Li, X., He, L., Yan, X., & Zhang, B. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.22344
“Following ShinkaEvolve [24], we optimize a Python scaffold for a fixed GPT-4.”
SHELDON: Reasoning through Dual Search over Hypothesis and Observation SpacesPreprint
Joshi, H., Shethia, P., & Lam, M. S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.18025
LLM Evolutionary Search Strategies Should be Compared by TokensPreprint
Oved, T., Pony, R., Naparstek, O., & Barzelay, U. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.23977
“The same loop drives heuristic design (Liu et al., 2024; Ye et al., 2024), prompt optimization (Fernando et al., 2023; Agrawal et al., 2026), and sample-efficient variants (Lange et al., 2026), and ADRS-Bench collects systems-optimization tasks for the setting (Cheng et al., 2025).”
HarnessTTT: Learn to Optimize Harness at Test-Time for Scientific DiscoveryPreprint
Ma, Y., Cao, Y., Li, N., Wu, F., Zhang, Y., Zou, J., Choi, Y., & Xiao, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.17370
DuoDiscover: Scaling Autonomous Discovery Beyond the Evaluation BottleneckPreprint
Agarwal, S., Gupta, A., Heddes, M., Soylu, D., Meng, R., Kon, P. T. J., Chen, J., Cemri, M., Liu, S., Gonzalez, A., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.17184
“Evaluation has long driven progress in AI, yet most prior work on autonomous discovery focuses on refining the search harness itself (Cemri et al., 2026; Lange et al., 2026), largely treating the evaluator as given and fixed.”
Small but Mighty: Test Suites as Improvement Signals for Self-Evolving AgentsPreprint
Cheng, Y., Cheng, W., Yu, W., Zhou, D., & Chen, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.15964 · Cited by 1
“LLMs used as variation operators under executable fitness make evolutionary search a strong inference-time optimizer [28, 41, 47], but a global objective concentrates the population around one mode [16, 19].”
Chen, J., Chen, W., & Vong, C. M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.15738
“Program and agent optimizers keep archives of evaluated candidates (Romera-Paredes et al., 2024; Novikov et al., 2025; Lange et al., 2025; Lee et al., 2025; Hu et al., 2025; Zhang et al., 2025a; Lee et al., 2026; Xia et al., 2026).”
EvoAlloc: A Self-Evolving Resource Allocation Agent for Efficient Program EvolutionPreprint
Dai, Y., Wang, Y., Li, N., Wang, W., & Schmidhuber, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.12086
“…selective evaluation takes the form of cascade evaluation through increasingly comprehensive tests (Novikov et al., 2025), novelty-based rejection of near-duplicate proposals (Lange et al., 2026), and LLM-based candidate ranking or filtering before costly evaluation (Foster et al., 2026).”
InfluentialOpen-ended Scientific Discovery with Possibilistic ReasoningPreprint
Yang, A., Chau, S. L., Wakayama, T., Muandet, K., & Adachi, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.11289
“Such implementation can be implemented by rejection sampling (Novikov et al., 2025; Lange et al., 2026; Adachi et al., 2026b).”
EliteTable: Self-Improving Program Search for Executable Control Policies
Farmer, S. (2026) · doi:10.48550/arxiv.2610.12536
“ShinkaEvolve combines parent sampling, code-novelty rejection, and bandit-based selection among language models [4].”
VEPA: Verified Evolving Formal Policy for Lifelong LLM Agent GuardrailsPreprint
Nguyen, D., Miculicich, L., Mishra, B. D., Li, S., Dvijotham, K., Bansal, M., Pfister, T., & Le, L. T. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.12813 · Cited by 1
EvoSignal: LLM-Guided Evolutionary Design of Modular Traffic Signal Control ProgramsPreprint
Wang, L., Duan, P., Qin, Z., Ling, Y., Xu, J., Wang, Y., Wang, H., & Ma, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.09563
“Combines LLM-based program revisions with evolutionary search and adaptive proposal mechanisms [15].”
AGAR: a reinforcement learning substrate for LLM program evolutionPreprint
Li, H., Ge, Z., Yuan, X., Lo, Y., Li, H., Zhou, S., Liu, Q., Ying, J., Liu, Y., Chen, M., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.09215
“Later systems differ in how they manage the population — ShinkaEvolve adds a bandit over LLM backends and novelty sampling (Lange et al., 2025); SMC-based variants replace generational selection with sequential Monte Carlo resampling (Jiang et al., 2026); EvoX makes the search strategy itself the…”
RLDISCOVER: LLM-driven co-evolution of reinforcement learning algorithmsPreprint
Li, H., Ge, Z., Yuan, X., Lo, Y., Zhou, S., Ying, J., Li, H., Liu, Q., Liu, Y., Liu, J., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.09218
“…the potential of this approach for mathematical discovery (Romera-Paredes et al., 2024); AlphaEvolve extended it to larger programs and computational optimization tasks (Novikov et al., 2025); and ShinkaEvolve developed mechanisms for more sample-efficient program evolution (Lange et al., 2025).”
Peng, B., Zhang, L., Zhou, Y., Wang, M., Wu, Y., Li, S., Fan, X., & Zhao, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.05950
“LLM-driven evolution automates model development by proposing code changes, evaluating them, and using the results to guide further edits (Liu et al., 2024; Novikov et al., 2025; Lange et al., 2025).”
AgentDiscover: Autonomous Discovery with Minimal Search ScaffoldingPreprint
Farahbakhsh, M., Sela, I., Doudi, F., Kunde, V. T., Narayanan, K. R., Chamberland, J., & Kalathil, D. M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.05334
“For baseline runs we conduct ourselves, OpenEvolve, ShinkaEvolve, and EvoX run through SkyDiscover (Liu et al., 2026b) with their default configurations, and CORAL uses its default configuration.”
InfluentialCURIO: Curiosity-Driven Test-Time Learning for Open-Ended DiscoveryPreprint
Feng, T., Yu, F., Lei, Z., Zou, J., Jiang, C., Yan, Y., You, J., & Lu, P. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.04851
“We compare two baseline families: frozen-LLM search , comprising A LPHA E-VOLVE (Novikov et al., 2025) via O PEN E VOLVE (Sharma, 2025), S HINKA E VOLVE (Lange et al., 2026), B EST -”
InfluentialULTRADISCOVERY: Abductive Exploration in an Interconnected, Epistemically Open Universe
Li, W.-H., ZHENG, T., Song, Y.-Q., et al. (2026) · doi:10.48550/arxiv.2610.03092
“Long-horizon agents are commonly strengthened through self-reflection, falsification-oriented experimentation, or search over candidate solutions (Shinn et al., 2023; Madaan et al., 2023; Huang et al., 2025; Romera-Paredes et al., 2024; Novikov et al., 2025; Lange et al., 2026).”
InfluentialTrading Strategy Optimization via Textual GradientPreprint
Yang, C., Wang, Q., Zhu, F., Lin, X., He, B., Zimmermann, R., & Chua, T.-S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.03128
“A (Liu et al., 2024), ReEvo (Ye et al., 2024), and LLaMEA (Van Stein & B¨ack, 2024), and further scaled or improved in AlphaEvolve (Novikov et al., 2025) and ShinkaEvolve (Lange et al., 2026).”
AdaEva: Accelerating LLM-Driven Algorithm Design with Adaptive Partial EvaluationPreprint
Nguyen, T., Liu, F., Le, P., Doerr, C., & Dang, N. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.03896
“…et al., 2025)), smaller evaluation subsets (HeuriGym (Chen et al., 2026)), learned fitness predictors or proxy eval-uators (Hercules-P (Wu et al., 2025); Janus (Liu et al., 2026c)), or more sample-efficient search policies (ShinkaE-volve (Lange et al., 2026); AdaEvolve (Cemri et al., 2026)).”
Distribution Matching Evolutionary Algorithms for Rare Event SamplingPreprint
Gideoni, Y., & Gal, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.03833
“Gideoni et al. [2026] found SCS to be a strong baseline when compared to open-source versions of AlphaEvolve, like ShinkaEvolve [Lange et al., 2026].”
SEDIMA: Cross-Run Hierarchical Insight Memory for Evolutionary Search Agents
Abaskohi, A., Mostajabdaveh, M., & Zhou, Z.-R. (2026) · doi:10.48550/arxiv.2610.02361
“This makes it a drop-in module rather than a redesign: we integrate the same memory, unchanged, into two independent open-source harnesses, OpenEvolve (Sharma, 2025) and ShinkaE-volve (Lange et al., 2025), with no modification to either codebase’s search logic.”
InfluentialBerghaus, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.01223
“…showed that searching over LLM-proposed programs can yield new results in mathematics [31]; AlphaEvolve broadened this into a general coding agent for scientific and algorithmic discovery [23]; and open frameworks such as ShinkaEvolve study how to make such program search sample-efficient [15].”
InfluentialInitialization Improves LLM-Driven DiscoveryPreprint
Sakarvadia, M., Ciccone, M., & Raffel, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.00707
“We consider three popular state-of-the-art harnesses that have been widely studied in this work: OpenEvolve (Sharma, 2025), ShinkaEvolve (Lange et al., 2026), and OA Agrawal et al. (2026a).”
InfluentialLu, C., Xue, K., Xu, S., Yuan, M., & Qian, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.38757
“…solves them from a high-level decision-making perspective by iteratively designing and refining algorithms, achieving outstanding results in scenarios like classical combinatorial optimization problems and open mathematical discoveries with pure in-context evolutionary frameworks [23, 32, 18].”
InfluentialLabBook: Harnessing Experimental History for Efficient LLM-Driven DiscoveryPreprint
Yuan, B., Ye, W., Zhao, Z., Moukheiber, L., Kautz, H., Zhang, A., & Chen, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.00675
“ShinkaEvolve, CodeEvolve, AdaEvolve, and EvoX further improve parent selection, nov-elty, quality-diversity, resource allocation, or adaptation of the search strategy (Lange et al., 2026; Assumpc¸˜ao et al., 2026; Cemri et al., 2026; Liu et al., 2026a).”
InfluentialZhao, R., Wu, J., Zhu, C., & Lu, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.40258
Evolving Towards Better Codes: LLM-Guided Search for High-Distance Binary Linear CodesPreprint
Seddas, A., Shashkov, V., Viazovska, M., & Abbe, E. A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.37056
“Building on ShinkaEvolve [LIC25], L IN C ODE E VOLVE coordinates agents that propose and implement experiments with specialist search islands focused on different construction strategies.”
AutoRef: Harness Optimization for Agentic Multi-Reference Image GenerationPreprint
Oshima, Y., Onoda, K., Iwasawa, Y., Suzuki, M., Matsuo, Y., & Furuta, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.35530
Jha, P., Ghosh, A., & Ganesh, V. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.34069
“Geant4 source and specifications define expected behavior, with Celeritas as an expert performance reference where available.”
Towards an AI Software Factory for Data SystemsPreprint
Pavlenko, A., Crivat, B., Brandon, H., Curino, C., Psallidas, F., Slawinski, J., Freischuetz, J., Sanchez, L. P., Weimer, M., Demarne, M. B., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.36323 · Cited by 1
“Darwin is inspired by the original genetic+LLM approaches such as [1, 9, 12, 16, 17], but employs an agent-first approach to search that delivers up to 22 × better token efficiency (vs [17]).”
Wu, F., Barbalho, H., Mellou, K., Molinaro, M., Gao, J., Menache, I., Zhang, X., & Li, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.36303
“…from seminal systems such as FunSearch (Romera-Paredes et al., 2024) and AlphaEvolve (Novikov et al., 2025) to a growing set of follow-up frameworks (Lehman et al., 2023; Lange et al., 2026; Jiang et al., 2026; Wang et al., 2025; Assumpção et al., 2025; Khrulkov et al., 2025; Wan et al., 2025).”
Beyond Scripted Search: Sample-Efficient Reward Discovery via Agentic Black-box OptimizationPreprint
Li, M., Tan, R., & Wang, R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.32394 · Cited by 1
“ShinkaEvolve (Lange et al., 2026) selects a score-weighted parent plus one archive and one top-k inspiration and expands with a patch type drawn at fixed rates, testing whether a strong general program-evolution searcher closes the gap.”
InfluentialLEMCA: LLM-Guided Synthesis of Efficient Mode-Switching Control ArchitecturesPreprint
Krishna, A., Pacelli, V., & Jayaraman, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.21319
“…that the semantic priors and reasoning capabilities of modern coding LLMs can be harnessed to serve as effective operators to hypothesize and refine interpretable programs [19], as evidenced in demonstrations of optimizing algorithmic heuristics [17, 18], reward functions [20], and curricula [21].”
InfluentialRecursive Self-Improvement LLM Agents for Inverter Dynamic Model IdentificationPreprint
Feng, J., Wang, X., Chen, X., & Shi, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.14260
“FunSearch paired an LLM with a program evaluator to discover new mathematical constructions [19], while AlphaEvolve extended this paradigm to evolutionary program discovery [20], followed by open-source and more sample-efficient variants [21], [22].”
Filho, R. I. O. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.29636 · Cited by 3
“Code, run data, the dated laboratory notebook with every pre-registration and amendment, and the analyses that generate every table are in the repository of the program ( https://github.com/ RobertoOno/interrupting-the-loop ): the search harness ( scripts/frontier_search.py ), the problem verifiers ( src/creative_mach…”
InfluentialDiscovering Adaptive Transmission Programs for Collective Innovation
Colas, C., Perez, J., Nisioti, E., et al. (2026) · doi:10.48550/arxiv.2608.24545
“…design space, building on recent work showing that large language models can generate and refine programs through iterative variation and selection (Lange et al., 2025; Lehman et al., 2023; Meyerson et al., 2024; Novikov et al., 2025; Pourcel et al., 2024; Romera-Paredes et al., 2024; Zhang et…”
Test-Time Scaling for Scientific Equation Discovery
Lin, H.-W., Lim, H., Wang, X.-Y., et al. (2026) · doi:10.48550/arxiv.2608.28660
“Under this lens, recent self-evolving systems (Lange et al., 2025; Assumpção et al., 2025) can be viewed as specialized instances of external TTS that differ primarily in their control flow.”
The Little Scientist: LLM Agent-Driven Discovery via the Scientific Method
Smith, T. (2026) · doi:10.48550/arxiv.2608.16951
“ShinkaEvolve [Lange et al., 2025] addresses the sample-efficiency limitations of these systems through parent sampling, code novelty rejection-sampling, and bandit-based LLM ensemble selection, discovering state-of-the-art solutions on several tasks using as few as 150 samples.”
Optimize Cheap, Deploy Strong: Cost-Aware Cross-Tier Transfer for Evolutionary Optimization
Oved, T., Pony, R., Naparstek, O., et al. (2026) · doi:10.48550/arxiv.2608.10694
“A second group instead varies which model acts inside the search, by mutation role (Tanveer, 2026), by a bandit over generators (Lange et al., 2026), by generation confidence (Ray et al., 2026), or by relaying a cheap population to a strong model mid-run (Luo et al., 2026).”
EvoMaestro: Toward Interpretable and Steerable LLM-Driven Program Evolution
Liang, F., Cheng, S., Li, Y., He, R., Wen, X., & Wang, Y. (2026) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3830398.3830626
“Recent systems (e.g.,[16, 23, 25]) push this further: instead of assisting programmers, large language models (LLMs) serve as the engine of evolutionary search over programs.”
InfluentialA Hybrid Nested Harness for Decoupling Structure and Parameters in LLM-Driven OptimizationPreprint
Gallego, V. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.08156
Inference-Time Agentic Decision Rules Beat Longer Evolving Search for Multi-Image Medical Reasoning
Li, S., Hao, J., & Liu, X. (2026) · arXiv.org · doi:10.48550/arxiv.2607.27564
“All prompt-program strategies are evolved with ShinkaEvolve [17].”
InfluentialObjective-Aligned Direct Answer SFT for Robust Multi-Frame Medical VQA
Li, S., Hao, J., & Liu, X. (2026) · arXiv.org · doi:10.48550/arxiv.2607.27566
“We contrast this direct family with controller-like and auxiliary-supervision alternatives inspired by medical reasoning or scaffolded search, including MedVLM-R1, Med-R1, and ShinkaEvolve-style program evolution [16, 17, 22].”
Efficiency Matters in Autonomous ResearchPreprint
Yang, H., & Cao, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.24647 · Cited by 1
“…ranging from Hy-perband for AutoML [33] to bandit-based allocation across acquisition strategies, tree branches, parallel evolutionary runs, and LLM choices [34–37]. fluid applies this principle to a portfolio of LLM-driven hill-climbing chains, using anytime performance as the central objective.”
Adaptive Graph-of-Islands Evolution for Automatic Feature Engineering with LLMsPreprint
Li, S., & Ramakrishnan, N. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.23286
“, program and algorithm discovery [Šurina et al., Assumpção et al., 2025, Novikov et al., 2025, Sharma, Lange et al., 2025], and scientific discovery [Shojaee et al., Yang et al., 2025b, Chen et al., 2026, Abhyankar et al., 2026].”
Reward-Free Evolving Agents via Pairwise Validator
Liu, M.-H., Wang, Y., Wang, J.-Y., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2607.14408
“We test the pairwise validator on code evolution under ADRS [4] and ShinkaEvolve [15], with Qwen3-8B as the agent model.”
InfluentialGAE: Graph-Augmented Evolution for Scientific Discovery via Reinforcement OptimizationPreprint
Chen, X., & Cheng, T. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.10127
“ShinkaEvolve (Lange et al., 2025) improves sample efficiency via a bandit-based LLM ensemble and novelty-based rejection filtering, yet the selection policy remains static and prompt-driven with no learned component.”
LLM-Driven Evolutionary Generation of Multi-Objective Bayesian Optimization AlgorithmsPreprint
Laskaris, G., Brasher, R., van Stein, B., Raponi, E., Bäck, T., & Neukart, F. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.08791
“…AlphaEvolve [11], which couples an ensemble of frontier LLMs with an evolutionary outer loop and an automated evaluation harness, reporting discoveries such as improved matrix-multiplication algorithms; open re-implementations (OpenEvolve [23]) and related systems (ShinkaEvolve [24]) have followed.”
Formally Verified Code Synthesis for Structured Data Translation in a Medical Internet of ThingsPreprint
Samplawski, C., & Cobb, A. D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.20776
“ShinkaEvolve is an open-source implementation of recent evolutionary code optimization techniques leveraging LLMs. ShinkaEvolve provides a multi-island evolutionary approach which builds an SQL database of programs and corresponding fitness scores.”
InfluentialCode evolution for link prediction in complex networksPreprint
Vlaskin, A., & Altmann, E. G. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.26132
“The combination of code-evolution systems [1] and large language models (LLMs) [2] has opened new promising approaches for automating the discovery of scientific algorithms [3, 4, 5, 6].”
REVES: REvision and VErification--Augmented Training for Test-Time ScalingPreprint
Liu, Y., Zhou, R., Zhao, X., Sharaf, A., Lin, H., Biswas, A., Ghavamzadeh, M., Wang, Z., & Hong, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.18910 · Cited by 1
“635983 ShinkaEvolve (Lange et al., 2025) Ensemble 2.635982 ThetaEvolve (Wang et al., 2025) R1-Qwen3-8B 2 .”
InfluentialFrom 0-to-1 to 1-to-N: Reproducible Engineering Evidence for MetaAI Recursive Self-Design
Li, D., Li, J., & Li, H.-Z. (2026) · arXiv.org · doi:10.48550/arxiv.2606.09663
“ShinkaEvolve is a complementary baseline for LLM-driven program evolution: LLM mutations, explicit evaluators, nov-elty filtering, adaptive parent sampling, and bandit model selection evolve target programs [22], [23].”
EvoSQL: Memory-Augmented Critic-Generator Co-Evolution for Text-to-SQL
Zhou, J., Wang, J., Zhou, C., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2607.20489
“Self-Evolution and recursive aggregation methods maintain populations of candidate solutions and improve them across rounds via selection, recombination, or self-aggregation (Novikov et al., 2025; Lange et al., 2025; Venkatraman et al., 2025).”
StorySpark: Module-wise Evolutionary Search for Story Premise GenerationPreprint
Yang, Y., Zhong, Z., Cao, Q., Li, J., Xu, B., Yuan, K., Yang, M., & Yue, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.12336
EvoDrive: Pareto Evolution for Safety-Critical Autonomous Driving via Self-Improving LLM AgentsPreprint
Nie, T., Mei, Y., Tang, Y., He, J., Deng, J., Sun, J., & MA, W. C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.03678
“Frameworks such as [17, 18, 19] have achieved notable success in software development and algorithm discovery.”
Hoppe, F., Röhrig-Zöllner, M., & Knechtges, P. (2026) · arXiv.org · doi:10.48550/arxiv.2606.01975
“…of the evolutionary search, LLM4AD [ LZX + 24 ] provides a broader experimental platform for LLM-based algorithm development, ShinkaEvolve [ LIC25 ] focuses strongly on sample-efficient evolution via parent/LLM selection and novelty rejection-sampling, and ThetaEvolve [ WSZa25 ] extends…”
Learning to Solve and Optimize by Evolving Code
Semmelrock, V., Strizzolo, B., Zuccato, F., et al. (2026) · Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence · doi:10.48550/arxiv.2605.31049
“I Furthermore, Al-phaEvolve [Novikov et al. , 2025] and its open-source implementations, such as OpenEvolve [Sharma, 2025], ShinkaE-volve [Lange et al. , 2025], DeepEvolve [Liu et al. , 2025], and CodeEvolve [Assumpc¸˜ao et al. , 2025], have applied code evolution to a variety of domains, focusing…”
Towards Feedback-to-Plan Decisions for Self-Evolving LLM Agents in CUDA Kernel GenerationPreprint
Chong, Y. H., Wu, J., Zhang, Y., & Qu, P. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.26720
“…in prior work commonly relies on end-to-end ablation, where the evolution is restarted or resumed under different feedback configurations, and the best-achieved kernels are reported (Novikov et al., 2025; Zhang et al., 2026a; Liu et al., 2024b; Lange et al., 2026; Assumpc¸˜ao et al., 2026).”
Declarative Data Services: Structured Agentic Discovery for Composing Data SystemsPreprint
Ye, S., & Lu, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.20690
“AlphaEvolve [1] structures evolutionary search over programs with benchmark verifiers; SkyDiscover [24], OpenEvolve [25], and ShinkaEvolve [26] extend the same paradigm to broader algorithmic and scientific discovery.”
CoupleEvo: Evolving Heuristics for Coupled Optimization Problems Using Large Language Models
Bömer, T., Amberg, B., Disselnmeyer, M., et al. (2026) · GECCO Companion · doi:10.1145/3795101.3814664
“ShinkaEvolve [13] introduces a evolutionary framework that improves sample efficiency in LLM-driven program evolution through adaptive parent selection, novelty-based rejection sampling, and bandit-based LLM ensemble selection.”
Co-evolving Agent Architectures and Interpretable Reasoning for Automated Optimization
Huang, J., Xu, P.-L., Nan, X.-Y., & Luo, W.-J. (2026) · arXiv.org · doi:10.48550/arxiv.2604.17708
“Examples include GEPA [44], ShinkaEvolve [45], AdaEvolve [46], DeltaEvolve [47], and ThetaEvolve [48], which study prompt optimization, adaptive resource allocation, context-efficient evolutionary updates, and learning-based variation control.”
When Do We Need LLMs? A Diagnostic for Language-Driven Bandits
Berdica, U., Acero, F., Ipsen, A., Zehtabi, P., Cashmore, M., & Veloso, M. (2026) · arXiv.org · doi:10.48550/arxiv.2604.05859
“More recently, Lange et al. (2025) use a UCB-style bandit to adaptively select which large language model in an ensemble proposes the most effective program mutation during an evolutionary optimization loop.”
EvoForest: A Novel Machine-Learning Paradigm via Open-Ended Evolution of Computational Graphs
Yüksel, K., & Sawaf, H. (2026) · arXiv.org · doi:10.48550/arxiv.2604.19761
SemRep: Generative Code Representation Learning with Code TransformationsPreprint
Li, W., Song, J., Stoica, B. A., Dhoot, A., Ryan, G., Fu, S., & Pei, K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.13640 · Cited by 1
“…code transformation can discover high-performance GPU kernels (Ouyang et al., 2025; Wei et al., 2025a), co-design system architectures (Cheng et al., 2025a;b), and even serve as a substrate for algorithmic and scientific discovery (Press et al., 2025; Novikov et al., 2025; Lange et al., 2025).”
Wu, C., Chen, H., Yuan, C., Yang, Q., Song, G., Gao, Y., Luo, L., Chen, F. Y., & Guo, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.21634
“More recent systems, such as AlphaResearch [43], further integrate execution-based verification with simulated peer review signals, while CodeEvolve [1] and ShinkaEvolve [18] improve scalability through evolutionary strategies and adaptive LLM scheduling.”
Test-Time Optimization of Physical Query Plans with LLMsPreprint
Erol, M. H., Hao, X., Bianchi, F., Greco, C., Tagliabue, J., & Zou, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.10387 · Cited by 3
Can LLMs Do Rocket Science? Exploring the Limits of Complex Reasoning with GTOC 12
del Campo, I., Cuervo, P., Rodríguez-Fernández, V., et al. (2026) · AIAA SCITECH 2026 Forum · doi:10.2514/6.2026-2379
“…a clear "implementation barrier," future work will focus on bridging the gap between strategic reasoning and execution through three primary pillars: • Architectural Evolution and Self-Correction: We intend to integrate more sophisticated, evolutionary agent frameworks such as Shinka Evolve [11].”
Escher-Loop: Mutual Evolution by Closed-Loop Self-Referential Optimization
Liu, Z., Guo, X., Wei, X., Hao, H., & Yang, L. (2026) · arXiv · doi:10.48550/arxiv.2604.23472
AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization
Cemri, M., Agrawal, S., Gupta, A., Liu, S., Cheng, A., Mang, Q., Naren, A., Erdogan, L. E., Sen, K., Zaharia, M., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.20133 · Cited by 2
“The largest gains appear on problems with deceptive fitness landscapes, such as Heilbronn Triangle and MinMaxDist , where fixed-policy baselines (OpenEvolve, GEPA, and ShinkaEvolve) frequently plateau after early progress.”
InfluentialMeta Context Engineering via Agentic Skill Evolution
Ye, H., He, X., Arak, V., Dong, H., & Song, G. (2026) · arXiv · doi:10.48550/arxiv.2601.21557 · Cited by 1
CodeEvolve: an open source evolutionary coding agent for algorithmic discovery and optimization
Assumpção, H., Ferreira, D., Campos, L., & Murai, F. (2025) · arXiv · doi:10.48550/arxiv.2510.14150
“In response, several open-source frameworks (Sharma, 2025; Lange et al., 2025; Wang et al., 2025; Yu et al., 2025) have begun to explore LLM-driven evolutionary agents, offering accessible baselines but leaving open questions about orchestration design, evaluation rigor, and quality– cost…”
InfluentialRecursive self-improvement of AI research agents
Srikanth, D., Zhao, B., Xu, D., Wu, Y., & Jiang, Z. (2026) · arXiv · doi:10.48550/arxiv.2609.26457
Understanding the Challenges in Iterative Generative Optimization with LLMs
Nie, A., Daull, X., Kuang, Z., Akkiraju, A., Chaudhuri, A., Piasevoli, M., Rong, R., Yuan, Y., Choudhary, P., Xiao, S., et al. (2026) · arXiv · doi:10.48550/arxiv.2603.23994
“We note that the learning loop can be implemented with any LLM optimization and search algorithms (Novikov et al., 2025; Pan et al., 2025a; Lange et al., 2025a; Agrawal et al., 2025; Ren et al., 2026) and the main focus of our paper is to study the factors that impact the learning loop, not the…”
EurekAgent: Agent Environment Engineering is All You Need For Autonomous Scientific Discovery
Xin, A., Siow, J., Wang, J., Yao, Z., Zhang, F., Song, J., Hou, L., & Li, J. (2026) · arXiv · doi:10.48550/arxiv.2606.13662
“Evolutionary systems such as AlphaEvolve explicitly maintain populations of candidate programs and use evaluator feedback to guide mutation and selection (Novikov et al., 2025; Lange et al., 2025; Liu et al., 2026b).”
LEVI: Stronger Search Architectures Can Substitute for Larger LLMs in Evolutionary Search
Tanveer, T. (2026) · arXiv · doi:10.48550/arxiv.2605.09764
“OpenEvolve [Sharma, 2025] combines island-based evolution with low-dimensional MAP-Elites [Mouret and Clune, 2015] archiving; ShinkaEvolve [Lange et al., 2025] adds weighted archive sampling, novelty filters, LLM novelty judges, and adaptive model selection; AdaEvolve [Cemri et al., 2026] adapts…”
InfluentialSafety Must Precede the Deployment of Open-Ended AI
Sheth, I., Wehner, J., Abdelnabi, S., Binkyte, R., & Fritz, M. (2025) · arXiv · doi:10.48550/arxiv.2502.04512
“There is a rapidly growing interest in OE AI as a pathway toward more general and continually improving artificial intelligence; examples of that are recent workshops (neu, 2023; 2024) and ICLR’25 keynote talk (Rocktaeschel, 2025).”
Metadata
- Authors
- Lange, R. T., Imajuku, Y., & Cetin, E.
- ORCID
- Imajuku, Yuki
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2509.19349
- OpenAlex
- W7083291402
- Cited by
- 0
- References
- 0
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Keywords
- software evolution, large language models, sample efficiency, circle packing, loss function design, scientific discovery
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-24, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-24 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
