ThetaEvolve: Test-time Learning on Open Problems
Open accessAbstract
Recent advances in large language models (LLMs) have enabled breakthroughs in mathematical discovery, exemplified by AlphaEvolve, a closed-source system that evolves programs to improve bounds on open problems. However, it relies on ensembles of frontier LLMs to achieve new bounds and is a pure inference system that models cannot internalize the evolving strategies. We introduce ThetaEvolve, an open-source framework that simplifies and extends AlphaEvolve to efficiently scale both in-context learning and Reinforcement Learning (RL) at test time, allowing models to continually learn from their experiences in improving open optimization problems. ThetaEvolve features a single LLM, a large program database for enhanced exploration, batch sampling for higher throughput, lazy penalties to discourage stagnant outputs, and optional reward shaping for stable training signals, etc. ThetaEvolve is the first evolving framework that enable a small open-source model, like DeepSeek-R1-0528-Qwen3-8B, to achieve new best-known bounds on open problems (circle packing and first auto-correlation inequality) mentioned in AlphaEvolve. Besides, across two models and four open tasks, we find that ThetaEvolve with RL at test-time consistently outperforms inference-only baselines, and the model indeed learns evolving capabilities, as the RL-trained checkpoints demonstrate faster progress and better final performance on both trained target task and other unseen tasks. We release our code publicly: https://github.com/ypwang61/ThetaEvolve
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Our five-run choice is larger than the common three-run protocol used in prior work.
“Our five-run choice is larger than the common three-run protocol used in prior work(Cemri et al., 2026;Wang et al., 2025; Liu et al., 2026b;Assumpção et al., 2025)”
From Automated Discovery Has No Universally Superior Harness (2026) · cites this paper as Wang et al., 2025
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Part of this paper’s reference list could not be split into single entries.
ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution
Lange, R. T., Imajuku, Y., & Cetin, E. (2025) · arXiv · doi:10.48550/arxiv.2509.19349 · cited 4×
“We consider two open tasks, circle packing (CP) and the first autocorrelation inequality (FACI), and report the best values mentioned in AlphaEvolve-v2 (Georgiev et al., 2025) and its variant ShinkaEvolve (Lange et al., 2025).”
Influential
Flagged first, then by citations
Research works printed but not linked: 16
- [2]2. Fernando, C., Banarse, D., Michalewski, H., Osindero, S., and Rockt¨ aschel, T. Promptbreeder: Self-referential self-improvement via prompt evolution. arXiv preprint arXiv:2309.16797 , 2023. · arXiv
- [4]4. Gao, J., Xu, S., Ye, W., Liu, W., He, C., Fu, W., Mei, Z., Wang, G., and Wu, Y. On designing effective rl reward at training time for llm reasoning. arXiv preprint arXiv:2410.15115 , 2024. · arXiv
- [5]5. Georgiev, B., G´ omez-Serrano, J., Tao, T., and Wagner, A. Z. Mathematical exploration and discovery at scale. arXiv preprint arXiv:2511.02864 , 2025. · arXiv
- [6]6. Google DeepMind. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261 , 2025. · arXiv
- [7]7. Guo, Q., Wang, R., Guo, J., Li, B., Song, K., Tan, X., Liu, G., Bian, J., and Yang, Y . Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. arXiv preprint arXiv:2309.08532 , 2023. · arXiv
- [9]9. Hubert, T., Mehta, R., Sartran, L., Horv´ ath, M. Z., ˇ Zuˇ zi´ c, G., Wieser, E., Huang, A., Schrittwieser, J., Schroecker, Y., Masoom, H., Bertolli, O., Zahavy, T., Mandhane, A., Yung, J., Beloshapka, I., Ibarz, B., Veeriah, V., Yu, L., Nash, O., Lezeau, P., Mercuri, S., S¨ onne, C., Mehta, B., Davies, A., Zheng, D., Pedregosa, F., Li, Y., von Glehn, I., Rowland, M., Albanie, S., Velingker, A., Schmitt, S., Lockhart, E., Hughes, E., Michalewski, H., Sonnerat, N., Hassabis, D., Kohli, P., and Silver, D. Olympiad-level formal mathematical reasoning with reinforcement learning. Nature , 2025. doi: 10.1038/s41586-025-09833-y. URL https://doi. org/10.1038/s41586-025-09833-y . · DOI · Paper page
- [10]10. Jaech, A. and Joseph, A. Further improvements to the lower bound for an autoconvolution inequality. arXiv preprint arXiv:2508.02803 , 2025a. · arXiv
- [11]11. Jaech, A. and Joseph, A. Further improvements to the lower bound for an autoconvolution inequality. arXiv preprint arXiv:2508.02803 , 2025b. · arXiv
- [12]12. Khattab, O., Singhvi, A., Maheshwari, P., Zhang, Z., Santhanam, K., Vardhamanan, S., Haq, S., Sharma, A., Joshi, T. T., Moazam, H., et al. Dspy: Compiling declarative language model calls into self-improving pipelines. arXiv preprint arXiv:2310.03714 , 2023. · arXiv
- [13]13. Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., and Stoica, I. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles , 2023.
- [14]14. Lambert, N., Morrison, J., Pyatkin, V., Huang, S., Ivison, H., Brahman, F., Miranda, L. J. V., Liu, A., Dziri, N., Lyu, S., Gu, Y., Malik, S., Graf, V., Hwang, J. D., Yang, J., Bras, R. L., Tafjord, O., Wilhelm, C., Soldaini, L., Smith, N. A., Wang, Y., Dasigi, P., and Hajishirzi, H. T¨ ulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15124 , 2024. · arXiv
- [18]18. Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al. Self-refine: Iterative refinement with selffeedback. Advances in Neural Information Processing Systems , 36:46534-46594, 2023.
- [19]19. Matolcsi, M. and Vinuesa, C. Improved bounds on the supremum of autoconvolutions. Journal of mathematical analysis and applications , 372(2):439-447, 2010.
- [20]20. Mouret, J.-B. and Clune, J. Illuminating search spaces by mapping elites. arXiv preprint arXiv:1504.04909 , 2015. · arXiv
- [21]21. Novikov, A., V˜ u, N., Eisenberger, M., Dupont, E., Huang, P.-S., Wagner, A. Z., Shirobokov, S., Kozlovskii, B., Ruiz, F. J., Mehrabian, A., et al. Alphaevolve: A coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.13131 , 2025. · arXiv
- [22]22. OpenAI. Openai o1 system card. arXiv preprint arXiv:2412.16720 , 2024. · arXiv
Cited by
Sorted by citations
EvoX: Meta-Evolution for Automated Discovery
Liu, S., Agarwal, S., Maheswaran, M., Cemri, M., Li, Z., Mang, Q., Naren, A., Boneh, E., Cheng, A., Pan, M. Z., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.23413 · Cited by 1 · see citing works
“SOAR [25] alternates between search and model fine-tuning, while ThetaEvolve [33], TTT-Discover [36], and FLEX [4] apply reinforcement learning to improve the generator model.”
MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution
Wang, Z., Yan, M., Bi, J., Yan, S., Tresp, V., & Ma, Y. (2026) · arXiv · doi:10.48550/arxiv.2607.05297
Automated Discovery Has No Universally Superior Harness
Gupta, A., Lei, J., Lu, A., Anumanchipalli, G., & Choshen, L. (2026) · arXiv · doi:10.48550/arxiv.2607.18235
Escher-Loop: Mutual Evolution by Closed-Loop Self-Referential Optimization
Liu, Z., Guo, X., Wei, X., Hao, H., & Yang, L. (2026) · arXiv · doi:10.48550/arxiv.2604.23472
- Authors
- Wang, Y., Su, S.-R., Zeng, Z., Xu, E., Ren, L., Yang, X., Huang, Z., He, X., Ma, L., Peng, B., Cheng, H., He, P., Chen, W., Wang, S., Du, S. S., & Shen, Y.
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2511.23473
- OpenAlex
- W4416932013
- Cited by
- 0
- References
- 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- open problems in mathematics, software evolution, large language models, reinforcement learning from human feedback, in-context learning, circle packing, reward shaping
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
