CoChat
Sign Up
PreprintCited by 3Evolutionary Algorithms and ApplicationsTeaching and Learning ProgrammingAI-based Problem Solving and Planning

Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGI

Open access
Pourcel, Julien, Colas, Cédric, Oudeyer, Pierre-Yves·2025·arXiv
DOI 10.48550/arxiv.2507.14172arXiv 2507.14172
Open this paper in CoChat — free accountOpen PDF

Abstract

Many program synthesis tasks prove too challenging for even state-of-the-art language models to solve in single attempts. Search-based evolutionary methods offer a promising alternative by exploring solution spaces iteratively, but their effectiveness remain limited by the fixed capabilities of the underlying generative model. We propose SOAR, a method that learns program synthesis by integrating language models into a self-improving evolutionary loop. SOAR alternates between (1) an evolutionary search that uses an LLM to sample and refine candidate solutions, and (2) a hindsight learning phase that converts search attempts into valid problem-solution pairs used to fine-tune the LLM's sampling and refinement capabilities\, -- \,enabling increasingly effective search in subsequent iterations. On the challenging ARC-AGI benchmark, SOAR achieves significant performance gains across model scales and iterations, leveraging positive transfer between the sampling and refinement finetuning tasks. These improvements carry over to test-time adaptation, enabling SOAR to solve 52\% of the public test set. Our code is open-sourced at: https://github.com/flowersteam/SOAR

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2507.14172

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The 2025 Paper Awards (TRM, SOAR, CompressARC) validate this direction, achieving competitive results through recursive refinement, self-improving search, and compression objectives rather than scale.

    “The 2025 Paper Awards (TRM, SOAR, Com-pressARC) validate this direction, achieving competitive results through recursive refinement, self-improving search, and compression objectives rather than scale.”

    From The ARC of Progress towards AGI: A Living Survey of Abstraction and Reasoning (2026) · cites this paper as Pourcel et al. 2025

    The Introduction and Section 4.4 support the SOAR portion: iterative refinement and self-improving search produced competitive ARC results, including gains beyond those from increasing model size alone. The cited paper does not establish that TRM, SOAR, and CompressARC received 2025 Paper Awards, or report TRM’s and CompressARC’s results or compression objectives. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • SOAR alternates between an evolutionary search phase to generate candidate programs and a 'hindsight learning' phase that uses all attempts to generate a rich dataset for fine-tuning the agent model, creating a virtuous cycle of self-improvement.

    “SOAR (Pourcel et al., 2025), for example, alternates between an evolutionary search phase to generate candidate programs and a”

    From A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence (2025) · cites this paper as Pourcel et al., 2025

  • Pourcel et al.'s SOAR framework fine-tunes LLMs on their own evolutionary search traces, achieving 52% on ARC-AGI-1 without hand-engineered DSLs.

    “Pourcel et al. 's SOAR framework fine-tunes LLMs on their own evolutionary search traces, achieving 52% on ARC-AGI-1 without hand-engineered DSLs [Pourcel et al. 2025].”

    From The ARC of Progress towards AGI: A Living Survey of Abstraction and Reasoning (2026) · cites this paper as Pourcel et al. 2025

  • CompressARC solved 20% with zero pretraining.

    “compression-based inference (CompressARC solving 20% with zero pretraining)”

    From The ARC of Progress towards AGI: A Living Survey of Abstraction and Reasoning (2026) · cites this paper as Pourcel et al. 2025

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 48

  1. Akyürek, E., Damani, M., Qiu, L., Guo, H., Kim, Y., and Andreas, J. The surprising effectiveness of test-time training for abstract reasoning, 2024. Preprint.
  2. Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Pieter Abbeel, O., and Zaremba, W. Hindsight experience replay. Advances in neural information processing systems, 30, 2017.
  3. Balog, M., Gaunt, A. L., Brockschmidt, M., Nowozin, S., and Tarlow, D. Deepcoder: Learning to write programs. arXiv preprint arXiv:1611.01989, 2016. · arXiv
  4. Bradley, H., Dai, A., Teufel, H., Zhang, J., Oostermeijer, K., Bellagente, M., Clune, J., Stanley, K., Schott, G., and Lehman, J. Quality-diversity through ai feedback. arXiv preprint arXiv:2310.13032, 2023. · arXiv
  5. Butt, N., Manczak, B., Wiggers, A., Rainone, C., Zhang, D. W., Defferrard, M., and Cohen, T. Codeit: Self- improving language models with prioritized hindsight replay. In International Conference on Machine Learn- ing, 2024.
  6. Chollet, F. On the measure of intelligence. arXiv preprint arXiv:1911.01547, 2019. · arXiv
  7. Chow, Y., Tennenholtz, G., Gur, I., Zhuang, V., Dai, B., Thiagarajan, S., Boutilier, C., Agarwal, R., Kumar, A., and Faust, A. Inference-aware fine-tuning for best-of- n sampling in large language models. arXiv preprint arXiv:2412.15287, 2024. · arXiv
  8. Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36, 2024.
  9. Dong, K. and Ma, T. Beyond limited data: Self-play llm theorem provers with iterative conjecturing and proving. arXiv preprint arXiv:2502.00212, 2025. · arXiv
  10. Ellis, K., Wong, C., Nye, M., Sablé-Meyer, M., Morales, L., Hewitt, L., Cary, L., Solar-Lezama, A., and Tenenbaum, J. B. Dreamcoder: Bootstrapping inductive program synthesis with wake-sleep library learning. In Proceed- ings of the 42nd acm sigplan international conference on programming language design and implementation, pp. 835-850, 2021.
  11. Gandhi, K., Lee, D., Grand, G., Liu, M., Cheng, W., Sharma, A., and Goodman, N. D. Stream of search (sos): Learning to search in language. arXiv preprint arXiv:2404.03683, 2024. · arXiv
  12. Gehring, J., Zheng, K., Copet, J., Mella, V., Carbonneaux, Q., Cohen, T., and Synnaeve, G. Rlef: Grounding code llms in execution feedback with reinforcement learning. arXiv preprint arXiv:2410.02089, 2024. · arXiv
  13. Gendron, G., Bao, Q., Witbrock, M., and Dobbie, G. Large language models are not strong abstract reasoners. arXiv preprint arXiv:2305.19555, 2023. · arXiv
  14. Goldberg, e. a. Genetic algorithms and machine learning. 3(2):95-99. ISSN 1573-0565. doi: 10.1023/A:102260 2019183. URL https://doi.org/10.1023/A: 1022602019183. · DOI · Paper page
  15. Guo, D., Zhu, Q., Yang, D., Xie, Z., Dong, K., Zhang, W., Chen, G., Bi, X., Wu, Y., Li, Y., et al. Deepseek-coder: When the large language model meets programming-the rise of code intelligence. arXiv preprint arXiv:2401.14196, 2024. · arXiv
  16. Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al. Deepseek-r1: In- centivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. · arXiv
  17. Holland, J. H. Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. The MIT Press. ISBN 978-0-262-27555-2. doi: 10.7551/mitpress/1090.001.00 · DOI
  18. Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. · arXiv
  19. Hui, B., Yang, J., Cui, Z., Yang, J., Liu, D., Zhang, L., Liu, T., Zhang, J., Yu, B., Lu, K., et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186, 2024. · arXiv
  20. Jain, N., Han, K., Gu, A., Li, W.-D., Yan, F., Zhang, T., Wang, S., Solar-Lezama, A., Sen, K., and Stoica, I. Live- codebench: Holistic and contamination free evaluation of large language models for code. In The Thirteenth International Conference on Learning Representations.
  21. Kalajdzievski, D. A rank stabilization scaling factor for fine-tuning with lora. arXiv preprint arXiv:2312.03732, 2023. · arXiv
  22. Kumar, A., Lu, C., Kirsch, L., Tang, Y., Stanley, K. O., Isola, P., and Ha, D. Automating the search for artificial life with foundation models. arXiv preprint arXiv:2412.17799, 2024. · arXiv
  23. Langdon, W. B. and Poli, R. Foundations of genetic pro- gramming. Springer Science & Business Media, 2013.
  24. LeGris, S., Vong, W. K., Lake, B. M., and Gureckis, T. M. H-arc: A robust estimate of human performance on the ab- straction and reasoning corpus benchmark. arXiv preprint arXiv:2409.01374, 2024. · arXiv
  25. Lehman, J., Gordon, J., Jain, S., Ndousse, K., Yeh, C., and Stanley, K. O. Evolution through large models. In Handbook of Evolutionary Machine Learning, pp. 331- 366. Springer, 2023.
  26. Li, W.-D. and Ellis, K. Is programming by example solved by llms? arXiv preprint arXiv:2406.08316, 2024. · arXiv
  27. Li, W.-D., Hu, K., Larsen, C., Wu, Y., Alford, S., Woo, C., Dunn, S. M., Tang, H., Naim, M., Nguyen, D., Zheng, W.-L., Tavares, Z., Pu, Y., and Ellis, K. Combining induction and transduction for abstract reasoning, 2024. URL https://arxiv.org/abs/2411.02272. · arXiv
  28. Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. Competition-level code generation with alpha- code. Science, 378(6624):1092-1097, 2022.
  29. Liang, P., Jordan, M. I., and Klein, D. Learning programs: A hierarchical bayesian approach. In ICML, volume 10, pp. 639-646, 2010.
  30. Liu, F., Tong, X., Yuan, M., Lin, X., Luo, F., Wang, Z., Lu, Z., and Zhang, Q. Evolution of heuristics: Towards efficient automatic algorithm design using large language model. arXiv preprint arXiv:2401.02051, 2024. · arXiv
  31. Menon, A., Tamuz, O., Gulwani, S., Lampson, B., and Kalai, A. A machine learning framework for programming by example. In International Conference on Machine Learning, pp. 187-195. PMLR, 2013.
  32. Meyerson, E., Nelson, M. J., Bradley, H., Gaier, A., Moradi, A., Hoover, A. K., and Lehman, J. Language model crossover: Variation through few-shot prompting. ACM Transactions on Evolutionary Learning, 4(4):1-40, 2024.
  33. Minh, N. N., Baker, A., Neo, C., Roush, A., Kirsch, A., Independent, W. A., Shwartz-Ziv, R., and AI, W. Turning up the heat: Min-p sampling for creative and coherent llm outputs. In The Thirteenth International Conference on Learning Representations, 2024.
  34. Mirchandani, S., Xia, F., Florence, P., Ichter, B., Driess, D., Arenas, M. G., Rao, K., Sadigh, D., and Zeng, A. Large language models as general pattern machines. arXiv preprint arXiv:2307.04721, 2023. · arXiv
  35. Olausson, T. X., Inala, J. P., Wang, C., Gao, J., and Solar- Lezama, A. Is self-repair a silver bullet for code genera- tion? arXiv preprint arXiv:2306.09896, 2023. · arXiv
  36. Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J., Ellenberg, J. S., Wang, P., Fawzi, O., et al. Mathematical discoveries from program search with large language models. Nature, 625 (7995):468-475, 2024.
  37. Roziere, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., et al. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950, 2023. · arXiv
  38. Saad, F. A., Cusumano-Towner, M. F., Schaechtle, U., Ri- nard, M. C., and Mansinghka, V. K. Bayesian synthesis of probabilistic programs for automatic data modeling. Proceedings of the ACM on Programming Languages, 3 (POPL):1-32, 2019.
  39. Shi, K., Bieber, D., and Singh, R. Tf-coder: Program synthesis for tensor manipulations. ACM Transactions on Programming Languages and Systems (TOPLAS), 44(2): 1-36, 2022.
  40. Tang, H., Hu, K., Zhou, J. P., Zhong, S. C., Zheng, W.-L., Si, X., and Ellis, K. Code repair with LLMs gives an exploration-exploitation tradeoff. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net /forum?id=o863gX6DxA.
  41. Tang, Y., Zheng, K., Synnaeve, G., and Munos, R. Optimiz- ing language models for inference time objectives using reinforcement learning. arXiv preprint arXiv:2503.19595, 2025. · arXiv
  42. Wang, R., Zelikman, E., Poesia, G., Pu, Y., Haber, N., and Goodman, N. D. Hypothesis search: Inductive reasoning with language models. arXiv preprint arXiv:2309.05660, 2023. · arXiv
  43. Xu, Y., Li, W., Vaezipoor, P., Sanner, S., and Khalil, E. B. Llms and the abstraction and reasoning corpus: Suc- cesses, failures, and the importance of object-based rep- resentations. arXiv preprint arXiv:2305.18354, 2023. · arXiv
  44. Yue, Y., Chen, Z., Lu, R., Zhao, A., Wang, Z., Song, S., and Huang, G. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv preprint arXiv:2504.13837, 2025. · arXiv
  45. Zelikman, E., Wu, Y., and Goodman, N. D. Star: Self-taught reasoner. arXiv preprint arXiv:2203.14465, 2022. · arXiv
  46. Zelikman, E., Harik, G., Shao, Y., Jayasiri, V., Haber, N., and Goodman, N. D. Quiet-star: Language models can teach themselves to think before speaking. arXiv preprint arXiv:2403.09629, 2024. · arXiv
  47. Zhang, Y., Diddee, H., Holm, S., Liu, H., Liu, X., Samuel, V., Wang, B., and Ippolito, D. Noveltybench: Evaluating language models for humanlike diversity. arXiv preprint arXiv:2504.05228, 2025. · arXiv
  48. Zheng, L., Yin, L., Xie, Z., Sun, C. L., Huang, J., Yu, C. H., Cao, S., Kozyrakis, C., Stoica, I., Gonzalez, J. E., et al. Sglang: Efficient execution of structured language model programs. Advances in Neural Information Processing Systems, 37:62557-62583, 2024. transformation_mapping = {((3, 1, 2), (3, 1, 2), (3, 1, 2)): ((4, 5, 6), (4, 5, 6), (4 , 5, 6)), ((2, 3, 8), (2, 3, 8), (2, 3, 8 )): ((6, 4, 9), (6, 4, 9), (6, 4, 9)), (( 5, 8, 6), (5, 8, 6), (5, 8, 6)): ((1, 9, 2), (1, 9, 2), (1, 9, 2)), ((9, 4, 2), (9 , 4, 2), (9, 4, 2)): ((8, 3, 6), (8, 3, 6 ), (8, 3, 6)), ((8, 1, 3), (8, 1, 3), (8, 1, 3)): ((7, 5, 4), (7, 5, 4), (7, 5, 4) )}

Cited by

Sorted by citations

Authors
Pourcel, J., Colas, C., & Oudeyer, P.-Y.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
DOI
10.48550/arxiv.2507.14172
arXiv
2507.14172
OpenAlex
W4416831911
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
Locations
  • HAL (submittedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • European Commission
  • Agence Nationale de la Recherche (ANR‐19, ANR-19-CHIA-0004)
  • Grand Équipement National De Calcul Intensif
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
  • Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
Keywords
program synthesis, evolutionary search, language models, LLM fine-tuning, transfer learning, test-time adaptation, SOAR

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review