CoChat
Sign Up
PreprintCited by 6Topic ModelingAI-based Problem Solving and PlanningNatural Language Processing Techniques

Evolving Deeper LLM Thinking

Open access
Lee, Kuang-Huei, Fischer, Ian, Wu, Yueh-Hua, Marwood, Dave, Baluja, Shumeet, Schuurmans, Dale, Chen, Xinyun·2025·arXiv
DOI 10.48550/arxiv.2501.09891arXiv 2501.09891
Open this paper in CoChat — free accountOpen PDF

Abstract

We explore an evolutionary search strategy for scaling inference time compute in Large Language Models. The proposed approach, Mind Evolution, uses a language model to generate, recombine and refine candidate responses. The proposed approach avoids the need to formalize the underlying inference problem whenever a solution evaluator is available. Controlling for inference cost, we find that Mind Evolution significantly outperforms other inference strategies such as Best-of-N and Sequential Revision in natural language planning tasks. In the TravelPlanner and Natural Plan benchmarks, Mind Evolution solves more than 98% of the problem instances using Gemini 1.5 Pro without the use of a formal solver.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2501.09891

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Existing TTS methods are largely independent, implying that LLMs have not yet evolved to progressively learn how to scale more effectively.

    “However, existing TTS methods are largely independent, implying that LLMs have not yet evolved to progressively learn how to scale more effectively.”

    From LatentEvolve: Self-Evolving Test-Time Scaling in Latent Space (2025) · cites this paper as Lee et al., 2025

    The Introduction (page unavailable) says Best-of-N generates independent candidates and contrasts it with Mind Evolution’s iterative refinement. But it also describes existing sequential revision and tree-search methods, so it does not establish that TTS methods are largely independent. Its evolutionary approach refines candidate solutions; it does not claim that LLMs have not yet learned to scale more effectively. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 42

  1. Y. Bai, S. Kadavath, S. Kundu, A. Askell, J. Kernion, A. Jones, A. Chen, A. Goldie, A. Mirhoseini, C. McKinnon, et al. Constitu- tional AI: Harmlessness from AI feedback. arXiv preprint arXiv:2212.08073, 2022. · arXiv
  2. B. Berger, M. S. Waterman, and Y. W. Yu. Leven- shtein distance, sequence comparison and bio- logical database search. IEEE Transactions on In- formation Theory, 67(6):3287-3294, 2021. doi: 10.1109/TIT.2020.2996543. · DOI
  3. S. Brahmachary, S. M. Joshi, A. Panda, K. Koner- ipalli, A. K. Sagotra, H. Patel, A. Sharma, A. D. Jagtap, and K. Kalyanaraman. Large language model-based evolutionary optimizer: Reasoning with elitism. arXiv preprint arXiv:2403.02054, 2024. · arXiv
  4. B. Brown, J. Juravsky, R. Ehrlich, R. Clark, Q. V. Le, C. Ré, and A. Mirhoseini. Large language monkeys: Scaling inference com- pute with repeated sampling. arXiv preprint arXiv:2407.21787, 2024. · arXiv
  5. E. Cantú-Paz et al. A survey of parallel genetic algorithms. Calculateurs paralleles, reseaux et systems repartis, 10(2):141-171, 1998.
  6. A. Chen, D. M. Dohan, and D. R. So. EvoPrompt- ing: Language models for code-level neural ar- chitecture search. In Proceedings of the 37th International Conference on Neural Information Processing Systems, pages 7787-7817, 2023.
  7. B. Chen, F. Zhang, A. Nguyen, D. Zan, Z. Lin, J.-G. Lou, and W. Chen. CodeT: Code gen- eration with generated tests. In The Eleventh International Conference on Learning Represen- tations, 2023. URL https://openreview. net/forum?id=ktrw68Cmu9c.
  8. X. Chen, M. Lin, N. Schärli, and D. Zhou. Teaching large language models to self- debug. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id= KuPixIqPiq.
  9. K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. · arXiv
  10. C. Fernando, D. Banarse, H. Michalewski, S. Osindero, and T. Rocktäschel. Promptbreeder: Self-referential self-improvement via prompt evolution. arXiv preprint arXiv:2309.16797, 2023. · arXiv
  11. D. E. Golberg. Genetic Algorithms in Search, Opti- mization, and Machine Learning. Addison Wesley, 1989.
  12. Q. Guo, R. Wang, J. Guo, B. Li, K. Song, X. Tan, G. Liu, J. Bian, and Y. Yang. Connecting large language models with evolutionary algo- rithms yields powerful prompt optimizers. arXiv preprint arXiv:2309.08532, 2023. · arXiv
  13. Y. Hao, Y. Chen, Y. Zhang, and C. Fan. Large language models can plan your travels rigorously with formal verification tools. arXiv preprint arXiv:2404.11891, 2024. · arXiv
  14. E. Hemberg, S. Moskal, and U.-M. O'Reilly. Evolv- ing code with a large language model. Genetic Programming and Evolvable Machines, 25(2):21, 2024.
  15. J. H. Holland. Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor, MI, 1975. second edition, 1992.
  16. G. Kim, P. Baldi, and S. McAleer. Language models can solve computer tasks. arXiv preprint arxiv:2303.17491, 2023. · arXiv
  17. J. H. Kirchner, Y. Chen, H. Edwards, J. Leike, N. McAleese, and Y. Burda. Prover-verifier games improve legibility of LLM outputs. arXiv preprint arXiv:2407.13692, 2024. · arXiv
  18. T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa. Large language models are zero- shot reasoners. Advances in Neural Information Processing Systems, 35:22199-22213, 2022.
  19. H. Le, Y. Wang, A. D. Gotmare, S. Savarese, and S. C. H. Hoi. CodeRL: Mastering code generation through pretrained models and deep reinforce- ment learning. Advances in Neural Information Processing Systems, 35:21314-21328, 2022.
  20. J. Lehman, J. Gordon, S. Jain, K. Ndousse, C. Yeh, and K. O. Stanley. Evolution through large mod- els. In Handbook of Evolutionary Machine Learn- ing, pages 331-366. Springer, 2023.
  21. Z. Liang, Y. Liu, T. Niu, X. Zhang, Y. Zhou, and S. Yavuz. Improving LLM reasoning through scaling inference computation with collaborative verification. arXiv preprint arXiv:2410.05318, 2024. · arXiv
  22. H. Lightman, V. Kosaraju, Y. Burda, H. Ed- wards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe. Let's verify step by step. arXiv preprint arXiv:2305.20050, 2023. · arXiv
  23. F. Liu, X. Lin, Z. Wang, S. Yao, X. Tong, M. Yuan, and Q. Zhang. Large language model for multi-objective evolutionary optimization. arXiv preprint arXiv:2310.12541, 2023. · arXiv
  24. J. Liu, Y. Zhu, K. Xiao, Q. FU, X. Han, Y. Wei, and D. Ye. RLTF: Reinforcement learning from unit test feedback. Transactions on Ma- chine Learning Research, 2023. ISSN 2835- 8856. URL https://openreview.net/ forum?id=hjYmsV6nXZ.
  25. S. Liu, C. Chen, X. Qu, K. Tang, and Y.-S. Ong. Large language models as evolutionary optimiz- ers. In 2024 IEEE Congress on Evolutionary Com- putation (CEC), pages 1-8. IEEE, 2024.
  26. V. Liventsev, A. Grishina, A. Härmä, and L. Moo- nen. Fully autonomous programming with large language models. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 1146-1155, 2023.
  27. M. Mitchell. An Introduction to Genetic Algo- rithms. MIT press, 1998.
  28. J. S. Park, J. O'Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein. Generative agents: Interactive simulacra of human behavior. In Pro- ceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, pages 1-22, 2023.
  29. N. Provos and P. Honeyman. Hide and seek: An introduction to steganography. IEEE security & privacy, 1(3):32-44, 2003.
  30. B. Romera-Paredes, M. Barekatain, A. Novikov, M. Balog, M. P. Kumar, E. Dupont, F. J. Ruiz, J. S. Ellenberg, P. Wang, O. Fawzi, et al. Mathemati- cal discoveries from program search with large language models. Nature, 625(7995):468-475, 2024.
  31. A. Setlur, C. Nagpal, A. Fisch, X. Geng, J. Eisen- stein, R. Agarwal, A. Agarwal, J. Berant, and A. Kumar. Rewarding progress: Scaling auto- mated process verifiers for LLM reasoning. arXiv preprint arXiv:2410.08146, 2024. · arXiv
  32. N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024.
  33. C. Snell, J. Lee, K. Xu, and A. Kumar. Scaling LLM test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314, 2024. · arXiv
  34. X. Wang, J. Wei, D. Schuurmans, Q. V. Le, E. H. Chi, S. Narang, A. Chowdhery, and D. Zhou. Self-consistency improves chain of thought rea- soning in language models. In The Eleventh International Conference on Learning Represen- tations, 2023. URL https://openreview. net/forum?id=1PL1NIMMrw.
  35. Z. Wang, Y. Li, Y. Wu, L. Luo, L. Hou, H. Yu, and J. Shang. Multi-step problem solving through a verifier: An empirical analysis on model- induced process supervision. arXiv preprint arXiv:2402.02658, 2024. · arXiv
  36. J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. Chain- of-thought prompting elicits reasoning in large language models. Advances in Neural Informa- tion Processing Systems, 35:24824-24837, 2022.
  37. J. Xie, K. Zhang, J. Chen, T. Zhu, R. Lou, Y. Tian, Y. Xiao, and Y. Su. Travelplanner: A benchmark for real-world planning with language agents. arXiv preprint arXiv:2402.01622, 2024. · arXiv
  38. S. Yao, D. Yu, J. Zhao, I. Shafran, T. L. Griffiths, Y. Cao, and K. Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In Proceedings of the 37th International Conference on Neural Information Processing Sys- tems, pages 11809-11822, 2023.
  39. H. Ye, J. Wang, Z. Cao, F. Berto, C. Hua, H. Kim, J. Park, and G. Song. ReEvo: Large language models as hyper-heuristics with reflective evolu- tion. arXiv preprint arXiv:2402.01145, 2024. · arXiv
  40. S. Yuan, K. Song, J. Chen, X. Tan, D. Li, and D. Yang. EvoAgent: Towards automatic multi- agent generation via evolutionary algorithms. arXiv preprint arXiv:2406.14228, 2024. · arXiv
  41. K. Zhang, D. Wang, J. Xia, W. Y. Wang, and L. Li. ALGO: Synthesizing algorithmic programs with LLM-generated oracle verifiers. In Proceedings of the 37th International Conference on Neural Information Processing Systems, pages 54769- 54784, 2023.
  42. H. S. Zheng, S. Mishra, H. Zhang, X. Chen, M. Chen, A. Nova, L. Hou, H.-T. Cheng, Q. V. Le, E. H. Chi, et al. NATURAL PLAN: Bench- marking LLMs on natural language planning. arXiv preprint arXiv:2406.04520, 2024. · arXiv

Cited by

Sorted by citations

Authors
Lee, K.-H., Fischer, I., Wu, Y.-H., Marwood, D., Baluja, S., Schuurmans, D., & Chen, X.
Affiliations
  • University of California San Diego (US)
  • University of Alberta (CA)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2501.09891
arXiv
2501.09891
OpenAlex
W4406603682
Cited by
2
References
0
Citations by year
2025: 2
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
DeepMind
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
large language models, inference cost reduction, evolutionary search, evolution of the mind, tourism planning, best-of-N sampling

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review