CoChat

Evolution Through Large Models

Open access
Lehman, Joel, Gordon, Jonathan, Jain, Shawn, Ndousse, Kamal, Yeh, Cathy, Stanley, Kenneth O. · 2024 · Genetic and Evolutionary Computation

The DOI matches its Crossref record; 43 of the 47 references it cites were found in a registry or an index.

Overview

Integrity signals

Registry

Crossref · 10.1007/978-981-99-3814-8_11

Cites

43 of 43 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 43 references

Research works printed but not linked: 23

  1. Baker, B., Kanitscheider, I., Markov, T., Wu, Y., Powell, G., McGrew, B., Mordatch, I.: Emergent tool use from multi-agent autocurricula (2019). arXiv:1909.07528 · arXiv
  2. Banzhaf, W., Nordin, P., Keller, R.E., Francone, F.D.: Genetic Programming - An Introduction. Morgan Kaufmann Publishers Inc. (1998)
  3. Bentley, P.J., Kumar, S.: Three ways to grow designs: a comparison of embryogenies for an evolutionary design problem. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-1999), pp. 35–43 (1999)
  4. Bongard, J.C., Pfeifer, R.: Repeated structure and dissociation of genotypic and phenotypic complexity in artificial ontogeny. In: Spector, L., Goodman, E.D., Wu, A., Langdon, W.B., Voigt, H.-M., Gen, M., Sen, S., Dorigo, M., Pezeshk, S., Garzon, M.H., Burke, E. (eds.) Genetic and Evolutionary Computation Conference, pp. 829–836 (2001)
  5. Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners (2020). arXiv:2005.14165 · arXiv
  6. Chen, L., Lu, K., Rajeswaran, A., Lee, K., Grover, A., Laskin, M., Abbeel, P., Srinivas, A., Mordatch, I.: Decision transformer: reinforcement learning via sequence modeling (2021). arXiv:2106.01345 · arXiv
  7. Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H.P., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al.: Evaluating large language models trained on code (2021). arXiv:2107.03374 · arXiv
  8. Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H.W., Sutton, C., Gehrmann, S., et al.: Palm: scaling language modeling with pathways (2022). arXiv:2204.02311 · arXiv
  9. Dennis, M., Jaques, N., Vinitsky, E., Bayen, A., Russell, S., Critch, A., Levine, S.: Emergent complexity and zero-shot transfer via unsupervised environment design. Adv. Neural Inf. Process. Syst. 33, 13049–13061 (2020)
  10. Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding (2018). arXiv:1810.04805 · arXiv
  11. Jonyer, I., Himes, A.: Improving modularity in genetic programming using graph-based data mining. In: FLAIRS Conference, pp. 556–561 (2006)
  12. Koza, J.R., Keane, M.A., Streeter, M.J., Mydlowec, W., Yu, J., Lanza, G.: Genetic Programming IV: Routine human-competitive machine intelligence, vol. 5. Springer Science & Business Media, 2006
  13. Langdon, W.B., Poli, R.: Foundations of Genetic Programming. Springer Science & Business Media (2013)
  14. Lehman, J., Gordon, J., Jain, S., Ndousse, K., Yeh, C., Stanley, K.O.: Evolution through large models (2022). arxiv:2206.08896 · arXiv
  15. Li, R., Allal, L.B., Zi, Y., Muennighoff, N., Kocetkov, D., Mou, C., Marone, M., Akiki, C., Li, J., Chim, J., et al.: Starcoder: may the source be with you! (2023). arXiv:2305.06161 · arXiv
  16. Mouret, J.-B., Clune, J.: Illuminating search spaces by mapping elites (2015). arXiv:1504.04909 · arXiv
  17. Schulman, J., Levine, S., Abbeel, P., Jordan, M., Moritz, P.: Trust region policy optimization. In: International Conference on Machine Learning, pp. 1889–1897 (2015)
  18. Schulman, J., Moritz, P., Levine, S., Jordan, M., Abbeel, P.: High-dimensional continuous control using generalized advantage estimation (2015). arXiv:1506.02438 · arXiv
  19. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms (2017). arXiv:1707.06347 · arXiv
  20. Seront, G.: External concepts reuse in genetic programming. In: Working Notes for the AAAI Symposium on Genetic Programming, pp. 94–98. MIT/AAAI Cambridge (1995)
  21. Taylor, T.: Exploring the concept of open-ended evolution. In: Artificial Life 13 (Proceedings of the Thirteenth International Conference on the Simulation and Synthesis of Living Systems), Cambridge, MA, pp. 540–541. MIT Press (2012)
  22. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Adv. Neural Inf. Proc. Syst. 30 (2017)
  23. Wang, R., Lehman, J., Rawal, A., Zhi, J., Li, Y., Clune, J., Stanley, K.O.: Enhanced poet: open-ended reinforcement learning through unbounded invention of learning challenges and their solutions. In: International Conference on Machine Learning, pp. 9940–9951. PMLR (2020) · DOI · Paper page

Cited by

See all 85

Metadata

Authors
Lehman, J., Gordon, J., Jain, S., Ndousse, K., Yeh, C., & Stanley, K. O.
Affiliations
  • OpenAI (United States) (US)
  • Anthropic (United States) (US)
Corresponding author
Lehman, Joel
Venue
Genetic and Evolutionary Computation
Published
2023-11-02
Pages
331-366
ISSN
  • 1932-0167 (ISSN-L)
  • 1932-0175
OpenAlex
W4388139328
Cited by
74
References
47
Citations by year
2026: 17 · 2025: 23 · 2024: 22 · 2023: 10
FWCI
39.94
Type
book chapter
Language
en
Open access
Licence
Locations
  • Genetic and evolutionary computation (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
Keywords
large language models, code generation, mutation operator, genetic programming, MAP-Elites, bootstrapping, open-endedness, zero-shot learning, Python

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Evolution Through Large Models. https://app.cochat.ai/papers/10.1007/978-981-99-3814-8_11
The paper
Lehman, J., Gordon, J., Jain, S., Ndousse, K., Yeh, C., & Stanley, K. O. (2024). Evolution Through Large Models. Genetic and Evolutionary Computation. https://doi.org/10.1007/978-981-99-3814-8_11

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.