CoChat
Sign Up
PreprintCited by 3

The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.

Large Language Models and Emergence: A Complex Systems Perspective

Krakauer, David C., Krakauer, John W., Mitchell, Melanie·2025·arXiv
DOI 10.48550/arxiv.2506.11135arXiv 2506.11135
Open this paper in CoChat — free account

Abstract

Emergence is a concept in complexity science that describes how many-body systems manifest novel higher-level properties, properties that can be described by replacing high-dimensional mechanisms with lower-dimensional effective variables and theories. This is captured by the idea "more is different". Intelligence is a consummate emergent property manifesting increasingly efficient -- cheaper and faster -- uses of emergent capabilities to solve problems. This is captured by the idea "less is more". In this paper, we first examine claims that Large Language Models exhibit emergent capabilities, reviewing several approaches to quantifying emergence, and secondly ask whether LLMs possess emergent intelligence.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2506.11135

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

  • Scaling Laws for Neural Language Models

    Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550 · cited 1×

    “For many tasks, at certain scales each model shows a sharp increase in accuracy, an increase that was not predicted by earlier empirical “scaling laws” [5].”

Flagged first, then by citations

Research works printed but not linked: 57

  1. Leonardo Berti, Flavio Giorgi, and Gjergji Kasneci. Emergent abilities in large language models: A survey. arXiv:2503.05788, 2025. · arXiv
  2. Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo. Are emergent abil- ities of large language models a mirage? Advances in Neural Information Processing Systems, 36:55565-55581, 2023.
  3. Sheng Lu, Irina Bigoulaeva, Rachneet Sachdeva, Harish Tayyar Madabushi, and Iryna Gurevych. Are emergent abilities in large language models just in-context learning? In Proceedings of The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), 2024.
  4. David Noever and Forrest McKee. Numeracy from literacy: Data science as an emergent skill from large language models. arXiv:2301.13382, 2023. · arXiv
  5. Taylor Webb, Keith J. Holyoak, and Hongjing Lu. Emergent analogical reasoning in large language models. Nature Human Behaviour, 7(9):1526- 1541, 2023.
  6. John J. Nay, David Karamardian, Sarah B. Lawsky, Wenting Tao, Meghana Bhat, Raghav Jain, Aaron Travis Lee, Jonathan H. Choi, and Jungo Ka- sai. Large language models as tax attorneys: A case study in legal ca- pabilities emergence. Philosophical Transactions of the Royal Society A, 382(2270):20230159, 2024.
  7. Kenneth Li, Aspen K Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Emergent world representations: Explor- ing a sequence model trained on a synthetic task. In Proceedings of the International Conference on Learning Representations (ICLR-2023), 2023.
  8. Anna Rogers and Alexandra S. Luccioni. Position: Key claims in LLM re- search have a long tail of footnotes. In Proceedings of the 41st International Conference on Machine Learning (ICML 2024), 2024.
  9. Mark A. Bedau and Paul Humphreys. Emergence: Contemporary Readings in Philosophy and Science. MIT Press, March 2008.
  10. Sophie Gibb, Robin F. Hendry, and Tom Lancaster. The Routledge Hand- book of Emergence. Routledge, March 2019.
  11. David J. Chalmers. Strong and weak emergence. In Phillip Claton and Paul Davies, editors, The Re-Emergence of Emergence. Oxford University Press, New York, 2006.
  12. Sean M. Carroll and Achyuth Parola. What emergence can possibly mean. arXiv:2410.15468, October 2024. · arXiv
  13. Navot Israeli and Nigel Goldenfeld. Coarse-graining of cellular automata, emergence, and the predictability of complex systems. Physical Review E-Statistical, Nonlinear, and Soft Matter Physics, 73(2):026203, 2006.
  14. P. W. Anderson. More is different. Science, 177(4047):393-396, August 1972.
  15. Wesley C. Salmon. Statistical Explanation and Statistical Relevance, vol- ume 69. University of Pittsburgh Press, 2010.
  16. David C. Krakauer. Symmetry-simplicity, broken symmetry-complexity. Interface Focus, 13(3):20220075, 2023.
  17. David C. Krakauer. The Complex World. Santa Fe Institute Press, Santa Fe, NM, 2024.
  18. Richard P. Feynman, Robert B. Leighton, and Matthew L. Sands. The Feynman Lectures on Physics. Addison-Wesley Publishing Company, 1963.
  19. Stuart A. Newman. Development and evolution: The physics connection. In Alan C. Love, editor, Conceptual Change in Biology: Scientific and Philosophical Perspectives on Evolution and Development, pages 421-440. Springer Netherlands, Dordrecht, 2015.
  20. A. M. Turing. The chemical basis of morphogenesis. Philosophical Trans- actions of the Royal Society, 237(641):37-72, 1952.
  21. Craig W. Reynolds. Flocks, herds and schools: A distributed behavioral model. In Proceedings of the 14th Annual Conference on Computer Graph- ics and Interactive Techniques, 1987.
  22. R. Linsker. Perceptual neural organization: Some approaches based on network models and information theory. Annual Reviews of Neuroscience, 13(1):257-281, 1990.
  23. J. J. Hopfield. Neural networks and physical systems with emergent collec- tive computational abilities. Proc. Natl. Acad. Sci. U. S. A., 79(8):2554- 2558, April 1982.
  24. Ricard V. Solé. Phase Transitions. Primers in Complex Systems. Princeton University Press, Princeton, NJ, 2011.
  25. Jessica C. Flack, Doug Erwin, Tanya Elliot, and David C. Krakauer. Timescales, symmetry, and uncertainty reduction in the origins of hier- archy in biological systems. Evolution Cooperation and Complexity, pages 45-74, 2013.
  26. Heinz G. Schuster. Criticality in Neural Systems. John Wiley & Sons, 2014.
  27. James P. O'Dwyer, Steven W. Kembel, and Thomas J. Sharpton. Back- bones of evolutionary history test biodiversity theory for microbes. Pro- ceedings of the National Academy of Sciences, 112(27):8356-8361, 2015.
  28. James H. Brown and Geoffrey B. West. Scaling in Biology. Oxford Univer- sity Press, 2000.
  29. Stefan Thurner, Rudolf Hanel, and Peter Klimek. Introduction to the The- ory of Complex Systems. Oxford University Press, 2018.
  30. Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Ilya Sutskever. Deep double descent: Where bigger models and more data hurt. Journal of Statistical Mechanics: Theory and Experiment, 2021(12):124003, 2021.
  31. Alicia Curth, Alan Jeffares, and Mihaela van der Schaar. A u-turn on double descent: Rethinking parameter counting in statistical learning. Ad- vances in Neural Information Processing Systems, 36:55932-55962, 2023.
  32. Rylan Schaeffer, Mikail Khona, Zachary Robertson, Akhilan Boopathy, Kateryna Pistunova, Jason W. Rocks, Ila Rani Fiete, and Oluwasanmi Koyejo. Double descent demystified: Identifying, interpreting & ablating the sources of a deep learning puzzle. arXiv preprint arXiv:2303.14151, 2023. · arXiv
  33. Angelica Chen, Ravid Shwartz-Ziv, Kyunghyun Cho, Matthew L. Leavitt, and Naomi Saphra. Sudden drops in the loss: Syntax acquisition, phase transitions, and simplicity bias in mlms. arXiv preprint arXiv:2309.07311, 2023. · arXiv
  34. Kai Nakaishi, Yoshihiko Nishikawa, and Koji Hukushima. Critical phase transition in large language models. arXiv preprint arXiv:2406.05335, 2024. · arXiv
  35. Bruno A. Olshausen and David J. Field. Sparse coding of sensory inputs. Current Opinion in Neurobiology, 14(4):481-487, 2004.
  36. Charles F. Stevens. What the fly's nose tells the fly's brain. Proceedings of the National Academy of Sciences, 112(30):9460-9465, 2015.
  37. Neel Nanda, Andrew Lee, and Martin Wattenberg. Emergent lin- ear representations in world models of self-supervised sequence models. arXiv:2309.00941, 2023. · arXiv
  38. Matthias Kaschube, Michael Schnabel, and Fred Wolf. Self-organization and the selection of pinwheel density in visual cortical development. New Journal of Physics, 10(1):015009, 2008.
  39. Daniel Polani. Foundations and formalizations of self-organization. Ad- vances in Applied Self-Organizing Systems, pages 19-37, 2008.
  40. Hujun Yin. Learning nonlinear principal manifolds by self-organising maps. In Principal Manifolds for Data Nisualization and Dimension Reduction, pages 68-95. Springer, 2008.
  41. Quentin RV. Ferry, Joshua Ching, and Takashi Kawai. Emergence and function of abstract representations in self-supervised transformers. arXiv preprint arXiv:2312.05361, 2023. · arXiv
  42. Florentin Guth, Brice Ménard, Gaspar Rochette, and Stéphane Mallat. A rainbow in deep network black boxes. Journal of Machine Learning Research, 25(350):1-59, 2024.
  43. Vladimir Vapnik, Esther Levin, and Yann LeCun. Measuring the vc- dimension of a learning machine. Neural Computation, 6(5):851-876, 1994.
  44. Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. GSM-Symbolic: Understand- ing the limitations of mathematical reasoning in large language models. arXiv:2410.05229, 2024. · arXiv
  45. Martha Lewis and Melanie Mitchell. Evaluating the robustness of analogical reasoning in GPT models. Transactions on Machine Learning Research, 2024.
  46. Toni J. B. Liu, Nicolas Boullé, Raphaël Sarfati, and Christopher J. Earls. Llms learn governing principles of dynamical systems, revealing an in- context neural scaling law. arXiv preprint arXiv:2402.00795, 2024. · arXiv
  47. Keyon Vafa, Justin Chen, Ashesh Rambachan, Jon Kleinberg, and Sendhil Mullainathan. Evaluating the world model implicit in a generative model. Advances in Neural Information Processing Systems, 37:26941-26975, 2024.
  48. François Chollet. On the measure of intelligence. arXiv:1911.01547, 2019. · arXiv
  49. Joseph Mazur. Enlightening symbols: A short history of mathematical no- tation and its hidden powers. Princeton University Press, 2014.
  50. Raúl Rojas. The Language of Mathematics: The Stories behind the Symbols. Princeton University Press, 2025.
  51. Hristos S Courellis, Juri Minxha, Araceli R Cardenas, Daniel L Kimmel, Chrystal M Reed, Taufik A Valiante, C Daniel Salzman, Adam N Mamelak, Stefano Fusi, and Ueli Rutishauser. Abstract representations emerge in human hippocampal neurons during inference. Nature, 632(8026):841-849, 2024.
  52. Pietro Mazzoni and John W Krakauer. An implicit plan overrides an explicit strategy during visuomotor adaptation. Journal of neuroscience, 26(14):3642-3645, 2006.
  53. M Felice Ghilardi, Clara Moisello, Giulia Silvestri, Claude Ghez, and John W Krakauer. Learning of a sequential motor skill comprises explicit and implicit components that consolidate differently. Journal of neurophys- iology, 101(5):2218-2229, 2009.
  54. Harry Manley, Peter Dayan, and Jörn Diedrichsen. When money is not enough: awareness, success, and variability in motor learning. PLoS One, 9(1):e86580, 2014.
  55. David C Krakauer, Jessica C Flack, Simon DeDeo, Doyne Farmer, and Daniel Rockmore. Intelligent data analysis of intelligent systems. In In- ternational Symposium on Intelligent Data Analysis, pages 8-17. Springer, 2010.
  56. Marijn JH Heule and Oliver Kullmann. The science of brute force. Com- munications of the ACM, 60(8):70-79, 2017.
  57. Harold Abelson and Gerald Jay Sussman. Structure and interpretation of computer programs. The MIT Press, 1996.

Cited by

Sorted by citations

Metadata

Authors
Krakauer, D. C., Krakauer, J. W., & Mitchell, M.
Venue
arXiv
Publisher
arXiv
Published
2025
DOI
10.48550/arxiv.2506.11135
arXiv
2506.11135
References
1
Type
Preprint
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review