Transmission Versus Truth, Imitation Versus Innovation: What Children Can Do That Large Language and Language-and-Vision Models Cannot (Yet)
Open accessThe DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Much discussion about large language models and language-and-vision models has focused on whether these models are intelligent agents. We present an alternative perspective. First, we argue that these artificial intelligence (AI) models are cultural technologies that enhance cultural transmission and are efficient and powerful imitation engines. Second, we explore what AI models can tell us about imitation and innovation by testing whether they can be used to discover new tools and novel causal structures and contrasting their responses with those of human children. Our work serves as a first step in determining which particular representations and competences, as well as which kinds of knowledge or skill, can be derived from particular learning techniques and data. In particular, we explore which kinds of cognitive capacities can be enabled by statistical analysis of large-scale linguistic data. Critically, our findings suggest that machines may need more than large-scale language and image data to allow the kinds of innovation that a small child can produce.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
(2010) · Intellectum (Universidad de La Sabana) · doi:10.1111/j.1467- · Cited by 16,131
Landauer, T. K., & Dumais, S. T. (1997) · Psychological Review · doi:10.1037/0033-295x.104.2.211 · Cited by 6,282
Barsalou, L. W. (2008) · Annual Review of Psychology · doi:10.1146/annurev.psych.59.103006.093639 · Cited by 5,936 · cited 1×
“…in general, human or artificial, can be acquired purely through cultural transmission itself and which require independent contact with the external world—a long-standing question in cognitive science (Barsalou, 2008; Gibson, 1979; Grand et al., 2022; Landauer & Dumais, 1997; Piantadosi, 2023).”
BackgroundStatistical learning by 8-month-old infants
Saffran, J. R., Aslin, R. N., & Newport, E. L. (1996) · Science · doi:10.1126/science.274.5294.1926 · Cited by 5,659
The standard definition of creativity
Runco, M. A., & Jaeger, G. J. (2012) · Creativity Research Journal · doi:10.1080/10400419.2012.650092 · Cited by 3,895
Human-level concept learning through probabilistic program induction
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015) · Science · doi:10.1126/science.aab3050 · Cited by 2,817 · cited 1×
“Similarly, a number of AI approaches have integrated causal inference and theory formation into learning mechanisms in an attempt to design more human-like systems (Goyal & Bengio, 2022; Lake et al., 2015; Pearl, 2000).”
BackgroundPaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
Tomasello, M., Kruger, A. C., & Ratner, H. H. (1993) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0003123x · Cited by 1,891
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.12712 · Cited by 1,526
Rational imitation in preverbal infants
Gergely, G., Bekkering, H., & Király, I. (2002) · Nature · doi:10.1038/415755a · Cited by 1,391
The cultural niche: Why social learning is essential for human adaptation
Boyd, R., Richerson, P. J., & Henrich, J. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1100290108 · Cited by 1,303 · cited 1×
“Imitation allows the transmission of knowledge or skill from one person to another (Boyd et al., 2011; Henrich, 2016).”
BackgroundSocial intelligence, innovation, and enhanced brain size in primates
Reader, S. M., & Laland, K. N. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.062041299 · Cited by 1,126 · cited 1×
“Tool innovation has often been taken to be a distinctive mark of intelligence in biological systems (Emery & Clayton, 2004; Reader & Laland, 2002).”
BackgroundVisual statistical learning in infancy: evidence for a domain general learning mechanism
Kirkham, N. Z., Slemmer, J. A., & Johnson, S. P. (2002) · Cognition · doi:10.1016/s0010-0277(02)00004-5 · Cited by 1,072 · cited 1×
“In particular, this discovery has interesting connections to the large body of empirical literature showing that infants are sensitive to the statistical structure of linguistic strings and visual images from a very young age (e.g., Kirkham et al., 2002; Saffran et al., 1996).”
BackgroundMotor Development: Embodied, Embedded, Enculturated, and Enabling
Adolph, K. E., & Hoch, J. E. (2019) · Annual Review of Psychology · doi:10.1146/annurev-psych-010418-102836 · Cited by 580
Using cognitive psychology to understand GPT-3
Binz, M., & Schulz, E. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2218523120 · Cited by 559 · cited 2×
“We set model outputs as deterministic with a temperature of 0 and kept the default values for all other parameters (Binz & Schulz, 2023; Hu et al., 2022).”
BackgroundMethodImitation and Innovation: The Dual Engines of Cultural Learning
Legare, C. H., & Nielsen, M. (2015) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2015.08.005 · Cited by 484
Dissociating language and thought in large language models
Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.01.011 · Cited by 345
Evaluating Large Language Models in Theory of Mind Tasks
Kosinski, M. (2023) · arXiv · doi:10.48550/arxiv.2302.02083 · Cited by 339
Language Models (Mostly) Know What They Know
Kadavath, S., Conerly, T., Askell, A., Henighan, T., Drain, D., Perez, E., Schiefer, N., Hatfield-Dodds, Z., DasSarma, N., Tran-Johnson, E., et al. (2022) · arXiv · doi:10.48550/arxiv.2207.05221 · Cited by 187 · cited 1×
“Even though state-of-the-art LLMs have been trained to estimate uncertainty over the validity of their claims (Kadavath et al., 2022), their output prediction probabilities do not distinguish between epistemic uncertainty, which relates to the lack of knowledge and can be resolved with more…”
BackgroundLarge Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks
Ullman, T. (2023) · arXiv · doi:10.48550/arxiv.2302.08399 · Cited by 81 · cited 1×
“…available in the training data, but often fail when they are asked to make inferences that involve novel events or relations in human thought (e.g., Binz & Schulz, 2023; Mahowald et al., 2023), sometimes even when these involve superficially slight changes to the training data (e.g., Ullman, 2023).”
BackgroundA fine-grained comparison of pragmatic language understanding in humans and language models
Hu, J., Floyd, S., Jouravlev, O., Fedorenko, E., & Gibson, E. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.230 · Cited by 78
How do we know how smart AI systems are?
Mitchell, M. (2023) · Science · doi:10.1126/science.adj5957 · Cited by 71
On the Paradox of Learning to Reason from Data
Zhang, H., Li, L. H., Meng, T., Chang, K.-W., & Van den Broeck, G. (2023) · Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/375 · Cited by 58
On the Paradox of Learning to Reason from Data
Zhang, H., Li, L. H., Meng, T., Chang, K.-W., & Broeck, G. V. D. (2022) · arXiv · doi:10.48550/arxiv.2205.11502 · Cited by 58 · cited 1×
“…almost perfectly (again reinforcing that they are perfect imitators) but then fail miserably to generalize to out-of-distribution examples that require the discovery of more abstract causal hypotheses and innovation (e.g., Chowdhery et al., 2022; Talmor et al., 2020; H. Zhang et al., 2022).”
BackgroundPutting GPT-3's Creativity to the (Alternative Uses) Test
Stevenson, C., Smal, I., Baas, M., Grasman, R., & van der Maas, H. (2022) · arXiv · doi:10.48550/arxiv.2206.08932 · Cited by 51
Cited by
Dissociating language and thought in large language models
Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.01.011 · Cited by 345
Testing theory of mind in large language models and humans
Strachan, J. W. A., Albergo, D., Borghini, G., Pansardi, O., Scaliti, E., Gupta, S., Saxena, K., Rufo, A., Panzeri, S., Manzi, G., et al. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01882-z · Cited by 299
Theory Is All You Need: AI, Human Cognition, and Causal Reasoning
Felin, T., & Holweg, M. (2024) · doi:10.2139/ssrn.4737265 · Cited by 152
Theory Is All You Need: AI, Human Cognition, and Causal Reasoning
(2024) · doi:10.1287/stsc.2024.0189 · Cited by 151
Hagendorff, T., Dasgupta, I., Binz, M., Chan, S. C. Y., Lampinen, A., Wang, J. X., Akata, Z., & Schulz, E. (2023) · arXiv · doi:10.48550/arxiv.2303.13988 · Cited by 68
Metadata
- Authors
- Yiu, E., Kosoy, E., & Gopnik, A.
- Venue
- Perspectives on Psychological Science
- Publisher
- SAGE Publications
- Published
- 2023-10-26
- Volume
- 19
- Issue
- 5
- Pages
- 874-883
- ISSN
- 1745-6916 (ISSN-L)
- 1745-6924
- OpenAlex
- W4387966945
- Cited by
- 48
- References
- 26
- FWCI
- 76.03
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.