Grounded language acquisition through the eyes and ears of a single child
The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Starting around 6 to 9 months of age, children begin acquiring their first words, linking spoken words to their visual counterparts. How much of this knowledge is learnable from sensory input with relatively generic learning mechanisms, and how much requires stronger inductive biases? Using longitudinal head-mounted camera recordings from one child aged 6 to 25 months, we trained a relatively generic neural network on 61 hours of correlated visual-linguistic data streams, learning feature-based representations and cross-modal associations. Our model acquires many word-referent mappings present in the child’s everyday experience, enables zero-shot generalization to new visual referents, and aligns its visual and linguistic conceptual systems. These results show how critical aspects of grounded word meaning are learnable through joint representation and associative learning from one child’s input.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Landauer, T. K., & Dumais, S. T. (1997) · Psychological Review · doi:10.1037/0033-295x.104.2.211 · Cited by 6,282
Emerging Properties in Self-Supervised Vision Transformers
Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., & Joulin, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00951 · Cited by 5,638
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
Jia, C., Yang, Y., Xia, Y., Chen, Y.-T., Parekh, Z., Pham, H., Le, Q. V., Sung, Y., Li, Z., & Duerig, T. (2021) · arXiv · doi:10.48550/arxiv.2102.05918 · Cited by 1,977
Joint attention and early language
Tomasello, M., & Farrar, M. J. (1986) · Child Development · doi:10.2307/1130423 · Cited by 1,822
The Structural Sources of Verb Meanings
Gleitman, L. (1990) · Language Acquisition · doi:10.1207/s15327817la0101_2 · Cited by 1,783
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555
Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., & Brendel, W. (2018) · arXiv · doi:10.48550/arxiv.1811.12231 · Cited by 1,490 · cited 1×
“Sub-sequent research could systematically test for their contributions on top of CVCL, by incorporating these biases into the architecture or training procedure ( 59 , 60 ).”
The importance of shape in early lexical learning
Landau, B., Smith, L. B., & Jones, S. S. (1988) · Cognitive Development · doi:10.1016/0885-2014(88)90014-7 · Cited by 1,174
The adaptive nature of human categorization.
Anderson, J. R. (1991) · Psychological Review · doi:10.1037/0033-295x.98.3.409 · Cited by 1,069
Statistical learning of new visual feature combinations by infants
Fiser, J., & Aslin, R. N. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.232472899 · Cited by 637
Learning words from sights and sounds: A computational model
Roy, D. K., & Pentland, A. P. (2002) · Cognitive Science · doi:10.1207/s15516709cog2601_4 · Cited by 521
Mapping the Early Language Environment Using All-Day Recordings and Automated Analysis
Gilkerson, J., Richards, J. A., Warren, S. F., Montgomery, J. K., Greenwood, C. R., Kimbrough Oller, D., Hansen, J. H. L., & Paul, T. D. (2017) · American Journal of Speech-Language Pathology · doi:10.1044/2016_ajslp-15-0169 · Cited by 472
Word meaning in minds and machines.
Lake, B. M., & Murphy, G. L. (2023) · Psychological Review · doi:10.1037/rev0000297 · Cited by 131
Self-citation
Cited by
Conwell, C., Prince, J. S., Kay, K. N., Alvarez, G. A., & Konkle, T. (2022) · doi:10.1101/2022.03.28.485868 · Cited by 160
Conwell, C., Prince, J. S., Kay, K. N., Alvarez, G. A., & Konkle, T. (2024) · Nature Communications · doi:10.1038/s41467-024-53147-y · Cited by 100
The Umwelt Representation Hypothesis: rethinking Universality
Bosch, V., Sommers, R. P., Doerig, A., & Kietzmann, T. C. (2026) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2026.07.004 · Cited by 2
Metadata
- Authors
- Vong, W. K., Wang, W., Orhan, A. E., & Lake, B. M.
- Venue
- Science
- Published
- 2024-02-02
- Volume
- 383
- Issue
- 6682
- Pages
- 504-511
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W4391428296
- Cited by
- 104
- References
- 17
- FWCI
- 80.28
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.