Neural network processing of natural language: I. Sensitivity to serial, temporal and abstract structure of language in the infant
The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Well before their first birthday, babies can acquire knowledge of serial order relations (Saffran et al., 1996a), as well as knowledge of more abstract rule-based structural relations (Marcus et al., 1999) between neighbouring speech sounds within 2 minutes of exposure. These early learners can likewise acquire knowledge of rhythmic or temporal structure of a new language within 5-10 minutes of exposure (Nazzi et al., 1998). All three of these types of knowledge likely play invaluable roles in "bootstrapping" language acquisition. Two important open questions that remain include: What are the mechanisms that provide this rapid learning ability, and how do they depend on pre-exposure to the environment? Here we show that a neurophysiologically validated temporal recurrent network simulates babies' capabilities to learn serial order and rhythmic structure. Indeed the recurrent network is capable of representing serial and temporal structure with no pre-exposure, and through exposure these internal representations can become bound to behavioural responses. In contrast, babies' performance in extracting abstract structure can only be simulated by a modified version of the model. We thus demonstrate how innate representational capabilities for serial and temporal structure of language could arise from a common neural architecture, distinct from that required for the representation of abstract structure, and we provide a predictive testable model of at least these aspects of the initial computational state of the language learner.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical learning by 8-month-old infants
Saffran, J. R., Aslin, R. N., & Newport, E. L. (1996) · Science · doi:10.1126/science.274.5294.1926 · Cited by 5,659
Harris, Z. S. (1954) · WORD · doi:10.1080/00437956.1954.11659520 · Cited by 4,807
Learning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 1×
“Given their general regularity detection capabilities, they can help discover new and correlated regularities in the input (Christiansen, Allen, & Seidenberg, 1998; Elman, 1990, 1991, 1993; Seidenberg, 1997).”
BackgroundCircuitry of Primate Prefrontal Cortex and Regulation of Behavior by Representational Memory
Goldman‐Rakic, P. S. (1987) · Comprehensive Physiology · doi:10.1002/cphy.cp010509 · Cited by 1,960 · cited 3×
“This recurrent state network was modelled after primate frontal cortex with its recurrent corticocortical connections (Goldman-Rakic, 1987), and allowed us to explain the electrophysiological encoding of visual space and sequential context (Dominey et al.”
InfluentialRethinking Innateness: A Connectionist Perspective on Development
Elman, J., Bates, E., Johnson, M. H., Karmiloff-Smith, A., Parisi, D., & Plunkett, K. (1996) · doi:10.7551/mitpress/5929.001.0001 · Cited by 1,511
Word Segmentation: The Role of Distributional Cues
Saffran, J. R., Newport, E. L., & Aslin, R. N. (1996) · Journal of Memory and Language · doi:10.1006/jmla.1996.0032 · Cited by 1,405 · cited 21×
“As noted by Saffran et al. (1996a) this problem could be solved by learning the serial order of events presented during the two minute training, and then recognising these sequences during testing, since the TABLE 1”
BackgroundMethodResultInfluentialDistributed representations, simple recurrent networks, and grammatical structure
Elman, J. L. (1991) · Machine Learning · doi:10.1007/bf00114844 · Cited by 1,354
Infants′ Detection of the Sound Patterns of Words in Fluent Speech
Jusczyk, P. W., & Aslin, R. N. (1995) · Cognitive Psychology · doi:10.1006/cogp.1995.1010 · Cited by 1,061
Responses of monkey dopamine neurons during learning of behavioral reactions
Ljungberg, T., Apicella, P., & Schultz, W. (1992) · Journal of Neurophysiology · doi:10.1152/jn.1992.67.1.145 · Cited by 987
Synaptic development of the cerebral cortex: implications for learning, memory and mental illness
(1994) · Progress in Brain Research · doi:10.1016/s0079-6123(08)60543-9 · Cited by 565
Distributional Information: A Powerful Cue for Acquiring Syntactic Categories
Redington, M., Chater, N., & Finch, S. (1998) · Cognitive Science · doi:10.1207/s15516709cog2204_2 · Cited by 535
Distributional regularity and phonotactic constraints are useful for segmentation
Brent, M. R., & Cartwright, T. A. (1996) · Cognition · doi:10.1016/s0010-0277(96)00719-6 · Cited by 514 · cited 4×
“Peter Ford Dominey”
BackgroundInfluentialLearning to Segment Speech Using Multiple Cues: A Connectionist Model
Christiansen, M. H., Allen, J., & Seidenberg, M. S. (1998) · Language and Cognitive Processes · doi:10.1080/016909698386528 · Cited by 356 · cited 7×
“In a number of recurrent network studies, learning modi es the recurrent connections (Christiansen et al., 1998; Cleeremans & McClelland, 1991; Elman, 1990).”
BackgroundMethodInfluentialLanguage Acquisition and Use: Learning and Applying Probabilistic Constraints
Seidenberg, M. S. (1997) · Science · doi:10.1126/science.275.5306.1599 · Cited by 338
Harris, Z. S. (1955) · Language · doi:10.2307/411036 · Cited by 256 · cited 1×
“Rhythm class: According to Pike (1945) and Abercrombie (1967).”
BackgroundDo infants perceive word boundaries? An empirical study of the bootstrapping of lexical acquisition
Christophe, A., Dupoux, E., Bertoncini, J., & Mehler, J. (1994) · The Journal of the Acoustical Society of America · doi:10.1121/1.408544 · Cited by 233
Cited by
Understanding and sharing intentions: The origins of cultural cognition
Tomasello, M., Carpenter, M., Call, J., Behne, T., & Moll, H. (2005) · Behavioral and Brain Sciences · doi:10.1017/s0140525x05000129 · Cited by 4,882
Reservoir computing approaches to recurrent neural network training
Lukoševičius, M., & Jaeger, H. (2009) · Computer Science Review · doi:10.1016/j.cosrev.2009.03.005 · Cited by 2,956
Recent advances in physical reservoir computing: A review
Tanaka, G., Yamane, T., Héroux, J. B., Nakane, R., Kanazawa, N., Takeda, S., Numata, H., Nakano, D., & Hirose, A. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.03.005 · Cited by 2,146
A Practical Guide to Applying Echo State Networks
Lukoševičius, M. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_36 · Cited by 867
Photonic matrix multiplication lights up photonic accelerator and beyond
Zhou, H., Dong, J., Cheng, J., Dong, W., Huang, C., Shen, Y., Zhang, Q., Gu, M., Qian, C., Chen, H., et al. (2022) · Light: Science & Applications · doi:10.1038/s41377-022-00717-8 · Cited by 563
Metadata
- Authors
- Dominey, P. F., & Ramus, F.
- Venue
- Language and Cognitive Processes
- Publisher
- Informa UK Limited
- Published
- 2000-02
- Volume
- 15
- Issue
- 1
- Pages
- 87-127
- ISSN
- 0169-0965 (ISSN-L)
- 1464-0732
- OpenAlex
- W2126357538
- Cited by
- 94
- References
- 16
- FWCI
- 12.44
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.