Recurrent temporal networks and language acquisition—from corticostriatal neurophysiology to reservoir computing
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
One of the most paradoxical aspects of human language is that it is so unlike any other form of behavior in the animal world, yet at the same time, it has developed in a species that is not far removed from ancestral species that do not possess language. While aspects of non-human primate and avian interaction clearly constitute communication, this communication appears distinct from the rich, combinatorial and abstract quality of human language. So how does the human primate brain allow for language? In an effort to answer this question, a line of research has been developed that attempts to build a language processing capability based in part on the gross neuroanatomy of the corticostriatal system of the human brain. This paper situates this research program in its historical context, that begins with the primate oculomotor system and sensorimotor sequencing, and passes, via recent advances in reservoir computing to provide insight into the open questions, and possible approaches, for future research that attempts to model language processing. One novel and useful idea from this research is that the overlap of cortical projections onto common regions in the striatum allows for adaptive binding of cortical signals from distinct circuits, under the control of dopamine, which has a strong adaptive advantage. A second idea is that recurrent cortical networks with fixed connections can represent arbitrary sequential and temporal structure, which is the basis of the reservoir computing framework. Finally, bringing these notions together, a relatively simple mechanism can be built for learning the grammatical constructions, as the mappings from surface structure of sentences to their meaning. This research suggests that the components of language that link conceptual structure to grammatical structure may be much simpler that has been proposed in other research programs. It also suggests that part of the residual complexity is in the conceptual system itself.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025 · cited 3×
“Subsequent work with such systems demonstrated their vast computational power (Maass et al., 2002; Jaeger and Haas, 2004).”
BackgroundHarnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication
Jaeger, H., & Haas, H. (2004) · Science · doi:10.1126/science.1091277 · Cited by 3,811 · cited 2×
“Subsequent work with such systems demonstrated their vast computational power (Maass et al., 2002; Jaeger and Haas, 2004).”
BackgroundReservoir computing approaches to recurrent neural network training
Lukoševičius, M., & Jaeger, H. (2009) · Computer Science Review · doi:10.1016/j.cosrev.2009.03.005 · Cited by 2,956
Towards a neural basis of auditory sentence processing
Friederici, A. D. (2002) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01839-8 · Cited by 2,231 · cited 1×
“We (Dominey and Inui, 2009; Dominey et al., 2009) attempted to reconcile the corticostriatal model with mainstream neuro-physiological models of language processing (Friederici, 2002; Hagoort, 2005) in more detail.”
MethodHickok, G., & Poeppel, D. (2004) · Cognition · doi:10.1016/j.cognition.2003.10.011 · Cited by 2,212
Learning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 1×
“A vast and productive line of research has characterized language processing in terms of predicting the next word in a sentence, initiated by the ground-breaking work of Elman (1990, 1991, 1993), and fruitfully pursued by others (e.g., Christiansen and Chater, 1999).”
BackgroundContributions of memory circuits to language: the declarative/procedural model
Ullman, M. T. (2004) · Cognition · doi:10.1016/j.cognition.2003.10.008 · Cited by 1,893
The importance of mixed selectivity in complex cognitive tasks
Rigotti, M., Barak, O., Warden, M. R., Wang, X.-J., Daw, N. D., Miller, E. K., & Fusi, S. (2013) · Nature · doi:10.1038/nature12160 · Cited by 1,639 · cited 1×
“Recent studies of primate electrophysiology provide evidence that indeed, the PFC operates based on reservoir-like properties (Rigotti et al., 2010, 2013).”
BackgroundOn Broca, brain, and binding: A new framework
Hagoort, P. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.07.004 · Cited by 1,411 · cited 2×
“We (Dominey and Inui, 2009; Dominey et al., 2009) attempted to reconcile the corticostriatal model with mainstream neuro-physiological models of language processing (Friederici, 2002; Hagoort, 2005) in more detail.”
MethodResultDistributed representations, simple recurrent networks, and grammatical structure
Elman, J. L. (1991) · Machine Learning · doi:10.1007/bf00114844 · Cited by 1,354
Chunking mechanisms in human learning
GOBET, F., LANE, P., CROKER, S., CHENG, P., JONES, G., OLIVER, I., & PINE, J. (2001) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01662-4 · Cited by 1,157
Simulation, situated conceptualization, and prediction
Barsalou, L. W. (2009) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2008.0319 · Cited by 1,017
A neurocognitive perspective on language: The declarative/procedural model
Ullman, M. T. (2001) · Nature Reviews Neuroscience · doi:10.1038/35094573 · Cited by 997
Optimization and applications of echo state networks with leaky- integrator neurons
Jaeger, H., Lukoševičius, M., Popovici, D., & Siewert, U. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.016 · Cited by 948 · cited 1×
“Such feedback connections have been demonstrated to play an important role in memory and sequence processing (Jaeger and Haas, 2004; Jaeger et al., 2007).”
BackgroundThe cortical language circuit: from auditory perception to sentence comprehension
Friederici, A. D. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2012.04.001 · Cited by 748
Organization of the nigrothalamocortical system in the rhesus monkey
Ilinsky, I. A., Jouandet, M. L., & Goldman‐Rakic, P. S. (1985) · Journal of Comparative Neurology · doi:10.1002/cne.902360304 · Cited by 406 · cited 1×
“Essentially all of neocortex projects in a topographically organized manner to the striatum, through the pallidum to the thalamus and back to cortex (Ilinsky et al., 1985), thus yielding what can be considered as a set of largely distinct and segregated corticostriatal circuits or loops (see Figure…”
BackgroundCross-linguistic research in aphasia: An overview
Bates, E., Wulfeck, B., & MacWhinney, B. (1991) · Brain and Language · doi:10.1016/0093-934x(91)90149-u · Cited by 325
Rigotti, M. (2010) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2010.00024 · Cited by 184
Why the P600 is not just a P300: The role of the basal ganglia
Frisch, S., Kotz, S. A., von Cramon, D. Y., & Friederici, A. D. (2003) · Clinical Neurophysiology · doi:10.1016/s1388-2457(02)00366-8 · Cited by 162 · cited 1×
“Several studies from Friederici and Kotz (2003; Friederici et al., 2003; Frisch et al., 2003; Kotz et al., 2003) in patients with striatal dysfunction due to lesion or Parkinson’s disease demonstrate that the P600 ERP evoked by syntactic anomalies or complexity in normal controls subjects is…”
BackgroundTopology and dynamics of the canonical circuit of cat V1
Binzegger, T., Douglas, R. J., & Martin, K. A. C. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.07.011 · Cited by 108
Dominey, P. F., Inui, T., & Hoen, M. (2009) · Brain and Language · doi:10.1016/j.bandl.2008.08.002 · Cited by 64 · cited 4×
“We (Dominey and Inui, 2009; Dominey et al., 2009) attempted to reconcile the corticostriatal model with mainstream neuro-physiological models of language processing (Friederici, 2002; Hagoort, 2005) in more detail.”
BackgroundMethodInfluentialSelf-citation
Cited by
Recent advances in physical reservoir computing: A review
Tanaka, G., Yamane, T., Héroux, J. B., Nakane, R., Kanazawa, N., Takeda, S., Numata, H., Nakano, D., & Hirose, A. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.03.005 · Cited by 2,146
Degree of automaticity and the prefrontal cortex
Jeon, H.-A., & Friederici, A. D. (2015) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2015.03.003 · Cited by 60
“How do humans make sense?” multiscale dynamics and emergent meaning
Dale, R., & Kello, C. T. (2018) · New Ideas in Psychology · doi:10.1016/j.newideapsych.2017.09.002 · Cited by 15
Metadata
- Authors
- Dominey, P. F.
- Venue
- Frontiers in Psychology
- Publisher
- Frontiers Media SA
- Published
- 2013
- Volume
- 4
- Pages
- 500-500
- ISSN
- 1664-1078 (ISSN-L)
- OpenAlex
- W2115071721
- Cited by
- 36
- References
- 21
- FWCI
- 2.19
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.