Finding Structure in Time
The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Abstract
Time underlies many interesting human behaviors. Thus, the question of how to represent time in connectionist models is very important. One approach is to represent time implicitly by its effects on processing rather than explicitly (as in a spatial representation). The current report develops a proposal along these lines first described by Jordan (1986) which involves the use of recurrent links in order to provide networks with a dynamic memory. In this approach, hidden unit patterns are fed back to themselves: the internal representations which develop thus reflect task demands in the context of prior internal states. A set of simulations is reported which range from relatively simple problems (temporal version of XOR) to discovering syntactic/semantic features for words. The networks are able to learn interesting internal representations which incorporate task demands with memory demands: indeed, in this approach the notion of memory is inextricably bound up with task processing. These representations reveal a rich structure, which allows them to be highly context‐dependent, while also expressing generalizations across classes of items. These representations suggest a method for representing lexical categories and the type/token distinction.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Language models have been used to model aspects of human language learning and processing for decades (Elman, 1990;Hale, 2001;Reali and Christiansen, 2005, o.a.).”
From Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora (2023) · cites this paper as Elman, 1990
In the citing paper
“recent computational models of language acquisition suggest that at least some pertinent language-learning mechanisms may induce structure from the input using statistical methods similar to those suggested by incidental-learning research”
From Incidental Language Learning: Listening (and Learning) Out of the Corner of Your Ear (1997) · cites this paper as Elman, 1990
In the citing paper
“Infants automaticallyanaly ze simple distributional statistics in the language theyhear, to discover the word boundaries (Brent, 1999; Elman, 1990).”
From FREQUENCY EFFECTS IN LANGUAGE PROCESSING: A Review with Implications for Theories of Implicit and Explicit Language Acquisition (2002) · cites this paper as Elman, 1990
In the citing paper
“These part-ofspeech categories emerge from the language input without anysemantics or real-world grounding, and their categorystructure is hierarchical, soft, and implicit.”
From FREQUENCY EFFECTS IN LANGUAGE PROCESSING: A Review with Implications for Theories of Implicit and Explicit Language Acquisition (2002) · cites this paper as Elman (1990)
In the citing paper
“Other recurrent connectionist models of language include those of Elman (1990) and Redington and Chater (1998), which demonstrate how learning the grammatical categories and requirements of words and word groups reduces to the analysis of the sequence in which words work in chunks.”
From FREQUENCY EFFECTS IN LANGUAGE PROCESSING: A Review with Implications for Theories of Implicit and Explicit Language Acquisition (2002) · cites this paper as Elman (1990)
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,172
Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations
Rumelhart, D. E., McClelland, J. L., & AU (1986) · doi:10.7551/mitpress/5236.001.0001 · Cited by 15,686
ASPECTS OF THE THEORY OF SYNTAX
Chomsky, N. (1964) · doi:10.21236/ad0616323 · Cited by 13,587
Chomsky, N. (1957) · doi:10.1515/9783112316009 · Cited by 7,885
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369 · cited 1×
“…in which this can be accomplished, and a number of interesting proposals have appeared in the literature (e.g., Jordan, 1986; Pineda, 1988; Stornetta, Hogg, & Huberman, 1987; Tank & Hopfield, 1987; Waibel, Hanazawa, Hinton, Shikano, & Lang, 1987; Watrous & Shastri, 1987; Williams & Zipser, 1988).”
Connectionism and cognitive architecture: A critical analysis
Fodor, J. A., & Pylyshyn, Z. W. (1988) · Cognition · doi:10.1016/0010-0277(88)90031-5 · Cited by 4,022
The problem of serial order in behavior
Lashley, K. S. (1951) · MPG.PuRe (Max Planck Society) · Cited by 2,734
Phoneme recognition using time-delay neural networks
Waibel, A., Hanazawa, T., Hinton, G., Shikano, K., & Lang, K. J. (1989) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/29.21701 · Cited by 2,643
On the proper treatment of connectionism
Smolensky, P. (1988) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00052432 · Cited by 2,441
Language Learnability and Language Development
Bialystok, E., & Pinker, S. (1986) · The Modern Language Journal · doi:10.2307/327334 · Cited by 1,816
Lexical access during sentence comprehension: (Re)consideration of context effects
Swinney, D. A. (1979) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(79)90355-4 · Cited by 1,786
Parallel Networks that Learn to Pronounce English Text
B, T. J. S. A. (1987) · Complex Systems · Cited by 1,671
Language Learnability and Language Development
Pye, C., & Pinker, S. (1985) · Language · doi:10.2307/414499 · Cited by 1,665
The sausage machine: A new two-stage parsing model
Frazier, L., & Fodor, J. D. (1978) · Cognition · doi:10.1016/0010-0277(78)90002-1 · Cited by 1,400
The temporal structure of spoken language understanding
Marslen-Wilson, W., & Tyler, L. K. (1980) · Cognition · doi:10.1016/0010-0277(80)90015-3 · Cited by 1,295
Serial Order: A Parallel Distributed Processing Approach
Jordan, M. I. (1997) · Advances in Psychology · doi:10.1016/s0166-4115(97)80111-2 · Cited by 1,242
Hinton, G. E. (2018) · KiltHub Repository · doi:10.1184/r1/6604925 · Cited by 1,025
Cerebral mechanisms in behavior.
indicated, N. A. (1952) · Journal of Consulting Psychology · doi:10.1037/h0051344 · Cited by 913
A Theory of Syntactic Recognition for Natural Language
Kac, M. B., & Marcus, M. P. (1982) · Language · doi:10.2307/414113 · Cited by 906
Potter, S., & Greenberg, J. H. (1964) · The Modern Language Review · doi:10.2307/3721182 · Cited by 847
Spoken word recognition processes and the gating paradigm
Grosjean, F. (1980) · Perception & Psychophysics · doi:10.3758/bf03204386 · Cited by 792
Neural Information Processing Systems
Anderson, D. Z. (1988) · neural information processing systems · Cited by 784
Skilled actions: A task-dynamic approach.
Saltzman, E. L., & Kelso, J. A. S. (1987) · Psychological Review · doi:10.1037//0033-295x.94.1.84 · Cited by 776
Cerebral Mechanisms in Behavior
Morgan, C. T., & Jeffress, L. A. (1953) · The American Journal of Psychology · doi:10.2307/1417992 · Cited by 727
The Acquisition of Morphophonology
MacWhinney, B. (1978) · Monographs of the Society for Research in Child Development · doi:10.2307/1166047 · Cited by 502
Motor control of serial ordering of speech.
MacNeilage, P. F. (1970) · Psychological Review · doi:10.1037/h0029070 · Cited by 455
Historical Linguistics: An Introduction
Fowkes, R. A., & Lehmann, W. P. (1964) · American Speech · doi:10.2307/453926 · Cited by 391
Coarticulation and theories of extrinsic timing
Fowler, C. A. (1980) · Journal of Phonetics · doi:10.1016/s0095-4470(19)31446-9 · Cited by 384
Neural computation by concentrating information in time.
Tank, D. W., & Hopfield, J. J. (1987) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.84.7.1896 · Cited by 354
The dynamical perspective on speech production: data and theory
Kelso, J. A. S., Saltzman, E. L., & Tuller, B. (1986) · Journal of Phonetics · doi:10.1016/s0095-4470(19)30608-4 · Cited by 341
Phoneme Recognition Using Time-Delay Neural Networks
WAIBEL, A., HANAZAWA, T., HINTON, G., SHIKANO, K., & LANG, K. J. (1990) · Readings in Speech Recognition · doi:10.1016/b978-0-08-051584-7.50037-1 · Cited by 302
Learning the hidden structure of speech
Elman, J. L., & Zipser, D. (1988) · The Journal of the Acoustical Society of America · doi:10.1121/1.395916 · Cited by 268
Self-citationGibbs, R. W., & Sharkey, N. E. (1988) · The American Journal of Psychology · doi:10.2307/1423234 · Cited by 257
The influence of sentence constraint on the scope of facilitation for upcoming words
Schwanenflugel, P. J., & Shoben, E. J. (1985) · Journal of Memory and Language · doi:10.1016/0749-596x(85)90026-9 · Cited by 210
Historical Linguistics: An Introduction
Reed, C. E. (1963) · Modern Language Quarterly · doi:10.1215/00267929-24-3-311 · Cited by 205
Representation and Structure in Connectionist Models
Elman, J. L. (1989) · doi:10.21236/ada259504 · Cited by 176
Self-citationLearning Sequential Structure in Simple Recurrent Networks
Servan‐Schreiber, D., Cleeremans, A., & McClelland, J. L. (1988) · Dépôt institutionnel de l'Université libre de Bruxelles (Université Libre de Bruxelles) · Cited by 175
Interaction of knowledge sources in spoken word identification
Salasoo, A., & Pisoni, D. B. (1985) · Journal of Memory and Language · doi:10.1016/0749-596x(85)90025-7 · Cited by 158
Accessing lexical ambiguity: Effects of context and dominance
Tabossi, P. L., Colombo, L., & Job, R. (1987) · Psychological Research · doi:10.1007/bf00308682 · Cited by 150
Smolensky, P. (1987) · CU Scholar (University of Colorado Boulder) · Cited by 145
Effects of context on the immediate interpretation of unambiguous nouns.
Tabossi, P. L. (1988) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.14.1.153 · Cited by 139
Historical Linguistics: An Introduction
Pei, M. A., & Lehmann, W. P. (1963) · Modern Language Journal · doi:10.2307/320620 · Cited by 135
Generalization of Back propagation to Recurrent and Higher Order Neural Networks
Pineda, F. J. (1987) · neural information processing systems · Cited by 127
A Dynamical Approach to Temporal Pattern Processing
Stornetta, W. S., Hogg, T., & Huberman, B. A. (1987) · neural information processing systems · Cited by 52
On the Proper Treatment of Connectionism
Smolensky, P. (1990) · Studies in cognitive systems · doi:10.1007/978-94-009-1882-5_6 · Cited by 43
On the proper treatment of Smolensky
Dreyfus, H. L., & Dreyfus, S. E. (1988) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00052511 · Cited by 41
Research works printed but not linked: 9
- Berwick R.C., 1984, The grammatical basis of linguistic performance.
- Fowler C., 1977, Timing control in speech production.
- Hanson S.J., 1987, Ninth Annual Conference of the Cognitive Science Society
- Hinton G.E., 1986, Parallel distributed processing: Explorations in the microstructure of cognition, 77
- Jordan M.I., 1988, Action
- 10.1037/0033-295X.94.1.84 · doi:10.1037/0033-295x.94.1.84 · Paper page
- Smolensky P., 1987, On variable binding and the representation of symbolic structures in connectionist systems
- Gelder T.J., Compositionality: Variations on a classical theme, Cognitive Science.
- Watrous R.L. &Shastri L.(1987).Learning phonetic features using connectionist networks: An experiment in speech recognition.Proceedings of the IEEE International Conference on Neural Networks. San Diego CA .
Cited by
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
“Representation of words as continuous vectors has a long history [10, 24, 8].”
BackgroundDeep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network
Sherstinsky, A. (2020) · Physica D: Nonlinear Phenomena · doi:10.1016/j.physd.2019.132306 · Cited by 5,680
“Hence, due to the repeated application of the recurrence relation as part of the unrolling, the state signal, s⃗[n], at some value of the index, n, no matter how large, encompasses the contributions of the state signal, s⃗[k], and the input signal, x⃗[k], for all indices, k < n, ending at k = 0, the start of the seque…”
BackgroundMethodA Review of Recurrent Neural Networks: LSTM Cells and Network Architectures
Yu, Y., Si, X., Hu, C., & Zhang, J. (2019) · Neural Computation · doi:10.1162/neco_a_01199 · Cited by 5,591
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390
“In our work, we have used an architecture that is usually called a simple recurrent neural network or Elman network [7].”
Method
Metadata
- Authors
- Elman, J. L.
- Affiliations
- University of California San Diego (US)
- Corresponding author
- Elman, Jeffrey L.
- ORCID
- Elman, Jeffrey L.
- Venue
- Cognitive Science
- Publisher
- Wiley
- Published
- 1990-03
- Volume
- 14
- Issue
- 2
- Pages
- 179-211
- ISSN
- 0364-0213 (ISSN-L)
- 1551-6709
- OpenAlex
- W2110485445
- Cited by
- 11,212
- References
- 46
- Citations by year
- 2026: 587 · 2025: 765 · 2024: 602 · 2023: 642 · 2022: 555 · 2021: 663 · 2020: 657 · 2019: 645 · 2018: 551 · 2017: 385 · 2016: 329 · 2015: 254 · 2014: 269 · 2013: 259 · 2012: 243
- FWCI
- 116.85
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Cognitive Science (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
- Life Sciences › Neuroscience › Cognitive Neuroscience › Cognitive Science and Education Research
- Keywords
- representation of time, recurrent neural networks, dynamic memory, temporal processing, parts of speech, type-token distinction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.