The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Learning State Space Trajectories in Recurrent Neural Networks
Abstract
Many neural network learning procedures compute gradients of the errors on the output layer of units after they have settled to their final values. We describe a procedure for finding ∂E/∂w ij, where E is an error functional of the temporal trajectory of the states of a continuous recurrent network and w ij are the weights of that network. Computing these quantities allows one to perform gradient descent in the weights to minimize E. Simulations in which networks are taught to move through limit cycles are shown. This type of recurrent network seems particularly suited for temporally continuous domains, such as signal processing, control, and speech.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Optimization by Simulated Annealing
Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983) · Science · doi:10.1126/science.220.4598.671 · Cited by 45,134
Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,172
“Neural” computation of decisions in optimization problems
Hopfield, J. J., & Tank, D. W. (1985) · Biological Cybernetics · doi:10.1007/bf00339943 · Cited by 6,031
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369
A learning algorithm for Boltzmann machines
Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985) · Cognitive Science · doi:10.1207/s15516709cog0901_7 · Cited by 3,402
Beyond regression : new fools for prediction and analysis in the behavioral sciences
Werbos, P. J. (1974) · Medical Entomology and Zoology · Cited by 2,739
Phoneme recognition using time-delay neural networks
Waibel, A., Hanazawa, T., Hinton, G., Shikano, K., & Lang, K. J. (1989) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/29.21701 · Cited by 2,643
Generalization of backpropagation with application to a recurrent gas market model
Werbos, P. J. (1988) · Neural Networks · doi:10.1016/0893-6080(88)90007-x · Cited by 944
Generalization of back-propagation to recurrent neural networks
Pineda, F. J. (1987) · Physical Review Letters · doi:10.1103/physrevlett.59.2229 · Cited by 931 · cited 3×
“We use the same class of networks used by Pineda [11], but he is concerned only with the limit behavior of these networks, and completely suppresses all other temporal behavior.”
BackgroundMethodA Steepest-Ascent Method for Solving Optimum Programming Problems
Bryson, A. E., & Denham, W. F. (1962) · Journal of Applied Mechanics · doi:10.1115/1.3640537 · Cited by 492
Cited by
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172
“Recurrent neural networks (RNN’s) [5], [8], [12], [13], [15] provide another alternative for incorporating temporal dynamics and are discussed in more detail in a later section.”
BackgroundFundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network
Sherstinsky, A. (2020) · Physica D: Nonlinear Phenomena · doi:10.1016/j.physd.2019.132306 · Cited by 5,680
A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures
Yu, Y., Si, X., Hu, C., & Zhang, J. (2019) · Neural Computation · doi:10.1162/neco_a_01199 · Cited by 5,591
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369
“…time-varying input or output have been suggested or investigated by Jordan (1986), Bachrach (1988), Mozer (1988), Elman (1988), Servan-Schreiber, Cleeremans, and McClelland (1988), Robinson and Fallside (1987), Stornetta, Hogg, and Huberman (1987), Gallant and King (1988), and Pearlmutter (1988).”
BackgroundUnderstanding normal and impaired word reading: Computational principles in quasi-regular domains.
Plaut, D. C., McClelland, J. L., Seidenberg, M. S., & Patterson, K. (1996) · Psychological Review · doi:10.1037/0033-295x.103.1.56 · Cited by 2,698
“In order to apply back-propagation through time to continuous units, the propagation of error in the backward pass must be made continuous as well (Pearlmutter, 1989).”
BackgroundMethod30 years of adaptive neural networks: perceptron, Madaline, and backpropagation
Widrow, B., & Lehr, M. A. (1990) · Proceedings of the IEEE · doi:10.1109/5.58323 · Cited by 2,242
Neural networks for control systems—A survey
Hunt, K. J., Sbarbaro, D., Żbikowski, R., & Gawthrop, P. J. (1992) · Automatica · doi:10.1016/0005-1098(92)90053-i · Cited by 1,864
Progress in supervised neural networks
Hush, D. R., & Horne, B. G. (1993) · IEEE Signal Processing Magazine · doi:10.1109/79.180705 · Cited by 1,247
Harm, M. W., & Seidenberg, M. S. (2004) · Psychological Review · doi:10.1037/0033-295x.111.3.662 · Cited by 1,133
Generating Coherent Patterns of Activity from Chaotic Neural Networks
Sussillo, D., & Abbott, L. F. (2009) · Neuron · doi:10.1016/j.neuron.2009.07.018 · Cited by 1,092
Self-organized control of bipedal locomotion by neural oscillators in unpredictable environment
Taga, G., Yamaguchi, Y., & Shimizu, H. (1991) · Biological Cybernetics · doi:10.1007/bf00198086 · Cited by 1,031
An ultra-sparse code underliesthe generation of neural sequences in a songbird
Hahnloser, R. H. R., Kozhevnikov, A. A., & Fee, M. S. (2002) · Nature · doi:10.1038/nature00974 · Cited by 986
Approximation of dynamical systems by continuous time recurrent neural networks
Funahashi, K.-I., & Nakamura, Y. (1993) · Neural Networks · doi:10.1016/s0893-6080(05)80125-x · Cited by 980
A Hybrid of Genetic Algorithm and Particle Swarm Optimization for Recurrent Network Design
Juang, C.-F. (2004) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2003.818557 · Cited by 964
A Dynamical Approach to Gestural Patterning in Speech Production
Saltzman, E. L., & Munhall, K. G. (1989) · Ecological Psychology · doi:10.1207/s15326969eco0104_2 · Cited by 946
Deep dyslexia: A case study of connectionist neuropsychology
Plaut, D. C., & Shallice, T. (1993) · Cognitive Neuropsychology · doi:10.1080/02643299308253469 · Cited by 856
Diagonal recurrent neural networks for dynamic systems control
Ku, C.-C., & Lee, K. Y. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.363441 · Cited by 725
Deep Learning and Its Applications in Biomedicine
Cao, C., Liu, F., Tan, H., Song, D., Shu, W., Li, W., Zhou, Y., Bo, X., & Xie, Z. (2018) · Genomics, Proteomics & Bioinformatics · doi:10.1016/j.gpb.2017.07.003 · Cited by 624
Gradient calculations for dynamic recurrent neural networks: a survey
Pearlmutter, B. A. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.410363 · Cited by 582
Self-citationAn Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories
Williams, R. J., & Peng, J. (1990) · Neural Computation · doi:10.1162/neco.1990.2.4.490 · Cited by 577
New results on recurrent network training: unifying the algorithms and accelerating convergence
Atiya, A. F., & Parlos, A. G. (2000) · IEEE Transactions on Neural Networks · doi:10.1109/72.846741 · Cited by 521
Evolving Dynamical Neural Networks for Adaptive Behavior
Beer, R. D., & Gallagher, J. C. (1992) · Adaptive Behavior · doi:10.1177/105971239200100105 · Cited by 500
On the Complexity of Neural Network Classifiers: A Comparison Between Shallow and Deep Architectures
Bianchini, M., & Scarselli, F. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2293637 · Cited by 500
A connectionist model of development
Mjolsness, E., Sharp, D. H., & Reinitz, J. (1991) · Journal of Theoretical Biology · doi:10.1016/s0022-5193(05)80391-1 · Cited by 434
How delays affect neural dynamics and learning
Baldi, P., & Atiya, A. F. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.298231 · Cited by 429
Learning in the Recurrent Random Neural Network
Gelenbe, E. (1993) · Neural Computation · doi:10.1162/neco.1993.5.1.154 · Cited by 366
Botvinick, M., & Plaut, D. C. (2004) · Psychological Review · doi:10.1037/0033-295x.111.2.395 · Cited by 363
Recurrent network models of sequence generation and memory
Rajan, K., Harvey, C. D., & Tank, D. W. (2016) · Neuron · doi:10.1016/j.neuron.2016.02.009 · Cited by 351
Learning to Control Fast-Weight Memories: An Alternative to Dynamic Recurrent Networks
Schmidhuber, J. (1992) · Neural Computation · doi:10.1162/neco.1992.4.1.131 · Cited by 357
Deep Learning Meets SAR: Concepts, models, pitfalls, and perspectives
Zhu, X. X., Montazeri, S., Ali, M., Hua, Y., Wang, Y., Mou, L., Shi, Y., Xu, F., & Bamler, R. (2021) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2020.3046356 · Cited by 357
Plaut, D. C., & Booth, J. R. (2000) · Psychological Review · doi:10.1037/0033-295x.107.4.786 · Cited by 355
Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences
Neil, D., Pfeiffer, M., & Liu, S. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.48550/arxiv.1610.09513 · Cited by 319
Machine Learning-based traffic prediction models for Intelligent Transportation Systems
Boukerche, A., & Wang, J. (2020) · Computer Networks · doi:10.1016/j.comnet.2020.107530 · Cited by 317
Faust, O., Shenfield, A., Kareem, M., San, T. R., Fujita, H., & Acharya, U. R. (2018) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2018.07.001 · Cited by 309
Speaking Up for Vocabulary: Reading Skill Differences in Young Adults
Braze, D., Tabor, W., Shankweiler, D. P., & Mencl, W. E. (2007) · Journal of Learning Disabilities · doi:10.1177/00222194070400030401 · Cited by 302
Input-output HMMs for sequence processing
Bengio, Y., & Frasconi, P. (1996) · IEEE Transactions on Neural Networks · doi:10.1109/72.536317 · Cited by 300
The induction of dynamical recognizers
Pollack, J. B. (1991) · Machine Learning · doi:10.1007/bf00114845 · Cited by 294
Artificial neural networks for automatic ECG analysis
Silipo, R., & Marchesi, C. (1998) · IEEE Transactions on Signal Processing · doi:10.1109/78.668803 · Cited by 282
Gradient descent learning algorithm overview: a general dynamical systems perspective
Baldi, P. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.363438 · Cited by 258
Hybrid physics-informed neural networks for lithium-ion battery modeling and prognosis
Nascimento, R. G., Corbetta, M., Kulkarni, C. S., & Viana, F. A. C. (2021) · Journal of Power Sources · doi:10.1016/j.jpowsour.2021.230526 · Cited by 224
An analog VLSI recurrent neural network learning a continuous-time trajectory
Cauwenberghs, G. (1996) · IEEE Transactions on Neural Networks · doi:10.1109/72.485671 · Cited by 127
Metadata
- Authors
- Pearlmutter, B. A.
- Venue
- Neural Computation
- Publisher
- MIT Press - Journals
- Published
- 1989-06
- Volume
- 1
- Issue
- 2
- Pages
- 263-269
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2143503258
- Cited by
- 656
- References
- 11
- FWCI
- 30.65
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.