Generalization of back-propagation to recurrent neural networks
Abstract
An adaptive neural network with asymmetric connections is introduced. This network is related to the Hopfield network with graded neurons and uses a recurrent generalization of the \ensuremath{\delta} rule of Rumelhart, Hinton, and Williams to modify adaptively the synaptic weights. The new network bears a resemblance to the master/slave network of Lapedes and Farber but it is architecturally simpler.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
For networks with more than three layers, this gives rise to a model based on columns containing parallel feedforward and feedback pathways, and for recurrent networks that settle into attractor states it gives a reinforcement-trained version of the Almeida/Pineda recurrent backpropagation algorithm.
“For networks with more than three layers, this gives rise to a model based on columns containing parallel feedforward and feedback pathways (Roelfsema and van Ooyen, 2005), and for recurrent networks that settle into attractor states it gives a reinforcement-trained version (Brosch et al., 2015) of the Almeida/Pineda recurrent backpropagation algorithm (Pineda, 1987).”
From Toward an integration of deep learning and neuroscience (2016) · cites this paper as Pineda, 1987
Cites
Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations
Rumelhart, D. E., McClelland, J. L., & AU (1986) · doi:10.7551/mitpress/5236.001.0001 · Cited by 15,686 · see citing works
Hopfield, J. J. (1984) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.81.10.3088 · Cited by 6,918 · see citing works
Characteristics of Random Nets of Analog Neuron-Like Elements
Amarimber, S.-I. (1972) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/tsmc.1972.4309193 · Cited by 304 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570 · see citing works
The graph neural network model
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., & Monfardini, G. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2005605 · Cited by 9,582 · see citing works
Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network
Sherstinsky, A. (2020) · Physica D: Nonlinear Phenomena · doi:10.1016/j.physd.2019.132306 · Cited by 5,680 · see citing works
Hansen, L. K., & Salamon, P. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.58871 · Cited by 4,260 · see citing works
Compressing Neural Networks with the Hashing Trick
Chen, W., Wilson, J. T., Tyree, S., Weinberger, K. Q., & Chen, Y. (2015) · arXiv · doi:10.48550/arxiv.1504.04788 · Cited by 974 · see citing works
The Microsoft 2017 Conversational Speech Recognition System
Xiong, W., Wu, L., Alleva, F., Droppo, J., Huang, X., & Stolcke, A. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461870 · Cited by 472 · see citing works
Theories of Error Back-Propagation in the Brain
Whittington, J. C. R., & Bogacz, R. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.12.005 · Cited by 420 · see citing works
O'Reilly, R. C. (1996) · Neural Computation · doi:10.1162/neco.1996.8.5.895 · Cited by 352 · see citing works
Model-based learning for mobile robot navigation from the dynamical systems perspective
Tani, J. (1996) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/3477.499793 · Cited by 335 · see citing works
The microsoft 2016 conversational speech recognition system
Xiong, W., Droppo, J., Huang, X., Seide, F., Seltzer, M., Stolcke, A., Yu, D., & Zweig, G. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7953159 · Cited by 326 · see citing works
Toward Human Parity in Conversational Speech Recognition
Xiong, W., Droppo, J., Huang, X., Seide, F., Seltzer, M. L., Stolcke, A., Yu, D., & Zweig, G. (2017) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2017.2756440 · Cited by 245 · see citing works
Computational Capabilities of Graph Neural Networks
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., & Monfardini, G. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2005141 · Cited by 225 · see citing works
State-of-Charge Estimation of Lithium-Ion Batteries via Long Short-Term Memory Network
Yang, F., Song, X., Xu, F., & Tsui, K.-L. (2019) · IEEE Access · doi:10.1109/access.2019.2912803 · Cited by 234 · see citing works
Integration of acoustic and visual speech signals using neural networks
Yuhas, B. P., Goldstein, M. H., & Sejnowski, T. J. (1989) · IEEE Communications Magazine · doi:10.1109/35.41402 · Cited by 209 · see citing works
DISCRETE- vs. CONTINUOUS-TIME NONLINEAR SIGNAL PROCESSING OF Cu ELECTRODISSOLUTION DATA
RICO-MARTÍNEZ, R., KRISCHER, K., KEVREKIDIS, I. G., KUBE, M. C., & HUDSON, J. L. (1992) · Chemical Engineering Communications · doi:10.1080/00986449208936084 · Cited by 144 · see citing works
Toward an integration of deep learning and neuroscience
Marblestone, A. H., Wayne, G., & Kording, K. P. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00094 · Cited by 119 · see citing works
Miller, M. L., Molinelli, E. J., Nair, J. S., Sheikh, T., Samy, R., Jing, X., He, Q., Korkut, A., Crago, A. M., Singer, S., et al. (2013) · Science Signaling · doi:10.1126/scisignal.2004014 · Cited by 86 · see citing works
When Code Completion Fails: A Case Study on Real-World Completions
Hellendoorn, V. J., Proksch, S., Gall, H. C., & Bacchelli, A. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2019.00101 · Cited by 76 · see citing works
On the Binding Problem in Artificial Neural Networks
Greff, K., van Steenkiste, S., & Schmidhuber, J. (2020) · arXiv · doi:10.48550/arxiv.2012.05208 · Cited by 102 · see citing works
Reinforcement learning of linking and tracing contours in recurrent neural networks
Brosch, T., Neumann, H., & Roelfsema, P. R. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004489 · Cited by 33 · see citing works
Shah, A., Varshney, S., & Mehrotra, M. (2024) · SN Computer Science · doi:10.1007/s42979-024-02839-9 · Cited by 16 · see citing works
An Instruction Set Architecture for Machine Learning
Chen, Y., Lan, H., Du, Z., Liu, S., Tao, J., Han, D., Luo, T., Guo, Q., Li, L., Xie, Y., et al. (2018) · ACM Transactions on Computer Systems · doi:10.1145/3331469 · Cited by 14 · see citing works
A predictive-coding network that is both discriminative and generative
Sun, W., & Orchard, J. (2020) · Neural Computation · doi:10.1162/neco_a_01311 · Cited by 11 · see citing works
- Authors
- Pineda, F. J.
- Affiliations
- Johns Hopkins University Applied Physics Laboratory (US)
- Corresponding author
- Pineda, Fernando J.
- ORCID
- Pineda, Fernando J.
- Venue
- Physical Review Letters
- Publisher
- American Physical Society (APS)
- Published
- 1987-11-09
- Volume
- 59
- Issue
- 19
- Pages
- 2229-2232
- ISSN
- 0031-9007 (ISSN-L)
- OpenAlex
- W2007431958
- Cited by
- 931
- References
- 3
- Citations by year
- 2026: 1 · 2025: 18 · 2024: 22 · 2023: 22 · 2022: 29 · 2021: 45 · 2020: 54 · 2019: 44 · 2018: 34 · 2017: 22 · 2016: 19 · 2015: 11 · 2014: 10 · 2013: 17 · 2012: 11
- FWCI
- 32.63
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Physical Review Letters (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Keywords
- backpropagation, recurrent neural networks, asymmetric connections, Hopfield neural network, delta rule, adaptive neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
