The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
LSTM recurrent networks learn simple context-free and context-sensitive languages
Abstract
Previous work on learning regular languages from exemplary training sequences showed that long short-term memory (LSTM) outperforms traditional recurrent neural networks (RNNs). We demonstrate LSTMs superior performance on context-free language benchmarks for RNNs, and show that it works even better than previous hardwired or highly specialized architectures. To the best of our knowledge, LSTM variants are also the first RNNs to learn a simple context-sensitive language, namely a(n)b(n)c(n).
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542
Learning to forget: Continual prediction with LSTM
Gers, F. A., Schmidhuber, J., & Cummins, F. (2000) · Neural Computation · doi:10.1162/089976600300015015 · Cited by 4,823
Self-citationGeneralization of backpropagation with application to a recurrent gas market model
Werbos, P. J. (1988) · Neural Networks · doi:10.1016/0893-6080(88)90007-x · Cited by 944
Recurrent nets that time and count
Gers, F. A., & Schmidhuber, J. (2000) · Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium · doi:10.1109/ijcnn.2000.861302 · Cited by 623
Self-citationGradient calculations for dynamic recurrent neural networks: a survey
Pearlmutter, B. A. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.410363 · Cited by 582
An Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories
Williams, R. J., & Peng, J. (1990) · Neural Computation · doi:10.1162/neco.1990.2.4.490 · Cited by 577
Gradient-based learning algorithms for recurrent networks and their computational complexity
Williams, R. J., & Zipser, D. (1995) · Cited by 424
Learning Complex, Extended Sequences Using the Principle of History Compression
Schmidhuber, J. (1992) · Neural Computation · doi:10.1162/neco.1992.4.2.234 · Cited by 398
Turing computability with neural nets
Siegelmann, H. T., & Sontag, E. D. (1991) · Applied Mathematics Letters · doi:10.1016/0893-9659(91)90080-f · Cited by 377
Experiments on Learning by Back Propagation.
Plaut, D. C., Nowlan, S. J., & Hinton, G. E. (1986) · Cited by 323
A Recurrent Neural Network that Learns to Count
RODRIGUEZ, P., WILES, J., & ELMAN, J. L. (1999) · Connection Science · doi:10.1080/095400999116340 · Cited by 241
Casey, M. (1996) · Neural Computation · doi:10.1162/neco.1996.8.6.1135 · Cited by 174
Recent advances of grammatical inference
Sakakibara, Y. (1997) · Theoretical Computer Science · doi:10.1016/s0304-3975(97)00014-5 · Cited by 157
On the Effect of Analog Noise in Discrete-Time Analog Computations
Maass, W., & Orponen, P. (1998) · Neural Computation · doi:10.1162/089976698300017359 · Cited by 89
A Local Learning Algorithm for Dynamic Feedforward and Recurrent Networks
Schmidhuber, J. (1989) · Connection Science · doi:10.1080/09540098908915650 · Cited by 86
Discrete recurrent neural networks for grammatical inference
Zeng, Z., Goodman, R. M., & Smyth, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279194 · Cited by 75
Maass, W., & Sontag, E. D. (1999) · Neural Computation · doi:10.1162/089976699300016656 · Cited by 67
The Neural Network Pushdown Automaton: Model, Stack and Learning SimulationsPreprint
Sun, G.-Z., Giles, C. L., Chen, H. H., & Lee, Y. C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.05738 · Cited by 41
Analysis of Dynamical Recognizers
Blair, A., & Pollack, J. B. (1997) · Neural Computation · doi:10.1162/neco.1997.9.5.1127 · Cited by 41
Recurrent Neural Networks Can Learn to Implement Symbol-Sensitive Counting
Rodriguez, P. F., & Wiles, J. (1997) · Cited by 28
Wiles, J., & Elman, J. (1995) · Cited by 17
Computation in recurrent neural networks: From counters to iterated function systems
Kalinke, Y., & Lehmann, H. (1998) · Lecture notes in computer science · doi:10.1007/bfb0095051 · Cited by 14
Cited by
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures
Yu, Y., Si, X., Hu, C., & Zhang, J. (2019) · Neural Computation · doi:10.1162/neco_a_01199 · Cited by 5,591
A Convolutional Neural Network for Modelling Sentences
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1062 · Cited by 3,601
Supervised Sequence Labelling with Recurrent Neural Networks
Graves, A. (2012) · Studies in Computational Intelligence · doi:10.1007/978-3-642-24797-2 · Cited by 3,120
Long short-term memory recurrent neural network architectures for large scale acoustic modeling
Sak, H., Senior, A., & Beaufays, F. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-80 · Cited by 2,973
Metadata
- Authors
- Gers, F. A., & Schmidhuber, E.
- Venue
- IEEE Transactions on Neural Networks
- Published
- 2001
- Volume
- 12
- Issue
- 6
- Pages
- 1333-1340
- ISSN
- 1045-9227 (ISSN-L)
- OpenAlex
- W2121029939
- Cited by
- 722
- References
- 23
- FWCI
- 9.53
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.