The DOI matches its Crossref record; 110 of the 110 references it cites were found in a registry or an index.
A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures
Abstract
Recurrent neural networks (RNNs) have been widely adopted in research areas concerned with sequential data, such as text, audio, and video. However, RNNs consisting of sigma cells or tanh cells are unable to learn the relevant information of input data when the input gap is large. By introducing gate functions into the cell structure, the long short-term memory (LSTM) could handle the problem of long-term dependencies well. Since its introduction, almost all the exciting results based on RNNs have been achieved by the LSTM. The LSTM has become the focus of deep learning. We review the LSTM cell and its variants to explore the learning capacity of the LSTM cell. Furthermore, the LSTM networks are divided into two broad categories: LSTM-dominated networks and integrated LSTM networks. In addition, their various applications are discussed. Finally, future research directions are presented for LSTM networks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
Elman, J. L. (1990) · Cognitive Science · doi:10.1207/s15516709cog1402_1 · Cited by 11,212
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955
Research works printed but not linked: 41
- Achanta R., 2010, SLIC superpixels
- 10.1109/ITSC.2017.8317913 · DOI
- Brown B., 2004, Circuits, Signals, and Systems: IASTED International Conference Proceedings
- 10.1109/ICNN.1996.549199 · doi:10.1109/icnn.1996.549199
- Cheng M., 2016, Proceedings of the IEEE 24th International Conference on Network Protocols
- Deng L., 2013, APSIPA transactions on signal and information processing
- Francesconi E., 1997, Proceedings of the International Workshop on Graphics Recognition, 104
- 10.1109/CEC.2005.1555002 · doi:10.1109/cec.2005.1555002
- Gers F. A., 2002, Journal of Machine Learning Research, 3, 115
- Goller C., 1996, Neural Networks, 1, 347
- Guo Y., 2017, International Journal of Multimedia Information Retrieval, 2, 1
- Hochreiter S., 1991, Untersuchungen zu dynamischen neuronalen Netzen
- Hsu W. N., 2016, Cell, 50, 395
- Jing L., 2017, Gated orthogonal recurrent units: On learning to forget
- Jozefowicz R., 2015, Proceedings of the International Conference on International Conference on Machine Learning, 2342
- Krause B., 2016, Multiplicative LSTM for sequence modelling
- 10.1145/2820783.2820801 · doi:10.1145/2820783.2820801
- Li J., 2016, Proceedings of the Conference on Automatic Speech Recognition and Understanding, 187
- Mallinar N., 2018, Deep canonically correlated LSTMs
- McCarter G., 2007, Air freight image segmentation database
- Moniz J. R. A., 2018, Nested LSTMs
- Nair V., 2010, Proceedings of the International Conference on International Conference on Machine Learning, 807
- Neil D., 2016, Advances in neural information processing systems, 16, 3882
- Nina O., 2016, Proceedings of the International Conference on Information, Communications and Signal Processing, 1
- Rahman L., 2017, Proceedings of the International Conference on Electrical Engineering and Information Communication Technology, 1
- Ranzato M. A., 2014, Video (language) modeling: A baseline for generative models of natural videos
- Robinson A. J., 1987, The utility driven dynamic error propagation network
- Saleh K., 2018, Proceedings of the IEEE International Conference on Intelligent Transportation Systems, 327
- Schlag I., 2017, NIPS Metalearning Workshop
- Shabanian S., 2017, Variational Bi-LSTMs
- Shi X., 2015, Advances in neural information processing systems, 802
- Sutskever I., 2014, Advances in neural information processing systems, 3104
- Tai K. S., 2015, Computer Science, 5, 36
- Weng J. J., 1993, Proceedings of the Fourth International Conference on Computer Vision, 121
- Williams R. J., 1989, Complexity of exact gradient computation algorithms for recurrent neural networks
- Wu H., 2016, An empirical exploration of skip connections for sequential tagging
- Yao K., 2015, Depth-gated LSTM
- 10.1109/DSC.2018.00019 · doi:10.1109/dsc.2018.00019
- Zaremba W., 2014, Learning to execute
- Zhang J., 2016, Proceedings of the IEEE Conference on Acoustics, Speech, and Signal Processing, 1655
- Zhou C., 2016, Computer Science, 1, 39
Cited by
A review on the long short-term memory model
Van Houdt, G., Mosquera, C., & Nápoles, G. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09838-1 · Cited by 1,991
A comprehensive review on ensemble deep learning: Opportunities and challenges
Mohammed, A., & Kora, R. (2023) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2023.01.014 · Cited by 1,127
Deep learning modelling techniques: Current progress, applications, advantages, and challenges
Ahmed, S. F., Alam, M. S. B., Hassan, M., Rozbu, M. R., Ishtiak, T., Rafa, N., Mofijur, M., Shawkat Ali, A. B. M., & Gandomi, A. H. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10466-8 · Cited by 1,079
Feature dimensionality reduction: a review
Jia, W., Sun, M., Lian, J., & Hou, S. (2022) · Complex & Intelligent Systems · doi:10.1007/s40747-021-00637-x · Cited by 797
“To alleviate this problem, a variant of the RNN network of long shortterm memory (LSTM) is proposed, which uses the cell state with gate structure tomake information flow continuously [145].”
MethodRecurrent Neural Networks: A Comprehensive Review of Architectures, Variants, and Applications
Mienye, I. D., Swart, T. G., & Obaido, G. (2024) · Information · doi:10.3390/info15090517 · Cited by 732
Financial Fraud: A Review of Anomaly Detection Techniques and Recent Advances
Hilal, W., Gadsden, S. A., & Yawney, J. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.116429 · Cited by 610
Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey
Jin, G., Liang, Y., Fang, Y., Shao, Z., Huang, J., Zhang, J., & Zheng, Y. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3333824 · Cited by 568
Self-citationRNN-LSTM: From applications to modeling techniques and beyond—Systematic review
Al-Selwi, S. M., Hassan, M. F., Abdulkadir, S. J., Muneer, A., Sumiea, E. H., Alqushaibi, A., & Ragab, M. G. (2024) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2024.102068 · Cited by 505
Machine Learning Methods for Small Data Challenges in Molecular Science
Dou, B., Zhu, Z., Merkurjev, E., Ke, L., Chen, L., Jiang, J., Zhu, Y., Liu, J., Zhang, B., & Wei, G.-W. (2023) · Chemical Reviews · doi:10.1021/acs.chemrev.3c00189 · Cited by 499
Deep learning and medical image processing for coronavirus (COVID-19) pandemic: A survey
Bhattacharya, S., Reddy Maddikunta, P. K., Pham, Q.-V., Gadekallu, T. R., Krishnan S, S. R., Chowdhary, C. L., Alazab, M., & Jalil Piran, M. (2021) · Sustainable Cities and Society · doi:10.1016/j.scs.2020.102589 · Cited by 483
EEG emotion recognition using fusion model of graph convolutional neural networks and LSTM
Yin, Y., Zheng, X., Hu, B., Zhang, Y., & Cui, X. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106954 · Cited by 466
Deep learning in drug discovery: an integrative review and future challenges
Askr, H., Elgeldawi, E., Aboul Ella, H., Elshaier, Y. A. M. M., Gomaa, M. M., & Hassanien, A. E. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10306-1 · Cited by 390
Autonomous robotic laparoscopic surgery for intestinal anastomosis
Saeidi, H., Opfermann, J. D., Kam, M., Wei, S., Leonard, S., Hsieh, M. H., Kang, J. U., & Krieger, A. (2022) · Science Robotics · doi:10.1126/scirobotics.abj2908 · Cited by 384
Dao, F., Zeng, Y., & Qian, J. (2024) · Energy · doi:10.1016/j.energy.2024.130326 · Cited by 366
Autonomous driving system: A comprehensive survey
Zhao, J., Zhao, W., Deng, B., Wang, Z., Zhang, F., Zheng, W., Cao, W., Nan, J., Lian, Y., & Burke, A. F. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122836 · Cited by 361
Time Series Prediction Based on LSTM-Attention-LSTM Model
Wen, X., & Li, W. (2023) · IEEE Access · doi:10.1109/access.2023.3276628 · Cited by 335
Gülmez, B. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120346 · Cited by 321
A review on deep learning in machining and tool monitoring: methods, opportunities, and challenges
Nasir, V., & Sassani, F. (2021) · The International Journal of Advanced Manufacturing Technology · doi:10.1007/s00170-021-07325-7 · Cited by 301
Machine Learning for Detection and Prediction of Crop Diseases and Pests: A Comprehensive Survey
Domingues, T., Brandão, T., & Ferreira, J. C. (2022) · Agriculture · doi:10.3390/agriculture12091350 · Cited by 290
IoT-Cloud-Based Smart Healthcare Monitoring System for Heart Disease Prediction via Deep Learning
Nancy, A. A., Ravindran, D., Raj Vincent, P. M. D., Srinivasan, K., & Gutierrez Reina, D. (2022) · Electronics · doi:10.3390/electronics11152292 · Cited by 286
A systematic literature review for network intrusion detection system (IDS)
Abdulganiyu, O. H., Ait Tchakoucht, T., & Saheed, Y. K. (2023) · International Journal of Information Security · doi:10.1007/s10207-023-00682-2 · Cited by 279
Time series forecasting of Covid-19 using deep learning models: India-USA comparative case study
Shastri, S., Singh, K., Kumar, S., Kour, P., & Mansotra, V. (2020) · Chaos, Solitons & Fractals · doi:10.1016/j.chaos.2020.110227 · Cited by 268
LSTM-Based Intrusion Detection System for In-Vehicle Can Bus Communications
Hossain, M. D., Inoue, H., Ochiai, H., Fall, D., & Kadobayashi, Y. (2020) · IEEE Access · doi:10.1109/access.2020.3029307 · Cited by 261
DCNNBiLSTM: An Efficient Hybrid Deep Learning-Based Intrusion Detection System
Hnamte, V., & Hussain, J. (2023) · Telematics and Informatics Reports · doi:10.1016/j.teler.2023.100053 · Cited by 259
Understanding Physics-Informed Neural Networks: Techniques, Applications, Trends, and Challenges
Farea, A., Yli-Harja, O., & Emmert-Streib, F. (2024) · AI · doi:10.3390/ai5030074 · Cited by 259
Predictive Battery Health Management With Transfer Learning and Online Model Correction
Che, Y., Deng, Z., Lin, X., Hu, L., & Hu, X. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3055811 · Cited by 257
Innovative Materials Science via Machine Learning
Gao, C., Min, X., Fang, M., Tao, T., Zheng, X., Liu, Y., Wu, X., & Huang, Z. (2022) · Advanced Functional Materials · doi:10.1002/adfm.202108044 · Cited by 198
Elallid, B. B., Benamar, N., Hafid, A. S., Rachidi, T., & Mrani, N. (2022) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2022.03.013 · Cited by 179
“This Neural Networks (NN) architecture is therefore capable of predicting the current action based on past actions of AVs (Yu et al., 2019).”
Toward an AI Era: Advances in Electronic Skins
Fu, X., Cheng, W., Wan, G., Yang, Z., & Tee, B. C. K. (2024) · Chemical Reviews · doi:10.1021/acs.chemrev.4c00049 · Cited by 172
Karijadi, I., & Chou, S.-Y. (2022) · Energy and Buildings · doi:10.1016/j.enbuild.2022.111908 · Cited by 159
Xu, J., Pan, S., Sun, P. Z. H., Hyeong Park, S., & Guo, K. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3225782 · Cited by 120
“More approaches have recently been proposed, such as convolutional neural network (CNN) [22], recurrent neural network (RNN) [23], auto encoder (AE) [24], etc .”
MethodMachine learning based approach to exam cheating detection
Kamalov, F., Sulieman, H., & Santandreu Calonge, D. (2021) · PLOS ONE · doi:10.1371/journal.pone.0254340 · Cited by 102
Zafar, M. H., Khan, N. M., Houran, M. A., Mansoor, M., Akhtar, N., & Sanfilippo, F. (2024) · Energy · doi:10.1016/j.energy.2024.130584 · Cited by 100
An Explainable AI framework for credit evaluation and analysis
Nallakaruppan, M. K., Balusamy, B., Shri, M. L., Malathi, V., & Bhattacharyya, S. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.111307 · Cited by 83
A Novel Method for SoH Prediction of Batteries Based on Stacked LSTM with Quick Charge Data
Yayan, U., Arslan, A. T., & Yucel, H. (2021) · Applied Artificial Intelligence · doi:10.1080/08839514.2021.1901033 · Cited by 70
“Beyond the traditional RNN models, LSTM provides a judge mechanism to judge if the information is useful or not (Yu et al. 2019).”
An Autocorrelation-based LSTM-Autoencoder for Anomaly Detection on Time-Series Data
Homayouni, H., Ghosh, S., Ray, I., Gondalia, S., Duggan, J., & Kahn, M. G. (2020) · 2020 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata50022.2020.9378192 · Cited by 66
“Examples are Multi Layer Perceptrons (MLPs) [26], Long Short Term Memory (LSTM) [9], and Hierarchical Temporal Memory (HTM) [27].”
BackgroundA Survey on Visual Navigation and Positioning for Autonomous UUVs
Qin, J., Li, M., Li, D., Zhong, J., & Yang, K. (2022) · Remote Sensing · doi:10.3390/rs14153794 · Cited by 65
He, H., Li, X., Chen, P., Chen, J., Liu, M., & Wu, L. (2024) · Journal of Cloud Computing · doi:10.1186/s13677-024-00677-x · Cited by 62
Parida, L., Moharana, S., Ferreira, V. M., Giri, S. K., & Ascensão, G. (2022) · Sensors · doi:10.3390/s22249920 · Cited by 58
Interactive Control of Lower Limb Exoskeleton Robots: A Review
Zhang, Y.-P., Cao, G.-Z., Li, L.-L., & Diao, D.-F. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3352005 · Cited by 58
Maarif, M. R., Saleh, A. R., Habibi, M., Fitriyani, N. L., & Syafrudin, M. (2023) · Information · doi:10.3390/info14050265 · Cited by 58
Ahmed, M. T., Rahman, M., Nur, S., Islam, A., & Das, D. (2021) · 2021 International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT) · doi:10.1109/icaect49130.2021.9392608 · Cited by 60
Fine-grained activity classification in assembly based on multi-visual modalities
Chen, H., Zendehdel, N., Leu, M. C., & Yin, Z. (2024) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-023-02152-x · Cited by 47
Forecasting Energy Consumption of a Public Building Using Transformer and Support Vector Regression
Huang, J., & Kaewunruen, S. (2023) · Energies · doi:10.3390/en16020966 · Cited by 38
Computer Vision-Based Wood Identification: A Review
Silva, J., Bordalo, R., Pissarra, J., & de Palacios, P. (2022) · Forests · doi:10.3390/f13122041 · Cited by 37
Correlation-Based Anomaly Detection in Industrial Control Systems
Jadidi, Z., Pal, S., Hussain, M., & Nguyen Thanh, K. (2023) · Sensors · doi:10.3390/s23031561 · Cited by 35
“For example, the data at the current time point are likely to be related to a previous time point or a time point in the long past, which can be referred to as short-term and long-term time dependency [32].”
BackgroundDing, H., Wang, B., Hamel, A. P., Melkonyan, M., Ang, T. F. A., Au, R., & Lin, H. (2023) · Frontiers in Dementia · doi:10.3389/frdem.2023.1271680 · Cited by 32
Human Gait phases recognition based on multi-source data fusion and BILSTM attention neural network
Zhan, H., Kou, J., Cao, Y., Guo, Q., Zhang, J., & Shi, Y. (2024) · Measurement · doi:10.1016/j.measurement.2024.115396 · Cited by 28
Self-citationDiagnosis of pathological speech with streamlined features for long short-term memory learning
Pham, T. D., Holmes, S. B., Zou, L., Patel, M., & Coulthard, P. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.107976 · Cited by 23
Zhao, M., Yuan, M., Yang, Y., & Xu, S. X. (2023) · IEEE/ACM Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2022.3225296 · Cited by 22
Augmenting Polymer Datasets by Iterative Rearrangement
(2023) · doi:10.1021/acs.jcim.3c00144 · Cited by 19
Bonci, A., Fredianelli, L., Kermenov, R., Longarini, L., Longhi, S., Pompei, G., Prist, M., & Verdini, C. (2024) · Applied Sciences · doi:10.3390/app14198686 · Cited by 16
Pronello, C., Anbarasan, D., Spoturno, F., & Terzolo, G. (2025) · Public Transport · doi:10.1007/s12469-023-00349-0 · Cited by 13
Lacalli, T. (2021) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2021.697129 · Cited by 11
Sparks of Large Audio Models: A Survey and Outlook
Latif, S., Shoukat, M., Shamshad, F., Usama, M., Ren, Y., Cuayáhuitl, H., Wang, W., Zhang, X., Togneri, R., Cambria, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.12792 · Cited by 17
“To combat this, LSTMs implement a gating mechanism alongside memory cells to regulate the information flow and mitigate issues related to gradients [74, 75].”
BackgroundAttAuth: An implicit authentication framework for smartphone users using multimodality data
Zhao, C., Gao, F., & Shen, Z. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3314717 · Cited by 10
Wang, Z., Zhang, H., Ji, Z., Yang, Y., & Wang, H. (2023) · Brain Science Advances · doi:10.26599/bsa.2023.9050013 · Cited by 9
“Finally, the weighted integration probability and the LSTM calibrated probability were added to obtain the final output.”
InfluentialTemperature Prediction of Mushrooms Based on a Data—Physics Hybrid Approach
Wang, M., Kong, X., Shan, F., Zheng, W., Ren, P., Wang, J., Chen, C., Zhang, X., & Zhao, C. (2024) · Agriculture · doi:10.3390/agriculture14010145 · Cited by 9
Xu, J., Yin, R., Huang, Y., Gao, H., Wu, Y., Guo, J., Smith, G. E., DeKosky, S. T., Wang, F., Guo, Y., et al. (2023) · doi:10.1101/2023.07.27.23293270 · Cited by 5
Prediction of Protein B-Factor Profiles Based on Bidirectional Long Short-Term Memory Network
Wang, Q., Xiao, X., Miao, Z., Zhang, X., Yang, D., Jiang, B., & Liu, M. (2025) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbbio.2025.3564284 · Cited by 5
Metadata
- Authors
- Yu, Y., Si, X., Hu, C., & Zhang, J.
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2019-07
- Volume
- 31
- Issue
- 7
- Pages
- 1235-1270
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2944851425
- Cited by
- 5,591
- References
- 110
- FWCI
- 111.78
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.