Cited by 6,955Neural Networks and ApplicationsSpeech Recognition and SynthesisTopic Modeling

The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.

LSTM: A Search Space Odyssey

Open access

Greff, Klaus, Srivastava, Rupesh K., Koutnik, Jan, Steunebrink, Bas R., Schmidhuber, Jurgen·2017·IEEE Transactions on Neural Networks and Learning Systems

DOI 10.1109/tnnls.2016.2582924

Open this paper in CoChat — free account Open PDF

Abstract

Several variants of the long short-term memory (LSTM) architecture for recurrent neural networks have been proposed since its inception in 1995. In recent years, these networks have become the state-of-the-art models for a variety of machine learning problems. This has led to a renewed interest in understanding the role and utility of various computational components of typical LSTM variants. In this paper, we present the first large-scale analysis of eight LSTM variants on three representative tasks: speech recognition, handwriting recognition, and polyphonic music modeling. The hyperparameters of all LSTM variants for each task were optimized separately using random search, and their importance was assessed using the powerful functional ANalysis Of VAriance framework. In total, we summarize the results of 5400 experimental runs ( ≈ 15 years of CPU time), which makes our study the largest of its kind on LSTM networks. Our results show that none of the variants can improve upon the standard LSTM architecture significantly, and demonstrate the forget gate and the output activation function to be its most critical components. We further observe that the studied hyperparameters are virtually independent and derive guidelines for their efficient adjustment.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1109/tnnls.2016.2582924

Cites

56 of 56 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Long Short-Term Memory
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Self-citation
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
Untitled work
(2012) · Cited by 8,009
Practical Bayesian Optimization of Machine Learning Algorithms
Snoek, J., Larochelle, H., & Adams, R. P. (2012) · arXiv · doi:10.48550/arxiv.1206.2944 · Cited by 6,691
Framewise phoneme classification with bidirectional LSTM and other neural network architectures
Graves, A., & Schmidhuber, J. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.06.042 · Cited by 6,152
Self-citation
Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks
Graves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143891 · Cited by 5,370
Self-citation
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
Untitled work
(2013) · Cited by 3,740

Show all 53 references

Research works printed but not linked: 2

gers, 2002, DEKF-LSTM, Proc European Symp Artificial Neural Networks (ESANN), 369
robinson, 1987, The utility driven dynamic error propagation network

Sorted by citations

Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network
Sherstinsky, A. (2020) · Physica D: Nonlinear Phenomena · doi:10.1016/j.physd.2019.132306 · Cited by 5,680
A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures
Yu, Y., Si, X., Hu, C., & Zhang, J. (2019) · Neural Computation · doi:10.1162/neco_a_01199 · Cited by 5,870
Character-level Convolutional Networks for Text ClassificationPreprint
Zhang, X., Zhao, J., & LeCun, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.01626 · Cited by 4,481
A review on the long short-term memory model
Houdt, G. V., Mosquera, C., & Nápoles, G. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09838-1 · Cited by 2,198
Towards the development of realistic botnet dataset in the Internet of Things for network forensic analytics: Bot-IoT dataset
Koroniotis, N., Moustafa, N., Sitnikova, E., & Turnbull, B. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.05.041 · Cited by 1,983
A Survey of the Usages of Deep Learning for Natural Language Processing
Otter, D. W., Medina, J. R., & Kalita, J. K. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2979670 · Cited by 1,836
Natural language processing: state of the art, current trends and challenges
Khurana, D., Koli, A., Khatter, K., & Singh, S. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13428-4 · Cited by 1,823
Neural Architecture Search: A Survey
Elsken, T., Metzen, J. H., & Hutter, F. (2018) · arXiv · doi:10.48550/arxiv.1808.05377 · Cited by 1,763
A State-of-the-Art Survey on Deep Learning Theory and Architectures
Alom, M. Z., Taha, T. M., Yakopcic, C., Westberg, S. K. E., Sidike, P., Nasrin, M. S., Hasan, M., Essen, B. C. V., Awwal, A. A. S., & Asari, V. K. (2019) · Electronics · doi:10.3390/electronics8030292 · Cited by 1,722
Generating Focused Molecule Libraries for Drug Discovery with Recurrent Neural Networks
Segler, M. H. S., Kogej, T., Tyrchan, C., & Waller, M. P. (2018) · ACS Central Science · doi:10.1021/acscentsci.7b00512 · Cited by 1,537
Predicting residential energy consumption using CNN-LSTM neural networks
Kim, T. Y., & Cho, S.-B. (2019) · Energy · doi:10.1016/j.energy.2019.05.230 · Cited by 1,492
Generación de series de tiempo financieras sintéticas para "data augmentation" usando redes neuronales generativas adversarias (GAN)
Villarraga Ossa, E. F. (2021) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) · Cited by 1,669
An Empirical Exploration of Recurrent Network Architectures
Józefowicz, R., Zaremba, W., & Sutskever, I. (2015) · Cited by 1,459
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
Li, Q., Wen, Z., Wu, Z., Hu, S., Wang, N., Li, Y., Liu, X., & He, B. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3124599 · Cited by 1,407
Generation and Comprehension of Unambiguous Object Descriptions
Mao, J., Huang, J., Toshev, A., Camburu, O., Yuille, A., & Murphy, K. P. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.9 · Cited by 1,281
Federated Learning for Mobile Keyboard Prediction
Hard, A., Rao, K., Mathews, R., Ramaswamy, S., Beaufays, F., Augenstein, S., Eichner, H., Kiddon, C., & Ramage, D. (2018) · arXiv · doi:10.48550/arxiv.1811.03604 · Cited by 1,148
Deep Neural Networks and Tabular Data: A Survey
Borisov, V., Leemann, T., Seßler, K., Haug, J., Pawelczyk, M., & Kasneci, G. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3229161 · Cited by 1,400
Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation
Liu, C., Chen, L.-C., Schroff, F., Adam, H., Hua, W., Yuille, A., & Fei-Fei, L. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00017 · Cited by 1,102
Neural Responding Machine for Short-Text Conversation
Shang, L., Lu, Z., & Li, H. (2015) · doi:10.3115/v1/p15-1152 · Cited by 1,146
Time series forecasting of petroleum production using deep LSTM recurrent networks
Sagheer, A., & Kotb, M. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.09.082 · Cited by 1,004
Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN
Li, S., Li, W., Cook, C., Zhu, C., & Gao, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00572 · Cited by 972
Siamese Recurrent Architectures for Learning Sentence Similarity
Mueller, J., & Thyagarajan, A. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10350 · Cited by 935
Stock market's price movement prediction with LSTM neural networks
Nelson, D. M. Q., Pereira, A. C. M., & de Oliveira, R. A. (2017) · 2017 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966019 · Cited by 951
Visualizing and Understanding Recurrent Networks
Karpathy, A., Johnson, J., & Fei-Fei, L. (2015) · arXiv · doi:10.48550/arxiv.1506.02078 · Cited by 845
Evolving Deep Neural Networks
Miikkulainen, R. P., Liang, J., Meyerson, E., Rawal, A., Fink, D., Francon, O., Raju, B., Shahrzad, H., Navruzyan, A., Duffy, N., et al. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-815480-9.00015-3 · Cited by 871
Neural Network Methods for Natural Language Processing
Goldberg, Y. (2017) · Synthesis Lectures on Human Language Technologies · doi:10.1007/978-3-031-02165-7 · Cited by 821
FakeBERT: Fake news detection in social media with a BERT-based deep learning approach
Kaliyar, R. K., Goswami, A., & Narang, P. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-020-10183-2 · Cited by 890
Remaining useful life estimation of engineered systems using vanilla LSTM neural networks
Wu, Y., Yuan, M., Dong, S., Lin, L., & Liu, Y. (2018) · Neurocomputing · doi:10.1016/j.neucom.2017.05.063 · Cited by 789
ConViT: improving vision transformers with soft convolutional inductive biases
d’Ascoli, S., Touvron, H., Leavitt, M. L., Morcos, A. S., Biroli, G., & Sagun, L. (2022) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/ac9830 · Cited by 782
A comparative analysis of K-Nearest Neighbor, Genetic, Support Vector Machine, Decision Tree, and Long Short Term Memory algorithms in machine learning
Bansal, M., Goyal, A., & Choudhary, A. (2022) · Decision Analytics Journal · doi:10.1016/j.dajour.2022.100071 · Cited by 911
Network Traffic Classifier With Convolutional and Recurrent Neural Networks for Internet of Things
López‐Martín, M., Carro, B., Sánchez-Esguevillas, A., & Lloret, J. (2017) · IEEE Access · doi:10.1109/access.2017.2747560 · Cited by 772
Deep Biaffine Attention for Neural Dependency ParsingPreprint
Dozat, T., & Manning, C. D. (2016) · arXiv (Cornell University) · doi:10.4230/oasics.ldk.2021.24 · Cited by 1,019
Action Recognition in Video Sequences using Deep Bi-Directional LSTM With CNN Features
Ullah, A., Ahmad, J., Muhammad, K., Sajjad, M., & Baik, S. W. (2017) · IEEE Access · doi:10.1109/access.2017.2778011 · Cited by 765
Recurrent Neural Networks: A Comprehensive Review of Architectures, Variants, and Applications
Mienye, I. D., Swart, T. G., & Obaido, G. (2024) · Information · doi:10.3390/info15090517 · Cited by 977
Automated diagnosis of arrhythmia using combination of CNN and LSTM techniques with variable length heart beats
Oh, S. L., Ng, E. Y. K., Tan, R., & Acharya, U. R. (2018) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2018.06.002 · Cited by 759
Long Short Term Memory Recurrent Neural Network Classifier for Intrusion Detection
Kim, J., Kim, J., Le, T.-T.-H., & Kim, H. (2016) · International Conference on Platform Technology and Service · doi:10.1109/platcon.2016.7456805 · Cited by 765
A Deep CNN-LSTM Model for Particulate Matter (PM2.5) Forecasting in Smart Cities
Huang, C., & Kuo, P. (2018) · Sensors · doi:10.3390/s18072220 · Cited by 727
Designing neural networks through neuroevolution
Stanley, K. O., Clune, J., Lehman, J., & Miikkulainen, R. P. (2018) · Nature Machine Intelligence · doi:10.1038/s42256-018-0006-z · Cited by 745
Short-term runoff prediction with GRU and LSTM networks without requiring time step optimization during sample generation
Gao, S., Huang, Y., Zhang, S., Han, J., Wang, G., Zhang, M., & Qingsheng, L. (2020) · Journal of Hydrology · doi:10.1016/j.jhydrol.2020.125188 · Cited by 713
A survey and critique of multiagent deep reinforcement learning
Hernández-Leal, P., Kartal, B., & Taylor, M. E. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09421-1 · Cited by 799
DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing
Yao, S., Hu, S., Zhao, Y., Zhang, A., & Abdelzaher, T. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052577 · Cited by 667
Deep Learning for solar power forecasting — An approach using AutoEncoder and LSTM Neural Networks
Gensler, A., Henze, J., Sick, B., & Raabe, N. (2016) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2016.7844673 · Cited by 693
Deep Learning with Long Short-Term Memory for Time Series Prediction
Hua, Y., Zhao, Z., Li, R., Chen, X., Liu, Z., & Zhang, H. (2019) · IEEE Communications Magazine · doi:10.1109/mcom.2019.1800155 · Cited by 651
Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables
Hammerla, N. Y., Halloran, S., & Ploetz, T. (2016) · arXiv · doi:10.48550/arxiv.1604.08880 · Cited by 606
Communication-Efficient Federated Deep Learning With Layerwise Asynchronous Model Update and Temporally Weighted Aggregation
Chen, Y., Sun, X., & Jin, Y. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2953131 · Cited by 597
A Survey of Visual Transformers
Liu, Y., Zhang, Y., Wang, Y., Hou, F., Yuan, J. S., Tian, J., Zhang, Y. S., Shi, Z., Jianping Fan, B., & He, Z. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3227717 · Cited by 652
A Siamese Long Short-Term Memory Architecture for Human Re-identification
Varior, R. R., Shuai, B., Lu, J., Xu, D., & Wang, G. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46478-7_9 · Cited by 551
Nonlinear Dynamic Soft Sensor Modeling With Supervised Long Short-Term Memory Network
Yuan, X., Li, L., & Wang, Y. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2902129 · Cited by 561
Model-Free Real-Time EV Charging Scheduling Based on Deep Reinforcement Learning
Wan, Z., Li, H., He, H., & Prokhorov, D. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2879572 · Cited by 548
A survey on LSTM memristive neural network architectures and applications
Smagulova, K., & James, A. P. (2019) · The European Physical Journal Special Topics · doi:10.1140/epjst/e2019-900046-x · Cited by 575
An optimized model using LSTM network for demand forecasting
Abbasimehr, H., Shabani, M., & Yousefi, M. (2020) · Computers & Industrial Engineering · doi:10.1016/j.cie.2020.106435 · Cited by 598
Remaining useful life predictions for turbofan engine degradation using semi-supervised deep architecture
Ellefsen, A. L., Bjørlykhaug, E., Æsøy, V., Ushakov, S., & Zhang, H. (2018) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2018.11.027 · Cited by 550
Online Multi-Target Tracking Using Recurrent Neural Networks
Milan, A., Rezatofighi, S. H., Dick, A., Reid, I. D., & Konrad Schindler, B. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11194 · Cited by 97
RNN-LSTM: From applications to modeling techniques and beyond—Systematic review
Al-Selwi, S. M., Hassan, M. F., Abdulkadir, S. J., Muneer, A., Sumiea, E. H., Alqushaibi, A., & Ragab, M. G. (2024) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2024.102068 · Cited by 593
A solution to the learning dilemma for recurrent networks of spiking neurons
Bellec, G., Scherr, F., Subramoney, A., Hajek, E., Salaj, D., Legenstein, R. A., & Maass, W. (2020) · Nature Communications · doi:10.1038/s41467-020-17236-y · Cited by 568
A Review of Deep Learning Models for Time Series Prediction
Han, Z., Zhao, J., Leung, H., Ma, K. F., & Wang, W. (2019) · IEEE Sensors Journal · doi:10.1109/jsen.2019.2923982 · Cited by 537
Machine learning for reliability engineering and safety applications: Review of current status and future opportunities
Xu, Z., & Saleh, J. H. (2021) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2021.107530 · Cited by 543
Forecasting and Anomaly Detection approaches using LSTM and LSTM Autoencoder techniques with the applications in supply chain management
Nguyen, H. D., Tran, K. P., Thomassey, S., & Hamad, M. (2020) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2020.102282 · Cited by 538
Deep Learning for Sensor-based Human Activity Recognition
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2022) · ACM Computing Surveys · doi:10.1145/3447744 · Cited by 690
“A multi-label RNN was created with each RNN cell responding to activity recognition of one occupant.”
BackgroundMethodInfluential
EA-LSTM: Evolutionary attention-based LSTM for time series prediction
Li, Y., Zhu, Z., Kong, D., Han, H., & Zhao, Y. (2019) · Knowledge-Based Systems · doi:10.1016/j.knosys.2019.05.028 · Cited by 510
Stacked bidirectional and unidirectional LSTM recurrent neural network for forecasting network-wide traffic state with missing values
Cui, Z., Ke, R., Pu, Z., & Wang, Y. (2020) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2020.102674 · Cited by 515
What to Do Next: Modeling User Behaviors by Time-LSTM
Zhu, Y., Li, H., Liao, Y., Wang, B., Guan, Z., Liu, H., & Cai, D. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/504 · Cited by 502
Massive Exploration of Neural Machine Translation Architectures
Britz, D., Goldie, A., Luong, M.-T., & Le, Q. V. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1151 · Cited by 594
Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamics
Vlachas, P. R., Pathak, J., Hunt, B. R., Sapsis, T. P., Girvan, M., Ott, E., & Koumoutsakos, P. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.02.016 · Cited by 532
A Bidirectional LSTM Prognostics Method Under Multiple Operational Conditions
Huang, C., Huang, H., & Li, Y. (2019) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2019.2891463 · Cited by 477
Stock Price Prediction via Discovering Multi-Frequency Trading Patterns
Zhang, L., Aggarwal, C. C., & Qi, G.-J. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098117 · Cited by 490
EEG emotion recognition using fusion model of graph convolutional neural networks and LSTM
Yin, Y., Zheng, X., Hu, B., Zhang, Y., & Cui, X. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106954 · Cited by 470
Long short-term memory for machine remaining life prediction
Zhang, J., Wang, P. E., Yan, R., & Gao, R. X. (2018) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2018.05.011 · Cited by 481
State-of-health estimation and remaining useful life prediction for the lithium-ion battery based on a variant long short term memory neural network
Li, P., Zhang, Z., Xiong, Q., Ding, B., Hou, J., Luo, D., Rong, Y., & Li, S. (2020) · Journal of Power Sources · doi:10.1016/j.jpowsour.2020.228069 · Cited by 463
A deep Recurrent Neural Network based approach for Internet of Things malware threat hunting
HaddadPajouh, H., Dehghantanha, A., Khayami, R., & Choo, K. R. (2018) · Future Generation Computer Systems · doi:10.1016/j.future.2018.03.007 · Cited by 466
Guiding the Long-Short Term Memory Model for Image Caption Generation
Jia, X., Gavves, E., Fernando, B., & Tuytelaars, T. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.277 · Cited by 480
BAT: Deep Learning Methods on Network Intrusion Detection Using NSL-KDD Dataset
Su, T., Sun, H., Zhu, J., Wang, S., & Li, Y. (2020) · IEEE Access · doi:10.1109/access.2020.2972627 · Cited by 436
Predicting process behaviour using deep learning
Evermann, J., Rehse, J.-R., & Fettke, P. (2017) · Decision Support Systems · doi:10.1016/j.dss.2017.04.003 · Cited by 454
A novel spatiotemporal convolutional long short-term neural network for air pollution prediction
Wen, C., Liu, S., Yao, X., Peng, L., Xiang, L., Hu, Y. C., & Chi, T. (2018) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2018.11.086 · Cited by 437
Hate Speech Dataset from a White Supremacy Forum
de Gibert, O., Pérez, N., Pablos, A. G., & Cuadros, M. (2018) · Workshop on Abusive Language Online (ALW) · doi:10.18653/v1/w18-5102 · Cited by 428
Overview of the Transformer-based Models for NLP Tasks
Gillioz, A., Casas, J., Mugellini, E., & Khaled, O. A. (2020) · Annals of Computer Science and Information Systems · doi:10.15439/2020f20 · Cited by 492
Fault Diagnosis for Electro-Mechanical Actuators Based on STL-HSTA-GRU and SM
Zhang, X., Li-wei, T., & Chen, J. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3127641 · Cited by 427
Multi-head CNN–RNN for multi-time series anomaly detection: An industrial case study
Cañizo, M., Triguero, I., Conde, Á., & Onieva, E. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.07.034 · Cited by 431
Anomaly Detection in Automobile Control Network Data with Long Short-Term Memory Networks
Taylor, A., Leblanc, S. P., & Japkowicz, N. (2016) · IEEE International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2016.20 · Cited by 433
Understanding Neural Networks through Representation ErasurePreprint
Li, J., Monroe, W., & Jurafsky, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.08220 · Cited by 625
Light Gated Recurrent Units for Speech Recognition
Ravanelli, M., Brakel, P., Omologo, M., & Bengio, Y. (2018) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2017.2762739 · Cited by 434
Fault diagnosis and remaining useful life estimation of aero engine using LSTM neural network
Yuan, M., Wu, Y., & Li, L. (2016) · IEEE International Conference on Aircraft Utility Systems · doi:10.1109/aus.2016.7748035 · Cited by 416
Why ResNet Works? Residuals Generalize
He, F., Liu, T., & Tao, D. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2966319 · Cited by 428
Deep Bidirectional and Unidirectional LSTM Recurrent Neural Network for Network-wide Traffic Speed PredictionPreprint
Cui, Z., Ke, R., Pu, Z., & Wang, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.02143 · Cited by 601
Rainfall–runoff prediction at multiple timescales with a single Long Short-Term Memory network
Gauch, M., Kratzert, F., Klotz, D., Nearing, G., Lin, J., & Hochreiter, S. (2021) · Hydrology and earth system sciences · doi:10.5194/hess-25-2045-2021 · Cited by 442
Where to Go Next: A Spatio-Temporal Gated Network for Next POI Recommendation
Zhao, P., Luo, A., Liu, Y., Xu, J., Li, Z., Zhuang, F., Sheng, V. S., & Zhou, X. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.3007194 · Cited by 412
Deep solar radiation forecasting with convolutional neural network and long short-term memory network algorithms
Ghimire, S., Deo, R. C., Raj, N., & Mi, J. (2019) · Applied Energy · doi:10.1016/j.apenergy.2019.113541 · Cited by 403
Convolutional Neural Network-Based Multiple-Rate Compressive Sensing for Massive MIMO CSI Feedback: Design, Simulation, and Analysis
Guo, J., Wen, C.-K., Jin, S., & Li, G. Y. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2968430 · Cited by 401
Deep Learning Models for Cyber Security in IoT Networks
Roopak, M., Tian, G. Y., & Chambers, J. A. (2019) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc.2019.8666588 · Cited by 390
Unsupervised Anomaly Detection With LSTM Neural Networks
Ergen, T., & Kozat, S. S. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2935975 · Cited by 417
Deep learning for load forecasting with smart meter data: Online Adaptive Recurrent Neural Network
Fekri, M. N., Patel, H., Grolinger, K., & Sharma, V. K. (2020) · Applied Energy · doi:10.1016/j.apenergy.2020.116177 · Cited by 387
Detecting Cyber Attacks in Industrial Control Systems Using Convolutional Neural Networks
Kravchik, M., & Shabtai, A. (2018) · Workshop on Cyber-Physical Systems Security and PrivaCy (CPS-SPC) · doi:10.1145/3264888.3264896 · Cited by 421
Stock price prediction based on deep neural networks
Yu, P., & Yan, X. (2019) · Neural Computing and Applications · doi:10.1007/s00521-019-04212-x · Cited by 390
All in One Bad Weather Removal Using Architectural Search
Li, R., Tan, R. T., & Cheong, L.-F. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00324 · Cited by 377
A method for state-of-charge estimation of lithium-ion batteries based on PSO-LSTM
Ren, X., Liu, S., Yu, X., & Dong, X. (2021) · Energy · doi:10.1016/j.energy.2021.121236 · Cited by 375
IONet: Learning to Cure the Curse of Drift in Inertial Odometry
Chen, C., Lu, C. X., Markham, A., & Trigoni, N. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12102 · Cited by 374
Optimizing LSTM for time series prediction in Indian stock market
Yadav, A., Jha, C. K., & Sharan, A. (2020) · Procedia Computer Science · doi:10.1016/j.procs.2020.03.257 · Cited by 368
A hybrid model integrating deep learning with investor sentiment analysis for stock price prediction
Jing, N., Wu, Z., & Wang, H. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115019 · Cited by 404
An Overview of Voice Conversion and Its Challenges: From Statistical Modeling to Deep Learning
Şişman, B., Yamagishi, J., King, S., & Li, H. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.3038524 · Cited by 399
Streamflow and rainfall forecasting by two long short-term memory-based models
Ni, L., Wang, D., Singh, V. P., Wu, J., Wang, Y., Tao, Y., & Zhang, J. (2019) · Journal of Hydrology · doi:10.1016/j.jhydrol.2019.124296 · Cited by 377

Authors: Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J.
Affiliations: University of Applied Sciences and Arts of Southern Switzerland (CH)
ORCID: Greff, Klaus
Venue: IEEE Transactions on Neural Networks and Learning Systems
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2017-10
Volume: 28
Issue: 10
Pages: 2222-2232
ISSN: 2162-237X (ISSN-L)
2162-2388
DOI: 10.1109/tnnls.2016.2582924
OpenAlex: W1689711448
Cited by: 6,955
References: 53
Citations by year: 2026: 602 · 2025: 812 · 2024: 810 · 2023: 822 · 2022: 872 · 2021: 881 · 2020: 778 · 2019: 686 · 2018: 383 · 2017: 188
FWCI: 370.42
Type: journal article
Language: en
Open access: green
arxiv.org
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
Locations: IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
PubMed (publishedVersion)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Funders: Swiss National Science Foundation through the Project Theory and Practice of Reinforcement Learning 2, and through the Project Advanced Reinforcement Learning (138219, 156682)
European Institute of Innovation and Technology through the Project Neural Dynamics, through the Project NASCENCE and through the Project WAY (FP7-ICT-270247, FP7-ICT-317662, FP7-ICT-288551)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords: long short-term memory, recurrent neural networks, hyperparameter optimization, random search, functional analysis of variance, forget gate, speech recognition, handwriting recognition

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

LSTM: A Search Space Odyssey

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks