Cited by 6,955Neural Networks and ApplicationsSpeech Recognition and SynthesisTopic Modeling

The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.

LSTM: A Search Space Odyssey

Open access

Greff, Klaus, Srivastava, Rupesh K., Koutnik, Jan, Steunebrink, Bas R., Schmidhuber, Jurgen·2017·IEEE Transactions on Neural Networks and Learning Systems

DOI 10.1109/tnnls.2016.2582924

Open this paper in CoChat — free account Open PDF

Abstract

Several variants of the long short-term memory (LSTM) architecture for recurrent neural networks have been proposed since its inception in 1995. In recent years, these networks have become the state-of-the-art models for a variety of machine learning problems. This has led to a renewed interest in understanding the role and utility of various computational components of typical LSTM variants. In this paper, we present the first large-scale analysis of eight LSTM variants on three representative tasks: speech recognition, handwriting recognition, and polyphonic music modeling. The hyperparameters of all LSTM variants for each task were optimized separately using random search, and their importance was assessed using the powerful functional ANalysis Of VAriance framework. In total, we summarize the results of 5400 experimental runs ( ≈ 15 years of CPU time), which makes our study the largest of its kind on LSTM networks. Our results show that none of the variants can improve upon the standard LSTM architecture significantly, and demonstrate the forget gate and the output activation function to be its most critical components. We further observe that the studied hyperparameters are virtually independent and derive guidelines for their efficient adjustment.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1109/tnnls.2016.2582924

Cites

56 of 56 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Long Short-Term Memory
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Self-citation
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
Untitled work
(2012) · Cited by 8,009
Practical Bayesian Optimization of Machine Learning Algorithms
Snoek, J., Larochelle, H., & Adams, R. P. (2012) · arXiv · doi:10.48550/arxiv.1206.2944 · Cited by 6,691
Framewise phoneme classification with bidirectional LSTM and other neural network architectures
Graves, A., & Schmidhuber, J. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.06.042 · Cited by 6,152
Self-citation
Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks
Graves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143891 · Cited by 5,370
Self-citation
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
Untitled work
(2013) · Cited by 3,740

Flagged first, then by citations

Show all 53 references

Research works printed but not linked: 2

gers, 2002, DEKF-LSTM, Proc European Symp Artificial Neural Networks (ESANN), 369
robinson, 1987, The utility driven dynamic error propagation network

Sorted by citations

Items 701–800 of 1,013

Temperature Prediction Using the Missing Data Refinement Model Based on a Long Short-Term Memory Neural Network
Park, I., Kim, H. S., Lee, J., Kim, J. H., Song, C. H., & Kim, H. K. (2019) · Atmosphere · doi:10.3390/atmos10110718 · Cited by 81
LSTM Based Phishing Detection for Big Email Data
Li, Q. L., Cheng, M., Wang, J., & Sun, B. (2020) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2020.2978915 · Cited by 76
A distributed WND-LSTM model on MapReduce for short-term traffic flow prediction
Xia, D., Zhang, M., Yan, X., Bai, Y., Zheng, Y., Li, Y., & Li, H. (2020) · Neural Computing and Applications · doi:10.1007/s00521-020-05076-2 · Cited by 76
CorrNet: Fine-Grained Emotion Recognition for Video Watching Using Wearable Physiological Sensors
Zhang, T., Ali, A. E., Wang, C., Hanjalić, A., & Cesar, P. S. (2020) · Sensors · doi:10.3390/s21010052 · Cited by 79
A multi-omics-based serial deep learning approach to predict clinical outcomes of single-agent anti-PD-1/PD-L1 immunotherapy in advanced stage non-small-cell lung cancer.
Yang, Y. A., Yang, J., Shen, L., Chen, J., Xia, L., Ni, B., Ge, L., Wang, Y., & Lü, S. (2021) · PubMed · Cited by 76
Federated Learning and Autonomous UAVs for Hazardous Zone Detection and AQI Prediction in IoT Environment
Chhikara, P., Tekchandani, R., Kumar, N., Guizani, M., & Hassan, M. M. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3074523 · Cited by 86
Short-term renewable energy consumption and generation forecasting: A case study of Western Australia
Abu-Salih, B., Wongthongtham, P., Morrison, G. M., Coutinho, K., Al‐Okaily, M., & Huneiti, A. (2022) · Heliyon · doi:10.1016/j.heliyon.2022.e09152 · Cited by 81
Genetic algorithm-based hyperparameter optimization of deep learning models for PM2.5 time-series prediction
Erden, C. (2023) · International Journal of Environmental Science and Technology · doi:10.1007/s13762-023-04763-6 · Cited by 82
A review on action recognition for accident detection in smart city transportation systems
Adewopo, V. A., ElSayed, Z., ElSayed, Z., Özer, M., Abdelgawad, A., & Bayoumi, M. (2023) · Journal of Electrical Systems and Information Technology · doi:10.1186/s43067-023-00124-y · Cited by 87
On the Potential of Simple Framewise Approaches to Piano TranscriptionPreprint
Kelz, R., Dorfer, M., Korzeniowski, F., Böck, S., Arzt, A., & Widmer, G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.05153 · Cited by 90
Automated Item Generation with Recurrent Neural Networks
von Davier, M. (2018) · Psychometrika · doi:10.1007/s11336-018-9608-y · Cited by 82
Evaluating Recurrent Neural Network Explanations
Arras, L., Osman, A., Müller, K., & Samek, W. (2019) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w19-4813 · Cited by 88
Investigation on Performance of Neural Networks Using Quadratic Relative Error Cost Function
Zhang, N., Shen, S., Zhou, A., & Xu, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2930520 · Cited by 77
LSTM-Based Wastewater Treatment Plants Operation Strategies for Effluent Quality Improvement
Pisa, I., Santín, I., Morell, A., Vicario, J. L., & Vilanova, R. (2019) · IEEE Access · doi:10.1109/access.2019.2950852 · Cited by 75
RNN-Based Subway Passenger Flow Rolling Prediction
Sha, S., Li, J., Zhang, K., Yang, Z., Wei, Z., Li, X., & Zhu, X. (2020) · IEEE Access · doi:10.1109/access.2020.2964680 · Cited by 76
Artificial Intelligence based accurately load forecasting system to forecast short and medium-term load demands
Butt, F. M., Hussain, L., Mahmood, A., & Lone, K. J. (2020) · Mathematical Biosciences & Engineering · doi:10.3934/mbe.2021022 · Cited by 82
Neural Networks Based Shunt Hybrid Active Power Filter for Harmonic Elimination
Iqbal, M., Jawad, M., Jaffery, M. H., Akhtar, S., Rafiq, M. N., Qureshi, M. B., Ansari, A. R., & Nawaz, R. (2021) · IEEE Access · doi:10.1109/access.2021.3077065 · Cited by 75
DynaNet: Neural Kalman Dynamical Model for Motion Estimation and Prediction
Chen, C., Lu, C. X., Wang, B., Trigoni, N., & Markham, A. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3112460 · Cited by 78
A model fusion strategy for identifying aircraft risk using CNN and Att-BiLSTM
Zhou, D., Zhuang, X. D., Zuo, H., Cai, J., Zhao, X., & Xiang, J. (2022) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2022.108750 · Cited by 75
Adaptive Spatiotemporal Transformer Graph Network for Traffic Flow Forecasting by IoT Loop Detectors
Huang, B., Dou, H., Luo, Y., Li, J., Wang, J., & Zhou, T. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3209523 · Cited by 80
Advancing horizons in remote sensing: a comprehensive survey of deep learning models and applications in image classification and beyond
Paheding, S., Saleem, A. I., Siddiqui, M. F. H., Rawashdeh, N. A., Essa, A. E., & Reyes-Angulo, A. A. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10165-7 · Cited by 87
Advanced data-driven techniques in AI for predicting lithium-ion battery remaining useful life: a comprehensive review
Wang, S., Zhou, R., Ren, Y., Jiao, M., Liu, H., & Lian, C. (2024) · Green Chemical Engineering · doi:10.1016/j.gce.2024.09.001 · Cited by 83
A Novel Deep Learning Framework for Industrial Multiphase Flow Characterization
Dang, W., Gao, Z., Hou, L., Lv, D., Qiu, S., & Chen, G. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2908211 · Cited by 78
Cyberbullying Detection in Social Networks Using Bi-GRU with Self-Attention Mechanism
Fang, Y., Yang, S., Zhao, B., & Huang, C. (2021) · Information · doi:10.3390/info12040171 · Cited by 74
“From the perspective of the recurrent unit structure, GRU uses fewer connections and fewer parameters in all places of networks, so model training and generalizing are easier and faster, especially on small sequence data [50].”
Background
Data Augmentation for Morphological Reinflection
Silfverberg, M., Wiemerslage, A., Liu, L., & Mao, L. J. (2017) · doi:10.18653/v1/k17-2010 · Cited by 74
Subsurface Structure Analysis Using Computational Interpretation and Learning: A Visual Signal Processing Perspective
AlRegib, G., Deriche, M. A., Long, Z., Di, H., Wang, Z., Alaudah, Y., Shafiq, M. A., & Alfarraj, M. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2785979 · Cited by 74
An Energy-Efficient Dual Prediction Scheme Using LMS Filter and LSTM in Wireless Sensor Networks for Environment Monitoring
Shu, T., Chen, J., Bhargava, V. K., & de Silva, C. W. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2911295 · Cited by 74
Multi-Period and Multi-Spatial Equilibrium Analysis in Imperfect Electricity Markets: A Novel Multi-Agent Deep Reinforcement Learning Approach
Ye, Y., Qiu, D., Li, J., & Štrbac, G. (2019) · IEEE Access · doi:10.1109/access.2019.2940005 · Cited by 74
A Text Abstraction Summary Model Based on BERT Word Embedding and Reinforcement Learning
Wang, Q., Liu, P., Zhu, Z., Yin, H., Zhang, Q., & Zhang, L. (2019) · Applied Sciences · doi:10.3390/app9214701 · Cited by 80
A Smartphone-Enabled Fall Detection Framework for Elderly People in Connected Home Healthcare
Hassan, M. M., Gumaei, A., Aloi, G., Fortino, G., & Zhou, M. (2019) · IEEE Network · doi:10.1109/mnet.001.1900100 · Cited by 74
Deep Learning for Price Movement Prediction Using Convolutional Neural Network and Long Short-Term Memory
Yang, C., Zhai, J., & Tao, G. (2020) · Mathematical Problems in Engineering · doi:10.1155/2020/2746845 · Cited by 74
Review of State-of-the-Art Design Techniques for Chatbots
Agarwal, R., & Wadhwa, M. (2020) · SN Computer Science · doi:10.1007/s42979-020-00255-3 · Cited by 83
Vessel trajectory prediction based on AIS data and bidirectional GRU
Wang, C., Ren, H., & Li, H. (2020) · International Conference on Computer Vision, Image and Deep Learning (CVIDL) · doi:10.1109/cvidl51233.2020.00-89 · Cited by 79
Industry 4.0-Oriented Deep Learning Models for Human Activity Recognition
Mohsen, S., Elkaseer, A., & Scholz, S. G. (2021) · IEEE Access · doi:10.1109/access.2021.3125733 · Cited by 88
Emotion Recognition With Audio, Video, EEG, and EMG: A Dataset and Baseline Approaches
Chen, J., Ro, T., & Zhu, Z. (2022) · IEEE Access · doi:10.1109/access.2022.3146729 · Cited by 78
Stock price prediction based on LSTM and LightGBM hybrid model
Tian, L., Feng, L., Yang, L., & Guo, Y. (2022) · The Journal of Supercomputing · doi:10.1007/s11227-022-04326-5 · Cited by 74
Digital Predistortion of RF Power Amplifiers With Phase-Gated Recurrent Neural Networks
Kobal, T., Li, Y., Wang, X., & Zhu, A. (2022) · IEEE Transactions on Microwave Theory and Techniques · doi:10.1109/tmtt.2022.3161024 · Cited by 78
FINN-L: Library Extensions and Design Trade-Off Analysis for Variable Precision LSTM Networks on FPGAs
Rybalkin, V., Pappalardo, A., Ghaffar, M. M., Gambardella, G., Wehn, N., & Blott, M. (2018) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2018.00024 · Cited by 78
Human Action Recognition Using Deep Multilevel Multimodal (${M}^{2}$ ) Fusion of Depth and Inertial Sensors
Ahmad, Z., & Khan, N. (2019) · IEEE Sensors Journal · doi:10.1109/jsen.2019.2947446 · Cited by 71
Stochastic simulation on reproducing long-term memory of hydroclimatological variables using deep learning model
Lee, T., Shin, J.-Y., Kim, J., & Singh, V. P. (2020) · Journal of Hydrology · doi:10.1016/j.jhydrol.2019.124540 · Cited by 75
Price Trailing for Financial Trading Using Deep Reinforcement Learning
Tsantekidis, A., Passalis, N., Toufa, A.-S., Saitas-Zarkias, K., Chairistanidis, S., & Tefas, A. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2997523 · Cited by 77
Stock Price Forecast Based on CNN-BiLSTM-ECA Model
Chen, Y., Fang, R., Liang, T., Sha, Z., Li, S., Yi, Y., Zhou, W., & Song, H. (2021) · Scientific Programming · doi:10.1155/2021/2446543 · Cited by 73
The Evolution of Artificial Intelligence in Medical Imaging: From Computer Science to Machine and Deep Learning
Avanzo, M., Stancanello, J., Pirrone, G., Drigo, A., & Retico, A. (2024) · Cancers · doi:10.3390/cancers16213702 · Cited by 87
Instantaneous heart rate as a robust feature for sleep apnea severity detection using deep learning
Pathinarupothi, R. K., Vinaykumar, R., Rangan, E. S., Gopalakrishnan, E. A., & Soman, K. P. (2017) · IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI ...) · doi:10.1109/bhi.2017.7897263 · Cited by 73
Deep neural models for ICD-10 coding of death certificates and autopsy reports in free-text
Duarte, F., Martins, B., Pinto, C. S., & Silva, M. J. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.02.011 · Cited by 74
Collaborative Filtering with Recurrent Neural NetworksPreprint
Devooght, R., & Bersini, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.07400 · Cited by 93
Duplicate Questions Pair Detection Using Siamese MaLSTM
Imtiaz, Z., Umer, M., Ahmad, M. I., Ullah, S., Choi, G. S., & Mehmood, A. (2020) · IEEE Access · doi:10.1109/access.2020.2969041 · Cited by 76
Critical Assessment of Artificial Intelligence Methods for Prediction of hERG Channel Inhibition in the “Big Data” Era
Siramshetty, V. B., Nguyễn, Ð.-T., Martinez, N. J., Southall, N. T., Simeonov, A. M., & Zakharov, A. V. (2020) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.0c00884 · Cited by 74
Short-Term Load Forecasting Based on Deep Learning Bidirectional LSTM Neural Network
Cai, C., Yuan, T., Zhu, T., & Deng, Z. (2021) · Applied Sciences · doi:10.3390/app11178129 · Cited by 72
Simultaneous feature engineering and interpretation: Forecasting harmful algal blooms using a deep learning approach
Kim, T., Shin, J., Lee, D., Kim, Y. W., Na, E. H., Park, J.-H., Lim, C., & Cha, Y. (2022) · Water Research · doi:10.1016/j.watres.2022.118289 · Cited by 72
Early warning of tunnel collapse based on Adam-optimised long short-term memory network and TBM operation parameters
Hou, S., & Liu, Y. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.104842 · Cited by 72
Deep-Learning-Based Diagnosis and Prognosis of Alzheimer’s Disease: A Comprehensive Review
Sharma, R., Goel, T., Tanveer, M., Lin, C., & Murugan, R. (2023) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2023.3254209 · Cited by 79
Word2Vec and LSTM based deep learning technique for context-free fake news detection
Mallik, A., & Kumar, S. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15364-3 · Cited by 77
A Causal Long Short-Term Memory Sequence to Sequence Model for TEC Prediction Using GNSS Observations
Kaselimi, M., Voulodimos, A. S., Doulamis, N. D., Doulamis, A. D., & Delikaraoglou, D. (2020) · Remote Sensing · doi:10.3390/rs12091354 · Cited by 72
An integrated long-short term memory algorithm for predicting polar westerlies wave height
Ni, C., & Ma, X. (2020) · Ocean Engineering · doi:10.1016/j.oceaneng.2020.107715 · Cited by 71
Attention based GRU-LSTM for software defect prediction
Munir, H. S., Ren, S., Mustafa, M., Siddique, C. N., & Qayyum, S. (2021) · PLoS ONE · doi:10.1371/journal.pone.0247444 · Cited by 71
A compositive architecture coupling outlier correction, EWT, nonlinear Volterra multi-model fusion with multi-objective optimization for short-term wind speed forecasting
Fang, P., Fu, W., Wang, K., Xiong, D., & Zhang, K. (2021) · Applied Energy · doi:10.1016/j.apenergy.2021.118191 · Cited by 70
CNN-LSTM Hybrid Model to Promote Signal Processing of Ultrasonic Guided Lamb Waves for Damage Detection in Metallic Pipelines
Shang, L., Zhang, Z. M., Tang, F., Cao, Q., Pan, H. X., & Lin, Z. (2023) · Sensors · doi:10.3390/s23167059 · Cited by 77
Estimating Brain Connectivity With Varying-Length Time Lags Using a Recurrent Neural Network
Wang, Y., Lin, K., Qi, Y., Lian, Q., Feng, S., Wu, Z., & Pan, G. (2018) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2018.2842769 · Cited by 69
Parsimonious memory unit for recurrent neural networks with application to natural language processing
Morchid, M. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.05.081 · Cited by 72
A stateless deep learning framework to predict net asset value
Koudjonou, K. M., & Rout, M. (2019) · Neural Computing and Applications · doi:10.1007/s00521-019-04525-x · Cited by 73
SAINT: self-attention augmented inception-inside-inception network improves protein secondary structure prediction
Uddin, M. R., Mahbub, S., Rahman, M. S., & Bayzid, M. S. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btaa531 · Cited by 69
A Comprehensive Review of Polyphonic Sound Event Detection
Chan, T. K., & Chin, C. S. (2020) · IEEE Access · doi:10.1109/access.2020.2999388 · Cited by 69
An LSTM-Based Model for Person Identification Using ECG Signal
Jyotishi, D., & Dandapat, S. (2020) · IEEE Sensors Letters · doi:10.1109/lsens.2020.3012653 · Cited by 70
Long Short-Term Memory Neural Networks for Modeling Nonlinear Electronic Components
Mahvash Moradi, A., Sadrossadat, S. A., & Derhami, V. (2021) · IEEE Transactions on Components Packaging and Manufacturing Technology · doi:10.1109/tcpmt.2021.3071351 · Cited by 73
Deep Neural Networks for Depression Recognition Based on 2D and 3D Facial Expressions Under Emotional Stimulus Tasks
Guo, W., Yang, H., Liu, Z., Xu, Y., & Hu, B. (2021) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.609760 · Cited by 71
Combining time-series evidence: A complex network model based on a visibility graph and belief entropy
Song, X., & Xiao, F. (2022) · Applied Intelligence · doi:10.1007/s10489-021-02956-5 · Cited by 70
An explainable framework for load forecasting of a regional integrated energy system based on coupled features and multi-task learning
Wu, K., Gu, J., Lu, M., Wen, H., & Ma, J. (2022) · Protection and Control of Modern Power Systems · doi:10.1186/s41601-022-00245-y · Cited by 70
Deep Learning Enabled Slow Fluid Antenna Multiple Access
Waqar, N., Wong, K., Tong, K., Sharples, A., & Zhang, Y. (2023) · IEEE Communications Letters · doi:10.1109/lcomm.2023.3237595 · Cited by 70
Fault Diagnosis and Tolerance Control of Five-Level Nested NPP Converter Using Wavelet Packet and LSTM
Ye, S., Jiang, J., Li, J., Liu, Y., Zhou, Z., & Liu, C. (2019) · IEEE Transactions on Power Electronics · doi:10.1109/tpel.2019.2921677 · Cited by 73
Detection and identification of external intrusion signals from 33 km optical fiber sensing system based on deep learning
Bai, Y., Xing, J., Xie, F., Liu, S., & Li, J. (2019) · Optical Fiber Technology · doi:10.1016/j.yofte.2019.102060 · Cited by 70
Dynamic Mortality Risk Predictions in Pediatric Critical Care Using Recurrent Neural NetworksPreprint
Aczon, M., Ledbetter, D., Ho, V. L., Gunny, A. M., Flynn, A., Williams, J., & Wetzel, R. C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.06675 · Cited by 91
Robust and accurate decoding of hand kinematics from entire spiking activity using deep learning
Ahmadi, N., Constandinou, T. G., & Bouganis, C.-S. (2021) · Journal of Neural Engineering · doi:10.1088/1741-2552/abde8a · Cited by 69
An Enhanced LSTM for Trend Following of Time Series
Hu, Y., Sun, X., Nie, X., Li, Y., & Liu, L. (2019) · IEEE Access · doi:10.1109/access.2019.2896621 · Cited by 68
Disturbance pattern recognition based on an ALSTM in a long‐distance φ‐OTDR sensing system
Xue, C., & Xu, C. (2019) · Microwave and Optical Technology Letters · doi:10.1002/mop.32025 · Cited by 69
Experimental and deep learning artificial neural network approach for evaluating grid-connected photovoltaic systems
Kazem, H. A., Yousif, J. H., Chaichan, M. T., & Al‐Waeli, A. H. A. (2019) · International Journal of Energy Research · doi:10.1002/er.4855 · Cited by 69
Detection of Respiratory Infections Using RGB-Infrared Sensors on Portable Device
Jiang, Z., Hu, M., Gao, Z., Fan, L., Dai, R., Pan, Y., Tang, W., Zhai, G., & Lü, Y. (2020) · IEEE Sensors Journal · doi:10.1109/jsen.2020.3004568 · Cited by 70
A Reduced Order Deep Data Assimilation model
Quilodrán-Casas, C., Arcucci, R., Wu, P., Pain, C. C., & Guo, Y. (2020) · Physica D Nonlinear Phenomena · doi:10.1016/j.physd.2020.132615 · Cited by 71
Towards Artificial Intelligence Driven Emotion Aware Fall Monitoring Framework Suitable for Elderly People with Neurological Disorder
Nahian, M. J. A., Ghosh, T., Uddin, M. N., Islam, M. M., Mahmud, M., & Kaiser, M. S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59277-6_25 · Cited by 70
Clustering-enhanced stock price prediction using deep learning
Li, M., Zhu, Y., Shen, Y., & Angelova, M. N. (2022) · World Wide Web · doi:10.1007/s11280-021-01003-0 · Cited by 81
Classification of Driver Cognitive Load: Exploring the Benefits of Fusing Eye-Tracking and Physiological Measures
He, D., Wang, Z., Khalil, E. B., Donmez, B., Qiao, G., & Kumar, S. (2022) · Transportation Research Record Journal of the Transportation Research Board · doi:10.1177/03611981221090937 · Cited by 73
What is the best RNN-cell structure to forecast each time series behavior?
Khaldi, R., Afia, A. E., Chiheb, R., & Tabik, S. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.119140 · Cited by 79
NEMO
Li, L., Jing, H., Tong, H., Yang, J., He, Q., & Chen, B.-C. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3041021.3054200 · Cited by 76
Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP
(2019) · doi:10.18653/v1/w19-48 · Cited by 80
Using recurrent neural networks to optimize dynamical decoupling for quantum memory
August, M., & Ni, X. (2017) · Physical Review A · doi:10.1103/physreva.95.012335 · Cited by 71
Deep facial analysis: A new phase I epilepsy evaluation using computer vision
Ahmedt‐Aristizabal, D., Fookes, C., Nguyen, K., Denman, S., Sridharan, S., & Dionisio, S. A. (2018) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2018.02.010 · Cited by 65
Wind Speed Prediction Model Using LSTM and 1D-CNN
Fukuoka, R., Suzuki, H., Kitajima, T., Kuwahara, A., & Yasuno, T. (2018) · Journal of Signal Processing · doi:10.2299/jsp.22.207 · Cited by 68
In-depth comparison of deep artificial neural network architectures on seismic events classification
Canário, J. P., de Mello, R. F., Curilem, M., Huenupán, F., & Rios, R. A. (2020) · Journal of Volcanology and Geothermal Research · doi:10.1016/j.jvolgeores.2020.106881 · Cited by 69
Learning Adversarial Transformer for Symbolic Music Generation
Zhang, N. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2990746 · Cited by 72
Length-Controllable Image Captioning
Deng, C., Ding, N., Tan, M., & Wu, Q. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58601-0_42 · Cited by 70
Landslide Deformation Prediction Based on a GNSS Time Series Analysis and Recurrent Neural Network Model
Wang, J., Nie, G., Gao, S., Wu, S., Li, H., & Ren, X. (2021) · Remote Sensing · doi:10.3390/rs13061055 · Cited by 74
Predicting medication adherence using ensemble learning and deep learning models with large scale healthcare data
Gu, Y., Zalkikar, A., Liu, M., Kelly, L., Hall, A. R., Daly, K., & Ward, T. E. (2021) · Scientific Reports · doi:10.1038/s41598-021-98387-w · Cited by 68
The derived demand for advertising expenses and implications on sustainability: a comparative study using deep learning and traditional machine learning methods
Birim, Ş. Ö., Kazançoğlu, İ., Mangla, S. K., Kahraman, A., & Kazançoğlu, Y. (2022) · Annals of Operations Research · doi:10.1007/s10479-021-04429-x · Cited by 71
Carbon price forecasting: a novel deep learning approach
Zhang, F., & Wen, N. (2022) · Environmental Science and Pollution Research · doi:10.1007/s11356-022-19713-x · Cited by 70
EEG-based emotion charting for Parkinson's disease patients using Convolutional Recurrent Neural Networks and cross dataset learning
Dar, M. N., Akram, M. U., Yuvaraj, R., Khawaja, S. G., & Murugappan, M. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105327 · Cited by 68
Deep Learning for Channel Tracking in IRS-Assisted UAV Communication Systems
Yu, J., Liu, X., Gao, Y., Zhang, C., & Zhang, W. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3160517 · Cited by 70
Stock Market Analysis and Prediction Using LSTM: A Case Study on Technology Stocks
Li, Z., Yu, H., Xu, J., Liu, J., & Mo, Y. (2023) · Innovations in Applied Engineering and Technology · doi:10.62836/iaet.v2i1.162 · Cited by 68
PowerLSTM: Power Demand Forecasting Using Long Short-Term Memory Neural Network
Cheng, Y., Xu, C., Mashima, D., Thing, V. L. L., & Wu, Y. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-69179-4_51 · Cited by 70
Segmenting and classifying activities in robot-assisted surgery with recurrent neural networks
DiPietro, R. S., Ahmidi, N., Malpani, A., Waldram, M. M., Lee, G. I., Lee, M. R., Vedula, S. S., & Hager, G. D. (2019) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-019-01953-x · Cited by 72
A predictive model of train delays on a railway line
Wen, C., Mou, W., Huang, P., & Li, Z. (2019) · Journal of Forecasting · doi:10.1002/for.2639 · Cited by 74

Authors: Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J.
Affiliations: University of Applied Sciences and Arts of Southern Switzerland (CH)
ORCID: Greff, Klaus
Venue: IEEE Transactions on Neural Networks and Learning Systems
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2017-10
Volume: 28
Issue: 10
Pages: 2222-2232
ISSN: 2162-237X (ISSN-L)
2162-2388
DOI: 10.1109/tnnls.2016.2582924
OpenAlex: W1689711448
Cited by: 6,955
References: 53
Citations by year: 2026: 602 · 2025: 812 · 2024: 810 · 2023: 822 · 2022: 872 · 2021: 881 · 2020: 778 · 2019: 686 · 2018: 383 · 2017: 188
FWCI: 370.42
Type: journal article
Language: en
Open access: green
arxiv.org
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
Locations: IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
PubMed (publishedVersion)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Funders: Swiss National Science Foundation through the Project Theory and Practice of Reinforcement Learning 2, and through the Project Advanced Reinforcement Learning (138219, 156682)
European Institute of Innovation and Technology through the Project Neural Dynamics, through the Project NASCENCE and through the Project WAY (FP7-ICT-270247, FP7-ICT-317662, FP7-ICT-288551)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords: long short-term memory, recurrent neural networks, hyperparameter optimization, random search, functional analysis of variance, forget gate, speech recognition, handwriting recognition

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

LSTM: A Search Space Odyssey

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks