Cited by 6,955Neural Networks and ApplicationsSpeech Recognition and SynthesisTopic Modeling

The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.

LSTM: A Search Space Odyssey

Open access

Greff, Klaus, Srivastava, Rupesh K., Koutnik, Jan, Steunebrink, Bas R., Schmidhuber, Jurgen·2017·IEEE Transactions on Neural Networks and Learning Systems

DOI 10.1109/tnnls.2016.2582924

Open this paper in CoChat — free account Open PDF

Abstract

Several variants of the long short-term memory (LSTM) architecture for recurrent neural networks have been proposed since its inception in 1995. In recent years, these networks have become the state-of-the-art models for a variety of machine learning problems. This has led to a renewed interest in understanding the role and utility of various computational components of typical LSTM variants. In this paper, we present the first large-scale analysis of eight LSTM variants on three representative tasks: speech recognition, handwriting recognition, and polyphonic music modeling. The hyperparameters of all LSTM variants for each task were optimized separately using random search, and their importance was assessed using the powerful functional ANalysis Of VAriance framework. In total, we summarize the results of 5400 experimental runs ( ≈ 15 years of CPU time), which makes our study the largest of its kind on LSTM networks. Our results show that none of the variants can improve upon the standard LSTM architecture significantly, and demonstrate the forget gate and the output activation function to be its most critical components. We further observe that the studied hyperparameters are virtually independent and derive guidelines for their efficient adjustment.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1109/tnnls.2016.2582924

Cites

56 of 56 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Long Short-Term Memory
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Self-citation
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
Untitled work
(2012) · Cited by 8,009
Practical Bayesian Optimization of Machine Learning Algorithms
Snoek, J., Larochelle, H., & Adams, R. P. (2012) · arXiv · doi:10.48550/arxiv.1206.2944 · Cited by 6,691
Framewise phoneme classification with bidirectional LSTM and other neural network architectures
Graves, A., & Schmidhuber, J. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.06.042 · Cited by 6,152
Self-citation
Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks
Graves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143891 · Cited by 5,370
Self-citation
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
Untitled work
(2013) · Cited by 3,740

Flagged first, then by citations

Show all 53 references

Research works printed but not linked: 2

gers, 2002, DEKF-LSTM, Proc European Symp Artificial Neural Networks (ESANN), 369
robinson, 1987, The utility driven dynamic error propagation network

Sorted by citations

Items 801–900 of 1,013

Deep Learning-based Extraction of Algorithmic Metadata in Full-Text Scholarly Documents
Safder, I., Hassan, S., Visvizi, A., Noraset, T., Nawaz, R., & Tuarob, S. (2020) · Information Processing & Management · doi:10.1016/j.ipm.2020.102269 · Cited by 73
The unreasonable effectiveness of the forget gatePreprint
van der Westhuizen, J., & Lasenby, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.04849 · Cited by 107
Fusion of tactile and visual information in deep learning models for object recognition
Babadian, R. P., Faez, K., Amiri, M., & Falotico, E. (2023) · Information Fusion · doi:10.1016/j.inffus.2022.11.032 · Cited by 67
A Two-Stage Neural Network for Sleep Stage Classification Based on Feature Learning, Sequence Learning, and Data Augmentation
Sun, C., Fan, J., Chen, C., Li, W., & Chen, W. (2019) · IEEE Access · doi:10.1109/access.2019.2933814 · Cited by 67
Remaining useful life prediction via a variational autoencoder and a time‐window‐based sequence neural network
Su, C., Li, L., & Wen, Z. (2020) · Quality and Reliability Engineering International · doi:10.1002/qre.2651 · Cited by 67
Deep Sequential Feature Learning in Clinical Image Classification of Infectious Keratitis
Xu, Y., Kong, M. C., Xie, W., Duan, R., Fang, Z., Lin, Y., Zhu, Q., Tang, S., Wu, F. Q., & Yao, Y. (2020) · Engineering · doi:10.1016/j.eng.2020.04.012 · Cited by 69
Time‐Series Prediction Approaches to Forecasting Deformation in Sentinel‐1 InSAR Data
Hill, P., Biggs, J., Ponce-López, V., & Bull, D. (2021) · Journal of Geophysical Research Solid Earth · doi:10.1029/2020jb020176 · Cited by 73
Rock Burst Precursor Electromagnetic Radiation Signal Recognition Method and Early Warning Application Based on Recurrent Neural Networks
Di, Y., & Wang, E. (2021) · Rock Mechanics and Rock Engineering · doi:10.1007/s00603-020-02314-w · Cited by 68
Thermal error prediction of ball screws based on PSO-LSTM
Gao, X., Guo, Y., Hanson, D. A., Liu, Z., Wang, M., & Zan, T. (2021) · The International Journal of Advanced Manufacturing Technology · doi:10.1007/s00170-021-07560-y · Cited by 67
Clock bias prediction algorithm for navigation satellites based on a supervised learning long short-term memory neural network
Huang, B., Ji, Z., Zhai, R., Xiao, C., Yang, F., Yang, B., & Wang, Y. (2021) · GPS Solutions · doi:10.1007/s10291-021-01115-0 · Cited by 70
Recurrent neural network from adder’s perspective: Carry-lookahead RNN
Jiang, H., Qin, F., Cao, J., Peng, Y. K., & Shao, Y. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.08.032 · Cited by 70
Uncertainty propagation for dropout-based Bayesian neural networks
Mae, Y., Kumagai, W., & Kanamori, T. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.09.005 · Cited by 70
An adaptive particle swarm optimization-based hybrid long short-term memory model for stock price time series forecasting
Kumar, G., Singh, U. P., & Jain, S. (2022) · Soft Computing · doi:10.1007/s00500-022-07451-8 · Cited by 75
Time Series Analysis Based on Informer Algorithms: A Survey
Zhu, Q., Han, J., Chai, K., & Zhao, C. (2023) · Symmetry · doi:10.3390/sym15040951 · Cited by 76
Deep Residual Convolutional and Recurrent Neural Networks for Temperature Estimation in Permanent Magnet Synchronous Motors
Kirchgässner, W., Wallscheid, O., & Böcker, J. (2019) · IEEE International Electric Machines & Drives Conference (IEMDC) · doi:10.1109/iemdc.2019.8785109 · Cited by 70
Guiding Long-Short Term Memory for Image Caption GenerationPreprint
Jia, X., Gavves, E., Fernando, B., & Tuytelaars, T. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.04942 · Cited by 67
Non-invasive wearable seizure detection using long–short-term memory networks with transfer learning
Nasseri, M., Pal Attia, T., Joseph, B., Gregg, N. M., Nurse, E. S., Viana, P. F., Schulze-Bonhage, A., Dümpelmann, M., Worrell, G., Freestone, D. R., et al. (2021) · Journal of Neural Engineering · doi:10.1088/1741-2552/abef8a · Cited by 66
Deep learning for mining protein data
Shi, Q., Chen, W., Huang, S., Wang, Y., & Xue, Z. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bbz156 · Cited by 68
Supervised data-driven approach to early kick detection during drilling operation
Muojeke, S., Venkatesan, R., & Khan, F. I. (2020) · Journal of Petroleum Science and Engineering · doi:10.1016/j.petrol.2020.107324 · Cited by 66
Indian sign language alphabet recognition system using CNN with diffGrad optimizer and stochastic pooling
Nandi, U., Ghorai, A., Singh, M. M., Changdar, C., Bhakta, S., & Pal, R. K. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-021-11595-4 · Cited by 69
Probabilistic Risk Metric for Highway Driving Leveraging Multi-Modal Trajectory Predictions
Wang, X., Alonso–Mora, J., & Wang, M. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3164469 · Cited by 69
Lightweight bidirectional long short-term memory based on automated model pruning with application to bearing remaining useful life prediction
Sun, J., Zhang, X., & Wang, J. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105662 · Cited by 68
Simplified gating in long short-term memory (LSTM) recurrent neural networks
Lu, Y., & Salem, F. M. (2017) · IEEE International Midwest Symposium on Circuits and Systems (MWSCAS) · doi:10.1109/mwscas.2017.8053244 · Cited by 76
Training Lightweight Deep Convolutional Neural Networks Using Bag-of-Features Pooling
Passalis, N., & Tefas, A. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2872995 · Cited by 68
Bit-Level Optimized Neural Network for Multi-Antenna Channel Quantization
Lu, C., Xu, W., Jin, S., & Wang, K. (2019) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2019.2942908 · Cited by 67
Quantity doesn’t buy quality syntax with neural language models
van Schijndel, M., Mueller, A., & Linzen, T. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1592 · Cited by 65
Accurate real time localization tracking in a clinical environment using Bluetooth Low Energy and deep learning
Iqbal, Z., Luo, D., Henry, P., Kazemifar, S., Rozario, T., Yan, Y., Westover, K. D., Lu, W., Nguyen, D. T. D., Long, T., et al. (2018) · PLoS ONE · doi:10.1371/journal.pone.0205392 · Cited by 69
Nonuniformly Sampled Data Processing Using LSTM Networks
Şahin, S. O., & Kozat, S. S. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2869822 · Cited by 67
Voltage Instability Prediction Using a Deep Recurrent Neural Network
Hagmar, H., Tong, L., Eriksson, R., & Tuấn, L. A. (2020) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2020.3008801 · Cited by 65
Effects of dataset characteristics on the performance of fatigue detection for crane operators using hybrid deep neural networks
Liu, P., Chi, H.-L., Li, X., & Guo, J. (2021) · Automation in Construction · doi:10.1016/j.autcon.2021.103901 · Cited by 66
Rolling Bearing Fault Diagnosis Method Based on Attention CNN and BiLSTM Network
Guo, Y., Mao, J., & Zhao, M. (2022) · Neural Processing Letters · doi:10.1007/s11063-022-11013-2 · Cited by 71
Multi-Modal Fusion Technology Based on Vehicle Information: A Survey
Zhang, X., Gong, Y., Lu, J., Wu, J., Li, Z., Jin, D., & Li, J. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3268051 · Cited by 73
Joint forecasting of source-load-price for integrated energy system based on multi-task learning and hybrid attention mechanism
Li, K., Mu, Y., Yang, F., Wang, H., Yan, Y., & Zhang, C. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.122821 · Cited by 69
Long short term memory networks for short-term electric load forecasting
Narayan, A., & Hipel, K. W. (2017) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2017.8123012 · Cited by 70
Exploiting Prediction Error Inconsistencies through LSTM-based Classifiers to Detect Deepfake Videos
Amerini, I., & Caldelli, R. (2020) · Proceedings of the 2020 ACM Workshop on Information Hiding and Multimedia Security · doi:10.1145/3369412.3395070 · Cited by 64
Unified Pragmatic Models for Generating and Following Instructions
Fried, D., Andreas, J., & Klein, D. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1177 · Cited by 64
Depth Estimation of Hard Inclusions in Soft Tissue by Autonomous Robotic Palpation Using Deep Recurrent Neural Network
Xiao, B. J., Xu, W., Guo, J., Lam, H., Jia, G., Hong, W., & Ren, H. (2020) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2020.2978881 · Cited by 65
Visual and Textual Deep Feature Fusion for Document Image Classification
Bakkali, S., Ming, Z., Coustaty, M., & Rusiñol, M. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00289 · Cited by 63
EMD and LSTM Hybrid Deep Learning Model for Predicting Sunspot Number Time Series with a Cyclic Pattern
Lee, T. (2020) · Solar Physics · doi:10.1007/s11207-020-01653-9 · Cited by 64
Long Short-Term Memory Spiking Networks and Their Applications
Rezaabad, A. L., & Vishwanath, S. (2020) · International Conference on Neuromorphic Systems (ICONS) · doi:10.1145/3407197.3407211 · Cited by 71
Theory of Gating in Recurrent Neural Networks
Krishnamurthy, K., Can, T., & Schwab, D. J. (2022) · Physical Review X · doi:10.1103/physrevx.12.011011 · Cited by 68
Machine learning based decline curve analysis for short-term oil production forecast
Tadjer, A., Hong, A., & Bratvold, R. B. (2021) · Energy Exploration & Exploitation · doi:10.1177/01445987211011784 · Cited by 71
Model-Based Representation and Deinterleaving of Mixed Radar Pulse Sequences With Neural Machine Translation Network
Zhu, M., Wang, S., & Li, Y. (2021) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2021.3122411 · Cited by 65
Deep-BERT: Transfer Learning for Classifying Multilingual Offensive Texts on Social Media
Wadud, M. A. H., Mridha, M. F., Shin, J., Nur, K. M., & Saha, A. K. (2022) · Computer Systems Science and Engineering · doi:10.32604/csse.2023.027841 · Cited by 70
Artificial intelligence‐enhanced epileptic seizure detection by wearables
Yu, S., Atrache, R. E., Tang, J., Jackson, M. C., Makarucha, A. J., Cantley, S., Sheehan, T., Vieluf, S., Zhang, B., Rogers, J. L., et al. (2023) · Epilepsia · doi:10.1111/epi.17774 · Cited by 64
A Historical Survey of Advances in Transformer Architectures
Sajun, A. R., Zualkernan, I. A., & Sankalpa, D. (2024) · Applied Sciences · doi:10.3390/app14104316 · Cited by 80
Combination of Deep Recurrent Neural Networks and Conditional Random Fields for Extracting Adverse Drug Reactions from User Reviews
Tutubalina, E. V., & Nikolenko, S. (2017) · Journal of Healthcare Engineering · doi:10.1155/2017/9451342 · Cited by 64
Regularized LSTM Based Deep Learning Model: First Step towards Real-Time Non-Intrusive Load Monitoring
Rafiq, H., Zhang, H., Li, H., & Ochani, M. K. (2018) · IEEE International Conference on Smart Energy Grid Engineering (SEGE) · doi:10.1109/sege.2018.8499519 · Cited by 66
A Hitchhiker’s Guide On Distributed Training Of Deep Neural Networks
Chahal, K., Grover, M. S., Dey, K., & Shah, R. R. (2019) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2019.10.004 · Cited by 69
An LSTM-based Indoor Positioning Method Using Wi-Fi Signals
Sahar, A., & Han, D. (2018) · International Conference on Vision, Image and Signal Processing (ICVISP) · doi:10.1145/3271553.3271566 · Cited by 64
Bidirectional Encoder Representations from Transformers (BERT): A sentiment analysis odysseyPreprint
Alaparthi, S., & Mishra, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.01127 · Cited by 64
The Sacred Infrastructure for Computational Research
Greff, K., Klein, A., Chovanec, M., Hutter, F., & Schmidhuber, J. (2017) · Proceedings of the Python in Science Conferences · doi:10.25080/shinma-7f4c6e7-008 · Cited by 63
Self-citation
Long short-term memory recurrent neural network-based acoustic model using connectionist temporal classification on a large-scale training corpus
Lee, D.-H., Lim, M., Park, H.-S., Kang, Y., Park, J., Jang, G., & Kim, J. (2017) · China Communications · doi:10.1109/cc.2017.8068761 · Cited by 67
When 5G Meets Deep Learning: A Systematic Review
Santos, G. L., Endo, P. T., Sadok, D. F. H., & Kelner, J. (2020) · Algorithms · doi:10.3390/a13090208 · Cited by 66
A Framework of Dynamic Data Driven Digital Twin for Complex Engineering Products: the Example of Aircraft Engine Health Management
Wu, Z., & Li, J. (2021) · Procedia Manufacturing · doi:10.1016/j.promfg.2021.10.020 · Cited by 63
A short-term probabilistic photovoltaic power prediction method based on feature selection and improved LSTM neural network
Liu, R., Wei, J., Sun, G., Muyeen, S. M., Lin, S., & Li, F. (2022) · Electric Power Systems Research · doi:10.1016/j.epsr.2022.108069 · Cited by 65
Multi-objective operation of cascade reservoirs based on short-term ensemble streamflow prediction
He, S., Guo, S., Zhang, J., Liu, Z., Cui, Z., Zhang, Y., & Zheng, Y. (2022) · Journal of Hydrology · doi:10.1016/j.jhydrol.2022.127936 · Cited by 63
A Deep RL-Based Algorithm for Coordinated Charging of Electric Vehicles
Zhang, Z., Wan, Y., Qin, J., Fu, W., & Kang, Y. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3170000 · Cited by 63
Two-stream transformer network for sensor-based human activity recognition
Xiao, S., Wang, S., Huang, Z., Wang, Y., & Jiang, H. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.09.099 · Cited by 66
Predicting next hour fine particulate matter (PM2.5) in the Istanbul Metropolitan City using deep learning algorithms with time windowing strategy
Eren, B., Aksangür, İ., & Erden, C. (2023) · Urban Climate · doi:10.1016/j.uclim.2023.101418 · Cited by 63
Investigation of long short-term memory networks to temperature prediction for permanent magnet synchronous motors
Wallscheid, O., Kirchgässner, W., & Böcker, J. (2017) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966088 · Cited by 68
Classification of motor imagery eeg using wavelet envelope analysis and LSTM networks
Zhou, J., Meng, M., Gao, Y., Ma, Y., & Zhang, Q. (2018) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc.2018.8408108 · Cited by 68
A novel hybrid model on the prediction of time series and its application for the gold price analysis and forecasting
Jianwei, E., Ye, J., & Jin, H. (2019) · Physica A Statistical Mechanics and its Applications · doi:10.1016/j.physa.2019.121454 · Cited by 75
An EEMD-BiLSTM Algorithm Integrated with Boruta Random Forest Optimiser for Significant Wave Height Forecasting along Coastal Areas of Queensland, Australia
Raj, N., & Brown, J. L. (2021) · Remote Sensing · doi:10.3390/rs13081456 · Cited by 63
Improving performance of recurrent neural network with relu nonlinearityPreprint
Talathi, S. S., & Vartak, A. A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.03771 · Cited by 91
Improved global 250 m 8-day NDVI and EVI products from 2000–2021 using the LSTM model
Xiong, C., Ma, H., Liang, S., He, T., Zhang, Y., Zhang, G., & Xu, J. (2023) · Scientific Data · doi:10.1038/s41597-023-02695-x · Cited by 63
AI-based techniques for multi-step streamflow forecasts: application for multi-objective reservoir operation optimization and performance assessment
Guo, Y., Xu, Y., Xu, Y., Chen, H., Gu, H., & Xie, J. (2021) · Hydrology and earth system sciences · doi:10.5194/hess-25-5951-2021 · Cited by 63
Open source dataset generator for power quality disturbances with deep-learning reference classifiers
Machlev, R., Chachkes, A., Belikov, J., Beck, Y., & Levron, Y. (2021) · Electric Power Systems Research · doi:10.1016/j.epsr.2021.107152 · Cited by 66
Digital twins based on bidirectional LSTM and GAN for modelling the COVID-19 pandemic
Quilodrán-Casas, C., Silva, V. L. S., Arcucci, R., Heaney, C. E., Guo, Y., & Pain, C. C. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.10.043 · Cited by 64
Machine Learning and Deep Learning for Phishing Email Classification using One-Hot Encoding
Bagui, S. S., Nandi, D., Bagui, S., & White, R. J. (2021) · Journal of Computer Science · doi:10.3844/jcssp.2021.610.623 · Cited by 80
Temporal deep learning architecture for prediction of COVID-19 cases in India
Verma, H., Mandal, S., & Gupta, A. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.116611 · Cited by 66
E-SFD: Explainable Sensor Fault Detection in the ICS Anomaly Detection System
Hwang, C., & Lee, T. (2021) · IEEE Access · doi:10.1109/access.2021.3119573 · Cited by 62
Spatial-Temporal Deep Learning for Hosting Capacity Analysis in Distribution Grids
Wu, J., Yuan, J., Weng, Y., & Ayyanar, R. (2022) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2022.3196943 · Cited by 65
Classification of Cassava Leaf Disease Based on a Non-Balanced Dataset Using Transformer-Embedded ResNet
Zhong, Y., Huang, B., & Tang, C. (2022) · Agriculture · doi:10.3390/agriculture12091360 · Cited by 62
XAI Transformer based Approach for Interpreting Depressed and Suicidal User Behavior on Online Social Networks
Malhotra, A., & Jindal, R. (2023) · Cognitive Systems Research · doi:10.1016/j.cogsys.2023.101186 · Cited by 72
Input quality aware convolutional LSTM networks for virtual marine sensors
Oehmcke, S., Zielinski, O., & Krämer, O. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.11.027 · Cited by 61
Deep-Q
Xiao, S., He, D., & Gong, Z. (2018) · ACM SIGCOMM Workshop on Network Meets AI & ML (NetAI) · doi:10.1145/3229543.3229549 · Cited by 62
Proceedings of the First Workshop on Abusive Language Online
(2017) · doi:10.18653/v1/w17-30 · Cited by 63
Time-Series Prediction of Environmental Noise for Urban IoT Based on Long Short-Term Memory Recurrent Neural Network
Zhang, X., Zhao, M. Y., & Dong, R. (2020) · Applied Sciences · doi:10.3390/app10031144 · Cited by 69
Deep6mA: A deep learning framework for exploring similar patterns in DNA N6-methyladenine sites across different species
Li, Z., Jiang, H., Kong, L., Chen, Y., Lang, K., Fan, X., Zhang, L., & Pian, C. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008767 · Cited by 61
Forecasting turning points in stock price by applying a novel hybrid CNN-LSTM-ResNet model fed by 2D segmented images
Khodaee, P., Esfahanipour, A., & Taheri, H. M. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105464 · Cited by 64
Traffic flow prediction using bi-directional gated recurrent unit method
Wang, S., Shao, C., Zhang, J., Zheng, Y., & Meng, M. (2022) · Urban Informatics · doi:10.1007/s44212-022-00015-z · Cited by 61
Deep learning approaches for seizure video analysis: A review
Ahmedt‐Aristizabal, D., Armin, M. A., Hayder, Z., Garcia‐Cairasco, N., Petersson, L., Fookes, C., Denman, S., & McGonigal, A. (2024) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2024.109735 · Cited by 65
Anomalous Sound Detection Using Deep Audio Representation and a BLSTM Network for Audio Surveillance of Roads
Li, Y., Li, X., Zhang, Y., Liu, M., & Wang, W. (2018) · IEEE Access · doi:10.1109/access.2018.2872931 · Cited by 61
Exploration of Complementary Features for Speech Emotion Recognition Based on Kernel Extreme Learning Machine
Guo, L., Wang, L., Dang, J., Liu, Z., & Guan, H. (2019) · IEEE Access · doi:10.1109/access.2019.2921390 · Cited by 62
A Decomposition-Ensemble Learning Model Based on LSTM Neural Network for Daily Reservoir Inflow Forecasting
Qi, Y., Zhou, Z., Yang, L., Quan, Y., & Miao, Q. (2019) · Water Resources Management · doi:10.1007/s11269-019-02345-1 · Cited by 63
Deep learning as phase retrieval tool for CARS spectra
Houhou, R., Barman, P., Schmitt, M., Meyer, T., Popp, J., & Bocklitz, T. W. (2020) · Optics Express · doi:10.1364/oe.390413 · Cited by 64
OxIOD: The Dataset for Deep Inertial OdometryPreprint
Chen, C., Zhao, P., Lu, C. X., Wang, W., Markham, A., & Trigoni, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.07491 · Cited by 76
Online Spatio-Temporal Learning in Deep Neural Networks
Bohnstingl, T., Woźniak, S., Pantazi, A., & Eleftheriou, E. S. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3153985 · Cited by 74
Multivariate Time Series Forecasting with Dilated Residual Convolutional Neural Networks for Urban Air Quality Prediction
Benhaddi, M., & Ouarzazi, J. (2021) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-020-05109-x · Cited by 61
Integrating Machine Learning, Radio Frequency Identification, and Consignment Policy for Reducing Unreliability in Smart Supply Chain Management
Sardar, S. K., Sarkar, B., & Kim, B. (2021) · Processes · doi:10.3390/pr9020247 · Cited by 60
A deep learning based approach for predicting the demand of electric vehicle charge
Mekkaoui, D. E., & Shen, Y. (2022) · The Journal of Supercomputing · doi:10.1007/s11227-022-04428-0 · Cited by 63
Spatial-temporal wave height forecast using deep learning and public reanalysis dataset
Ti, Z., Yubing, S., & Xiaowei, D. (2022) · Applied Energy · doi:10.1016/j.apenergy.2022.120027 · Cited by 60
Forecasting seasonal demand for retail: A Fourier time-varying grey model
Ye, L., Xie, N., Boylan, J. E., & Shang, Z. (2024) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2023.12.006 · Cited by 60
Deep learning-enabled integration of renewable energy sources through photovoltaics in buildings
Arun, M., Le, T. T., Barik, D., Sharma, P., Osman, S. M., Huynh, V. K., Kowalski, J., Dong, V. H., & Le, V. V. (2024) · Case Studies in Thermal Engineering · doi:10.1016/j.csite.2024.105115 · Cited by 60
Assessing dengue forecasting methods: a comparative study of statistical models and machine learning techniques in Rio de Janeiro, Brazil
Chen, X., & Moraga, P. (2025) · Tropical Medicine and Health · doi:10.1186/s41182-025-00723-7 · Cited by 70
Residential Energy Management with Deep Reinforcement Learning
Wan, Z., Li, H., & He, H. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489210 · Cited by 60
A Hybrid Deep Model Using Deep Learning and Dense Optical Flow Approaches for Human Activity Recognition
Tanberk, S., Kilimci, Z. H., Tükel, D. B., Uysal, M., & Akyokuş, S. (2020) · IEEE Access · doi:10.1109/access.2020.2968529 · Cited by 64
Information Entropy-Based Intention Prediction of Aerial Targets under Uncertain and Incomplete Information
Zhou, T., Chen, M., Wang, Y., He, J., & Yang, C. (2020) · Entropy · doi:10.3390/e22030279 · Cited by 59
Determining Physical Activity Characteristics From Wristband Data for Use in Automated Insulin Delivery Systems
Sevil, M., Rashid, M. M., Maloney, Z., Hajizadeh, I., Samadi, S., Askari, M. R., Hobbs, N., Brandt, R., Park, M., Quinn, L., et al. (2020) · IEEE Sensors Journal · doi:10.1109/jsen.2020.3000772 · Cited by 61

Authors: Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J.
Affiliations: University of Applied Sciences and Arts of Southern Switzerland (CH)
ORCID: Greff, Klaus
Venue: IEEE Transactions on Neural Networks and Learning Systems
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2017-10
Volume: 28
Issue: 10
Pages: 2222-2232
ISSN: 2162-237X (ISSN-L)
2162-2388
DOI: 10.1109/tnnls.2016.2582924
OpenAlex: W1689711448
Cited by: 6,955
References: 53
Citations by year: 2026: 602 · 2025: 812 · 2024: 810 · 2023: 822 · 2022: 872 · 2021: 881 · 2020: 778 · 2019: 686 · 2018: 383 · 2017: 188
FWCI: 370.42
Type: journal article
Language: en
Open access: green
arxiv.org
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
Locations: IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
PubMed (publishedVersion)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Funders: Swiss National Science Foundation through the Project Theory and Practice of Reinforcement Learning 2, and through the Project Advanced Reinforcement Learning (138219, 156682)
European Institute of Innovation and Technology through the Project Neural Dynamics, through the Project NASCENCE and through the Project WAY (FP7-ICT-270247, FP7-ICT-317662, FP7-ICT-288551)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords: long short-term memory, recurrent neural networks, hyperparameter optimization, random search, functional analysis of variance, forget gate, speech recognition, handwriting recognition

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

LSTM: A Search Space Odyssey

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks