Cited by 6,955Neural Networks and ApplicationsSpeech Recognition and SynthesisTopic Modeling

The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.

LSTM: A Search Space Odyssey

Open access

Greff, Klaus, Srivastava, Rupesh K., Koutnik, Jan, Steunebrink, Bas R., Schmidhuber, Jurgen·2017·IEEE Transactions on Neural Networks and Learning Systems

DOI 10.1109/tnnls.2016.2582924

Open this paper in CoChat — free account Open PDF

Abstract

Several variants of the long short-term memory (LSTM) architecture for recurrent neural networks have been proposed since its inception in 1995. In recent years, these networks have become the state-of-the-art models for a variety of machine learning problems. This has led to a renewed interest in understanding the role and utility of various computational components of typical LSTM variants. In this paper, we present the first large-scale analysis of eight LSTM variants on three representative tasks: speech recognition, handwriting recognition, and polyphonic music modeling. The hyperparameters of all LSTM variants for each task were optimized separately using random search, and their importance was assessed using the powerful functional ANalysis Of VAriance framework. In total, we summarize the results of 5400 experimental runs ( ≈ 15 years of CPU time), which makes our study the largest of its kind on LSTM networks. Our results show that none of the variants can improve upon the standard LSTM architecture significantly, and demonstrate the forget gate and the output activation function to be its most critical components. We further observe that the studied hyperparameters are virtually independent and derive guidelines for their efficient adjustment.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1109/tnnls.2016.2582924

Cites

56 of 56 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Long Short-Term Memory
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Self-citation
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
Untitled work
(2012) · Cited by 8,009
Practical Bayesian Optimization of Machine Learning Algorithms
Snoek, J., Larochelle, H., & Adams, R. P. (2012) · arXiv · doi:10.48550/arxiv.1206.2944 · Cited by 6,691
Framewise phoneme classification with bidirectional LSTM and other neural network architectures
Graves, A., & Schmidhuber, J. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.06.042 · Cited by 6,152
Self-citation
Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks
Graves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143891 · Cited by 5,370
Self-citation
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
Untitled work
(2013) · Cited by 3,740

Flagged first, then by citations

Show all 53 references

Research works printed but not linked: 2

gers, 2002, DEKF-LSTM, Proc European Symp Artificial Neural Networks (ESANN), 369
robinson, 1987, The utility driven dynamic error propagation network

Sorted by citations

Items 401–500 of 1,013

A robust deep learning framework for short-term wind power forecast of a full-scale wind farm using atmospheric variables
Meka, R., Alaeddini, A., & Bhaganagar, K. (2021) · Energy · doi:10.1016/j.energy.2021.119759 · Cited by 132
Stacking integration algorithm based on CNN-BiLSTM-Attention with XGBoost for short-term electricity load forecasting
Luo, S., Wang, B., Gao, Q., Wang, Y., & Pang, X. (2024) · Energy Reports · doi:10.1016/j.egyr.2024.08.078 · Cited by 125
A hybrid deep learning-based method for short-term building energy load prediction combined with an interpretation process
Zhang, C., Li, J., Zhao, Y., Li, T., Chen, Q., & Zhang, X. (2020) · Energy and Buildings · doi:10.1016/j.enbuild.2020.110301 · Cited by 124
Deep Learning for Fall Risk Assessment With Inertial Sensors: Utilizing Domain Knowledge in Spatio-Temporal Gait Parameters
Tunca, C., Salur, G., & Ersoy, C. (2019) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2019.2958879 · Cited by 130
Pretrained domain-specific language model for natural language processing tasks in the AEC domain
Zheng, Z., Lu, X., Chen, K., Zhou, Y., & Lin, J. (2022) · Computers in Industry · doi:10.1016/j.compind.2022.103733 · Cited by 139
ReRAM-Based Processing-in-Memory Architecture for Recurrent Neural Network Acceleration
Long, Y., Na, T., & Mukhopadhyay, S. (2018) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2018.2819190 · Cited by 125
Investigating gated recurrent networks for speech synthesis
Wu, Z., & King, S. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472657 · Cited by 136
Machine Learning in Predictive Toxicology: Recent Applications and Future Directions for Classification Models
Wang, M. W. H., Goodman, J. M., & Allen, T. E. H. (2020) · Chemical Research in Toxicology · doi:10.1021/acs.chemrestox.0c00316 · Cited by 133
Over a decade of social opinion mining: a systematic review
Cortis, K., & Davis, B. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-10030-2 · Cited by 151
Attention-based Knowledge Graph Representation Learning for Predicting Drug-drug Interactions
Su, X., Hu, L., You, Z., Hu, P., & Zhao, B.-W. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac140 · Cited by 125
Recurrent Neural Networks Hardware Implementation on FPGAPreprint
Chang, A. X. M., Martini, B., & Culurciello, E. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.05552 · Cited by 148
A Spiking Neural Network Framework for Robust Sound Classification
Wu, J., Chua, Y., Zhang, M., Li, H., & Tan, K. C. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00836 · Cited by 131
DeepSite: bidirectional LSTM and CNN models for predicting DNA–protein binding
Zhang, Y., Qiao, S., Ji, S., & Li, Y. (2019) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-019-00990-x · Cited by 122
Software Vulnerability Discovery via Learning Multi-Domain Knowledge Bases
Lin, G., Zhang, J., Luo, W., Pan, L., Vel, O. D., Montague, P., & Xiang, Y. (2019) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2019.2954088 · Cited by 129
Multivariate Air Quality Forecasting With Nested Long Short Term Memory Neural Network
Jin, N., Zeng, Y., Yan, K., & Ji, Z. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3065425 · Cited by 126
A hybrid model for multi-step coal price forecasting using decomposition technique and deep learning algorithms
Zhang, K. F., Cao, H., Thé, J. V. G., & Yu, H. (2021) · Applied Energy · doi:10.1016/j.apenergy.2021.118011 · Cited by 126
Data-driven stock forecasting models based on neural networks: A review
Bao, W., Cao, Y., Yang, Y., Che, H., Huang, J., & Wen, S. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102616 · Cited by 166
Born to learn: The inspiration, progress, and future of evolved plastic artificial neural networks
Soltoggio, A., Stanley, K. O., & Risi, S. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.07.013 · Cited by 132
A Deep Evolutionary Approach to Bioinspired Classifier Optimisation for Brain‐Machine Interaction
Bird, J. J., Faria, D. R., Manso, L. J., Ekárt, A., & Buckingham, C. D. (2019) · Complexity · doi:10.1155/2019/4316548 · Cited by 136
Machine learning for condensed matter physics
Bedolla-Montiel, E. A., Padierna, L. C., & Castañeda-Priego, R. (2020) · Journal of Physics Condensed Matter · doi:10.1088/1361-648x/abb895 · Cited by 134
SSGRU: A novel hybrid stacked GRU-based traffic volume prediction approach in a road network
Sun, P., Boukerche, A., & Tao, Y. (2020) · Computer Communications · doi:10.1016/j.comcom.2020.06.028 · Cited by 121
Hierarchical Recurrent Neural Hashing for Image Retrieval With Hierarchical Convolutional Features
Lu, X., Chen, Y., & Li, X. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2755766 · Cited by 119
LASSO and LSTM Integrated Temporal Model for Short-Term Solar Intensity Forecasting
Wang, Y., Shen, Y., Mao, S., Chen, X., & Zou, H. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2877510 · Cited by 125
Long-short-term-memory-based crop classification using high-resolution optical images and multi-temporal SAR data
Zhou, Y., Luo, J., Feng, L., Yang, Y., Chen, Y., & Wu, W. (2019) · GIScience & Remote Sensing · doi:10.1080/15481603.2019.1628412 · Cited by 125
A robust deep learning approach for automatic classification of seizures against non-seizures
Yao, X., Li, X., Ye, Q., Huang, Y., Cheng, Q. S., & Zhang, G. (2020) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2020.102215 · Cited by 124
Safe Deep Reinforcement Learning for Microgrid Energy Management in Distribution Networks With Leveraged Spatial–Temporal Perception
Ye, Y., Wang, H., Chen, P., Tang, Y. P., & Štrbac, G. (2023) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2023.3243170 · Cited by 122
Machine Learning Solutions for the Security of Wireless Sensor Networks: A Review
Ghadi, Y. Y., Mazhar, T., Shloul, T. A., Shahzad, T., Salaria, U. A., Ahmed, A., & Hamam, H. (2024) · IEEE Access · doi:10.1109/access.2024.3355312 · Cited by 131
A Survey on Intelligent Network Operations and Performance Optimization Based on Large Language Models
Long, S., Tan, J., Mao, B., Tang, F., Li, Y., Zhao, M., & Kato, N. (2025) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2025.3526606 · Cited by 139
Neural Machine Translation and Sequence-to-sequence Models: A TutorialPreprint
Neubig, G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01619 · Cited by 194
Recent Advances in Deep Learning: An OverviewPreprint
Minar, M. R., & Naher, J. (2018) · arXiv (Cornell University) · doi:10.13140/rg.2.2.24831.10403 · Cited by 164
Abstractive Summarization: A Survey of the State of the Art
Lin, H., & Ng, V. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019815 · Cited by 120
Bilateral LSTM: A Two-Dimensional Long Short-Term Memory Model With Multiply Memory Units for Short-Term Cycle Time Forecasting in Re-entrant Manufacturing Systems
Wang, J., Zhang, J., & Wang, X. (2017) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2017.2754641 · Cited by 120
An Integrated Missing-Data Tolerant Model for Probabilistic PV Power Generation Forecasting
Li, Q., Xu, Y. H., Chew, B. S. H., Ding, H., & Zhao, G. (2022) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2022.3146982 · Cited by 128
Long short-term memory-based deep recurrent neural networks for target tracking
Gao, C., Yan, J., Zhou, S., Varshney, P. K., & Liu, H. (2019) · Information Sciences · doi:10.1016/j.ins.2019.06.039 · Cited by 118
Deep learning in biomedical data science
Baldi, P. (2018) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-080917-013343 · Cited by 119
Encoding Syntactic Knowledge in Neural Networks for Sentiment Classification
Huang, M., Qian, Q., & Zhu, X. (2017) · ACM Transactions on Information Systems · doi:10.1145/3052770 · Cited by 118
Memory-controlled deep LSTM neural network post-equalizer used in high-speed PAM VLC system
Lu, X., Lu, C., Yu, W., Qiao, L., Liang, S., Lau, A. P. T., & Chi, N. (2019) · Optics Express · doi:10.1364/oe.27.007822 · Cited by 118
Rainfall and runoff time-series trend analysis using LSTM recurrent neural network and wavelet neural network with satellite-based meteorological data: case study of Nzoia hydrologic basin
Ouma, Y. O., Cheruyot, R., & Wachera, A. N. (2021) · Complex & Intelligent Systems · doi:10.1007/s40747-021-00365-2 · Cited by 131
SigNet: A Novel Deep Learning Framework for Radio Signal Classification
Chen, Z., Cui, H., Xiang, J., Qiu, K., Huang, L. P., Zheng, S., Chen, S., Xuan, Q., & Yang, X. (2021) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2021.3120997 · Cited by 118
Machine learning research that matters for music creation: A case study
Sturm, B. L. T., Ben‐Tal, O., Monaghan, Ú., Collins, N., Herremans, D., Chew, E., Hadjeres, G., Deruty, E., & Francois Pachet, B. (2018) · Journal of New Music Research · doi:10.1080/09298215.2018.1515233 · Cited by 117
An Attention-based Bi-LSTM Method for Visual Object Classification via EEG
Zheng, X. H., & Chen, W. (2020) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2020.102174 · Cited by 117
Autoencoder-based Condition Monitoring and Anomaly Detection Method for Rotating Machines
Ahmad, S., Styp-Rekowski, K., Nedelkoski, S., & Kao, O. (2020) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata50022.2020.9378015 · Cited by 131
Tunnel boring machine vibration-based deep learning for the ground identification of working faces
Liu, M., Liao, S., Yang, Y., Men, Y., He, J., & Huang, Y. (2021) · Journal of Rock Mechanics and Geotechnical Engineering · doi:10.1016/j.jrmge.2021.09.004 · Cited by 118
RUDDER: Return Decomposition for Delayed RewardsPreprint
Arjona-Medina, J. A., Gillhofer, M., Widrich, M., Unterthiner, T., Brandstetter, J., & Hochreiter, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.07857 · Cited by 145
Rolling bearing health prognosis using a modified health index based hierarchical gated recurrent unit network
Li, X., Jiang, H., Xiong, X., & Shao, H. (2018) · Mechanism and Machine Theory · doi:10.1016/j.mechmachtheory.2018.11.005 · Cited by 116
Improved prediction of daily pan evaporation using Deep-LSTM model
Majhi, B., Naidu, D., Mishra, A. P., & Satapathy, S. C. (2019) · Neural Computing and Applications · doi:10.1007/s00521-019-04127-7 · Cited by 118
A Comprehensive Review of Driver Behavior Analysis Utilizing Smartphones
Chan, T. K., Chin, C. S., Chen, H., & Zhong, X. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2940481 · Cited by 116
Deep Learning for User Comment Moderation
Pavlopoulos, J., Malakasiotis, P., & Androutsopoulos, I. (2017) · Workshop on Abusive Language Online (ALW) · doi:10.18653/v1/w17-3004 · Cited by 136
Data-driven prognosis method using hybrid deep recurrent neural network
Xia, M., Zheng, X., Imran, M. A., & Shoaib, M. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106351 · Cited by 115
Machine learning for demand forecasting in the physical internet: a case study of agricultural products in Thailand
Kantasa–ard, A., Nouiri, M., Bekrar, A., el cadi, A. A., & Sallez, Y. (2020) · International Journal of Production Research · doi:10.1080/00207543.2020.1844332 · Cited by 128
A hybrid CNN-LSTM model for pre-miRNA classification
Taşdelen, A., & Şen, B. (2021) · Scientific Reports · doi:10.1038/s41598-021-93656-0 · Cited by 125
A novel decomposition-ensemble prediction model for ultra-short-term wind speed
Tian, Z., & Chen, H. (2021) · Energy Conversion and Management · doi:10.1016/j.enconman.2021.114775 · Cited by 115
Adult content detection in videos with convolutional and recurrent neural networks
Wehrmann, J., Simões, G. S., Barros, R. C., & Cavalcante, V. F. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.07.012 · Cited by 116
Computer network intrusion detection using sequential LSTM Neural Networks autoencoders
Mirza, A. H., & Coşan, S. (2018) · Signal Processing and Communications Applications Conference (SIU) · doi:10.1109/siu.2018.8404689 · Cited by 114
Artificial neural network based hybrid modeling approach for flood inundation modeling
Xie, S., Wu, W., Mooser, S., Wang, Q. J., Nathan, R. J., & Huang, Y. (2020) · Journal of Hydrology · doi:10.1016/j.jhydrol.2020.125605 · Cited by 115
Impact of climate change on river water temperature and dissolved oxygen: Indian riverine thermal regimes
Rajesh, M., & Rehana, S. (2022) · Scientific Reports · doi:10.1038/s41598-022-12996-7 · Cited by 130
Architectural Complexity Measures of Recurrent Neural NetworksPreprint
Zhang, S., Wu, Y., Che, T., Lin, Z., Memisevic, R., Salakhutdinov, R. R., & Bengio, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.08210 · Cited by 150
Large Language Model‐Based Chatbots in Higher Education
Yigci, D., Eryilmaz, M., Yetisen, A. K., Tasoglu, S., & Ozcan, A. (2025) · Advanced Intelligent Systems · doi:10.1002/aisy.202400429 · Cited by 113
A survey of machine learning methods applied to anomaly detection on drinking-water quality data
Dogo, E. M., Nwulu, N. I., Twala, B., & Aigbavboa, C. O. (2019) · Urban Water Journal · doi:10.1080/1573062x.2019.1637002 · Cited by 122
Learning Student Networks via Feature Embedding
Chen, H., Wang, Y., Xu, C., Xu, C., & Tao, D. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2970494 · Cited by 116
Probabilistic Prediction of Regional Wind Power Based on Spatiotemporal Quantile Regression
Yu, Y., Han, X., Yang, M. Y., & Yang, J. (2020) · IEEE Transactions on Industry Applications · doi:10.1109/tia.2020.2992945 · Cited by 113
Explainable stock prices prediction from financial news articles using sentiment analysis
Gite, S., Khatavkar, H., Kotecha, K. V., Srivastava, S., Maheshwari, P., & Pandey, N. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.340 · Cited by 126
Large-scale water quality prediction with integrated deep neural network
Bi, J., Lin, Y., Dong, Q., Yuan, H., & Zhou, M. (2021) · Information Sciences · doi:10.1016/j.ins.2021.04.057 · Cited by 118
Dual-side Sparse Tensor Core
Wang, Y., Zhang, C., Xie, Z., Guo, C., Liu, Y., & Leng, J. (2021) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca52012.2021.00088 · Cited by 118
Deep Learning Based Modeling of Groundwater Storage Change
Haq, M. A., Jilani, A. K., & Prabu, P. (2021) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2022.020495 · Cited by 113
Financial time series forecasting with deep learning : A systematic literature review: 2005–2019
Sezer, Ö. B., Gudelek, M. U., & Ozbayoglu, A. M. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106181 · Cited by 141
Android Malware Detection Based on a Hybrid Deep Learning Model
Lu, T., Du, Y., Ouyang, L., Chen, Q., & Wang, X. (2020) · Security and Communication Networks · doi:10.1155/2020/8863617 · Cited by 112
Deep Learning for Short-Term Voltage Stability Assessment of Power Systems
Zhang, M., Li, J., Li, Y., & Xu, R. (2021) · IEEE Access · doi:10.1109/access.2021.3057659 · Cited by 116
Data fusion in predicting internal heat gains for office buildings through a deep learning approach
Wang, Z., Hong, T., & Piette, M. A. (2019) · Applied Energy · doi:10.1016/j.apenergy.2019.02.066 · Cited by 113
A hybrid DL-driven intelligent SDN-enabled malware detection framework for Internet of Medical Things (IoMT)
Khan, S., & Akhunzada, A. (2021) · Computer Communications · doi:10.1016/j.comcom.2021.01.013 · Cited by 111
Energy consumption prediction and optimization of industrial robots based on LSTM
Jiang, P., Wang, Z., Li, X., Wang, X. V., Yang, B., & Zheng, J. (2023) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2023.07.009 · Cited by 111
Learning Recurrent Span Representations for Extractive Question AnsweringPreprint
Lee, K., Salant, S., Kwiatkowski, T., Parikh, A. P., Das, D., & Berant, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01436 · Cited by 118
Semantic Communication: A Survey on Research Landscape, Challenges, and Future Directions
Getu, T. M., Kaddoum, G., & Bennis, M. (2024) · Proceedings of the IEEE · doi:10.1109/jproc.2024.3520707 · Cited by 110
CNN-LSTM Neural Network Model for Quantitative Strategy Analysis in Stock Markets
Liu, S., Zhang, C., & Ma, J. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-70096-0_21 · Cited by 115
Long short-term memory neural network based fault detection and isolation for electro-mechanical actuators
Yang, J., Guo, Y., & Zhao, W. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.06.029 · Cited by 113
A Novel Wind Speed Interval Prediction Based on Error Prediction Method
Tang, G., Wu, Y., Li, C., Wong, P. K., Xiao, Z., & An, X. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.2973413 · Cited by 111
NAS-Bench-NLP: Neural Architecture Search Benchmark for Natural Language Processing
Klyuchnikov, N., Trofimov, I., Artemova, E., Salnikov, M., Fedorov, M. V., Filippov, A., & Burnaev, E. V. (2022) · IEEE Access · doi:10.1109/access.2022.3169897 · Cited by 135
An Enhanced Deep Learning-Based Fusion Prognostic Method for RUL Prediction
Huang, C., Yin, X., Huang, H., & Li, Y. (2019) · IEEE Transactions on Reliability · doi:10.1109/tr.2019.2948705 · Cited by 110
Physically Interpretable Wavelet-Guided Networks With Dynamic Frequency Decomposition for Machine Intelligence Fault Prediction
Wang, H., Li, Y., Men, T., & Li, L. (2024) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2024.3389068 · Cited by 114
Trellis Networks for Sequence ModelingPreprint
Bai, S., Kolter, J. Z., & Koltun, V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.06682 · Cited by 137
Stock Market Prediction with High Accuracy using Machine Learning Techniques
Bansal, M., Goyal, A., & Choudhary, A. (2022) · Procedia Computer Science · doi:10.1016/j.procs.2022.12.028 · Cited by 108
Towards Accurate Duplicate Bug Retrieval Using Deep Learning Techniques
Deshmukh, J., Annervaz, K. M., Podder, S., Sengupta, S., & Dubash, N. (2017) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2017.69 · Cited by 111
A Novel Equivalent Model of Active Distribution Networks Based on LSTM
Zheng, C., Wang, S., Liu, Y., Liu, C., Xie, W. C., Fang, C., & Liu, S. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2885219 · Cited by 109
Learning physical properties in complex visual scenes: An intelligent machine for perceiving blood flow dynamics from static CT angiography imaging
Gao, Z., Wang, X., Sun, S., Wu, D., Bai, J., Yin, Y., Liu, X., Zhang, H., & de Albuquerque, V. H. C. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.11.017 · Cited by 111
Enhanced Accuracy for Multiclass Mental Workload Detection Using Long Short-Term Memory for Brain–Computer Interface
Asgher, U., Khalil, K., Khan, M. J., Ahmad, R., Butt, S. I., Ayaz, Y., Naseer, N., & Nazir, S. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00584 · Cited by 111
Influence maximization in social networks using transfer learning via graph-based LSTM
Kumar, S., Mallik, A., & Panda, B. S. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118770 · Cited by 108
Human activity recognition from inertial sensor time-series using batch normalized deep LSTM recurrent networks
Zebin, T., Sperrin, M., Peek, N., & Casson, A. J. (2018) · 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2018.8513115 · Cited by 107
Morphological Inflection Generation Using Character Sequence to Sequence Learning
Faruqui, M., Tsvetkov, Y., Neubig, G., & Dyer, C. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1077 · Cited by 125
An integrated autoencoder-based hybrid CNN-LSTM model for COVID-19 severity prediction from lung ultrasound
Dastider, A. G., Sadik, F., & Fattah, S. A. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104296 · Cited by 112
Referring Expression Comprehension: A Survey of Methods and Datasets
Qiao, Y., Deng, C., & Wu, Q. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3042066 · Cited by 115
A hybrid deep learning framework with CNN and Bi-directional LSTM for store item demand forecasting
Joseph, R. V., Mohanty, A., Tyagi, S., Mishra, S., Satapathy, S. K., & Mohanty, S. N. (2022) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2022.108358 · Cited by 112
Deep Recurrent Neural Network for Protein Function Prediction from SequencePreprint
Liu, X. L. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/103994 · Cited by 120
Comparative Study of Deep Learning-Based Sentiment Classification
Seo, S., Kim, C., Kim, H., Mo, K., & Kang, P. (2020) · IEEE Access · doi:10.1109/access.2019.2963426 · Cited by 110
Sequential Model Based Intrusion Detection System for IoT Servers Using Deep Learning Methods
Zhong, M. J., Zhou, Y., & Chen, G. (2021) · Sensors · doi:10.3390/s21041113 · Cited by 105
A Multi-Layer Classification Approach for Intrusion Detection in IoT Networks Based on Deep Learning
Qaddoura, R., Al-Zoubi, A. M., Faris, H., & Almomani, I. (2021) · Sensors · doi:10.3390/s21092987 · Cited by 110
Multimodal Pretraining Unmasked: A Meta-Analysis and a Unified Framework of Vision-and-Language BERTs
Bugliarello, E., Cotterell, R., Okazaki, N., & Elliott, D. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00408 · Cited by 133
Proposing an Integrated Approach to Analyzing ESG Data via Machine Learning and Deep Learning Algorithms
Lee, O., Joo, H., Choi, H., & Cheon, M. (2022) · Sustainability · doi:10.3390/su14148745 · Cited by 105
Artificial intelligence for load forecasting: A stacking learning approach based on ensemble diversity regularization
Shi, J., Li, C., & Yan, X. (2022) · Energy · doi:10.1016/j.energy.2022.125295 · Cited by 105
DeepIntent
Zhai, S., Chang, K.-H., Zhang, R., & Zhang, Z. M. (2016) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2939672.2939759 · Cited by 105
AN OVERVIEW OF POPULAR DEEP LEARNING METHODS
Coşkun, M., Yıldırım, Ö., Uçar, A., & Demir, Y. (2017) · European Journal of Technic · doi:10.23884/ejt.2017.7.2.11 · Cited by 108

Authors: Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J.
Affiliations: University of Applied Sciences and Arts of Southern Switzerland (CH)
ORCID: Greff, Klaus
Venue: IEEE Transactions on Neural Networks and Learning Systems
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2017-10
Volume: 28
Issue: 10
Pages: 2222-2232
ISSN: 2162-237X (ISSN-L)
2162-2388
DOI: 10.1109/tnnls.2016.2582924
OpenAlex: W1689711448
Cited by: 6,955
References: 53
Citations by year: 2026: 602 · 2025: 812 · 2024: 810 · 2023: 822 · 2022: 872 · 2021: 881 · 2020: 778 · 2019: 686 · 2018: 383 · 2017: 188
FWCI: 370.42
Type: journal article
Language: en
Open access: green
arxiv.org
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
Locations: IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
PubMed (publishedVersion)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Funders: Swiss National Science Foundation through the Project Theory and Practice of Reinforcement Learning 2, and through the Project Advanced Reinforcement Learning (138219, 156682)
European Institute of Innovation and Technology through the Project Neural Dynamics, through the Project NASCENCE and through the Project WAY (FP7-ICT-270247, FP7-ICT-317662, FP7-ICT-288551)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords: long short-term memory, recurrent neural networks, hyperparameter optimization, random search, functional analysis of variance, forget gate, speech recognition, handwriting recognition

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

LSTM: A Search Space Odyssey

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks