Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Open accessAbstract
In this paper we compare different types of recurrent units in recurrent neural networks (RNNs). Especially, we focus on more sophisticated units that implement a gating mechanism, such as a long short-term memory (LSTM) unit and a recently proposed gated recurrent unit (GRU). We evaluate these recurrent units on the tasks of polyphonic music modeling and speech signal modeling. Our experiments revealed that these advanced recurrent units are indeed better than more traditional recurrent units such as tanh units. Also, we found GRU to be comparable to LSTM.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Recurrent neural networks and their variants, including long short-term memory and gated recurrent units, are effective for sequence modeling, capturing the impact of historical measurements on the current SOC.
“RNNs and their variants, including long short-term memory (LSTM)[23] and gated recurrent units (GRUs)[24], are effective for sequence modeling, capturing the impact of historical measurements on the current SOC[25].”
From Advancements in the estimation of the state of charge of lithium-ion battery: a comprehensive review of traditional and deep learning approaches (2025) · cites this paper as [24]
The Background (p. 2) says RNNs using LSTM or GRU units perform well on tasks requiring long-term dependencies, and the Conclusion (p. 7) reports their effectiveness in sequence modeling. The paper studies music and speech data, however; it does not establish that these networks capture the effect of historical measurements on current battery state of charge (SOC). (LLM, conf=0.88)
Experimental — accuracy not yet measuredGRU offers a significant reduction in computational complexity compared to LSTM network, making it a more lightweight and efficient alternative.
“Additionally, it offers a significant reduction in computational complexity when examined to the LSTM network, making it a more lightweight and efficient alternative.”
From X-GEVON- a novel explainable intelligent network to detect the multiple attacks in vanet systems (2025) · cites this paper as [34]
The Introduction (p. 1) reports that, on some datasets and with approximately equal parameter counts, GRU models converged faster than LSTM models in CPU time. The Results and Analysis (p. 6) likewise reports faster progress on the music datasets but says the overall GRU–LSTM comparison is inconclusive. These findings do not establish a significant reduction in computational complexity or that GRU is generally a more lightweight alternative. (LLM, conf=0.85)
Experimental — accuracy not yet measuredRecurrent neural networks, including long short-term memory and gated recurrent networks, are state-of-the-art approaches in sequence modeling and transduction problems such as language modeling and machine translation.
“Recurrent neural networks, long short-term memory [13] and gated recurrent [7] neural networks in particular, have been firmly established as state of the art approaches in sequence modeling and transduction problems such as language modeling and machine translation [35, 2, 5].”
From Attention Is All You Need (2017) · cites this paper as [7]
GRU is particularly effective in dealing with extended sequences and retaining long-term dependencies.
“GRU is particularly effective in dealing with extended sequences and retaining long-term dependencies.”
From X-GEVON- a novel explainable intelligent network to detect the multiple attacks in vanet systems (2025) · cites this paper as [34]
The fourth layer of CPC is the Gated Recurrent Unit, which also learns representations similar to supervised learning.
“The fourth layer of CPC is the Gated Recurrent Unit [8], which also learns representations similar to supervised learning.”
From Assessing the State of Self-Supervised Human Activity Recognition Using Wearables (2022) · cites this paper as [8]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · see citing works
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · see citing works
Self-citationSpeech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · see citing works
(2012) · Cited by 8,009 · see citing works
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches
Cho, K., van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014) · SSST@EMNLP · doi:10.3115/v1/w14-4012 · Cited by 7,251 · see citing works
Self-citationSupervised Sequence Labelling with Recurrent Neural Networks
Graves, A. (2012) · Studies in Computational Intelligence · doi:10.1007/978-3-642-24797-2 · Cited by 3,120 · see citing works
Generating Sequences With Recurrent Neural Networks
Graves, A. (2013) · arXiv · doi:10.48550/arxiv.1308.0850 · Cited by 2,894 · see citing works
Practical Variational Inference for Neural Networks
Graves, A. (2011) · Cited by 1,079 · see citing works
Theano: new features and speed improvements
Bastien, F., Lamblin, P., Pascanu, R., Bergstra, J., Goodfellow, I., Bergeron, A., Bouchard, N., Warde-Farley, D., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5590 · Cited by 1,005 · see citing works
Self-citationLearning Recurrent Neural Networks with Hessian-Free Optimization
Martens, J., & Sutskever, I. (2011) · Cited by 525 · see citing works
Boulanger-Lewandowski, N., Bengio, Y., & Vincent, P. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6392 · Cited by 484 · see citing works
Self-citationAdvances in optimizing recurrent networks
Bengio, Y., Boulanger-Lewandowski, N., & Pascanu, R. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639349 · Cited by 478 · see citing works
Self-citationPylearn2: a machine learning research libraryPreprint
Goodfellow, I., Warde-Farley, D., Lamblin, P., Dumoulin, V., Mirza, M., Pascanu, R., Bergstra, J., Bastien, F., & Bengio, Y. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1308.4214 · Cited by 245 · see citing works
Self-citationLearned-Norm Pooling for Deep Feedforward and Recurrent Neural Networks
Gülçehre, Ç., Cho, K., Pascanu, R., & Bengio, Y. (2014) · Lecture notes in computer science · doi:10.1007/978-3-662-44848-9_34 · Cited by 174 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 4
- Y. Bengio, P. Simard, and P. Frasconi. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks , 5(2):157-166, 1994.
- J. Bergstra, O. Breuleux, F. Bastien, P. Lamblin, R. Pascanu, G. Desjardins, J. Turian, D. WardeFarley, and Y. Bengio. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy) , June 2010. Oral Presentation.
- S. Hochreiter. Untersuchungen zu dynamischen neuronalen Netzen. Diploma thesis, Institut f¨ ur Informatik, Lehrstuhl Prof. Brauer, Technische Universit¨ at M¨ unchen, 1991. URL http:// www7.informatik.tu-muenchen.de/˜Ehochreit .
- R. Pascanu, T. Mikolov, and Y. Bengio. On the difficulty of training recurrent neural networks. In Proceedings of the 30th International Conference on Machine Learning (ICML'13) . ACM, 2013. URL http://icml.cc/2013/ .
Cited by
Sorted by citations
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955 · see citing works
Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules
Gómez-Bombarelli, R., Wei, J. N., Duvenaud, D., Hernández-Lobato, J. M., Sánchez-Lengeling, B., Sheberla, D., Aguilera-Iparraguirre, J., Hirzel, T. D., Adams, R. P., & Aspuru-Guzik, A. (2018) · ACS Central Science · doi:10.1021/acscentsci.7b00572 · Cited by 3,106 · see citing works
Attention Based Spatial-Temporal Graph Convolutional Networks for Traffic Flow Forecasting
Guo, S., Lin, Y., Feng, N., Song, C., & Wan, H. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301922 · Cited by 3,123 · see citing works
Recent Trends in Deep Learning Based Natural Language Processing [Review Article]
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2018) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2018.2840738 · Cited by 2,921 · see citing works
Deep learning and its applications to machine health monitoring
Zhao, R., Yan, R., Chen, Z., Mao, K., Wang, P., & Gao, R. X. (2019) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2018.05.050 · Cited by 2,894 · see citing works
Film: Visual reasoning with a general conditioning layer
Perez, E., Strub, F., De Vries, H., Dumoulin, V., & Courville, A. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11671 · Cited by 2,490 · see citing works
Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond
Nallapati, R., Zhou, B., dos Santos, C., Gulcehre, C., & Xiang, B. (2016) · Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning · doi:10.18653/v1/k16-1028 · Cited by 2,361 · see citing works
Self-citationZhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., & Fidler, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.11 · Cited by 2,264 · see citing works
Deep Learning for Hyperspectral Image Classification: An Overview
Li, S., Song, W., Fang, L., Chen, Y., Ghamisi, P., & Benediktsson, J. A. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2907932 · Cited by 2,007 · see citing works
Review of Deep Learning Algorithms and Architectures
Shrestha, A., & Mahmood, A. (2019) · IEEE Access · doi:10.1109/access.2019.2912200 · Cited by 1,883 · see citing works
“GRUs are smaller in size than LSTM as they don’t include the output gate, and can perform better than LSTM on only some simpler datasets [44], [45].”
BackgroundConceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning
Sharma, P., Ding, N., Goodman, S., & Soricut, R. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1238 · Cited by 1,879 · see citing works
Natural language processing: state of the art, current trends and challenges
Khurana, D., Koli, A., Khatter, K., & Singh, S. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13428-4 · Cited by 1,823 · see citing works
Robust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network
Su, Y., Zhao, Y., Niu, C., Liu, R., Sun, W., & Pei, D. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330672 · Cited by 1,804 · see citing works
Tacotron: towards end-to-end speech synthesis
Wang, Y., Skerry-Ryan, R. J., Stanton, D., Wu, Y., Weiss, R. J., Jaitly, N., Yang, Z., Xiao, Y., Chen, Z., Bengio, S., et al. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1452 · Cited by 1,746 · see citing works
A Structured Self-attentive Sentence Embedding
Lin, Z., Feng, M., Santos, C. N. D., Yu, M., Xiang, B., Zhou, B., & Bengio, Y. (2017) · arXiv · doi:10.48550/arxiv.1703.03130 · Cited by 1,743 · see citing works
“Several models have been proposed along this line, by using recurrent networks (Hochreiter & Schmidhuber, 1997; Chung et al., 2014), recursive networks (Socher et al.”
MethodSelf-citationPre-trained models for natural language processing: A survey
Qiu, X., Sun, T., Xu, Y., Shao, Y., Dai, N., & Huang, X. (2020) · Science China Technological Sciences · doi:10.1007/s11431-020-1647-3 · Cited by 1,591 · see citing works
A clinically applicable approach to continuous prediction of future acute kidney injury
Tomašev, N., Glorot, X., Rae, J. W., Zielinski, M., Askham, H., Saraiva, A., Mottram, A., Meyer, C., Ravuri, S., Protsyuk, I., et al. (2019) · Nature · doi:10.1038/s41586-019-1390-1 · Cited by 1,086 · see citing works
Learning quadrupedal locomotion over challenging terrain
Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., & Hutter, M. (2020) · Science Robotics · doi:10.1126/scirobotics.abc5986 · Cited by 1,156 · see citing works
Machine Learning and Deep Learning Methods for Intrusion Detection Systems: A Survey
Liu, H., & Lang, B. (2019) · Applied Sciences · doi:10.3390/app9204396 · Cited by 1,133 · see citing works
Deep Interest Evolution Network for Click-Through Rate Prediction
Zhou, G., Mou, N., Fan, Y., Pi, Q., Bian, W., Zhou, C., Zhu, X., & Gai, K. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33015941 · Cited by 1,099 · see citing works
Chatbots: History, technology, and applications
Adamopoulou, E., & Moussiades, L. (2020) · Machine Learning with Applications · doi:10.1016/j.mlwa.2020.100006 · Cited by 1,090 · see citing works
Speech-Transformer: A No-Recurrence Sequence-to-Sequence Model for Speech Recognition
Dong, L., Xu, S., & Xu, B. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8462506 · Cited by 1,100 · see citing works
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Gu, A., & Dao, T. (2023) · arXiv · doi:10.48550/arxiv.2312.00752 · Cited by 1,579 · see citing works
Nweke, H. F., Teh, Y. W., Al-garadi, M. A., & Alo, U. R. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.03.056 · Cited by 930 · see citing works
Rasmy, L., Xiang, Y., Xie, Z., Tao, C., & Zhi, D. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00455-y · Cited by 886 · see citing works
A Comprehensive Survey of Deep Learning for Image Captioning
Hossain, M. Z., Sohel, F., Shiratuddin, M. F., & Laga, H. (2019) · ACM Computing Surveys · doi:10.1145/3295748 · Cited by 879 · see citing works
“Another network, Gated Recurrent Unit (GRU) [25] has a similar structure to LSTM but it does not use separate memory cells and uses fewer gates to control the flow of information.”
BackgroundSpeaker Recognition from Raw Waveform with SincNet
Ravanelli, M., & Bengio, Y. (2018) · 2018 IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639585 · Cited by 896 · see citing works
Self-citationDeep Reinforcement Learning: An Overview
Li, Y. (2017) · arXiv · doi:10.48550/arxiv.1701.07274 · Cited by 762 · see citing works
DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding
Shen, T., Zhou, T., Long, G., Jiang, J., Pan, S., & Zhang, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11941 · Cited by 779 · see citing works
DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing
Yao, S., Hu, S., Zhao, Y., Zhang, A., & Abdelzaher, T. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052577 · Cited by 667 · see citing works
Qiu, S., Zhao, H., Jiang, N., Wang, Z., Liu, L., An, Y., Zhao, H., Miao, X., Liu, R., & Fortino, G. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.11.006 · Cited by 671 · see citing works
Deep learning in medical image registration: a review
Fu, Y., Lei, Y., Wang, T., Curran, W. J., Liu, T., & Yang, X. (2020) · Physics in Medicine & Biology · doi:10.1088/1361-6560/ab843e · Cited by 661 · see citing works
Liu, Y., & Wu, Y.-F. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11268 · Cited by 663 · see citing works
Lago, J., De Ridder, F., & De Schutter, B. (2018) · Applied Energy · doi:10.1016/j.apenergy.2018.02.069 · Cited by 642 · see citing works
DÏoT: A Federated Self-learning Anomaly Detection System for IoT
Nguyen, T. D., Marchal, S., Miettinen, M., Fereidooni, H., Asokan, N., & Sadeghi, A.-R. (2019) · 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2019.00080 · Cited by 663 · see citing works
Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances
Zhang, S., Li, Y., Zhang, S., Shahabi, F., Xia, S., Deng, Y., & Alshurafa, N. (2022) · Sensors · doi:10.3390/s22041476 · Cited by 625 · see citing works
“In an effort to address these challenges, long short-term memory (LSTM)-based RNNs [127], and Gated Recurrent Units (GRUs) [75] are introduced to model temporal sequences and their broad dependencies.”
BackgroundDeepCpG: accurate prediction of single-cell DNA methylation states using deep learning
Angermueller, C., Lee, H. J., Reik, W., & Stegle, O. (2017) · Genome Biology · doi:10.1186/s13059-017-1189-z · Cited by 536 · see citing works
State-of-charge estimation of lithium-ion batteries based on gated recurrent neural network
Yang, F., Li, W., Li, C., & Miao, Q. (2019) · Energy · doi:10.1016/j.energy.2019.03.059 · Cited by 543 · see citing works
Zero-Effort Cross-Domain Gesture Recognition with Wi-Fi
Zheng, Y., Zhang, Y., Qian, K., Zhang, G., Liu, Y., Wu, C., & Yang, Z. (2019) · Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3307334.3326081 · Cited by 557 · see citing works
Deep Multimodal Representation Learning: A Survey
Guo, W., Wang, J., & Wang, S. (2019) · IEEE Access · doi:10.1109/access.2019.2916887 · Cited by 535 · see citing works
Quantum advantage in learning from experiments
Huang, H.-Y., Broughton, M., Cotler, J., Chen, S., Li, J., Mohseni, M., Neven, H., Babbush, R., Kueng, R., Preskill, J., et al. (2022) · Science · doi:10.1126/science.abn7293 · Cited by 519 · see citing works
Multimodal intelligence: Representation learning, information fusion, and applications
Zhang, C., Yang, Z., He, X., & Deng, L. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.2987728 · Cited by 511 · see citing works
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352 · see citing works
GCAN: Graph-aware Co-Attention Networks for Explainable Fake News Detection on Social Media
Lu, Y.-J., & Li, C.-T. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.48 · Cited by 427 · see citing works
Kraus, M., Feuerriegel, S., & Oztekin, A. (2020) · European Journal of Operational Research · doi:10.1016/j.ejor.2019.09.018 · Cited by 392 · see citing works
STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing
Liu, M., Ding, Y., Xia, M., Liu, X., Ding, E., Zuo, W., & Wen, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00379 · Cited by 394 · see citing works
Application of Generative Autoencoder in <i>De Novo</i> Molecular Design
Blaschke, T., Olivecrona, M., Engkvist, O., Bajorath, J., & Chen, H. (2018) · Molecular Informatics · doi:10.1002/minf.201700123 · Cited by 411 · see citing works
How social learning amplifies moral outrage expression in online social networks
Brady, W. J., McLoughlin, K., Doan, T. N., & Crockett, M. J. (2021) · Science Advances · doi:10.1126/sciadv.abe5641 · Cited by 375 · see citing works
Machine learning for real-time prediction of complications in critical care: a retrospective study
Meyer, A., Zverinski, D., Pfahringer, B., Kempfert, J., Kuehne, T., Sündermann, S. H., Stamm, C., Hofmann, T., Falk, V., & Eickhoff, C. (2018) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(18)30300-x · Cited by 350 · see citing works
An Intrusion Detection System Using a Deep Neural Network With Gated Recurrent Units
Xu, C., Shen, J., Du, X., & Zhang, F. (2018) · IEEE Access · doi:10.1109/access.2018.2867564 · Cited by 339 · see citing works
A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4
Kalyan, K. S. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100048 · Cited by 335 · see citing works
A Survey on Multimodal Large Language Models for Autonomous Driving
Cui, C., Ma, Y., Cao, X., Ye, W., Zhou, Y., Liang, K., Chen, J., Lu, J., Yang, Z., Liao, K.-D., et al. (2024) · 2024 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw60836.2024.00106 · Cited by 369 · see citing works
Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
De-Arteaga, M., Romanov, A., Wallach, H., Chayes, J., Borgs, C., Chouldechova, A., Geyik, S., Kenthapadi, K., & Kalai, A. T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287572 · Cited by 337 · see citing works
Bidirectional RNN for Medical Event Detection in Electronic Health Records
Jagannatha, A. N., & Yu, H. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1056 · Cited by 314 · see citing works
ArunKumar, K. E., Kalaga, D. V., Mohan Sai Kumar, C., Kawaji, M., & Brenza, T. M. (2022) · Alexandria Engineering Journal · doi:10.1016/j.aej.2022.01.011 · Cited by 287 · see citing works
Deep learning for edge computing applications: A State-of-the-Art survey
Wang, F., Zhang, M., Wang, X., Ma, X., & Liu, J. (2020) · IEEE Access · doi:10.1109/access.2020.2982411 · Cited by 259 · see citing works
Azam, Z., Islam, M. M., & Huda, M. N. (2023) · IEEE Access · doi:10.1109/access.2023.3296444 · Cited by 265 · see citing works
AttnSense: Multi-level Attention Mechanism For Multimodal Human Activity Recognition
Ma, H., Li, W., Zhang, X., Gao, S., & Lu, S. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/431 · Cited by 266 · see citing works
Spatiotemporal Co-Attention Recurrent Neural Networks for Human-Skeleton Motion Prediction
Shu, X., Zhang, L., Qi, G.-J., Liu, W., & Tang, J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3050918 · Cited by 276 · see citing works
Iterative Visual Reasoning Beyond Convolutions
Chen, X., Li, L.-J., Fei-Fei, L., & Gupta, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00756 · Cited by 236 · see citing works
Camps, J., Samà, A., Martín, M., Rodríguez-Martín, D., Pérez-López, C., Moreno Arostegui, J. M., Cabestany, J., Català, A., Alcaine, S., Mestre, B., et al. (2018) · Knowledge-Based Systems · doi:10.1016/j.knosys.2017.10.017 · Cited by 224 · see citing works
The major human AP endonuclease (Ape1) is involved in the nucleotide incision repair pathway
Gros, L. (2004) · Nucleic Acids Research · doi:10.1093/nar/gkh165 · Cited by 209 · see citing works
Generative AI design for building structures
Liao, W., Lu, X., Fei, Y., Gu, Y., & Huang, Y. (2024) · Automation in Construction · doi:10.1016/j.autcon.2023.105187 · Cited by 213 · see citing works
Joint Audio-Visual Deepfake Detection
Zhou, Y., & Lim, S.-N. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01453 · Cited by 217 · see citing works
A Review on Explainability in Multimodal Deep Neural Nets
Joshi, G., Walambe, R., & Kotecha, K. (2021) · IEEE Access · doi:10.1109/access.2021.3070212 · Cited by 204 · see citing works
A Survey of Evaluation Metrics Used for NLG Systems
Sai, A. B., Mohankumar, A. K., & Khapra, M. M. (2023) · ACM Computing Surveys · doi:10.1145/3485766 · Cited by 235 · see citing works
Erase or Fill? Deep Joint Recurrent Rain Removal and Reconstruction in Videos
Liu, J., Yang, W., Yang, S., & Guo, Z. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00341 · Cited by 193 · see citing works
Stabilizing Transformers for Reinforcement Learning
Parisotto, E., Song, H. F., Rae, J. W., Pascanu, R., Gulcehre, C., Jayakumar, S. M., Jaderberg, M., Kaufman, R. L., Clark, A., Noury, S., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.06764 · Cited by 199 · see citing works
Weakly Supervised Action Learning with RNN Based Fine-to-Coarse Modeling
Richard, A., Kuehne, H., & Gall, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.140 · Cited by 195 · see citing works
Naseem, U., Razzak, I., Khan, S. K., & Prasad, M. (2021) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3434237 · Cited by 194 · see citing works
Poisoning Attacks on Federated Learning-based IoT Intrusion Detection System
Nguyen, T. D., Rieger, P., Miettinen, M., & Sadeghi, A.-R. (2020) · Proceedings 2020 Workshop on Decentralized IoT Systems and Security · doi:10.14722/diss.2020.23003 · Cited by 200 · see citing works
A Neural Temporal Model for Human Motion Prediction
Gopalakrishnan, A., Mali, A., Kifer, D., Giles, L., & Ororbia, A. G. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01239 · Cited by 185 · see citing works
“The cell update equations for the top-level GRU remain the same as described in [3] except that we note its non-state input is the action label (which remains fixed over the length of the sequence).”
BackgroundMethodA Systematic Review of Detecting Sleep Apnea Using Deep Learning
Mostafa, S. S., Mendonça, F., G. Ravelo-García, A., & Morgado-Dias, F. (2019) · Sensors · doi:10.3390/s19224934 · Cited by 182 · see citing works
Big code != big vocabulary: open-vocabulary models for source code
Karampatsis, R.-M., Babii, H., Robbes, R., Sutton, C., & Janes, A. (2020) · Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering · doi:10.1145/3377811.3380342 · Cited by 182 · see citing works
Neural Map: Structured Memory for Deep Reinforcement Learning
Parisotto, E., & Salakhutdinov, R. (2017) · arXiv · doi:10.48550/arxiv.1702.08360 · Cited by 178 · see citing works
MART: Memory-Augmented Recurrent Transformer for Coherent Video Paragraph Captioning
Lei, J., Wang, L., Shen, Y., Yu, D., Berg, T., & Bansal, M. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.233 · Cited by 179 · see citing works
Toward an AI Era: Advances in Electronic Skins
Fu, X., Cheng, W., Wan, G., Yang, Z., & Tee, B. C. K. (2024) · Chemical Reviews · doi:10.1021/acs.chemrev.4c00049 · Cited by 172 · see citing works
Jung, J.-W., Heo, H.-S., Kim, J.-H., Shim, H.-J., & Yu, H.-J. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-1982 · Cited by 166 · see citing works
A Recurrent BERT-based Model for Question Generation
Chan, Y.-H., & Fan, Y.-C. (2019) · Proceedings of the 2nd Workshop on Machine Reading for Question Answering · doi:10.18653/v1/d19-5821 · Cited by 165 · see citing works
DeepMood: Modeling Mobile Phone Typing Dynamics for Mood Detection
Cao, B., Zheng, L., Zhang, C., Yu, P. S., Piscitello, A., Zulueta, J., Ajilore, O., Ryan, K., & Leow, A. D. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098086 · Cited by 145 · see citing works
Contrastive Predictive Coding for Human Activity Recognition
Haresamudram, H., Essa, I., & Plötz, T. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3463506 · Cited by 150 · see citing works
“For the autoregressive network gar , we utilize a two-layer gated recurrent units (GRU) network [14] containing 256 units and dropout with p=0.”
MethodXu, H., Zhou, P., Tan, R., Li, M., & Shen, G. (2021) · Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems · doi:10.1145/3485730.3485937 · Cited by 150 · see citing works
Model agnostic generation of counterfactual explanations for molecules
Wellawatte, G. P., Seshadri, A., & White, A. D. (2022) · Chemical Science · doi:10.1039/d1sc05259d · Cited by 143 · see citing works
Mekruksavanich, S., & Jitpattanakul, A. (2021) · Electronics · doi:10.3390/electronics10141685 · Cited by 138 · see citing works
Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Series
Tipirneni, S., & Reddy, C. K. (2022) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3516367 · Cited by 130 · see citing works
A Comprehensive Study of Activity Recognition Using Accelerometers
Twomey, N., Diethe, T., Fafoutis, X., Elsts, A., McConville, R., Flach, P., & Craddock, I. (2018) · Informatics · doi:10.3390/informatics5020027 · Cited by 143 · see citing works
Learning for Personalized Medicine: A Comprehensive Review From a Deep Learning Perspective
Zhang, S., Bamakan, S. M. H., Qu, Q., & Li, S. (2019) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2018.2864254 · Cited by 125 · see citing works
<i>De Novo</i> Design of New Chemical Entities for SARS-CoV-2 Using Artificial Intelligence
Bung, N., Krishnan, S. R., Bulusu, G., & Roy, A. (2021) · Future Medicinal Chemistry · doi:10.4155/fmc-2020-0262 · Cited by 118 · see citing works
Navigating to objects in the real world
Gervet, T., Chintala, S., Batra, D., Malik, J., & Chaplot, D. S. (2023) · Science Robotics · doi:10.1126/scirobotics.adf6991 · Cited by 125 · see citing works
Toward an integration of deep learning and neuroscience
Marblestone, A. H., Wayne, G., & Kording, K. P. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00094 · Cited by 119 · see citing works
Gait Recognition Based on Deep Learning: A Survey
Filipi Gonçalves dos Santos, C., Oliveira, D. D. S., A. Passos, L., Gonçalves Pires, R., Felipe Silva Santos, D., Pascotti Valem, L., P. Moreira, T., Cleison S. Santana, M., Roder, M., Paulo Papa, J., et al. (2023) · ACM Computing Surveys · doi:10.1145/3490235 · Cited by 108 · see citing works
Assessing the State of Self-Supervised Human Activity Recognition Using Wearables
Haresamudram, H., Essa, I., & Plötz, T. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3550299 · Cited by 107 · see citing works
A Review on Deep-Learning-Based Cyberbullying Detection
Hasan, M. T., Hossain, M. A. E., Mukta, M. S. H., Akter, A., Ahmed, M., & Islam, S. (2023) · Future Internet · doi:10.3390/fi15050179 · Cited by 103 · see citing works
Hierarchical Attention Networks for Cyberbullying Detection on the Instagram Social Network
Cheng, L., Guo, R., Silva, Y., Hall, D., & Liu, H. (2019) · Proceedings of the 2019 SIAM International Conference on Data Mining · doi:10.1137/1.9781611975673.27 · Cited by 95 · see citing works
LEARN Codes: Inventing Low-Latency Codes via Recurrent Neural Networks
Jiang, Y., Kim, H., Asnani, H., Kannan, S., Oh, S., & Viswanath, P. (2020) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2020.2988577 · Cited by 87
Understanding Dataset Design Choices for Multi-hop Reasoning
Chen, J., & Durrett, G. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1405 · Cited by 82 · see citing works
Clune, J. (2019) · arXiv · doi:10.48550/arxiv.1905.10985 · Cited by 82 · see citing works
Deep neural networks predicting oil movement in a development unit
Temirchev, P., Simonov, M., Kostoev, R., Burnaev, E., Oseledets, I., Akhmetov, A., Margarit, A., Sitnikov, A., & Koroteev, D. (2020) · Journal of Petroleum Science and Engineering · doi:10.1016/j.petrol.2019.106513 · Cited by 75 · see citing works
Reinforcement learning-based adaptive PID controller for DPS
Lee, D., Lee, S. J., & Yim, S. C. (2020) · Ocean Engineering · doi:10.1016/j.oceaneng.2020.108053 · Cited by 80 · see citing works
Robust and accurate decoding of hand kinematics from entire spiking activity using deep learning
Ahmadi, N., Constandinou, T. G., & Bouganis, C.-S. (2021) · Journal of Neural Engineering · doi:10.1088/1741-2552/abde8a · Cited by 69 · see citing works
- Authors
- Chung, J., Gulcehre, C., Cho, K., & Bengio, Y.
- Affiliations
- Alcatel-Lucent (France) (FR)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2014
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1412.3555
- OpenAlex
- W1924770834
- Cited by
- 11,008
- References
- 15
- Citations by year
- 2026: 90 · 2025: 768 · 2024: 1415 · 2023: 1354 · 2022: 1193 · 2021: 1688 · 2020: 1596 · 2019: 1225 · 2018: 921 · 2017: 505 · 2016: 190 · 2015: 38 · 2014: 3 · 2013: 1 · 2012: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- gated recurrent neural networks, gated recurrent unit, long short-term memory, sequence modeling, speech signal modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
