Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Open accessThe DOI matches its Crossref record; 29 of the 31 references it cites were found in a registry or an index.
Abstract
Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, Yoshua Bengio. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2014.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“GRU [185] is a variant based on the LSTM, which only contains an update gate and reset gate and has a faster training speed than the LSTM with fewer parameters.”
From Deep Learning for Structural Health Monitoring: Data, Algorithms, Applications, Challenges, and Trends (2023) · cites this paper as [185]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“They are thus well-suited for detecting dependencies and patterns in language-like input.”
From The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model (2026) · cites this paper as [87]
The cited source says what the claim attributes to it.
In the citing paper
“GRU cells are a specific type of Recurrent Neural Networks (RNNs) which have been proven useful for dealing with sequence data such as language.”
From The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model (2026) · cites this paper as [87]
The cited source says what the claim attributes to it.
In the citing paper
“Early work [139-141] mainly relied on recurrent neural networks (e.g., LSTM [142], GRU [143]). Limited by the model's expressiveness, they often suffer from the issue of insufficient diversity.”
From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [143]
The cited source says what the claim attributes to it.
In the citing paper
“Moreover, [Tang et al., 2020] adopts GRU [Cho et al., 2014] on the sequence of hyperplanes to further capture the dynamics among hyperplanes.”
From Temporal Knowledge Graph Completion: A Survey (2023) · cites this paper as Cho et al., 2014
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“At each time step t , the hidden state h t (cid:105) of the RNN is updated by where f is a non-linear activation function. f may be as simple as an element-wise logistic sigmoid function and as complex as a long short-term memory (LSTM) unit (Hochreiter and Schmidhuber, 1997).”
BackgroundDistributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 1×
“We have done so by applying the data selection method proposed in (Moore and Lewis, 2010), and its extension to bi-texts (Axelrod et al., 2011).”
Method(2011) · Cited by 6,993
Self-citationADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322 · cited 2×
“95 (Zeiler, 2012).”
MethodStatistical phrase-based translation
Koehn, P., Och, F. J., & Marcu, D. (2003) · Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - NAACL '03 · doi:10.3115/1073445.1073462 · Cited by 3,197 · cited 2×
“In the phrase-based SMT framework introduced in (Koehn et al., 2003) and (Marcu and Wong, 2002), the translation model log p ( e | f ) is factorized into the translation probabilities of matching phrases in the source and target sentences.”
Method(2005) · Cited by 3,125
Supervised Sequence Labelling with Recurrent Neural Networks
Graves, A. (2012) · Studies in Computational Intelligence · doi:10.1007/978-3-642-24797-2 · Cited by 3,120
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011 · cited 1×
“Deep neural networks have shown great success in various applications such as objection recognition (see, e.g., (Krizhevsky et al., 2012)) and speech recognition (see, e.g., (Dahl et al., 2012)).”
BackgroundRecurrent Continuous Translation Models
Kalchbrenner, N., & Blunsom, P. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1176 · Cited by 1,232 · cited 1×
“The closest approach related to the proposed RNN Encoder–Decoder is the Recurrent Continuous Translation Model (Model 2) proposed in (Kalchbrenner and Blunsom, 2013).”
Method
Cited by
Items 101–200 of 1,296
Understanding Back-Translation at Scale
Edunov, S., Ott, M., Auli, M., & Grangier, D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1045 · Cited by 993
Cui, Z., Henrickson, K., Ke, R., & Wang, Y. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2950416 · Cited by 981
“The reason might be that GRU has the no cell state to store historical information in its gate units comparing to LSTM.”
BackgroundInfluentialNeural Responding Machine for Short-Text Conversation
Shang, L., Lu, Z., & Li, H. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1152 · Cited by 978
“(2014) has to use 4, 000 dimension for satisfying performance on machine translation, while (Cho et al., 2014) with a smaller dimension perform poorly on translating an entire sentence.”
BackgroundMethodInfluentialNeural Machine Translation of Rare Words with Subword Units
Sennrich, R., Haddow, B., & Birch, A. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1162 · Cited by 974
“The encoder is a bidirectional neural network with gated recurrent units (Cho et al., 2014) that reads an input sequence x = ( x 1 , ..., x m ) and calculates a forward sequence of hidden states ( −→ h 1 , ..., −→ h m ) , and a backward sequence ←− −→ ←− The hidden states h j and h j are…”
BackgroundIndependently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN
Li, S., Li, W., Cook, C., Zhu, C., & Gao, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00572 · Cited by 972
Recurrent Neural Network for Text Classification with Multi-Task Learning
Liu, P., Qiu, X., & Huang, X. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.05101 · Cited by 972
Long-term Recurrent Convolutional Networks for Visual Recognition and Description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Saenko, K., & Darrell, T. (2014) · doi:10.21236/ada623249 · Cited by 968
A review of wind speed and wind power forecasting with deep neural networks
Wang, Y., Zou, R., Liu, F., Zhang, L., & Liu, Q. (2021) · Applied Energy · doi:10.1016/j.apenergy.2021.117766 · Cited by 959
The Case for Learned Index Structures
Kraska, T., Beutel, A., Chi, E. H., Dean, J., & Polyzotis, N. (2018) · Proceedings of the 2018 International Conference on Management of Data · doi:10.1145/3183713.3196909 · Cited by 956
Siamese Recurrent Architectures for Learning Sentence Similarity
Mueller, J., & Thyagarajan, A. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10350 · Cited by 935
Deep Forest: Towards An Alternative to Deep Neural Networks
Zhou, Z.-H., & Feng, J. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/497 · Cited by 935
Deep Anomaly Detection with Outlier Exposure
Hendrycks, D., Mazeika, M., & Dietterich, T. (2018) · arXiv · doi:10.48550/arxiv.1812.04606 · Cited by 932
“For NLP experiments, we train 2-layer GRUs (Cho et al., 2014) for 5 epochs, then fine-tune network copies with OE for 2 epochs.”
MethodNweke, H. F., Teh, Y. W., Al-garadi, M. A., & Alo, U. R. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.03.056 · Cited by 930
Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining
Li, X., Wu, J., Lin, Z., Liu, H., & Zha, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_16 · Cited by 927
A Survey on Knowledge Graph-Based Recommender Systems
Guo, Q., Zhuang, F., Qin, C., Zhu, H., Xie, X., Xiong, H., & He, Q. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.3028705 · Cited by 926
Wang, T., & Isola, P. (2020) · arXiv · doi:10.48550/arxiv.2005.10242 · Cited by 921
“Our experiment settings below mostly the official implementation by Logeswaran & Lee (2018): • Sentence encoder architecture is bi-directional Gated Recurrent Unit (GRU) (Cho et al., 2014) with inputs from a 620-dimensional word embedding trained jointly from scratch.”
MethodA Unifying Review of Deep and Shallow Anomaly Detection
Ruff, L., Kauffmann, J. R., Vandermeulen, R. A., Montavon, G., Samek, W., Kloft, M., Dietterich, T. G., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3052449 · Cited by 914
Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling
Marcheggiani, D., & Titov, I. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1159 · Cited by 907
ASTER: An Attentional Scene Text Recognizer with Flexible Rectification
Shi, B., Yang, M., Wang, X., Lyu, P., Yao, C., & Bai, X. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2848939 · Cited by 906
Application of Long Short-Term Memory (LSTM) Neural Network for Flood Forecasting
Le, X.-H., Ho, H. V., Lee, G., & Jung, S. (2019) · Water · doi:10.3390/w11071387 · Cited by 889
Abstractive Sentence Summarization with Attentive Recurrent Neural Networks
Chopra, S., Auli, M., & Rush, A. M. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1012 · Cited by 881
Machine Health Monitoring Using Local Feature-Based Gated Recurrent Unit Networks
Zhao, R., Wang, D., Yan, R., Mao, K., Shen, F., & Wang, J. (2018) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2017.2733438 · Cited by 877
Show and Tell: Lessons Learned from the 2015 MSCOCO Image Captioning Challenge
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2587640 · Cited by 875
“An “encoder” RNN reads the source sentence and transforms it into a rich fixed-length vector representation, which in turn in used as the initial hidden state of a “decoder” RNN that generates the target sentence.”
BackgroundMethodInfluentialDeep Learning for Cardiac Image Segmentation: A Review
Chen, C., Qin, C., Qiu, H., Tarroni, G., Duan, J., Bai, W., & Rueckert, D. (2020) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2020.00025 · Cited by 869
“Machine learning terminology: AE, autoencoder; ASM, active shape model; BN, batch normalization; CONV, convolution; CNN, convolutional neural network; CRF, conditional random field; DBN, deep belief network; DL, deep learning; DNN, deep neural network; EM, expectation maximization; FCN, fully convolutional neural netw…”
MethodSemEval-2019 Task 5: Multilingual Detection of Hate Speech Against Immigrants and Women in Twitter
Basile, V., Bosco, C., Fersini, E., Nozza, D., Patti, V., Rangel Pardo, F. M., Rosso, P., & Sanguinetti, M. (2019) · Proceedings of the 13th International Workshop on Semantic Evaluation · doi:10.18653/v1/s19-2007 · Cited by 864
Contrastive Representation Learning: A Framework and Review
Le-Khac, P. H., Healy, G., & Smeaton, A. F. (2020) · IEEE Access · doi:10.1109/access.2020.3031549 · Cited by 862
“In Contrastive Predictive Coding (CPC) [77] where similarity is defined from the past-present relationship, a GRU [19] head is applied over previous time steps to aggregate the past information into a contextualised embed-”
MethodOn Using Very Large Target Vocabulary for Neural Machine Translation
Jean, S., Cho, K., Memisevic, R., & Bengio, Y. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1001 · Cited by 862
Self-citationZhang, C., Song, D., Chen, Y., Feng, X., Lumezanu, C., Cheng, W., Ni, J., Zong, B., Chen, H., & Chawla, N. V. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33011409 · Cited by 855
Zhang, A., Lipton, Z. C., Mu, L., & Smola, A. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.11342 · Cited by 76
Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models
Polykovskiy, D., Zhebrak, A., Sanchez-Lengeling, B., Golovanov, S., Tatanov, O., Belyaev, S., Kurbanov, R., Artamonov, A., Aladinskiy, V., Veselov, M., et al. (2020) · Frontiers in Pharmacology · doi:10.3389/fphar.2020.565644 · Cited by 849
“Utilizing a bidirectional [85] Gated Recurrent Unit (GRU) [86] with a linear output layer as an encoder.”
Structured Sequence Modeling with Graph Convolutional Recurrent Networks
Seo, Y., Defferrard, M., Vandergheynst, P., & Bresson, X. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-04167-0_33 · Cited by 843
Sequence-Level Knowledge Distillation
Kim, Y., & Rush, A. M. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1139 · Cited by 836
Exploring the Limits of Language Modeling
Józefowicz, R., Vinyals, O., Schuster, M., Shazeer, N., & Wu, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.02410 · Cited by 823
“…recently proposed sequence-to-sequence models employ conditional language models (Mikolov & Zweig, 2012) as their key component to solve diverse tasks like machine translation (Sutskever et al., 2014; Cho et al., 2014; Kalch-brenner et al., 2014) or video generation (Srivastava et al., 2015a).”
BackgroundLSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection
Malhotra, P., Ramakrishnan, A., Anand, G., Vig, L., Agarwal, P., & Shroff, G. (2016) · arXiv · doi:10.48550/arxiv.1607.00148 · Cited by 819
“LSTM Encoder-Decoder models have been recently proposed for sequence-to-sequence learning tasks like machine translation (Cho et al., 2014; Sutskever et al., 2014).”
BackgroundLearning robust perceptive locomotion for quadrupedal robots in the wild
Miki, T., Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., & Hutter, M. (2022) · Science Robotics · doi:10.1126/scirobotics.abk2822 · Cited by 813
Deep Audio-Visual Speech Recognition
Afouras, T., Chung, J. S., Senior, A., Vinyals, O., & Zisserman, A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2889052 · Cited by 801
An Introduction to Machine Learning
Badillo, S., Banfai, B., Birzele, F., Davydov, I. I., Hutchinson, L., Kam‐Thong, T., Siebourg‐Polster, J., Steiert, B., & Zhang, J. D. (2020) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1796 · Cited by 799
“Further notable neural networks that are out of scope for this article but we recommend further reading on are convolutional neural networks, 53 encoder-decoder networks, 54,55 and generative models.”
BackgroundDeep Learning in Medical Ultrasound Analysis: A Review
Liu, S., Wang, Y., Yang, X., Lei, B., Liu, L., Li, S. X., Ni, D., & Wang, T. (2019) · Engineering · doi:10.1016/j.eng.2018.11.020 · Cited by 798
Feature dimensionality reduction: a review
Jia, W., Sun, M., Lian, J., & Hou, S. (2022) · Complex & Intelligent Systems · doi:10.1007/s40747-021-00637-x · Cited by 797
“designed a new RNN Encoder-Decoder, containing two recurrent neural networks [147].”
MethodXiao, C., Choi, E., & Sun, J. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy068 · Cited by 792
“Two prominent RNN variants with gating mechanisms are widely used: the LSTM unit,(99) and the GRU.(100) They are designed to overcome the vanishing gradient problem as well as capture the effect of long-term dependencies.”
MethodA Survey of Optimization Methods From a Machine Learning Perspective
Sun, S., Cao, Z., Zhu, H., & Zhao, J. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2950779 · Cited by 789
“RNNs are a kind of sequential model and very active in NLP [12], [13], [14], [15].”
BackgroundTraining Spiking Neural Networks Using Lessons From Deep Learning
Eshraghian, J. K., Ward, M., Neftci, E. O., Wang, X., Lenz, G., Dwivedi, G., Bennamoun, M., Jeong, D. S., & Lu, W. D. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3308088 · Cited by 782
“LSTMs [181] and, later, GRUs [182] introduced slow dynamics designed to overcome memory and vanishing gradient problems in RNNs.”
BackgroundA Network-based End-to-End Trainable Task-oriented Dialogue System
Wen, T.-H., Vandyke, D., Mrkšić, N., Gasic, M., Rojas Barahona, L. M., Su, P.-H., Ultes, S., & Young, S. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers · doi:10.18653/v1/e17-1042 · Cited by 778
From Recognition to Cognition: Visual Commonsense Reasoning
Zellers, R., Bisk, Y., Farhadi, A., & Choi, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00688 · Cited by 769
“Because VCR is highly diverse (Supp A), for these models we represent each response r(i) using a GRU [11].11 The output logit for response i is given by the dot product between the final hidden state of the GRU encoding r(i), and the final representation from the model.”
MethodInfluentialA Rainfall‐Runoff Model With LSTM‐Based Sequence‐to‐Sequence Learning
Xiang, Z., Yan, J., & Demir, I. (2020) · Water Resources Research · doi:10.1029/2019wr025326 · Cited by 768
“Thus, Cho et al. (2014) proposed a neural network structure called Encoder ‐ Decoder, or seq2seq, which allows the model to be established on different input and output time steps.”
BackgroundMethodMotion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning
Everett, M., Chen, Y. F., & How, J. P. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593871 · Cited by 767
A Neural Network Approach to Context-Sensitive Generation of Conversational Responses
Sordoni, A., Galley, M., Auli, M., Brockett, C., Ji, Y., Mitchell, M., Nie, J.-Y., Gao, J., & Dolan, B. (2015) · Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.3115/v1/n15-1020 · Cited by 764
Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory
Zhou, H., Huang, M., Zhang, T., Zhu, X., & Liu, B. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11325 · Cited by 763
Miikkulainen, R., Liang, J., Meyerson, E., Rawal, A., Fink, D., Francon, O., Raju, B., Shahrzad, H., Navruzyan, A., Duffy, N., et al. (2019) · Artificial Intelligence in the Age of Neural Networks and Brain Computing · doi:10.1016/b978-0-12-815480-9.00015-3 · Cited by 760
Neural Speech Synthesis with Transformer Network
Li, N., Liu, S., Liu, Y., Zhao, S., & Liu, M. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016706 · Cited by 754
“For both encoder and decoder, recurrent neural networks (RNNs) are usually leveraged, such as LSTM (Hochreiter and Schmidhuber 1997) and GRU (Cho et al. 2014).”
MethodEnd-to-End Multimodal Emotion Recognition using Deep Neural Networks
Tzirakis, P., Trigeorgis, G., Nicolaou, M. A., Schuller, B., & Zafeiriou, S. (2017) · arXiv · doi:10.48550/arxiv.1704.08619 · Cited by 752
“Due to memory and training instability concerns [45] it is not always optimal to use very large sequences in recurrent networks.”
BackgroundThe Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems
Lowe, R., Pow, N., Serban, I., & Pineau, J. (2015) · Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w15-4640 · Cited by 748
Rahman, A., Srikumar, V., & Smith, A. D. (2018) · Applied Energy · doi:10.1016/j.apenergy.2017.12.051 · Cited by 743
Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies
Linzen, T., Dupoux, E., & Goldberg, Y. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00115 · Cited by 737
Artificial intelligence and machine learning in design of mechanical materials
Guo, K., Yang, Z., Yu, C.-H., & Buehler, M. J. (2021) · Materials Horizons · doi:10.1039/d0mh01451f · Cited by 735
MASS: Masked Sequence to Sequence Pre-training for Language Generation
Song, K., Tan, X., Qin, T., Lu, J., & Liu, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.02450 · Cited by 733
Recurrent Neural Networks: A Comprehensive Review of Architectures, Variants, and Applications
Mienye, I. D., Swart, T. G., & Obaido, G. (2024) · Information · doi:10.3390/info15090517 · Cited by 732
A review of the application of deep learning in intelligent fault diagnosis of rotating machinery
Zhu, Z., Lei, Y., Qi, G., Chai, Y., Mazur, N., An, Y., & Huang, X. (2023) · Measurement · doi:10.1016/j.measurement.2022.112346 · Cited by 731
Reinforcement learning for combinatorial optimization: A survey
Mazyavkina, N., Sviridov, S., Ivanov, S., & Burnaev, E. (2021) · Computers & Operations Research · doi:10.1016/j.cor.2021.105400 · Cited by 723
Federated-Learning-Based Anomaly Detection for IoT Security Attacks
Mothukuri, V., Khare, P., Parizi, R. M., Pouriyeh, S., Dehghantanha, A., & Srivastava, G. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3077803 · Cited by 722
A survey on long short-term memory networks for time series prediction
Lindemann, B., Müller, T., Vietz, H., Jazdi, N., & Weyrich, M. (2021) · Procedia CIRP · doi:10.1016/j.procir.2021.03.088 · Cited by 720
“GRU was developed by [8] to model time series with the aim of creating a mechanism that complements the ability to predict long-term dependencies with an improved integration of short-term information.”
BackgroundMethodApplications of deep learning in stock market prediction: Recent progress
Jiang, W. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115537 · Cited by 714
“(2019b); Zhong & Enke (2017); Wang & Wang (2015); Singh & Srivastava (2017); Chong et al. (2017). Zhong & Enke (2017) even gives a comparison between different versions of PCA, and finds that the PCA-ANN model give a slightly higher prediction accuracy for the daily direction of SPY for next day, compared to the use o…”
BackgroundMethodDeep Unordered Composition Rivals Syntactic Methods for Text Classification
Iyyer, M., Manjunatha, V., Boyd-Graber, J., & Daumé III, H. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1162 · Cited by 714
Privacy-Preserving Traffic Flow Prediction: A Federated Learning Approach
Liu, Y., Yu, J. J. Q., Kang, J., Niyato, D., & Zhang, S. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2991401 · Cited by 712
“[45] proposed the GRU neural network in 2014, which is a variant of RNN that handles time-series data.”
BackgroundAttention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling
Liu, B., & Lane, I. (2016) · Interspeech 2016 · doi:10.21437/interspeech.2016-1352 · Cited by 709
Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting
Bai, L., Yao, L., Li, C., Wang, X., & Wang, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.02842 · Cited by 701
Language to Logical Form with Neural Attention
Dong, L., & Lapata, M. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1004 · Cited by 700
A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues
Serban, I., Sordoni, A., Lowe, R., Charlin, L., Pineau, J., Courville, A., & Bengio, Y. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10983 · Cited by 697
Self-citationLearning to Ask: Neural Question Generation for Reading Comprehension
Du, X., Shao, J., & Cardie, C. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1123 · Cited by 696
“We model the conditional probability us-ing RNN encoder-decoder architecture (Bahdanau et al., 2015; Cho et al., 2014), and adopt the global attention mechanism (Luong et al., 2015a) to make the model focus on certain elements of the input when generating each word during decoding.”
MethodModeling Coverage for Neural Machine Translation
Tu, Z., Lu, Z., Liu, Y., Liu, X., & Li, H. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1008 · Cited by 692
Gao, S., Huang, Y., Zhang, S., Han, J., Wang, G., Zhang, M., & Lin, Q. (2020) · Journal of Hydrology · doi:10.1016/j.jhydrol.2020.125188 · Cited by 691
An Attentive Survey of Attention Models
Chaudhari, S., Mithal, V., Polatkan, G., & Ramanath, R. (2021) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3465055 · Cited by 689
Adversarial Attacks and Defenses in Deep Learning
Ren, K., Zheng, T., Qin, Z., & Liu, X. (2020) · Engineering · doi:10.1016/j.eng.2019.12.012 · Cited by 678
“However, efforts in this direction—especially those yielding obfuscated gradients—actually give a false sense of security [2].”
BackgroundDeep Learning-Based Vehicle Behavior Prediction for Autonomous Driving Applications: A Review
Mozaffari, S., Al-Jarrah, O. Y., Dianati, M., Jennings, P., & Mouzakitis, A. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3012034 · Cited by 671
Evaluating the Factual Consistency of Abstractive Text Summarization
Kryscinski, W., McCann, B., Xiong, C., & Socher, R. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.750 · Cited by 669
Gated Self-Matching Networks for Reading Comprehension and Question Answering
Wang, W., Yang, N., Wei, F., Chang, B., & Zhou, M. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1018 · Cited by 666
Recurrent MVSNet for High-Resolution Multi-View Stereo Depth Inference
Yao, Y., Luo, Z., Li, S., Shen, T., Fang, T., & Quan, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00567 · Cited by 666
“2D spatial cost aggregation with depth-wise winner-take-all [29, 31]. In this work, we follow the idea of sequential processing, and propose to regularize the cost volume using the convolutional GRU [6]. GRU is a RNN architecture [8] initially proposed for learning sequential speech and text data, and is recently appl…”
MethodA Survey on Deep Learning for Data-Driven Soft Sensors
Sun, Q., & Ge, Z. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3053128 · Cited by 665
Contrastive Learning for Sequential Recommendation
Xie, X., Sun, F., Liu, Z., Wu, S., Gao, J., Zhang, J., Ding, B., & Cui, B. (2022) · 2022 IEEE 38th International Conference on Data Engineering (ICDE) · doi:10.1109/icde53745.2022.00099 · Cited by 663
“With the recent advance of deep learning, Recurrent Neural Networks (RNN) [6], and its variants, such as Long Short-Term Memory (LSTM) [20], and Gated Recurrent Unit (GRU) [11, 19], are applied for user behavior sequences.”
MethodHarnessing multimodal data integration to advance precision oncology
Boehm, K. M., Khosravi, P., Vanguri, R., Gao, J., & Shah, S. P. (2022) · Nature Reviews Cancer · doi:10.1038/s41568-021-00408-3 · Cited by 662
“The most successful variants are long short-term memory (lSTm) and gated recurrent unit (Gru) networks 165,166 . more recently, transformer networks 167 have demonstrably outperformed rNNs in sequential learning. although rNNs and transformers have not yet been widely applied in oncology,…”
BackgroundDeep learning in medical image registration: a review
Fu, Y., Lei, Y., Wang, T., Curran, W. J., Liu, T., & Yang, X. (2020) · Physics in Medicine & Biology · doi:10.1088/1361-6560/ab843e · Cited by 661
Self-referencing embedded strings (SELFIES): A 100% robust molecular string representation
Krenn, M., Häse, F., Nigam, A., Friederich, P., & Aspuru-Guzik, A. (2020) · Machine Learning: Science and Technology · doi:10.1088/2632-2153/aba947 · Cited by 660
STGAT: Modeling Spatial-Temporal Interactions for Human Trajectory Prediction
Huang, Y., Bi, H., Li, Z., Mao, T., & Wang, Z. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00637 · Cited by 658
A survey on fake news and rumour detection techniques
Bondielli, A., & Marcelloni, F. (2019) · Information Sciences · doi:10.1016/j.ins.2019.05.035 · Cited by 658
“Authors have proposed different RNN architectures, namely tanh-RNN, LSTM and Gated Recurrent Unit (GRU) [21].”
MethodInfluentialWhat you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties
Conneau, A., Kruszewski, G., Lample, G., Barrault, L., & Baroni, M. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1198 · Cited by 657
“…single $&!#* vector, sentence embedding meth-ods have achieved impressive results in tasks ranging from machine translation (Sutskever et al., 2014; Cho et al., 2014) to entailment detection (Williams et al., 2018), spurring the quest for “universal embeddings” trained once and used in a variety…”
BackgroundExploring Models and Data for Remote Sensing Image Caption Generation
Lu, X., Wang, B., Zheng, X., & Li, X. (2018) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2017.2776321 · Cited by 647
“Inspired by recent advances in computer vision and machine translation [38], a deep model [39] is proposed to maximize the likelihood of the target description sentence given the training image.”
MethodSemantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition
Zhang, P., Lan, C., Zeng, W., Xing, J., Xue, J., & Zheng, N. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00119 · Cited by 647
Joint Event Extraction via Recurrent Neural Networks
Nguyen, T. H., Cho, K., & Grishman, R. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1034 · Cited by 647
“…in this work, we propose to solve the EE problem with the joint approach via recurrent neural networks (RNNs) (Hochreiter and Schmidhuber, 1997; Cho et al., 2014) augmented with the discrete features, thus inheriting all the benefits from both systems as well as overcoming their inherent…”
MethodSelf-citationMirrorGAN: Learning Text-To-Image Generation by Redescription
Qiao, T., Zhang, J., Xu, D., & Tao, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00160 · Cited by 644
“The image encoder is a convolutional neural network (CNN) [11] pretrained on ImageNet [5], and the decoder is a RNN [12].”
MethodInfluentialLi, D., Opazo, C. R., Yu, X., & Li, H. (2020) · 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv45572.2020.9093512 · Cited by 641
Searching for a Robust Neural Architecture in Four GPU Hours
Dong, X., & Yang, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00186 · Cited by 640
“Since the birth of AlexNet [20] in 2012, human experts have conducted a huge number of experiments, and consequently devised several useful structures, such as attention [7] and residual connection [12].”
BackgroundGRAM: Graph-based Attention Model for Healthcare Representation Learning
Choi, E., Bahadori, M. T., Song, L., Stewart, W. F., & Sun, J. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098126 · Cited by 637
“• RNN : Input x t is transformed by an embedding matrix W emb ∈ R k ×|C| , then fed to the GRU with a single hidden layer.”
MethodInfluentialA Recurrent Latent Variable Model for Sequential Data
Chung, J., Kastner, K., Dinh, L., Goel, K., Courville, A. C., & Bengio, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.02216 · Cited by 635
“The transition function f can be implemented with gated activation functions such as long short-term memory [LSTM, 9] or gated recurrent unit [GRU, 5].”
BackgroundSelf-citationDeep learning predicts path-dependent plasticity
Mozaffar, M., Bostanabad, R., Chen, W., Ehmann, K., Cao, J., & Bessa, M. A. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1911815116 · Cited by 634
Ask Your Neurons: A Neural-Based Approach to Answering Questions about Images
Malinowski, M., Rohrbach, M., & Fritz, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.9 · Cited by 631
AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting
Yuan, Y., Weng, X., Ou, Y., & Kitani, K. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00967 · Cited by 631
FS-Net: A Flow Sequence Network For Encrypted Traffic Classification
Liu, C., He, L., Xiong, G., Cao, Z., & Li, Z. (2019) · IEEE INFOCOM 2019 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2019.8737507 · Cited by 630
“The Gated Recurrent Unit (GRU) cell [30] adds the gating mechanism to control the information transformation between the hidden states, and tracks the states of the input sequences without using separate memory cells.”
BackgroundMethodInfluentialDeep Learning Predicts Lung Cancer Treatment Response from Serial Medical Imaging
Xu, Y., Hosny, A., Zeleznik, R., Parmar, C., Coroller, T., Franco, I., Mak, R. H., & Aerts, H. J. W. L. (2019) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-18-2495 · Cited by 628
Phan, H., Andreotti, F., Cooray, N., Chen, O. Y., & De Vos, M. (2019) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2019.2896659 · Cited by 623
Metadata
- Authors
- Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y.
- Affiliations
- Le Mans Université (FR)
- Polytechnique Montréal (CA)
- Venue
- Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Published
- 2014
- Pages
- 1724-1734
- OpenAlex
- W2157331557
- Cited by
- 25,545
- References
- 31
- Citations by year
- 2026: 1895 · 2025: 2591 · 2024: 2721 · 2023: 3047 · 2022: 3245 · 2021: 3534 · 2020: 3063 · 2019: 2494 · 2018: 1644 · 2017: 816 · 2016: 349 · 2015: 119 · 2014: 22
- FWCI
- 256.40
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- HAL (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- statistical machine translation, neural machine translation, phrase embeddings, sequence-to-sequence models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.