Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Open accessThe DOI matches its Crossref record; 29 of the 31 references it cites were found in a registry or an index.
Abstract
Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, Yoshua Bengio. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2014.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“GRU [185] is a variant based on the LSTM, which only contains an update gate and reset gate and has a faster training speed than the LSTM with fewer parameters.”
From Deep Learning for Structural Health Monitoring: Data, Algorithms, Applications, Challenges, and Trends (2023) · cites this paper as [185]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“They are thus well-suited for detecting dependencies and patterns in language-like input.”
From The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model (2026) · cites this paper as [87]
The cited source says what the claim attributes to it.
In the citing paper
“GRU cells are a specific type of Recurrent Neural Networks (RNNs) which have been proven useful for dealing with sequence data such as language.”
From The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model (2026) · cites this paper as [87]
The cited source says what the claim attributes to it.
In the citing paper
“Early work [139-141] mainly relied on recurrent neural networks (e.g., LSTM [142], GRU [143]). Limited by the model's expressiveness, they often suffer from the issue of insufficient diversity.”
From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [143]
The cited source says what the claim attributes to it.
In the citing paper
“Moreover, [Tang et al., 2020] adopts GRU [Cho et al., 2014] on the sequence of hyperplanes to further capture the dynamics among hyperplanes.”
From Temporal Knowledge Graph Completion: A Survey (2023) · cites this paper as Cho et al., 2014
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“At each time step t , the hidden state h t (cid:105) of the RNN is updated by where f is a non-linear activation function. f may be as simple as an element-wise logistic sigmoid function and as complex as a long short-term memory (LSTM) unit (Hochreiter and Schmidhuber, 1997).”
BackgroundDistributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 1×
“We have done so by applying the data selection method proposed in (Moore and Lewis, 2010), and its extension to bi-texts (Axelrod et al., 2011).”
Method(2011) · Cited by 6,993
Self-citationADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322 · cited 2×
“95 (Zeiler, 2012).”
MethodStatistical phrase-based translation
Koehn, P., Och, F. J., & Marcu, D. (2003) · Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - NAACL '03 · doi:10.3115/1073445.1073462 · Cited by 3,197 · cited 2×
“In the phrase-based SMT framework introduced in (Koehn et al., 2003) and (Marcu and Wong, 2002), the translation model log p ( e | f ) is factorized into the translation probabilities of matching phrases in the source and target sentences.”
Method(2005) · Cited by 3,125
Supervised Sequence Labelling with Recurrent Neural Networks
Graves, A. (2012) · Studies in Computational Intelligence · doi:10.1007/978-3-642-24797-2 · Cited by 3,120
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011 · cited 1×
“Deep neural networks have shown great success in various applications such as objection recognition (see, e.g., (Krizhevsky et al., 2012)) and speech recognition (see, e.g., (Dahl et al., 2012)).”
BackgroundRecurrent Continuous Translation Models
Kalchbrenner, N., & Blunsom, P. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1176 · Cited by 1,232 · cited 1×
“The closest approach related to the proposed RNN Encoder–Decoder is the Recurrent Continuous Translation Model (Model 2) proposed in (Kalchbrenner and Blunsom, 2013).”
Method
Cited by
Items 1,001–1,100 of 1,297
Attention-over-Attention Neural Networks for Reading Comprehension
Cui, Y., Chen, Z., Wei, S., Wang, S., Liu, T., & Hu, G. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1055 · Cited by 171
“• Hidden Layer: Internal weights of GRUs are initialized with random orthogonal matrices (Saxe et al., 2013).”
MethodInfluentialA Taxonomy of Deep Convolutional Neural Nets for Computer Vision
Srinivas, S., Sarvadevabhatla, R. K., Mopuri, K. R., Prabhu, N., Kruthiventi, S. S. S., & Babu, R. V. (2016) · Frontiers in Robotics and AI · doi:10.3389/frobt.2015.00036 · Cited by 171
Knowledge Graph Embedding: A Survey from the Perspective of Representation Spaces
Cao, J., Fang, J., Meng, Z., & Liang, S. (2024) · ACM Computing Surveys · doi:10.1145/3643806 · Cited by 171
A Survey on Machine Learning Techniques for Routing Optimization in SDN
Amin, R., Rojas, E., Aqdus, A., Ramzan, S., Casillas-Perez, D., & Arco, J. M. (2021) · IEEE Access · doi:10.1109/access.2021.3099092 · Cited by 171
Recent progresses in deep learning based acoustic models
Yu, D., & Li, J. (2017) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2017.7510508 · Cited by 171
Chattopadhyay, A., Hassanzadeh, P., & Subramanian, D. (2020) · Nonlinear Processes in Geophysics · doi:10.5194/npg-27-373-2020 · Cited by 169
Sparse GPU Kernels for Deep Learning
Gale, T., Zaharia, M., Young, C., & Elsen, E. (2020) · SC20: International Conference for High Performance Computing, Networking, Storage and Analysis · doi:10.1109/sc41405.2020.00021 · Cited by 169
Jha, A., & Mamidi, R. (2017) · Proceedings of the Second Workshop on NLP and Computational Social Science · doi:10.18653/v1/w17-2902 · Cited by 169
“A basic sequence-to-sequence model consists of an encoder and a decoder (Sutskever et al., 2014; Cho et al., 2014).”
BackgroundLi, G., Iyer, B., Prasath, V. B. S., Ni, Y., & Salomonis, N. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab160 · Cited by 168
“For deep learning based methods, we benchmark against a Gated Recurrent Unit (GRU) [30] based deep learning model DeepHLApan [10].”
E-RAFT: Dense Optical Flow from Event Cameras
Gehrig, M., Millhausler, M., Gehrig, D., & Scaramuzza, D. (2021) · 2021 International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00030 · Cited by 168
Visualizing and Understanding Neural Machine Translation
Ding, Y., Liu, Y., Luan, H., & Sun, M. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1106 · Cited by 168
“…the encoder first uses source word embeddings to map each source word xi to a real-valued vector xi.1 Then, a forward recurrent neural network (RNN) with GRU units (Cho et al., 2014) runs to calculate source forward hidden states: −→ h i = f( −→ h i−1,xi), (2) where f(·) is a non-linear function.”
MethodTeMP: Temporal Message Passing for Temporal Knowledge Graph Completion
Wu, J., Cao, M., Cheung, J. C. K., & Hamilton, W. L. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.462 · Cited by 167
A Survey on an Emerging Area: Deep Learning for Smart City Data
Chen, Q., Wang, W., Wu, F., De, S., Wang, R., Zhang, B., & Huang, X. (2019) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2019.2907718 · Cited by 167
“It has become increasingly popular in natural language processing [23], speech recognition [30], traffic prediction [75]”
BackgroundMethodPredicting sepsis with a recurrent neural network using the MIMIC III database
Scherpf, M., Gräßer, F., Malberg, H., & Zaunseder, S. (2019) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2019.103395 · Cited by 167
Jung, J.-W., Heo, H.-S., Kim, J.-H., Shim, H.-J., & Yu, H.-J. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-1982 · Cited by 166
A Selective Overview of Deep Learning
Fan, J., Ma, C., & Zhong, Y. (2021) · Statistical Science · doi:10.1214/20-sts783 · Cited by 166
Translation between Molecules and Natural Language
Edwards, C., Lai, T., Ros, K., Honke, G., Cho, K., & Ji, H. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.26 · Cited by 165
Self-citationStock Prediction Based on Optimized LSTM and GRU Models
Gao, Y., Wang, R., & Zhou, E. (2021) · Scientific Programming · doi:10.1155/2021/4055281 · Cited by 165
“[12] reduced the LSTM structure and created GRU, a new deep learning architecture that integrates longterm and short-term memory.”
BackgroundSSIM—A Deep Learning Approach for Recovering Missing Time Series Sensor Data
Zhang, Y.-F., Thorburn, P. J., Xiang, W., & Fitch, P. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2909038 · Cited by 164
“as machine translation [15], image caption generation [16], and speech recognition [17].”
BackgroundPredicting prime editing efficiency and product purity by deep learning
Mathis, N., Allam, A., Kissling, L., Marquart, K. F., Schmidheini, L., Solari, C., Balázs, Z., Krauthammer, M., & Schwank, G. (2023) · Nature Biotechnology · doi:10.1038/s41587-022-01613-7 · Cited by 162
Mobile Video Object Detection with Temporally-Aware Feature Maps
Zhu, M., & Liu, M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00596 · Cited by 160
M3: Multimodal Memory Modelling for Video Captioning
Wang, J., Wang, W., Huang, Y., Wang, L., & Tan, T. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00784 · Cited by 159
Multi-Sensor Prognostics using an Unsupervised Health Index based on LSTM Encoder-Decoder
Malhotra, P., Tv, V., Ramakrishnan, A., Anand, G., Vig, L., Agarwal, P., & Shroff, G. M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.06154 · Cited by 159
Karijadi, I., & Chou, S.-Y. (2022) · Energy and Buildings · doi:10.1016/j.enbuild.2022.111908 · Cited by 159
Antibody complementarity determining region design using high-capacity machine learning
Liu, G., Zeng, H., Mueller, J., Carter, B., Wang, Z., Schilz, J., Horny, G., Birnbaum, M. E., Ewert, S., & Gifford, D. K. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz895 · Cited by 157
Vision-Language Pre-Training: Basics, Recent Advances, and Future Trends
Gan, Z., Li, L., Li, C., Wang, L., Liu, Z., & Gao, J. (2022) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000105 · Cited by 155
Towards Interpretable Mental Health Analysis with Large Language Models
Yang, K., Ji, S., Zhang, T., Xie, Q., Kuang, Z., & Ananiadou, S. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.370 · Cited by 155
“Compared with supervised methods, ChatGPT ZS significantly outperforms traditional light-weighted neural network-based methods such as CNN and GRU on binary detection and cause/factor detection, showing its potential in cause analysis for mental health-related texts.”
MethodInfluentialDeepMood: Modeling Mobile Phone Typing Dynamics for Mood Detection
Cao, B., Zheng, L., Zhang, C., Yu, P. S., Piscitello, A., Zulueta, J., Ajilore, O., Ryan, K., & Leow, A. D. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098086 · Cited by 145
“A typical GRU is formulated as: rk = sigmoid(Wr xk + Ur hk−1) zk = sigmoid(Wzxk + Uzhk−1) h̃k = tanh(Wxk + U(rk ⊙ hk−1)) hk = zk ⊙ hk−1 + (1 − zk ) ⊙ h̃k (3) where ⊙ is the element-wise multiplication operator, a reset gate rk allows the GRU to forget the previously computed state hk−1, and an update gate zk balances…”
BackgroundMethodInfluentialMetzger, S. L., Liu, J. R., Moses, D. A., Dougherty, M. E., Seaton, M. P., Littlejohn, K. T., Chartier, J., Anumanchipalli, G. K., Tu-Chan, A., Ganguly, K., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-33611-3 · Cited by 153
Modeling Relation Paths for Knowledge Graph Completion
Shen, Y., Ding, N., Zheng, H.-T., Li, Y., & Yang, M. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.2970044 · Cited by 153
Chu, M., Li, H., Liao, X., & Cui, S. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2872440 · Cited by 153
Tracing the evolution of AI in the past decade and forecasting the emerging trends
Shao, Z., Zhao, R., Yuan, S., Ding, M., & Wang, Y. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118221 · Cited by 153
Shickel, B., Loftus, T. J., Adhikari, L., Ozrazgat-Baslanti, T., Bihorac, A., & Rashidi, P. (2019) · Scientific Reports · doi:10.1038/s41598-019-38491-0 · Cited by 165
The digital scribe in clinical practice: a scoping review and research agenda
van Buchem, M. M., Boosman, H., Bauer, M. P., Kant, I. M. J., Cammel, S. A., & Steyerberg, E. W. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00432-5 · Cited by 152
Machine-learning-based anomaly detection in optical fiber monitoring
Abdelli, K., Cho, J. Y., Azendorf, F., Griesser, H., Tropschug, C., & Pachnicke, S. (2022) · Journal of Optical Communications and Networking · doi:10.1364/jocn.451289 · Cited by 149
Jia, J., & Li, Y. (2023) · Sensors · doi:10.3390/s23218824 · Cited by 182
“Users can decide whether to add, delete, and pass information GRU [185] is a variant based on the LSTM, which only contains an update gate and reset gate and has a faster training speed than the LSTM with fewer parameters.”
BackgroundGenerative AI in cybersecurity: A comprehensive review of LLM applications and vulnerabilities
Ferrag, M. A., Alwahedi, F., Battah, A., Cherif, B., Mechri, A., Tihanyi, N., Bisztray, T., & Debbah, M. (2025) · Internet of Things and Cyber-Physical Systems · doi:10.1016/j.iotcps.2025.01.001 · Cited by 148
Neural Machine Translation with Byte-Level Subwords
Wang, C., Cho, K., & Gu, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6451 · Cited by 147
Self-citationA Practical Survey on Faster and Lighter Transformers
Fournier, Q., Caron, G. M., & Aloise, D. (2023) · ACM Computing Surveys · doi:10.1145/3586074 · Cited by 145
“[3] later improved over the LSTM with the simpler gated recurrent unit (GRU).”
BackgroundA Comprehensive Study of Activity Recognition Using Accelerometers
Twomey, N., Diethe, T., Fafoutis, X., Elsts, A., McConville, R., Flach, P., & Craddock, I. (2018) · Informatics · doi:10.3390/informatics5020027 · Cited by 143
“Our experiments use LSTM since they have been shown to perform similarly to other methods (e.g., GRU) in many settings [87].”
MethodA Comprehensive Study of Activity Recognition Using Accelerometers
Twomey, N., Diethe, T., Fafoutis, X., Elsts, A., McConville, R., Flach, P., & Craddock, I. (2018) · doi:10.20944/preprints201803.0147.v1 · Cited by 143
Interpretation of Natural Language Rules in Conversational Machine Reading
Saeidi, M., Bartolo, M., Lewis, P., Singh, S., Rocktäschel, T., Sheldon, M., Bouchard, G., & Riedel, S. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1233 · Cited by 141
“In practice, we test this hypothesis using a seq2seq model (Sutskever et al., 2014; Cho et al., 2014), with and without copy mechanisms (Gu et al., 2016) to reflect how follow-up questions often use lexical content from the rule text.”
MethodSoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation
Lin, X., Wang, Y., Olkin, J., & Held, D. (2020) · arXiv · doi:10.48550/arxiv.2011.07215 · Cited by 140
“We use a GRU [2] with 200 hidden nodes as the deterministic path in the dynamics model.”
BackgroundMethodDeepfakes: current and future trends
Gambín, Á. F., Yazidi, A., Vasilakos, A., Haugerud, H., & Djenouri, Y. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-023-10679-x · Cited by 145
Fortelny, N., & Bock, C. (2020) · Genome Biology · doi:10.1186/s13059-020-02100-5 · Cited by 135
Multimodal Sentiment Detection Based on Multi-channel Graph Neural Networks
Yang, X., Feng, S., Zhang, Y., & Wang, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.28 · Cited by 145
“For text modality , we first encode words by GloVe (Pennington et al., 2014) to obtain the embedding vector and then obtain the text memory bank, M t , by BiGRU (Cho et al., 2014): (1) where T is a text sequence, L t is the maximum length of a padded text sequence, and d t is the dimension of hidden units in the BiGRU.”
MethodTemporal Knowledge Graph Completion: A Survey
Cai, B., Xiang, Y., Gao, L., Zhang, H., Li, Y., & Li, J. (2023) · Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/734 · Cited by 133
“Moreover, [Tang et al. , 2020] adopts GRU [Cho et al. , 2014] on the sequence of hyper-planes to further capture the dynamics among hyperplanes.”
BackgroundMulla, N., & Gharpure, P. (2023) · Progress in Artificial Intelligence · doi:10.1007/s13748-023-00295-9 · Cited by 132
“Among the generative models, maximum entropy language model (MELM) [31, 75, 76], machine translation (MT) sequence-to-sequence model, and gated recurrent neural networks (GRNN), derived from [77, 78] were constructed and evaluated.”
MethodWord meaning in minds and machines.
Lake, B. M., & Murphy, G. L. (2023) · Psychological Review · doi:10.1037/rev0000297 · Cited by 131
Deep Learning in the State of Charge Estimation for Li-Ion Batteries of Electric Vehicles: A Review
Zhang, D., Zhong, C., Xu, P., & Tian, Y. (2022) · Machines · doi:10.3390/machines10100912 · Cited by 131
Hierarchical Reinforcement Learning: A Survey and Open Research Challenges
Hutsebaut-Buysse, M., Mets, K., & Latré, S. (2022) · Machine Learning and Knowledge Extraction · doi:10.3390/make4010009 · Cited by 129
“Building blocks such as Long Short-Term Memory (LSTM) [107] recurrent networks and Gated Recurrent Units (GRU ([108] allow deep learning to work with data which is sequential in nature.”
MethodLook before you leap: Detecting phishing web pages by exploiting raw URL and HTML characteristics
Opara, C., Chen, Y., & Wei, B. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121183 · Cited by 129
Raj, C., Agarwal, A., Bharathy, G., Narayan, B., & Prasad, M. (2021) · Electronics · doi:10.3390/electronics10222810 · Cited by 128
“GRU: Gated Recurrent Units (GRUs) [55] are also a type of RNN with a gated mechanism designed to deal with the vanishing and exploding gradient problem.”
BackgroundNo Need to Lift a Finger Anymore? Assessing the Quality of Code Generation by ChatGPT
Liu, Z., Tang, Y., Luo, X., Zhou, Y., & Zhang, L. F. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3392499 · Cited by 128
“Language models have a wide range of applications in NLP, including machine translation, question answering, summarization, text generation, code generation and so on [4], [9], [16], [18], [68], [69], [70], [71], [72], [73].”
BackgroundTang, X., Yao, H., Sun, Y., Aggarwal, C., Mitra, P., & Wang, S. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6056 · Cited by 125
Personalized Transformer for Explainable Recommendation
Li, L., Zhang, Y., & Chen, L. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.383 · Cited by 124
“Later works (Liu et al., 2018; Devlin et al., 2019) show that it remains effective, even when the encoder or the decoder is removed, reducing nearly half of the parameters.”
BackgroundMethodInfluentialLiu, Z., Liu, Q., Xu, W., Liu, Z., Zhou, Z., & Chen, J. (2019) · Procedia CIRP · doi:10.1016/j.procir.2019.04.080 · Cited by 123
De Novo Design of New Chemical Entities for SARS-CoV-2 Using Artificial Intelligence
Bung, N., Krishnan, S. R., Bulusu, G., & Roy, A. (2021) · Future Medicinal Chemistry · doi:10.4155/fmc-2020-0262 · Cited by 118
Learning Functional Causal Models with Generative Neural Networks
Goudet, O., Kalainathan, D., Caillou, P., Guyon, I., Lopez-Paz, D., & Sebag, M. (2018) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-319-98131-4_3 · Cited by 123
Knowledge Graph Representation with Jointly Structural and Textual Encoding
Xu, J., Qiu, X., Chen, K., & Huang, X. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/183 · Cited by 121
Abstractive Summarization: A Survey of the State of the Art
Lin, H., & Ng, V. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019815 · Cited by 120
“In some cases, GRUs (Cho et al. 2014) have been shown to be a better alternative to LSTMs (Chen et al. 2016; Kim et al. 2016; Li et al. 2017; Zhou et al. 2017), as they have fewer parameters and are faster to train while achieving comparable results (Chung et al. 2014; Greff et al. 2017).”
BackgroundResultYan, M. Y., Gustad, L. T., & Nytrø, Ø. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab236 · Cited by 119
Rengasamy, D., Jafari, M., Rothwell, B., Chen, X., & Figueredo, G. P. (2020) · Sensors · doi:10.3390/s20030723 · Cited by 118
“The gated recurrent unit (GRU) [18] is proposed as alternative to LSTM.”
BackgroundAuxiliary Tasks and Exploration Enable ObjectGoal Navigation
Ye, J., Batra, D., Das, A., & Wijmans, E. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01581 · Cited by 120
“We term output cell states from all GRUs as “beliefs.””
InfluentialWei, M., Gu, H., Ye, M., Wang, Q., Xu, X., & Wu, C. (2021) · Energy Reports · doi:10.1016/j.egyr.2021.05.019 · Cited by 116
Semantic similarity metrics for evaluating source code summarization
Haque, S., Eberhart, Z., Bansal, A., & McMillan, C. (2022) · Proceedings of the 30th IEEE/ACM International Conference on Program Comprehension · doi:10.1145/3524610.3527909 · Cited by 116
“Both the encoder and decoder layers of the at-tendgru model uses a GRU [68] to encode the text.”
MethodInfluentialA novel monitoring system for fall detection in older people
Taramasco, C., Rodenas, T., Martinez, F., Fuentes, P., Munoz, R., Olivares, R., De Albuquerque, V. H. C., & Demongeot, J. (2018) · IEEE Access · doi:10.1109/access.2018.2861331 · Cited by 113
“These are LSTM, GRU, and Bi-LSTM; we have used these RNNs because they have been highly used for classification problems [24]–[26].”
MethodIteratively Questioning and Answering for Interpretable Legal Judgment Prediction
Zhong, H., Wang, Y., Tu, C., Zhang, T., Liu, Z., & Sun, M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5479 · Cited by 113
“GRU adds a gating mechanism in the recurrent neural network unit which couples the input and forget gates to decrease the number of parameters.”
CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text
Sinha, K., Sodhani, S., Dong, J., Pineau, J., & Hamilton, W. L. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1458 · Cited by 110
“We consider bidirectional LSTMs (Hochreiter and Schmidhuber, 1997; Cho et al., 2014) (with and without attention), as well as recently proposed models that aim to incorporate inductive biases towards relational reasoning: Relation Networks (RN) (Santoro et al.”
MethodInfluentialRobust Multimodal Emotion Recognition from Conversation with Transformer-Based Crossmodality Fusion
Xie, B., Sidulova, M., & Park, C. H. (2021) · Sensors · doi:10.3390/s21144913 · Cited by 108
DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generation
Hong, S., Moon, S., Kim, J., Lee, S., Kim, M., Lee, D., & Kim, J.-Y. (2022) · 2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro56248.2022.00051 · Cited by 107
Real-Time Intended Knee Joint Motion Prediction by Deep-Recurrent Neural Networks
Huang, Y., He, Z., Liu, Y., Yang, R., Zhang, X., Cheng, G., Yi, J., Ferreira, J. P., & Liu, T. (2019) · IEEE Sensors Journal · doi:10.1109/jsen.2019.2933603 · Cited by 105
“GRU and other forms of neural network layers [23], [24], are”
BackgroundOh, B.-D., & Schuler, W. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00548 · Cited by 107
“…from several well-established types of LMs, including ngram models, Simple Recurrent Networks (Elman, 1991), Gated Recurrent Unit networks (GRU; Cho et al., 2014), and Long Short-Term Memory networks (LSTM; Hochreiter and Schmidhuber, 1997), have been compared against behavioral measures of…”
BackgroundTowards Unsupervised Image Captioning With Shared Multimodal Embeddings
Laina, I., Rupprecht, C., & Navab, N. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00751 · Cited by 111
“We evaluate our method with the official COCO evaluation code and report performance under the commonly used metrics, BLEU 1-4 [10], ROUGE [37], METEOR [13], CIDEr [58], SPICE [1] and WMD [32].”
MethodBabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning
Chevalier-Boisvert, M., Bahdanau, D., Lahlou, S., Willems, L., Saharia, C., Nguyen, T. H., & Bengio, Y. (2018) · arXiv · doi:10.48550/arxiv.1810.08272 · Cited by 104
“The Small model uses a smaller memory of 128 units and encodes the instruction with a unidirectional GRU and no attention mechanism.”
MethodInfluentialSelf-citationOptimizing risk-based breast cancer screening policies with reinforcement learning
Yala, A., Mikhael, P. G., Lehman, C., Lin, G., Strand, F., Wan, Y.-L., Hughes, K., Satuluru, S., Kim, T., Banerjee, I., et al. (2022) · Nature Medicine · doi:10.1038/s41591-021-01599-w · Cited by 103
“For our RNN model, we utilized a Gated Recurrent Unit (GRU) [47] with an additive hazard layer [12] to estimate future risk.”
MethodAutonomous Molecular Design: Then and Now
Dimitrov, T., Kreisbeck, C., Becker, J. S., Aspuru-Guzik, A., & Saikin, S. K. (2019) · ACS Applied Materials & Interfaces · doi:10.1021/acsami.9b01226 · Cited by 103
DreamCoder: bootstrapping inductive program synthesis with wake-sleep library learning
Ellis, K., Wong, C., Nye, M., Sablé-Meyer, M., Morales, L., Hewitt, L., Cary, L., Solar-Lezama, A., & Tenenbaum, J. B. (2021) · Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation · doi:10.1145/3453483.3454080 · Cited by 103
Decoding spoken English from intracortical electrode arrays in dorsal precentral gyrus
Wilson, G. H., Stavisky, S. D., Willett, F. R., Avansino, D. T., Kelemen, J. N., Hochberg, L. R., Henderson, J. M., Druckmann, S., & Shenoy, K. V. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/abbfef · Cited by 103
“The RNN was built as a single layer of 512 bidirectional gated recurrent units (Cho et al 2014) implemented with the cuDNN library (Nvidia Corp., Santa Clara, USA).”
MethodAuditing Data Provenance in Text-Generation Models
Song, C., & Shmatikov, V. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330885 · Cited by 103
Cascaded multi-modal mixing transformers for Alzheimer’s disease classification with incomplete data
Liu, L., Liu, S., Zhang, L., To, X. V., Nasrallah, F., & Chandra, S. S. (2023) · NeuroImage · doi:10.1016/j.neuroimage.2023.120267 · Cited by 101
Exploring Human-Like Attention Supervision in Visual Question Answering
Qiao, T., Dong, J., & Xu, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12272 · Cited by 100
Technical and clinical overview of deep learning in radiology
Ueda, D., Shimazaki, A., & Miki, Y. (2019) · Japanese Journal of Radiology · doi:10.1007/s11604-018-0795-3 · Cited by 100
A Review on Deep-Learning-Based Cyberbullying Detection
Hasan, M. T., Hossain, M. A. E., Mukta, M. S. H., Akter, A., Ahmed, M., & Islam, S. (2023) · Future Internet · doi:10.3390/fi15050179 · Cited by 103
“Deep Neural Networks such as Recurrent Neural Network (RNN), Gated Recurrent Unit (GRU) [8], Long Short-term Memory (LSTM) [9], Bi-LSTM [10] and several other DL models can be used to detect this problem.”
MethodZhou, J., Qin, Y., Luo, J., & Zhu, T. (2023) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3218665 · Cited by 98
“For a single GRU cell, the output hidden state at time t is calculated by [16]”
MethodPredicting functional effect of missense variants using graph attention neural networks
Zhang, H., Xu, M. S., Fan, X., Chung, W. K., & Shen, Y. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00561-w · Cited by 98
Graph Convolutional Encoders for Syntax-aware Neural Machine Translation
Bastings, J., Titov, I., Aziz, W., Marcheggiani, D., & Simaan, K. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1209 · Cited by 97
“We use the BiRNN + GCN model, i.e. a bidirectional GRU with a 1-layer GCN on top.”
BackgroundMethodInfluentialMastering Diverse Domains through World Models
Hafner, D., Pasukonis, J., Ba, J., & Lillicrap, T. (2023) · arXiv · doi:10.48550/arxiv.2301.04104 · Cited by 124
Hierarchical Attention Networks for Cyberbullying Detection on the Instagram Social Network
Cheng, L., Guo, R., Silva, Y., Hall, D., & Liu, H. (2019) · Proceedings of the 2019 SIAM International Conference on Data Mining · doi:10.1137/1.9781611975673.27 · Cited by 95
“3.1 Bidirectional GRU-RNN To model the continuous temporal phenomena, we use the bi-directional GRU [1] based RNN to encode the sequence of words and comments.”
MethodInfluentialA Survey of Generative Adversarial Networks for Synthesizing Structured Electronic Health Records
Ghosheh, G. O., Li, J., & Zhu, T. (2024) · ACM Computing Surveys · doi:10.1145/3636424 · Cited by 95
Incorporating Literals into Knowledge Graph Embeddings
Kristiadi, A., Khan, M. A., Lukovnikov, D., Lehmann, J., & Fischer, A. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-30793-6_20 · Cited by 95
ATOM: Commit Message Generation Based on Abstract Syntax Tree and Hybrid Ranking
Liu, S., Gao, C., Chen, S., Nie, L. Y., & Liu, Y. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2020.3038681 · Cited by 93
Wireless Network Intelligence at the Edge
Park, J., Samarakoon, S., Bennis, M., & Debbah, M. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2941458 · Cited by 92
“GRUs, a gating mechanism in RNNs, are commonly used for modeling speech signals and musics.”
BackgroundInfluentialHabitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scale
Ramrakhya, R., Undersander, E., Batra, D., & Das, A. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00511 · Cited by 90
“RGBD and instruction features are concatenated to form an observation embedding, which is fed into a 2-layer, 512-d GRU at every timestep.”
MethodInfluentialOnline Fall Detection Using Recurrent Neural Networks on Smart Wearable Devices
Musci, M., De Martini, D., Blago, N., Facchinetti, T., & Piastra, M. (2021) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2020.3027454 · Cited by 128
Multi-Model Long Short-Term Memory Network for Gait Recognition Using Window-Based Data Segment
Tran, L., Hoang, T., Nguyen, T., Kim, H., & Choi, D. (2021) · IEEE Access · doi:10.1109/access.2021.3056880 · Cited by 90
“Although Recurrent Neural Network (RNN) techniques ( i.e., LSTM [21], GRU [34]) are powerful for sequential data, only a few studies applied them for gait recognition [19], [22].”
MethodDeep Learning in Surgical Workflow Analysis: A Review of Phase and Step Recognition
Demir, K. C., Schieber, H., Weise, T., Roth, D., May, M., Maier, A., & Yang, S. H. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3311628 · Cited by 90
Kotei, E., & Thirunavukarasu, R. (2023) · Information · doi:10.3390/info14030187 · Cited by 89
“The transformer network has two parts (the encoder and the decoder), with selfattention for neural sequence transduction [37,38].”
BackgroundMai, S., Xing, S., & Hu, H. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2925966 · Cited by 89
Mai, S., Xing, S., & Hu, H. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3068598 · Cited by 89
“In addition, temporal convolution network (TCN) [9] has achieved excellent performance and consistently outperformed RNN variants [10]–[12] in a great number of language modeling tasks.”
BackgroundMethod
Metadata
- Authors
- Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y.
- Affiliations
- Le Mans Université (FR)
- Polytechnique Montréal (CA)
- Venue
- Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Published
- 2014
- Pages
- 1724-1734
- OpenAlex
- W2157331557
- Cited by
- 25,545
- References
- 31
- Citations by year
- 2026: 1895 · 2025: 2591 · 2024: 2721 · 2023: 3047 · 2022: 3245 · 2021: 3534 · 2020: 3063 · 2019: 2494 · 2018: 1644 · 2017: 816 · 2016: 349 · 2015: 119 · 2014: 22
- FWCI
- 256.40
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- HAL (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- statistical machine translation, neural machine translation, phrase embeddings, sequence-to-sequence models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.