Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Open accessAbstract
In this paper, we propose a novel neural network model called RNN Encoder-Decoder that consists of two recurrent neural networks (RNN). One RNN encodes a sequence of symbols into a fixed-length vector representation, and the other decodes the representation into another sequence of symbols. The encoder and decoder of the proposed model are jointly trained to maximize the conditional probability of a target sequence given a source sequence. The performance of a statistical machine translation system is empirically found to improve by using the conditional probabilities of phrase pairs computed by the RNN Encoder-Decoder as an additional feature in the existing log-linear model. Qualitatively, we show that the proposed model learns a semantically and syntactically meaningful representation of linguistic phrases.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Recurrent neural networks, including long short-term memory and gated recurrent networks, are state-of-the-art approaches in sequence modeling and transduction problems such as language modeling and machine translation.
“Recurrent neural networks, long short-term memory [13] and gated recurrent [7] neural networks in particular, have been firmly established as state of the art approaches in sequence modeling and transduction problems such as language modeling and machine translation [35, 2, 5].”
From Attention Is All You Need (2017) · cites this paper as [5]
It is not always optimal to use very large sequences in recurrent networks due to memory and training instability concerns.
“Due to memory and training instability concerns [44] its not always optimal to use very large sequences in recurrent networks.”
From End-to-End Multimodal Emotion Recognition using Deep Neural Networks (2017) · cites this paper as [44]
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · see citing works
(2011) · Cited by 6,993 · see citing works
Self-citationADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322 · see citing works
Statistical phrase-based translation
Koehn, P., Och, F. J., & Marcu, D. (2003) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/1073445.1073462 · Cited by 3,265 · see citing works
(2005) · Cited by 3,125 · see citing works
Supervised Sequence Labelling with Recurrent Neural Networks
Graves, A. (2012) · Studies in Computational Intelligence · doi:10.1007/978-3-642-24797-2 · Cited by 3,120 · see citing works
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011 · see citing works
Recurrent Continuous Translation Models
Kalchbrenner, N., & Blunsom, P. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1176 · Cited by 1,277 · see citing works
Exact solutions to the nonlinear dynamics of learning in deep linear neural networksPreprint
Saxe, A., McClelland, J. L., & Ganguli, S. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1312.6120 · Cited by 1,010 · see citing works
Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection
Socher, R., Huang, E., Pennin, J., Manning, C. D., & Ng, A. Y. (2011) · Cited by 785 · see citing works
(2014) · International Conference on Learning Representations · Cited by 634 · see citing works
Self-citationIntelligent Selection of Language Model Training Data
Moore, R. C., & Lewis, W. D. (2010) · Cited by 530 · see citing works
Bilingual Word Embeddings for Phrase-Based Machine Translation
Zou, W. Y., Socher, R., Cer, D. M., & Manning, C. D. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1141 · Cited by 529 · see citing works
Continuous space language models
Schwenk, H. (2006) · Computer Speech & Language · doi:10.1016/j.csl.2006.09.003 · Cited by 524 · see citing works
Self-citationFast and Robust Neural Network Joint Models for Statistical Machine Translation
Devlin, J., Zbib, R., Huang, Z., Lamar, T., Schwartz, R., & Makhoul, J. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1129 · Cited by 481 · see citing works
Advances in optimizing recurrent networks
Bengio, Y., Boulanger-Lewandowski, N., & Pascanu, R. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639349 · Cited by 478 · see citing works
Self-citationDomain Adaptation via Pseudo In-Domain Data Selection
Axelrod, A. E., He, X., & Gao, J. (2011) · Cited by 473 · see citing works
A phrase-based, joint probability model for statistical machine translation
Marcu, D., & Wong, W. (2002) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1118693.1118711 · Cited by 473 · see citing works
An Autoencoder Approach to Learning Bilingual Word RepresentationsPreprint
P, S. C. A., Lauly, S., Larochelle, H., Khapra, M. M., Ravindran, B., Raykar, V. C., & Saha, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1402.1454 · Cited by 309 · see citing works
Joint Language and Translation Modeling with Recurrent Neural Networks
Auli, M., Galley, M., Quirk, C., & Zweig, G. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1106 · Cited by 260 · see citing works
Decoding with Large-Scale Neural Language Models Improves Translation
Vaswani, A., Zhao, Y., Fossum, V., & Chiang, D. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1140 · Cited by 215 · see citing works
Continuous space translation models with neural networks
Le, H.-S., Allauzen, A., & Yvon, F. (2012) · HAL · Cited by 118 · see citing works
Continuous Space Translation Models for Phrase-Based Statistical Machine Translation
Schwenk, H. (2025) · doi:10.57702/tyvla89t · Cited by 112 · see citing works
Self-citationContinuous Space Language Models for the IWSLT 2006 Task
Schwenk, H., Costa‐jussà, M. R., & Fonollosa, J. A. R. (2006) · Cited by 22 · see citing works
Self-citation
Flagged first, then by citations
1 of the 27 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · see citing works
Self-citationA survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332 · see citing works
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., & Bengio, Y. (2017) · arXiv · doi:10.48550/arxiv.1710.10903 · Cited by 12,107
Self-citationGraph Attention NetworksPreprint
Veličković, P., Casanova, A., Liò, P., Cucurull, G., Romero, A., & Bengio, Y. (2017) · arXiv (Cornell University) · doi:10.17863/cam.48429 · Cited by 12,099 · see citing works
Self-citationConvolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-K., & Woo, W.-C. (2015) · arXiv · doi:10.48550/arxiv.1506.04214 · Cited by 7,709 · see citing works
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955 · see citing works
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242 · see citing works
A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures
Yu, Y., Si, X., Hu, C., & Zhang, J. (2019) · Neural Computation · doi:10.1162/neco_a_01199 · Cited by 5,662 · see citing works
Convolutional Sequence to Sequence Learning
Gehring, J., Auli, M., Grangier, D., Yarats, D., & Dauphin, Y. N. (2017) · arXiv · doi:10.48550/arxiv.1705.03122 · Cited by 2,850 · see citing works
“Popular choices for recurrent networks in encoder-decoder models are long short term memory networks (LSTM; Hochreiter & Schmidhuber, 1997) and gated recurrent units (GRU; Cho et al., 2014).”
BackgroundA Neural Attention Model for Abstractive Sentence Summarization
Rush, A. M., Chopra, S., & Weston, J. (2015) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1044 · Cited by 2,879 · see citing works
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., Jiang, D., et al. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.139 · Cited by 2,824 · see citing works
All-optical machine learning using diffractive deep neural networks
Lin, X., Rivenson, Y., Yardimci, N. T., Veli, M., Luo, Y., Jarrahi, M., & Ozcan, A. (2018) · Science · doi:10.1126/science.aat8084 · Cited by 2,568 · see citing works
Gated Graph Sequence Neural NetworksPreprint
Li, Y., Tarlow, D., Brockschmidt, M., & Zemel, R. (2015) · International Conference on Learning Representations · doi:10.48550/arxiv.1511.05493 · Cited by 3,735 · see citing works
Self-Attention with Relative Position Representations
Shaw, P., Uszkoreit, J., & Vaswani, A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2074 · Cited by 2,505 · see citing works
Film: Visual reasoning with a general conditioning layer
Perez, E., Strub, F., De Vries, H., Dumoulin, V., & Courville, A. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11671 · Cited by 2,490 · see citing works
Deep learning in remote sensing applications: A meta-analysis and review
Ma, L., Liu, Y., Zhang, X., Ye, Y., Yin, G., & Johnson, B. A. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.04.015 · Cited by 2,456 · see citing works
Power of Deep Learning for Channel Estimation and Signal Detection in OFDM Systems
Ye, H., Li, G. Y., & Juang, B.-H. (2018) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2017.2757490 · Cited by 2,000 · see citing works
Review of Deep Learning Algorithms and Architectures
Shrestha, A., & Mahmood, A. (2019) · IEEE Access · doi:10.1109/access.2019.2912200 · Cited by 1,883 · see citing works
“GRUs are smaller in size than LSTM as they don’t include the output gate, and can perform better than LSTM on only some simpler datasets [44], [45].”
BackgroundDeep learning in environmental remote sensing: Achievements and challenges
Yuan, Q., Shen, H., Li, T., Li, Z., Li, S., Jiang, Y., Xu, H., Tan, W., Yang, Q., Wang, J., et al. (2020) · Remote Sensing of Environment · doi:10.1016/j.rse.2020.111716 · Cited by 1,697 · see citing works
Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models
Serban, I. V., Sordoni, A., Bengio, Y., Courville, A. C., & Pineau, J. (2016) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v30i1.9883 · Cited by 1,639 · see citing works
Self-citationLarge-Scale Point Cloud Semantic Segmentation with Superpoint Graphs
Landrieu, L., & Simonovsky, M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00479 · Cited by 1,543 · see citing works
Character-Aware Neural Language Models
Kim, Y., Jernite, Y., Sontag, D. A., & Rush, A. M. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10362 · Cited by 1,546 · see citing works
A Survey of Machine and Deep Learning Methods for Internet of Things (IoT) Security
Al-Garadi, M. A., Mohamed, A., Al-Ali, A. K., Du, X., Ali, I., & Guizani, M. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2988293 · Cited by 1,392 · see citing works
AllenNLP: A Deep Semantic Natural Language Processing Platform
Gardner, M., Grus, J., Neumann, M., Tafjord, O., Dasigi, P., Liu, N. F., Peters, M. E., Schmitz, M., & Zettlemoyer, L. (2018) · Workshop for NLP Open Source Software (NLP-OSS) · doi:10.18653/v1/w18-2501 · Cited by 1,273 · see citing works
Justifying Recommendations using Distantly-Labeled Reviews and Fine-Grained Aspects
Ni, J., Li, J., & McAuley, J. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1018 · Cited by 1,283 · see citing works
Norouzzadeh, M. S., Nguyen, A., Kosmala, M., Swanson, A., Palmer, M. S., Packer, C., & Clune, J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1719367115 · Cited by 1,230 · see citing works
Deep Learning for Anomaly Detection: A Survey
Chalapathy, R., & Chawla, S. (2019) · arXiv · doi:10.48550/arxiv.1901.03407 · Cited by 1,240 · see citing works
Micikevicius, P., Narang, S., Alben, J., Diamos, G., Elsen, E., Garcia, D., Ginsburg, B., Houston, M., Kuchaiev, O., Venkatesh, G., et al. (2017) · arXiv · doi:10.48550/arxiv.1710.03740 · Cited by 1,179 · see citing works
End-to-end attention-based large vocabulary speech recognition
Bahdanau, D., Chorowski, J., Serdyuk, D., Brakel, P., & Bengio, Y. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472618 · Cited by 1,129 · see citing works
Self-citationArtificial Intelligence in Precision Cardiovascular Medicine
Krittanawong, C., Zhang, H., Wang, Z., Aydar, M., & Kitai, T. (2017) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2017.03.571 · Cited by 1,126 · see citing works
Lian, J., Zhou, X., Zhang, F., Chen, Z., Xie, X., & Sun, G. (2018) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3219819.3220023 · Cited by 1,050 · see citing works
MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding
Fu, X., Zhang, J., Meng, Z., & King, I. (2020) · International World Wide Web Conference (WWW) · doi:10.1145/3366423.3380297 · Cited by 1,056 · see citing works
Neural Network Acceptability Judgments
Warstadt, A., Singh, A., & Bowman, S. R. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00290 · Cited by 984 · see citing works
A review of wind speed and wind power forecasting with deep neural networks
Wang, Y., Zou, R., Liu, F., Zhang, L., & Liu, Q. (2021) · Applied Energy · doi:10.1016/j.apenergy.2021.117766 · Cited by 974 · see citing works
Deep Reinforcement Learning: A Survey
Wang, X., Wang, S., Liang, X., Zhao, D., Huang, J., Xu, X., Dai, B., & Miao, Q. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3207346 · Cited by 972 · see citing works
Deep Forest: Towards An Alternative to Deep Neural Networks
Zhou, Z., & Feng, J. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/497 · Cited by 963 · see citing works
The Case for Learned Index Structures
Kraska, T., Beutel, A., Chi, E. H., Dean, J., & Polyzotis, N. (2018) · Proceedings of the 2018 International Conference on Management of Data · doi:10.1145/3183713.3196909 · Cited by 956 · see citing works
Abstractive Sentence Summarization with Attentive Recurrent Neural Networks
Chopra, S., Auli, M., & Rush, A. M. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1012 · Cited by 896 · see citing works
Application of Long Short-Term Memory (LSTM) Neural Network for Flood Forecasting
Le, X.-H., Ho, H. V., Lee, G., & Jung, S. (2019) · Water · doi:10.3390/w11071387 · Cited by 893 · see citing works
Xiao, C., Choi, E., & Sun, J. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy068 · Cited by 792 · see citing works
Deep Audio-Visual Speech Recognition
Afouras, T., Chung, J. S., Senior, A., Vinyals, O., & Zisserman, A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2889052 · Cited by 801 · see citing works
Feature dimensionality reduction: a review
Jia, W., Sun, M., Lian, J., & Hou, S. (2022) · Complex & Intelligent Systems · doi:10.1007/s40747-021-00637-x · Cited by 797 · see citing works
Deep Learning in Medical Ultrasound Analysis: A Review
Liu, S., Wang, Y., Yang, X., Lei, B., Liu, L., Li, S. X., Ni, D., & Wang, T. (2019) · Engineering · doi:10.1016/j.eng.2018.11.020 · Cited by 798 · see citing works
A Theoretically Grounded Application of Dropout in Recurrent Neural NetworksPreprint
Gal, Y., & Zoubin Ghahramani, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.05287 · Cited by 765 · see citing works
Deep Reinforcement Learning: An Overview
Li, Y. (2017) · arXiv · doi:10.48550/arxiv.1701.07274 · Cited by 762 · see citing works
Contrastive Learning for Sequential Recommendation
Xie, X., Sun, F., Liu, Z., Wu, S., Gao, J., Zhang, J., Ding, B., & Cui, B. (2022) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde53745.2022.00099 · Cited by 810 · see citing works
AI applications to medical images: From machine learning to deep learning
Castiglioni, I., Rundo, L., Codari, M., Leo, G. D., Salvatore, C., Interlenghi, M., Gallivanone, F., Cozzi, A., D’Amico, N. C., & Sardanelli, F. (2021) · Physica Medica · doi:10.1016/j.ejmp.2021.02.006 · Cited by 737 · see citing works
Privacy-Preserving Traffic Flow Prediction: A Federated Learning Approach
Liu, Y., Yu, J. J. Q., Kang, J., Niyato, D., & Zhang, S. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2991401 · Cited by 712 · see citing works
Modeling Coverage for Neural Machine Translation
Tu, Z., Lu, Z., Liu, Y., Liu, X., & Li, H. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1008 · Cited by 702 · see citing works
Order Matters: Sequence to sequence for setsPreprint
Vinyals, O., Samy Bengio, A., & Kudlur, M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06391 · Cited by 696 · see citing works
A C-LSTM Neural Network for Text ClassificationPreprint
Zhou, C., Sun, C., Liu, Z., & Lau, F. C. M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.08630 · Cited by 666 · see citing works
Deep learning in medical image registration: a review
Fu, Y., Lei, Y., Wang, T., Curran, W. J., Liu, T., & Yang, X. (2020) · Physics in Medicine & Biology · doi:10.1088/1361-6560/ab843e · Cited by 661 · see citing works
Dual Learning for Machine TranslationPreprint
Xia, Y., He, D., Qin, T., Wang, L., Yu, N., Liu, T., & Ma, W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.00179 · Cited by 640 · see citing works
A survey on fake news and rumour detection techniques
Bondielli, A., & Marcelloni, F. (2019) · Information Sciences · doi:10.1016/j.ins.2019.05.035 · Cited by 658 · see citing works
Reinforcement Learning for Solving the Vehicle Routing ProblemPreprint
Nazari, M., Oroojlooy, A., Snyder, L., & Takáč, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04240 · Cited by 638 · see citing works
Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning
Everett, M., Chen, Y. F., & How, J. P. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593871 · Cited by 632 · see citing works
Phrase-Based & Neural Unsupervised Machine Translation
Lample, G., Ott, M., Conneau, A., Denoyer, L., & Ranzato, M. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1549 · Cited by 647 · see citing works
Self-referencing embedded strings (SELFIES): A 100% robust molecular string representation
Krenn, M., Häse, F., Nigam, A., Friederich, P., & Aspuru‐Guzik, A. (2020) · Repository KITopen (Karlsruhe Institute of Technology) · doi:10.5445/ir/1000128111 · Cited by 616 · see citing works
Using the Output Embedding to Improve Language Models
Press, O., & Wolf, L. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-2025 · Cited by 642 · see citing works
Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection
Çakır, E., Parascandolo, G., Heittola, T., Huttunen, H. J., & Virtanen, T. I. (2017) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2017.2690575 · Cited by 604 · see citing works
Delving Deeper into Convolutional Networks for Learning Video RepresentationsPreprint
Ballas, N., Yao, L., Pal, C., & Courville, A. C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06432 · Cited by 715 · see citing works
Large-scale comparison of machine learning methods for drug target prediction on ChEMBL
Mayr, A., Klambauer, G., Unterthiner, T., Steijaert, M., Wegner, J. K., Ceulemans, H., Clevert, D.-A., & Hochreiter, S. (2018) · Chemical Science · doi:10.1039/c8sc00148k · Cited by 584 · see citing works
Hyperspectral Image Classification With Deep Learning Models
Yang, X., Ye, Y., Li, X., Lau, R. Y. K., Zhang, X., & Huang, X. (2018) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2018.2815613 · Cited by 582 · see citing works
Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism
Fırat, O., Cho, K., & Bengio, Y. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1101 · Cited by 570 · see citing works
Self-citationTransductive LSTM for time-series prediction: An application to weather forecasting
Karevan, Z., & Suykens, J. A. K. (2020) · Neural Networks · doi:10.1016/j.neunet.2019.12.030 · Cited by 578 · see citing works
A Survey of Deep Learning Methods for Cyber Security
Berman, D. S., Buczak, A. L., Chavis, J. S., & Corbett, C. L. (2019) · Information · doi:10.3390/info10040122 · Cited by 569 · see citing works
Artificial Intelligence and COVID-19: Deep Learning Approaches for Diagnosis and Treatment
Jamshidi, M., Lalbakhsh, A., Talla, J., Peroutka, Z., Hadjilooei, F., Lalbakhsh, P., Jamshidi, M., Spada, L. L., Mirmozafari, M., Dehghani, M., et al. (2020) · IEEE Access · doi:10.1109/access.2020.3001973 · Cited by 551 · see citing works
Survey on semantic segmentation using deep learning techniques
Lateef, F., & Ruichek, Y. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.02.003 · Cited by 561 · see citing works
Deep Learning for Entity Matching
Mudgal, S., Li, H., Rekatsinas, T., Doan, A., Park, Y., Krishnan, G., Deep, R., Arcaute, E., & Raghavendra, V. (2018) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3183713.3196926 · Cited by 563 · see citing works
State-of-charge estimation of lithium-ion batteries based on gated recurrent neural network
Yang, F., Li, W., Li, C., & Miao, Q. (2019) · Energy · doi:10.1016/j.energy.2019.03.059 · Cited by 543 · see citing works
Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection
Haliassos, A., Vougioukas, K., Petridis, S., & Pantic, M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00500 · Cited by 537 · see citing works
From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label ClassificationPreprint
Martins, A. F. T., & Astudillo, R. F. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.02068 · Cited by 531 · see citing works
Sequence-to-Sequence Prediction of Vehicle Trajectory via LSTM Encoder-Decoder Architecture
Park, S. H., Kim, B., Kang, C. M., Chung, C. C., & Choi, J. W. (2018) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2018.8500658 · Cited by 528 · see citing works
Exploration by Random Network Distillation
Burda, Y., Edwards, H., Storkey, A., & Klimov, O. (2018) · arXiv · doi:10.48550/arxiv.1810.12894 · Cited by 527 · see citing works
Predicting citywide crowd flows using deep spatio-temporal residual networks
Zhang, J., Zheng, Y., Qi, D., Li, R., Yi, X., & Li, T. (2018) · Artificial Intelligence · doi:10.1016/j.artint.2018.03.002 · Cited by 524 · see citing works
Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and GaugesPreprint
Bronstein, M. M., Bruna, J., Cohen, T., & Veličković, P. (2021) · arXiv (Cornell University) · doi:10.4230/oasics.dx.2026.1 · Cited by 928 · see citing works
TransFuser: Imitation With Transformer-Based Sensor Fusion for Autonomous Driving
Chitta, K., Prakash, A., Jaeger, B., Yu, Z., Renz, K., & Geiger, A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3200245 · Cited by 528 · see citing works
MORAN: A Multi-Object Rectified Attention Network for scene text recognition
Luo, C., Jin, L., & Sun, Z. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.01.020 · Cited by 511 · see citing works
Efficient Content-Based Sparse Attention with Routing Transformers
Roy, A., Saffar, M., Vaswani, A., & Grangier, D. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00353 · Cited by 509 · see citing works
Deep learning methods for forecasting COVID-19 time-Series data: A Comparative study
Zeroual, A., Harrou, F., Dairi, A., & Sun, Y. (2020) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2020.110121 · Cited by 508 · see citing works
SeqTrack: Sequence to Sequence Learning for Visual Object Tracking
Chen, X., Peng, H., Wang, D., Lu, H., & Han, H. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01400 · Cited by 501 · see citing works
Cui, Z., Ke, R., Pu, Z., & Wang, Y. (2020) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2020.102674 · Cited by 490 · see citing works
On Using Monolingual Corpora in Neural Machine TranslationPreprint
Gülçehre, Ç., Fırat, O., Xu, K., Cho, K., Barrault, L., Lin, H.-C., Bougares, F., Schwenk, H., & Bengio, Y. (2015) · HAL · doi:10.48550/arxiv.1503.03535 · Cited by 470 · see citing works
Self-citationGraph-Structured Representations for Visual Question Answering
Teney, D., Liu, L., & van den Hengel, A. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.344 · Cited by 464 · see citing works
Deep learning for deepfakes creation and detection: A survey
Nguyen, T. T., Nguyen, Q. V. H., Nguyen, D. T., Nguyen, D. T., Huynh-The, T., Nahavandi, S., Nguyen, T. T., Pham, Q.-V., & Nguyen, C. M. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2022.103525 · Cited by 464 · see citing works
Tourism demand forecasting: A deep learning approach
Law, R., Li, G., Fong, D. K. C., & Han, X. (2019) · Annals of Tourism Research · doi:10.1016/j.annals.2019.01.014 · Cited by 453 · see citing works
Deep learning models for solar irradiance forecasting: A comprehensive review
Kumari, P., & Toshniwal, D. (2021) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2021.128566 · Cited by 463 · see citing works
Multivariate time series forecasting via attention-based encoder–decoder framework
Du, S., Li, T., Yang, Y. Z., & Horng, S. (2020) · Neurocomputing · doi:10.1016/j.neucom.2019.12.118 · Cited by 442 · see citing works
Deep learning and its application in geochemical mapping
Zuo, R., Xiong, Y., Wang, J., & Carranza, E. J. M. (2019) · Earth-Science Reviews · doi:10.1016/j.earscirev.2019.02.023 · Cited by 435 · see citing works
Convolutional Image Captioning
Aneja, J., Deshpande, A., & Schwing, A. G. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00583 · Cited by 426 · see citing works
An Actor-Critic Algorithm for Sequence Prediction
Bahdanau, D., Brakel, P., Xu, K., Goyal, A., Lowe, R., Pineau, J., Courville, A., & Bengio, Y. (2016) · arXiv · doi:10.48550/arxiv.1607.07086 · Cited by 423 · see citing works
Self-citationDeep learning and its application to LHC physics
Guest, D., Cranmer, K., & Whiteson, D. (2018) · Annual Review of Nuclear and Particle Science · doi:10.1146/annurev-nucl-101917-021019 · Cited by 417 · see citing works
Graph-based Knowledge Tracing: Modeling Student Proficiency Using Graph Neural Network
Hiromi, N., Iwasawa, Y., & Matsuo, Y. (2019) · IEEE/WIC/ACM International Conference on Web Intelligence · doi:10.1145/3350546.3352513 · Cited by 414 · see citing works
Entangled Transformer for Image Captioning
Li, G., Zhu, L., Liu, P., & Yang, Y. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00902 · Cited by 406 · see citing works
Predicting alzheimer’s disease progression using multi-modal deep learning approach
Lee, G., Nho, K., Kang, B., Sohn, K.-A., Kim, D., Initiative, F. A. D. N., Weiner, M. W., Aisen, P., Petersen, R., Jack, C. R., et al. (2019) · Scientific Reports · doi:10.1038/s41598-018-37769-z · Cited by 406 · see citing works
Simple and Effective Multi-Paragraph Reading Comprehension
Clark, C., & Gardner, M. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1078 · Cited by 402 · see citing works
Probabilistic forecasting with temporal convolutional neural network
Chen, Y., Kang, Y., Chen, Y., & Wang, Z. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.03.011 · Cited by 403 · see citing works
Tips and Tricks for Visual Question Answering: Learnings from the 2017 Challenge
Teney, D., Anderson, P., He, X., & van den Hengel, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00444 · Cited by 398 · see citing works
Deep Speaker: an End-to-End Neural Speaker Embedding SystemPreprint
Li, C., Ma, X., Jiang, B., Li, X., Zhang, X., Liu, X., Cao, Y., Kannan, A., & Zhu, Z. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.02304 · Cited by 392 · see citing works
Chen, Y., Peng, G., Zhu, Z., & Li, S. (2019) · Applied Soft Computing · doi:10.1016/j.asoc.2019.105919 · Cited by 390 · see citing works
- Authors
- Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y.
- Affiliations
- Université de Montréal (CA)
- Constructor University (DE)
- Le Mans Université (FR)
- Corresponding author
- Bahdanau, Dzmitry
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2014
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1406.1078
- OpenAlex
- W2950635152
- Cited by
- 3,317
- References
- 27
- Citations by year
- 2026: 335 · 2025: 348 · 2024: 369 · 2023: 216 · 2022: 93 · 2021: 434 · 2020: 495 · 2019: 436 · 2018: 296 · 2017: 172 · 2016: 87 · 2015: 29 · 2014: 7
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Canadian Institute for Advanced Research
- Compute Canada
- Canada Research Chairs
- Defense Advanced Research Projects Agency
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- statistical machine translation, phrase representation, recurrent neural networks, encoder-decoder architecture, log-linear models, joint training
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
