The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
Open accessAbstract
We propose a simple solution to use a single Neural Machine Translation (NMT) model to translate between multiple languages. Our solution requires no changes to the model architecture from a standard NMT system but instead introduces an artificial token at the beginning of the input sentence to specify the required target language. Using a shared wordpiece vocabulary, our approach enables Multilingual NMT systems using a single model. On the WMT’14 benchmarks, a single multilingual model achieves comparable performance for English→French and surpasses state-of-theart results for English→German. Similarly, a single multilingual model surpasses state-of-the-art results for French→English and German→English on WMT’14 and WMT’15 benchmarks, respectively. On production corpora, multilingual models of up to twelve language pairs allow for better translation of many individual pairs. Our models can also learn to perform implicit bridging between language pairs never seen explicitly during training, showing that transfer learning and zero-shot translation is possible for neural translation. Finally, we show analyses that hints at a universal interlingua representation in our models and also show some interesting examples when mixing languages.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382
Japanese and Korean voice search
Schuster, M., & Nakajima, K. (2012) · 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2012.6289079 · Cited by 862
Self-citationControlling Politeness in Neural Machine Translation via Side Constraints
Sennrich, R., Haddow, B., & Birch, A. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1005 · Cited by 273
Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translation
Zhou, J., Cao, Y., Wang, X., Li, P., & Xu, W. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00105 · Cited by 216
Cited by
Unsupervised Cross-lingual Representation Learning at Scale
Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., & Stoyanov, V. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.747 · Cited by 5,637
Parameter-efficient transfer learning for NLP
Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., de Laroussilhe, Q., Gesmundo, A., Attariyan, M., & Gelly, S. (2019) · arXiv · doi:10.48550/arxiv.1902.00751 · Cited by 2,270
A Survey of the Usages of Deep Learning for Natural Language Processing
Otter, D. W., Medina, J. R., & Kalita, J. K. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2979670 · Cited by 1,836
Cross-lingual Language Model Pretraining
Lample, G., & Conneau, A. (2019) · arXiv · doi:10.48550/arxiv.1901.07291 · Cited by 1,586
Artificial intelligence in surgery: promises and perils
Hashimoto, D. A., Rosman, G., Rus, D., & Meireles, O. R. (2018) · Annals of Surgery · doi:10.1097/sla.0000000000002693 · Cited by 1,468
Finetuned Language Models Are Zero-Shot Learners
Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2021) · arXiv · doi:10.48550/arxiv.2109.01652 · Cited by 1,101
Self-citationImageBind One Embedding Space to Bind Them All
Girdhar, R., El-Nouby, A., Liu, Z., Singh, M., Alwala, K. V., Joulin, A., & Misra, I. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01457 · Cited by 863
CTRL: A Conditional Transformer Language Model for Controllable Generation
Keskar, N. S., McCann, B., Varshney, L. R., Xiong, C., & Socher, R. (2019) · arXiv · doi:10.48550/arxiv.1909.05858 · Cited by 751
Unsupervised Neural Machine Translation
Artetxe, M., Labaka, G., Agirre, E., & Cho, K. (2017) · arXiv · doi:10.48550/arxiv.1710.11041 · Cited by 632
Unsupervised Machine Translation Using Monolingual Corpora Only
Lample, G., Conneau, A., Denoyer, L., & Ranzato, M. (2017) · arXiv · doi:10.48550/arxiv.1711.00043 · Cited by 584
Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers
Hohman, F., Kahng, M., Pienta, R., & Chau, D. H. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2843369 · Cited by 582
A Survey of Zero-Shot Learning: Settings, Methods, and Applications
Wang, W., Zheng, V. W., Yu, H., & Miao, C. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3293318 · Cited by 528
The Natural Language Decathlon: Multitask Learning as Question Answering
McCann, B., Keskar, N. S., Xiong, C., & Socher, R. (2018) · arXiv · doi:10.48550/arxiv.1806.08730 · Cited by 448
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
Lepikhin, D., Lee, H., Xu, Y., Chen, D., Firat, O., Huang, Y., Krikun, M., Shazeer, N., & Chen, Z. (2020) · arXiv · doi:10.48550/arxiv.2006.16668 · Cited by 412
Self-citationAssessing gender bias in machine translation: a case study with Google Translate
Prates, M. O. R., Avelar, P. H., & Lamb, L. C. (2020) · Neural Computing and Applications · doi:10.1007/s00521-019-04144-6 · Cited by 410
Massively Multilingual Neural Machine Translation
Aharoni, R., Johnson, M., & Firat, O. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1388 · Cited by 402
Self-citationStyle transfer through back-translation
Prabhumoye, S., Tsvetkov, Y., Salakhutdinov, R., & Black, A. W. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1080 · Cited by 348
What do Neural Machine Translation Models Learn about Morphology?
Belinkov, Y., Durrani, N., Dalvi, F., Sajjad, H., & Glass, J. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1080 · Cited by 307
Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
Arivazhagan, N., Bapna, A., Firat, O., Lepikhin, D., Johnson, M., Krikun, M., Chen, M. X., Cao, Y., Foster, G., Cherry, C., et al. (2019) · arXiv · doi:10.48550/arxiv.1907.05019 · Cited by 279
Self-citation4K-memristor analog-grade passive crossbar circuit
Kim, H., Mahmoodi, M. R., Nili, H., & Strukov, D. B. (2021) · Nature Communications · doi:10.1038/s41467-021-25455-0 · Cited by 265
Kaiser, L., Gomez, A. N., Shazeer, N., Vaswani, A., Parmar, N., Jones, L., & Uszkoreit, J. (2017) · arXiv · doi:10.48550/arxiv.1706.05137 · Cited by 253
A comprehensive survey of ChatGPT: Advancements, applications, prospects, and challenges
Nazir, A., & Wang, Z. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100022 · Cited by 245
An Empirical Comparison of Domain Adaptation Methods for Neural Machine Translation
Chu, C., Dabre, R., & Kurohashi, S. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p17-2061 · Cited by 215
Gmail Smart Compose: Real-Time Assisted Writing
Chen, M. X., Lee, B. N., Bansal, G., Cao, Y., Zhang, S., Lu, J., Tsay, J., Wang, Y., Dai, A. M., Chen, Z., et al. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330723 · Cited by 169
Self-citationA New Golden Age in Computer Architecture: Empowering the Machine-Learning Revolution
Dean, J., Patterson, D., & Young, C. (2018) · IEEE Micro · doi:10.1109/mm.2018.112130030 · Cited by 168
Self-citationArtificial intelligence in food science and nutrition: A narrative review
Miyazawa, T., Hiratsuka, Y., Toda, M., Hatakeyama, N., Ozawa, H., Abe, C., Cheng, T.-Y., Matsushima, Y., Miyawaki, Y., Ashida, K., et al. (2022) · Nutrition Reviews · doi:10.1093/nutrit/nuac033 · Cited by 91
Business value appropriation roadmap for artificial intelligence
Mishra, A. N., & Pani, A. K. (2021) · VINE Journal of Information and Knowledge Management Systems · doi:10.1108/vjikms-07-2019-0107 · Cited by 82
Perception science in the age of deep neural networks.
VanRullen, R. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00142 · Cited by 82
HOP: History-and-Order Aware Pretraining for Vision-and-Language Navigation
Qiao, Y., Qi, Y., Hong, Y., Yu, Z., Wang, P., & Wu, Q. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01498 · Cited by 82
Domain Robustness in Neural Machine Translation
Müller, M., Rios, A., & Sennrich, R. (2019) · arXiv · doi:10.48550/arxiv.1911.03109 · Cited by 68
Multi-task Learning for Multilingual Neural Machine Translation
Wang, Y., Zhai, C., & Hassan, H. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.75 · Cited by 59
To protect science, we must use LLMs as zero-shot translators
Mittelstadt, B., Wachter, S., & Russell, C. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01744-0 · Cited by 58
Erdem, E., Kuyu, M., Yagcioglu, S., Frank, A., Parcalabescu, L., Plank, B., Babii, A., Turuta, O., Erdem, A., Calixto, I., et al. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12918 · Cited by 54
Multilingual training for software engineering
Ahmed, T., & Devanbu, P. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510049 · Cited by 57
CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation
Yan, W., Tian, Y., Li, Y., Chen, Q., & Wang, W. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.337 · Cited by 31
A Multi-task Approach to Learning Multilingual Representations
Singla, K., Can, D., & Narayanan, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2035 · Cited by 22
Question Generation With Doubly Adversarial Nets
Bao, J., Gong, Y., Duan, N., Zhou, M., & Zhao, T. (2018) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2018.2859777 · Cited by 14
On the Shortcut Learning in Multilingual Neural Machine Translation
Wang, W., Jiao, W., Huang, J.-T., Tu, Z., & Lyu, M. R. (2023) · doi:10.2139/ssrn.4474513 · Cited by 1
Gmail Smart Compose: Real-Time Assisted Writing
Chen, M. X., Lee, B. N., Bansal, G., Cao, Y., Zhang, S., Lu, J., Tsay, J., Wang, Y., Dai, A. M., Chen, Z., et al. (2019) · arXiv · doi:10.48550/arxiv.1906.00080
Self-citation
Metadata
- Authors
- Johnson, M., Schuster, M., Le, Q. V., Krikun, M., Wu, Y., Chen, Z., Thorat, N., Viégas, F., Wattenberg, M., Corrado, G., Hughes, M., & Dean, J.
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press - Journals
- Published
- 2017-12
- Volume
- 5
- Pages
- 339-351
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W2550821151
- Cited by
- 1,747
- References
- 4
- FWCI
- 138.91
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
