Multi-task Learning for Multilingual Neural Machine Translation
Open accessAbstract
While monolingual data has been shown to be useful in improving bilingual neural machine translation (NMT), effectively and efficiently leveraging monolingual data for Multilingual NMT (MNMT) systems is a less explored area.In this work, we propose a multi-task learning (MTL) framework that jointly trains the model with the translation task on bitext data and two denoising tasks on the monolingual data.We conduct extensive empirical studies on MNMT systems with 10 language pairs from WMT datasets.We show that the proposed approach can effectively improve the translation quality for both high-resource and low-resource languages with large margin, achieving significantly better results than the individual bilingual models.We also demonstrate the efficacy of the proposed approach in the zero-shot setup for language pairs without bitext training data.Furthermore, we show the effectiveness of MTL over pre-training approaches for both NMT and cross-lingual transfer learning NLU tasks; the proposed approach outperforms massive scale models trained on single task.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Multilingual modeling has not been systematically explored for code translation.
“Multilingual modeling shows great potential in neural machine translation (Aharoni et al., 2019;Wang et al., 2020;Zhu et al., 2023), but it has not been systematically explored for code translation.”
From CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation (2023) · cites this paper as Wang et al., 2020
Multilingual modeling has been pivotal in broadening the applicability of neural machine translation, primarily evidenced in enhancing performance of low-resource languages and cross-language transfer learning.
“Multilingual modeling has been pivotal in broadening the applicability of neural machine translation (Aharoni et al., 2019;Wang et al., 2020;Zhu et al., 2023;Johnson et al., 2017).This is primarily evidenced in enhancing the performance of low-resource languages and cross-language transfer learning (Mohammadshahi et al., 2022;Zoph et al., 2016;Nguyen and Chiang, 2017;Johnson et al., 2017).”
From CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation (2023) · cites this paper as Wang et al., 2020
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138 · see citing works
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · see citing works
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · see citing works
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821 · see citing works
Extracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436 · see citing works
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636 · see citing works
Unsupervised Cross-lingual Representation Learning at Scale
Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., & Stoyanov, V. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.747 · Cited by 5,637 · see citing works
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · see citing works
Kudo, T., & Richardson, J. (2018) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-2012 · Cited by 4,454 · see citing works
fairseq: A Fast, Extensible Toolkit for Sequence Modeling
Ott, M., Edunov, S., Baevski, A., Fan, A., Gross, S., Ng, N., Grangier, D., & Auli, M. (2019) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n19-4009 · Cited by 3,538 · see citing works
An Overview of Multi-Task Learning in Deep Neural NetworksPreprint
Ruder, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05098 · Cited by 3,195 · see citing works
Convolutional Sequence to Sequence Learning
Gehring, J., Auli, M., Grangier, D., Yarats, D., & Dauphin, Y. N. (2017) · arXiv · doi:10.48550/arxiv.1705.03122 · Cited by 2,850 · see citing works
A Call for Clarity in Reporting BLEU Scores
Post, M. (2018) · Proceedings of the Third Conference on Machine Translation: Research Papers · doi:10.18653/v1/w18-6319 · Cited by 2,112 · see citing works
Learning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988 · cited 2× · see citing works
“ing algorithms perform better when starting with easier tasks and gradually move to harder ones as promoted in curriculum learning (Elman, 1993).”
BackgroundGoogle’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
Johnson, M., Schuster, M., Le, Q. V., Krikun, M., Wu, Y., Chen, Z., Thorat, N., Viégas, F. B., WATTENBERG, M. P., Corrado, G. S., et al. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00065 · Cited by 1,842 · see citing works
Multilingual Denoising Pre-training for Neural Machine Translation
Liu, Y., Gu, J., Goyal, N., Li, X., Edunov, S., Ghazvininejad, M., Lewis, M., & Zettlemoyer, L. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00343 · Cited by 1,608 · see citing works
Cross-lingual Language Model Pretraining
Lample, G., & Conneau, A. (2019) · arXiv · doi:10.48550/arxiv.1901.07291 · Cited by 1,586 · see citing works
Deng, L., Hinton, G. E., & Kingsbury, B. E. D. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639344 · Cited by 1,222 · see citing works
XNLI: Evaluating Cross-lingual Sentence Representations
Conneau, A., Rinott, R., Lample, G., Williams, A., Bowman, S. R., Schwenk, H., & Stoyanov, V. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1269 · Cited by 949 · see citing works
Transfer Learning for Low-Resource Neural Machine Translation
Zoph, B., Yüret, D., May, J., & Knight, K. K. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1163 · Cited by 802 · see citing works
Phrase-Based & Neural Unsupervised Machine Translation
Lample, G., Ott, M., Conneau, A., Denoyer, L., & Ranzato, M. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1549 · Cited by 647 · see citing works
Dual Learning for Machine TranslationPreprint
Xia, Y., He, D., Qin, T., Wang, L., Yu, N., Liu, T., & Ma, W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.00179 · Cited by 640 · see citing works
Multi-task Sequence to Sequence LearningPreprint
Luong, M.-T., Le, Q. V., Sutskever, I., Vinyals, O., & Kaiser, Ł. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06114 · Cited by 606 · see citing works
Unsupervised Machine Translation Using Monolingual Corpora Only
Lample, G., Conneau, A., Denoyer, L., & Ranzato, M. (2017) · arXiv · doi:10.48550/arxiv.1711.00043 · Cited by 584 · see citing works
Multi-Task Learning for Multiple Language Translation
Dong, D., Wu, H. R., He, W. P., Yu, D., & Wang, H. (2015) · doi:10.3115/v1/p15-1166 · Cited by 573 · see citing works
Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism
Fırat, O., Cho, K., & Bengio, Y. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1101 · Cited by 570 · see citing works
CCNet: Extracting High Quality Monolingual Datasets from Web Crawl DataPreprint
Wenzek, G., Lachaux, M.-A., Conneau, A., Chaudhary, V., Guzmán, F., Joulin, A., & Grave, É. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00359 · Cited by 362 · see citing works
Simple, Scalable Adaptation for Neural Machine Translation
Bapna, A., & Fırat, O. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1165 · Cited by 301 · see citing works
Exploiting Source-side Monolingual Data in Neural Machine Translation
Zhang, J., & Zong, C. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1160 · Cited by 280 · see citing works
Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
Arivazhagan, N., Bapna, A., Firat, O., Lepikhin, D., Johnson, M., Krikun, M., Chen, M. X., Cao, Y., Foster, G., Cherry, C., et al. (2019) · arXiv · doi:10.48550/arxiv.1907.05019 · Cited by 279 · see citing works
Universal Neural Machine Translation for Extremely Low Resource Languages
Gu, J., Hassan, H., Devlin, J., & Li, V. O. K. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1032 · Cited by 274 · see citing works
Self-citationZero-Resource Translation with Multi-Lingual Neural Machine Translation
Fırat, O., Sankaran, B., Al-Onaizan, Y., Vural, F. T. Y., & Cho, K. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1026 · Cited by 262 · see citing works
Toward Multilingual Neural Machine Translation with Universal Encoder and DecoderPreprint
Ha, T.-L., Niehues, J., & Waibel, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.04798 · Cited by 241 · see citing works
Transfer Learning across Low-Resource, Related Languages for Neural Machine TranslationPreprint
Nguyen, T. Q., & Chiang, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.09803 · Cited by 168 · see citing works
A Comparison of Pivot Methods for Phrase-Based Statistical Machine Translation
Utiyama, M., & Isahara, H. (2007) · Cited by 167 · see citing works
Learning to Parse and Translate Improves Neural Machine Translation
Eriguchi, A., Tsuruoka, Y., & Cho, K. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-2012 · Cited by 144 · see citing works
Exploiting Linguistic Resources for Neural Machine Translation Using Multi-task Learning
Niehues, J., & Cho, E. (2017) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w17-4708 · Cited by 131 · see citing works
Overcoming Catastrophic Forgetting During Domain Adaptation of Neural Machine Translation
Thompson, B. J., Gwinnup, J., Khayrallah, H., Duh, K., & Koehn, P. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1209 · Cited by 119 · see citing works
Scheduled Multi-Task Learning: From Syntax to Translation
Kiperwasser, E., & Ballesteros, M. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00017 · Cited by 76 · see citing works
Multilingual Neural Machine Translation with Task-Specific AttentionPreprint
Blackwood, G. W., Ballesteros, M., & Ward, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.03280 · Cited by 69 · see citing works
XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and GenerationPreprint
Liang, Y., Duan, N., Gong, Y., Wu, N., Guo, F., Qi, W., Gong, M., Shou, L., Jiang, D., Cao, G., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.01401 · Cited by 65 · see citing works
Exploiting Monolingual Data at Scale for Neural Machine Translation
Wu, L., Wang, Y., Xia, Y., Qin, T., Lai, J., & Liu, T. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1430 · Cited by 50 · see citing works
Self-citationMultilingual Unsupervised NMT using Shared Encoder and Language-Specific Decoders
Sen, S., Gupta, K. K., Ekbal, A., & Bhattacharyya, P. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1297 · Cited by 40 · see citing works
Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation
Siddhant, A., Johnson, M., Tsai, H., Arivazhagan, N., Riesa, J., Bapna, A., Fırat, O., & Raman, K. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6414 · Cited by 15 · see citing works
Neural Machine Translation for Bilingually Scarce Scenarios: a Deep Multi-Task Learning Approach
Zaremoodi, P., & Haffari, G. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1123 · Cited by 12 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
6 of the 55 references couldn’t be matched to a known paper.
Research works printed but not linked: 4
- Adrià De Gispert and Jose B Marino. 2006. Catalan- english statistical machine translation without paral- lel corpus: bridging through spanish. In Proc. of 5th International Conference on Language Resources and Evaluation (LREC), pages 65-68.
- Jonas Gehring, Michael Auli, David Grangier, De- nis Yarats, and Yann N Dauphin. 2017. Convolu- tional sequence to sequence learning. In Proceed- ings of the 34th International Conference on Ma- chine Learning, pages 1243-1252. JMLR. org.
- Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving neural machine translation mod- els with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Compu- tational Linguistics, pages 86-96.
- Yiren Wang, Yingce Xia, Tianyu He, Fei Tian, Tao Qin, ChengXiang Zhai, and Tie-Yan Liu. 2019. Multi- agent dual learning. In International Conference on Learning Representations.
Cited by
Sorted by citations
Neural Machine Translation for Low-resource Languages: A Survey
Ranathunga, S., Lee, E.-S. A., Skënduli, M. P., Shekhar, R., Alam, M., & Kaur, R. (2022) · ACM Computing Surveys · doi:10.1145/3567592 · Cited by 348 · see citing works
N-LTP: An Open-source Neural Language Technology Platform for Chinese
Che, W., Feng, Y., Qin, L., & Liu, T. (2021) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2021.emnlp-demo.6 · Cited by 152 · see citing works
Survey of Low-Resource Machine Translation
Haddow, B., Bawden, R., Barone, A. V. M., Helcl, J., & Birch, A. (2022) · Computational Linguistics · doi:10.1162/coli_a_00446 · Cited by 185 · see citing works
Ma, S., Dong, L., Huang, S., Zhang, D., Muzio, A. F. V., Singhal, S., Awadalla, H. H., Song, X., & Wei, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.13736 · Cited by 63 · see citing works
Improving Multilingual Neural Machine Translation System for Indic Languages
Das, S. B., Biradar, A., Mishra, T. K., & Patra, B. K. (2023) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3587932 · Cited by 46 · see citing works
Fast and Accurate Neural Machine Translation with Translation Memory
He, Q., Huang, G., Cui, Q., Li, L., & Liu, L. (2021) · doi:10.18653/v1/2021.acl-long.246 · Cited by 42 · see citing works
StableMoE: Stable Routing Strategy for Mixture of Experts
Dai, D., Dong, L., Ma, S., Zheng, B., Sui, Z., Chang, B., & Wei, F. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.489 · Cited by 47 · see citing works
Plug and Play Autoencoders for Conditional Text Generation
Mai, F., Pappas, N., Montero, I., Smith, N. A., & Henderson, J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.491 · Cited by 40 · see citing works
Yu, D., Shen, L., Hao, H., Gong, W., Wu, H., Bian, J., Dai, L.-R., & Xiong, H. (2024) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2024.3399654 · Cited by 47 · see citing works
Scalable and Efficient MoE Training for Multitask Multilingual ModelsPreprint
Kim, Y. J., Awan, A. A., Muzio, A. F. V., Cruz-Salinas, A. F., Lu, L., Hendy, A., Rajbhandari, S., He, Y., & Awadalla, H. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.10465 · Cited by 57 · see citing works
CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation
Yan, W., Tian, Y., Li, Y., Chen, Q., & Wang, W. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.337 · Cited by 31 · see citing works
Ma, S., Yang, J., Huang, H., Chi, Z., Dong, L., Zhang, D., Awadalla, H. H., Muzio, A. F. V., Eriguchi, A., Singhal, S., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.15547 · Cited by 33 · see citing works
BiTIIMT: A Bilingual Text-infilling Method for Interactive Machine Translation
Xiao, Y., Liu, L., Huang, G., Cui, Q., Huang, S., Shi, S., & Chen, J. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.138 · Cited by 21 · see citing works
Multilingual Translation via Grafting Pre-trained Language Models
Sun, Z., Wang, M., & Li, L. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.233 · Cited by 19 · see citing works
Improving Multilingual Translation by Representation and Gradient Regularization
Yang, Y., Eriguchi, A., Muzio, A. F. V., Tadepalli, P. V., Lee, S., & Hassan, H. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.578 · Cited by 19 · see citing works
Self-citationMultilingual Agreement for Multilingual Neural Machine Translation
Yang, J., Yin, Y., Ma, S., Huang, H., Zhang, D., Li, Z., & Wei, F. (2021) · doi:10.18653/v1/2021.acl-short.31 · Cited by 17 · see citing works
Deep multi-task learning: a review of concepts, methods, and cross-domain applications
Abdelsamie, M. M., Azab, S. S., & Hefny, H. A. (2025) · International Journal of Data Science and Analytics · doi:10.1007/s41060-025-00892-y · Cited by 16 · see citing works
Boosting the Transformer with the BERT Supervision in Low-Resource Machine Translation
Yan, R., Li, J., Su, X., Wang, X., & Gao, G. (2022) · Applied Sciences · doi:10.3390/app12147195 · Cited by 16 · see citing works
Multilingual Machine Translation Systems from Microsoft for WMT21 Shared TaskPreprint
Yang, J., Ma, S., Huang, H., Zhang, D., Dong, L., Huang, S., Muzio, A. F. V., Singhal, S., Awadalla, H. H., Song, X., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.02086 · Cited by 15 · see citing works
High-resource Language-specific Training for Multilingual Neural Machine Translation
Yang, J., Yin, Y., Ma, S., Zhang, D., Li, Z., & Wei, F. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/619 · Cited by 13 · see citing works
PoE: A Panel of Experts for Generalized Automatic Dialogue Assessment
Zhang, C., D’Haro, L. F., Zhang, Q., Friedrichs, T., & Li, H. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3250825 · Cited by 12 · see citing works
mt4CrossOIE: Multi-stage tuning for cross-lingual open information extraction
Li, T., Wang, Z., Chai, L., Yang, J., Bai, J., Yin, Y., Liu, J., Guo, H., Yang, L., el-abidine, H. Z., et al. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124760 · Cited by 13 · see citing works
Yang, J., Yin, Y., Ma, S., Zhang, D., Wu, S., Guo, H., Li, Z., & Wei, F. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/618 · Cited by 11 · see citing works
Polyglot Prompt: Multilingual Multitask Prompt Training
Fu, J., Ng, S.-K., & Liu, P. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.674 · Cited by 23 · see citing works
Linguistically Driven Multi-Task Pre-Training for Low-Resource Neural Machine Translation
Mao, Z., Chu, C., & Kurohashi, S. (2022) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3491065 · Cited by 11 · see citing works
HexT5: Unified Pre-Training for Stripped Binary Code Information Inference
Xiong, J., Chen, G., Chen, K., Gao, H., Cheng, S., & Zhang, W. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00099 · Cited by 10 · see citing works
Rethinking the Exploitation of Monolingual Data for Low-Resource Neural Machine Translation
Pang, J., Yang, B., Wong, D. F., Wan, Y., Liu, D., Chao, L. S., & Xie, J. (2023) · Computational Linguistics · doi:10.1162/coli_a_00496 · Cited by 13 · see citing works
On the Off-Target Problem of Zero-Shot Multilingual Neural Machine Translation
Chen, L., Ma, S., Zhang, D., Wei, F., & Chang, B. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.608 · Cited by 9 · see citing works
Unifying the Convergences in Multilingual Neural Machine Translation
Huang, Y., Feng, X., Geng, X., & Qin, B. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.458 · Cited by 9 · see citing works
Deep Fusing Pre-trained Models into Neural Machine Translation
Weng, R., Yu, H., Luo, W., & Zhang, M. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21399 · Cited by 9 · see citing works
Performance analysis and modeling for quantum computing simulation on distributed GPU platforms
Ahmadzadeh, A., & Sarbazi‐Azad, H. (2024) · Quantum Information Processing · doi:10.1007/s11128-024-04580-x · Cited by 8 · see citing works
Sheshadri, S. K., Gupta, D., Paul, B., & Sivabhavani, J. (2025) · IEEE Access · doi:10.1109/access.2025.3570699 · Cited by 8 · see citing works
GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator
Yang, J. Y., Ma, S., Dong, L., Huang, S., Huang, H., Yin, Y., Zhang, D., Yang, L., Wei, F., & Li, Z. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.522 · Cited by 10 · see citing works
Multi-granularity Knowledge Sharing in Low-resource Neural Machine Translation
Mi, C., Xie, S., & Fan, Y. H. (2024) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3639930 · Cited by 8 · see citing works
HintedBT: Augmenting Back-Translation with Quality and Transliteration Hints
Ramnath, S., Johnson, M., Gupta, A., & Raghuveer, A. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.129 · Cited by 9 · see citing works
Hwang, M., Rachim, V. P., Yoo, J., Lee, Y., & Park, S. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01994-4 · Cited by 7 · see citing works
Raganato, A., Vázquez, R., Creutz, M., & Tiedemann, J. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.664 · Cited by 7 · see citing works
Improving Multilingual Neural Machine Translation with Auxiliary Source Languages
Xu, W., Yin, Y., Ma, S., Zhang, D., & Huang, H. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.260 · Cited by 6 · see citing works
Video question answering supported by a multi-task learning objective
Falcon, A., Serra, G., & Lanz, O. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-14333-0 · Cited by 5 · see citing works
Exploring Unsupervised Pretraining Objectives for Machine Translation
Baziotis, C., Titov, I., Birch, A., & Haddow, B. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.261 · Cited by 4 · see citing works
Language as a latent sequence: Deep latent variable models for semi-supervised paraphrase generation
Yu, J., Cristea, A. I., Harit, A., Sun, Z., Aduragba, O. T., Shi, L., & Moubayed, N. A. (2023) · AI Open · doi:10.1016/j.aiopen.2023.05.001 · Cited by 4 · see citing works
Kozhirbayev, Z., & Yessenbayev, Z. (2025) · Electronics · doi:10.3390/electronics14153126 · Cited by 3 · see citing works
Zhang, J., Huang, H., Hu, Y., Guo, P., & Xie, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-40292-0_12 · Cited by 3 · see citing works
Part-of-Speech Tags Guide Low-Resource Machine Translation
Kadeer, Z., Yi, N., & Wumaier, A. (2023) · Electronics · doi:10.3390/electronics12163401 · Cited by 3 · see citing works
Modeling Task-Aware MIMO Cardinality for Efficient Multilingual Neural Machine Translation
Xu, H., Liu, Q., van Genabith, J., & Xiong, D. (2021) · doi:10.18653/v1/2021.acl-short.46 · Cited by 3 · see citing works
Language-Aware Multilingual Machine Translation with Self-Supervised Learning
Xu, H., Maillard, J., & Goswami, V. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.38 · Cited by 2 · see citing works
Netmarble AI Center's WMT21 Automatic Post-Editing Shared Task SubmissionPreprint
Oh, S., Jang, S., Xu, H., An, S., & Oh, I. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.06515 · Cited by 3 · see citing works
N-LTP: An Open-source Neural Language Technology Platform for ChinesePreprint
Che, W., Feng, Y., Qin, L., & Liu, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.11616 · Cited by 20 · see citing works
Fan, Y., Li, P., Kong, F., & Zhu, Q. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3568211 · Cited by 1 · see citing works
An Innovative Speech Translation System for Accurate Idiomatic and Contextual Interpretation
Koreddi, V., Yasmin, N., Divya, N. S., & Somanath, N. (2025) · Algorithms for intelligent systems · doi:10.1007/978-981-96-5265-5_48 · Cited by 1 · see citing works
Ensembling of Distilled Models from Multi-task Teachers for Constrained Resource Language PairsPreprint
Hendy, A., Gad, E. A., Abdelghaffar, M., ElMosalami, J. S., Afify, M., Tawfik, A. Y., & Awadalla, H. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.13284 · Cited by 1 · see citing works
Improving Chinese-Centric Low-Resource Translation Using English-Centric Pivoted Parallel Data
wang, X., Mu, L., & Xu, H. (2023) · International Conference on Asian Language Processing · doi:10.1109/ialp61005.2023.10337297
Collaborative Multi-Task Representation for Natural Language Understanding
Yang, Y., Cao, D., Zeng, M., Yu, J. X., Ren, X., Tong, Y., & Wang, Y. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650257
Self-citationA Unified Language Model Approach for the Semantic Interpretation of Traffic Signs
Strelnikoff, S., Pullabhotla, N., Xu, J., & Ashari, A. E. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825930
IPC-Tree-Enhanced Structured Patent Machine Translation
Ye, W., & Liu, W. (2025) · International Conference on Neural Networks, Information and Communication Engineering (NNICE) · doi:10.1109/nnice64954.2025.11064346
Luo, Z., Yang, W., Yin, Y., Wei, F., He, J., & Chen, J. B. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11581453
Zhang, J., Lü, M., & Deng, Y. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729491
- Authors
- Wang, Y., Zhai, C., & Hassan, H.
- Affiliations
- Microsoft (United States) (US)
- Corresponding author
- Zhai, ChengXiang
- ORCID
- Zhai, ChengXiang
- Venue
- Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Published
- 2020
- Pages
- 1022-1034
- OpenAlex
- W3105038888
- Cited by
- 59
- References
- 55
- Citations by year
- 2026: 2 · 2025: 7 · 2024: 6 · 2023: 13 · 2022: 12 · 2021: 15 · 2020: 4
- FWCI
- 4.20
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- multilingual neural machine translation, multi-task learning, monolingual data, zero-shot translation, low-resource languages, cross-lingual transfer learning, pre-training, joint training
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
