The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Meta-Learning for Low-Resource Neural Machine Translation
Open accessAbstract
In this paper, we propose to extend the recently introduced model-agnostic meta-learning algorithm (MAML, Finn et al., 2017) for lowresource neural machine translation (NMT).We frame low-resource translation as a metalearning problem, and we learn to adapt to low-resource languages based on multilingual high-resource language tasks.We use the universal lexical representation (Gu et al., 2018b) to overcome the input-output mismatch across different languages.We evaluate the proposed meta-learning strategy using eighteen European languages (Bg, Cs, Da, De, El, Es, Et, Fr, Hu, It, Lt, Nl, Pl, Pt, Sk, Sl, Sv and Ru) as source tasks and five diverse languages (Ro, Lv, Fi, Tr and Ko) as target tasks.We show that the proposed approach significantly outperforms the multilingual, transfer learning based approach (Zoph et al., 2016) and enables us to train a competitive NMT system with only a fraction of training examples.For instance, the proposed approach can achieve as high as 22.04 BLEU on Romanian-English WMT'16 by seeing only 16,000 translated words (⇠ 600 parallel sentences).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Self-citationEnriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645
A Stochastic Approximation Method
Robbins, H., & Monro, S. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729586 · Cited by 9,643
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches
Cho, K., van Merrienboer, B., Bahdanau, D., & Bengio, Y. (2014) · Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation · doi:10.3115/v1/w14-4012 · Cited by 7,099
Self-citationPrototypical Networks for Few-shot Learning
Snell, J., Swersky, K., & Zemel, R. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.05175 · Cited by 4,755
Statistical phrase-based translation
Koehn, P., Och, F. J., & Marcu, D. (2003) · Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - NAACL '03 · doi:10.3115/1073445.1073462 · Cited by 3,197
Convolutional Sequence to Sequence Learning
Gehring, J., Auli, M., Grangier, D., Yarats, D., & Dauphin, Y. N. (2017) · arXiv · doi:10.48550/arxiv.1705.03122 · Cited by 2,850 · cited 1×
“More recently, architectures without any recurrent structures (Gehring et al., 2017; Vaswani et al., 2017) have been proposed and shown to speed up training while achieving state-of-the-art performance.”
BackgroundHuman-level concept learning through probabilistic program induction
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015) · Science · doi:10.1126/science.aab3050 · Cited by 2,817
Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
Johnson, M., Schuster, M., Le, Q. V., Krikun, M., Wu, Y., Chen, Z., Thorat, N., Viégas, F., Wattenberg, M., Corrado, G., et al. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00065 · Cited by 1,747
(2016) · doi:10.18653/v1/d16-1147 · Cited by 1,035
Statistical Phrase-Based Translation
Koehn, P., Och, F. J., & Marcu, D. (2003) · doi:10.21236/ada461156 · Cited by 965
Word translation without parallel data
Lample, G., Conneau, A., Ranzato, M., Denoyer, L., & Jeǵou, H. (2018) · International Conference on Learning Representations · Cited by 653
Dual Learning for Machine Translation
Xia, Y., He, D., Qin, T., Wang, L., Yu, N., Liu, T., & Ma, W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.00179 · Cited by 636
Unsupervised Machine Translation Using Monolingual Corpora Only
Lample, G., Conneau, A., Denoyer, L., & Ranzato, M. (2017) · arXiv · doi:10.48550/arxiv.1711.00043 · Cited by 584
Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism
Firat, O., Cho, K., & Bengio, Y. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1101 · Cited by 562
Self-citationOn Using Monolingual Corpora in Neural Machine Translation
Gülçehre, Ç., Fırat, O., Xu, K., Cho, K., Barrault, L., Lin, H.-C., Bougares, F., Schwenk, H., & Bengio, Y. (2015) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.48550/arxiv.1503.03535 · Cited by 466
Self-citationLearning bilingual word embeddings with (almost) no bilingual data
Artetxe, M., Labaka, G., & Agirre, E. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1042 · Cited by 455
Non-Autoregressive Neural Machine Translation
Gu, J., Bradbury, J., Xiong, C., Li, V. O. K., & Socher, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.02281 · Cited by 454
Self-citationFully Character-Level Neural Machine Translation without Explicit Segmentation
Lee, J., Cho, K., & Hofmann, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00067 · Cited by 394
Self-citationExploiting Source-side Monolingual Data in Neural Machine Translation
Zhang, J., & Zong, C. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1160 · Cited by 277
Zero-Resource Translation with Multi-Lingual Neural Machine Translation
Firat, O., Sankaran, B., Al-Onaizan, Y., Yarman Vural, F. T., & Cho, K. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1026 · Cited by 261
Self-citationOffline bilingual word vectors, orthogonal transformations and the inverted softmax
Smith, S. L., Turban, D. H. P., Hamblin, S., & Hammerla, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.03859 · Cited by 238
Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder
Ha, T.-L., Niehues, J., & Waibel, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.04798 · Cited by 236
Universal Neural Machine Translation for Extremely Low Resource Languages
Gu, J., Hassan, H., Devlin, J., & Li, V. O. K. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1032 · Cited by 228
Self-citationA Teacher-Student Framework for Zero-Resource Neural Machine Translation
Chen, Y., Liu, Y., Cheng, Y., & Li, V. O. K. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1176 · Cited by 171
Self-citationEarth Mover's Distance Minimization for Unsupervised Bilingual Lexicon Induction
Zhang, M., Liu, Y., Luan, H., & Sun, M. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1207 · Cited by 170
Unsupervised Neural Machine Translation with Weight Sharing
Yang, Z. C., Chen, W., Wang, F., & Xu, B. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1005 · Cited by 139
Dynamic Neural Turing Machine with Continuous and Discrete Addressing Schemes
Gülçehre, Ç., Chandar, S., Cho, K., & Bengio, Y. (2018) · Neural Computation · doi:10.1162/neco_a_01060 · Cited by 111
Self-citationHa, D., Dai, A., & Le, Q. V. (2016) · arXiv · doi:10.48550/arxiv.1609.09106 · Cited by 100
Meta-Learning with Temporal Convolutions.Preprint
Mishra, N., Rohaninejad, M., Chen, X., & Abbeel, P. (2017) · arXiv (Cornell University) · Cited by 71
Emergent Translation in Multi-Agent CommunicationPreprint
Lee, J., Cho, K., Weston, J., & Kiela, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.06922 · Cited by 66
Self-citationZero-Resource Neural Machine Translation with Multi-Agent Communication Game
Chen, Y., Liu, Y., & Li, V. O. K. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11976 · Cited by 40
Self-citation
Cited by
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Meta-Learning in Neural Networks: A Survey
Hospedales, T., Antoniou, A., Micaelli, P., & Storkey, A. (2020) · arXiv · doi:10.48550/arxiv.2004.05439 · Cited by 886
“Many more tasks have since been tackled, including text classification [135], neural program induction [263] and synthesis [264], English to SQL program synthesis [265], text-based relationship graph extractor [266], machine translation [267], and quickly adapting to new personas in dialogue”
Meta-Learning in Neural Networks: A Survey
Hospedales, T. M., Antoniou, A., Micaelli, P., & Storkey, A. J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3079209 · Cited by 886
Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems
Wu, C.-S., Madotto, A., Hosseini-Asl, E., Xiong, C., Socher, R., & Fung, P. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1078 · Cited by 415
Advances and Challenges in Meta-Learning: A Technical Review
Vettoruzzo, A., Bouguelia, M.-R., Vanschoren, J., Rögnvaldsson, T., & Santosh, K. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3357847 · Cited by 314
Metadata
- Authors
- Gu, J., Wang, Y., Chen, Y., Li, V. O. K., & Cho, K.
- Venue
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Published
- 2018
- Pages
- 3622-3631
- OpenAlex
- W2888541716
- Cited by
- 325
- References
- 37
- FWCI
- 20.91
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.