Adaptive Knowledge Sharing in Multi-Task Learning: Improving Low-Resource Neural Machine Translation
Open accessThe DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
Neural Machine Translation (NMT) is notorious for its need for large amounts of bilingual data.An effective approach to compensate for this requirement is Multi-Task Learning (MTL) to leverage different linguistic resources as a source of inductive bias.Current MTL architectures are based on the SEQ2SEQ transduction, and (partially) share different components of the models among the tasks.However, this MTL approach often suffers from task interference, and is not able to fully capture commonalities among subsets of tasks.We address this issue by extending the recurrent units with multiple blocks along with a trainable routing network.The routing network enables adaptive collaboration by dynamic sharing of blocks conditioned on the task at hand, input, and model state.Empirical evaluation of two low-resource translation tasks, English to Vietnamese and Farsi, show +1 BLEU score improvements compared to strong baselines.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Moses: open source toolkit for statistical machine translation
Koehn, P., Zens, R., Dyer, C., Bojar, O., Constantin, A., Herbst, E., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., et al. (2007) · Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions - ACL '07 · doi:10.3115/1557769.1557821 · Cited by 4,735
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., & Dean, J. (2017) · arXiv · doi:10.48550/arxiv.1701.06538 · Cited by 933 · cited 2×
“Recently, (Niehues and Cho, 2017) have made use of several linguistic tasks to improve NMT.”
MethodVinyals, O., Kaiser, L., Koo, T., Petrov, S., Sutskever, I., & Hinton, G. (2014) · arXiv · doi:10.48550/arxiv.1412.7449 · Cited by 464
Cited by
ERNIE: Enhanced Language Representation with Informative Entities
Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., & Liu, Q. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1139 · Cited by 1,482
“…next layer. significantly enhance original models, such as reading comprehension (Mihaylov and Frank, 2018; Zhong et al., 2018), machine translation (Zaremoodi et al., 2018), natural language inference (Chen et al., 2018), knowledge acquisition (Han et al., 2018a), and dialog systems (Madotto et…”
BackgroundKEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation
Wang, X., Gao, T., Zhu, Z., Zhang, Z., Liu, Z., Li, J., & Tang, J. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00360 · Cited by 676
“…not only many works integrating knowledge embed-dings into NLP models to improve the performance of NLP applications such as machine translation (Zaremoodi et al., 2018), reading comprehension (Mihaylov and Frank, 2018; Zhong et al., 2019) and dialogue system (Madotto et al., 2018), but also…”
BackgroundMassively Multilingual Neural Machine Translation
Aharoni, R., Johnson, M., & Firat, O. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1388 · Cited by 402
“Zaremoodi et al. (2018) utilize recurrent units with multiple blocks together with a trainable routing network.”
Neural Machine Translation for Low-resource Languages: A Survey
Ranathunga, S., Lee, E.-S. A., Prifti Skenduli, M., Shekhar, R., Alam, M., & Kaur, R. (2023) · ACM Computing Surveys · doi:10.1145/3567592 · Cited by 313
A Survey of Multilingual Neural Machine Translation
Dabre, R., Chu, C., & Kunchukuttan, A. (2021) · ACM Computing Surveys · doi:10.1145/3406095 · Cited by 294
Metadata
- Authors
- Zaremoodi, P., Buntine, W., & Haffari, G.
- Venue
- Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
- Published
- 2018
- Pages
- 656-661
- OpenAlex
- W2798465082
- Cited by
- 53
- References
- 4
- FWCI
- 3.75
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Adaptive Knowledge Sharing in Multi-Task Learning: Improving Low-Resource Neural Machine Translation. https://app.cochat.ai/papers/10.18653/v1/p18-2104
- The paper
- Zaremoodi, P., Buntine, W., & Haffari, G. (2018). Adaptive Knowledge Sharing in Multi-Task Learning: Improving Low-Resource Neural Machine Translation. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). https://doi.org/10.18653/v1/p18-2104
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.