Latent Multi-Task Architecture Learning
Open accessAbstract
Multi-task learning (MTL) allows deep neural networks to learn from related tasks by sharing parameters with other networks. In practice, however, MTL involves searching an enormous space of possible parameter sharing architectures to find (a) the layers or subspaces that benefit from sharing, (b) the appropriate amount of sharing, and (c) the appropriate relative weights of the different task losses. Recent work has addressed each of the above problems in isolation. In this work we present an approach that learns a latent multi-task architecture that jointly addresses (a)–(c). We present experiments on synthetic data and data from OntoNotes 5.0, including four different tasks and seven different domains. Our extension consistently outperforms previous approaches to learning latent architectures for multi-task problems and achieves up to 15% average error reductions over common approaches to MTL.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Multitask learning has become an increasingly popular topic in the deep learning community.
“More recently, multitask learning (MTL) [108, 109] has become an increasingly popular topic in the deep learning community.”
From A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets (2022) · cites this paper as [109]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cited by
Sorted by citations
AdapterFusion: Non-Destructive Task Composition for Transfer Learning
Pfeiffer, J., Kamath, A., Rücklé, A., Cho, K., & Gurevych, I. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.39 · Cited by 636 · see citing works
“In this section, we formalize our goal of transfer learning (Pan and Yang, 2010; Torrey and Shavlik, 2010; Ruder, 2019), highlight its key challenges, and provide a brief overview of common methods that can be used to address them.”
BackgroundBayoudh, K., Knani, R., Hamdaoui, F., & Mtibaa, A. (2022) · The Visual Computer · doi:10.1007/s00371-021-02166-7 · Cited by 456 · see citing works
Yao, J., & Kowal, J. (2025) · Energy and AI · doi:10.1016/j.egyai.2025.100585 · Cited by 30 · see citing works
- Authors
- Ruder, S., Bingel, J., Augenstein, I., & Søgaard, A.
- Affiliations
- National University of Ireland (IE)
- University of Copenhagen (DK)
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2019-07-17
- Volume
- 33
- Issue
- 01
- Pages
- 4822-4829
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2900964459
- Cited by
- 14
- References
- 0
- Citations by year
- 2025: 2 · 2024: 1 · 2023: 1 · 2022: 1 · 2021: 1 · 2020: 2 · 2019: 6
- FWCI
- 0.96
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- www.aaai.org/ (unspecified)
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Research at the University of Copenhagen (University of Copenhagen) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Science Foundation Ireland (SFI/12/RC/, SFI/12/RC/2289., 12/RC/, 12/RC/2289)
- Irish Research Council
- European Regional Development Fund (SFI/12/RC/2289)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- multi-task learning, parameter sharing, joint optimization, OntoNotes 5.0, natural language processing tasks, domain adaptation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
