Student or educator? Sign up with your school email and get 6 months of Pro free.
Cited by 2,878

The DOI matches its Crossref record; 230 of the 230 references it cites were found in a registry or an index.

A Survey on Multi-Task Learning

Open access
Zhang, Yu, Yang, Qiang · 2022 · IEEE Transactions on Knowledge and Data Engineering

Abstract

Multi-Task Learning (MTL) is a learning paradigm in machine learning and its aim is to leverage useful information contained in multiple related tasks to help improve the generalization performance of all the tasks. In this paper, we give a survey for MTL from the perspective of algorithmic modeling, applications and theoretical analyses. For algorithmic modeling, we give a definition of MTL and then classify different MTL algorithms into five categories, including feature learning approach, low-rank approach, task clustering approach, task relation learning approach and decomposition approach as well as discussing the characteristics of each approach. In order to improve the performance of learning tasks further, MTL can be combined with other learning paradigms including semi-supervised learning, active learning, unsupervised learning, reinforcement learning, multi-view learning and graphical models. When the number of tasks is large or the data dimensionality is high, we review online, parallel and distributed MTL models as well as dimensionality reduction and feature hashing to reveal their computational and storage advantages. Many real-world applications use MTL to boost their performance and we review representative works in this paper. Finally, we present theoretical analyses and discuss several future directions for MTL.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tkde.2021.3070203

Cites

230 of 230 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 34

  1. gong, 2013, Multi-stage multi-task feature learning, J Mach Learn Res, 14, 2979
  2. 10.1007/978-1-4419-9011-2 · DOI
  3. smith, 2017, Federated multi-task learning, Proc Int Conf Neural Inf Process, 4427
  4. wang, 2016, Distributed multi-task learning, Proc Int Conf Artif Intell Statist, 751
  5. cavallanti, 2010, Linear algorithms for online multitask classification, J Mach Learn Res, 11, 2901
  6. liu, 2009, Multi-task feature learning via efficient $l_{2,1}$l2,1-norm minimization, Proc 25th Conf Uncertainty Artif Intell, 339
  7. rao, 2013, Sparse overlapping sets lasso for multitask learning and its application to fMRI analysis, Proc Int Conf Neural Inf Process, 2202
  8. omidshafiei, 2017, Deep decentralized multi-task multi-agent reinforcement learning under partial observability, Proc Int Conf Mach Learn, 2681
  9. rusu, 2016, Policy distillation, Proc Int Conf Learn Representations
  10. parisotto, 2016, Actor-mimic: Deep multitask and transfer reinforcement learning, Proc Int Conf Learn Representations
  11. yu, 2007, Robust multi-task learning with t-processes, Proc 24th Int Conf Mach Learn, 1103
  12. sharma, 2018, Learning to multi-task by active sampling, Proc Int Conf Learn Representations
  13. espeholt, 2018, IMPALA: Scalable distributed deep-RL with importance weighted actor-learner architectures, Proc Int Conf Mach Learn, 1406
  14. mcdonald, 2014, Spectral $k$k-support norm regularization, Proc 27th Int Conf Neural Inf Process Syst, 3644
  15. yang, 2017, Trace norm regularised deep multi-task learning, Proc 5th Int Conf Learn Representation Workshop Track
  16. görnitz, 2011, Hierarchical multitask structured output learning for large-scale sequence segmentation, Proc 24th Int Conf Neural Inf Process Syst, 2690
  17. zhang, 2010, Multi-task learning using generalized $t$t process, Proc 13th Int Conf Artif Intell Statist, 964
  18. yang, 2017, Deep multi-task representation learning: A tensor factorisation approach, Proc Int Conf Mach Representations
  19. zhou, 2011, Clustered multi-task learning via alternating structure optimization, Proc Int Conf Neural Inf Process
  20. jawanpuria, 2015, Efficient output kernel learning for multiple tasks, Proc Int Conf Neural Process Syst, 1189
  21. luong, 2016, Multi-task sequence to sequence learning, Proc Int Conf Learn Representations
  22. wang, 2009, Boosted multi-task learning for face verification with applications to web image and video search, Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit, 142
  23. pentina, 2017, Multi-task learning with labeled and unlabeled tasks, Proc Int Conf Mach Learn, 2807
  24. zhang, 2016, Self-adapted multi-task clustering, Proc Int Joint Conf Artif Intell, 2357
  25. andreas, 2017, Modular multitask reinforcement learning with policy sketches, Proc Int Conf Mach Learn, 166
  26. zhang, 2018, Learning to multitask, Proc 32nd Int Conf Neural Inf Process Syst, 5776
  27. jawanpuria, 2012, A convex feature learning formulation for latent task structure discovery, Proc 29th Int Conf Mach Learn, 1531
  28. maurer, 2016, The benefit of multitask representation learning, J Mach Learn Res, 17, 1
  29. yu, 2020, Gradient surgery for multi-task learning, Proc Int Conf Neural Inf Process
  30. sener, 2018, Multi-task learning as multi-objective optimization, Proc Int Conf Neural Inf Process, 525
  31. liu, 2007, Semi-supervised multitask learning, Proc Int Conf Neural Inf Process, 937
  32. argyriou, 2009, When is there a representer theorem? Vector versus matrix regularizers, J Mach Learn Res, 10, 2507
  33. argyriou, 2010, On spectral learning, J Mach Learn Res, 11, 935
  34. kolar, 2011, Union support recovery in multi-task learning, J Mach Learn Res, 12, 2415

Cited by

See all 2,878

Metadata

Authors
Zhang, Y., & Yang, Q.
Venue
IEEE Transactions on Knowledge and Data Engineering
Published
2022-12-01
Volume
34
Issue
12
Pages
5586-5609
ISSN
  • 1041-4347 (ISSN-L)
  • 1558-2191
  • 2326-3865
OpenAlex
W3141797743
Cited by
2,878
References
230
FWCI
131.00
Type
journal article
Language
en
Open access
green
Funders
NSFC (62076118)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.