Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 2,878

The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.

A Survey on Multi-Task Learning

Open access
Zhang, Yu, Yang, Qiang·2022·IEEE Transactions on Knowledge and Data Engineering
DOI 10.1109/tkde.2021.3070203
Open this paper in CoChat — free account

Abstract

Multi-Task Learning (MTL) is a learning paradigm in machine learning and its aim is to leverage useful information contained in multiple related tasks to help improve the generalization performance of all the tasks. In this paper, we give a survey for MTL from the perspective of algorithmic modeling, applications and theoretical analyses. For algorithmic modeling, we give a definition of MTL and then classify different MTL algorithms into five categories, including feature learning approach, low-rank approach, task clustering approach, task relation learning approach and decomposition approach as well as discussing the characteristics of each approach. In order to improve the performance of learning tasks further, MTL can be combined with other learning paradigms including semi-supervised learning, active learning, unsupervised learning, reinforcement learning, multi-view learning and graphical models. When the number of tasks is large or the data dimensionality is high, we review online, parallel and distributed MTL models as well as dimensionality reduction and feature hashing to reveal their computational and storage advantages. Many real-world applications use MTL to boost their performance and we review representative works in this paper. Finally, we present theoretical analyses and discuss several future directions for MTL.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tkde.2021.3070203

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 5 references

Research works printed but not linked: 258

  1. 10.1137/1.9781611975673.2 · DOI
  2. honorio, 2010, Multi-task learning of Gaussian graphical models, Proc Int Conf Mach Learn, 447
  3. niculescu-mizil, 2012, Inductive transfer for Bayesian network structure learning, Proc Mach Learn, 167
  4. 10.1145/2939672.2939834 · DOI
  5. oyen, 2012, Leveraging domain knowledge in multitask Bayesian network structure learning, Proc AAAI Conf Artif Intell, 1091
  6. 10.1007/11776420_34 · DOI
  7. 10.1109/ICDM.2015.130 · DOI
  8. lugosi, 2009, Online multi-task learning with hard constraints, Proc 22nd Conf Learn Theory
  9. dekel, 2007, Online learning of multiple tasks with a shared loss, J Mach Learn Res, 8, 2233
  10. 10.1109/TKDE.2016.2603983 · DOI
  11. 10.1109/TCYB.2015.2403356 · DOI · Paper page
  12. hernández-lobato, 2015, A probabilistic model for dirty multi-task feature selection, Proc Int Conf Mach Learn, 1073
  13. hernández-lobato, 2013, Learning feature selection dependencies in multi-task learning, Proc Int Conf Neural Inf Process, 746
  14. 10.1609/aaai.v28i1.9009 · DOI
  15. wang, 2014, On multiplicative multitask feature learning, Proc Int Conf Neural Inf Process, 2411
  16. lozano, 2012, Multi-level lasso for sparse multi-task regression, Proc 29th Int Conf Mach Learn, 595
  17. gong, 2013, Multi-stage multi-task feature learning, J Mach Learn Res, 14, 2979
  18. zhang, 2010, Probabilistic multi-task feature selection, Proc Int Conf Neural Inf Process, 2559
  19. zhou, 2010, Exclusive lasso for multi-task feature selection, Proc 13th Int Conf Artif Intell Statist, 988
  20. kim, 2010, Tree-guided group lasso for multi-task regression with structured sparsity, Proc 27th Int Conf Mach Learn, 543
  21. 10.1007/978-1-4419-9011-2 · DOI
  22. saha, 2011, Online learning of multiple tasks and their relationships, Proc Int Conf Artif Intell Statist, 643
  23. 10.1109/TPAMI.2008.297 · DOI
  24. 10.1609/aaai.v33i01.33015636 · DOI
  25. 10.24963/ijcai.2017/253 · DOI
  26. 10.1109/TKDE.2017.2703106 · DOI
  27. murugesan, 2016, Adaptive smoothed online multi-task learning, Proc Int Conf Neural Inf Process, 4303
  28. smith, 2017, Federated multi-task learning, Proc Int Conf Neural Inf Process, 4427
  29. 10.1145/3097983.3098152 · DOI
  30. 10.1145/3097983.3098136 · DOI
  31. wang, 2016, Distributed multi-task learning, Proc Int Conf Artif Intell Statist, 751
  32. wang, 2015, Safe screening for multi-task feature learning with multiple data matrices, Proc Int Conf Mach Learn, 1747
  33. 10.1145/2623330.2623641 · DOI
  34. cavallanti, 2010, Linear algorithms for online multitask classification, J Mach Learn Res, 11, 2901
  35. 10.1145/1553374.1553458 · DOI · Paper page
  36. 10.21437/Interspeech.2016-879 · DOI · Paper page
  37. 10.1109/CVPR.2016.433 · DOI · Paper page
  38. 10.18653/v1/P17-1001 · DOI · Paper page
  39. liu, 2009, Multi-task feature learning via efficient $l_{2,1}$l2,1-norm minimization, Proc 25th Conf Uncertainty Artif Intell, 339
  40. rao, 2013, Sparse overlapping sets lasso for multitask learning and its application to fMRI analysis, Proc Int Conf Neural Inf Process, 2202
  41. lee, 2010, Adaptive multi-task lasso: With application to eQTL detection, Proc Int Conf Neural Inf Process, 1306
  42. bakker, 2003, Task clustering and gating for Bayesian multitask learning, J Mach Learn Res, 4, 83
  43. 10.1145/1102351.1102479 · DOI · Paper page
  44. teh, 2017, Distral: Robust multitask reinforcement learning, Proc Int Conf Neural Inf Process, 4499
  45. omidshafiei, 2017, Deep decentralized multi-task multi-agent reinforcement learning under partial observability, Proc Int Conf Mach Learn, 2681
  46. sharma, 2017, Online multi-task learning using active sampling, Proc Int Conf Learn Representations Workshop
  47. bsat, 2017, Scalable multitask policy gradient reinforcement learning, Proc AAAI Conf Artif Intell, 1847
  48. igl, 2020, Multitask soft option learning, Proc Conf Uncertainty Artif Intell, 969
  49. rusu, 2016, Policy distillation, Proc Int Conf Learn Representations
  50. parisotto, 2016, Actor-mimic: Deep multitask and transfer reinforcement learning, Proc Int Conf Learn Representations
  51. deshmukh, 2017, Multi-task learning for contextual bandits, Proc Int Conf Neural Inf Process, 4851
  52. saxe, 2017, Hierarchy through composition with multitask LMDPs, Proc Int Conf Mach Learn, 3017
  53. bräm, 2019, Attentive multi-task deep reinforcement learning, Proc Eur Conf Mach Learn Knowl Discovery Databases, 134
  54. 10.24963/ijcai.2019/505 · DOI
  55. 10.1109/TSP.2008.924798 · DOI
  56. 10.1145/1273496.1273583 · DOI
  57. 10.1145/1390156.1390253 · DOI
  58. 10.1109/ICASSP.2011.5946926 · DOI
  59. 10.1145/1273496.1273630 · DOI
  60. xue, 2007, Multi-task learning for classification with Dirichlet process priors, J Mach Learn Res, 8, 35
  61. lian, 2015, A multitask point process predictive model, Proc Int Conf Mach Learn, 2030
  62. yu, 2007, Robust multi-task learning with t-processes, Proc 24th Int Conf Mach Learn, 1103
  63. ando, 2005, A framework for learning predictive structures from multiple tasks and unlabeled data, J Mach Learn Res, 6, 1817
  64. zhang, 2015, Multi-task multi-view clustering for non-negative data, Proc Int Joint Conf Artif Intell, 4055
  65. 10.1145/2339530.2339617 · DOI
  66. he, 2011, A graph-based framework for multi-task multi-view learning, Proc Int Conf Mach Learn, 25
  67. guo, 2020, Bootstrap latent-predictive representations for multitask reinforcement learning, Proc Int Conf Mach Learn, 3875
  68. 10.1609/aaai.v33i01.33013796 · DOI · Paper page
  69. shu, 2018, Hierarchical and interpretable skill acquisition in multi-task reinforcement learning, Proc Int Conf Learn Representations
  70. d’eramo, 2020, Sharing knowledge in multi-task deep reinforcement learning, Proc Int Conf Learn Representations, 1
  71. lin, 2019, Adaptive auxiliary task weighting for reinforcement learning, Proc Int Conf Neural Inf Process, 4773
  72. 10.1109/TKDE.2013.39 · DOI · Paper page
  73. yang, 2009, Heterogeneous multitask learning with joint sparsity constraints, Proc Int Conf Neural Inf Process, 2151
  74. 10.1093/nsr/nwx105 · DOI · Paper page
  75. liao, 2005, Radial basis function network for multi-task learning, Proc Int Conf Neural Inf Process, 792
  76. tutunov, 2018, Distributed multitask reinforcement learning with quadratic convergence, Proc Int Conf Neural Inf Process, 8921
  77. crammer, 2012, Learning multiple tasks using shared hypotheses, Proc 25th Int Conf Neural Inf Process Syst, 1475
  78. 10.1145/1390156.1390164 · DOI · Paper page
  79. sharma, 2018, Learning to multi-task by active sampling, Proc Int Conf Learn Representations
  80. 10.1007/s10994-008-5088-0 · DOI
  81. espeholt, 2018, IMPALA: Scalable distributed deep-RL with importance weighted actor-learner architectures, Proc Int Conf Mach Learn, 1406
  82. mcdonald, 2014, Spectral $k$k-support norm regularization, Proc 27th Int Conf Neural Inf Process Syst, 3644
  83. 10.1609/aaai.v30i1.10261 · DOI
  84. thrun, 1996, Discovering structure in multiple learning tasks: The TC algorithm, Proc 13th Int Conf Mach Learn, 489
  85. yang, 2017, Trace norm regularised deep multi-task learning, Proc 5th Int Conf Learn Representation Workshop Track
  86. agarwal, 2010, Learning multiple tasks using manifold regularization, Proc Adv Neural Inf Process Syst, 46
  87. 10.1145/1553374.1553392 · DOI · Paper page
  88. 10.1137/090763184 · DOI · Paper page
  89. zhang, 2005, Learning multiple related tasks using latent independent component analysis, Proc Int Conf Neural Inf Process, 1585
  90. feldman, 2014, Revisiting Stein's paradox: Multi-task averaging, J Mach Learn Res, 15, 3621
  91. kato, 2007, Multi-task learning via conic programming, Proc Int Conf Neural Inf Process, 737
  92. evgeniou, 2005, Learning multiple tasks with kernel methods, J Mach Learn Res, 6, 615
  93. parameswaran, 2010, Large margin multi-task metric learning, Proc Int Conf Neural Inf Process, 1867
  94. bonilla, 2007, Multi-task Gaussian process prediction, Proc Int Neural Inf Process Syst, 153
  95. chai, 2009, Generalization errors and learning curves for regression with multi-task Gaussian processes, Proc 22nd Int Conf Neural Inf Process Syst, 279
  96. 10.1162/NECO_a_00844 · DOI
  97. görnitz, 2011, Hierarchical multitask structured output learning for large-scale sequence segmentation, Proc 24th Int Conf Neural Inf Process Syst, 2690
  98. zhang, 2010, Multi-task learning using generalized $t$t process, Proc 13th Int Conf Artif Intell Statist, 964
  99. zhang, 2010, A convex formulation for learning task relationships in multi-task learning, Proc 26th Conf Uncertainty Artif Intell, 733
  100. passos, 2012, Flexible modeling of latent task structures in multitask learning, Proc 29th Int Conf Mach Learn, 1283
  101. kang, 2011, Learning with whom to share in multi-task feature learning, Proc 28th Int Conf Mach Learn, 521
  102. jacob, 2008, Clustered multi-task learning: A convex formulation, Proc 18th Int Conf Artif Intell Statist, 745
  103. 10.1609/aaai.v29i1.9581 · DOI
  104. barzilai, 2015, Convex multi-task learning by clustering, Proc Artif Intell Statist
  105. 10.1109/TPAMI.2015.2452911 · DOI
  106. kumar, 2012, Learning task grouping and overlap in multi-task learning, Proc 29th Int Conf Mach Learn
  107. yang, 2017, Deep multi-task representation learning: A tensor factorisation approach, Proc Int Conf Mach Representations
  108. zhou, 2011, Clustered multi-task learning via alternating structure optimization, Proc Int Conf Neural Inf Process
  109. 10.1145/1014052.1014067 · DOI · Paper page
  110. 10.1145/1645953.1646169 · DOI
  111. ghosn, 1996, Multi-task learning for stock selection, Proc Int Conf Neural Inf Process, 946
  112. 10.1109/CVPR.2013.61 · DOI
  113. 10.1007/s11263-012-0582-z · DOI · Paper page
  114. 10.1109/TCBB.2010.22 · DOI
  115. 10.1109/ICASSP.2015.7178814 · DOI · Paper page
  116. 10.21437/Interspeech.2015-265 · DOI
  117. long, 2017, Learning multiple tasks with multilinear relationship networks, Proc 31st Int Conf Neural Inf Process Syst, 1593
  118. gonçalves, 2016, Multi-task sparse structure learning with Gaussian copula models, J Mach Learn Res, 7, 1
  119. 10.1007/s10994-017-5676-y · DOI
  120. rakitsch, 2013, It is all in the noise: Efficient multi-task Gaussian process inference with structured residuals, Proc 26th Int Conf Neural Inf Process Syst, 1466
  121. 10.1145/1553374.1553516 · DOI · Paper page
  122. rai, 2012, Simultaneously leveraging output and task structures for multiple-output regression, Proc 25th Int Conf Neural Inf Process Syst, 3185
  123. zhang, 2012, Robust visual tracking via multi-task sparse learning, Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit, 2042
  124. zhang, 2013, Learning high-order task relationships in multi-task learning, Proc 23rd Int Joint Conf Artif Intell, 1917
  125. yang, 2013, Multi-task learning with Gaussian matrix generalized inverse Gaussian model, Proc Int Conf Mach Learn, 423
  126. jalali, 2010, A dirty model for multi-task learning, Proc 23rd Int Conf Neural Inf Process Syst, 964
  127. 10.1145/1835804.1835952 · DOI
  128. zhang, 2013, Heterogeneous-neighborhood-based multi-task local learning algorithms, Proc Int Conf Neural Inf Process, 1896
  129. lee, 2016, Asymmetric multi-task learning based on task relatedness and loss, Proc Int Conf Mach Learn, 230
  130. 10.1145/2499907.2499910 · DOI
  131. 10.1007/978-3-642-33460-3_50 · DOI
  132. ciliberto, 2015, Convex learning of multiple tasks and their structure, Proc 32nd Int Conf Mach Learn, 1548
  133. dinuzzo, 2011, Learning output kernels with block coordinate descent, Proc 28th Int Conf Mach Learn, 49
  134. 10.1145/2538028 · DOI
  135. archambeau, 2011, Sparse Bayesian multi-task learning, Proc Int Conf Neural Inf Process, 1755
  136. 10.1109/CVPR.2015.7298608 · DOI
  137. jawanpuria, 2015, Efficient output kernel learning for multiple tasks, Proc Int Conf Neural Process Syst, 1189
  138. 10.1609/aaai.v31i1.10820 · DOI
  139. zhang, 2010, Learning multiple tasks with a sparse matrix-normal penalty, Proc 23rd Int Conf Neural Inf Process Syst, 2550
  140. 10.1093/bioinformatics/btq394 · DOI
  141. 10.1145/2339530.2339672 · DOI · Paper page
  142. 10.1145/2020408.2020423 · DOI · Paper page
  143. 10.1109/ICCV.2015.383 · DOI
  144. zheng, 2008, Transferring multi-device localization models using latent multi-task learning, Proc AAAI Conf Artif Intell, 1427
  145. 10.1109/ICASSP.2015.7178780 · DOI
  146. 10.21437/Interspeech.2015-659 · DOI
  147. luong, 2016, Multi-task sequence to sequence learning, Proc Int Conf Learn Representations
  148. yim, 2015, Rotating your face using multi-task deep neural network, Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit, 676
  149. 10.1109/CVPR.2014.186 · DOI
  150. 10.1109/TMM.2015.2477680 · DOI · Paper page
  151. 10.1109/CVPR.2010.5539967 · DOI · Paper page
  152. wang, 2009, Boosted multi-task learning for face verification with applications to web image and video search, Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit, 142
  153. 10.1186/1471-2105-11-181 · DOI
  154. 10.1145/2396761.2398508 · DOI
  155. wang, 2011, Sparse multi-task regression and feature selection to identify brain imaging predictors for memory performance, Proc IEEE Int Conf Comput Vis, 557
  156. 10.1093/bioinformatics/btq191 · DOI
  157. 10.1145/2020408.2020549 · DOI · Paper page
  158. wan, 2012, Sparse Bayesian multi-task learning for predicting cognitive outcomes from neuroimaging measures in Alzheimer's disease, Proc IEEE Conf Comput Vis Pattern Recognit, 940
  159. 10.1093/bioinformatics/btw249 · DOI
  160. 10.1093/bioinformatics/btq181 · DOI
  161. 10.1109/ICCV.2011.6126528 · DOI · Paper page
  162. 10.1109/TIP.2011.2169274 · DOI
  163. 10.1137/1.9781611974348.74 · DOI
  164. 10.1109/TPAMI.2016.2537337 · DOI · Paper page
  165. 10.1109/ICCV.2015.430 · DOI
  166. 10.1137/1.9781611974010.56 · DOI
  167. 10.1109/CVPR.2010.5539975 · DOI · Paper page
  168. alamgir, 2010, Multitask learning for brain-computer interfaces, Mach Learn Res, 17
  169. 10.1145/1390156.1390159 · DOI
  170. wang, 2012, High-order multi-task feature learning to identify longitudinal phenotypic markers for Alzheimer's disease progression prediction, Proc Adv Neural Inf Process Syst, 1277
  171. 10.1137/1.9781611974010.57 · DOI
  172. 10.1137/1.9781611973440.22 · DOI
  173. reichart, 2008, Multi-task active learning for linguistic annotations, Proc Annual Meeting of the Assoc Computational Linguistics, 861
  174. 10.1007/978-3-642-04174-7_40 · DOI
  175. 10.1109/ICDM.2016.0035 · DOI
  176. li, 2009, Active learning for semi-supervised multi-task learning, Proc IEEE Int Conf Acoust Speech Signal Process, 1637
  177. pentina, 2017, Multi-task learning with labeled and unlabeled tasks, Proc Int Conf Mach Learn, 2807
  178. 10.1145/2747879 · DOI
  179. 10.1609/aaai.v25i1.7914 · DOI
  180. 10.1609/aaai.v24i1.7674 · DOI
  181. 10.1609/aaai.v27i1.8557 · DOI
  182. 10.1007/978-3-642-12683-3_34 · DOI
  183. zhang, 2010, Multi-domain collaborative filtering, Proc Conf Uncertainty Artif Intell, 725
  184. 10.1109/ICDM.2015.68 · DOI
  185. 10.1145/1835804.1835953 · DOI
  186. 10.1109/TITS.2016.2598356 · DOI
  187. 10.1609/aaai.v27i1.8577 · DOI
  188. 10.1007/978-0-387-85774-9_8 · DOI
  189. chai, 2008, Multi-task Gaussian process learning of robot inverse dynamics, Proc Int Conf Neural Inf Process, 265
  190. 10.1145/2556195.2556264 · DOI
  191. 10.1186/1471-2105-11-S8-S5 · DOI
  192. wang, 2015, Multi-task learning for subspace segmentation, Proc Int Conf Mach Learn, 1209
  193. 10.1109/TPAMI.2014.2343221 · DOI
  194. 10.1109/TCYB.2014.2344015 · DOI
  195. zhang, 2016, Self-adapted multi-task clustering, Proc Int Joint Conf Artif Intell, 2357
  196. 10.1109/TNNLS.2016.2521602 · DOI · Paper page
  197. 10.24963/ijcai.2018/435 · DOI
  198. 10.1145/1273496.1273624 · DOI · Paper page
  199. li, 2009, Multi-task reinforcement learning in partially observable stochastic environments, J Mach Learn Res, 10, 1131
  200. lazaric, 2010, Bayesian multi-task reinforcement learning, Proc Int Conf Mach Learn, 599

200 of 258 shown

Cited by

See all 2,878

Metadata

Authors
Zhang, Y., & Yang, Q.
Venue
IEEE Transactions on Knowledge and Data Engineering
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2022-12-01
Volume
34
Issue
12
Pages
5586-5609
ISSN
  • 1041-4347 (ISSN-L)
  • 1558-2191
  • 2326-3865
DOI
10.1109/tkde.2021.3070203
OpenAlex
W3141797743
Cited by
2,878
References
5
FWCI
131.00
Type
journal article
Language
en
Open access
green
Licence
Funders
NSFC (62076118)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A Survey on Multi-Task Learning. https://app.cochat.ai/papers/10.1109/tkde.2021.3070203
The paper
Zhang, Y., & Yang, Q. (2022). A Survey on Multi-Task Learning. IEEE Transactions on Knowledge and Data Engineering. https://doi.org/10.1109/tkde.2021.3070203

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.