The DOI matches its Crossref record; 230 of the 230 references it cites were found in a registry or an index.
A Survey on Multi-Task Learning
Open accessAbstract
Multi-Task Learning (MTL) is a learning paradigm in machine learning and its aim is to leverage useful information contained in multiple related tasks to help improve the generalization performance of all the tasks. In this paper, we give a survey for MTL from the perspective of algorithmic modeling, applications and theoretical analyses. For algorithmic modeling, we give a definition of MTL and then classify different MTL algorithms into five categories, including feature learning approach, low-rank approach, task clustering approach, task relation learning approach and decomposition approach as well as discussing the characteristics of each approach. In order to improve the performance of learning tasks further, MTL can be combined with other learning paradigms including semi-supervised learning, active learning, unsupervised learning, reinforcement learning, multi-view learning and graphical models. When the number of tasks is large or the data dimensionality is high, we review online, parallel and distributed MTL models as well as dimensionality reduction and feature hashing to reveal their computational and storage advantages. Many real-world applications use MTL to boost their performance and we review representative works in this paper. Finally, we present theoretical analyses and discuss several future directions for MTL.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–230 of 230
Attentive Multi-task Deep Reinforcement Learning
Bräm, T., Brunner, G., Richter, O., & Wattenhofer, R. P. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-46133-1_9 · Cited by 22
Multi-task learning for sequential data via iHMMs and the nested Dirichlet process
Ni, K., Carin, L., & Dunson, D. B. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273583 · Cited by 21
Multi-task Gaussian Process Learning of Robot Inverse Dynamics
Chai, K. M., Klanke, S., Williams, C., & Vijayakumar, S. (2008) · ERA · Cited by 21
Robust Online Multi-Task Learning with Correlative and Personalized Structures
Yang, P., Zhao, P., & Gao, X. (2017) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2017.2703106 · Cited by 20
GSpartan: a Geospatio-Temporal Multi-task Learning Framework for Multi-location Prediction
Xu, J., Tan, P., Luo, L., & Zhou, J. (2016) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611974348.74 · Cited by 19
Multi-Task Clustering with Model Relation Learning
Zhang, X., Zhang, X., Liu, H., & Luo, J. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/435 · Cited by 19
Safe Screening for Multi-Task Feature Learning with Multiple Data MatricesPreprint
Wang, J., & Ye, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.04073 · Cited by 18
Online Multi-Task Learning Using Active Sampling
Sharma, S., & Ravindran, B. (2017) · International Conference on Learning Representations · Cited by 18
Hierarchy Through Composition with Multitask LMDPs
Saxe, A., Earle, A. C., & Rosman, B. (2017) · International Conference on Machine Learning · Cited by 18
Multi-task learning for subspace segmentation
Wang, Y., Wipf, D., Ling, Q. H., Chen, W. R., & Wassell, I. J. (2015) · Apollo (University of Cambridge) · Cited by 17
Local Rademacher Complexity-based Learning Guarantees for Multi-Task LearningPreprint
Yousefi, N., Lei, Y., Kloft, M., Mollaghasemi, M., & Anagnostopoulos, G. C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.05916 · Cited by 17
Smart Multi-Task Bregman Clustering and Multi-Task Kernel Clustering
Zhang, X., & Zhang, X. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v27i1.8557 · Cited by 16
Learning Inverse Dynamics by Gaussian process Begrression under the Multi-Task Learning Framework
Yeung, D., & Zhang, Y. (2008) · Springer eBooks · doi:10.1007/978-0-387-85774-9_8 · Cited by 16
Self-citationScalable Multitask Policy Gradient Reinforcement Learning
Bsat, S. E., Ammar, H. B., & Taylor, M. E. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10942 · Cited by 16
Sharing Experience in Multitask Reinforcement Learning
Vuong, T.-L., Nguyen, D. V., Nguyen, T.-L., Bui, C.-M., Kieu, H.-D., Ta, V. C., Tran, Q.-L., & Le, T. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/505 · Cited by 16
Multitask Soft Option LearningPreprint
Igl, M., Gambardella, A., He, J., Nardelli, N., Siddharth, N., Böhmer, W., & Whiteson, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.01033 · Cited by 16
Cross-Domain Multitask Learning with Latent Probit ModelsPreprint
Han, S., Liao, X., & Carin, L. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6419 · Cited by 15
Web-scale multi-task feature selection for behavioral targeting
Ahmed, A. A., Aly, M., Das, A., Smola, A. J., & Anastasakos, T. (2012) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2396761.2398508 · Cited by 13
Revisiting Stein's paradox: multi-task averaging
Feldman, S., Gupta, M., & Frigyik, A. B. (2014) · Journal of machine learning research · doi:10.5555/2627435.2750355 · Cited by 13
Multitask Matrix Completion for Learning Protein Interactions Across Diseases
Kshirsagar, M., Murugesan, K., Carbonell, J., & Klein-Seetharaman, J. (2016) · Annual International Conference on Research in Computational Molecular Biology · doi:10.1089/cmb.2016.0201 · Cited by 13
Multi-Task Learning for Analyzing and Sorting Large Databases of Sequential Data
Ni, K., Paisley, J. W., Carin, L., & Dunson, D. B. (2008) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2008.924798 · Cited by 12
Deep Multimodality Model for Multi-task Multi-view Learning
Zheng, L., Cheng, Y., & He, J. (2019) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611975673.2 · Cited by 12
Interactive Multi-task Relationship Learning
Lin, K., & Zhou, J. (2016) · Proceedings · doi:10.1109/icdm.2016.0035 · Cited by 11
Online Multitask Relative Similarity Learning
Hao, S., Zhao, P., Liu, Y., Hoi, S. C. H., & Miao, C. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/253 · Cited by 11
Active learning for semi-supervised multi-task learning
Li, H., Liao, X., & Carin, L. (2009) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2009.4959914 · Cited by 9
Confidence Weighted Multitask Learning
Yang, P., Zhao, P., Zhou, J., & Gao, X. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33015636 · Cited by 8
Learning multiple visual tasks while discovering their structure
Ciliberto, C., Rosasco, L., & Villa, S. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298608 · Cited by 7
Regularized Multitask Learning for Multidimensional Log-Density Gradient Estimation
Yamane, I., Sasaki, H., & Sugiyama, M. (2016) · Neural Computation · doi:10.1162/neco_a_00844 · Cited by 6
Distributed Multitask Reinforcement Learning with Quadratic Convergence
Tutunov, R., Kim, D., & Ammar, H. B. (2018) · neural information processing systems · Cited by 6
Nonparametric Bayesian feature selection for multi-task learning
Li, H., Liao, X., & Carin, L. (2011) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2011.5946926 · Cited by 4
Research works printed but not linked: 34
- gong, 2013, Multi-stage multi-task feature learning, J Mach Learn Res, 14, 2979
- 10.1007/978-1-4419-9011-2 · DOI
- smith, 2017, Federated multi-task learning, Proc Int Conf Neural Inf Process, 4427
- wang, 2016, Distributed multi-task learning, Proc Int Conf Artif Intell Statist, 751
- cavallanti, 2010, Linear algorithms for online multitask classification, J Mach Learn Res, 11, 2901
- liu, 2009, Multi-task feature learning via efficient $l_{2,1}$l2,1-norm minimization, Proc 25th Conf Uncertainty Artif Intell, 339
- rao, 2013, Sparse overlapping sets lasso for multitask learning and its application to fMRI analysis, Proc Int Conf Neural Inf Process, 2202
- omidshafiei, 2017, Deep decentralized multi-task multi-agent reinforcement learning under partial observability, Proc Int Conf Mach Learn, 2681
- rusu, 2016, Policy distillation, Proc Int Conf Learn Representations
- parisotto, 2016, Actor-mimic: Deep multitask and transfer reinforcement learning, Proc Int Conf Learn Representations
- yu, 2007, Robust multi-task learning with t-processes, Proc 24th Int Conf Mach Learn, 1103
- sharma, 2018, Learning to multi-task by active sampling, Proc Int Conf Learn Representations
- espeholt, 2018, IMPALA: Scalable distributed deep-RL with importance weighted actor-learner architectures, Proc Int Conf Mach Learn, 1406
- mcdonald, 2014, Spectral $k$k-support norm regularization, Proc 27th Int Conf Neural Inf Process Syst, 3644
- yang, 2017, Trace norm regularised deep multi-task learning, Proc 5th Int Conf Learn Representation Workshop Track
- görnitz, 2011, Hierarchical multitask structured output learning for large-scale sequence segmentation, Proc 24th Int Conf Neural Inf Process Syst, 2690
- zhang, 2010, Multi-task learning using generalized $t$t process, Proc 13th Int Conf Artif Intell Statist, 964
- yang, 2017, Deep multi-task representation learning: A tensor factorisation approach, Proc Int Conf Mach Representations
- zhou, 2011, Clustered multi-task learning via alternating structure optimization, Proc Int Conf Neural Inf Process
- jawanpuria, 2015, Efficient output kernel learning for multiple tasks, Proc Int Conf Neural Process Syst, 1189
- luong, 2016, Multi-task sequence to sequence learning, Proc Int Conf Learn Representations
- wang, 2009, Boosted multi-task learning for face verification with applications to web image and video search, Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit, 142
- pentina, 2017, Multi-task learning with labeled and unlabeled tasks, Proc Int Conf Mach Learn, 2807
- zhang, 2016, Self-adapted multi-task clustering, Proc Int Joint Conf Artif Intell, 2357
- andreas, 2017, Modular multitask reinforcement learning with policy sketches, Proc Int Conf Mach Learn, 166
- zhang, 2018, Learning to multitask, Proc 32nd Int Conf Neural Inf Process Syst, 5776
- jawanpuria, 2012, A convex feature learning formulation for latent task structure discovery, Proc 29th Int Conf Mach Learn, 1531
- maurer, 2016, The benefit of multitask representation learning, J Mach Learn Res, 17, 1
- yu, 2020, Gradient surgery for multi-task learning, Proc Int Conf Neural Inf Process
- sener, 2018, Multi-task learning as multi-objective optimization, Proc Int Conf Neural Inf Process, 525
- liu, 2007, Semi-supervised multitask learning, Proc Int Conf Neural Inf Process, 937
- argyriou, 2009, When is there a representer theorem? Vector versus matrix regularizers, J Mach Learn Res, 10, 2507
- argyriou, 2010, On spectral learning, J Mach Learn Res, 11, 935
- kolar, 2011, Union support recovery in multi-task learning, J Mach Learn Res, 12, 2415
Cited by
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772
UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation
Zhou, Z., Siddiquee, M. M. R., Tajbakhsh, N., & Liang, J. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2959609 · Cited by 4,402
A Survey on Multi-Task Learning
Zhang, Y., & Yang, Q. (2017) · arXiv · doi:10.48550/arxiv.1707.08114 · Cited by 2,910
Self-citationA Continual Learning Survey: Defying Forgetting in Classification Tasks
Delange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., & Tuytelaars, T. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3057446 · Cited by 1,647
A continual learning survey: Defying forgetting in classification tasks
De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., & Tuytelaars, T. (2019) · arXiv · doi:10.48550/arxiv.1909.08383 · Cited by 1,647
Metadata
- Authors
- Zhang, Y., & Yang, Q.
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2022-12-01
- Volume
- 34
- Issue
- 12
- Pages
- 5586-5609
- ISSN
- 1041-4347 (ISSN-L)
- 1558-2191
- 2326-3865
- OpenAlex
- W3141797743
- Cited by
- 2,878
- References
- 230
- FWCI
- 131.00
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- Funders
- NSFC (62076118)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.