CoChat
Sign Up
PreprintCited by 2,910Domain Adaptation and Few-Shot LearningMachine Learning and AlgorithmsStochastic Gradient Optimization Techniques

A Survey on Multi-Task Learning

Open access
Zhang, Yu, Yang, Qiang·2017·arXiv
DOI 10.48550/arxiv.1707.08114arXiv 1707.08114
Open this paper in CoChat — free accountOpen PDF

Abstract

Multi-Task Learning (MTL) is a learning paradigm in machine learning and its aim is to leverage useful information contained in multiple related tasks to help improve the generalization performance of all the tasks. In this paper, we give a survey for MTL from the perspective of algorithmic modeling, applications and theoretical analyses. For algorithmic modeling, we give a definition of MTL and then classify different MTL algorithms into five categories, including feature learning approach, low-rank approach, task clustering approach, task relation learning approach and decomposition approach as well as discussing the characteristics of each approach. In order to improve the performance of learning tasks further, MTL can be combined with other learning paradigms including semi-supervised learning, active learning, unsupervised learning, reinforcement learning, multi-view learning and graphical models. When the number of tasks is large or the data dimensionality is high, we review online, parallel and distributed MTL models as well as dimensionality reduction and feature hashing to reveal their computational and storage advantages. Many real-world applications use MTL to boost their performance and we review representative works in this paper. Finally, we present theoretical analyses and discuss several future directions for MTL.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1707.08114

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Fully decentralized variants of SGD and other optimization algorithms have recently been considered in machine learning both for improved scalability in datacenters and for decentralized networks of devices.

    “fully decentralized variants of SGD and other optimization algorithms have recently been considered in machine learning both for improved scalability in datacenters [29] as well as for decentralized networks of devices [127, 459, 443, 59, 278, 291, 173].”

    From Advances and Open Problems in Federated Learning (2021) · cites this paper as [127]

    “Handling Big Data” (p. 15) says training data can reside on different devices and describes distributed multi-task learning algorithms, supporting the device-network part of the citation. “MTL with Other Learning Paradigms” (p. 13) also mentions a decentralized solver. Neither passage establishes that fully decentralized variants of SGD are used for those device networks; the datacenter clause is attached to a different citation. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 235

  1. [2][2] Q. Yang, Y. Zhang, W. Dai, and S. J. Pan, Transfer Learning . Cambridge University Press, 2020. -
  2. [3][3] M.-L. Zhang and Z.-H. Zhou, 'A review on multi-label learning algorithms,' IEEE TKDE , 2014. -
  3. [4][4] G. I. Parisi, R. Kemker, J. L. Part, C. Kanan, and S. Wermter, 'Continual lifelong learning with neural networks: A review,' Neural Networks , vol. 113, pp. 54-71, 2019. -
  4. [5][5] Y. Zhang and Q. Yang, 'An overview of multi-task learning,' National Science Review , 2018. -
  5. [6][6] X. Yang, S. Kim, and E. P. Xing, 'Heterogeneous multitask learning with joint sparsity constraints,' in NIPS , 2009. -
  6. [7][7] S. Bickel, J. Bogojeska, T. Lengauer, and T. Scheffer, 'Multi-task learning for HIV therapy screening,' in ICML , 2008. -
  7. [8][8] X. Liao and L. Carin, 'Radial basis function network for multi-task learning,' in NIPS , 2005. -
  8. [11][11] A. Argyriou, C. A. Micchelli, M. Pontil, and Y. Ying, 'A spectral regularization framework for multi-task structure learning,' in NIPS , 2007. -
  9. [12][12] A. Maurer, M. Pontil, and B. Romera-Paredes, 'Sparse coding for multitask and transfer learning,' in ICML , 2013. -
  10. [13][13] J. Zhu, N. Chen, and E. P. Xing, 'Infinite latent SVM for classification and multi-task learning,' in NIPS , 2011. -
  11. [14][14] M. K. Titsias and M. L´ azaro-Gredilla, 'Spike and slab variational inference for multi-task and multiple kernel learning,' in NIPS , 2011. -
  12. [15][15] Z. Zhang, P. Luo, C. C. Loy, and X. Tang, 'Facial landmark detection by deep multi-task learning,' in ECCV , 2014. -
  13. [16][16] W. Liu, T. Mei, Y. Zhang, C. Che, and J. Luo, 'Multi-task deep visualsemantic embedding for video thumbnail selection,' in CVPR , 2015. -
  14. [17][17] W. Zhang, R. Li, T. Zeng, Q. Sun, S. Kumar, J. Ye, and S. Ji, 'Deep model based transfer and multi-task learning for biological image analysis,' in KDD , 2015. -
  15. [18][18] N. Mrksic, D. ´ O. S´ eaghdha, B. Thomson, M. Gasic, P. Su, D. Vandyke, T. Wen, and S. J. Young, 'Multi-domain dialog state tracking using recurrent neural networks,' in ACL , 2015. -
  16. [19][19] S. Li, Z. Liu, and A. B. Chan, 'Heterogeneous multi-task learning for human pose estimation with deep convolutional neural network,' IJCV , 2015. -
  17. [20][20] Y. Shinohara, 'Adversarial multi-task learning of deep neural networks for robust speech recognition,' in Interspeech , 2016. -
  18. [21][21] P. Liu, X. Qiu, and X. Huang, 'Adversarial multi-task learning for text classification,' in ACL , 2017. -
  19. [22][22] I. Misra, A. Shrivastava, A. Gupta, and M. Hebert, 'Cross-stitch networks for multi-task learning,' in CVPR , 2016. -
  20. [24][24] J. Liu, S. Ji, and J. Ye, 'Multi-task feature learning via efficient l 2 , 1 -norm minimization,' in UAI , 2009. -
  21. [25][25] S. Lee, J. Zhu, and E. P. Xing, 'Adaptive multi-task lasso: With application to eQTL detection,' in NIPS , 2010. -
  22. [26][26] N. S. Rao, C. R. Cox, R. D. Nowak, and T. T. Rogers, 'Sparse overlapping sets lasso for multitask learning and its application to fMRI analysis,' in NIPS , 2013. -
  23. [27][27] P. Gong, J. Zhou, W. Fan, and J. Ye, 'Efficient multi-task feature learning with calibration,' in KDD , 2014. -
  24. [28][28] J. Wang and J. Ye, 'Safe screening for multi-task feature learning with multiple data matrices,' in ICML , 2015. -
  25. [29][29] H. Liu, M. Palatucci, and J. Zhang, 'Blockwise coordinate descent procedures for the multi-task lasso, with applications to neural semantic basis discovery,' in ICML , 2009. -
  26. [30][30] P. Gong, J. Ye, and C. Zhang, 'Multi-stage multi-task feature learning,' JMLR , 2013. -
  27. [31][31] A. C. Lozano and G. Swirszcz, 'Multi-level lasso for sparse multi-task regression,' in ICML , 2012. -
  28. [32][32] X. Wang, J. Bi, S. Yu, and J. Sun, 'On multiplicative multitask feature learning,' in NIPS , 2014. -
  29. [33][33] L. Han, Y. Zhang, G. Song, and K. Xie, 'Encoding tree sparsity in multi-task learning: A probabilistic framework,' in AAAI , 2014. -
  30. [34][34] T. Jebara, 'Multi-task feature and kernel selection for SVMs,' in ICML , 2004. -
  31. [35][35] S. Kim and E. P. Xing, 'Tree-guided group lasso for multi-task regression with structured sparsity,' in ICML , 2010. -
  32. [36][36] Y. Zhou, R. Jin, and S. C. H. Hoi, 'Exclusive lasso for multi-task feature selection,' in AISTATS , 2010. -
  33. [37][37] Y. Zhang, D.-Y. Yeung, and Q. Xu, 'Probabilistic multi-task feature selection,' in NIPS , 2010. -
  34. [38][38] D. Hern´ andez-Lobato and J. M. Hern´ andez-Lobato, 'Learning feature selection dependencies in multi-task learning,' in NIPS , 2013. -
  35. [39][39] D. Hern´ andez-Lobato, J. M. Hern´ andez-Lobato, and Z. Ghahramani, 'A probabilistic model for dirty multi-task feature selection,' in ICML , 2015. -
  36. [40][40] R. K. Ando and T. Zhang, 'A framework for learning predictive structures from multiple tasks and unlabeled data,' JMLR , 2005. -
  37. [41][41] J. Chen, L. Tang, J. Liu, and J. Ye, 'A convex formulation for learning shared structures from multiple tasks,' in ICML , 2009. -
  38. [42][42] A. Agarwal, H. Daum´ e III, and S. Gerber, 'Learning multiple tasks using manifold regularization,' in NIPS , 2010. -
  39. [43][43] J. Zhang, Z. Ghahramani, and Y. Yang, 'Learning multiple related tasks using latent independent component analysis,' in NIPS , 2005. -
  40. [44][44] T. K. Pong, P. Tseng, S. Ji, and J. Ye, 'Trace norm regularization: Reformulations, algorithms, and multi-task learning,' SIAM Journal on Optimization , 2010. -
  41. [45][45] L. Han and Y. Zhang, 'Multi-stage multi-task learning with reduced rank,' in AAAI , 2016. -
  42. [46][46] A. M. McDonald, M. Pontil, and D. Stamos, 'Spectral k -support norm regularization,' in NIPS , 2014. -
  43. [47][47] Y. Yang and T. M. Hospedales, 'Trace norm regularised deep multi-task learning,' in ICLR, Workshop Track , 2017. -
  44. [48][48] S. Thrun and J. O'Sullivan, 'Discovering structure in multiple learning tasks: The TC algorithm,' in ICML , 1996. -
  45. [49][49] K. Crammer and Y. Mansour, 'Learning multiple tasks using shared hypotheses,' in NIPS , 2012. -
  46. [50][50] B. Bakker and T. Heskes, 'Task clustering and gating for Bayesian multitask learning,' JMLR , 2003. -
  47. [51][51] K. Yu, V. Tresp, and A. Schwaighofer, 'Learning Gaussian processes from multiple tasks,' in ICML , 2005. -
  48. [52][52] S. Yu, V. Tresp, and K. Yu, 'Robust multi-task learning with t -processes,' in ICML , 2007. -
  49. [53][53] W. Lian, R. Henao, V. Rao, J. E. Lucas, and L. Carin, 'A multitask point process predictive model,' in ICML , 2015. -
  50. [54][54] Y. Xue, X. Liao, L. Carin, and B. Krishnapuram, 'Multi-task learning for classification with Dirichlet process priors,' JMLR , 2007. -
  51. [55][55] Y. Xue, D. B. Dunson, and L. Carin, 'The matrix stick-breaking process for flexible multi-task learning,' in ICML , 2007. -
  52. [56][56] H. Li, X. Liao, and L. Carin, 'Nonparametric Bayesian feature selection for multi-task learning,' in ICASSP , 2011. -
  53. [57][57] Y. Qi, D. Liu, D. B. Dunson, and L. Carin, 'Multi-task compressive sensing with Dirichlet process priors,' in ICML , 2008. -
  54. [58][58] K. Ni, L. Carin, and D. B. Dunson, 'Multi-task learning for sequential data via iHMMs and the nested Dirichlet process,' in ICML , 2007. -
  55. [59][59] K. Ni, J. W. Paisley, L. Carin, and D. B. Dunson, 'Multi-task learning for analyzing and sorting large databases of sequential data,' IEEE TSP , 2008. -
  56. [60][60] A. Passos, P. Rai, J. Wainer, and H. Daum´ e III, 'Flexible modeling of latent task structures in multitask learning,' in ICML , 2012. -
  57. [61][61] L. Jacob, F. R. Bach, and J.-P. Vert, 'Clustered multi-task learning: A convex formulation,' in NIPS , 2008. -
  58. [62][62] Z. Kang, K. Grauman, and F. Sha, 'Learning with whom to share in multi-task feature learning,' in ICML , 2011. -
  59. [63][63] L. Han and Y. Zhang, 'Learning multi-level task groups in multi-task learning,' in AAAI , 2015. -
  60. [64][64] A. Barzilai and K. Crammer, 'Convex multi-task learning by clustering,' in AISTATS , 2015. -
  61. [65][65] Q. Zhou and Q. Zhao, 'Flexible clustered multi-task learning by learning representative tasks,' IEEE TPAMI , 2016. -
  62. [66][66] A. Kumar and H. Daum´ e III, 'Learning task grouping and overlap in multi-task learning,' in ICML , 2012. -
  63. [67][67] Y. Yang and T. M. Hospedales, 'Deep multi-task representation learning: A tensor factorisation approach,' in ICLR , 2017. -
  64. [68][68] J. Zhou, J. Chen, and J. Ye, 'Clustered multi-task learning via alternating structure optimization,' in NIPS , 2011. -
  65. [69][69] T. Evgeniou and M. Pontil, 'Regularized multi-task learning,' in KDD , 2004. -
  66. [70][70] S. Parameswaran and K. Q. Weinberger, 'Large margin multi-task metric learning,' in NIPS , 2010. -
  67. [71][71] T. Evgeniou, C. A. Micchelli, and M. Pontil, 'Learning multiple tasks with kernel methods,' JMLR , 2005. -
  68. [72][72] T. Kato, H. Kashima, M. Sugiyama, and K. Asai, 'Multi-task learning via conic programming,' in NIPS , 2007. -
  69. [73][73] S. Feldman, M. R. Gupta, and B. A. Frigyik, 'Revisiting Stein's paradox: Multi-task averaging,' JMLR , 2014. -
  70. [75][75] N. G¨ ornitz, C. Widmer, G. Zeller, A. Kahles, S. Sonnenburg, and G. R¨ atsch, 'Hierarchical multitask structured output learning for largescale sequence segmentation,' in NIPS , 2011. -
  71. [76][76] E. V. Bonilla, K. M. A. Chai, and C. K. I. Williams, 'Multi-task Gaussian process prediction,' in NIPS , 2007. -
  72. [77][77] K. M. A. Chai, 'Generalization errors and learning curves for regression with multi-task Gaussian processes,' in NIPS , 2009. -
  73. [78][78] Y. Zhang and D.-Y. Yeung, 'Multi-task learning using generalized t process,' in AISTATS , 2010. -
  74. [79][79] Y. Zhang and D.-Y. Yeung, 'A convex formulation for learning task relationships in multi-task learning,' in UAI , 2010. -
  75. [80][80] Y. Zhang and D.-Y. Yeung, 'A regularization approach to learning task relationships in multitask learning,' ACM TKDD , 2014. -
  76. [82][82] Y. Zhang and D.-Y. Yeung, 'Multilabel relationship learning,' ACM TKDD , 2013. -
  77. [83][83] F. Dinuzzo, C. S. Ong, P. V. Gehler, and G. Pillonetto, 'Learning output kernels with block coordinate descent,' in ICML , 2011. -
  78. [84][84] C. Ciliberto, Y. Mroueh, T. A. Poggio, and L. Rosasco, 'Convex learning of multiple tasks and their structure,' in ICML , 2015. -
  79. [85][85] C. Ciliberto, L. Rosasco, and S. Villa, 'Learning multiple visual tasks while discovering their structure,' in CVPR , 2015. -
  80. [86][86] P. Jawanpuria, M. Lapin, M. Hein, and B. Schiele, 'Efficient output kernel learning for multiple tasks,' in NIPS , 2015. -
  81. [87][87] Y. Zhang and Q. Yang, 'Learning sparse task relations in multi-task learning,' in AAAI , 2017. -
  82. [88][88] Y. Zhang and J. G. Schneider, 'Learning multiple tasks with a sparse matrix-normal penalty,' in NIPS , 2010. -
  83. [89][89] C. Archambeau, S. Guo, and O. Zoeter, 'Sparse Bayesian multi-task learning,' in NIPS , 2011. -
  84. [90][90] M. Yang, Y. Li, and Z. Zhang, 'Multi-task learning with Gaussian matrix generalized inverse Gaussian model,' in ICML , 2013. -
  85. [91][91] Y. Zhang and D.-Y. Yeung, 'Learning high-order task relationships in multi-task learning,' in IJCAI , 2013. -
  86. [92][92] P. Rai, A. Kumar, and H. Daum´ e III, 'Simultaneously leveraging output and task structures for multiple-output regression,' in NIPS , 2012. -
  87. [93][93] B. Rakitsch, C. Lippert, K. M. Borgwardt, and O. Stegle, 'It is all in the noise: Efficient multi-task Gaussian process inference with structured residuals,' in NIPS , 2013. -
  88. [94][94] A. R. Goncalves, F. J. V. Zuben, and A. Banerjee, 'Multi-task sparse structure learning with Gaussian copula models,' JMLR , 2016. -
  89. [95][95] M. Long, Z. Cao, J. Wang, and P. S. Yu, 'Learning multiple tasks with multilinear relationship networks,' in NIPS , 2017. -
  90. [96][96] Y. Zhang, 'Heterogeneous-neighborhood-based multi-task local learning algorithms,' in NIPS , 2013. -
  91. [97][97] G. Lee, E. Yang, and S. J. Hwang, 'Asymmetric multi-task learning based on task relatedness and loss,' in ICML , 2016. -
  92. [98][98] A. Jalali, P. Ravikumar, S. Sanghavi, and C. Ruan, 'A dirty model for multi-task learning,' in NIPS , 2010. -
  93. [99][99] J. Chen, J. Liu, and J. Ye, 'Learning incoherent sparse and low-rank patterns from multiple tasks,' in KDD , 2010. -
  94. [100][100] J. Chen, J. Zhou, and J. Ye, 'Integrating low-rank and group-sparse structures for robust multi-task learning,' in KDD , 2011. -
  95. [101][101] P. Gong, J. Ye, and C. Zhang, 'Robust multi-task feature learning,' in KDD , 2012. -
  96. [102][102] W. Zhong and J. T. Kwok, 'Convex multitask learning with flexible task clusters,' in ICML , 2012. -
  97. [103][103] A. Zweig and D. Weinshall, 'Hierarchical regularization cascade for joint learning,' in ICML , 2013. -
  98. [104][104] L. Han and Y. Zhang, 'Learning tree structure in multi-task learning,' in KDD , 2015. -
  99. [105][105] P. Jawanpuria and J. S. Nath, 'A convex feature learning formulation for latent task structure discovery,' in ICML , 2012. -
  100. [106][106] B. Gong, Y. Shi, F. Sha, and K. Grauman, 'Geodesic flow kernel for unsupervised domain adaptation,' in CVPR , 2012. -
  101. [107][107] M. Solnon, S. Arlot, and F. R. Bach, 'Multi-task regression using minimal penalties,' JMLR , 2012. -
  102. [108][108] Y. Zhang, Y. Wei, and Q. Yang, 'Learning to multitask,' in NIPS , 2018. -
  103. [109][109] Y. Zhang and D.-Y. Yeung, 'Multi-task learning in heterogeneous feature spaces,' in AAAI , 2011. -
  104. [110][110] S. Han, X. Liao, and L. Carin, 'Cross-domain multitask learning with latent probit models,' in ICML , 2012. -
  105. [112][112] N. Quadrianto, A. J. Smola, T. S. Caetano, S. V. N. Vishwanathan, and J. Petterson, 'Multitask learning without label correspondences,' in NIPS , 2010. -
  106. [113][113] B. Romera-Paredes, H. Aung, N. Bianchi-Berthouze, and M. Pontil, 'Multilinear multitask learning,' in ICML , 2013. -
  107. [114][114] K. Wimalawarne, M. Sugiyama, and R. Tomioka, 'Multitask learning meets tensor factorization: Task imputation via convex optimization,' in NIPS , 2014. -
  108. [115][115] O. Sener and V. Koltun, 'Multi-task learning as multi-objective optimization,' in NeurIPS , 2018. -
  109. [116][116] T. Yu, S. Kumar, A. Gupta, S. Levine, K. Hausman, and C. Finn, 'Gradient surgery for multi-task learning,' in NeurIPS , 2020. -
  110. [117][117] Z. Chen, V. Badrinarayanan, C. Lee, and A. Rabinovich, 'Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks,' in ICML , 2018. -
  111. [118][118] N. Parikh and S. P. Boyd, 'Proximal algorithms,' Foundations and Trends in Optimization , vol. 1, no. 3, pp. 127-239, 2014. -
  112. [119][119] L. Zhao, Q. Sun, J. Ye, F. Chen, C. Lu, and N. Ramakrishnan, 'Multitask learning for spatio-temporal event forecasting,' in KDD , 2015. -
  113. [120][120] Y. Li, J. Wang, J. Ye, and C. K. Reddy, 'A multi-task learning formulation for survival analysis,' in KDD , 2016. -
  114. [121][121] L. Zhao, Q. Sun, J. Ye, F. Chen, C. Lu, and N. Ramakrishnan, 'Feature constrained multi-task learning models for spatiotemporal event forecasting,' IEEE TKDE , 2017. -
  115. [122][122] Q. Liu, X. Liao, and L. Carin, 'Semi-supervised multitask learning,' in NIPS , 2007. -
  116. [123][123] Q. Liu, X. Liao, H. Li, J. R. Stack, and L. Carin, 'Semisupervised multitask learning,' IEEE TPAMI , 2009. -
  117. [125][125] R. Reichart, K. Tomanek, U. Hahn, and A. Rappoport, 'Multi-task active learning for linguistic annotations,' in ACL , 2008. -
  118. [128][128] H. Li, X. Liao, and L. Carin, 'Active learning for semi-supervised multitask learning,' in ICASSP , 2009. -
  119. [129][129] K. Lin and J. Zhou, 'Interactive multi-task relationship learning,' in ICDM , 2016. -
  120. [130][130] A. Pentina and C. H. Lampert, 'Multi-task learning with labeled and unlabeled tasks,' in ICML , 2017. -
  121. [131][131] J. Zhang and C. Zhang, 'Multitask Bregman clustering,' in AAAI , 2010. -
  122. [132][132] X. Zhang and X. Zhang, 'Smart multi-task Bregman clustering and multi-task kernel clustering,' in AAAI , 2013. -
  123. [133][133] X. Zhang, X. Zhang, and H. Liu, 'Smart multitask Bregman clustering and multitask kernel clustering,' ACM TKDD , 2015. -
  124. [134][134] Q. Gu, Z. Li, and J. Han, 'Learning a kernel for multi-task clustering,' in AAAI , 2011. -
  125. [135][135] X. Zhang, 'Convex discriminative multitask clustering,' IEEE TPAMI , 2015. -
  126. [136][136] Y. Wang, D. P. Wipf, Q. Ling, W. Chen, and I. J. Wassell, 'Multi-task learning for subspace segmentation,' in ICML , 2015. -
  127. [137][137] X. Zhang, X. Zhang, and H. Liu, 'Self-adapted multi-task clustering,' in IJCAI , 2016. -
  128. [138][138] Y. Yang, Z. Ma, Y. Yang, F. Nie, and H. T. Shen, 'Multitask spectral clustering by exploring intertask correlation,' IEEE TCYB , 2015. -
  129. [139][139] X. Zhu, X. Li, S. Zhang, C. Ju, and X. Wu, 'Robust joint graph sparse coding for unsupervised spectral feature selection,' IEEE TNNLS , 2017. -
  130. [140][140] X. Zhang, X. Zhang, H. Liu, and J. Luo, 'Multi-task clustering with model relation learning,' in IJCAI , 2018. -
  131. [141][141] A. Wilson, A. Fern, S. Ray, and P. Tadepalli, 'Multi-task reinforcement learning: A hierarchical Bayesian approach,' in ICML , 2007. -
  132. [142][142] H. Li, X. Liao, and L. Carin, 'Multi-task reinforcement learning in partially observable stochastic environments,' JMLR , 2009. -
  133. [143][143] A. Lazaric and M. Ghavamzadeh, 'Bayesian multi-task reinforcement learning,' in ICML , 2010. -
  134. [144][144] D. Calandriello, A. Lazaric, and M. Restelli, 'Sparse multi-task reinforcement learning,' in NIPS , 2014. -
  135. [145][145] J. Andreas, D. Klein, and S. Levine, 'Modular multitask reinforcement learning with policy sketches,' in ICML , 2017. -
  136. [146][146] A. A. Deshmukh, ¨ U. Dogan, and C. Scott, 'Multi-task learning for contextual bandits,' in NIPS , 2017. -
  137. [147][147] A. M. Saxe, A. C. Earle, and B. Rosman, 'Hierarchy through composition with multitask LMDPs,' in ICML , 2017. -
  138. [149][149] T. Vuong, D. V. Nguyen, T. Nguyen, C. Bui, H. Kieu, V. Ta, Q. Tran, and T. H. Le, 'Sharing experience in multitask reinforcement learning,' in IJCAI , 2019. -
  139. [150][150] M. Igl, A. Gambardella, N. Nardelli, N. Siddharth, W. B¨ ohmer, and S. Whiteson, 'Multitask soft option learning,' in UAI , 2020. -
  140. [151][151] E. Parisotto, J. Ba, and R. Salakhutdinov, 'Actor-mimic: Deep multitask and transfer reinforcement learning,' in ICLR , 2016. -
  141. [152][152] A. A. Rusu, S. G. Colmenarejo, C ¸ . G¨ ulc ¸ehre, G. Desjardins, J. Kirkpatrick, R. Pascanu, V. Mnih, K. Kavukcuoglu, and R. Hadsell, 'Policy distillation,' in ICLR , 2016. -
  142. [153][153] S. Omidshafiei, J. Pazis, C. Amato, J. P. How, and J. Vian, 'Deep decentralized multi-task multi-agent reinforcement learning under partial observability,' in ICML , 2017. -
  143. [154][154] Y. W. Teh, V. Bapst, W. M. Czarnecki, J. Quan, J. Kirkpatrick, R. Hadsell, N. Heess, and R. Pascanu, 'Distral: Robust multitask reinforcement learning,' in NIPS , 2017. -
  144. [155][155] S. E. Bsat, H. Bou-Ammar, and M. E. Taylor, 'Scalable multitask policy gradient reinforcement learning,' in AAAI , 2017. -
  145. [156][156] S. Sharma and B. Ravindran, 'Online multi-task learning using active sampling,' in ICLR Workshop , 2017. -
  146. [157][157] S. Sharma, A. K. Jha, P. Hegde, and B. Ravindran, 'Learning to multitask by active sampling,' in ICLR , 2018. -
  147. [158][158] L. Espeholt, H. Soyer, R. Munos, K. Simonyan, V. Mnih, T. Ward, Y. Doron, V. Firoiu, T. Harley, I. Dunning, S. Legg, and K. Kavukcuoglu, 'IMPALA: scalable distributed deep-RL with importance weighted actor-learner architectures,' in ICML , 2018. -
  148. [159][159] R. Tutunov, D. Kim, and H. Bou-Ammar, 'Distributed multitask reinforcement learning with quadratic convergence,' in NeurIPS , 2018. -
  149. [160][160] X. Lin, H. S. Baweja, G. Kantor, and D. Held, 'Adaptive auxiliary task weighting for reinforcement learning,' in NeurIPS , 2019. -
  150. [161][161] C. D'Eramo, D. Tateo, A. Bonarini, M. Restelli, and J. Peters, 'Sharing knowledge in multi-task deep reinforcement learning,' in ICLR , 2020. -
  151. [162][162] T. Shu, C. Xiong, and R. Socher, 'Hierarchical and interpretable skill acquisition in multi-task reinforcement learning,' in ICLR , 2018. -
  152. [163][163] M. Hessel, H. Soyer, L. Espeholt, W. Czarnecki, S. Schmitt, and H. van Hasselt, 'Multi-task deep reinforcement learning with PopArt,' in AAAI , 2019. -
  153. [164][164] Z. Guo, B. A. Pires, M. G. Azar, B. Piot, F. Altch´ e, J.-B. Grill, and R. Munos, 'Bootstrap latent-predictive representations for multitask reinforcement learning,' in ICML , 2020. -
  154. [165][165] J. He and R. Lawrence, 'A graph-based framework for multi-task multiview learning,' in ICML , 2011. -
  155. [166][166] J. Zhang and J. Huan, 'Inductive multi-task learning with multiple view data,' in KDD , 2012. -
  156. [167][167] X. Zhang, X. Zhang, and H. Liu, 'Multi-task multi-view clustering for non-negative data,' in IJCAI , 2015. -
  157. [168][168] X. Zhang, X. Zhang, H. Liu, and X. Liu, 'Multi-task multi-view clustering,' IEEE TKDE , 2016. -
  158. [169][169] X. Zhu, X. Li, and S. Zhang, 'Block-row sparse multiview multilabel learning for image classification,' IEEE TCYB , 2016. -
  159. [171][171] A. Niculescu-Mizil and R. Caruana, 'Inductive transfer for Bayesian network structure learning,' in AISTATS , 2007. -
  160. [172][172] J. Honorio and D. Samaras, 'Multi-task learning of Gaussian graphical models,' in ICML , 2010. -
  161. [173][173] D. Oyen and T. Lane, 'Leveraging domain knowledge in multitask Bayesian network structure learning,' in AAAI , 2012. -
  162. [174][174] K. Lin, J. Xu, I. M. Baytas, S. Ji, and J. Zhou, 'Multi-task feature interaction learning,' in KDD , 2016. -
  163. [175][175] Y. Zhang, 'Parallel multi-task learning,' in ICDM , 2015. -
  164. [176][176] O. Dekel, P. M. Long, and Y. Singer, 'Online multitask learning,' in COLT , 2006. -
  165. [177][177] O. Dekel, P. M. Long, and Y. Singer, 'Online learning of multiple tasks with a shared loss,' JMLR , 2007. -
  166. [178][178] G. Lugosi, O. Papaspiliopoulos, and G. Stoltz, 'Online multi-task learning with hard constraints,' in COLT , 2009. -
  167. [179][179] G. Cavallanti, N. Cesa-Bianchi, and C. Gentile, 'Linear algorithms for online multitask classification,' JMLR , 2010. -
  168. [180][180] G. Pillonetto, F. Dinuzzo, and G. D. Nicolao, 'Bayesian online multitask learning of Gaussian processes,' IEEE TPAMI , 2010. -
  169. [181][181] A. Saha, P. Rai, H. Daum´ e III, and S. Venkatasubramanian, 'Online learning of multiple tasks and their relationships,' in AISTATS , 2011. -
  170. [182][182] K. Murugesan, H. Liu, J. G. Carbonell, and Y. Yang, 'Adaptive smoothed online multi-task learning,' in NIPS , 2016. -
  171. [183][183] P. Yang, P. Zhao, and X. Gao, 'Robust online multi-task learning with correlative and personalized structures,' IEEE TKDE , 2017. -
  172. [184][184] S. Hao, P. Zhao, Y. Liu, S. C. H. Hoi, and C. Miao, 'Online multitask relative similarity learning,' in IJCAI , 2017. -
  173. [185][185] P. Yang, P. Zhao, J. Zhou, and X. Gao, 'Confidence weighted multitask learning,' in AAAI , 2019. -
  174. [186][186] J. Wang, M. Kolar, and N. Srebro, 'Distributed multi-task learning,' in AISTATS , 2016. -
  175. [187][187] S. Liu, S. J. Pan, and Q. Ho, 'Distributed multi-task relationship learning,' in KDD , 2017. -
  176. [188][188] L. Xie, I. M. Baytas, K. Lin, and J. Zhou, 'Privacy-preserving distributed multi-task learning with asynchronous updates,' in KDD , 2017. -
  177. [189][189] V. Smith, C. Chiang, M. Sanjabi, and A. S. Talwalkar, 'Federated multitask learning,' in NIPS , 2017. -
  178. [191][191] K. Q. Weinberger, A. Dasgupta, J. Langford, A. J. Smola, and J. Attenberg, 'Feature hashing for large scale multitask learning,' in ICML , 2009. -
  179. [192][192] T. Zhang, B. Ghanem, S. Liu, and N. Ahuja, 'Robust visual tracking via multi-task sparse learning,' in CVPR , 2012. -
  180. [193][193] T. Zhang, B. Ghanem, S. Liu, and N. Ahuja, 'Robust visual tracking via structured multi-task sparse learning,' IJCV , 2013. -
  181. [194][194] Q. Xu, S. J. Pan, H. H. Xue, and Q. Yang, 'Multitask learning for protein subcellular location prediction,' IEEE/ACM TCBB , 2011. -
  182. [195][195] Z. Wu, C. Valentini-Botinhao, O. Watts, and S. King, 'Deep neural networks employing multi-task learning and stacked bottleneck features for speech synthesis,' in ICASSP , 2015. -
  183. [197][197] J. Bai, K. Zhou, G. Xue, H. Zha, G. Sun, B. L. Tseng, Z. Zheng, and Y. Chang, 'Multi-task learning for learning to rank in web search,' in CIKM , 2009. -
  184. [198][198] J. Ghosn and Y. Bengio, 'Multi-task learning for stock selection,' in NIPS , 1996. -
  185. [199][199] C. Yuan, W. Hu, G. Tian, S. Yang, and H. Wang, 'Multi-task sparse learning with Beta process prior for action recognition,' in CVPR , 2013. -
  186. [201][201] P. Bell and S. Renals, 'Regularization of context-dependent deep neural networks with context-independent multi-task training,' in ICASSP , 2015. -
  187. [203][203] V. W. Zheng, S. J. Pan, Q. Yang, and J. J. Pan, 'Transferring multidevice localization models using latent multi-task learning,' in AAAI , 2008. -
  188. [204][204] R. Collobert and J. Weston, 'A unified architecture for natural language processing: Deep neural networks with multitask learning,' in ICML , 2008. -
  189. [205][205] M. Lapin, B. Schiele, and M. Hein, 'Scalable multitask representation learning for scene classification,' in CVPR , 2014. -
  190. [206][206] A. H. Abdulnabi, G. Wang, J. Lu, and K. Jia, 'Multi-task CNN model for attribute prediction,' IEEE TMM , 2015. -
  191. [207][207] M. Luong, Q. V. Le, I. Sutskever, O. Vinyals, and L. Kaiser, 'Multi-task sequence to sequence learning,' in ICLR , 2016. -
  192. [208][208] J. Yim, H. Jung, B. Yoo, C. Choi, D. Park, and J. Kim, 'Rotating your face using multi-task deep neural network,' in CVPR , 2015. -
  193. [209][209] X. Chu, W. Ouyang, W. Yang, and X. Wang, 'Multi-task recurrent neural network for immediacy prediction,' in ICCV , 2015. -
  194. [210][210] X. Wang, C. Zhang, and Z. Zhang, 'Boosted multi-task learning for face verification with applications to web image and video search,' in CVPR , 2009. -
  195. [211][211] X. Yuan and S. Yan, 'Visual classification with multi-task joint sparse representation,' in CVPR , 2010. -
  196. [213][213] A. Ahmed, M. Aly, A. Das, A. J. Smola, and T. Anastasakos, 'Webscale multi-task feature selection for behavioral targeting,' in CIKM , 2012. -
  197. [214][214] H. Wang, F. Nie, H. Huang, S. L. Risacher, C. H. Q. Ding, A. J. Saykin, and L. Shen, 'Sparse multi-task regression and feature selection to identify brain imaging predictors for memory performance,' in ICCV , 2011. -
  198. [216][216] J. Zhou, L. Yuan, J. Liu, and J. Ye, 'A multi-task learning formulation for predicting disease progression,' in KDD , 2011. -
  199. [217][217] J. Wan, Z. Zhang, J. Yan, T. Li, B. D. Rao, S. Fang, S. Kim, S. L. Risacher, A. J. Saykin, and L. Shen, 'Sparse Bayesian multi-task learning for predicting cognitive outcomes from neuroimaging measures in Alzheimer's disease,' in CVPR , 2012. -
  200. [220][220] B. Cheng, G. Liu, J. Wang, Z. Huang, and S. Yan, 'Multi-task low-rank affinity pursuit for image segmentation,' in ICCV , 2011. -

200 of 235 shown

Cited by

Sorted by citations

Authors
Zhang, Y., & Yang, Q.
Affiliations
  • Southern University of Science and Technology (CN)
  • Hong Kong University of Science and Technology (HK)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2017
Volume
34
Issue
12
Pages
5586-5609
ISSN
1041-4347 (ISSN-L)
DOI
10.48550/arxiv.1707.08114
arXiv
1707.08114
OpenAlex
W3141797743
Cited by
2,910
References
0
Citations by year
2026: 439 · 2025: 665 · 2024: 507 · 2023: 417 · 2022: 316 · 2021: 243 · 2020: 172 · 2019: 108 · 2018: 37 · 2017: 6
FWCI
131.19
Type
journal article
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • IEEE Transactions on Knowledge and Data Engineering (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
  • arXiv (Cornell University)
Funders
National Natural Science Foundation of China (62076118)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
  • Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
Keywords
multi-task learning, related tasks, generalization performance, representation learning, low-rank models, task clustering, decomposition method, semi-supervised learning, active learning (machine learning), unsupervised learning, reinforcement learning, multi-view learning, probabilistic graphical models, online learning, distributed learning, dimensionality reduction, feature hashing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review