The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Integrating low-rank and group-sparse structures for robust multi-task learning
Abstract
Multi-task learning (MTL) aims at improving the generalization performance by utilizing the intrinsic relationships among multiple related tasks. A key assumption in most MTL algorithms is that all tasks are related, which, however, may not be the case in many real-world applications. In this paper, we propose a robust multi-task learning (RMTL) algorithm which learns multiple tasks simultaneously as well as identifies the irrelevant (outlier) tasks. Specifically, the proposed RMTL algorithm captures the task relationships using a low-rank structure, and simultaneously identifies the outlier tasks using a group-sparse structure. The proposed RMTL algorithm is formulated as a non-smooth convex (unconstrained) optimization problem. We propose to adopt the accelerated proximal method (APM) for solving such an optimization problem. The key component in APM is the computation of the proximal operator, which can be shown to admit an analytic solution. We also theoretically analyze the effectiveness of the RMTL algorithm. In particular, we derive a key property of the optimal solution to RMTL; moreover, based on this key property, we establish a theoretical bound for characterizing the learning performance of RMTL. Our experimental results on benchmark data sets demonstrate the effectiveness and efficiency of the proposed algorithm.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
Beck, A., & Teboulle, M. (2009) · SIAM Journal on Imaging Sciences · doi:10.1137/080716542 · Cited by 11,952
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636
A Singular Value Thresholding Algorithm for Matrix Completion
Cai, J.-F., Candès, E. J., & Shen, Z. (2010) · SIAM Journal on Optimization · doi:10.1137/080738970 · Cited by 5,809
Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization
Recht, B., Fazel, M., & Parrilo, P. A. (2010) · SIAM Review · doi:10.1137/070697835 · Cited by 3,471
Simultaneous analysis of Lasso and Dantzig selector
Bickel, P. J., Ritov, Y., & Tsybakov, A. B. (2009) · The Annals of Statistics · doi:10.1214/08-aos620 · Cited by 2,396
Convex multi-task feature learning
Argyriou, A., Evgeniou, T., & Pontil, M. (2008) · Machine Learning · doi:10.1007/s10994-007-5040-8 · Cited by 1,528
Efficient and Robust Feature Selection via Joint ℓ2,1-Norms Minimization
Nie, F., Huang, H., Cai, X., & Ding, C. H. Q. (2010) · Cited by 1,522
Proximité et dualité dans un espace hilbertien
Moreau, J. J. (1965) · Bulletin de la Société mathématique de France · doi:10.24033/bsmf.1625 · Cited by 1,462
A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data
Ando, R. K., & Zhang, T. (2005) · doi:10.5555/1046920.1194905 · Cited by 1,311
Caruana, R. (1998) · Cited by 1,110
Learning Multiple Tasks with Kernel Methods
Evgeniou, T., Micchelli, C. A., & Pontil, M. (2005) · doi:10.5555/1046920.1088693 · Cited by 799
SLEP: Sparse Learning with Efficient Projections
Liu, J., Ji, S., & Ye, J. (2011) · Cited by 585
Self-citationMulti-Task Feature Learning Via Efficient l2,1-Norm Minimization
Liu, J., Ji, S., & Ye, J. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2631 · Cited by 555
Self-citationAn accelerated gradient method for trace norm minimization
Ji, S., & Ye, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553434 · Cited by 517
Self-citationMulti-task learning for classification with dirichlet process priors
Xue, Y., Liao, X., Carin, L., Krishnapuram, B., & Bartlett, P. (2007) · Cited by 485
Clustered Multi-Task Learning: A Convex Formulation
Jacob, L., Bach, F., & Vert, J. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0809.2085 · Cited by 400
Learning Gaussian processes from multiple tasks
Yu, K., Tresp, V., & Schwaighofer, A. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102479 · Cited by 380
Tree-Guided Group Lasso for Multi-Task Regression with Structured Sparsity
Kim, S., & Xing, E. P. (2018) · Figshare · doi:10.1184/r1/6476414 · Cited by 374
A Dirty Model for Multi-task Learning
Jalali, A., Sanghavi, S., Ruan, C., & Ravikumar, P. (2010) · Cited by 368
Learning to learn with the informative vector machine
Lawrence, N. D., & Platt, J. C. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015382 · Cited by 338
Discovering Structure in Multiple Learning Tasks: The TC Algorithm.
Thrun, S., & O’Sullivan, J. A. (1996) · Cited by 208
A convex formulation for learning shared structures from multiple tasks
Chen, J., Tang, L., Liu, J., & Ye, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553392 · Cited by 197
Self-citation(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244304322765658 · Cited by 192
Robust PCA via Outlier Pursuit
Xu, H., Caramanis, C., & Sanghavi, S. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1010.4237 · Cited by 192
Large-scale sparse logistic regression
Liu, J., Chen, J., & Ye, J. (2009) · Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1557019.1557082 · Cited by 191
Self-citationTaking Advantage of Sparsity in Multi-Task Learning
Lounici, K., Pontil, M., Tsybakov, A. B., & van de Geer, S. (2009) · Cited by 179
Learning Gaussian Process Kernels via Hierarchical Bayes
Schwaighofer, A., Tresp, V., & Yu, K. (2004) · Cited by 172
Learning incoherent sparse and low-rank patterns from multiple tasks
Chen, J.-H., Liu, J., & Ye, J. (2010) · Knowledge Discovery and Data Mining · doi:10.1145/1835804.1835952 · Cited by 166
Self-citationAn efficient algorithm for a class of fused lasso problems
Liu, J., Yuan, L., & Ye, J. (2010) · Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1835804.1835847 · Cited by 163
Self-citationConvergence of Fixed-Point Continuation Algorithms for Matrix Rank Minimization
Goldfarb, D., & Ma, S. (2011) · Foundations of Computational Mathematics · doi:10.1007/s10208-011-9084-6 · Cited by 144
EFFICIENT METHODS IN CONVEX PROGRAMMING
Nemirovski, A. S. (2007) · Cited by 114
Learning Multiple Related Tasks using Latent Independent Component Analysis
Zhang, J. A., Ghahramani, Z., & Yang, Y. (2005) · Cited by 110
Robust multi-task learning witht-processes
Yu, S., Volker Tresp, C., & Yu, K. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273635 · Cited by 77
Heterogeneous multitask learning with joint sparsity constraints
Yang, X., Kim, S. Y., & Xing, E. P. (2009) · Cited by 76
Learning Incoherent Sparse and Low-Rank Patterns from Multiple Tasks
Chen, J.-H., Liu, J., & Ye, J. (2012) · TKDD · doi:10.1145/2086737.2086742 · Cited by 75
Self-citationMulti-task Learning Using Generalized t Process
Zhang, Y., & Yeung, D. (2010) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · Cited by 57
Multi-Task Learning for Classification with Dirichlet Process Priors
Xueya, LiaoXuejun, CarinLawrence, & KrishnapuramBalaji (2007) · Journal of Machine Learning Research · doi:10.5555/1248659.1248661 · Cited by 26
Cited by
A Survey on Multi-Task Learning
Zhang, Y., & Yang, Q. (2017) · arXiv · doi:10.48550/arxiv.1707.08114 · Cited by 2,910
A Survey on Multi-Task Learning
Zhang, Y., & Yang, Q. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3070203 · Cited by 2,878
Cross-Stitch Networks for Multi-task Learning
Misra, I., Shrivastava, A., Gupta, A., & Hebert, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.433 · Cited by 1,397
An overview of multi-task learning
Zhang, Y., & Yang, Q. (2018) · National Science Review · doi:10.1093/nsr/nwx105 · Cited by 1,117
Chen, Z., & Liu, B. (2018) · Synthesis Lectures on Artificial Intelligence and Machine Learning · doi:10.1007/978-3-031-01581-6 · Cited by 466
Metadata
- Authors
- Chen, J., Zhou, J., & Ye, J.
- Venue
- Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining
- Publisher
- ACM
- Published
- 2011-08-21
- Pages
- 42-50
- OpenAlex
- W2018096278
- Cited by
- 311
- References
- 37
- FWCI
- 35.63
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.