Distributed Multi-Task Relationship Learning
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
Multi-task learning aims to learn multiple tasks jointly by exploiting their relatedness to improve the generalization performance for each task. Traditionally, to perform multi-task learning, one needs to centralize data from all the tasks to a single machine. However, in many real-world applications, data of different tasks may be geo-distributed over different local machines. Due to heavy communication caused by transmitting the data and the issue of data privacy and security, it is impossible to send data of different task to a master machine to perform multi-task learning. Therefore, in this paper, we propose a distributed multi-task learning framework that simultaneously learns predictive models for each task as well as task relationships between tasks alternatingly in the parameter server paradigm. In our framework, we first offer a general dual form for a family of regularized multi-task relationship learning methods. Subsequently, we propose a communication-efficient primal-dual distributed optimization algorithm to solve the dual problem by carefully designing local subproblems to make the dual problem decomposable. Moreover, we provide a theoretical convergence analysis for the proposed algorithm, which is specific for distributed multi-task relationship learning. We conduct extensive experiments on both synthetic and real-world datasets to evaluate our proposed framework in terms of effectiveness and convergence.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636
Communication-Efficient Distributed Optimization using an Approximate Newton-type Method
Shamir, O., Srebro, N., & Zhang, T. (2013) · arXiv · doi:10.48550/arxiv.1312.7853 · Cited by 424 · cited 2×
“Although there have been developments in data-parallel distributed algorithms for single task learning [17, 27, 32], distributed MTL under the aforementioned setting remains challenging as these algorithms do not suit the MTL formulation as MTL requires joint optimization of parameters of different tasks.”
BackgroundDistributed Learning, Communication Complexity and Privacy
Balcan, M.-F., Blum, A., Fine, S., & Mansour, Y. (2012) · arXiv · doi:10.48550/arxiv.1204.3514 · Cited by 117
Cited by
A Survey on Multi-Task Learning
Zhang, Y., & Yang, Q. (2017) · arXiv · doi:10.48550/arxiv.1707.08114 · Cited by 2,910
A Survey on Multi-Task Learning
Zhang, Y., & Yang, Q. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3070203 · Cited by 2,878
An Overview of Multi-Task Learning in Deep Neural Networks
Ruder, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05098 · Cited by 2,265
Federated learning with hierarchical clustering of local updates to improve training on non-IID data
Briggs, C., Fan, Z., & Andras, P. (2020) · 2020 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207469 · Cited by 676
Personalized Federated Learning With Differential Privacy
Hu, R., Guo, Y., Li, H., Pei, Q., & Gong, Y. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2991416 · Cited by 350
“Besides, some works recently studied the problem of collaborative learning of personalized models, however, they did not address any privacy issue [7], [24], [27].”
Background
Metadata
- Authors
- Liu, S., Pan, S. J., & Ho, Q.
- Venue
- Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
- Publisher
- ACM
- Published
- 2017-08-04
- Pages
- 937-946
- OpenAlex
- W2562969225
- Cited by
- 61
- References
- 3
- FWCI
- 3.18
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.