The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
FedProto: Federated Prototype Learning across Heterogeneous Clients
Open accessAbstract
Heterogeneity across clients in federated learning (FL) usually hinders the optimization convergence and generalization performance when the aggregation of clients' knowledge occurs in the gradient space. For example, clients may differ in terms of data distribution, network latency, input/output space, and/or model architecture, which can easily lead to the misalignment of their local gradients. To improve the tolerance to heterogeneity, we propose a novel federated prototype learning (FedProto) framework in which the clients and server communicate the abstract class prototypes instead of the gradients. FedProto aggregates the local prototypes collected from different clients, and then sends the global prototypes back to all clients to regularize the training of local models. The training on each client aims to minimize the classification error on the local data while keeping the resulting local prototypes sufficiently close to the corresponding global ones. Moreover, we provide a theoretical analysis to the convergence rate of FedProto under non-convex objectives. In experiments, we propose a benchmark setting tailored for heterogeneous FL, with FedProto outperforming several recent FL approaches on multiple datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
(2008) · Journal of Machine Learning Research · Cited by 33,051
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Prototypical Networks for Few-shot Learning
Snell, J., Swersky, K., & Zemel, R. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.05175 · Cited by 4,755
Federated Optimization in Heterogeneous Networks
Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., & Smith, V. (2018) · arXiv · doi:10.48550/arxiv.1812.06127 · Cited by 4,023
The Future of Digital Health with Federated Learning
Rieke, N., Hancox, J., Li, W., Milletari, F., Roth, H., Albarqouni, S., Bakas, S., Galtier, M. N., Landman, B., Maier-Hein, K., et al. (2020) · arXiv · doi:10.48550/arxiv.2003.08119 · Cited by 3,495
Zhu, L., & Han, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-63076-8_2 · Cited by 1,906
Federated Learning for Healthcare Informatics
Xu, J., Glicksberg, B. S., Su, C., Walker, P., Bian, J., & Wang, F. (2021) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-020-00082-4 · Cited by 1,542 · cited 2×
“2020) and in related applications, such as healthcare (Rieke et al. 2020; Xu et al. 2020) and street image object detection (Luo et al.”
BackgroundMethodFederated Learning for Healthcare Informatics
Xu, J., Glicksberg, B. S., Su, C., Walker, P., Bian, J., & Wang, F. (2019) · arXiv · doi:10.48550/arxiv.1911.06270 · Cited by 1,538
Model-Contrastive Federated Learning
Li, Q., He, B., & Song, D. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01057 · Cited by 1,534
Sattler, F., Muller, K.-R., & Samek, W. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3015958 · Cited by 1,355
Exploiting Shared Representations for Personalized Federated LearningPreprint
Collins, L., Hassani, H., Mokhtari, A., & Shakkottai, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.07078 · Cited by 1,317
Think Locally, Act Globally: Federated Learning with Local and Global RepresentationsPreprint
Liang, P. P., Liu, T., Ziyin, L., Allen, N. B., Auerbach, R. P., Brent, D., Salakhutdinov, R., & Morency, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.01523 · Cited by 932
Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization
Wang, J., Liu, Q., Liang, H., Joshi, G., & Poor, H. V. (2020) · arXiv · doi:10.48550/arxiv.2007.07481 · Cited by 927 · cited 2×
“Moreover, functionality-based neural matching across local models (Wang et al. 2020a) can aggregate neurons with similar functionality regardless of the variance of the model architectures.”
BackgroundMethodAggregating Local Deep Features for Image Retrieval
Yandex, A. B., & Lempitsky, V. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.150 · Cited by 674
An Efficient Framework for Clustered Federated Learning
Ghosh, A., Chung, J., Yin, D., & Ramchandran, K. (2022) · IEEE Transactions on Information Theory · doi:10.1109/tit.2022.3192506 · Cited by 623
Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach
Fallah, A., Mokhtari, A., & Ozdaglar, A. (2020) · neural information processing systems · Cited by 596
Graph Self-Supervised Learning: A Survey
Liu, Y., Jin, M., Pan, S., Zhou, C., Zheng, Y., Xia, F., & Yu, P. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3172903 · Cited by 510
Federated Learning with Personalization Layers
Arivazhagan, M. G., Aggarwal, V., Singh, A. K., & Choudhary, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.00818 · Cited by 495
Ensemble Distillation for Robust Model Fusion in Federated Learning
Lin, T., Kong, L., Stich, S. U., & Jaggi, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07242 · Cited by 476
Jeong, E., Oh, S., Kim, H., Park, J., Bennis, M., & Kim, S.-L. (2018) · arXiv · doi:10.48550/arxiv.1811.11479 · Cited by 465
FedMD: Heterogenous Federated Learning via Model Distillation
Li, D., & Wang, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.03581 · Cited by 450
FedML: A Research Library and Benchmark for Federated Machine Learning
He, C., Li, S., So, J., Zeng, X., Zhang, M., Wang, H., Wang, X., Vepakomma, P., Singh, A., Qiu, H., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.13518 · Cited by 314
Adaptive Personalized Federated Learning
Deng, Y., Kamani, M. M., & Mahdavi, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.13461 · Cited by 310
Three Approaches for Personalization with Applications to Federated Learning
Mansour, Y., Mohri, M., Ro, J. H., & Suresh, A. T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.10619 · Cited by 301
From federated learning to federated neural architecture search: a survey
Zhu, H., Zhang, H., & Jin, Y. (2021) · Complex & Intelligent Systems · doi:10.1007/s40747-020-00247-z · Cited by 177
Anomaly Detection in Dynamic Graphs via Transformer
Liu, Y., Pan, S., Wang, Y. G., Xiong, F., Wang, L., Chen, Q., & Lee, V. C. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3124061 · Cited by 169 · cited 3×
“Recently, self-supervised learning strategies are incorporated into the local training phase to handle the heterogeneity challenges (Li, He, and Song 2021; Liu et al. 2021a; Yang et al. 2021).”
BackgroundReal-World Image Datasets for Federated LearningPreprint
Luo, J., Wu, X., Luo, Y., Huang, A., Huang, Y., Liu, Y., & Yang, Q. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.11089 · Cited by 131
A Universal Representation Transformer Layer for Few-Shot Image ClassificationPreprint
Liu, L., Hamilton, W. L., Long, G., Jiang, J. B., & Larochelle, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.11702 · Cited by 112
Self-citationFederated Learning for Privacy-Preserving Open Innovation Future on Digital Health
Long, G., Shen, T., Tan, Y., Gerrard, L., Clarke, A., & Jiang, J. (2022) · Humanity Driven AI · doi:10.1007/978-3-030-72188-6_6 · Cited by 85
Self-citation(2020) · arXiv (Cornell University) · Cited by 64
Prototype Guided Federated Learning of Visual Feature RepresentationsPreprint
Michieli, U., & Özay, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.08982 · Cited by 56
Robust Federated Recommendation SystemPreprint
Chen, C., Zhang, J., Tung, A. K. H., Kankanhalli, M., & Chen, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.08259 · Cited by 44
Differentially-private Federated Neural Architecture SearchPreprint
Singh, I., Zhou, H., Yang, K., Ding, M., Lin, B., & Xie, P. (2020) · doi:10.36227/techrxiv.12503420 · Cited by 22
Collective Model Fusion for Multiple Black-Box Experts
Hoang, Q. M., Hoang, T. N., Low, B. K. H., et al. (2019) · International Conference on Machine Learning · Cited by 15
Differentially-private Federated Neural Architecture SearchPreprint
Singh, I., Zhou, H., Yang, K., Ding, M., Lin, B., & Xie, P. (2020) · doi:10.36227/techrxiv.12503420.v1 · Cited by 13
Learning Task-Agnostic Embedding of Multiple Black-Box Experts for Multi-Task Model Fusion
Hoang, T. N., Lam, C. T., Low, B. K. H., & Jaillet, P. (2020) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 8
Cited by
Multi-center federated learning: clients clustering for better personalization
Long, G., Xie, M., Shen, T., Zhou, T., Wang, X., & Jiang, J. (2023) · World Wide Web · doi:10.1007/s11280-022-01046-x · Cited by 320
Self-citationA Review of Federated Learning Methods in Heterogeneous Scenarios
Pei, J., Liu, W., Li, J., Wang, L., & Liu, C. (2024) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2024.3385440 · Cited by 281
Federated learning for healthcare applications
Chaddad, A., Wu, Y., & Desrosiers, C. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3325822 · Cited by 136
ProtoHAR: Prototype Guided Personalized Federated Learning for Human Activity Recognition
Cheng, D., Zhang, L., Bu, C., Wang, X., Wu, H., & Song, A. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3275438 · Cited by 92
“A more detailed proof and analysis can be found in [34].”
BackgroundMethodInfluentialModel-Heterogeneous Semi-Supervised Federated Learning for Medical Image Segmentation
Ma, Y., Wang, J., Yang, J., & Wang, L. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3348982 · Cited by 55
Metadata
- Authors
- Tan, Y., Long, G., LIU, L., Zhou, T., Lu, Q., Jiang, J., & Zhang, C.
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2022-06-28
- Volume
- 36
- Issue
- 8
- Pages
- 8432-8440
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W4283796083
- Cited by
- 761
- References
- 39
- FWCI
- 85.67
- Type
- conference paper
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.