The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Model-Contrastive Federated Learning
Open accessAbstract
Federated learning enables multiple parties to collaboratively train a machine learning model without communicating their local data. A key challenge in federated learning is to handle the heterogeneity of local data distribution across parties. Although many studies have been proposed to address this challenge, we find that they fail to achieve high performance in image datasets with deep learning models. In this paper, we propose MOON: model-contrastive federated learning. MOON is a simple and effective federated learning framework. The key idea of MOON is to utilize the similarity between model representations to correct the local training of individual parties, i.e., conducting contrastive learning in model-level. Our extensive experiments show that MOON significantly outperforms the other state-of-the-art federated learning algorithms on various image classification tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
(2008) · Journal of Machine Learning Research · Cited by 33,051
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857
Federated Machine Learning: Concept and Applications
Yang, Q., Liu, Y., Chen, T., & Tong, Y. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3298981 · Cited by 6,322
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242
Cited by
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
Li, Q., Wen, Z., Wu, Z., Hu, S., Wang, N., Li, Y., Liu, X., & He, B. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3124599 · Cited by 1,407
Self-citationTowards Personalized Federated Learning
Tan, A. Z., Yu, H., Cui, L., & Yang, Q. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3160699 · Cited by 1,182
Federated Learning on Non-IID Data Silos: An Experimental Study
Li, Q., Diao, Y., Chen, Q., & He, B. (2022) · 2022 IEEE 38th International Conference on Data Engineering (ICDE) · doi:10.1109/icde53745.2022.00077 · Cited by 1,132
Self-citationFedProto: Federated Prototype Learning across Heterogeneous Clients
Tan, Y., Long, G., LIU, L., Zhou, T., Lu, Q., Jiang, J., & Zhang, C. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i8.20819 · Cited by 761
FedALA: Adaptive Local Aggregation for Personalized Federated Learning
Zhang, J., Hua, Y., Wang, H., Song, T., Xue, Z., Ma, R., & Guan, H. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i9.26330 · Cited by 420
Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning
Zhang, L., Shen, L., Ding, L., Tao, D., & Duan, L.-Y. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00993 · Cited by 371
Rauniyar, A., Hagos, D. H., Jha, D., Håkegård, J. E., Bagci, U., Rawat, D. B., & Vlassov, V. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3329061 · Cited by 315
“In this regard, contrastive learning has recently become a popular research field for learning unlabeled data representations by training an FL model on unlabeled data [211].”
BackgroundAdvances and Challenges in Meta-Learning: A Technical Review
Vettoruzzo, A., Bouguelia, M.-R., Vanschoren, J., Rögnvaldsson, T., & Santosh, K. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3357847 · Cited by 314
Learn from Others and Be Yourself in Heterogeneous Federated Learning
Huang, W., Ye, M., & Du, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00990 · Cited by 280
FedProc: Prototypical contrastive federated learning on non-IID data
Mu, X., Shen, Y., Cheng, K., Geng, X., Fu, J., Zhang, T., & Zhang, Z. (2023) · Future Generation Computer Systems · doi:10.1016/j.future.2023.01.019 · Cited by 274
Yan, R., Qu, L., Wei, Q., Huang, S.-C., Shen, L., Rubin, D. L., Xing, L., & Zhou, Y. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3233574 · Cited by 183
ProtoHAR: Prototype Guided Personalized Federated Learning for Human Activity Recognition
Cheng, D., Zhang, L., Bu, C., Wang, X., Wu, H., & Song, A. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3275438 · Cited by 92
“We compare our algorithm with eight state-of-the-art FL baselines in Non-IID setting: SOLO: Select all clients for lo-cal training without using FL(i.e., the computational cost is ignored); FedAvg [5]: The original federated averaging algo-rithm selects a subset of clients for local training during each round to ensur…”
BackgroundMethodInfluentialFederated Contrastive Learning With Feature-Based Distillation for Human Activity Recognition
Xiao, Z., & Tong, H. (2025) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3510428 · Cited by 86
Adam, M., & Baroudi, U. (2024) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2024.3506214 · Cited by 46
Yue, G., Wei, P., Zhou, T., Song, Y., Zhao, C., Wang, T., & Lei, B. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3319516 · Cited by 24
A distribution information sharing federated learning approach for medical image data
Zhao, L., & Huang, J. (2023) · Complex & Intelligent Systems · doi:10.1007/s40747-023-01035-1 · Cited by 23
“[11] shows that these methods perform as poorly as FedAvg [12] on image datasets with deep learning models.”
MethodCommunication-efficient federated learning for multi-institutional medical image classification
Zhou, S., Landman, B., Huo, Y., & Gokhale, A. (2022) · Medical Imaging 2022: Imaging Informatics for Healthcare, Research, and Applications · doi:10.1117/12.2611654 · Cited by 7
“Like previous studies 8 , 9 we use Dirichlet distribution to generate non-i.i.d datasets among batches.”
MethodEdgeHive: Federated Learning on IoT Clusters in Unreliable Networks
Gupta, R. (2025) · 2025 13th International Conference on Intelligent Systems and Embedded Design (ISED) · doi:10.1109/ised67359.2025.11405287
Metadata
- Authors
- Li, Q., He, B., & Song, D.
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 10708-10717
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3182158470
- Cited by
- 1,534
- References
- 47
- FWCI
- 93.04
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.