The DOI matches its Crossref record; 58 of the 65 references it cites were found in a registry or an index.
Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imaging
Open accessAbstract
The collection and curation of large-scale medical datasets from multiple institutions is essential for training accurate deep learning models, but privacy concerns often hinder data sharing. Federated learning (FL) is a promising solution that enables privacy-preserving collaborative learning among different institutions, but it generally suffers from performance deterioration due to heterogeneous data distributions and a lack of quality labeled data. In this paper, we present a robust and label-efficient self-supervised FL framework for medical image analysis. Our method introduces a novel Transformer-based self-supervised pre-training paradigm that pre-trains models directly on decentralized target task datasets using masked image modeling, to facilitate more robust representation learning on heterogeneous data and effective knowledge transfer to downstream models. Extensive empirical results on simulated and real-world medical imaging non-IID federated datasets show that masked image modeling with Transformers significantly improves the robustness of models against various degrees of data heterogeneity. Notably, under severe data heterogeneity, our method, without relying on any additional pre-training data, achieves an improvement of 5.06%, 1.53% and 4.58% in test accuracy on retinal, dermatology and chest X-ray classification compared to the supervised baseline with ImageNet pre-training. In addition, we show that our federated self-supervised pre-training methods yield models that generalize better to out-of-distribution data and perform more effectively when fine-tuning with limited labeled data, compared to existing FL algorithms. The code is available at https://github.com/rui-yan/SSL-FL.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Nevertheless, using federated learning, Yan et al. have demonstrated the possibility of training self-supervised models with data from multiple healthcare centers without the need for explicitly sharing data, and have shown improvement in robustness and performance over models trained using data from only one institution97.”
From Self-supervised learning for medical image classification: a systematic review and implementation guidelines (2023) · cites this paper as [97]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857
Masked Autoencoders Are Scalable Vision Learners
He, K., Chen, X., Xie, S., Li, Y., Dollar, P., & Girshick, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01553 · Cited by 8,489
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242
Federated learning: Challenges, methods, and future directions
Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2975749 · Cited by 6,210
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691
Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning
Kermany, D. S., Goldbaum, M., Cai, W., Valentim, C. C. S., Liang, H., Baxter, S. L., McKeown, A., Yang, G., Wu, X., Yan, F., et al. (2018) · Cell · doi:10.1016/j.cell.2018.02.010 · Cited by 4,842
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., & Summers, R. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.369 · Cited by 4,616
Federated Optimization in Heterogeneous Networks
Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., & Smith, V. (2018) · arXiv · doi:10.48550/arxiv.1812.06127 · Cited by 4,023
The future of digital health with federated learning
Rieke, N., Hancox, J., Li, W., Milletarì, F., Roth, H. R., Albarqouni, S., Bakas, S., Galtier, M. N., Landman, B. A., Maier-Hein, K., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00323-1 · Cited by 4,015 · cited 2×
“Therefore, this training paradigm has been widely adopted for critical medical applications such as the detection of brain tumors [2] and COVID-19 [3], [4], and applied to various types of data, including medical imaging data, electronic health records and sensor data [2], [5], [6].”
BackgroundMethodBootstrap Your Own Latent: A New Approach to Self-Supervised Learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. A., Guo, Z. D., Azar, M. G., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.07733 · Cited by 3,846
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles
Noroozi, M., & Favaro, P. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_5 · Cited by 2,775
MixMatch: A Holistic Approach to Semi-Supervised LearningPreprint
Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., & Raffel, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.02249 · Cited by 2,523
Zero-shot text-to-image generation
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., & Sutskever, I. (2021) · arXiv · doi:10.48550/arxiv.2102.12092 · Cited by 2,417
Federated Learning with Non-IID Data
Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., & Chandra, V. (2018) · arXiv · doi:10.48550/arxiv.1806.00582 · Cited by 2,077
A brief introduction to weakly supervised learning
Zhou, Z.-H. (2018) · National Science Review · doi:10.1093/nsr/nwx106 · Cited by 1,899
Measuring the Effects of Non-Identical Data Distribution for Federated Visual ClassificationPreprint
Hsu, T., Qi, & Brown, M. (2019) · arXiv.org · doi:10.48550/arxiv.1909.06335 · Cited by 1,757
Model-Contrastive Federated Learning
Li, Q., He, B., & Song, D. X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01057 · Cited by 1,738
SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
Karimireddy, S. P., Kale, S., Mohri, M., Reddi, S. J., Stich, S. U., & Suresh, A. T. (2019) · arXiv · doi:10.48550/arxiv.1910.06378 · Cited by 1,734
An Empirical Study of Training Self-Supervised Vision Transformers
Chen, X., Xie, S., & He, K. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00950 · Cited by 1,648
Sheller, M. J., Edwards, B., Reina, G. A., Martin, J., Pati, S., Kotrotsou, A., Milchenko, M., Xu, W., Marcus, D., Colen, R. R., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-69250-1 · Cited by 1,646 · cited 1×
“Therefore, this training paradigm has been widely adopted for critical medical applications such as the detection of brain tumors [2] and COVID-19 [3], [4], and applied to various types of data, including medical imaging data, electronic health records and sensor data [2], [5], [6].”
MethodOn the Convergence of FedAvg on Non-IID Data
Li, X., Huang, K., Yang, W., Wang, S., & Zhang, Z. (2019) · arXiv · doi:10.48550/arxiv.1907.02189 · Cited by 1,575
Self-Supervised Learning of Pretext-Invariant Representations
Misra, I., & van der Maaten, L. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00674 · Cited by 1,435
BEiT: BERT Pre-Training of Image Transformers
Bao, H., Dong, L., Piao, S., & Wei, F. (2021) · arXiv · doi:10.48550/arxiv.2106.08254 · Cited by 1,392
Federated Learning with Matched AveragingPreprint
Wang, H., Yurochkin, M., Sun, Y., Papailiopoulos, D., & Khazaeni, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.06440 · Cited by 1,036
Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization
Wang, J., Liu, Q., Liang, H., Joshi, G., & Poor, H. V. (2020) · arXiv · doi:10.48550/arxiv.2007.07481 · Cited by 927
Federated learning for predicting clinical outcomes in patients with COVID-19
Dayan, I., Roth, H. R., Zhong, A., Harouni, A., Gentili, A., Abidin, A. Z., Liu, A., Costa, A. B., Wood, B. J., Tsai, C.-S., et al. (2021) · Nature Medicine · doi:10.1038/s41591-021-01506-3 · Cited by 842 · cited 8×
“[3] I. Dayan et al., “Federated learning for predicting clinical outcomes in patients with COVID-19,” Nature Med., vol. 27, no. 10, pp. 1735–1743, 2021.”
MethodInfluentialEnsemble Distillation for Robust Model Fusion in Federated LearningPreprint
Lin, T., Kong, L., Stich, S. U., & Jaggi, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07242 · Cited by 761
Think Locally, Act Globally: Federated Learning with Local and Global RepresentationsPreprint
Liang, P. P., Liu, T., Ziyin, L., Allen, N. B., Auerbach, R. P., Brent, D., Salakhutdinov, R., & Morency, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.01523 · Cited by 712
Big Self-Supervised Models Advance Medical Image Classification
Azizi, S., Mustafa, B., Ryan, F., Beaver, Z., Freyberg, J., Deaton, J., Loh, A., Karthikesalingam, A., Kornblith, S., Chen, T., et al. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00346 · Cited by 685
Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty
Hendrycks, D., Mazeika, M., Kadavath, S., & Song, D. (2019) · arXiv · doi:10.48550/arxiv.1906.12340 · Cited by 634
A patient-centric dataset of images and metadata for identifying melanomas using clinical context
Rotemberg, V. M., Kurtansky, N. R., Betz‐Stablein, B., Caffery, L. J., Chousakos, E., Codella, N., Combalia, M., Dusza, S. W., Guitera, P., Gutman, D. A., et al. (2021) · Scientific Data · doi:10.1038/s41597-021-00815-z · Cited by 617
Seven-point checklist and skin lesion classification using multitask multimodal neural nets
Kawahara, J., Daneshvar, S., Argenziano, G., & Hamarneh, G. (2019) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2018.2824327 · Cited by 584
End-to-end privacy preserving deep learning on multi-institutional medical imaging
Kaissis, G., Ziller, A., Passerat-Palmbach, J., Ryffel, T., Usynin, D., Trask, A., Lima, I., Mancuso, J., Jungmann, F., Steinborn, M.-M., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00337-8 · Cited by 513
Bayesian Nonparametric Federated Learning of Neural NetworksPreprint
Yurochkin, M., Agarwal, M., Ghosh, S., Greenewald, K. H., Hoang, T. N., & Khazaeni, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.12022 · Cited by 491
Data-Free Knowledge Distillation for Heterogeneous Federated Learning
Zhu, Z., Hong, J., & Zhou, J. (2021) · PubMed · doi:10.48550/arxiv.2105.10056 · Cited by 474
Zhou, Z., Sodha, V., Pang, J., Gotway, M. B., & Liang, J. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101840 · Cited by 375
The Non-IID Data Quagmire of Decentralized Machine LearningPreprint
Hsieh, K., Phanishayee, A., Mutlu, O., & Gibbons, P. B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.00189 · Cited by 353
Dynamic-Fusion-Based Federated Learning for COVID-19 Detection
Zhang, W., Zhou, T., Lu, Q., Wang, X., Zhu, C., Sun, H., Wang, Z., Lo, S. K., & Wang, F.-Y. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3056185 · Cited by 329
Group Knowledge Transfer: Federated Learning of Large CNNs at the EdgePreprint
He, C., Annavaram, M., & Avestimehr, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.14513 · Cited by 316
BIMCV COVID-19+: a large annotated dataset of RX and CT images from COVID-19 patientsPreprint
MdlI, V., Saborit-Torres, J. M., Ja, M., Pertusa, A., Bustos, A., Cazorla, M., Galant, J., Barber, X., Orozco‐Beltrán, D., García-García, F., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.01174 · Cited by 288
Yang, D., Xu, Z., Li, W., Myronenko, A., Roth, H. R., Harmon, S., Xu, S., Turkbey, B., Turkbey, E., Wang, X., et al. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.101992 · Cited by 279
Weakly Supervised Object Localization and Detection: A Survey
Zhang, D., Han, J., Cheng, G., & Yang, M.-H. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3074313 · Cited by 277
The impact of patient clinical information on automated skin cancer detection
Pacheco, A. G. C., & Krohling, R. A. (2019) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2019.103545 · Cited by 270
Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning
Qu, L., Zhou, Y., Liang, P. P., Xia, Y., Wang, F., Adeli, E., Fei-Fei, L., & Rubin, D. L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00982 · Cited by 235
Self-citationFederated unsupervised representation learning
Zhang, F., Kuang, K., Chen, L., You, Z., Shen, T., Xiao, J., Zhang⋆, Y., Wu, C., Wu, F. Q., Zhuang, Y., et al. (2023) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2200268 · Cited by 166
Divergence-aware Federated Self-Supervised LearningPreprint
Zhuang, W.-M., Wen, Y., & Zhang, S. (2022) · International Conference on Learning Representations · doi:10.48550/arxiv.2204.04385 · Cited by 135
Weakly Supervised Object Detection Using Proposal- and Semantic-Level Relationships
Zhang, D., Zeng, W., Yao, J., & Han, J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3046647 · Cited by 132
Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint LearningPreprint
Jeong, W., Yoon, J., Yang, E., & Hwang, S. J. (2020) · arXiv (Cornell University) · Cited by 108
Rotation-Oriented Collaborative Self-Supervised Learning for Retinal Disease Diagnosis
Li, X., Hu, X., Qi, X., Yu, L., Zhao, W., Heng, P.-A., & Xing, L. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3075244 · Cited by 105
Self-citationCollaborative Unsupervised Visual Representation Learning from Decentralized Data
Zhuang, W., Gan, X., Wen, Y., Zhang, S., & Yi, S. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00487 · Cited by 96
Federated Contrastive Learning for Volumetric Medical Image Segmentation
Wu, Y.-W., Zeng, D., Wang, Z., Shi, Y.-Y., & Hu, J. (2022) · International Conference on Medical Image Computing and Computer-Assisted Intervention · doi:10.1007/978-3-030-87199-4_35 · Cited by 77
Federated Contrastive Learning for Decentralized Unlabeled Medical Images
Dong, N., & Voiculescu, I. D. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87199-4_36 · Cited by 64
FedPerl: Semi-supervised Peer Learning for Skin Lesion Classification
Bdair, T. M., Navab, N., & Albarqouni, S. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87199-4_32 · Cited by 49
Federated Unsupervised Representation LearningPreprint
Zhang, F., Kuang, K., You, Z., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.08982 · Cited by 38
Semi-Supervised Federated Peer Learning for Skin Lesion Classification
Bdair, T. M., Navab, N., & Albarqouni, S. (2022) · The Journal of Machine Learning for Biomedical Imaging · doi:10.59275/j.melba.2022-8g82 · Cited by 11
Flagged first, then by citations
7 of the 65 references couldn’t be matched to a known paper.
Research works printed but not linked: 6
- tschandl, 2018, The HAM 10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions, Data Science Journal, 5, 1
- dosovitskiy, 2021, An image is worth 16×16 words: Transformers for image recognition at scale, Proc ICLR, 1
- chen, 2022, On pre-training for federated learning, arXiv 2206 11488
- nguyen, 2022, Where to begin? Exploring the impact of pre-training and initialization in federated learning, arXiv 2206 15387
- 10.1109/TPAMI.2020.3046647 · DOI · Paper page
- zhou, 2022, Self pre-training with masked autoencoders for medical image analysis, arXiv 2203 05573
Cited by
Sorted by citations
Federated learning for medical image analysis: A survey
Guan, H., Yap, P.-T., Bozoki, A., & Liu, M. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110424 · Cited by 518
Advances in medical image analysis with vision Transformers: A comprehensive review
Azad, R., Kazerouni, A., Heidari, M., Aghdam, E. K., Molaei, A., Jia, Y., Jose, A., Roy, R., & Merhof, D. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.103000 · Cited by 557
Huang, S.-C., Pareek, A., Jensen, M., Lungren, M. P., Yeung, S., & Chaudhari, A. S. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00811-0 · Cited by 479
Self-citationEmerging trends in federated learning: from model fusion to federated X learning
Ji, S., Tan, Y., Saravirta, T., Yang, Z. Q., Liu, Y., Vasankari, L., Pan, S., Long, G., & Walid, A. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-024-02119-1 · Cited by 192
Federated Learning for Medical Image Analysis with Deep Neural Networks
Nazir, S., & Kaleem, M. (2023) · Diagnostics · doi:10.3390/diagnostics13091532 · Cited by 163
Metadata
- Authors
- Yan, R., Qu, L., Wei, Q., Huang, S.-C., Shen, L., Rubin, D. L., Xing, L., & Zhou, Y.
- Affiliations
- Stanford University (US)
- University of Hong Kong (HK)
- Stanford Medicine (US)
- University of California, Santa Cruz (US)
- ORCID
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2023-07
- Volume
- 42
- Issue
- 7
- Pages
- 1932-1943
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- PMC
- PMC10880587
- OpenAlex
- W4280619489
- Cited by
- 183
- References
- 65
- Citations by year
- 2026: 38 · 2025: 76 · 2024: 43 · 2023: 24
- FWCI
- 17.06
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- Locations
- IEEE Transactions on Medical Imaging (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- National Institutes of Health (R01CA256890, R01CA227713, U01CA242879, R01HL155410, R01LM012966)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- federated learning, self-supervised learning, data heterogeneity, non-IID data, label efficiency, masked image modeling, transformer pre-training, medical image classification, privacy-preserving collaborative learning, out-of-distribution generalization, limited labeled data, retinal image classification, dermatological image classification, chest X-ray classification
- MeSH terms
- Algorithms, Diagnostic Imaging, Radiography, Retina
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
