Cited by 183Privacy-Preserving Technologies in DataDomain Adaptation and Few-Shot LearningArtificial Intelligence in Healthcare and Education

Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imaging

Open access
Yan, Rui, Qu, Liangqiong, Wei, Qingyue, Huang, Shih-Cheng, Shen, Liyue, Rubin, Daniel L., Xing, Lei, Zhou, Yuyin·2023·IEEE Transactions on Medical Imaging

Abstract

The collection and curation of large-scale medical datasets from multiple institutions is essential for training accurate deep learning models, but privacy concerns often hinder data sharing. Federated learning (FL) is a promising solution that enables privacy-preserving collaborative learning among different institutions, but it generally suffers from performance deterioration due to heterogeneous data distributions and a lack of quality labeled data. In this paper, we present a robust and label-efficient self-supervised FL framework for medical image analysis. Our method introduces a novel Transformer-based self-supervised pre-training paradigm that pre-trains models directly on decentralized target task datasets using masked image modeling, to facilitate more robust representation learning on heterogeneous data and effective knowledge transfer to downstream models. Extensive empirical results on simulated and real-world medical imaging non-IID federated datasets show that masked image modeling with Transformers significantly improves the robustness of models against various degrees of data heterogeneity. Notably, under severe data heterogeneity, our method, without relying on any additional pre-training data, achieves an improvement of 5.06%, 1.53% and 4.58% in test accuracy on retinal, dermatology and chest X-ray classification compared to the supervised baseline with ImageNet pre-training. In addition, we show that our federated self-supervised pre-training methods yield models that generalize better to out-of-distribution data and perform more effectively when fine-tuning with limited labeled data, compared to existing FL algorithms. The code is available at https://github.com/rui-yan/SSL-FL.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tmi.2022.3233574

Cites

58 of 58 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Yan et al. have demonstrated the possibility of training self-supervised models with data from multiple healthcare centers without explicitly sharing data, and have shown improvement in robustness and performance over models trained using data from only one institution.

    “Nevertheless, using federated learning, Yan et al. have demonstrated the possibility of training self-supervised models with data from multiple healthcare centers without the need for explicitly sharing data, and have shown improvement in robustness and performance over models trained using data from only one institution97.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

7 of the 65 references couldn’t be matched to a known paper.

Research works printed but not linked: 9

  1. [16][16] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, 'An image is worth 16x16 words: Transformers for image recognition at scale,' in ICLR , 2021. -
  2. [21][21] L. Zhou, H. Liu, J. Bae, J. He, D. Samaras, and P. Prasanna, 'Self pretraining with masked autoencoders for medical image analysis,' arXiv , 2022. -
  3. [27][27] W. Zhuang, Y. Wen, and S. Zhang, 'Divergence-aware federated selfsupervised learning,' in ICLR , 2021. -
  4. [35][35] H.-Y. Chen, C.-H. Tu, Z. Li, H.-W. Shen, and W.-L. Chao, 'On pretraining for federated learning,' arXiv , 2022. -
  5. [36][36] J. Nguyen, K. Malik, M. Sanjabi, and M. Rabbat, 'Where to begin? exploring the impact of pre-training and initialization in federated learning,' arXiv , 2022. -
  6. [38][38] I. Misra and L. v. d. Maaten, 'Self-supervised learning of pretextinvariant representations,' in CVPR , 2020, pp. 6707-6717. -
  7. [53][53] P. Tschandl, C. Rosendahl, and H. Kittler, 'The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions,' Scientific data , vol. 5, no. 1, pp. 1-9, 2018. -
  8. [60][60] W. Jeong, J. Yoon, E. Yang, and S. J. Hwang, 'Federated semisupervised learning with inter-client consistency & disjoint learning,' arXiv , 2020. -
  9. [62][62] X. Chen, S. Xie, and K. He, 'An empirical study of training selfsupervised vision transformers,' in ICCV , 2021, pp. 9640-9649.

Cited by

Sorted by citations

Authors
Yan, R., Qu, L., Wei, Q., Huang, S.-C., Shen, L., Rubin, D. L., Xing, L., & Zhou, Y.
Affiliations
  • Stanford University (US)
  • University of Hong Kong (HK)
  • Stanford Medicine (US)
  • University of California, Santa Cruz (US)
Venue
IEEE Transactions on Medical Imaging
Published
2023-07
Volume
42
Issue
7
Pages
1932-1943
ISSN
  • 0278-0062 (ISSN-L)
  • 1558-254X
OpenAlex
W4280619489
Cited by
183
References
65
Citations by year
2026: 38 · 2025: 76 · 2024: 43 · 2023: 24 · 2022: 2
FWCI
17.06
Type
journal article
Language
en
Open access
Locations
  • IEEE Transactions on Medical Imaging (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
National Institutes of Health (R01CA256890, R01CA227713, U01CA242879, R01HL155410, R01LM012966)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
federated learning, self-supervised learning, data heterogeneity, non-IID data, label efficiency, masked image modeling, transformer pre-training, medical image classification, privacy-preserving collaborative learning, out-of-distribution generalization, limited labeled data, retinal image classification, dermatological image classification, chest X-ray classification
MeSH terms
Algorithms, Diagnostic Imaging, Radiography, Retina

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review