CoChat
Sign Up
Cited by 183Privacy-Preserving Technologies in DataDomain Adaptation and Few-Shot LearningArtificial Intelligence in Healthcare and Education

The DOI matches its Crossref record; 58 of the 65 references it cites were found in a registry or an index.

Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imaging

Open access
Yan, Rui, Qu, Liangqiong, Wei, Qingyue, Huang, Shih-Cheng, Shen, Liyue, Rubin, Daniel L., Xing, Lei, Zhou, Yuyin·2023·IEEE Transactions on Medical Imaging
DOI 10.1109/tmi.2022.3233574PMC PMC10880587
Open this paper in CoChat — free accountOpen PDF

Abstract

The collection and curation of large-scale medical datasets from multiple institutions is essential for training accurate deep learning models, but privacy concerns often hinder data sharing. Federated learning (FL) is a promising solution that enables privacy-preserving collaborative learning among different institutions, but it generally suffers from performance deterioration due to heterogeneous data distributions and a lack of quality labeled data. In this paper, we present a robust and label-efficient self-supervised FL framework for medical image analysis. Our method introduces a novel Transformer-based self-supervised pre-training paradigm that pre-trains models directly on decentralized target task datasets using masked image modeling, to facilitate more robust representation learning on heterogeneous data and effective knowledge transfer to downstream models. Extensive empirical results on simulated and real-world medical imaging non-IID federated datasets show that masked image modeling with Transformers significantly improves the robustness of models against various degrees of data heterogeneity. Notably, under severe data heterogeneity, our method, without relying on any additional pre-training data, achieves an improvement of 5.06%, 1.53% and 4.58% in test accuracy on retinal, dermatology and chest X-ray classification compared to the supervised baseline with ImageNet pre-training. In addition, we show that our federated self-supervised pre-training methods yield models that generalize better to out-of-distribution data and perform more effectively when fine-tuning with limited labeled data, compared to existing FL algorithms. The code is available at https://github.com/rui-yan/SSL-FL.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tmi.2022.3233574

Cites

58 of 58 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Show all 65 references

7 of the 65 references couldn’t be matched to a known paper.

Research works printed but not linked: 6

  1. tschandl, 2018, The HAM 10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions, Data Science Journal, 5, 1
  2. dosovitskiy, 2021, An image is worth 16×16 words: Transformers for image recognition at scale, Proc ICLR, 1
  3. chen, 2022, On pre-training for federated learning, arXiv 2206 11488
  4. nguyen, 2022, Where to begin? Exploring the impact of pre-training and initialization in federated learning, arXiv 2206 15387
  5. 10.1109/TPAMI.2020.3046647 · DOI · Paper page
  6. zhou, 2022, Self pre-training with masked autoencoders for medical image analysis, arXiv 2203 05573

Cited by

Sorted by citations

Items 101–183 of 183

Related

See all

Metadata

Authors
Yan, R., Qu, L., Wei, Q., Huang, S.-C., Shen, L., Rubin, D. L., Xing, L., & Zhou, Y.
Affiliations
  • Stanford University (US)
  • University of Hong Kong (HK)
  • Stanford Medicine (US)
  • University of California, Santa Cruz (US)
ORCID
Venue
IEEE Transactions on Medical Imaging
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2023-07
Volume
42
Issue
7
Pages
1932-1943
ISSN
  • 0278-0062 (ISSN-L)
  • 1558-254X
DOI
10.1109/tmi.2022.3233574
PMC
PMC10880587
OpenAlex
W4280619489
Cited by
183
References
65
Citations by year
2026: 38 · 2025: 76 · 2024: 43 · 2023: 24
FWCI
17.06
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Transactions on Medical Imaging (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
National Institutes of Health (R01CA256890, R01CA227713, U01CA242879, R01HL155410, R01LM012966)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
federated learning, self-supervised learning, data heterogeneity, non-IID data, label efficiency, masked image modeling, transformer pre-training, medical image classification, privacy-preserving collaborative learning, out-of-distribution generalization, limited labeled data, retinal image classification, dermatological image classification, chest X-ray classification
MeSH terms
Algorithms, Diagnostic Imaging, Radiography, Retina

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review