Cited by 250Domain Adaptation and Few-Shot LearningArtificial Intelligence in Healthcare and EducationMedical Image Segmentation Techniques

Robust and data-efficient generalization of self-supervised machine learning for diagnostic imaging

Azizi, Shekoofeh, Culp, Laura, Freyberg, Jan, Mustafa, Basil, Baur, Sebastien, Kornblith, Simon, Chen, Ting, Tomasev, Nenad +28 more·2023·Nature Biomedical Engineering

Abstract

Machine-learning models for medical tasks can match or surpass the performance of clinical experts. However, in settings differing from those of the training dataset, the performance of a model can deteriorate substantially. Here we report a representation-learning strategy for machine-learning models applied to medical-imaging tasks that mitigates such 'out of distribution' performance problem and that improves model robustness and training efficiency. The strategy, which we named REMEDIS (for 'Robust and Efficient Medical Imaging with Self-supervision'), combines large-scale supervised transfer learning on natural images and intermediate contrastive self-supervised learning on medical images and requires minimal task-specific customization. We show the utility of REMEDIS in a range of diagnostic-imaging tasks covering six imaging domains and 15 test datasets, and by simulating three realistic out-of-distribution scenarios. REMEDIS improved in-distribution diagnostic accuracies up to 11.5% with respect to strong supervised baseline models, and in out-of-distribution settings required only 1-33% of the data for retraining to match the performance of supervised models retrained using all available data. REMEDIS may accelerate the development lifecycle of machine-learning models for medical imaging.

Integrity signals

Registry

Crossref · 10.1038/s41551-023-01049-7

Cites

148 of 148 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Weakly-supervised deep learning models can be trained to perform various downstream computational pathology tasks such as tumor subtyping and prognosis without any region or pixel-level annotations.

    “To overcome these limitations, several research groups have alternatively proposed the use of weakly-supervised deep learning models, which can be trained to perform various downstream computational pathology tasks such as tumor subtyping and prognosis32–37 without any region or pixel-level annotations.”
  • Virchow is the largest foundation model for computational pathology to date.

    “we present Virchow, the largest foundation model for computational pathology to date.”
  • Recent pioneering works have utilized datasets ranging from 30,000 to 400,000 WSIs to train foundation models ranging in size from 28 million to 307 million parameters.

    “Despite the challenges in collecting large-scale datasets in the pathology domain, recent pioneering works have utilized datasets ranging from 30,000 to 400,000 WSIs to train foundation models ranging in size from 28 million to 307 million parameters35–42”
  • These works demonstrate that image features produced with self-supervised learning of pathology images outperform image features trained on natural images and that performance improves with scale.

    “These works demonstrate that image features produced with self-supervised learning of pathology images outperform image features trained on natural images and that performance improves with scale.”
  • Self-supervised learning is enabling in-domain transfer by removing the label requirement, driving a second wave of scaling to tens of thousands of WSIs.

    “Now self-supervised learning is enabling in-domain transfer by removing the label requirement, driving a second wave of scaling to tens of thousands of WSIs to inform image representation35–39,56.”

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Azizi, S., Culp, L., Freyberg, J., Mustafa, B., Baur, S., Kornblith, S., Chen, T., Tomasev, N., Mitrović, J., Strachan, P., Mahdavi, S. S., Wulczyn, E., Babenko, B., Walker, M., Loh, A., Chen, P.-H. C., Liu, Y., Bavishi, P., McKinney, S. M., Winkens, J., Roy, A. G., Beaver, Z., Ryan, F., Krogue, J., Etemadi, M., Telang, U., Liu, Y., Peng, L., Corrado, G. S., Webster, D. R., Fleet, D., Hinton, G., Houlsby, N., Karthikesalingam, A., Norouzi, M., & Natarajan, V.
Affiliations
  • Google (United States) (US)
  • Google DeepMind (United Kingdom) (GB)
  • Georgia Institute of Technology (US)
  • Northwestern University (US)
Corresponding author
  • Azizi, Shekoofeh
  • Karthikesalingam, Alan
  • Natarajan, Vivek
Venue
Nature Biomedical Engineering
Published
2023-06-08
Volume
7
Issue
6
Pages
756-779
ISSN
2157-846X (ISSN-L)
OpenAlex
W4379878309
Cited by
250
References
141
Citations by year
2026: 68 · 2025: 92 · 2024: 77 · 2023: 12 · 2021: 1
FWCI
23.44
Type
journal article
Language
en
Open access
closed
Locations
  • Nature Biomedical Engineering (publishedVersion)
  • PubMed (publishedVersion)
Funders
Cancer Research UK
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
Keywords
self-supervised learning, representation learning, out-of-distribution generalization, model robustness, transfer learning, contrastive learning, diagnostic imaging, data efficiency
MeSH terms
Machine Learning, Supervised Machine Learning, Diagnostic Imaging

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review