The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Self-supervised learning in medicine and healthcare
Abstract
The development of medical applications of machine learning has required manual annotation of data, often by medical experts. Yet, the availability of large-scale unannotated data provides opportunities for the development of better machine-learning models. In this Review, we highlight self-supervised methods and models for use in medicine and healthcare, and discuss the advantages and limitations of their application to tasks involving electronic health records and datasets of medical images, bioelectrical signals, and sequences and structures of genes and proteins. We also discuss promising applications of self-supervised learning for the development of models leveraging multimodal datasets, and the challenges in collecting unbiased data for their training. Self-supervised learning may accelerate the development of medical artificial intelligence.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, translating the capabilities of current patch-based foundation models to address patient- and slide-level clinical challenges still remains complex due to the immense scale of gigapixel WSIs, compounded by the small size of patient cohorts in real-world evidence39–42, especially for rare diseases with limited training data43–45.”
From A multimodal whole-slide foundation model for pathology (2025) · cites this paper as [40]
In the citing paper
“Self-supervised learning is an important approach for addressing the challenge by fully leveraging the unlabeled data.”
From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [146]
In the citing paper
“Based on contrastive learning and data augmentation, the self-supervised learning approach can learn robust data embeddings from unlabeled data and the trained model can be fitted to many downstream tasks [146,147].”
From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [146]
In the citing paper
“self-supervised pretraining, a technique in which models learn to extract meaningful patterns from unlabeled data by generating knowledge based on the data itself, allowing more effective utilization of available raw data across various modalities [100].”
From Multimodal Artificial Intelligence for Precision Critical Care: A Scoping Review (2026) · cites this paper as [100]
In the citing paper
“Several studies23–25 have demonstrated the efficacy of SSL in medicine where only limited data might be available for training deep-learning networks.”
From Foundation Model for Cancer Imaging Biomarkers (2024) · cites this paper as [24]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“Masked-word pre-training (guessing the hidden word via context words) has been extensively used after the development of transformers—a state-of-the-art class of neural-network architectures for language tasks 36 .”
MethodHighly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764 · cited 1×
“Techniques for natural language processing have been applied directly as a pre-training task for protein data by framing them as supervised tasks for predicting the structure or functions of proteins with partially masked sequences 40,41 .”
MethodImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 2×
“crowdsourcing marketplace for outsourcing tasks that typically require human intelligence) and hundreds of academics and citizen scientists to label approximately 15 million images of 21,000 classes (such as ‘broccoli’ and ‘hummingbird’) 3 .”
BackgroundMethodLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“Models trained on biased data are vulnerable to become biased themselves 57 .”
BackgroundLearning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · cited 2×
“…self-supervised models can learn to segment images into particular objects without any labels (as exemplified by the Distillation with NO Labels model 49 which, after training with paired image data without any segmentation labels, correctly identified pixels belonging to different object classes).”
BackgroundInternational Conference on Learning Representations (ICLR 2013)
松尾, 豊. (2013) · 人工知能学会誌 = Journal of Japanese Society for Artificial Intelligence · Cited by 6,286
Rajpurkar, P., Chen, E., Banerjee, O., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-021-01614-0 · Cited by 3,212
Self-citationShortcut learning in deep neural networks
Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., & Wichmann, F. A. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00257-z · Cited by 2,224 · cited 1×
“Strongly supervised models are liable to short - cut learning 50 by associating erroneous distortions of facets of the image with the label.”
BackgroundImproved Baselines with Momentum Contrastive Learning
Chen, X., Fan, H., Girshick, R., & He, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.04297 · Cited by 1,956 · cited 1×
“In what follows, we highlight specific examples of self-supervised pre-training tasks across medical domains and data types in its two major forms: contrastive learning and generative learning 6,7 .”
BackgroundRasmy, L., Xiang, Y., Xie, Z., Tao, C., & Zhi, D. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00455-y · Cited by 886
Research works printed but not linked: 12
- Chen, T., Kornblith, S., Norouzi, M. & Hinton, G. A simple framework for contrastive learning of visual representations. In Proc. 37th International Conference on Machine Learning (eds. Daumé, H. III & Singh, A.) 1597–1607 (PMLR, 2020).
- Zbontar, J., Jing, L., Misra, I., LeCun, Y. & Deny, S. Barlow Twins: self-supervised learning via redundancy reduction. In Proc. 38th International Conference on Machine Learning (eds. Meila, M. & Zhang, T.) 12310–12320 (PMLR, 2021).
- Sowrirajan, H., Yang, J., Ng, A. Y. & Rajpurkar, P. MoCo-CXR: MoCo pretraining improves representation and transferability of chest X-ray models. In Medical Imaging with Deep Learning 2021 727–743 (PMLR, 2021).
- Kiyasseh, D., Zhu, T. & Clifton, D. A. CLOCS: Contrastive Learning of Cardiac Signals across space, time, and patients. In Proc. 38th International Conference on Machine Learning (eds. Meila, M. & Zhang, T.) 5606–5615 (PMLR, 2021).
- Gopal, B. et al. 3KG: contrastive learning of 12-lead electrocardiograms using physiologically-inspired augmentations. in Proc. Machine Learning for Health (eds. Roy, S. et al.) 156–167 (PMLR, 2021).
- Meng, X., Ganoe, C. H., Sieberg, R. T., Cheung, Y. Y. & Hassanpour, S. Self-supervised contextual language representation of radiology reports to improve the identification of communication urgency. AMIA Jt. Summits Transl. Sci. Proc. 2020, 413–421 (2020).
- Baevski, A., Zhou, Y., Mohamed, A. & Auli, M. wav2vec 2.0: a framework for self-supervised learning of speech representations. In Advances in Neural Information Processing Systems (eds Larochelle, H. et al.) 12449–12460 (Curran Associates, 2020).
- Jaegle, A. et al. Perceiver IO: a general architecture for structured inputs and outputs. In International Conference on Learning Representations 4039 (ICLR, 2022).
- Rao, R. M. et al. MSA Transformer. In Proc. 38th International Conference on Machine Learning (eds Meila, M. & Zhang, T.) 8844–8856 (PMLR, 2021).
- Endo, M., Krishnan, R., Krishna, V., Ng, A. Y. & Rajpurkar, P. Retrieval-based chest X-ray report generation using a pre-trained contrastive language-image model. in Proc. Machine Learning for Health (eds. Roy, S. et al.) 209–219 (PMLR, 2021).
- Sriram, A. et al. COVID-19 prognosis via self-supervised representation learning and multi-image prediction. Preprint at https://doi.org/10.48550/arXiv.2101.04909 (2021). · DOI
- Chen, R. J. & Krishnan, R. G. Self-supervised vision transformers learn visual concepts in histopathology. in LMLR at Neural Information Processing Systems (NeurIPS, 2021).
Cited by
Foundation models for generalist medical artificial intelligence
Moor, M., Banerjee, O., Abad, Z. S. H., Krumholz, H. M., Leskovec, J., Topol, E. J., & Rajpurkar, P. (2023) · Nature · doi:10.1038/s41586-023-05881-4 · Cited by 2,028
Self-citationA foundation model for generalizable disease detection from retinal images
Zhou, Y., Chia, M. A., Wagner, S. K., Ayhan, M. S., Williamson, D. J., Struyven, R. R., Liu, T., Xu, M., Lozano, M. G., Woodward-Court, P., et al. (2023) · Nature · doi:10.1038/s41586-023-06555-x · Cited by 1,014
“The combined qualities of strong generalization capacity of representations, and high performance achieved by fine-tuned models in many downstream tasks, indicate the great potential of SSL in medical AI in which data are abundant and healthcare tasks are diverse but labels are scarce 1,8 .”
BackgroundSelf-citationA pathology foundation model for cancer diagnosis and prognosis prediction
Wang, X., Zhao, J., Marostica, E., Yuan, W., Jin, J., Zhang, J., Li, R., Tang, H., Wang, K., Li, Y., et al. (2024) · Nature · doi:10.1038/s41586-024-07894-z · Cited by 580
Zhao, Z., Alzubaidi, L., Zhang, J., Duan, Y., & Gu, Y. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122807 · Cited by 410
Huang, S.-C., Pareek, A., Jensen, M., Lungren, M. P., Yeung, S., & Chaudhari, A. S. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00811-0 · Cited by 479
“1), as well as a few narrative reviews 21,22 , the most suitable strategies and best practices for medical images have not been suf fi ciently investigated.”
Background
Metadata
- Authors
- Krishnan, R., Rajpurkar, P., & Topol, E. J.
- Affiliations
- Stanford University (US)
- Harvard University (US)
- Scripps Research Institute (US)
- Corresponding author
- Rajpurkar, Pranav
- Venue
- Nature Biomedical Engineering
- Published
- 2022-08-11
- Volume
- 6
- Issue
- 12
- Pages
- 1346-1352
- ISSN
- 2157-846X (ISSN-L)
- OpenAlex
- W4291023040
- Cited by
- 605
- References
- 50
- Citations by year
- 2026: 104 · 2025: 220 · 2024: 185 · 2023: 84 · 2022: 11
- FWCI
- 8.71
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Nature Biomedical Engineering (publishedVersion)
- PubMed (publishedVersion)
- Funders
- U.S. Department of Health & Human Services | National Institutes of Health (UL1TR002550)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- self-supervised learning, artificial intelligence in healthcare, electronic health records, medical image analysis, bioelectric signals, protein structure prediction, gene sequence analysis, multimodal data, data annotation, data bias
- MeSH terms
- Machine Learning, Supervised Machine Learning, Artificial Intelligence, Delivery of Health Care, Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Self-supervised learning in medicine and healthcare. https://app.cochat.ai/papers/10.1038/s41551-022-00914-1
- The paper
- Krishnan, R., Rajpurkar, P., & Topol, E. J. (2022). Self-supervised learning in medicine and healthcare. Nature Biomedical Engineering. https://doi.org/10.1038/s41551-022-00914-1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.