Cited by 1,560AI in cancer detectionRadiomics and Machine Learning in Medical ImagingDomain Adaptation and Few-Shot Learning

Towards a general-purpose foundation model for computational pathology

Open access
Chen, Richard J., Ding, Tong, Lu, Ming Y., Williamson, Drew F. K., Jaume, Guillaume, Song, Andrew H., Chen, Bowen, Zhang, Andrew +12 more·2024·Nature Medicine

Abstract

Quantitative evaluation of tissue images is crucial for computational pathology (CPath) tasks, requiring the objective characterization of histopathological entities from whole-slide images (WSIs). The high resolution of WSIs and the variability of morphological features present significant challenges, complicating the large-scale annotation of data for high-performance applications. To address this challenge, current efforts have proposed the use of pretrained image encoders through transfer learning from natural image datasets or self-supervised learning on publicly available histopathology datasets, but have not been extensively developed and evaluated across diverse tissue types at scale. We introduce UNI, a general-purpose self-supervised model for pathology, pretrained using more than 100 million images from over 100,000 diagnostic H&E-stained WSIs (>77 TB of data) across 20 major tissue types. The model was evaluated on 34 representative CPath tasks of varying diagnostic difficulty. In addition to outperforming previous state-of-the-art models, we demonstrate new modeling capabilities in CPath such as resolution-agnostic tissue classification, slide classification using few-shot class prototypes, and disease subtyping generalization in classifying up to 108 cancer types in the OncoTree classification system. UNI advances unsupervised representation learning at scale in CPath in terms of both pretraining data and downstream evaluation, enabling data-efficient artificial intelligence models that can generalize and transfer to a wide range of diagnostically challenging tasks and clinical workflows in anatomic pathology. Pretrained using over 100,000 diagnostic histopathological slides across 20 major tissue types, a self-supervised model is shown to outperform existing baselines across various clinically relevant computational pathology tasks.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41591-024-02857-3

Cites

186 of 186 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Increasing the patch size from 256 × 256 pixels to 512 × 512 pixels reduces the sequence length by four without impacting representation quality, leveraging the robustness of patch-level foundation models to higher resolutions.

    “By increasing the patch size from the widely used 256 × 256 pixels, we effectively reduce the sequence length by four without impacting the representation quality due to higher resolution patch input, leveraging the robustness of the patch-level foundation models in generalizing to higher resolutions9,10,87.”

    From A multimodal whole-slide foundation model for pathology (2025) · cites this paper as [9]

    The Results section, ‘Robustness to high image resolution’ (page unavailable), reports that UNI retains relatively strong feature quality across image resolutions, but its KNN performance falls by 6.3% at higher resolution in one task. The Methods describe extracting 256 × 256 tissue patches, but do not test changing them to 512 × 512 or report a fourfold reduction in slide-level sequence length. Thus the source supports the general robustness premise, not the claim of unchanged quality for that specific patch-size change. (LLM, conf=0.88)

    Experimental — accuracy not yet measured
  • Weakly-supervised deep learning models can be trained to perform various downstream computational pathology tasks such as tumor subtyping and prognosis without any region or pixel-level annotations.

    “To overcome these limitations, several research groups have alternatively proposed the use of weakly-supervised deep learning models, which can be trained to perform various downstream computational pathology tasks such as tumor subtyping and prognosis32–37 without any region or pixel-level annotations.”

    From Predicting the tumor microenvironment composition and immunotherapy response in non-small cell lung cancer from digital histopathology images (2024) · cites this paper as [32]

    The Results section on weakly supervised slide classification reports tumor-subtyping tasks, and the Results section on prompt-based slide classification says that weakly supervised learning via multiple-instance learning no longer requires region-of-interest annotations. The cited paper does not report prognosis as one of its weakly supervised downstream tasks; prognosis appears in reference titles rather than in the paper’s account of those tasks. Page numbers were not available in our copy. (LLM, conf=0.84)

    Experimental — accuracy not yet measured
  • UNI, Prov-GigaPath, and Virchow were pretrained solely on pathology image data to perform cancer classification, diagnosis, and prognosis.

    “UNI [191], Prov-GigaPath [192], and Virchow [193] were pretrained solely on pathology image data to perform cancer classification, diagnosis, and prognosis.”

    From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [191]

    The Introduction describes UNI as pretrained on H&E histology images, and the Results report its use for cancer classification and diagnostic tasks. The cited paper does not report a UNI prognosis task, so that part of the claim is unsupported. Citation [191] identifies UNI; this verdict does not assess the separately cited Prov-GigaPath or Virchow. (LLM, conf=0.85)

    Experimental — accuracy not yet measured
  • The vision module is based on the patch encoder CTransPath for the small and medium models, and UNI for the large model.

    “The vision module is based on the patch encoder CTransPath17 for the small and medium models, and UNI18 for the large model.”
  • The UNI foundation model, when paired with multi-task AB-MIL, achieved the best predictive performance for the various TME signature prediction tasks.

    “We found out that the UNI foundation model32, when paired with multi-task AB-MIL, achieved the best predictive performance for the various TME signature prediction tasks.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 13

  1. [26]Chen T, Kornblith S, Norouzi M & Hinton G A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, 1597– 1607 (PMLR, 2020).
  2. [36]Chen RJ & Krishnan RG Self-supervised vision transformers learn visual concepts in histopathology. In Learning Meaningful Representations of Life, NeurIPS 2021 (2022).
  3. [75]Dosovitskiy A An image is worth 16×16 words: transformers for image recognition at scale. In International Conference on Learning Representations (2021).
  4. [96]Xiang J & Zhang J Exploring low-rank property in multiple instance learning for whole slide image classification. In The Eleventh International Conference on Learning Representations (2022).
  5. [112]Wang X SCL-WC: cross-slide contrastive learning for weakly-supervised whole-slide image classification. Advances in Neural Information Processing Systems 35, 18009– 18021 (2022).
  6. [117]Zhou J iBOT: image BERT pre-training with online tokenizer. In International Conference on Learning Representations (2022).
  7. [120]Devlin J, Chang M-W, Lee K & Toutanova K BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (2018).
  8. [122]Sablayrolles A, Douze M, Schmid C & Jégou H Spreading vectors for similarity search. In International Conference on Learning Representations (2019).
  9. [123]Touvron H, Vedaldi A, Douze M & Jegou H Fixing the train–test resolution discrepancy. In Advances in Neural Information Processing Systems, Vol. 32 (eds Wallach H) (Curran Associates, 2019).
  10. [124]Dao T, Fu DY, Ermon S, Rudra A & Ré C FlashAttention: fast and memory-efficient exact attention with IO-awareness. In Advances in Neural Information Processing Systems (2022).
  11. [128]Loshchilov I & Hutter F Decoupled weight decay regularization. In International Conference on Learning Representations (2019).
  12. [131]Sarıyıldız MB, Kalantidis Y, Alahari K & Larlus D No reason for no supervision: improved generalization in supervised models. In The Eleventh International Conference on Learning Representations (2023).
  13. [141]Yang J, Chen H, Yan J, Chen X & Yao J Towards better understanding and better generalization of low-shot classification in histology images with contrastive learning. In International Conference on Learning Representations (2021).

Cited by

Sorted by citations

Authors
Chen, R. J., Ding, T., Lu, M. Y., Williamson, D. F. K., Jaume, G., Song, A. H., Chen, B., Zhang, A., Shao, D., Shaban, M., Williams, M., Oldenburg, L., Weishaupt, L. L., Wang, J. J., Vaidya, A., Le, L. P., Gerber, G., Sahai, S., Williams, W., & Mahmood, F.
Affiliations
  • Broad Institute (US)
  • Brigham and Women's Hospital (US)
  • Harvard University (US)
Corresponding author
Mahmood, Faisal
Venue
Nature Medicine
Published
2024-03
Volume
30
Issue
3
Pages
850-862
ISSN
  • 1078-8956 (ISSN-L)
  • 1546-170X
OpenAlex
W4392947521
Cited by
1,560
References
170
Citations by year
2026: 789 · 2025: 613 · 2024: 154 · 2023: 4
FWCI
229.15
Type
journal article
Language
en
Open access
Locations
  • Nature Medicine (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
  • National Human Genome Research Institute (T32HG002295)
  • National Cancer Institute (T32CA251062)
  • National Institute of General Medical Sciences (R35GM149270)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
computational pathology, whole slide images, self-supervised learning, foundation models, unsupervised feature learning, tissue classification, few-shot learning, cancer subtyping, histopathology, transfer learning
MeSH terms
Artificial Intelligence, Workflow

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review