The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index; 4 of the 19 sources its claims cite were checked for support.
A clinical benchmark of public self-supervised pathology foundation models
Open accessAbstract
The use of self-supervised learning to train pathology foundation models has increased substantially in the past few years. Notably, several models trained on large quantities of clinical data have been made publicly available in recent months. This will significantly enhance scientific research in computational pathology and help bridge the gap between research and clinical deployment. With the increase in availability of public foundation models of different sizes, trained using different algorithms on different datasets, it becomes important to establish a benchmark to compare the performance of such models on a variety of clinically relevant tasks spanning multiple organs and diseases. In this work, we present a collection of pathology datasets comprising clinical slides associated with clinically relevant endpoints including cancer diagnoses and a variety of biomarkers generated during standard hospital operation from three medical centers. We leverage these datasets to systematically assess the performance of public pathology foundation models and provide insights into best practices for training foundation models and selecting appropriate pretrained models. To enable the community to evaluate their models on our clinical datasets, we make available an automated benchmarking pipeline for external use.
Overview
CoChat Summary
TL;DR
This paper presents a clinical benchmark of publicly available self-supervised pathology foundation models across 22 slide-level tasks from three health systems, covering disease detection and computational biomarker prediction. It finds that pathology-pretrained models generally outperform an ImageNet baseline, but scaling with model size, dataset size, or compute is weak, and performance depends more on pretraining dataset composition and task.
Key findings
- For disease detection, all models achieved AUCs above 0.9; the ImageNet-pretrained encoder consistently underperformed pathology-trained encoders, and CTransPath was consistently inferior. The top 3 ranked models by average AUC were H-optimus-0, Prov-GigaPath, and SP85M.
- Biomarker prediction was more variable. H-optimus-0, Prov-GigaPath, and UNI were top 3 ranked by average AUC; for ICI response in NSCLC, all models were barely above chance, with UNI performing significantly better at an average AUC of 0.6.
- Model size correlated weakly with detection performance (Pearson 0.055, p = 2.59e-2) and more with biomarker performance (Pearson 0.091, p = 9.52e-6). Pretraining dataset size showed no strong correlation: slides detection r = 0.008, p = 7.58e-01; biomarker r = −0.033, p = 1.07e-01; tiles detection r = 0.050, p = 7.79e-02; biomarker r = −0.048, p = 4.03e-02.
- Inference trade-offs: SP85M had average detection AUC 0.978 with 0.387 GB minimal memory, versus H-optimus-0 at 4.643 GB for a 0.2% average increase; UNI had average biomarker AUC 0.773 with 1.259 GB, versus H-optimus-0 at 4.643 GB and average AUC 0.785. Throughput ranged from 4417.4 TPS for truncated ResNet50 to 75.3 TPS for H-optimus-0; SP85M achieved 1063.6 TPS with a 0.2% average detection AUC decrease versus H-optimus-0.
Methods
- The benchmark used clinical slides from three institutions—Mount Sinai Health System (MSHS), Sahlgrenska University Hospital (SUH), and Memorial Sloan Kettering Cancer Center (MSKCC)—with detection tasks and biomarker prediction tasks summarized in Tables 2 and 3; all analysis was at 20x magnification (0.5 microns per pixel).
- For each slide, tiles were embedded with a frozen foundation model, then a Gated MIL Attention (GMA) aggregator with a linear classifier was trained. Generalization was estimated with Monte Carlo Cross-Validation: 80% train / 20% validation, 20 fixed MCCV folds, each run twice, 50 epochs with AdamW and peak learning rate 0.0001.
- Models benchmarked included CTransPath, UNI, Virchow, Prov-GigaPath, Virchow2, H-optimus-0, Phikon-v2, a truncated ResNet50 ImageNet baseline, and two in-house DINO-trained ViTs (SP22M, 21.7M parameters; SP85M, 85.8M parameters) pretrained on 423,563 H&E slides from 88,035 cases and 76,794 patients.
Limitations
- Stated: technical variability is limited to three institutions; the study focused on tile-level features using GMA with a linear classifier, with more complex aggregators out of scope; and for Virchow and Virchow2, overlap with MSKCC-based clinical tasks cannot be ruled out because they were trained on MSKCC slides.
- Stated: several questions could not be addressed due to lack of evidence, including optimal magnification, use of IHC slides, and benefits of multi-institutional pretraining data; the test data are not released to avoid data contamination.
- Evident from the text: the benchmark is not exhaustive, some scaling analyses excluded models due to missing data (e.g., Prov-GigaPath and H-optimus-0 for computational resources), and reported p-values were not adjusted for multiple hypothesis testing.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
2 Valid · 1 Not valid · 8 Not checked · 71 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“ICI response prediction from H&E slides is a challenging task, yet there is evidence that descriptors of local cellular networks42, that better model the tumor microenvironment (TME) can achieve AUCs of around 0.7, on-par with PD-L1 IHC, the current clinical gold standard.”
§ A clinical benchmark of public self-supervised pathology foundation models > Results > Computational Biomarker Prediction Tasks
What the sources say
- Supportedcites [42] Xie, C. et al. Computational biomarker predicts lung ici response via deep learning-driven hierarchical spatial modelling from h&e. https://www.researchsquare.com/article/rs-1251762/v1 (2022). source
Split into 2 atoms — 2 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“It is becoming abundantly clear that using SSL to train image encoders on unlabeled pathology data is superior to relying on models pretrained on other domains such as natural images26,36,37.”
§ A clinical benchmark of public self-supervised pathology foundation models > Introduction
What the sources say
- Not checked yetcites [26] Campanella, G. et al. Computational pathology at health system scale–self-supervised foundation models from three billion images. Preprint at Xiv:2310.07033 (2023).
- Not checked yetcites [36] Kang, M., Song, H., Park, S., Yoo, D. & Pereira, S. Benchmarking self-supervised learning on diverse pathology datasets. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 3344–3354 (IEEE, 2023). source
- Supportedcites [37] Campanella, G. et al. A clinical benchmark of public self-supervised pathology foundation models. Preprint at arXiv:2407.06508 (2024). source
The cited sentence appears verbatim in our copy's Introduction ('It is becoming abundantly clear that using SSL to train image encoders on unlabeled pathology data is superior to relying on models pretrained on other domains such as natural images'), and the paper's own results back it: Results > Disease Detection Tasks states 'The ImageNet pretrained encoder is consistently under-performing the pathology trained encoders,' statistically significant across tasks except the Thyroid cohort, and the Discussion summarizes that 'the ImageNet pretrained encoder, and CTransPath to a lesser degree, consistently underperformed compared to newer models.' The Introduction likewise frames the prior reliance on natural-image encoders as a limitation SSL for pathology addresses. No qualifier in the claim is unsupported by our copy. (LLM, conf=0.92)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We also found that model size and pretraining dataset composition influenced performance, particularly for biomarker prediction, but not necessarily for disease detection.”
§ A clinical benchmark of public self-supervised pathology foundation models > Discussion
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Requirements range from 0.127GB for SP22M, to 4.643GB for H-optimus-0 (see Supplementary Table 8).”
§ A clinical benchmark of public self-supervised pathology foundation models > Results > Foundation Model Inference
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Ranking the models based on the average AUC across tasks (Supplementary Fig. 9), the top 3 ranked models are H-optimus-0, Prov-GigaPath, and SP85M.”
§ A clinical benchmark of public self-supervised pathology foundation models > Results > Disease Detection Tasks
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
42 more claims
Create a free account to see all 82 claims and to verify the 15 references still open.
- Probably not a substantial claim. 11
- It may state two separate claims as one. 28
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“To ensure the diversity of the pretraining dataset, which has proven to be a key factor in successful patch encoders21, Mass-340K is distributed across 20 organs, different stains, diverse tissue types and scanned with various scanner types (Fig. 1a and Supplementary Table 1).”
From A multimodal whole-slide foundation model for pathology (2025) · cites this paper as [21]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“It is becoming abundantly clear that using SSL to train image encoders on unlabeled pathology data is superior to relying on models pretrained on other domains such as natural images26,36,37.”
From A clinical benchmark of public self-supervised pathology foundation models (2025) · cites this paper as [37]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Building on the success of knowledge distillation and masked image modeling86,87 for patch encoder pretraining21,22, TITAN introduces a large-scale pretraining paradigm that leverages millions of high-resolution region-of-interests (ROIs; at 8,192 × 8,192 pixels at 20× magnification) for scalable WSI encoding.”
From A multimodal whole-slide foundation model for pathology (2025) · cites this paper as [21]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A clinical benchmark of public self-supervised pathology foundation models > Results > Foundation Model Inference
“the best-performing H-optimus-0 which requires 4643GB”
§ A clinical benchmark of public self-supervised pathology foundation models > Results > Foundation Model Inference
“Requirements range from 0.127GB for SP22M, to 4.643GB for H-optimus-0”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A clinical benchmark of public self-supervised pathology foundation models > Results > Foundation Model Inference
“The best-performing H-optimus-0 required 4.643GB to achieve an average AUC of 0.785, a 1.3% increase in performance over UNI.”
§ A clinical benchmark of public self-supervised pathology foundation models > Results > Foundation Model Inference
“UNI shows a 1.5% drop in average AUC (0.773 for UNI vs 0.785 for H-optimus-0)”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A clinical benchmark of public self-supervised pathology foundation models > Results > Pretraining Dataset Composition
“For colon and rectum, we have two detection tasks.”
§ A clinical benchmark of public self-supervised pathology foundation models > Results > Pretraining Dataset Composition
“We didn’t observe a significant correlation between performance and tissue prevalence for the breast tasks considered with a Pearson statistic of 0.122 (p value: 8.59e-2, Supplementary Fig. 4c).”
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472 · cited 1×
“Convolutional neural networks (CNNs) and vision transformers 2 (ViTs) have been used to solve numerous problems using supervised learning and have enabled the training of predictive models for a variety of tasks with high performance 3 – 9 .”
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · cited 1×
“Wang et al. 18 proposed SRCL, an SSL method based on MoCo v3 10 , along with CTransPath, a model architecture that combines convolutional layers with the Swin Transformer 19 model.”
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · cited 1×
“Given the evidence from the natural language and vision domains that larger datasets and higher capacity models will produce better performance especially in the SSL setting 38 – 40 , training on larger pathology datasets should be a priority.”
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
The cancer genome atlas pan-cancer analysis project
The Cancer Genome Atlas Research Network, Weinstein, J. N., Collisson, E. A., Mills, G. B., Shaw, K. R. M., Ozenberger, B. A., Ellrott, K., Shmulevich, I., Sander, C., & Stuart, J. M. (2013) · Nature Genetics · doi:10.1038/ng.2764 · Cited by 9,092
Masked Autoencoders Are Scalable Vision Learners
He, K., Chen, X., Xie, S., Li, Y., Dollar, P., & Girshick, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01553 · Cited by 8,489 · cited 2×
“Xu et al. 30 introduced Prov-GigaPath, that was created by tile-level pretraining using DINOv2 24 , followed by slide-level pretraining using a masked autoencoder 27 and LongNet 31 .”
Comprehensive molecular profiling of lung adenocarcinoma
The Cancer Genome Atlas Research Network (2014) · Nature · doi:10.1038/nature13385 · Cited by 5,732 · cited 1×
“Prov-GigaPath was evaluated on 17 genomic prediction tasks and 9 cancer subtyping tasks using both Providence and TCGA 32 data.”
Emerging Properties in Self-Supervised Vision Transformers
Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., & Joulin, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00951 · Cited by 5,638 · cited 2×
“Campanella et al. 26 compared the performance of masked autoencoders 27 (MAE) and DINO 28 using over 3 billion tiles sourced from more than 423,000 pathology slides.”
InfluentialZehir, A., Benayed, R., Shah, R. H., Syed, A., Middha, S., Kim, H. R., Srinivasan, P., Gao, J., Chakravarty, D., Devlin, S. M., et al. (2017) · Nature Medicine · doi:10.1038/nm.4333 · Cited by 3,380
Cited by
Hallmarks of cancer—Then and now, and beyond
Hanahan, D. (2026) · Cell · doi:10.1016/j.cell.2025.12.049 · Cited by 225
A multimodal whole-slide foundation model for pathology
Ding, T., Wagner, S. J., Song, A. H., Chen, R. J., Lu, M. Y., Zhang, A., Vaidya, A. J., Jaume, G., Shaban, M., Kim, A., et al. (2025) · Nature Medicine · doi:10.1038/s41591-025-03982-3 · Cited by 171
“Building on the success of knowledge distillation and masked image modeling 86,87 for patch encoder pretraining 21,22 , TITAN introduces a large-scale pretraining paradigm that leverages millions of high-resolution region-of-interests (ROIs; at 8,192 × 8,192 pixels at 20× magnification) for…”
InfluentialBreen, J., Allen, K., Zucker, K., Godson, L., Orsi, N. M., & Ravikumar, N. (2025) · npj Precision Oncology · doi:10.1038/s41698-025-00799-8 · Cited by 81
“Histopathology foundation models have exhibited impressive performance across diverse tasks 37,46 – 49 including ovarian cancer subtyping 15,24,25 , although analyses have been relatively shallow, without thorough hyperparameter tuning and rigorous statistical comparison of downstream models.”
Application of artificial intelligence and digital tools in cancer pathology
Shaktah, L., Carrero, Z. I., Hewitt, K. J., Gustav, M., Cecchini, M. J., Foersch, S., Berezowska, S., & Kather, J. N. (2025) · The Lancet Digital Health · doi:10.1016/j.landig.2025.100933 · Cited by 50
Uncertainty-aware ensemble of foundation models differentiates glioblastoma from its mimics
Zhao, J., Lin, S.-Y., Attias, R., Mathews, L., Engel, C., Larghero, G., Vremenko, D., Kao, T., Lee, T.-H., Wang, Y.-H., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-64249-6 · Cited by 26
Metadata
- Authors
- Campanella, G., Chen, S., Singh, M., Verma, R., Muehlstedt, S., Zeng, J., Stock, A., Croken, M., Veremis, B., Elmas, A., Shujski, I., Neittaanmäki, N., Huang, K.-L., Kwan, R., Houldsworth, J., Schoenfeld, A. J., & Vanderbilt, C.
- Affiliations
- Icahn School of Medicine at Mount Sinai (US)
- Sahlgrenska University Hospital (SE)
- Memorial Sloan Kettering Cancer Center (US)
- Corresponding author
- Campanella, Gabriele
- Vanderbilt, Chad
- ORCID
- Venue
- Nature Communications
- Published
- 2025-04-17
- Volume
- 16
- Issue
- 1
- Pages
- 3640-3640
- ISSN
- 2041-1723 (ISSN-L)
- PMC
- PMC12003829
- OpenAlex
- W4409488923
- Semantic Scholar
- 7f9ceffdf4ddd8effa16679d8746c6a3844a7dbf
- Cited by
- 126
- References
- 42
- Citations by year
- 2026: 90 · 2025: 34
- FWCI
- 49.74
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc-nd
- creativecommons.org/licenses/by-nc-nd/4.0 (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0 (vor)
- Locations
- Nature Communications (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University) (cc-by)
- Funders
- U.S. Department of Health & Human Services | NIH | U.S. National Library of Medicine (LM013766)
- Grant from the Warren Alpert Foundation through the Warren Alpert Center for Digital and Computational Pathology at Memorial Sloan Kettering Cancer Center
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- pathology foundation models, self-supervised learning, computational pathology, benchmarking, cancer diagnosis, biomarkers
- MeSH terms
- Supervised Machine Learning, Algorithms, Humans, Neoplasms, Benchmarking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A clinical benchmark of public self-supervised pathology foundation models. https://app.cochat.ai/papers/10.1038/s41467-025-58796-1
- The paper
- Campanella, G., Chen, S., Singh, M., Verma, R., Muehlstedt, S., Zeng, J., Stock, A., Croken, M., Veremis, B., Elmas, A., Shujski, I., Neittaanmäki, N., Huang, K.-L., Kwan, R., Houldsworth, J., Schoenfeld, A. J., & Vanderbilt, C. (2025). A clinical benchmark of public self-supervised pathology foundation models. Nature Communications. https://doi.org/10.1038/s41467-025-58796-1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.