Student or educator? Sign up with your school email and get 6 months of Pro free.

The DOI matches its Crossref record; 138 of the 138 references it cites were found in a registry or an index; 63 of the 104 sources its claims cite were checked for support.

A multimodal whole-slide foundation model for pathology

Open access

Abstract

The field of computational pathology has been transformed with recent advances in foundation models that encode histopathology region-of-interests (ROIs) into versatile and transferable feature representations via self-supervised learning. However, translating these advancements to address complex clinical challenges at the patient and slide level remains constrained by limited clinical data in disease-specific cohorts, especially for rare clinical conditions. We propose Transformer-based pathology Image and Text Alignment Network (TITAN), a multimodal whole-slide foundation model pretrained using 335,645 whole-slide images via visual self-supervised learning and vision-language alignment with corresponding pathology reports and 423,122 synthetic captions generated from a multimodal generative AI copilot for pathology. Without any fine-tuning or requiring clinical labels, TITAN can extract general-purpose slide representations and generate pathology reports that generalize to resource-limited clinical scenarios such as rare disease retrieval and cancer prognosis. We evaluate TITAN on diverse clinical tasks and find that it outperforms both ROI and slide foundation models across machine learning settings, including linear probing, few-shot and zero-shot classification, rare cancer retrieval, cross-modal retrieval and pathology report generation. Pretrained using 335,645 whole-slide images, a foundation model is developed to provide representations for slide- and patient-level tasks. It is capable of performing clinical tasks and generating reports even in data-scarce scenarios, such as rare cancer diagnosis and survival prediction, without requiring further fine-tuning.

Overview

CoChat Summary

## TL;DR

TITAN is a multimodal whole-slide foundation model for pathology pretrained on 335,645 WSIs and 182,862 reports, plus 423,122 synthetic ROI captions, to produce transferable slide embeddings and report-generation capability. Across diverse pathology tasks, it outperforms prior slide foundation models in classification, few-shot learning, rare cancer retrieval, cross-modal retrieval, and report generation.

## Key findings

  • TITAN was pretrained on 335,645 whole-slide images from 20 organs, with multimodal alignment using 423,122 synthetic ROI captions and 182,862 slide-report pairs; the abstract also states 423,122 synthetic captions and 335,645 WSIs.
  • On downstream tasks, the paper reports TITAN/TITANV outperforming prior slide encoders across morphological subtyping (14 tasks), grading (3 tasks), molecular classification (39 tasks), and survival prediction (6 tasks).
  • In low-data few-shot learning, TITAN significantly outperformed all other encoders (P < 0.0001); for 16-shot learning it beat CHIEF by 22.4% on TCGA-UT-8K and 18.7% on TCGA-OT, and on 1-shot OT108 it beat ABMIL by 56.7%.
  • For zero-shot classification on 13 subtyping tasks, TITAN outperformed PRISM by +56.52% balanced accuracy on multiclass tasks and +13.8% AUROC on binary tasks; on EBRAINS, balanced accuracy was more than double PRISM’s (+121.9%).
  • For rare-cancer retrieval, TITAN improved over the next best model PRISM by +14.8% in Accuracy@K and +18.1% in MVAcc@K on average; on Rare-Cancer-External, TITAN and TITANV improved over GigaPath by +30.8% / +41.5% in Accuracy@K and +31.2% / +26.7% in MVAcc@K.
  • For report generation on 10,108 slide-report pairs from TCGA-Slide-Reports, TITAN outperformed PRISM by 161% on average across METEOR, ROUGE, and BLEU; for cross-modal retrieval, it improved by +10.5% on report-to-slide and +20.5% on slide-to-report Recall@K.

## Methods

  • Design: Two-stage multimodal continual pretraining on top of a vision-only whole-slide model: stage 1 SSL on ROI feature grids with iBOT, stage 2 ROI-caption alignment using synthetic PathChat captions, and stage 3 WSI-report alignment using clinical pathology reports; the multimodal stages use a CoCa-style image encoder, text encoder, and text decoder.
  • Data/sample: Pretraining used Mass-340K with 335,645 WSIs and 182,862 medical reports; stage 2 used 423,122 ROIs of 8,192 × 8,192 pixels, and stage 3 used 182,862 slide-report pairs. Downstream evaluation spans multiple public and in-house datasets, including TCGA-UT-8K (25,495 ROIs), TCGA-OT (11,186 WSIs), TCGA-Slide-Reports (10,108 WSIs with paired reports), and rare-cancer retrieval sets.
  • Main analysis: The paper evaluates TITAN/TITANV with linear probing, k-NN probing, few-shot classification, slide retrieval, cross-modal retrieval, zero-shot classification, report generation, and survival analysis, comparing against baselines including PRISM, GigaPath, CHIEF, ABMIL, and mean pooling.

## Limitations

  • The authors state that pretraining on 8k × 8k region crops and extrapolating with ALiBi to the entire WSI may still not capture the full contextual information.
  • They note that slide models may encode nonbiological features such as tissue processing sites and scanners, which can affect translational impact.
  • They state that clinical report processing remains challenging for vision-language alignment and that Mass-340K contains fewer slides than some other pretraining datasets.
  • Evident from the text: several reported gains are based on internal retrospective data and curated benchmark datasets, so generalization beyond the evaluated cohorts is only partially tested.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41591-025-03982-3

Cites

138 of 138 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Overview of TITAN. a, Tissue site distribution of Mass-340K used for TITANV pretraining (stage 1). Mass-340K includes 335,645 WSIs across 20 organs with a mix of tissue sections stained with H&E (89.7%), IHC (7.9%), special stains (2.3%) and others (0.1%) or a mix of neoplastic (70.0%), tissue damage response (8.4%), normal (4.7%), inflammatory (3.4%) and others (13.5%) scanned with diverse scanner types. TITAN pretraining (stages 2 and 3) uses a subset of Mass-340K with paired captions and medical reports. b–d, Block diagram of TITANV pretraining. b, TITAN uses a ViT to encode a WSI into a slide embedding. c, TITANV (stage 1) is pretrained using SSL with student–teacher knowledge distillation. d, TITAN (stage 2 and 3) is pretrained using vision-language modeling, first by aligning the slide embedding with synthetic captions (stage 2) and then with medical reports (stage 3). e, UMAP visualization of TCGA slide embeddings obtained with TITAN, color-coded by organ. UMAP, uniform manifold approximation and projection; px, pixel.
Fig. 1 Overview of TITAN. a, Tissue site distribution of Mass-340K used for TITANV pretraining (stage 1). Mass-340K includes 335,645 WSIs across 20 organs with a mix of tissue sections stained with H&E (89.7%), IHC (7.9%), special stains (2.3%) and others (0.1%) or a mix of neoplastic (70.0%), tissue damage response (8.4%), normal (4.7%), inflammatory (3.4%) and others (13.5%) scanned with diverse scanner types. TITAN pretraining (stages 2 and 3) uses a subset of Mass-340K with paired captions and medical reports. b–d, Block diagram of TITANV pretraining. b, TITAN uses a ViT to encode a WSI into a slide embedding. c, TITANV (stage 1) is pretrained using SSL with student–teacher knowledge distillation. d, TITAN (stage 2 and 3) is pretrained using vision-language modeling, first by aligning the slide embedding with synthetic captions (stage 2) and then with medical reports (stage 3). e, UMAP visualization of TCGA slide embeddings obtained with TITAN, color-coded by organ. UMAP, uniform manifold approximation and projection; px, pixel.CC BY-NC-ND · PubMed Central
Fig. 2 TITAN evaluation. a, Impact of pretraining data size on TITANV and baselines across four challenging subtyping tasks. TITANV is pretrained with 12.5%, 25%, 50% and 100% of Mass-340K. b, The average performance of the four tasks against the number of parameters. c, Linear probe evaluation of TITAN and baselines on morphological classification, molecular status and survival prediction tasks. The mean pooling baseline uses the same patch encoder as TITAN (CONCHv1.5). Multiclass tasks are evaluated with balanced accuracy, binary tasks with AUROC and survival tasks with the concordance index. For external cohorts (DHMC, CPTAC), the classifier is trained on the corresponding TCGA cohort. All error bars represent s.d. based on bootstrapping (n = 1,000) or k-fold evaluation (k = 5). d, Ablation for positional encoding, number of transformer layers and inclusion of vision-pretraining stage. The performance is averaged across the four subtyping tasks. e, Change in performance of slide encoders averaged across the four subtyping tasks for different learning paradigms. For mean pooling and ABMIL, the respective patch encoder for each framework is used. PRISM fine-tuning is not evaluated as the fine-tuning recipes are not provided. f, Linear probe few-shot performance using K shots, K ∈ {1, 2, 4, 8, 16}, comparing baselines and ABMIL with CONCHv1.5. For each setting, 50 runs were performed. The center of each box plot (horizontal line) represents the median, with whiskers extending to data points within 1.5× the interquartile range. Statistical significance was assessed by fitting generalized linear mixed-effects model and two-sided Wald z test on the fitted model. Significance shown with respect to TITAN. P values for nonsignificant results are shown. **P ≤ 0.01, ***P ≤ 0.001, ****P ≤ 0.0001. C, number of classes; Ft., fine-tune.
Fig. 2 TITAN evaluation. a, Impact of pretraining data size on TITANV and baselines across four challenging subtyping tasks. TITANV is pretrained with 12.5%, 25%, 50% and 100% of Mass-340K. b, The average performance of the four tasks against the number of parameters. c, Linear probe evaluation of TITAN and baselines on morphological classification, molecular status and survival prediction tasks. The mean pooling baseline uses the same patch encoder as TITAN (CONCHv1.5). Multiclass tasks are evaluated with balanced accuracy, binary tasks with AUROC and survival tasks with the concordance index. For external cohorts (DHMC, CPTAC), the classifier is trained on the corresponding TCGA cohort. All error bars represent s.d. based on bootstrapping (n = 1,000) or k-fold evaluation (k = 5). d, Ablation for positional encoding, number of transformer layers and inclusion of vision-pretraining stage. The performance is averaged across the four subtyping tasks. e, Change in performance of slide encoders averaged across the four subtyping tasks for different learning paradigms. For mean pooling and ABMIL, the respective patch encoder for each framework is used. PRISM fine-tuning is not evaluated as the fine-tuning recipes are not provided. f, Linear probe few-shot performance using K shots, K ∈ {1, 2, 4, 8, 16}, comparing baselines and ABMIL with CONCHv1.5. For each setting, 50 runs were performed. The center of each box plot (horizontal line) represents the median, with whiskers extending to data points within 1.5× the interquartile range. Statistical significance was assessed by fitting generalized linear mixed-effects model and two-sided Wald z test on the fitted model. Significance shown with respect to TITAN. P values for nonsignificant results are shown. **P ≤ 0.01, ***P ≤ 0.001, ****P ≤ 0.0001. C, number of classes; Ft., fine-tune.CC BY-NC-ND · PubMed Central
Fig. 3 Visual-language evaluation of TITAN. a, A schematic for zero-shot evaluation. The query slide is classified by identifying the closest text prompt embedding in the slide embedding space. b, Zero-shot performance of TITAN and PRISM. All multiclass tasks are evaluated with balanced accuracy and binary tasks are evaluated with AUROC. All error bars represent s.d. based on bootstrapping (n = 1,000). Dashed lines represent average performance for respective models (red, TITAN; teal, PRISM) c, Ablation study comparing different pretraining strategies, and assessed with zero-shot performance averaged across TCGA-UT-8K, TCGA-OT, OT108 and EBRAINS. Evaluations are based on the percentage changes of balanced accuracy from the reference zero-shot performance of TITAN. d, Report-generation evaluation on TCGA-Slide-Reports, and evaluated using METEOR, ROUGE and BLEU. All error bars represent s.d. based on bootstrapping (n = 1,000). e, TCGA examples of generated reports of TITAN and PRISM, with the corresponding clinical reports. Additional examples of generated reports are available in Extended Data Fig. 7. Statistical significance was assessed by fitting a generalized linear mixed-effects model and performing a two-sided Wald z test on the fitted model. Significance shown with respect to TITAN. ****P ≤ 0.0001.
Fig. 3 Visual-language evaluation of TITAN. a, A schematic for zero-shot evaluation. The query slide is classified by identifying the closest text prompt embedding in the slide embedding space. b, Zero-shot performance of TITAN and PRISM. All multiclass tasks are evaluated with balanced accuracy and binary tasks are evaluated with AUROC. All error bars represent s.d. based on bootstrapping (n = 1,000). Dashed lines represent average performance for respective models (red, TITAN; teal, PRISM) c, Ablation study comparing different pretraining strategies, and assessed with zero-shot performance averaged across TCGA-UT-8K, TCGA-OT, OT108 and EBRAINS. Evaluations are based on the percentage changes of balanced accuracy from the reference zero-shot performance of TITAN. d, Report-generation evaluation on TCGA-Slide-Reports, and evaluated using METEOR, ROUGE and BLEU. All error bars represent s.d. based on bootstrapping (n = 1,000). e, TCGA examples of generated reports of TITAN and PRISM, with the corresponding clinical reports. Additional examples of generated reports are available in Extended Data Fig. 7. Statistical significance was assessed by fitting a generalized linear mixed-effects model and performing a two-sided Wald z test on the fitted model. Significance shown with respect to TITAN. ****P ≤ 0.0001.CC BY-NC-ND · PubMed Central
Fig. 4 Retrieval capabilities of TITAN. a, Slide retrieval results on rare cancer retrieval tasks assessed with Accuracy@K, with K = {1, 3, 5}. Rare-Cancer (internal rare cancer cohort) consists of TCGA, EBRAINS and the MGB internal cohort, with 43 rare and 143 common cancer types for a total of 186 classes. Rare-Cancer-Public (public rare cancer cohort) consists of TCGA and EBRAINS only, with 29 rare and 98 common cancer types for a total of 127 classes. Rare-Cancer-External consists of 12 rare cancer types for the ovary and soft tissue, curated at Kanagawa Cancer Center Hospital, Japan. b, Example of rare cancer retrieval on Rare-Cancer with the query slide and four representative retrieved slides. The number indicates the cosine similarity between the query and the retrieved slide. Additional examples of rare cancer retrieval are available in Extended Data Fig. 8. c, Slide retrieval results on five subtyping tasks. Mean represents the average performance across three shots. d, Report-to-slide and slide-to-report cross-modal retrieval performance assessed with Recall@K, with K = {1, 3, 5, 10} on TCGA cohort of 10,108 pairs of WSIs and reports for TITAN and PRISM. Mean represents the average performance across four shots. All error bars represent s.d. based on bootstrapping (n = 1,000). Statistical significance was assessed using TITAN by the fitting of a generalized linear mixed-effects model and a two-sided Wald z test on the fitted model. Significance shown with respect to TITAN. P values for nonsignificant results are shown. **P ≤ 0.01, ***P ≤ 0.001, ****P ≤ 0.0001.
Fig. 4 Retrieval capabilities of TITAN. a, Slide retrieval results on rare cancer retrieval tasks assessed with Accuracy@K, with K = {1, 3, 5}. Rare-Cancer (internal rare cancer cohort) consists of TCGA, EBRAINS and the MGB internal cohort, with 43 rare and 143 common cancer types for a total of 186 classes. Rare-Cancer-Public (public rare cancer cohort) consists of TCGA and EBRAINS only, with 29 rare and 98 common cancer types for a total of 127 classes. Rare-Cancer-External consists of 12 rare cancer types for the ovary and soft tissue, curated at Kanagawa Cancer Center Hospital, Japan. b, Example of rare cancer retrieval on Rare-Cancer with the query slide and four representative retrieved slides. The number indicates the cosine similarity between the query and the retrieved slide. Additional examples of rare cancer retrieval are available in Extended Data Fig. 8. c, Slide retrieval results on five subtyping tasks. Mean represents the average performance across three shots. d, Report-to-slide and slide-to-report cross-modal retrieval performance assessed with Recall@K, with K = {1, 3, 5, 10} on TCGA cohort of 10,108 pairs of WSIs and reports for TITAN and PRISM. Mean represents the average performance across four shots. All error bars represent s.d. based on bootstrapping (n = 1,000). Statistical significance was assessed using TITAN by the fitting of a generalized linear mixed-effects model and a two-sided Wald z test on the fitted model. Significance shown with respect to TITAN. P values for nonsignificant results are shown. **P ≤ 0.01, ***P ≤ 0.001, ****P ≤ 0.0001.CC BY-NC-ND · PubMed Central
Extended Data Fig. 1 Examples of TCGA-UT-8K dataset. Examples of TCGA-UT-8K, which are ROIs of 8,192 × 8,192 pixels selected by the pathologists. The green contours illustrate the cancer region annotations, with the red number indicating the ROI index within a given TCGA slide.
Extended Data Fig. 1 Examples of TCGA-UT-8K dataset. Examples of TCGA-UT-8K, which are ROIs of 8,192 × 8,192 pixels selected by the pathologists. The green contours illustrate the cancer region annotations, with the red number indicating the ROI index within a given TCGA slide.CC BY-NC-ND · PubMed Central
Extended Data Fig. 2 Linear probe results for molecular classification tasks. (a) Linear models are fitted and evaluated on binary molecular status predictions for BCNB and MUT-HET. We observe that TITAN consistently performs best with + 0.9% on BCNB and MUT-HET, +1.7% on TCGA, and +3.7% on internal molecular classification of BRCA and LUAD, in averaged AUROC scores over the next best model PRISM. (b) Linear models are fitted and evaluated on five-fold splits on TCGA. (c) The same models are evaluated on the corresponding external datasets from CPTAC and EBRAINS. (d) 6-level ER and PR prediction from Mass General Hospital (MGH) and 3-level PD-L1 prediction, all from immunohistochemistry (IHC) slides. (e) Molecular classification tasks for BRCA and LUAD from Mass General Brigham (MGB). All error bars represent standard deviations based on bootstrapping (n = 1,000) or k-fold evaluation (k = 5).
Extended Data Fig. 2 Linear probe results for molecular classification tasks. (a) Linear models are fitted and evaluated on binary molecular status predictions for BCNB and MUT-HET. We observe that TITAN consistently performs best with + 0.9% on BCNB and MUT-HET, +1.7% on TCGA, and +3.7% on internal molecular classification of BRCA and LUAD, in averaged AUROC scores over the next best model PRISM. (b) Linear models are fitted and evaluated on five-fold splits on TCGA. (c) The same models are evaluated on the corresponding external datasets from CPTAC and EBRAINS. (d) 6-level ER and PR prediction from Mass General Hospital (MGH) and 3-level PD-L1 prediction, all from immunohistochemistry (IHC) slides. (e) Molecular classification tasks for BRCA and LUAD from Mass General Brigham (MGB). All error bars represent standard deviations based on bootstrapping (n = 1,000) or k-fold evaluation (k = 5).CC BY-NC-ND · PubMed Central
Extended Data Fig. 3 UMAP of slide embedding space for TCGA-OT. UMAP visualization of slide embeddings in TCGA-OT cohort (n = 11,186) for all slide encoder baselines, including TITAN and TITANV, color-coded by different organs for visual decluttering.
Extended Data Fig. 3 UMAP of slide embedding space for TCGA-OT. UMAP visualization of slide embeddings in TCGA-OT cohort (n = 11,186) for all slide encoder baselines, including TITAN and TITANV, color-coded by different organs for visual decluttering.CC BY-NC-ND · PubMed Central
Extended Data Fig. 4 UMAP of TCGA-OT slide representations (n = 11,186) from all slide encoders. The first row is labeled by OncoTreeCode, the second row by OncoTreeSiteCode, and the third row by submission site. Clustering metrics, mean local diversity (mLD), adjusted rand index (ARI), and normalized mutual information (NMI), are computed for all labels. Note that CHIEF includes TCGA in the pretraining dataset.
Extended Data Fig. 4 UMAP of TCGA-OT slide representations (n = 11,186) from all slide encoders. The first row is labeled by OncoTreeCode, the second row by OncoTreeSiteCode, and the third row by submission site. Clustering metrics, mean local diversity (mLD), adjusted rand index (ARI), and normalized mutual information (NMI), are computed for all labels. Note that CHIEF includes TCGA in the pretraining dataset.CC BY-NC-ND · PubMed Central
Extended Data Fig. 5 Attention heatmaps of TITAN. Exemplar attention heatmaps for three Transformer attention heads of TITAN (head #4, #10, #11) are shown across three different TCGA WSIs. Out of the 12 attention heads, we find that most attention heads focus on dense tumor regions, with certain attention heads such as head #10 focusing on tumor-adjacent stroma and head #11 focusing on non-tumor areas. Across different cancer types, while head #11 attends to tissue-specific morphologies such as peritumoral stroma in the thymoma WSI and the tumor-adjacent stroma and ducts in the BRCA WSI, we do observe that general morphological patterns such as tumor/non-tumor are conserved across tissue types.
Extended Data Fig. 5 Attention heatmaps of TITAN. Exemplar attention heatmaps for three Transformer attention heads of TITAN (head #4, #10, #11) are shown across three different TCGA WSIs. Out of the 12 attention heads, we find that most attention heads focus on dense tumor regions, with certain attention heads such as head #10 focusing on tumor-adjacent stroma and head #11 focusing on non-tumor areas. Across different cancer types, while head #11 attends to tissue-specific morphologies such as peritumoral stroma in the thymoma WSI and the tumor-adjacent stroma and ducts in the BRCA WSI, we do observe that general morphological patterns such as tumor/non-tumor are conserved across tissue types.CC BY-NC-ND · PubMed Central
Extended Data Fig. 6 Ablation experiments on different learning paradigms. Change in balanced accuracy performance for several learning paradigms on four subtyping tasks with respect to the linear probe. The baselines include mean pooling, ABMIL, linear probe, and finetuned from pretrained or randomly initialized weights. The number under each task name indicates the linear probe performance. TITAN-L represents the variation of TITAN without vision-pretraining. For mean pooling and ABMIL, we use the respective patch encoder for each framework, as specified under each slide encoder name. Finetuning results are not provided for PRISM, as the finetuning recipes were not available.
Extended Data Fig. 6 Ablation experiments on different learning paradigms. Change in balanced accuracy performance for several learning paradigms on four subtyping tasks with respect to the linear probe. The baselines include mean pooling, ABMIL, linear probe, and finetuned from pretrained or randomly initialized weights. The number under each task name indicates the linear probe performance. TITAN-L represents the variation of TITAN without vision-pretraining. For mean pooling and ABMIL, we use the respective patch encoder for each framework, as specified under each slide encoder name. Finetuning results are not provided for PRISM, as the finetuning recipes were not available.CC BY-NC-ND · PubMed Central
Extended Data Fig. 7 Examples of generated reports. TCGA examples of generated reports of TITAN and PRISM, with the corresponding clinical reports.
Extended Data Fig. 7 Examples of generated reports. TCGA examples of generated reports of TITAN and PRISM, with the corresponding clinical reports.CC BY-NC-ND · PubMed Central
Extended Data Fig. 8 Rare cancer retrieval with TITAN. (a)–(c) Examples of slide retrieval on Rare-Cancer. The number for each retrieved slide represents the cosine similarity between the query and the retrieved slide. The retrieved slides with high similarity are either of the same diagnostic label or from the same organ as the query slide. (a) Thyroid (THFO) query (b) Pleura (PLBMESO) query (c) Adrenal gland (ACC) query.
Extended Data Fig. 8 Rare cancer retrieval with TITAN. (a)–(c) Examples of slide retrieval on Rare-Cancer. The number for each retrieved slide represents the cosine similarity between the query and the retrieved slide. The retrieved slides with high similarity are either of the same diagnostic label or from the same organ as the query slide. (a) Thyroid (THFO) query (b) Pleura (PLBMESO) query (c) Adrenal gland (ACC) query.CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

28 Valid · 1 Not valid · 7 Not checked · 92 Own results and interpretations

39 more claims · 5 with their cited sources checked

Create a free account to see all 128 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 53
  • Probably not a substantial claim. 39

Create a free account to see these claims.

Citations

Cites

Show all 112 references

Research works printed but not linked: 16

  1. Juyal, D. et al. Pluto: pathology-universal transformer. In ICML 2024 Workshop on Foundation Models in the Wild. https://openreview.net/forum?id=7yn50e6tVX (OpenReview, 2024).
  2. Shao, D. et al. Do multiple instance learning models transfer? In Proc. 42nd International Conference on Machine Learning, (eds Singh, A. et al.) 54219–54238 (PMLR, 2025).
  3. Ilse, M., Tomczak, J. M. & Welling, M. Attention-based deep multiple instance learning. In Proc. 35th International Conference on Machine Learning (eds Dy, J. & Krause, A.) 2127–2136 (PMLR, 2018).
  4. Xiang, J. & Zhang, J. Exploring low-rank property in multiple instance learning for whole slide image classification. Proceedings of the 11th International Conference on Learning Representations https://openreview.net/forum?id=01KmhBsEPFO (OpenReview, 2023).
  5. Ahmed, F. et al. PathAlign: a vision-language model for whole slide images in histopathology. In Proc. MICCAI Workshop on Computational Pathology (eds Ciompi, F. et al.) 72–108 (PMLR, 2024).
  6. Zhou, J. et al. iBOT: image BERT pre-training with online tokenizer. In Proc. International Conference on Learning Representations https://openreview.net/forum?id=ydopy-e6Dg (OpenReview, 2022).
  7. Oquab, M. et al. Dinov2: learning robust visual features without supervision. Trans. Mach. Learn. Res. https://openreview.net/forum?id=a68SUt6zFt (2024).
  8. Dosovitskiy, A. et al. An image is worth 16 × 16 words: transformers for image recognition at scale. In Proc. International Conference on Learning Representations https://openreview.net/forum?id=YicbFdNTTy (OpenReview, 2021).
  9. Press, O., Smith, N. & Lewis, M. Train short, test long: attention with linear biases enables input length extrapolation. In Proc. International Conference on Learning Representations https://openreview.net/forum?id=R8sQPpGCv0 (OpenReview, 2022).
  10. Yu J. et al. CoCa: contrastive captioners are image-text foundation models Trans. Mach. Learn. Res. https://openreview.net/forum?id=Ee277P3AYC (2022).
  11. Song, A. H. et al. Multimodal prototyping for cancer survival prediction. In Proc. 41st International Conference on Machine Learning (eds Salakhutdinov, R. et al.) 46050–46073 (PMLR, 2024).
  12. Snell, J., Swersky, K. & Zemel, R. Prototypical networks for few-shot learning. In Proc. 31st International Conference on Neural Information Processing Systems (eds Guyon, I. et al.) 4080–4090 (Curran Associates, 2017).
  13. Radford, A. et al. Learning transferable visual models from natural language supervision. In Proc. 38th International Conference on Machine Learning (eds Meila, M. & Zhang, T.) 8748–8763 (PMLR, 2021).
  14. Banerjee, S. & Lavie, A. METEOR: an automatic metric for MT evaluation with improved correlation with human judgments. In Proc. ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization (ed Goldstein, J. et al) 65–72 (Association for Computational Linguistics, 2005).
  15. Lin, C.-Y. Rouge: a package for automatic evaluation of summaries. In Text Summarization Branches Out: Proceedings of the ACL-04 Workshop 74–81 (Association for Computational Linguistics, 2004).
  16. Jaegle, A. et al. Perceiver: general perception with iterative attention. In Proc. 38th International Conference on Machine Learning (eds Meila, M. & Zhang, T.) 4651–4664 (PMLR, 2021).

Cited by

See all 171

Metadata

Authors
Ding, T., Wagner, S. J., Song, A. H., Chen, R. J., Lu, M. Y., Zhang, A., Vaidya, A. J., Jaume, G., Shaban, M., Kim, A., Williamson, D. F. K., Robertson, H., Chen, B., Almagro-Pérez, C., Doucet, P., Sahai, S., Chen, C., Chen, C. S., Komura, D., Kawabe, A., Ochi, M., Sato, S., Yokose, T., Miyagi, Y., Ishikawa, S., Gerber, G., Peng, T., Le, L. P., & Mahmood, F.
Affiliations
  • Broad Institute (US)
  • Harvard University (US)
  • The University of Tokyo (JP)
  • Kanagawa Cancer Center (JP)
  • Helmholtz Munich (DE)
Corresponding author
  • Le, Long Phi
  • Mahmood, Faisal
Venue
Nature Medicine
Published
2025-11
Volume
31
Issue
11
Pages
3749-3761
ISSN
  • 1078-8956 (ISSN-L)
  • 1546-170X
OpenAlex
W4415913530
Cited by
171
References
112
Citations by year
2026: 154 · 2025: 16
FWCI
67.65
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Medicine (publishedVersion, cc-by-nc-nd)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
Funders
National Institute of General Medical Sciences (R35GM149270, R35GM138216)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
Keywords
whole slide images, foundation models, computational pathology, vision-language models, self-supervised learning, multimodal generative AI, cancer prognosis, cross-modal retrieval, zero-shot classification, few-shot classification, linear probing, histopathology
MeSH terms
Machine Learning, Humans, Image Processing, Computer-Assisted, Neoplasms, Pathology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A multimodal whole-slide foundation model for pathology. https://app.cochat.ai/papers/10.1038/s41591-025-03982-3
The paper
Ding, T., Wagner, S. J., Song, A. H., Chen, R. J., Lu, M. Y., Zhang, A., Vaidya, A. J., Jaume, G., Shaban, M., Kim, A., Williamson, D. F. K., Robertson, H., Chen, B., Almagro-Pérez, C., Doucet, P., Sahai, S., Chen, C., Chen, C. S., Komura, D., . . . Mahmood, F. (2025). A multimodal whole-slide foundation model for pathology. Nature Medicine. https://doi.org/10.1038/s41591-025-03982-3

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.