The DOI matches its Crossref record; 80 of the 88 references it cites were found in a registry or an index; none of the 2 sources its claims cite is checked for support yet.
Pan-cancer integrative histology-genomic analysis via multimodal deep learning
Open accessAbstract
The rapidly emerging field of computational pathology has demonstrated promise in developing objective prognostic models from histology images. However, most prognostic models are either based on histology or genomics alone and do not address how these data sources can be integrated to develop joint image-omic prognostic models. Additionally, identifying explainable morphological and molecular descriptors from these models that govern such prognosis is of interest. We use multimodal deep learning to jointly examine pathology whole-slide images and molecular profile data from 14 cancer types. Our weakly supervised, multimodal deep-learning algorithm is able to fuse these heterogeneous modalities to predict outcomes and discover prognostic features that correlate with poor and favorable outcomes. We present all analyses for morphological and molecular correlates of patient prognosis across the 14 cancer types at both a disease and a patient level in an interactive open-access database to allow for further exploration, biomarker discovery, and feature assessment.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The authors built a weakly supervised multimodal deep learning model that combines pathology whole-slide images with molecular profiles to predict cancer survival across 14 TCGA cancer types. They report improved risk stratification over unimodal models for most cancers and provide an interactive platform for exploring image and molecular explanations.
Key findings
- Multimodal fusion achieved an overall cross-validated c-index of 0.645, compared with 0.585 for image-only AMIL and 0.607 for molecular-only SNN; Survival AUC was 0.662 for multimodal, versus 0.616 for SNN and 0.598 for AMIL.
- The multimodal model had the highest c-index in 12 of 14 cancer types, with statistically significant binary risk stratification in 10 of 14. KIRP showed the largest gain, reaching a c-index of 0.816 versus 0.539 for AMIL and 0.779 for SNN.
- Whole-slide images contributed an average of 16.8% of input attributions across cancers, but this varied widely, including 55.1% for UCEC, 32.4% for PAAD, and 5.8% for LUSC.
- Interpretability analyses linked low-risk attention regions to greater immune-cell presence and high-risk regions to tumor presence, higher grade, or invasion; lymphocyte fractions differed significantly in 8 of 14 cancers and tumor-cell fractions in 6 of 14. TIL presence was significantly higher in predicted low-risk patients for 9 of 14 cancers.
Methods
- The study used 6,592 diagnostic H&E whole-slide images from 5,720 patients across 14 TCGA cancer types, with paired mutation, copy-number, and RNA-Seq data where available.
- The multimodal model combined an attention-based multiple-instance-learning branch for images, a self-normalizing network for molecular features, and a Kronecker-product fusion layer. Training used patient-level five-fold cross-validation with 80/20 train-test splits, and performance was assessed with c-index, Survival AUC, Kaplan-Meier curves, and logrank tests.
- Interpretability used attention heatmaps for image regions and Integrated Gradients for molecular features, with HoVerNet-based cell segmentation to quantify cell types and TILs in high-attention regions. Results were made available through the PORPOISE platform.
Limitations
- The paper states that the approach can identify “what” and “where” but not always “why,” so discovered features require further human quantification and biological interpretation.
- The paper notes that attention sometimes shifted away from tumor regions toward stroma or normal tissue, and speculates this may reflect training dynamics favoring the simpler modality; molecular data are bulk and lack spatial resolution.
- Evident from the text: the study is retrospective and TCGA-based, with no independent external validation; SKCM includes metastatic cases that may optimistically bias performance, and data-alignment criteria were relaxed for SKCM and UCEC due to missingness.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
10 Own result · 7 Own method · 1 Sources not found · 1 No full text of the sources
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, however, model performances were found to improve following multimodal integration for almost all cancer types”
§ Pan-Cancer Integrative Histology-Genomic Analysis via Multimodal Deep Learning > Results > Multimodal Integration Improves Patient Risk Stratification
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Across the 14 cancer types, MMF achieved an overall c-Index of 0.645, whereas AMIL and SNN had overall c-Indices of 0.585 and 0.607 respectively. On Survival AUC, similar improvement in multimodal integration was found with an overall performance of 0.662 in comparison to 0.616 and 0.598 in SNN and AMIL respectively (Table S2).”
§ Pan-Cancer Integrative Histology-Genomic Analysis via Multimodal Deep Learning > Results > Multimodal Integration Improves Patient Risk Stratification
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In one-versus-all model performance comparisons on individual cancer types, MMF achieved the highest c-Index on 12 out of 14 (12/14) cancer types, with models for 10/14 cancer types demonstrating statistical significance in binary patient stratification (Figure 2A,2B, 2C).”
§ Pan-Cancer Integrative Histology-Genomic Analysis via Multimodal Deep Learning > Results > Multimodal Integration Improves Patient Risk Stratification
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our weakly-supervised, multimodal deep learning algorithm is able to fuse these heterogeneous modalities to predict outcomes and discover prognostic features that correlate with poor and favorable outcomes.”
§ Pan-Cancer Integrative Histology-Genomic Analysis via Multimodal Deep Learning > Abstract
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we found that 9 out of 14 cancer types had a statistically significant increase in TIL presence amongst patients with predicted low-risk, indicating that model attention was localized to more immune-hot regions.”
§ Pan-Cancer Integrative Histology-Genomic Analysis via Multimodal Deep Learning > Results > Immune response as a prognostic marker
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, we note that for MMF models evaluated on UCEC, WSIs contributed to 55.1% of all input attributions, which corroborates with high AMIL performance on this cancer type.”
§ Pan-Cancer Integrative Histology-Genomic Analysis via Multimodal Deep Learning > Results > Multimodal Integration Improves Patient Risk Stratification
In the paper
“Across all cancer types, gradient-based interpretability was able to identify many well-known oncogenes and immune-related genes established in existing biomedical literature and used in targeted molecular therapies (Uhlen et al.,2017).”
§ Pan-Cancer Integrative Histology-Genomic Analysis via Multimodal Deep Learning > Results > Multimodal Interpretability for Joint Image-Omic Biomarker Discovery
What the sources say
- Not checked — no full text of the sourcecites Uhlen et al.,2017 source
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For cancer types in our dataset that have been FDA approved immune checkpoint inhibitor therapies, TIL presence was used as model explanations for favorable prognosis in BRCA (P-Value, 5.17×10−11, t-test), HNSC (P-Value, 1.97×10−18, t-test), KIRC (P-Value, 1.86×10−3, t-test), LIHC (P-Value, 2.54 × 10−17, t-test), LUAD (P-Value, 1.54 × 10−21, t-test), LUSC (P-Value, 2.92 × 10−12, t-test), PAAD (P-Value, 3.77×10−6, t-test), STAD (P-Value, 1.09×10−9, t-test), and SKCM (P-Value, 6.29×10−10, t-test).”
§ Pan-Cancer Integrative Histology-Genomic Analysis via Multimodal Deep Learning > Results > Immune response as a prognostic marker
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“WSIs on average accounted for 16.8% of input attributions in MMF for all cancer types, which suggests that molecular features drive most of the risk prediction in MMF (Figure 2C, Table S3).”
§ Pan-Cancer Integrative Histology-Genomic Analysis via Multimodal Deep Learning > Results > Multimodal Integration Improves Patient Risk Stratification
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Across all cancer types, we generally observed that high attention regions in low-risk patients corresponded with greater immune cell presence and lower tumor grade than that of high-risk patients, with 8/14 cancer types demonstrating statistically significant differences in lymphocyte cell fractions in high attention regions (Figure 6F, S3-10).”
§ Pan-Cancer Integrative Histology-Genomic Analysis via Multimodal Deep Learning > Results > Multimodal Interpretability for Joint Image-Omic Biomarker Discovery
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Notably, a vital explainability component was incorporated with heatmaps for histopathology and SHAP values for genomic markers.”
From Navigating the landscape of multimodal AI in medicine: A scoping review on technical challenges and clinical applications (2025) · cites this paper as Chen et al., 2022
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“and integrated with genomic features for prognosis prediction [20].”
From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [20]
The cited source says what the claim attributes to it.
In the citing paper
“In the PORPOISE framework, Chen et al. introduced a method to integrate WSIs with genomic, and molecular profiles for prognosis prediction and helped identify poor prognosis-related joint biomarkers [20].”
From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [20]
The cited source says what the claim attributes to it.
In the citing paper
“Chen et al. (2022) developed a multimodal deep learning model to jointly examine pathology whole-slide images and molecular profile data from 14 cancer types to predict outcomes and discover prognostic features correlating with poor and favorable outcomes.”
From Navigating the landscape of multimodal AI in medicine: A scoping review on technical challenges and clinical applications (2025) · cites this paper as Chen et al., 2022
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“The fundamental criterion for evaluating a medical AI agent is its precision in generating diagnostic and treatment recommendations.80,100,118,119,120”
From A foundational architecture for AI agents in healthcare (2025) · cites this paper as [120]
Experimental — accuracy not yet measured
In the citing paper
“Additionally, ML and AI allow for integration of multiple data types, including EHRs, genomic data, and environmental exposures, and of multimodal data, such as kinds of information from individual’s various types of records, thus providing a comprehensive perspective of the determinants of health outcomes across the different stages of life course [32].”
From Integrating machine learning and artificial intelligence in life-course epidemiology: pathways to innovative public health solutions (2024) · cites this paper as [32]
In the citing paper
“Multimodal AI systems have been evaluated for various applications in precision oncology, such as outcome predictions14,15.”
From Large language models and multimodal foundation models for precision oncology (2024) · cites this paper as [14]
In the citing paper
“there are increasing efforts of using digital pathology images to guide treatment selection by predicting patient prognosis (73)(74)(75) and/or existing treatment biomarkers (76-79) using AI.”
From Artificial Intelligence in Oncology: Current Landscape, Challenges, and Future Directions (2024) · cites this paper as [75]
In the citing paper
“For instance, Chen and colleagues (75) developed an approach to fuse genomic data and histopathology images to predict survival in a pan-cancer cohort.”
From Artificial Intelligence in Oncology: Current Landscape, Challenges, and Future Directions (2024) · cites this paper as [75]
In the citing paper
“Recently, several such AI systems have been introduced6. Examples include models that analyze radiology images together with clinical data7 or integrate information from histopathology with genomic8 or text-based information9.”
From Development and validation of an autonomous artificial intelligence agent for clinical decision-making in oncology (2025) · cites this paper as [8]
In the citing paper
“However, learning the spatial relevance of non-imaging data such as bulk gene expression is not straightforward using late fusion.”
From Graph attention-based fusion of pathology images and gene expression for prediction of cancer survival (2024) · cites this paper as [5]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Pan-Cancer Integrative Histology-Genomic Analysis via Multimodal Deep Learning > Results > Multimodal Interpretability for Joint Image-Omic Biomarker Discovery
“the top 25% (high-risk group) and bottom 25% (low-risk group) of predicted patient risks for each cancer type, which reflect favorable and poor cancer prognosis respectively.”
§ Pan-Cancer Integrative Histology-Genomic Analysis via Multimodal Deep Learning > Star Methods > Method Details
“WSIs with predicted high-risk scores would have high attention on high tumor grade or tumor invasion regions used in explaining poor survival, whereas WSIs with predicted low risk scores would have high attention on low tumor grade or lymphocyte-containing regions used in explaining favorable survival.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Pan-Cancer Integrative Histology-Genomic Analysis via Multimodal Deep Learning > Results > Deep learning-based Multimodal Integration
“Pathology-Omics Research Platform for Integrative Survival Estimation (PORPOISE)”
§ Pan-Cancer Integrative Histology-Genomic Analysis via Multimodal Deep Learning > Star Methods > Method Details
“PORPOISE (Pathology-Omics Research Platform for Integrated Survival Estimation)”
Citations
Cites
Subramanian, A., Tamayo, P., Mootha, V. K., Mukherjee, S., Ebert, B. L., Gillette, M. A., Paulovich, A., Pomeroy, S. L., Golub, T. R., Lander, E. S., et al. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0506580102 · Cited by 53,625
PD-1 and PD-L1 Immune Checkpoint Blockade to Treat Breast Cancer
Hartkopf, A. D., Taran, F., Wallwiener, M., Walter, C. B., Krämer, B., Grischke, E., & Brucker, S. Y. (2016) · Breast Care · doi:10.1159/000453569 · Cited by 30,989
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · doi:10.1109/cvpr.1993 · Cited by 17,989
The 2016 World Health Organization Classification of Tumors of the Central Nervous System: a summary
Louis, D. N., Perry, A., Reifenberger, G., von Deimling, A., Figarella‐Branger, D., Cavenee, W. K., Ohgaki, H., Wiestler, O. D., Kleihues, P., & Ellison, D. W. (2016) · Acta Neuropathologica · doi:10.1007/s00401-016-1545-1 · Cited by 15,703
Tissue-based map of the human proteome
Uhlén, M., Fagerberg, L., Hallström, B. M., Lindskog, C., Oksvold, P., Mardinoglu, A., Sivertsson, Å., Kampf, C., Sjöstedt, E., Asplund, A., et al. (2015) · Science · doi:10.1126/science.1260419 · Cited by 15,286
From local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551
Mutational landscape determines sensitivity to PD-1 blockade in non–small cell lung cancer
Rizvi, N. A., Hellmann, M. D., Snyder, A., Kvistborg, P., Makarov, V., Havel, J. J., Lee, W., Yuan, J., Wong, P., Ho, T. S., et al. (2015) · Science · doi:10.1126/science.aaa1348 · Cited by 7,802
Amin, M. B., Greene, F. L., Edge, S. B., Compton, C. C., Gershenwald, J. E., Brookland, R. K., Meyer, L., Gress, D. M., Byrd, D. R., & Winchester, D. P. (2017) · CA A Cancer Journal for Clinicians · doi:10.3322/caac.21388 · Cited by 6,531
The Immune Landscape of Cancer
Thorsson, V., Gibbs, D. L., Brown, S. D., Wolf, D., Bortone, D. S., Ou Yang, T.-H., Porta-Pardo, E., Gao, G. F., Plaisier, C. L., Eddy, J. A., et al. (2018) · Immunity · doi:10.1016/j.immuni.2018.03.023 · Cited by 5,870
Mutational heterogeneity in cancer and the search for new cancer-associated genes
Lawrence, M. S., Stojanov, P., Polak, P., Kryukov, G. V., Cibulskis, K., Sivachenko, A., Carter, S. L., Stewart, C., Mermel, C. H., Roberts, S. A., et al. (2013) · Nature · doi:10.1038/nature12213 · Cited by 5,461
Multimodal Machine Learning: A Survey and Taxonomy
Baltrusaitis, T., Ahuja, C., & Morency, L.-P. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2798607 · Cited by 5,018
The Somatic Genomic Landscape of Glioblastoma
Brennan, C. W., Verhaak, R. G. W., McKenna, A., Campos, B., Noushmehr, H., Salama, S. R., Zheng, S., Chakravarty, D., Sanborn, J. Z., Berman, S. H., et al. (2013) · Cell · doi:10.1016/j.cell.2013.09.034 · Cited by 5,016
Axiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087
Comprehensive molecular characterization of clear cell renal cell carcinoma
The Cancer Genome Atlas Research Network (2013) · Nature · doi:10.1038/nature12222 · Cited by 3,485
A pathology atlas of the human cancer transcriptome
Uhlen, M., Zhang, C., Lee, S., Sjöstedt, E., Fagerberg, L., Bidkhori, G., Benfeitas, R., Arif, M., Liu, Z., Edfors, F., et al. (2017) · Science · doi:10.1126/science.aan2507 · Cited by 3,371
Coudray, N., Ocampo, P. S., Sakellaropoulos, T., Narula, N., Snuderl, M., Fenyö, D., Moreira, A. L., Razavian, N., & Tsirigos, A. (2018) · Nature Medicine · doi:10.1038/s41591-018-0177-5 · Cited by 2,930
Clinical-grade computational pathology using weakly supervised deep learning on whole slide images
Campanella, G., Hanna, M. G., Geneslaw, L., Miraflor, A., Werneck Krauss Silva, V., Busam, K. J., Brogi, E., Reuter, V. E., Klimstra, D. S., & Fuchs, T. J. (2019) · Nature Medicine · doi:10.1038/s41591-019-0508-1 · Cited by 2,799
Data-efficient and weakly supervised computational pathology on whole-slide images
Lu, M. Y., Williamson, D. F. K., Chen, T. Y., Chen, R. J., Barbieri, M., & Mahmood, F. (2021) · Nature Biomedical Engineering · doi:10.1038/s41551-020-00682-w · Cited by 2,282
Self-citationSelf-Normalizing Neural Networks
Klambauer, G., Unterthiner, T., Mayr, A., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.02515 · Cited by 2,267
The immune contexture in cancer prognosis and treatment
Fridman, W. H., Zitvogel, L., Sautès–Fridman, C., & Kroemer, G. (2017) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2017.101 · Cited by 2,263
Tensor Fusion Network for Multimodal Sentiment Analysis
Zadeh, A., Chen, M., Poria, S., Cambria, E., & Morency, L.-P. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1115 · Cited by 2,102
Intra-tumour heterogeneity: a looking glass for cancer?
Marusyk, A., Almendro, V., & Polyák, K. (2012) · Nature reviews. Cancer · doi:10.1038/nrc3261 · Cited by 2,058
Highly multiplexed imaging of tumor tissues with subcellular resolution by mass cytometry
Giesen, C., Wang, H. A. O., Schapiro, D., Zivanovic, N., Jacobs, A., Hattendorf, B., Schüffler, P. J., Grolimund, D., Buhmann, J. M., Brandt, S., et al. (2014) · Nature Methods · doi:10.1038/nmeth.2869 · Cited by 1,951
Artificial intelligence in digital pathology — new tools for diagnosis and precision oncology
Bera, K., Schalper, K. A., Rimm, D. L., Velcheti, V., & Madabhushi, A. (2019) · Nature Reviews Clinical Oncology · doi:10.1038/s41571-019-0252-y · Cited by 1,698
Biomarkers in Cancer Staging, Prognosis and Treatment Selection
Ludwig, J. A., & Weinstein, J. N. (2005) · Nature reviews. Cancer · doi:10.1038/nrc1739 · Cited by 1,682
2013 IEEE 10th International Symposium on Biomedical Imaging
(2013) · doi:10.1109/isbi20048.2013 · Cited by 1,637
Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images
Graham, S., Vu, Q. D., Raza, S. E. A., Azam, A. S., Tsang, Y. W., Kwak, J. T., & Rajpoot, N. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.101563 · Cited by 1,516
Kather, J. N., Pearson, A. T., Halama, N., Jäger, D., Krause, J., Loosen, S. H., Marx, A., Boor, P., Tacke, F., Neumann, U. P., et al. (2019) · Nature Medicine · doi:10.1038/s41591-019-0462-y · Cited by 1,458
Attention-based Deep Multiple Instance LearningPreprint
Ilse, M., Tomczak, J. M., & Welling, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04712 · Cited by 1,428
Cloughesy, T. F., Mochizuki, A. Y., Orpilla, J. R., Hugo, W., Lee, A. H., Davidson, T. B., Wang, A. C., Ellingson, B. M., Rytlewski, J. A., Sanders, C. M., et al. (2019) · Nature Medicine · doi:10.1038/s41591-018-0337-7 · Cited by 1,382
Predicting cancer outcomes from histology and genomics using convolutional networks
Mobadersany, P., Yousefi, S., Amgad, M., Gutman, D. A., Barnholtz-Sloan, J. S., Velázquez Vega, J. E., Brat, D. J., & Cooper, L. A. D. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1717139115 · Cited by 1,084
Kather, J. N., Krisam, J., Charoentong, P., Luedde, T., Herpel, E., Weis, C.-A., Gaiser, T., Marx, A., Valous, N. A., Ferber, D., et al. (2019) · PLOS Medicine · doi:10.1371/journal.pmed.1002730 · Cited by 1,082
An Immune Atlas of Clear Cell Renal Cell Carcinoma
Chevrier, S., Levine, J., Zanotelli, V. R., Siliņa, K., Schulz, D., Bacac, M., Ries, C. H., Ailles, L., Jewett, M. A. S., Moch, H., et al. (2017) · Cell · doi:10.1016/j.cell.2017.04.016 · Cited by 1,036
Saltz, J., Gupta, R., Hou, L., Kurc, T., Singh, P., Nguyen, V., Samaras, D., Shroyer, K. R., Zhao, T., Batiste, R., et al. (2018) · Cell Reports · doi:10.1016/j.celrep.2018.03.086 · Cited by 1,027
Bland, M., & Altman, D. G. (2004) · BMJ · doi:10.1136/bmj.328.7447.1073 · Cited by 972
The single-cell pathology landscape of breast cancer
Jackson, H. W., Fischer, J. R., Zanotelli, V. R., Ali, H. R., Mechera, R., Soysal, S. D., Moch, H., Muenst, S. E., Varga, Z., Weber, W. P., et al. (2020) · Nature · doi:10.1038/s41586-019-1876-x · Cited by 892
Systematic Analysis of Breast Cancer Morphology Uncovers Stromal Features Associated with Survival
Beck, A. H., Sangoi, A. R., Leung, S. C. Y., Marinelli, R. J., Nielsen, T. O., van de Vijver, M. J., West, R. B., van de Rijn, M., & Koller, D. (2011) · Science Translational Medicine · doi:10.1126/scitranslmed.3002564 · Cited by 780
Chen, R. J., Lu, M. Y., Wang, J., Williamson, D. F. K., Rodig, S. J., Lindeman, N. I., & Mahmood, F. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.3021387 · Cited by 729
Self-citationPan-cancer image-based detection of clinically actionable genetic alterations
Kather, J. N., Heij, L. R., Grabsch, H. I., Loeffler, C. M. L., Echle, A., Muti, H. S., Krause, J., Niehues, J., Sommer, K. A. J., Bankhead, P., et al. (2020) · Nature Cancer · doi:10.1038/s43018-020-0087-6 · Cited by 686
Deep learning in cancer pathology: a new generation of clinical biomarkers
Echle, A., Rindtorff, N. T., Brinker, T. J., Luedde, T., Pearson, A. T., & Kather, J. N. (2021) · British Journal of Cancer · doi:10.1038/s41416-020-01122-x · Cited by 667
histoCAT: analysis of cell phenotypes and interactions in multiplex image cytometry data
Schapiro, D., Jackson, H. W., Raghuraman, S., Fischer, J. R., Zanotelli, V. R., Schulz, D., Giesen, C., Catena, R., Varga, Z., & Bodenmiller, B. (2017) · Nature Methods · doi:10.1038/nmeth.4391 · Cited by 661
Captum: A unified and generic model interpretability library for PyTorch
Kokhlikyan, N., Miglani, V., Martin, M., Wang, E., Alsallakh, B., Reynolds, J., Melnikov, A., Kliushkina, N., Araya, C., Yan, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2009.07896 · Cited by 658
Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis
Fu, Y., Jung, A. W., Viñas, R., González, S., Vöhringer, H. S., Shmatko, A., Yates, L. R., Jimenez‐Liñan, M., Moore, L., & Gerstung, M. (2020) · Nature Cancer · doi:10.1038/s43018-020-0085-8 · Cited by 655
A genome-wide transcriptomic analysis of protein-coding genes in human blood cells
Uhlen, M., Karlsson, M. J., Zhong, W., Tebani, A., Pou, C., Mikes, J., Lakshmikanth, T., Forsström, B., Edfors, F., Odeberg, J., et al. (2019) · Science · doi:10.1126/science.aax9198 · Cited by 594
What Makes Training Multi-modal Classification Networks Hard?
Wang, W., Tran, D., & Feiszli, M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01271 · Cited by 588
Spatial maps of prostate cancer transcriptomes reveal an unexplored landscape of heterogeneity
Berglund, E., Maaskola, J., Schultz, N., Friedrich, S., Marklund, M., Bergenstråhle, J., Tarish, F., Tanoglidi, A., Vicković, S., Larsson, L., et al. (2018) · Nature Communications · doi:10.1038/s41467-018-04724-5 · Cited by 558
Deep learning-based classification of mesothelioma improves prediction of patient outcome
Courtiol, P., Maussion, C., Moarii, M., Pronier, E., Pilcer, S., Sefta, M., Manceron, P., Toldo, S., Zaslavskiy, M., Le Stang, N., et al. (2019) · Nature Medicine · doi:10.1038/s41591-019-0583-3 · Cited by 544
Integrating spatial gene expression and breast tumour morphology via deep learning
He, B., Bergenstråhle, L., Stenbeck, L., Abid, A., Andersson, A., Borg, Å., Maaskola, J., Lundeberg, J., & Zou, J. (2020) · Nature Biomedical Engineering · doi:10.1038/s41551-020-0578-x · Cited by 507
Deep learning with multimodal representation for pancancer prognosis prediction
Cheerla, A., & Gevaert, O. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz342 · Cited by 421
Integrated genomic characterization of IDH1-mutant glioma malignant progression
Bai, H., Harmancı, A. S., Erson-Omay, Z., Li, J., Coşkun, S., Simon, M., Krischek, B., Özduman, K., Omay, S. B., Sorensen, E. A., et al. (2015) · Nature Genetics · doi:10.1038/ng.3457 · Cited by 328
Diao, J. A., Wang, J. K., Chui, W. F., Mountain, V., Gullapally, S. C., Srinivasan, R., Mitchell, R. N., Glass, B., Hoffman, S., Rao, S. K., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-21896-9 · Cited by 289
Geospatial immune variability illuminates differential evolution of lung adenocarcinoma
AbdulJabbar, K., Raza, S. E. A., Rosenthal, R., Jamal‐Hanjani, M., Veeriah, S., Akarca, A. U., Lund, T., Moore, D. A., Salgado, R., Bakir, M. A., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0900-x · Cited by 287
Topography of cancer-associated immune cells in human solid tumors
Kather, J. N., Suarez‐Carmona, M., Charoentong, P., Weis, C., Hirsch, D. D., Bankhead, P., Horning, M., Ferber, D., Kel, I., Herpel, E., et al. (2018) · eLife · doi:10.7554/elife.36967 · Cited by 283
Long-term cancer survival prediction using multimodal deep learning
Vale-Silva, L. A., & Rohr, K. (2021) · Scientific Reports · doi:10.1038/s41598-021-92799-4 · Cited by 268
Deep learning-based survival prediction for multiple cancer types using histopathology images
Wulczyn, E., Steiner, D. F., Xu, Z., Sadhwani, A., Wang, H., Flament-Auvigne, I., Mermel, C. H., Chen, P.-H. C., Liu, Y., & Stumpe, M. C. (2020) · PLOS ONE · doi:10.1371/journal.pone.0233678 · Cited by 266
LeDell, E., Petersen, M. L., & van der Laan, M. J. (2015) · Electronic Journal of Statistics · doi:10.1214/15-ejs1035 · Cited by 241
Zhou, Q. H., Xu, C., Cheng, Y., Liu, Y., Chen, G., Cui, J., Yang, N., Song, Y., Li, X., Lü, S., et al. (2021) · Cancer Cell · doi:10.1016/j.ccell.2021.07.005 · Cited by 236
Interpretable survival prediction for colorectal cancer using deep learning
Wulczyn, E., Steiner, D. F., Moran, M., Plass, M., Reihs, R., Tan, F., Flament-Auvigne, I., Brown, T., Regitnig, P., Chen, P.-H. C., et al. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00427-2 · Cited by 231
Bootstrapping the out-of-sample predictions for efficient and accurate cross-validation
Tsamardinos, I., Greasidou, E., & Borboudakis, G. (2018) · Machine Learning · doi:10.1007/s10994-018-5714-4 · Cited by 230
Mapping spatial heterogeneity in the tumor microenvironment: a new era for digital pathology
Heindl, A., Nawaz, S., & Yuan, Y. (2015) · Laboratory Investigation · doi:10.1038/labinvest.2014.155 · Cited by 220
Bias in Cross-Entropy-Based Training of Deep Survival Networks
Zadeh, S. G., & Schmid, M. C. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2979450 · Cited by 202
Shi, J.-Y., Wang, X., Ding, G., Dong, Z., Han, J., Guan, Z., Ma, L., Zheng, Y., Zhang, L., Yu, G., et al. (2020) · Gut · doi:10.1136/gutjnl-2020-320930 · Cited by 198
Abdelmoula, W. M., Balluff, B., Englert, S., Dijkstra, J., Reinders, M. J. T., Walch, A. K., McDonnell, L. A., & Lelieveldt, B. P. F. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1510227113 · Cited by 184
Current Landscape of Targeted Therapy in Lung Cancer
Mayekar, M. K., & Bivona, T. G. (2017) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.810 · Cited by 184
Shaban, M., Khurram, S. A., Fraz, M. M., Alsubaie, N. M., Masood, I., Mushtaq, S., Hassan, M., Loya, A., & Rajpoot, N. (2019) · Scientific Reports · doi:10.1038/s41598-019-49710-z · Cited by 176
Self-citationDetermining breast cancer biomarker status and associated morphological features using deep learning
Gamble, P., Jaroensri, R., Wang, H., Tan, F., Moran, M., Brown, T., Flament-Auvigne, I., Rakha, E. A., Toss, M., Dabbs, D. J., et al. (2021) · Communications Medicine · doi:10.1038/s43856-021-00013-3 · Cited by 154
Hyman, D. M., Solit, D. B., Arcila, M. E., Cheng, D. T., Sabbatini, P. J., Baselga, J. M., Berger, M. F., & Ladanyi, M. (2015) · Drug Discovery Today · doi:10.1016/j.drudis.2015.08.005 · Cited by 149
Spontaneous mutations in the single TTN gene represent high tumor mutation burden
Oh, J., Jang, S. J., Kim, J., Sohn, I., Lee, J., Cho, E. J., Chun, S., & Sung, C. O. (2020) · npj Genomic Medicine · doi:10.1038/s41525-019-0107-6 · Cited by 135
Grading Systems in Renal Cell Carcinoma
Novara, G., Martignoni, G., Artibani, W., et al. (2007) · The Journal of Urology · doi:10.1016/j.juro.2006.09.034 · Cited by 133
Kulkarni, P. M., Robinson, E. J., Pradhan, J. S., Gartrell, R. D., Rohr, B. R., Trager, M. H., Geskin, L. J., Kluger, H. M., Wong, P. F., Ács, B., et al. (2019) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-19-1495 · Cited by 128
An ecological measure of immune-cancer colocalization as a prognostic factor for breast cancer
Maley, C. C., Koelble, K., Natrajan, R., Aktipis, A., & Yuan, Y. (2015) · Breast Cancer Research · doi:10.1186/s13058-015-0638-4 · Cited by 104
Nicholson, A. G., Perry, L. J. P., Cury, P. M., Jackson, P., MCCORMICK, C., CORRIN, B., & Wells, A. U. (2001) · Histopathology · doi:10.1046/j.1365-2559.2001.01078.x · Cited by 102
Bejnordi, B. E., Lin, J., Glass, B., Mullooly, M., Gierach, G. L., Sherman, M. E., Karssemeijer, N., van der Laak, J. A. W. M., & Beck, A. H. (2017) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2017.7950668 · Cited by 79
Development of AI-based pathology biomarkers in gastrointestinal and liver cancer
Kather, J. N., & Caldéraro, J. (2020) · Nature Reviews Gastroenterology & Hepatology · doi:10.1038/s41575-020-0343-3 · Cited by 79
Petrelli, F., Ghidini, M., Cabiddu, M., Pezzica, E., Corti, D. D., Turati, L., Costanzo, A., VARRICCHIO, A., Ghidini, A., Barni, S., et al. (2019) · Anticancer Research · doi:10.21873/anticanres.13857 · Cited by 77
Harder, N., Schönmeyer, R., Nekolla, K., Meier, A., Brieu, N., Vanegas, M. C., Madonna, G., Capone, M., Botti, G., Ascierto, P. A., et al. (2019) · Scientific Reports · doi:10.1038/s41598-019-43525-8 · Cited by 59
Interobserver variability in breast carcinoma grading results in prognostic stage differences
Rabe, K. M., Snir, O. L., Bossuyt, V., Harigopal, M., Celli, R., & Reisenbichler, E. (2019) · Human Pathology · doi:10.1016/j.humpath.2019.09.006 · Cited by 52
Classification of Tumor Histology via Morphometric Context
Chang, H., Borowsky, A. D., Spellman, P. T., & Parvin, B. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.286 · Cited by 46
Tarantino, P., Mazzarella, L., Marra, A., Trapani, D., & Curigliano, G. (2021) · Cancer Treatment Reviews · doi:10.1016/j.ctrv.2021.102169 · Cited by 18
Gentzler, R. D., Yentz, S. E., Johnson, M. L., RADEMAKER, A. W., & Patel, J. D. (2014) · Cancer · doi:10.1002/cncr.28956 · Cited by 10
Cited by
Sorted by citations
Towards a general-purpose foundation model for computational pathology
Chen, R. J., Ding, T., Lu, M. Y., Williamson, D. F. K., Jaume, G., Song, A. H., Chen, B., Zhang, A., Shao, D., Shaban, M., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02857-3 · Cited by 1,560
Self-citationA visual-language foundation model for computational pathology
Lu, M. Y., Chen, B., Williamson, D. F. K., Chen, R. J., Liang, I., Ding, T., Jaume, G., Odintsov, I., Le, L. P., Gerber, G., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02856-4 · Cited by 966
Self-citationA whole-slide foundation model for digital pathology from real-world data
Xu, H., Usuyama, N., Bagga, J., Zhang, S., Rao, R., Naumann, T., Wong, C., Gero, Z., González, J., Gu, Y., et al. (2024) · Nature · doi:10.1038/s41586-024-07441-w · Cited by 921
Artificial intelligence for multimodal data integration in oncology
Lipkova, J., Chen, R. J., Chen, B., Lu, M. Y., Barbieri, M., Shao, D., Vaidya, A. J., Chen, C., Zhuang, L., Williamson, D. F. K., et al. (2022) · Cancer Cell · doi:10.1016/j.ccell.2022.09.012 · Cited by 777
Self-citationMultimodal deep learning for biomedical data fusion: a review
Stahlschmidt, S. R., Ulfenborg, B., & Synnergren, J. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbab569 · Cited by 766
Metadata
- Authors
- Chen, R. J., Lu, M. Y., Williamson, D. F. K., Chen, T. Y., Lipkova, J., Noor, Z., Shaban, M., Shady, M., Williams, M., Joo, B., & Mahmood, F.
- Affiliations
- Broad Institute (US)
- Brigham and Women's Hospital (US)
- Corresponding author
- Mahmood, Faisal
- ORCID
- Venue
- Cancer Cell
- Publisher
- Elsevier BV
- Published
- 2022-08
- Volume
- 40
- Issue
- 8
- Pages
- 865-878.e6
- ISSN
- 1535-6108 (ISSN-L)
- PMC
- PMC10397370
- OpenAlex
- W4291021272
- Cited by
- 660
- References
- 88
- Citations by year
- 2026: 219 · 2025: 215 · 2024: 133 · 2023: 83 · 2022: 9
- FWCI
- 36.27
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Cancer Cell (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- U.S. National Library of Medicine (T15LM007092)
- National Science Foundation
- Brigham and Women's Hospital
- National Institutes of Health
- National Human Genome Research Institute
- National Institute of General Medical Sciences (R35GM138216)
- National Cancer Institute (T32CA251062)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Cancer Research › Cancer Genomics and Diagnostics
- Keywords
- pan-cancer, multimodal deep learning, whole slide images, prognostic model, weakly supervised learning, morphological features, molecular profiles, explainable features, biomarker discovery, interactive database
- MeSH terms
- Deep Learning, Algorithms, Humans, Neoplasms, Prognosis, Genomics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-09-25 (prompt extract-v2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
