The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Deep learning with multimodal representation for pancancer prognosis prediction
Open accessPublished version of a preprint
Abstract
Motivation: Estimating the future course of patients with cancer lesions is invaluable to physicians; however, current clinical methods fail to effectively use the vast amount of multimodal data that is available for cancer patients. To tackle this problem, we constructed a multimodal neural network-based model to predict the survival of patients for 20 different cancer types using clinical data, mRNA expression data, microRNA expression data and histopathology whole slide images (WSIs). We developed an unsupervised encoder to compress these four data modalities into a single feature vector for each patient, handling missing data through a resilient, multimodal dropout method. Encoding methods were tailored to each data type—using deep highway networks to extract features from clinical and genomic data, and convolutional neural networks to extract features from WSIs. Results: We used pancancer data to train these feature encodings and predict single cancer and pancancer overall survival, achieving a C-index of 0.78 overall. This work shows that it is possible to build a pancancer model for prognosis that also predicts prognosis in single cancer sites. Furthermore, our model handles multiple data modalities, efficiently analyzes WSIs and represents patient multimodal data flexibly into an unsupervised, informative representation. We thus present a powerful automated tool to accurately determine prognosis, a key step towards personalized treatment for cancer patients. Availability and implementation: https://github.com/gevaertlab/MultimodalPrognosis
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Different approaches have been proposed, such as multi-task learning [70], generative models [63] and multimodal dropout [78, 101].”
From Multimodal deep learning for biomedical data fusion: a review (2022) · cites this paper as [78]
The cited source says what the claim attributes to it.
In the citing paper
“Cheerla and Gevaert (2019) constructed a multimodal neural network-based model to predict the survival of patients for 20 different cancer types using clinical data, mRNA expression data, microRNA expression data, and histopathology whole slide images (WSIs).”
From Navigating the landscape of multimodal AI in medicine: A scoping review on technical challenges and clinical applications (2025) · cites this paper as Cheerla and Gevaert, 2019
In the citing paper
“Indeed, it has been shown that training a model on multiple tissues or cancer variants can improve detection performance for each cancer53, as many morphological features are observed across cancers from different topographies54.”
From A foundation model for clinical-grade computational pathology and rare cancers detection (2024) · cites this paper as [53]
In the citing paper
“although we posit that increased diversity helps to learn robust and rare features. Indeed, it has been shown that training a model on multiple tissues or cancer variants can improve detection performance for each cancer53, as many morphological features are observed across cancers from different topographies54.”
From A foundation model for clinical-grade computational pathology and rare cancers detection (2024) · cites this paper as [53]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
(2014) · Cited by 33,419
(2008) · Journal of Machine Learning Research · Cited by 33,051
The cancer genome atlas pan-cancer analysis project
The Cancer Genome Atlas Research Network, Weinstein, J. N., Collisson, E. A., Mills, G. B., Shaw, K. R. M., Ozenberger, B. A., Ellrott, K., Shmulevich, I., Sander, C., & Stuart, J. M. (2013) · Nature Genetics · doi:10.1038/ng.2764 · Cited by 9,092
MicroRNA signatures in human cancers
Calin, G. A., & Croce, C. M. (2006) · Nature reviews. Cancer · doi:10.1038/nrc1997 · Cited by 7,668
Cox, D. R., & Oakes, D. (2018) · Chapman & Hall/CRC eBooks · doi:10.1201/9781315137438 · Cited by 6,269
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016) · arXiv · doi:10.48550/arxiv.1602.07360 · Cited by 6,201
Oncomirs—MicroRNAs with a role in cancer
Esquela-Kerscher, A., & Slack, F. J. (2006) · Nature Reviews Cancer · doi:10.1038/nrc1840 · Cited by 5,884
Learning a Similarity Metric Discriminatively, with Application to Face Verification
Chopra, S., Hadsell, R., & LeCun, Y. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.202 · Cited by 4,036
Flagged first, then by citations
Research works printed but not linked: 4
- Gevaert, 2013, Pacific Symposium on Biocomputing 2013, 123
- Gevaert, 2008, Pacific Symposium on Biocomputing 2008, 279
- Momeni, 2018, International MICCAI Brainlesion Workshop, 407
- Yao, 2016, International Conference on Medical Image Computing and Computer-Assisted Intervention, 649
Cited by
Sorted by citations
Items 401–424 of 424
Kolavennu, G. P., Shi, Y., Suo, K., & Nguyen, T. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-2891-1_16
Ji, J., Zhou, W., Chen, Y., 杨照安, Fu, Y., Yang, Y., Huang, C., & Guo, Q. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-3551-3_1
Application of deep learning in prognostic prediction of non-small cell lung cancer
Lun, X. (2026) · Frontiers in Cell and Developmental Biology · doi:10.3389/fcell.2026.1878798
Self-supervised Multimodal Pre-training for Lung Adenocarcinoma Overall Survival Prediction
Carrillo‐Pérez, F., Pizurica, M., Rojas, I., Marchal, K., Herrera, L. J., & Gevaert, O. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-22199-5_1
Self-citationTowards an Integrated CRC Risk Prediction using Machine Learning - Driven Digital Twin Framework
Gardašević, G., Grbić, M., Ždralević, M., & Mihailović, A. (2026) · International Balkan Conference on Communications and Networking (BalkanCom) · doi:10.1109/balkancom71095.2026.11605028
YiJiang, D., Yuanwei, J., & Zhang, W. (2026) · Frontiers in Oncology · doi:10.3389/fonc.2026.1825286
FORESEE: Multimodal and multiview representation learning for robust cancer survival prediction
Pan, L., Nie, C., Wang, X., & Peng, S. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.111196
SHHN: A Sequential Hierarchy Hybrid Network for multimodal cancer survival prediction
Peng, H., Qu, A., & Zeng, Y. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.111159
FusionSurv: A multi-modal fusion representation network for pan-cancer survival analysis
Gao, R., Yang, Z., & Liu, Z. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.111197
Farhang, Y., Öztel, B., & Yildiz, G. (2026) · Pamukkale University Journal of Engineering Sciences · doi:10.65206/pajes.1934256
Ji, Z., Chen, Q., Shi, X., TIAN, M., Gao, H., Wan, S., Yin, Y., Zhou, X., & Huang, S. (2026) · International Conference on Computer Vision, Image and Deep Learning (CVIDL) · doi:10.1109/cvidl70130.2026.11637865
Artificial intelligence in oncology: linking biological discovery to clinical utility
Kuchana, S. K., Alahari, N. V., Kode, R., Jalapu, K. R., & Alahari, S. K. (2026) · Molecular Cancer · doi:10.1186/s12943-026-02771-x
Hypernetwork-guided fusion with intra-class MixUp for breast cancer subtyping
Abdullakutty, F. C., Akbari, Y., Al-Maadeed, S. A., Bouridane, A., Hamoudi, R., & Kunhoth, S. (2026) · Medical Engineering & Physics · doi:10.1088/1873-4030/ae9ffa
Digital Twins in Oncology: From Multimodal Data Integration to Precision Clinical Practice
Vairavan, A., Bhattacharjee, R., & Raghavan, B. (2026) · Indian journal of radiology and imaging - new series/Indian journal of radiology and imaging/Indian Journal of Radiology & Imaging · doi:10.1055/s-0046-1827804
Saravia, F. H. V., Huanca-Frías, J. O., Huanca-Frias, R. E., Mamani, V. E., Calisaya, L. A. F., Paredes-Quispe, J. R., & Tumi-Figueroa, A. (2026) · Journal Of Advanced Pharmacy Education And Research · doi:10.51847/fwl7k3bfxf
Lu, Z., Wang, J., Liang, Y., Lin, Z., Tan, Y., Ma, Q., Zhou, W., Zhao, Y. C., & Xu, S. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag500
Kapoor, S., & Verma, D. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-40338-5.00001-9
Ming, W., Ye, W., Liu, M., Chen, D., Jiao, Y., Xu, J., & Wang, X. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38172-9_46
Yiming, N., Saibaidoula, Y., Abulimiti, K., & Ou, J. (2026) · Frontiers in Oncology · doi:10.3389/fonc.2026.1912317
Zheng, J., Su, Z., Zhuo, X., Chai, H., Tang, H., & Lan, T. (2026) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2026.109465
Oni, R. O. (2023) · International Journal of Scientific Research and Modern Technology. · doi:10.38124/ijsrmt.v2i1.1707
Explainable Multimodal Brain Tumor Classification via Co-Attention Gated Fusion
Syed, A. A., & kumar, K. R. V. G. (2026) · International Conference on Soft Computing for Security Applications (ICSCSA) · doi:10.1109/icscsa69367.2026.11708114
Lei, K.-X., Luo, Z.-X., Tang, W.-J., Guo, Y., Kong, Q., Liu, Z.-W., Ding, H.-J., Wang, J.-H., & Huang, B.-S. (2026) · Expert systems with applications · doi:10.1016/j.eswa.2026.134710
Taslimi, J., & Minagar, A. (2026) · Archives of Breast Cancer · doi:10.32768/abc.4829103847-562
Metadata
- Authors
- Cheerla, A., & Gevaert, O.
- Affiliations
- Monta Vista High School, Cupertino, CA, USA
- Stanford University (US)
- Corresponding author
- Gevaert, Olivier
- ORCID
- Gevaert, Olivier
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2019-07
- Volume
- 35
- Issue
- 14
- Pages
- i446-i454
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- PMC
- PMC6612862
- OpenAlex
- W2954499361
- Cited by
- 421
- References
- 52
- Citations by year
- 2026: 69 · 2025: 74 · 2024: 68 · 2023: 64 · 2022: 59 · 2021: 69 · 2020: 17 · 2019: 1
- FWCI
- 16.65
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Bioinformatics (publishedVersion, cc-by-nc)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health (R01EB020527)
- National Institute of Dental and Craniofacial Research (U01DE025188)
- National Cancer Institute (U01CA199241, U01CA217851)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- cancer prognosis prediction, pan-cancer analysis, multimodal data integration, overall survival prediction, clinical data, mRNA expression, microRNA expression, convolutional neural networks, missing data handling, precision oncology
- MeSH terms
- Deep Learning, Humans, Neoplasms, Neural Networks, Computer, Genome
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
