The DOI matches its Crossref record; 161 of the 161 references it cites were found in a registry or an index.
Multimodal data fusion for cancer biomarker discovery with deep learning
Open accessAbstract
Technological advances have made it possible to study a patient from multiple angles with high-dimensional, high-throughput multiscale biomedical data. In oncology, massive amounts of data are being generated, ranging from molecular, histopathology, radiology to clinical records. The introduction of deep learning has greatly advanced the analysis of biomedical data. However, most approaches focus on single data modalities, leading to slow progress in methods to integrate complementary data types. Development of effective multimodal fusion approaches is becoming increasingly important as a single modality might not be consistent and sufficient to capture the heterogeneity of complex diseases to tailor medical care and improve personalized medicine. Many initiatives now focus on integrating these disparate modalities to unravel the biological processes involved in multifactorial diseases such as cancer. However, many obstacles remain, including lack of usable data as well as methods for clinical validation and interpretation. Here, we cover these current challenges and reflect on opportunities through deep learning to tackle data sparsity and scarcity, multimodal interpretability and standardization of datasets. Cancer diagnosis and treatment decisions often focus on one data source. Steyaert and colleagues discuss the current status and challenges of data fusion, including electronic health records, molecular data, digital pathology and radiographic images, in cancer research and translational development.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“NLU is central to enhancing the comprehension and contextualization of human language in AI systems, and involves key capabilities such as semantic analysis, named entity recognition, sentiment analysis, textual entailment, and machine reading comprehension [190], [191], [192], [193].”
From From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape (2025) · cites this paper as [192]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“AI systems can profit significantly from assimilating multimodal data into prediction and classification models to imitate integrative human clinical decision-making.”
From Orchestrating explainable artificial intelligence for multimodal and longitudinal data in medical imaging (2024) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This can boost their robustness and accuracy, enable the discovery of new biomarkers and therapeutic targets6,14, as well as improve model performance15–17.”
From Orchestrating explainable artificial intelligence for multimodal and longitudinal data in medical imaging (2024) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The understanding of important gene features or cross-modality regulatory networks is essential to uncover the mechanism of disease development and identify new disease biomarkers and drug targets [28].”
From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [28]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The high costs of biomedical data generation, difficulties in patient follow-up, and privacy restrictions limit the size of training datasets and lead to missing data modalities [28].”
From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [28]
Experimental — accuracy not yet measured
The cited source says what the claim attributes to it.
In the citing paper
“The data generated by these diverse modalities is generally complementary, with each modality contributing unique information to the status of a patient.”
From Navigating the landscape of multimodal AI in medicine: A scoping review on technical challenges and clinical applications (2025) · cites this paper as Steyaert et al., 2023
In the citing paper
“Advances in NLU have been crucial in improving AI's proficiency in interpreting and analyzing language across a spectrum of contexts, ranging from straightforward conversational exchanges to intricate textual data [190], [192], [193].”
From From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape (2025) · cites this paper as [192]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
The FAIR Guiding Principles for scientific data management and stewardship
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.18 · Cited by 18,423 · cited 1×
“Together, these efforts have the shared aim to process, analyse and share data using a community-embraced standard in a FAIR (findable, accessible, interoperable and reusable) way 156 .”
BackgroundThe 2016 World Health Organization Classification of Tumors of the Central Nervous System: a summary
Louis, D. N., Perry, A., Reifenberger, G., von Deimling, A., Figarella‐Branger, D., Cavenee, W. K., Ohgaki, H., Wiestler, O. D., Kleihues, P., & Ellison, D. W. (2016) · Acta Neuropathologica · doi:10.1007/s00401-016-1545-1 · Cited by 15,703
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Circos: An information aesthetic for comparative genomicsPreprint
Krzywinski, M. I., Schein, J. E., Birol, İ., Connors, J. M., Gascoyne, R. D., Horsman, D. E., Jones, S. J. M., & Marra, M. A. (2009) · Genome Research · doi:10.1101/gr.092759.109 · Cited by 11,073
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450
Hallmarks of Cancer: New Dimensions
Hanahan, D. (2022) · Cancer Discovery · doi:10.1158/2159-8290.cd-21-1059 · Cited by 9,887
Research works printed but not linked: 4
- Mikolov, T., Sutskever, I., Chen, K., Corrado, G. & Dean, J. Distributed representations of words and phrases and their compositionality. In Proc. 26th International Conference on Neural Information Processing Systems 3111–3119 (Curran Associates, Inc., 2013).
- Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies 4171–4186 (Association for Computational Linguistics, 2019).
- Jie, Y., Xutong, L. & Mingyue, Z. Current status of active learning for drug discovery. Artif. Intell. Life Sci. 1, 100023 (2021).
- Lundberg, S. M. & Lee, S. I. A unified approach to interpreting model predictions. Adv. Neural Inf. Process. Syst. 30, 4765–4774 (2017).
Cited by
Sorted by citations
Artificial intelligence in drug development
Zhang, K., Yang, X., Wang, Y., Yu, Y., Huang, N., Li, G. L., Li, X., Wu, J. C., & Yang, S. (2025) · Nature Medicine · doi:10.1038/s41591-024-03434-4 · Cited by 585
Deep learning based multimodal biomedical data fusion: an overview and comparative review
Duan, J., Xiong, J., Li, Y., & Ding, W. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102536 · Cited by 284
Shaik, T., Tao, X., Li, L., Xie, H., & Velasquez, J. D. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102040 · Cited by 228
Cembrowska-Lech, D., Krzemińska, A., Miller, T. I., Nowakowska, A., Adamski, C., Radaczyńska, M., Mikiciuk, G., & Mikiciuk, M. (2023) · Biology · doi:10.3390/biology12101298 · Cited by 193
Medical digital twins: enabling precision medicine and medical artificial intelligence
Sadée, C., Testa, S., Barba, T., Hartmann, K., Schuessler, M., Thieme, A. H., Church, G. M., Okoye, I. J., Hernandez‐Boussard, T., Hood, L., et al. (2025) · The Lancet Digital Health · doi:10.1016/j.landig.2025.02.004 · Cited by 167
Self-citation
Metadata
- Authors
- Steyaert, S., Pizurica, M., Nagaraj, D., Khandelwal, P., Hernandez-Boussard, T., Gentles, A. J., & Gevaert, O.
- Affiliations
- Stanford Medicine (US)
- Stanford University (US)
- Corresponding author
- Steyaert, Sandra
- Gevaert, Olivier
- ORCID
- Venue
- Nature Machine Intelligence
- Published
- 2023-04-06
- Volume
- 5
- Issue
- 4
- Pages
- 351-362
- ISSN
- 2522-5839 (ISSN-L)
- PMC
- PMC10484010
- OpenAlex
- W4362673398
- Semantic Scholar
- 304b640c1b2559ac1442ac9be54853ac80ec248c
- Cited by
- 402
- References
- 159
- Citations by year
- 2026: 122 · 2025: 173 · 2024: 92 · 2023: 15
- FWCI
- 66.57
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- Locations
- Nature Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Ghent University Academic Bibliography (Ghent University) (submittedVersion)
- Funders
- National Cancer Institute (R01CA260271)
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Keywords
- multimodal fusion, cancer biomarker discovery, deep learning, data sparsity, interpretability, electronic health records, digital pathology, radiographic images, molecular data, data standardization, personalized medicine, cancer diagnosis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
