CoChat

The DOI matches its Crossref record; 63 of the 64 references it cites were found in a registry or an index; 10 of the 16 sources its claims cite were checked for support.

The role of multimodality in clinical disease diagnosis: advances, challenges, and opportunities

Open access

Abstract

Advances in artificial intelligence (AI) have significantly improved medical diagnosis, with deep learning models achieving expert-level performance across unimodal tasks such as medical imaging, physiological signal analysis, electronic health record (EHR) modeling, and omics-based prediction. However, clinical decision-making is inherently multimodal, as diseases manifest through complex interactions among imaging phenotypes, molecular signatures, physiological measurements, and textual clinical documentation. Consequently, unimodal systems often lack robustness, generalizability, and clinical reliability. This survey provides a comprehensive and methodologically grounded review of multimodal learning for disease diagnosis, emphasizing the paradigm shifts that have emerged over the past five years. Beyond classical early, intermediate, and late fusion strategies, we synthesize modern cross-modal representation learning frameworks, including contrastive alignment, vision-language pretraining, graph and hypergraph-based multimodal reasoning, modality-agnostic representation learning, and missing-modality robust architectures. We further examine large-scale foundation-model style multimodal pretraining and recent advances in histology-transcriptomics and image-omics integration, which exemplify biologically grounded cross-modal learning beyond traditional fusion pipelines. In addition to summarizing widely used datasets and clinical applications across oncology, neurology, cardiovascular medicine, and intensive care, we critically appraise translation barriers, including data heterogeneity, missing data modalities, model interpretability, bias, and evaluation pitfalls. Our goal is to provide a unified reference that bridges technical AI research and clinical utility, paving the way for the next generation of safe and effective diagnostic decision-support systems.

Overview

CoChat Summary

## TL;DR

This survey argues that multimodal AI is a necessary next step for clinical diagnosis because diseases are expressed across imaging, text, EHR, signals, and omics, and single-modality models miss important complementary information. It synthesizes fusion methods and newer representation-learning paradigms, while highlighting persistent obstacles such as missing modalities, imbalance, fairness, interpretability, and deployment constraints.

## Key findings

  • The survey states that multimodal approaches consistently outperform unimodal baselines by leveraging complementary information, improving diagnostic accuracy, robustness to missing/noisy data, and fairness across diverse patient populations.
  • It organizes multimodal methods into early, intermediate, and late fusion, and reports that each has trade-offs: early fusion is straightforward but sensitive to missing data; intermediate fusion captures cross-modal interactions but is computationally demanding; late fusion is robust to missingness but weaker at modeling fine-grained dependencies.
  • The paper highlights newer paradigms including cross-modal contrastive learning, vision–language pretraining, graph/hypergraph reasoning, modality-agnostic representation learning, and missing-modality robust architectures as major advances beyond classical fusion.
  • It identifies common application areas and datasets, including oncology, neurology, cardiology, pulmonology, and ophthalmology, and cites multimodal resources such as MIMIC-CXR + MIMIC-IV, CheXpert, TCGA, CPTAC, and ADNI.

## Methods

  • This is a narrative/systematic survey “inspired by PRISMA guidelines,” organized into literature search, study selection, and data extraction/analysis.
  • Literature was searched in PubMed, IEEE Xplore, ACM Digital Library, Scopus, and arXiv for papers published from January 2020 to August 2025, with references of included papers also screened.
  • Inclusion required human disease diagnosis studies using at least two modalities and reporting quantitative diagnostic metrics plus comparison to unimodal or ablation baselines; exclusions covered prognosis-only, synthetic, non-clinical, purely methodological, and review/opinion papers.

## Limitations

  • The paper states that multimodal systems still face missing modalities, data imbalance, interpretability and fairness issues, computational cost, and poor generalization across institutions and populations.
  • It also states that evaluation is often incomplete, with studies frequently reporting discrimination metrics only and lacking calibration, decision-curve analysis, cost-utility, external validation, or prospective testing.
  • Evident from the text: this is a survey rather than an original empirical study, so its conclusions depend on the included literature and the specific search window of 2020–August 2025.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3389/fpubh.2026.1788454

Cites

63 of 64 references verified0 citation errors · 1 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Global causes of death 148.
Figure 1 Global causes of death 148.CC BY · PubMed Central
Figure 2 Taxonomy of clinical data modalities.
Figure 2 Taxonomy of clinical data modalities.CC BY · PubMed Central
Figure 3 Literature search flow of the proposed work based on PRISMA-style.
Figure 3 Literature search flow of the proposed work based on PRISMA-style.CC BY · PubMed Central
Figure 4 Literature search strategy.
Figure 4 Literature search strategy.CC BY · PubMed Central
Figure 5 Fusion strategies for multimodal disease diagnosis.
Figure 5 Fusion strategies for multimodal disease diagnosis.CC BY · PubMed Central
Figure 6 Heatmap of modalities used across disease domains (illustrative).
Figure 6 Heatmap of modalities used across disease domains (illustrative).CC BY · PubMed Central

Verification

Claims the paper makes

4 Valid · 34 Not valid · 4 Not checked · 86 Own results and interpretations

41 more claims

Create a free account to see all 128 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 30
  • Probably not a substantial claim. 65

Create a free account to see these claims.

Citations

Cites

Show all 54 references

Research works printed but not linked: 85

  1. Chen, 2020, Pathomic fusion: an integrated framework for fusing histopathology and genomic features, IEEE Trans Med Imaging, 39, 2045, 10.1109/TMI.2020.3021387 · DOI · Paper page
  2. Soto, 2022, LVH-fusion: multimodal deep learning combining ECG and echocardiogram for hypertrophic cardiomyopathy classification, NPJ Digit Med, 5, 123
  3. Jahanian, 2025, RadFuse: multimodal transformer for joint disease diagnosis and image-text retrieval in chest X-rays, Med Image Anal, 92, 103123
  4. Han, 2022, Synthetic data augmentation for multimodal cancer diagnosis via cross-modal GANs, Artif Intell Med, 124, 102159
  5. Loey, 2022, Multimodal deep learning for COVID-19 diagnosis using CT scans and clinical data, Comput Biol Med, 142, 105233
  6. El-Ateif, 2024, Multimodal fusion strategies for ophthalmic disease diagnosis: a comparative evaluation, J Med Imag, 11, 024503
  7. Ngiam, 2022, Self-supervised multimodal representation learning for healthcare
  8. Rajpurkar, 2022, Cross-hospital evaluation of multimodal chest X-ray diagnosis with clinical notes, NPJ Digit Med, 5, 88
  9. Banerjee, 2023, Federated multimodal learning for oncology across institutions, Artif Intell Med, 136, 102430
  10. Wu, 2022, Fair multimodal dermatology AI via adversarial debiasing, Med Image Anal, 80, 102517
  11. Lal, 2022, Interpretable multimodal fusion using graph neural networks for cancer diagnosis, Artif Intell Med, 130, 102310
  12. Sun, 2023, Prototype-based multimodal fusion for interpretable clinical decision support, J Biomed Inform, 139, 104221, 10.1016/j.jbi.2023.104221 · doi:10.1016/j.jbi.2023.104221
  13. Bhardwaj, 2023, Interpretable multimodal deep learning for diabetic retinopathy detection, Diagnostics, 13, 1233
  14. Ghorbani, 2019, Towards interpretable multimodal medical AI: concept attribution for cardiology, Nat Mach Intell, 1, 386
  15. Li, 2024, Causal inference–guided multimodal fusion for interpretable neurodegenerative disease diagnosis, Front Neurosci, 18, 122345
  16. Rajpurkar, 2017, CheXNet: radiologist-level pneumonia detection on chest X-rays with deep learning, arXiv, 1, 1
  17. Yildirim, 2019, Automated detection of diabetic retinopathy using deep neural networks on retinal fundus images, Med Hypotheses, 122, 123
  18. Song, 2020, Sleep apnea detection using a recurrent neural network on ECG signals, IEEE Access, 8, 172923
  19. Huang, 2019, Applications of deep learning to electronic health records: a review, J Biomed Inform, 95, 103182
  20. Solares, 2020, Deep learning for electronic health records: a comparative review of multiple data representations, J Biomed Inform, 106, 103426
  21. Way, 2018, Extracting biological meaning from RNA-seq data using unsupervised learning, PLoS Comput Biol, 14, e1006147
  22. Khodabakhshi, 2020, Deep learning for cancer diagnosis using omics data: a review, Brief Bioinform, 21, 2064
  23. Chen, 2021, Deep learning methods for omics-based cancer classification: a comparative study, Front Genet, 12, 650370
  24. Divate, 2022, Deep learning for tissue-of-origin classification using transcriptomic data, Front Oncol, 12, 853264
  25. Couture, 2021, Image analysis with deep learning to predict breast cancer molecular subtype, NPJ Breast Cancer, 7, 59
  26. Hao, 2023, Multimodal deep learning for integrative analysis of genomics and pathology, Bioinformatics, 39, btad123
  27. Sun, 2024, Attention-based multimodal survival prediction with histology and omics, Artif Intell Med, 149, 102905
  28. Liu, 2018, Multimodal neuroimaging feature learning for Alzheimer’s disease, IEEE Trans Biomed Eng, 65, 2242
  29. Zhang, 2019, Multimodal classification of Alzheimer’s using MRI and PET, NeuroImage, 61, 119
  30. Qiu, 2020, Multimodal biomarkers in Alzheimer’s: fusion of MRI, PET, and clinical data, Front Neurosci, 14, 589
  31. Zhang, 2020, Multimodal deep learning for pneumonia diagnosis with chest X-rays and clinical data, Nat Commun, 11, 4080
  32. Johnson, 2021, Multimodal machine learning in critical care: predicting outcomes with imaging and EHR, NPJ Digit Med, 4, 98
  33. Huang, 2022, Transformer-based multimodal fusion for ICU risk prediction, Artif Intell Med, 127, 102286
  34. Wu, 2023, Cross-modal transformers for radiology report and image fusion, Med Image Anal, 85, 102753
  35. Tang, 2024, Clinical multimodal transformers for pulmonary disease triage, Front Med, 11, 120933
  36. Chakravarty, 2020, Multimodal ophthalmic fusion of fundus and OCT imaging, IEEE Access, 8, 121234
  37. Liu, 2021, Attention-based multimodal fusion for ophthalmic disease detection, Comput Biol Med, 135, 104610
  38. Jiang, 2024, Multimodal fusion for glaucoma detection with fundus and OCT, Front Ophthalmol, 2, 101245
  39. Harutyunyan, 2019, Multimodal deep learning for ICU mortality prediction with physiological signals and EHR, J Biomed Inform, 91, 103112
  40. Shashikumar, 2020, Multimodal deep learning for early ICU mortality prediction, Sci Rep, 10, 1402
  41. Choi, 2021, Using multimodal data for cardiovascular risk prediction, J Am Med Inform Assoc, 28, 80
  42. Reyna, 2022, Multimodal deep learning for ECG and labs in cardiac risk, Circ Cardiovasc Qual Outcomes, 15, e009345
  43. Zhao, 2023, Lightweight multimodal fusion for cardiac event prediction, Comput Biol Med, 154, 106672
  44. Subhalakshmi, 2022, DLMMF: deep learning–based multimodal fusion for COVID-19 diagnosis from CT and clinical data, Comput Biol Med, 145, 105403
  45. Ou, 2022, Multimodal attention networks for skin cancer diagnosis from dermoscopic images and metadata, Med Image Anal, 77, 102359
  46. Veluchamy, 2024, Automated COVID-19 detection using multimodal deep learning from CT scans and RT-PCR signals, J Healthc Eng, 2024, 1234567
  47. Li, 2025, Cross-modal alignment and contrastive learning for multimodal survival prediction in pathology, Elsevier Sci J, 263, 108633
  48. Zhang, 2022, BioViL: self-supervised biomedical vision–language pre-training using radiology reports
  49. Li, 2023, Aligning medical images and reports through contrastive language–image pretraining, Med Image Anal, 86, 102763
  50. Li, 2022, Graph neural networks in medical image analysis: a survey, IEEE Rev Biomed Eng, 15, 3
  51. Gao, 2022, Multi-modal graph neural network for survival prediction in cancer, Med Image Anal, 75, 102237
  52. Wang, 2023, Hypergraph neural networks for multimodal medical data integration, Pattern Recogn, 137, 109287
  53. Gao, 2021, Multimodal graph learning for disease diagnosis using imaging and electronic health records, IEEE J Biomed Health Inform, 25, 4384
  54. Fu, 2023, Graph-based multimodal fusion for histopathology and genomics integration, IEEE Trans Med Imaging, 42, 1874
  55. Zhang, 2024, Multimodal relational learning for clinical risk prediction using graph attention networks, Artif Intell Med, 146, 102628
  56. Ganin, 2015, Unsupervised domain adaptation by backpropagation, 1180
  57. Ngiam, 2011, Multimodal deep learning, 689
  58. Wu, 2022, Domain-invariant multimodal representation learning for medical diagnosis, IEEE J Biomed Health Inform, 26, 4302
  59. Zhou, 2023, Learning modality-invariant representations for multimodal medical data integration, Med Image Anal, 86, 102748
  60. Zhang, 2024, Unified multimodal representation learning for clinical risk prediction, Artif Intell Med, 146, 102628
  61. Hazarika, 2022, MISA: modality-invariant and modality-specific representations for multimodal sentiment analysis, 1122
  62. Han, 2023, Robust multimodal learning with missing modalities via gated mixture-of-experts, Pattern Recogn, 136, 109214
  63. Nguyen, 2021, Learning robust joint representations under missing modalities, IEEE Trans Neural Netw Learn Syst, 32, 3987
  64. Tran, 2017, Missing modalities imputation via generative adversarial networks, 123
  65. Li, 2024, Uncertainty-aware multimodal fusion for incomplete medical data, Artif Intell Med, 147, 102650
  66. Shao, 2021, TransMIL: transformer based correlated multiple instance learning for whole slide image classification, Adv Neural Inf Process Syst, 34, 2136
  67. He, 2020, Integrating spatial gene expression and histology images with deep learning, Nat Biotechnol, 38, 1019
  68. Ding, 2023, Cross-modal contrastive learning for histopathology and transcriptomics integration, Med Image Anal, 86, 102756
  69. Chen, 2023, Pan-cancer computational histopathology reveals conserved tumor molecular signatures, Nat Commun, 14, 1234
  70. Hao, 2022, Predicting gene expression from histology using deep learning across multiple cancer types, Brief Bioinform, 23, bbac239
  71. Yang, 2024, Domain adaptation for multimodal neuroimaging in Alzheimer’s disease diagnosis, Front Neurosci, 18, 120334
  72. Xu, 2022, Knowledge distillation for efficient multimodal cancer diagnosis using pathology and omics data, IEEE J Biomed Health Inform, 26, 6102
  73. Kim, 2023, Accelerating multimodal chest X-ray analysis with pruning and quantization, Med Image Anal, 87, 102777
  74. Albahli, 2023, Edge deployment of lightweight multimodal deep learning for diabetic retinopathy screening, Comput Biol Med, 155, 106694
  75. Chen, 2024, Early-exit multimodal transformers for real-time ICU triage, Artif Intell Med, 148, 102891
  76. Hassan, 2024, Lightweight multimodal fusion for dermatology diagnosis on mobile devices, Diagnostics (Basel), 14, 765
  77. Yan, 2022, Blockchain-based secure data sharing for multimodal medical AI, IEEE J Biomed Health Inform, 26, 5612
  78. Wang, 2023, Secure multiparty computation for privacy-preserving multimodal omics analysis, BMC Med Inform Decis Mak, 23, 77
  79. Wang, 2023, Multimodal foundation models for healthcare: opportunities and challenges, NPJ Digit Med, 6, 150
  80. Zhang, 2023, Causal multimodal machine learning for clinical applications, J Biomed Inform, 145, 104478
  81. Qiu, 2023, Precision medicine with multimodal data fusion: opportunities and challenges, Front Artif Intell, 6, 119832
  82. Huang, 2024, Personalized multimodal deep learning for precision medicine: a systematic review, Artif Intell Med, 150, 102943
  83. Liu, 2021, Modality dropout for robust multimodal learning in healthcare, Proc AAAI Conf Art Intell, 35, 984
  84. Ramesh, 2023, Mixture-of-experts fusion for multimodal clinical data: robust diagnosis under missing modalities, Artif Intell Med, 140, 102612
  85. Saeed, 2024, Self-supervised representation learning for multimodal clinical data, Patterns, 5, 100789

Cited by

See all 3

Metadata

Authors
Nawaz, A., Edinat, A., Rana, M. R. R., Ali, T., Mustafa, G., Tahir, S., & Lee, S. W.
Affiliations
  • Amman Arab University (JO)
  • Shaheed Zulfiqar Ali Bhutto Institute of Science and Technology (PK)
  • Pir Mehr Ali Shah Arid Agriculture University (PK)
  • Kangbuk Samsung Hospital (KR)
Corresponding author
  • Nawaz, Asif
  • Lee, Seung Won
Venue
Frontiers in public health
Published
2026-04-10
Volume
14
Pages
1788454-1788454
ISSN
2296-2565 (ISSN-L)
OpenAlex
W7152978444
Cited by
3
References
54
Citations by year
2026: 3
FWCI
4.49
Type
journal article
Language
en
Open access
Locations
  • Frontiers in Public Health (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Ministry of Education and Ministry of Science & ICT, Republic of Korea (RS [2025-2544 (3209)], RS [2019-II19 (0421)], RS [2024-0044 (0881)], RS [2024-0040 (5650)], NRF [2021-R1-I1A2 (059735)])
  • Korea Government (RS-2025-02314328)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
multimodal learning, cross-modal representation learning, contrastive alignment, vision-language pre-training, missing modality robustness, foundation models, modality heterogeneity, missing data, interpretability, clinical decision support
MeSH terms
Deep Learning, Artificial Intelligence, Humans

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The role of multimodality in clinical disease diagnosis: advances, challenges, and opportunities. https://app.cochat.ai/papers/10.3389/fpubh.2026.1788454
The paper
Nawaz, A., Edinat, A., Rana, M. R. R., Ali, T., Mustafa, G., Tahir, S., & Lee, S. W. (2026). The role of multimodality in clinical disease diagnosis: advances, challenges, and opportunities. Frontiers in public health. https://doi.org/10.3389/fpubh.2026.1788454

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.