A Novel Prediction Model for Multimodal Medical Data Based on Graph Neural Networks
Open accessAbstract
Multimodal medical data provides a wide and real basis for disease diagnosis. Computer-aided diagnosis (CAD) powered by artificial intelligence (AI) is becoming increasingly prominent in disease diagnosis. CAD for multimodal medical data requires addressing the issues of data fusion and prediction. Traditionally, the prediction performance of CAD models has not been good enough due to the complicated dimensionality reduction. Therefore, this paper proposes a fusion and prediction model—EPGC—for multimodal medical data based on graph neural networks. Firstly, we select features from unstructured multimodal medical data and quantify them. Then, we transform the multimodal medical data into a graph data structure by establishing each patient as a node, and establishing edges based on the similarity of features between the patients. Normalization of data is also essential in this process. Finally, we build a node prediction model based on graph neural networks and predict the node classification, which predicts the patients’ diseases. The model is validated on two publicly available datasets of heart diseases. Compared to the existing models that typically involve dimensionality reduction, classification, or the establishment of complex deep learning networks, the proposed model achieves outstanding results with the experimental dataset. This demonstrates that the fusion and diagnosis of multimodal data can be effectively achieved without dimension reduction or intricate deep learning networks. We take pride in exploring unstructured multimodal medical data using deep learning and hope to make breakthroughs in various fields.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Comparisons were made with state-of-the-art graph-based multimodal prediction models.
“Furthermore, the effectiveness of the proposed knowledge enhanced spatial temporal attention mechanism was substantiated by comparisons with state, of, the, art graph, based multimodal prediction models63.”
From K-STAMM: a knowledge-enhanced spatial - temporal attention model with multimodal fusion for pneumonia prediction (2026) · cites this paper as [63]
The Conclusions report comparing EPGC with current mainstream and best-performing methods. However, §3.2.2 (Control Group Experiment) identifies the comparators as feature-selection, dimensionality-reduction, and deep-learning models—not graph-based multimodal prediction models. No page numbers are available in our copy. (LLM, conf=0.90)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Graph neural networks: A review of methods and applications
Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., & Sun, M. (2020) · AI Open · doi:10.1016/j.aiopen.2021.01.001 · Cited by 6,880 · see citing works
Geometric Deep Learning: Going beyond Euclidean data
Bronstein, M. M., Bruna, J., LeCun, Y., Szlam, A., & Vandergheynst, P. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2693418 · Cited by 3,661 · see citing works
Reed, G. W., Rossi, J., & Cannon, C. P. (2016) · The Lancet · doi:10.1016/s0140-6736(16)30677-8 · Cited by 1,570 · see citing works
Acosta, J. N., Falcone, G. J., Rajpurkar, P., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-022-01981-2 · Cited by 1,365 · see citing works
Clustering Using a Similarity Measure Based on Shared Near Neighbors
Jarvis, R. A., & Patrick, E. A. (1973) · IEEE Transactions on Computers · doi:10.1109/t-c.1973.223640 · Cited by 970 · see citing works
Artificial intelligence for multimodal data integration in oncology
Lipkova, J., Chen, R. J., Chen, B., Lu, M. Y., Barbieri, M., Shao, D., Vaidya, A. J., Chen, C., Zhuang, L., Williamson, D. F. K., et al. (2022) · Cancer Cell · doi:10.1016/j.ccell.2022.09.012 · Cited by 777 · see citing works
Multimodal deep learning for biomedical data fusion: a review
Stahlschmidt, S. R., Ulfenborg, B., & Synnergren, J. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbab569 · Cited by 766 · see citing works
On the Euclidean distance of images
Wang, L., Zhang, Y., & Feng, J. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.165 · Cited by 659 · see citing works
Entropy, distance measure and similarity measure of fuzzy sets and their relations
Xuecheng, L. (1992) · Fuzzy Sets and Systems · doi:10.1016/0165-0114(92)90239-z · Cited by 577 · see citing works
Multimodal machine learning in precision health: A scoping review
Kline, A., Wang, H., Li, Y., Dennis, S., Hutch, M., Xu, Z., Wang, F., Cheng, F., & Luo, Y. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00712-8 · Cited by 528 · cited 1× · see citing works
“Kline et al. [9] described the diversity of forms and modalities of multimodal medical data in their study.”
Transfer Learning for Brain–Computer Interfaces: A Euclidean Space Data Alignment Approach
He, H., & Wu, D. (2020) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2019.2913914 · Cited by 525 · cited 1× · see citing works
“In Euclidean space data, for example, text and speech data are one-dimensional sequence data structures [20], and image data [21,22] are typically rectangular structures.”
Azam, M. A., Khan, K. B., Salahuddin, S., Rehman, E., Khan, S. A., Khan, M. A., Kadry, S., & Gandomi, A. H. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105253 · Cited by 449 · see citing works
Deep Learning-Based Image Segmentation on Multimodal Medical Imaging
Guo, Z., Li, X., Huang, H., Guo, N., & Li, Q. (2019) · IEEE Transactions on Radiation and Plasma Medical Sciences · doi:10.1109/trpms.2018.2890359 · Cited by 445 · see citing works
CT or Invasive Coronary Angiography in Stable Chest Pain
The DISCHARGE Trial Group (2022) · New England Journal of Medicine · doi:10.1056/nejmoa2200963 · Cited by 402 · see citing works
A Similarity Measure for Text Classification and Clustering
Lin, Y.-S., Jiang, J.-Y., & Lee, S.-J. (2013) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2013.19 · Cited by 293 · see citing works
Deep learning based multimodal biomedical data fusion: an overview and comparative review
Duan, J., Xiong, J., Li, Y., & Ding, W. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102536 · Cited by 284 · see citing works
Yang, J., Ju, J., Guo, L., Ji, B., Shi, S., Yang, Z., Gao, S., Yuan, X., Tian, G., Liang, Y., et al. (2022) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.12.028 · Cited by 233 · see citing works
Multimodal medical image fusion algorithm in the era of big data
Tan, W., Tiwari, P., Pandey, H. M., Moreira, C., & Jaiswal, A. K. (2020) · Neural Computing and Applications · doi:10.1007/s00521-020-05173-2 · Cited by 210 · see citing works
Clinical Concept Embeddings Learned from Massive Sources of Multimodal Medical Data
Beam, A. L., Kompa, B., Schmaltz, A., Fried, I., Weber, G. M., Palmer, N., Shi, X., Cai, T., & Kohane, I. S. (2019) · Pacific Symposium on Biocomputing (PSB) · doi:10.1142/9789811215636_0027 · Cited by 197 · see citing works
A Survey on Multimodal Data-Driven Smart Healthcare Systems: Approaches and Applications
Cai, Q., Wang, H. H., Li, Z., & Liu, X. (2019) · IEEE Access · doi:10.1109/access.2019.2941419 · Cited by 165 · see citing works
Alizadehsani, R., Hosseini, M. J., Khosravi, A., Khozeimeh, F., Roshanzamir, M., Sarrafzadegan, N., & Nahavandi, S. (2018) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2018.05.009 · Cited by 128 · see citing works
Hossain, M. E., Uddin, S., & Khan, A. M. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113918 · Cited by 107 · see citing works
Causal GraphSAGE: A robust graph method for classification based on causal sampling
Zhang, T., Shan, H., & Little, M. A. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.108696 · Cited by 75 · see citing works
Branchy-GNN: A Device-Edge Co-Inference Framework for Efficient Point Cloud Processing
Shao, J., Zhang, H., Mao, Y., & Zhang, J. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414831 · Cited by 55 · see citing works
Chen, H., Gao, M., Zhang, Y., Liang, W., & Zou, X. (2019) · BioMed Research International · doi:10.1155/2019/9523719 · Cited by 47 · see citing works
CheXGAT: A disease correlation-aware network for thorax disease diagnosis from chest X-ray images
Lee, Y., Huang, S.-K., & Chang, R. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102382 · Cited by 46 · see citing works
Olawade, D. B., Soladoye, A. A., Omodunbi, B. A., Aderinto, N., & Adeyanju, I. (2025) · International Journal of Cardiology · doi:10.1016/j.ijcard.2025.133443 · Cited by 38 · see citing works
Data-driven decision-making for precision diagnosis of digestive diseases
Jiang, S., Wang, T., & Zhang, K.-H. (2023) · BioMedical Engineering OnLine · doi:10.1186/s12938-023-01148-1 · Cited by 29 · see citing works
Complications in Acute Myocardial Infarction: Navigating Challenges in Diagnosis and Management
Moras, E., Yakkali, S., Gandhi, K., Virk, H. U. H., Alam, M., Zaid, S., Barman, N., Jneid, H. M., Vallabhajosyula, S., Sharma, S. K., et al. (2024) · Hearts · doi:10.3390/hearts5010009 · Cited by 21 · see citing works
Vital sign monitoring using wearable devices in a Vietnamese intensive care unit
Van, H. M. T., Hảo, N. V., Khanh, P. N. Q., Hai, H. B., Khoa, L. D. V., Yen, L. M., Nhat, P. T. H., Duong, H. T. H., Thuy, D. B., Zhu, T., et al. (2021) · BMJ Innovations · doi:10.1136/bmjinnov-2021-000707 · Cited by 20 · see citing works
Xuan, P., Wu, X., Cui, H., Jin, Q., Wang, L., Zhang, T., Nakaguchi, T., & Duh, H. B. L. (2022) · Applied Soft Computing · doi:10.1016/j.asoc.2022.109905 · Cited by 20 · see citing works
Self-citationThe importance of graph databases and graph learning for clinical applications
Walke, D., Micheel, D., Schallert, K., Muth, T., Broneske, D., Saake, G., & Heyer, R. (2023) · Database · doi:10.1093/database/baad045 · Cited by 17 · see citing works
A GNN-Based False Data Detection Scheme for Smart Grids
Qiu, J., Zhang, X., Wang, T., Hou, H., Wang, S., & Yang, T. (2025) · Algorithms · doi:10.3390/a18030166 · Cited by 17 · see citing works
Graph Neural Network Training Systems: A Performance Comparison of Full-Graph and Mini-Batch
Bajaj, S., Son, H., Liu, J., Guan, H., & Serafini, M. (2024) · Proceedings of the VLDB Endowment · doi:10.14778/3717755.3717776 · Cited by 15 · see citing works
Mohanraj, P., Raman, V., & Ramanathan, S. (2024) · Diagnostics · doi:10.3390/diagnostics14192181 · Cited by 10 · see citing works
A review of feature selection strategies utilizing graph data structures and Knowledge Graphs
Shao, S., Ribeiro, P. H., Ramirez, C. M., & Moore, J. H. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae521 · Cited by 10 · see citing works
Tahirovic, A. A., Angeli, D., Tahirović, A., & Štrbac, G. (2025) · Scientific Reports · doi:10.1038/s41598-025-01856-9 · Cited by 6 · see citing works
OmniSage: Large Scale, Multi-Entity Heterogeneous Graph Representation Learning
Badrinath, A., Yang, A., Rajesh, K., Agarwal, P., Yang, J., Chen, H., Xu, J., & Rosenberg, C. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737253 · Cited by 6 · see citing works
PREDICTING MYOCARDIAL INFARCTION COMPLICATIONS AND OUTCOMES WITH DEEP LEARNING
YAVRU, İ. B., & Yılmaz, S. (2022) · Eskişehir Technical University Journal of Science and Technology A - Applied Sciences and Engineering · doi:10.18038/estubtda.1056821 · Cited by 5 · see citing works
A Multitask Deep Learning Model for Predicting Myocardial Infarction Complications
Makhmudov, F., Равшанов, Н., Akhmedov, D., Pekos, O., Turimov, D., & Cho, Y. I. (2025) · Bioengineering · doi:10.3390/bioengineering12050520 · Cited by 5 · see citing works
Research on Medical Multi-Source Data Fusion Based on Big Data
Sheng-li, H. (2020) · Recent Advances in Computer Science and Communications · doi:10.2174/2666255813666200831181155 · Cited by 4 · see citing works
Alawad, D. M., Katebi, A., & Hoque, M. T. (2024) · Machine Learning and Knowledge Extraction · doi:10.3390/make6030089 · Cited by 4 · see citing works
Graph Neural Network Training Systems: A Performance Comparison of Full-Graph and Mini-BatchPreprint
Bajaj, S., Son, H., Liu, J., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.00552 · Cited by 4 · see citing works
Евгеньевич, Г. С., Dorrer, M. G., Nikulina, S. Y., Orlova, Y. V., & Pelipeckaya, E. Y. (2020) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1679/4/042017 · Cited by 3 · see citing works
Sokač, M., Skračić, B., Kučak, D., & Mršić, L. (2024) · Machine Learning and Knowledge Extraction · doi:10.3390/make6030100 · Cited by 3 · see citing works
Enhancing Coronary Artery Disease Classification Using Optimized MLP Based on Genetic Algorithm
Hashemi, M., Komamardakhi, S. S. S., Maftoun, M., Zare, O., Joloudari, J. H., Nematollahi, M. A., Alizadehsani, R., Sala, P., & Górriz, J. M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-61140-7_11 · Cited by 2 · see citing works
Harnessing GraphSAGE for Learning Representations of Massive Transactional Networks
Tare, M., Rattasits, C., Wu, Y., & Wielewski, E. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-94139-9_17 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- Gori, M.; Monfardini, G.; Scarselli, F. A New Model for Learning in Graph Domains. In Proceedings of the IEEE International Joint Conference on Neural Network (IJCNN), Montreal, QC, Canada, 31 July-4 August 2005; pp. 729-734.
- Mann, D.L.; Zipes, D.P.; Libby, P.; Bonow, R.O. Braunwald's Heart Disease: A Textbook of Cardiovascular Medicine, 10th ed.; Elsevier Health Sciences: Amsterdam, The Netherlands, 2018; pp. 636-661.
- Alizadehsani, R.; Roshanzamir, M.; Sani, Z. Extention of Z-Alizadeh Sani Dataset Data Set; UCI Irvine-Machine Leaming-Repository: Irvine, CA, USA, 2013. [CrossRef] · DOI
- Golovenkin, S.E.; Shulman, V.A.; Rossiev, D.A.; Shesternya, P.A.; Nikulina, S.Y.; Orlova, Y.V. Myocardial Infarction Complications Data Set; UC Irvine-Machine Leaming-Repository: Irvine, CA, USA, 2020. [CrossRef] · DOI
Cited by
Sorted by citations
Anbukkarasi, S., Hemalatha, S., Balakrishnan, A., Varadhaganapathy, S., & Easwaramoorthy, S. V. (2026) · Scientific reports · doi:10.1038/s41598-026-47146-w · Cited by 1 · see citing works
“…with the help of a knowledge projection function: that made it possible to integrate semantic data from various heterogeneous biomedical sources and to facilitate knowledge guided reasoning as can be seen from the ablation and performance improvements mentioned in the results Sects 53,63 ..”
Adaptive Cross-Modal Gated Framework for Intelligent Medical Diagnosis using Multimodal Data Fusion
Chang, H. (2025) · International Conference on Mobile Networks and Wireless Communications (ICMNWC) · doi:10.1109/icmnwc66779.2025.11354373
A Comprehensive Study of Graph Neural Networks: Types and Application
Dubey, R., Kaur, R., & Ahuja, N. (2026) · Expert Systems · doi:10.1111/exsy.70429
Liao, R., Yang, X., & Wang, Z. (2026) · IEEE Access · doi:10.1109/access.2026.3742949
- Authors
- Zhang, L., Li, T., Cui, H., Zhang, Q., Jiang, Z., Li, J., Welsch, R. E., & Jia, Z.
- Affiliations
- Peking University (CN)
- Chinese Academy of Medical Sciences & Peking Union Medical College (CN)
- Beijing University of Posts and Telecommunications (CN)
- China University of Petroleum, Beijing (CN)
- Massachusetts Institute of Technology (US)
- Corresponding author
- Jia, Zhongwei
- Venue
- Machine Learning and Knowledge Extraction
- Publisher
- MDPI AG
- Published
- 2025-09-02
- Volume
- 7
- Issue
- 3
- Pages
- 92
- ISSN
- 2504-4990 (ISSN-L)
- OpenAlex
- W4413916395
- Cited by
- 4
- References
- 45
- Citations by year
- 2026: 3 · 2025: 1
- FWCI
- 1.61
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Machine Learning and Knowledge Extraction (publishedVersion, cc-by)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Research on Vaccination Strategy and Policy of COVID-19 Pneumonia Based on Big Data Modeling (72174004, 2023YFC2308703, Discipline Backbone-02-12, 72274210)
- National Key Research and Development Program of China (72174004, 2023YFC2308703, Discipline Backbone-02-12, 72274210)
- High-level Public Health Talent Development Program of Beijing (72174004, 2023YFC2308703, Discipline Backbone-02-12, 72274210)
- Research on Early Warning Signal Recognition and Evaluation of Acute Respiratory Infectious Diseases in Hospitals Based on Medical Prevention Integration (72174004, 2023YFC2308703, Discipline Backbone-02-12, 72274210)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- multimodal medical data, graph neural networks, computer-aided diagnosis, data fusion, patient classification, heart disease prediction, node classification, unstructured medical data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
