An intelligent healthcare system for rare disease diagnosis utilizing electronic health records based on a knowledge-guided multimodal transformer framework
Open accessAbstract
Rare diseases are a common problem with millions of patients globally, but their diagnosis is difficult because of varied clinical presentations, small sample size, and disparate biomedical data sources. Current diagnostic tools are not able to combine multimodal information effectively, which results in a timely or wrong diagnosis. To fill this gap, this paper suggests a smart multimodal healthcare framework integrating electronic health records (EHRs), genomic sequences, and medical imaging to improve the detection of rare diseases. The framework uses Swin Transformer to extract hierarchical visual features in radiographic scans, Med-BERT and Transformer-XL to learn semantic and long-term temporal relations in longitudinal electronic health record narratives, and a Graph Neural Network (GNN)-based encoder to learn functional and structural relations in genomic sequences. The alignment of the cross-modal representation is further boosted with a Knowledge-Guided Contrastive Learning (KGCL) mechanism, which takes advantage of rare disease ontologies in Orphanet to improve the interpretability of the model and infusion of knowledge. To achieve strong performance, the Nutcracker Optimization Algorithm (NOA) is proposed to optimize hyperparameters, calibrate attention mechanisms, and enhance multimodal fusion. Experimental results on MIMIC-IV (EHR), ClinVar (genomics), and CheXpert (imaging) datasets show that the proposed framework significantly outperforms the state-of-the-art multimodal baselines in terms of accuracy and robustness of early rare disease diagnosis. This paper presents the opportunity to integrate hierarchical vision transformers, domain-specific language models, graph-based genomic encoders, and knowledge-directed optimization to make explainable, accurate, and clinically applicable healthcare decisions in rare disease settings.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Knowledge-guided multimodal transformer frameworks use contrastive learning and cross-modal alignment to increase interpretability and model robustness.
“Knowledge guided multimodal transformer frameworks also use contrastive learning and cross modal alignment to increase interpretability and model robustness42.”
From K-STAMM: a knowledge-enhanced spatial - temporal attention model with multimodal fusion for pneumonia prediction (2026) · cites this paper as [42]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · cited 1× · see citing works
“Swin Transformer [27] architecture is constructed with Layer Normalization (LN), Win-dow-based Multi-Head Self-Attention (W-MSA), Shifted Window-based Multi-Head Self-Attention (SW-MSA), and a two-layer Multi-Layer Perceptron (MLP) with GELU nonlinearity.”
MIMIC-IV, a freely accessible electronic health record dataset
Johnson, A. E. W., Bulgarelli, L., Shen, L., Gayles, A., Shammout, A., Horng, S., Pollard, T. J., Hao, S., Moody, B., Gow, B., et al. (2023) · Scientific Data · doi:10.1038/s41597-022-01899-x · Cited by 3,401 · cited 1× · see citing works
“The MIMIC-IV [33] is a large-scale, de-identified database that offers comprehensive electronic health records (EHRs) of the patients who were admitted to the Beth Israel Deaconess Medical Center in Boston, Massachusetts.”
Khemani, B., Patil, S., Kotecha, K. V., & Tanwar, S. (2024) · Journal Of Big Data · doi:10.1186/s40537-023-00876-4 · Cited by 638 · see citing works
Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook.
AlSaad, R., Abd-alrazaq, A., Boughorbel, S., Ahmed, A., Renault, M.-A., Damseh, R., & Sheikh, J. (2024) · Journal of medical Internet research · doi:10.2196/59505 · Cited by 349 · cited 1× · see citing works
“Transformers are more adaptable, have better feature representation, and better ability to incorporate heterogeneous data, which makes them especially suitable for multimodal healthcare applications compared to CNNs [10].”
Zhou, H.-Y., Yu, Y., Wang, C., Zhang, S., Gao, Y., Pan, J., Shao, J., Lu, G., Zhang, K., & Li, W. (2023) · Nature Biomedical Engineering · doi:10.1038/s41551-023-01045-x · Cited by 343 · see citing works
Wani, N. A., Kumar, R., & Bedi, J. (2023) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2023.107879 · Cited by 318 · see citing works
Transformer Architecture and Attention Mechanisms in Genome Data Analysis: A Comprehensive Review
Choi, S. R., & Lee, M. (2023) · Biology · doi:10.3390/biology12071033 · Cited by 265 · see citing works
A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis
Xu, X., Li, J., Zhu, Z., Zhao, L., Wang, H., Song, C., Chen, Y., Zhao, Q., Yang, J., & Pei, Y. (2024) · Bioengineering · doi:10.3390/bioengineering11030219 · Cited by 242 · see citing works
Boulahia, S. Y., Amamra, A., Madi, M. R., & Daikh, S. (2021) · Machine Vision and Applications · doi:10.1007/s00138-021-01249-8 · Cited by 236 · see citing works
Recent trend in medical imaging modalities and their applications in disease diagnosis: a review
Abhisheka, B., Biswas, S. K., Purkayastha, B., Das, D., & Escargueil, A. E. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17326-1 · Cited by 181 · see citing works
The use of SNOMED CT, 2013-2020: a literature review
Chang, E., & Mostafa, J. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab084 · Cited by 151 · see citing works
Progress, challenges and global approaches to rare diseases
Groft, S. C., de la Paz, M. P., & Taruscio, D. (2021) · Acta Paediatrica · doi:10.1111/apa.15974 · Cited by 119 · see citing works
Knowledge-Adaptive Contrastive Learning for Recommendation
Wang, H., Xu, Y., Yang, C. H., et al. (2023) · doi:10.1145/3539597.3570483 · Cited by 90 · see citing works
Med-BERT: A Pretraining Framework for Medical Records Named Entity Recognition
Liu, N., Hu, Q., Xu, H., Xu, X., & Chen, M. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3131180 · Cited by 89 · see citing works
EHR-KnowGen: Knowledge-enhanced multimodal learning for disease diagnosis generation
Niu, S., Ma, J., Bai, L., Wang, Z., Guo, L., & Yang, X. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102069 · Cited by 59 · see citing works
A Review on CLAHE Based Enhancement Techniques
Sharma, R., & Kamra, A. (2023) · International Conference on Contemporary Computing and Informatics (IC3I) · doi:10.1109/ic3i59117.2023.10397722 · Cited by 55 · see citing works
Handbook of Medical and Health Sciences in Developing Countries
Al‐Worafi, Y. M. (2023) · Springer eBooks · doi:10.1007/978-3-030-74786-2 · Cited by 53 · see citing works
Parameter Extraction of Solar Photovoltaic Model Based on Nutcracker Optimization Algorithm
Duan, Z., Yu, H., Zhang, Q., et al. (2023) · Applied Sciences · doi:10.3390/app13116710 · Cited by 42 · see citing works
Mir, A. N., Rizvi, D. R., & Ahmad, M. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110519 · Cited by 40 · see citing works
Singh, S., Wani, N. A., Kumar, R., & Bedi, J. (2025) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2025.110470 · Cited by 36 · see citing works
Mir, A. N., & Rizvi, D. R. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111413 · Cited by 34 · see citing works
Mao, X., Huang, Y., Jin, Y., Wang, L., Chen, X., Liu, H., Yang, X., Xu, H., Luan, X., Xiao, Y., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01452-1 · Cited by 33 · see citing works
Rare diseases: a comprehensive literature review and future directions
Chaudhary, A., & Kumar, V. (2025) · Journal of Rare Diseases · doi:10.1007/s44162-025-00099-6 · Cited by 33 · see citing works
The interdisciplinary diagnosis of rare diseases—results of the Translate-NAMSE project
Rillig, F., Grüters, A., Schramm, C., & Krude, H. (2022) · Deutsches Ärzteblatt international · doi:10.3238/arztebl.m2022.0219 · Cited by 32 · see citing works
Ontology-driven and weakly supervised rare disease identification from clinical notes
Dong, H., Suárez-Paniagua, V., Zhang, H., Wang, M., Casey, A., Davidson, E., Chen, J., Alex, B., Whiteley, W. N., & Wu, H. (2023) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-023-02181-9 · Cited by 31 · see citing works
Manemann, S. M., Sauver, J. L. S., Liu, H., Larson, N. B., Moon, S., Takahashi, P. Y., Olson, J. E., Rocca, W. A., Miller, V. M., Therneau, T. M., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-044353 · Cited by 30 · see citing works
Identification of Genomic Variants Causing Variation in Quantitative Traits: A Review
Meuwissen, T. H. E., Hayes, B. J., MacLeod, I. M., & Goddard, M. E. (2022) · Agriculture · doi:10.3390/agriculture12101713 · Cited by 27 · see citing works
Botros, J., Mourad-Chehade, F., & Laplanche, D. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106997 · Cited by 24 · see citing works
Li, T. Z., Still, J. M., Xu, K., Lee, H. H., Cai, L. Y., Krishnan, A. R., Gao, R., Khan, M. S., Antic, S., Kammer, M. N., et al. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43895-0_61 · Cited by 22 · see citing works
Neves, F. S. P., Claro, R. M., & Pinto, A. M. (2023) · Sensors · doi:10.3390/s23052434 · Cited by 19 · see citing works
van Karnebeek, C., O’Donnell‐Luria, A., Baynam, G. S., Baudot, A., Groza, T., Jans, J. J. M., Lassmann, T., Letinturier, M. C. V., Montgomery, S. B., Robinson, P. N., et al. (2024) · Orphanet Journal of Rare Diseases · doi:10.1186/s13023-024-03361-0 · Cited by 17 · see citing works
Moynihan, D., Monaco, S., Ting, T. W., Narasimhalu, K., Hsieh, J. Y. C., Kam, S., Lim, J. Y., Lim, W. K., Dávila, S., Bylstra, Y. M., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-55424-8 · Cited by 14 · see citing works
Chen, Y., Guo, X., Xia, Y., et al. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3349284 · Cited by 13 · see citing works
Jordan, D. M., Vy, H. M. T., & Do, R. (2023) · medRxiv · doi:10.1101/2023.12.21.23300393 · Cited by 7 · see citing works
Rare Diseases Severity Prediction System Using a Machine Learning-Based Technique
Bahadure, N. B., Dash, S., Padhy, S. K., et al. (2023) · International Conference on Artificial Intelligence for Innovations in Healthcare Industries (ICAIIHI) · doi:10.1109/icaiihi57871.2023.10489179 · Cited by 5 · see citing works
A U-Shaped Architecture Based on Hybrid CNN and Mamba for Medical Image Segmentation
Ma, X., Du, Y., & Sui, D. (2025) · Applied Sciences · doi:10.3390/app15147821 · Cited by 5 · see citing works
Biologically Informed Variational Inference Enables Interpretable Cell Phenotyping and DiscoveryPreprint
Arnoldt, L., zu Belzen, J. U., Herrmann, L., Nguyen, K. P., Ishaque, N., Theis, F. J., Wild, B., & Eils, R. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.06.10.657924 · Cited by 4 · see citing works
Exploiting Uncertainty: A Transformer-Based Multi-Disease Diagnostic Approach to Chest X-RAYS
Songfeng, D., & Xingwu, C. (2023) · International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP) · doi:10.1109/iccwamtip60502.2023.10387037 · Cited by 2 · see citing works
Semantic-guided Unknown-aware Rare Disease Diagnosis System
Luo, Y., & Yuan, Y. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385449 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- [29]Li Z, Huang Q, Yang X, Chen Q. Li Zhang. Automatic Composition system based on Transformer-XL. Appl Sci. 2024;14:13.
- [34]ClinVar D. Retrieved from https://www.ncbi.nlm.nih.gov/clinvar/
- [39]Belete DM, Manjaiah D, Huchaiah. Grid search in hyperparameter optimization of machine learning models for prediction of HIV/AIDS test results. Int J Comput Appl. 2022;44(9):875–86.
- [45]Arnoldt L. Julius Upmeier zu Belzen, Luis Herrmann, Khue Nguyen, Fabian Theis, Benjamin Wild, and Roland Eils. Biologically guided variational inference for interpretable multimodal single-cell integration and mechanistic discovery. bioRxiv. 2025:2025-06.
Cited by
Sorted by citations
Anbukkarasi, S., Hemalatha, S., Balakrishnan, A., Varadhaganapathy, S., & Easwaramoorthy, S. V. (2026) · Scientific reports · doi:10.1038/s41598-026-47146-w · Cited by 1 · see citing works
- Authors
- Abugabah, A., Shukla, P. K., Shukla, P. K., & Pandey, A.
- Affiliations
- Zayed University (AE)
- Rajiv Gandhi Technical University (IN)
- Manipal University Jaipur (IN)
- Corresponding author
- Pandey, Ankur
- ORCID
- Abugabah, Ahed
- Venue
- BioData Mining
- Published
- 2025-10-07
- Volume
- 18
- Issue
- 1
- Pages
- 70-70
- ISSN
- 1756-0381 (ISSN-L)
- PMC
- PMC12505588
- OpenAlex
- W4414891414
- Cited by
- 11
- References
- 39
- Citations by year
- 2026: 10 · 2025: 1
- FWCI
- 4.41
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BioData Mining (publishedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Manipal University Jaipur
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- rare disease diagnosis, electronic health records, multimodal fusion, Swin Transformer, Transformer-XL, nutcracker optimization algorithm, diagnostic imaging, genome sequence, MIMIC-IV, ClinVar, CheXpert
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
