Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index.

Artificial intelligence-based methods for fusion of electronic health records and imaging data

Open access
Mohsen, Farida, Ali, Hazrat, El Hajj, Nady, Shah, Zubair · 2022 · Scientific Reports

Abstract

Healthcare data are inherently multimodal, including electronic health records (EHR), medical images, and multi-omics data. Combining these multimodal data sources contributes to a better understanding of human health and provides optimal personalized healthcare. The most important question when using multimodal data is how to fuse them—a field of growing interest among researchers. Advances in artificial intelligence (AI) technologies, particularly machine learning (ML), enable the fusion of these different data modalities to provide multimodal insights. To this end, in this scoping review, we focus on synthesizing and analyzing the literature that uses AI techniques to fuse multimodal medical data for different clinical applications. More specifically, we focus on studies that only fused EHR with medical imaging data to develop various AI methods for clinical applications. We present a comprehensive analysis of the various fusion strategies, the diseases and clinical outcomes for which multimodal fusion was used, the ML algorithms used to perform multimodal fusion for each clinical application, and the available multimodal medical datasets. We followed the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines. We searched Embase, PubMed, Scopus, and Google Scholar to retrieve relevant studies. After pre-processing and screening, we extracted data from 34 studies that fulfilled the inclusion criteria. We found that studies fusing imaging data with EHR are increasing and doubling from 2020 to 2021. In our analysis, a typical workflow was observed: feeding raw data, fusing different data modalities by applying conventional machine learning (ML) or deep learning (DL) algorithms, and finally, evaluating the multimodal fusion through clinical outcome predictions. Specifically, early fusion was the most used technique in most applications for multimodal learning (22 out of 34 studies). We found that multimodality fusion models outperformed traditional single-modality models for the same task. Disease diagnosis and prediction were the most common clinical outcomes (reported in 20 and 10 studies, respectively) from a clinical outcome perspective. Neurological disorders were the dominant category (16 studies). From an AI perspective, conventional ML models were the most used (19 studies), followed by DL models (16 studies). Multimodal data used in the included studies were mostly from private repositories (21 studies). Through this scoping review, we offer new insights for researchers interested in knowing the current state of knowledge within this research field.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41598-022-22514-4

Cites

72 of 72 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 PRISMA flow chart for study identification, screening, and selection.
Figure 1 PRISMA flow chart for study identification, screening, and selection.CC BY · PubMed Central
Figure 2 The distribution of studies by the type of publication and the year.
Figure 2 The distribution of studies by the type of publication and the year.CC BY · PubMed Central
Figure 3 Fusion strategies associated with clinical outcomes for different diseases.
Figure 3 Fusion strategies associated with clinical outcomes for different diseases.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 68 references

Research works printed but not linked: 2

  1. Alistair, J. et al. Mimic-iv (version 0.4). PhysioNet https://doi.org/10.13026/a3wn-hq05 (2020). · DOI
  2. Goodfellow, I., Bengio, Y. & Courville, A. Deep Learning (MIT press, 2016).

Cited by

See all 251

Metadata

Authors
Mohsen, F., Ali, H., El Hajj, N., & Shah, Z.
Affiliations
Hamad bin Khalifa University (QA)
Corresponding author
Shah, Zubair
Venue
Scientific Reports
Published
2022-10-26
Volume
12
Issue
1
Pages
17981-17981
ISSN
2045-2322 (ISSN-L)
OpenAlex
W4307539314
Cited by
250
References
68
Citations by year
2026: 75 · 2025: 101 · 2024: 49 · 2023: 23 · 2022: 2
FWCI
14.40
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Figshare (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
Qatar National Library
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
multimodal fusion, electronic health records, diagnostic imaging, multimodal learning, feature-level fusion, disease diagnosis, clinical outcome prediction, neurological disorders, deep learning, machine learning
MeSH terms
Machine Learning, Algorithms, Artificial Intelligence, Diagnostic Imaging, Humans, Electronic Health Records

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Artificial intelligence-based methods for fusion of electronic health records and imaging data. https://app.cochat.ai/papers/10.1038/s41598-022-22514-4
The paper
Mohsen, F., Ali, H., El Hajj, N., & Shah, Z. (2022). Artificial intelligence-based methods for fusion of electronic health records and imaging data. Scientific Reports. https://doi.org/10.1038/s41598-022-22514-4

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.