Cited by 59Machine Learning in HealthcareMultimodal Machine Learning ApplicationsArtificial Intelligence in Healthcare and Education

EHR-KnowGen: Knowledge-enhanced multimodal learning for disease diagnosis generation

Open access
Niu, Shuai, Ma, Jing, Bai, Liang, Wang, Zhihua, Guo, Li, Yang, Xian·2024·Information Fusion

Abstract

Electronic health records (EHRs) contain diverse patient information, including medical notes, clinical events, and laboratory test results. Integrating this multimodal data can improve disease diagnoses using deep learning models. However, effectively combining different modalities for diagnosis remains challenging. Previous approaches, such as attention mechanisms and contrastive learning, have attempted to address this but do not fully integrate the modalities into a unified feature space. This paper presents EHR-KnowGen, a multimodal learning model enhanced with external domain knowledge, for improved disease diagnosis generation from diverse patient information in EHRs. Unlike previous approaches, our model integrates different modalities into a unified feature space with soft prompts learning and leverages large language models (LLMs) to generate disease diagnoses. By incorporating external domain knowledge from different levels of granularity, we enhance the extraction and fusion of multimodal information, resulting in more accurate diagnosis generation. Experimental results on real-world EHR datasets demonstrate the superiority of our generative model over comparative methods, providing explainable evidence to enhance the understanding of diagnosis results.

Integrity signals

Registry

Crossref · 10.1016/j.inffus.2023.102069

Cites

50 of 50 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Every study conducting mechanistic or internal analysis necessarily used open-source models.

    “Notably, every study conducting mechanistic or internal analysis: influence functions [129], attention visualization [113,116, 128], activation patching [114], necessarily used open-source models.”

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Niu, S., Ma, J., Bai, L., Wang, Z., Guo, L., & Yang, X.
Affiliations
  • Hong Kong Baptist University (HK)
  • Shanxi University (CN)
  • Zhejiang University (CN)
  • Manchester Metropolitan University (GB)
  • University of Manchester (GB)
Corresponding author
Yang, Xian
Venue
Information Fusion
Publisher
Elsevier BV
Published
2024-02
Volume
102
Pages
102069
ISSN
1566-2535 (ISSN-L)
OpenAlex
W4387627515
Cited by
59
References
50
Citations by year
2026: 17 · 2025: 30 · 2024: 12
FWCI
5.56
Type
journal article
Language
en
Open access
Locations
  • Information Fusion (publishedVersion, cc-by)
  • e-space (Manchester Metropolitan University) (submittedVersion)
  • Research Explorer (The University of Manchester) (submittedVersion, cc-by)
Funders
  • Key Technologies Research and Development Program
  • National Key Research and Development Program of China (2021ZD0113303)
  • National Natural Science Foundation of China (62022052, 62276159)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
electronic health records, multimodal learning, soft prompts, large language models, clinical notes, clinical events, laboratory findings, explainable diagnosis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review