Cited by 207Multimodal Machine Learning ApplicationsArtificial Intelligence in Healthcare and EducationRadiomics and Machine Learning in Medical Imaging

Dynamic Graph Enhanced Contrastive Learning for Chest X-Ray Report Generation

Open access
Li, Mingjie, Lin, Bingqian, Chen, Zicong, Lin, Haokun, Liang, Xiaodan, Chang, Xiaojun·2023·2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Abstract

Automatic radiology reporting has great clinical potential to relieve radiologists from heavy workloads and improve diagnosis interpretation. Recently, researchers have enhanced data-driven neural networks with medical knowledge graphs to eliminate the severe visual and textual bias in this task. The structures of such graphs are exploited by using the clinical dependencies formed by the disease topic tags via general knowledge and usually do not update during the training process. Consequently, the fixed graphs can not guarantee the most appropriate scope of knowledge and limit the effectiveness. To address the limitation, we propose a knowledge graph with Dynamic structure and nodes to facilitate chest X-ray report generation with Contrastive Learning, named DCL. In detail, the fundamental structure of our graph is pre-constructed from general knowledge. Then we explore specific knowledge extracted from the retrieved reports to add additional nodes or redefine their relations in a bottom-up manner. Each image feature is integrated with its very own updated graph before being fed into the decoder module for report generation. Finally, this paper introduces Image-Report Contrastive and Image-Report Matching losses to better represent visual features and textual information. Evaluated on IU-Xray and MIMIC-CXR datasets, our DCL outperforms previous state-of-the-art models on these two benchmarks.

Integrity signals

Registry

Crossref · 10.1109/cvpr52729.2023.00325

Cites

45 of 45 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • It is not possible to quantitatively compare MAIRA-2 to models trained to generate other sections, such as Impression or both Findings and Impression together.

    “It is not possible to quantitatively compare to models trained to generate other sections, such as Impression 17 or both Findings and Impression together.15,58”

    From MAIRA-2: Grounded Radiology Report Generation (2024) · cites this paper as [15]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Li, M., Lin, B., Chen, Z., Lin, H., Liang, X., & Chang, X.
Affiliations
  • University of Technology Sydney (AU)
  • Sun Yat-sen University (CN)
  • University of Hong Kong (HK)
Venue
2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2023-06
Pages
3334-3343
ISSN
1063-6919 (ISSN-L)
OpenAlex
W4386076127
Cited by
206
References
45
Citations by year
2026: 52 · 2025: 86 · 2024: 54 · 2023: 14
FWCI
22.03
Type
conference paper
Language
en
Open access
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
  • arXiv (Cornell University)
Funders
National Key R&D Program of China (2020AAA0109700)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
Keywords
chest X-ray report generation, contrastive learning, dynamic knowledge graphs, medical knowledge graph, MIMIC-CXR

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review