Automatic radiology report generation with deep learning: a comprehensive review of methods and advances
Open accessAbstract
Automatic report generation refers to the process of generating medical reports from medical images without the need for manual intervention, enabling faster, more consistent, and objective analysis of radiological data. The rapid progress in deep learning, particularly in the fields of computer vision and natural language processing, has significantly improved the efficacy of this approach. By leveraging deep learning techniques, which seamlessly integrate image analysis with natural language generation, these methods have shown promise in interpreting complex medical images and producing highly accurate textual descriptions. In this paper, we provide a thorough review of various deep learning models and techniques employed for generating radiological reports, with a focus on chest X-ray images as a representative case. We propose a unified encoder-decoder framework that consists of an image encoder for extracting feature representations from medical images, a language decoder for generating textual reports, and enhancement components designed to refine model performance. Through a comprehensive comparison of existing state-of-the-art methods on the widely utilized MIMIC-CXR dataset, we highlight the innovative contributions made by recent advancements in the field. Furthermore, we discuss the current challenges and identify potential research directions for future advancements in this field.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Many existing vision-language models rely on simplistic feature concatenation strategies that fail to capture fine-grained correspondences between localised radiographic findings and their semantic descriptions.
“these models often rely on simplistic feature concatenation strategies that fail to capture fine-grained correspondences between localised radiographic findings and their semantic descriptions (11, 16).”
From Swin-Qwen3: a three-stage vision–language framework for automated radiology report generation with multi-agent verification (2026) · cites this paper as [16]
Section 4.2.2 describes some report-generation methods that use straightforward concatenation, and §3.1 identifies a deficiency in precise image–text correspondence. The review does not say that many vision-language models use concatenation or establish that concatenation causes the failure to align localized findings with their descriptions. Page numbers were not available in our copy. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 136 · Cited by 7
Research works printed but not linked: 20
- Banerjee S, Lavie A (2005) Meteor: an automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pp 65-72
- Chen W, Liu Y, Wang C, Li G, Zhu J, Lin L (2023a) Visual-linguistic causal intervention for radiology report generation. arXiv preprint arXiv:2303.09117. https://doi.org/10.48550/arXiv.2303.09117 · DOI
- Forgues G, Pineau J, Larchevêque JM, Tremblay R (2014) Bootstrapping dialog systems with word embed- dings. In Nips modern machine learning and natural language processing workshop 2:168
- Irvin J, Rajpurkar P, Ko M, Yu Y, Ciurea-Ilcus S, Chute C, Marklund H, Haghgoo B, Ball R, Shpanskaya K et al (2019) Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. Proc AAAI Conf Artif Intell 33:590-597
- Jain S, Agrawal A, Saporta A, Truong SQ, Duong DN, Bui T, Chambon P, Zhang Y, Lungren MP, Ng AY et al (2021b) Radgraph: Extracting clinical entities and relations from radiology reports. arXiv preprint arXiv:2106.14463. https://doi.org/10.48550/arXiv.2106.14463 · DOI · Paper page
- Johnson AE, Pollard TJ, Berkowitz SJ, Greenbaum NR, Lungren MP, Deng C, Mark RG, Horng S (2019b) Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports. Sci Data 6(1):317. https://doi.org/10.1038/s41597-019-0322-0 · DOI · Paper page
- Kiros R, Zhu Y, Salakhutdinov RR, Zemel R, Urtasun R, Torralba A, Fidler S (2015) Skip-thought vectors. Adv Neural Inf Process Syst 28
- Li Y, Liang X, Hu Z, Xing EP (2018) Hybrid retrieval-generation reinforced agent for medical image report generation. Adv Neural Inf Process Syst 10(5555/3326943):3327084
- Li M, Cai W, Liu R, Weng Y, Zhao X, Wang C, Chen X, Liu Z, Pan C, Li M et al (2021b) Ffa-ir: Towards an explainable and reliable medical report generation benchmark. In Thirty-fifth conference on neural information processing systems datasets and benchmarks track (Round 2)
- Liu G, Hsu TMH, McDermott M, Boag W, Weng WH, Szolovits P, Ghassemi M (2019) Clinically accurate chest x-ray report generation. In Doshi-Velez F, Fackler J, Jung K, Kale D, Ranganath R, Wallace B, and Wiens J (Eds.), Proceedings of the 4th machine learning for healthcare conference, volume 106 of proceedings of machine learning research. PMLR, pp 249-269
- 0 4 3 · DOI
- Peng Y, Wang X, Lu L, Bagheri M, Summers R, Lu Z (2018) Negbio: a high-performance tool for negation and uncertainty detection in radiology reports. AMIA Summits on Translational Science Proceedings 2018:188. https://doi.org/10.13140/RG.2.2.19745.10087 · DOI
- Ramos J et al (2003) Using tf-idf to determine word relevance in document queries. In Proceedings of the first instructional conference on machine learning, 242, pp 29-48.
- Rückert J, Ben Abacha A, Seco de Herrera AG, Bloch L, Brüngel R, Idrissi-Yaghir A, Schäfer H, Müller H, Friedrich CM (2023) Overview of imageclefmedical 2023-caption prediction and concept detection. In Working notes of the conference and labs of the evaluation forum (CLEF 2023), 3497, pp 1328-1346
- Song X, Zhang X, Ji J, Liu Y, Wei P (2022) Cross-modal contrastive attention model for medical report generation. In Proceedings of the 29th international conference on computational linguistics, Gyeongju, Republic of Korea, pp 2388-2397. International Committee on Computational Linguistics
- Wang Z, Yu J, Yu AW, Dai Z, Tsvetkov Y, Cao Y (2022g) SimVLM: Simple visual language model pretrain- ing with weak supervision. In International conference on learning representations
- Wu JT, Agu NN, Lourentzou I, Sharma A, Paguio JA, Yao JS, Dee EC, Mitchell WG, Kashyap S, Giovannini A, Celi LA, Moradi M (2021b) Chest imagenome dataset for clinical reasoning. In Thirty-fifth confer- ence on neural information processing systems datasets and benchmarks track (Round 2)
- Xu K Ba J, Kiros R, Cho K, Courville A, Salakhudinov R, Zemel R, Bengio Y (2015) Show, attend and tell: neural image caption generation with visual attention. In F Bach and D Blei (Eds.), Proceedings of the 32nd international conference on machine learning, Volume 37 of Proceedings of Machine Learning Research, Lille, France. PMLR, pp 2048-2057
- Zhang Y, Jiang H, Miura Y, Manning CD, Langlotz CP (2022) Contrastive learning of medical visual repre- sentations from paired images and text. In Z Lipton, R Ranganath, M Sendak, M Sjoding, and S Yeung (Eds.), Proceedings of the 7th machine learning for healthcare conference, Volume 182 of Proceedings of Machine Learning Research, pp 2-25.
- PMLR
- Authors
- Li, Y., Kong, C., Zhao, G., & Zhao, Z.
- Affiliations
- Shandong University (CN)
- Shandong Tumor Hospital (CN)
- Corresponding author
- Zhao, Zijian
- ORCID
- Zhao, Guosheng
- Venue
- Artificial Intelligence Review
- Published
- 2025-08-21
- Volume
- 58
- Issue
- 11
- ISSN
- 0269-2821 (ISSN-L)
- 1573-7462
- OpenAlex
- W4413403999
- Cited by
- 7
- References
- 136
- Citations by year
- 2026: 7
- FWCI
- 2.47
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Artificial Intelligence Review (publishedVersion, cc-by-nc-nd)
- Funders
- National Key Research and Development Program of China (2019YFB1311300)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
- Keywords
- chest X-ray reports, encoder-decoder architecture, natural language generation, MIMIC-CXR, deep learning for medical imaging
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
