Radiology report generation with a learned knowledge base and multi-modal alignment
Open accessAbstract
In clinics, a radiology report is crucial for guiding a patient's treatment. However, writing radiology reports is a heavy burden for radiologists. To this end, we present an automatic, multi-modal approach for report generation from a chest x-ray. Our approach, motivated by the observation that the descriptions in radiology reports are highly correlated with specific information of the x-ray images, features two distinct modules: (i) Learned knowledge base: To absorb the knowledge embedded in the radiology reports, we build a knowledge base that can automatically distill and restore medical knowledge from textual embedding without manual labor; (ii) Multi-modal alignment: to promote the semantic alignment among reports, disease labels, and images, we explicitly utilize textual embedding to guide the learning of the visual feature space. We evaluate the performance of the proposed model using metrics from both natural language generation and clinic efficacy on the public IU-Xray and MIMIC-CXR datasets. Our ablation study shows that each module contributes to improving the quality of generated reports. Furthermore, the assistance of both modules, our approach outperforms state-of-the-art methods over almost all the metrics. Code is available at https://github.com/LX-doctorAI1/M2KT.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The uncategorized keyword list in CheXpert may not be transferable from one image modality to different modalities.
“the uncategorized keyword list in CheXpert may not be transferable from one image modality to different modalities”
From Towards a holistic framework for multimodal LLM in 3D brain CT radiology report generation (2025) · cites this paper as [12]
The 14-feature retrieval rates in CheXpert cannot reflect the multi-semantic context of disease nature over an extended clinical course.
“the 14-feature retrieval rates cannot reflect the multi-semantic context (inclusive or exclusive; progression or resolution; relative location and size) of the disease nature over an extended clinical course.”
From Towards a holistic framework for multimodal LLM in 3D brain CT radiology report generation (2025) · cites this paper as [12]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
(2004) · Cited by 9,718 · see citing works
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · see citing works
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318 · see citing works
CIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845 · see citing works
Unsupervised Feature Learning via Non-parametric Instance Discrimination
Wu, Z., Xiong, Y., Yu, S. X., & Lin, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00393 · Cited by 3,664 · see citing works
Improved Baselines with Momentum Contrastive LearningPreprint
Chen, X., Fan, H., Girshick, R., & He, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.04297 · Cited by 2,579 · see citing works
MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports
Johnson, A. E. W., Pollard, T. J., Berkowitz, S. J., Greenbaum, N. R., Lungren, M. P., Deng, C.-Y., Mark, R. G., & Horng, S. (2019) · Scientific Data · doi:10.1038/s41597-019-0322-0 · Cited by 1,816 · see citing works
Knowing When to Look: Adaptive Attention via a Visual Sentinel for Image Captioning
Lu, J., Xiong, C., Parikh, D., & Socher, R. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.345 · Cited by 1,724 · see citing works
Hybrid computing using a neural network with dynamic external memory
Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., Colmenarejo, S. G., Grefenstette, E., Ramalho, T., Agapiou, J., et al. (2016) · Nature · doi:10.1038/nature20101 · Cited by 1,412 · see citing works
Zhou, S. K., Greenspan, H. K., Davatzikos, C. A., Duncan, J. S., van Ginneken, B., Madabhushi, A., Prince, J. L., Rueckert, D., & Summers, R. M. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3054390 · Cited by 1,294 · see citing works
Self-citationPreparing a collection of radiology examinations for distribution and retrieval
Demner-Fushman, D., Kohli, M. D., Rosenman, M. B., Shooshan, S. E., Rodriguez, L., Antani, S., Thoma, G. R., & McDonald, C. J. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv080 · Cited by 1,164 · see citing works
Attention on Attention for Image Captioning
Huang, L., Wang, W., Chen, J., & Wei, X.-Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00473 · Cited by 1,092 · see citing works
(2016) · doi:10.18653/v1/d16-1147 · Cited by 1,035 · see citing works
Generating Radiology Reports via Memory-driven Transformer
Chen, Z., Song, Y., Chang, T.-H., & Wan, X. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.112 · Cited by 693 · see citing works
On the Automatic Generation of Medical Imaging Reports
Jing, B., Xie, P., & Xing, E. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1240 · Cited by 657 · see citing works
Bruno, M. A., Walker, E. A., & Abujudeh, H. H. (2015) · Radiographics · doi:10.1148/rg.2015150023 · Cited by 577 · see citing works
Wang, X., Peng, Y., Lü, L., Lu, Z., & Summers, R. M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00943 · Cited by 574 · see citing works
Learning local feature descriptors with triplets and shallow convolutional neural networks
Balntas, V., Riba, E., Ponsa, D., & Mikolajczyk, K. (2016) · British Machine Vision Conference (BMVC) · doi:10.5244/c.30.119 · Cited by 555 · see citing works
Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generation
Liu, F., Wu, X., Ge, S., Fan, W., & Zou, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01354 · Cited by 396 · see citing works
Self-citationCheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301590 · Cited by 392 · see citing works
Learning to Read Chest X-Rays: Recurrent Neural Cascade Model for Automated Image Annotation
Shin, H.-C., Roberts, K. E., Lü, L., Demner‐Fushman, D., Yao, J., & Summers, R. M. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.274 · Cited by 390 · see citing works
When Radiology Report Generation Meets Knowledge Graph
Zhang, Y., Wang, X., Xu, Z., Yu, Q., Yuille, A., & Xu, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6989 · Cited by 348 · see citing works
Knowledge-Driven Encode, Retrieve, Paraphrase for Medical Image Report Generation
Li, C. Y., Liang, X., Hu, Z., & Xing, E. P. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016666 · Cited by 316 · see citing works
Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation
Li, C. Y., Liang, X., Hu, Z., & Xing, E. P. (2018) · arXiv · doi:10.48550/arxiv.1805.08298 · Cited by 271 · see citing works
Barzilay, R., & Kan, M.-Y. (2017) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p17-1 · Cited by 269 · see citing works
Bottom-Up and Top-Down Attention for Image Captioning and VQA.Preprint
Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S. J., & Zhang, L. (2017) · arXiv (Cornell University) · Cited by 252 · see citing works
Multimodal Recurrent Model with Attention for Automated Radiology Report Generation
Xue, Y., Xu, T., Long, L. R., Xue, Z., Antani, S., Thoma, G. R., & Huang, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00928-1_52 · Cited by 241 · see citing works
Knowledge matters: Chest radiology report generation with general and specific knowledge
Yang, S., Wu, X., Ge, S., Zhou, S. K., & Xiao, L. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102510 · Cited by 225 · see citing works
Self-citationYuan, J., Liao, H., Luo, R., & Luo, J. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32226-7_80 · Cited by 215 · see citing works
Handbook of Medical Image Computing and Computer Assisted Intervention
(2019) · Elsevier eBooks · doi:10.1016/c2017-0-04608-6 · Cited by 198 · see citing works
Contrastive Attention for Automatic Chest X-ray Report Generation
Liu, F., Yin, C., Wu, X., Ge, S., Zhang, P., & Sun, X. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.23 · Cited by 175 · see citing works
Self-citationYou, D., Liu, F., Ge, S., Xie, X., Zhang, J., & Wu, X. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87199-4_7 · Cited by 170 · see citing works
Self-citationCompetence-based Multimodal Curriculum Learning for Medical Report Generation
Liu, F., Ge, S., & Wu, X. (2021) · doi:10.18653/v1/2021.acl-long.234 · Cited by 145 · see citing works
Self-citationShow, Describe and Conclude: On Exploiting the Structure Information of Chest X-ray Reports
Jing, B., Wang, Z., & Xing, E. (2019) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/p19-1657 · Cited by 131 · see citing works
Progressive Transformer-Based Generation of Radiology Reports
Nooralahzadeh, F., Gonzalez, N. A. P., Frauenfelder, T., Fujimoto, K., & Krauthammer, M. O. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.241 · Cited by 121 · see citing works
Joint Modeling of Chest Radiographs and Radiology Reports for Pulmonary Edema Assessment
Chauhan, G., Liao, R., Wells, W. M., Andreas, J., Wang, X., Berkowitz, S. J., Horng, S., Szolovits, P., & Golland, P. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59713-9_51 · Cited by 59 · see citing works
Chest X-Ray Report Generation Through Fine-Grained Label Learning
Syeda-Mahmood, T. F., Wong, K. C. L., Gur, Y., Wu, J. T.-Y., Jadhav, A., Kashyap, S., Karargyris, A., Pillai, A., Sharma, A., Syed, A., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59713-9_54 · Cited by 57 · see citing works
Flagged first, then by citations
1 of the 47 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
METransformer: Radiology Report Generation by Transformer with Multiple Learnable Expert Tokens
Wang, Z., Liu, L., Wang, L., & Zhou, L. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01112 · Cited by 184 · see citing works
“Most recently, many works [16, 21, 22, 40, 41, 47] attempt to integrate knowledge graphs into medical report generation pipeline to improve the quality of the generated reports.”
BackgroundExplainable Deep Learning Methods in Medical Image Classification: A Survey
Patrício, C., Neves, J. C., & Teixeira, L. F. (2023) · ACM Computing Surveys · doi:10.1145/3625287 · Cited by 190 · see citing works
PromptMRG: Diagnosis-Driven Prompts for Medical Report Generation
Jin, H., Che, H., Lin, Y., & Chen, H. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i3.28038 · Cited by 151 · see citing works
Multimodal generative AI for medical image interpretation
Rao, V. M., Hla, M., Moor, M., Adithan, S., Kwak, S., Topol, E. J., & Rajpurkar, P. (2025) · Nature · doi:10.1038/s41586-025-08675-y · Cited by 128 · see citing works
Automated Radiology Report Generation: A Review of Recent Advances
Sloan, P., Clatworthy, P. L., Simpson, E., & Mirmehdi, M. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2024.3408456 · Cited by 98 · see citing works
Towards a holistic framework for multimodal LLM in 3D brain CT radiology report generation
Li, C.-Y., Chang, K.-J., Yang, C.-F., Wu, H.-Y., Chen, W., Bansal, H., Chen, L., Yang, Y.-P., Chen, Y.-C., Chen, S.-P., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-57426-0 · Cited by 75 · see citing works
“6,7 8-11 have been applied to chest X-ray (CXR) report generation in datasets such as the MIMIC-CXR and OpenI to relieve radiologists' time-consuming and tedious burden.”
FFA-GPT: an automated pipeline for fundus fluorescein angiography interpretation and question-answer
Chen, X., Zhang, W., Xu, P., Zhao, Z., Zheng, Y., Shi, D., & He, M. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01101-z · Cited by 72 · see citing works
Multi-modal transformer architecture for medical image analysis and automated report generation
Raminedi, S., Shridevi, S., & Won, D. (2024) · Scientific Reports · doi:10.1038/s41598-024-69981-5 · Cited by 74 · see citing works
A survey of deep-learning-based radiology report generation using multimodal inputs
Wang, X., Figueredo, G. P., Li, R., Zhang, W. E., Chen, W., & Chen, X. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103627 · Cited by 50 · see citing works
Creswell, J., Vo, L. N. Q., Qin, Z. Z., Muyoyeta, M., Tovar, M. A., Wong, E. B., Ahmed, S., Vijayan, S., John, S., Maniar, R., et al. (2023) · BMC Global and Public Health · doi:10.1186/s44263-023-00033-2 · Cited by 33 · see citing works
Yi, X., Fu, Y., Yu, J., Liu, R., Zhang, H., & Hua, R. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3507073 · Cited by 35 · see citing works
MITER: Medical Image–TExt joint adaptive pretRaining with multi-level contrastive learning
Shu, C., Zhu, Y., Tang, X., Xiao, J., Chen, Y., Li, X., Zhang, Q., & Lu, Z. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121526 · Cited by 34 · see citing works
Dia-LLaMA: Towards Large Language Model-Driven CT Report Generation
Chen, Z., Luo, L., Bie, Y., & Chen, H. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04981-0_14 · Cited by 41 · see citing works
A Comprehensive Study of GPT-4V’s Multimodal Capabilities in Medical ImagingPreprint
Li, Y., Liu, Y., Wang, Z., Liang, X., Liu, L., Wang, L., Cui, L., Tu, Z., Wang, L., & Zhou, L. (2023) · medRxiv · doi:10.1101/2023.11.03.23298067 · Cited by 28 · see citing works
Medical report generation based on multimodal federated learning
Chen, J., & Pan, R. (2024) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2024.102342 · Cited by 29 · see citing works
Chest X-Ray Report Generation Using Abnormality Guided Vision Language Model
Edirisinghe, D., Nimalsiri, W., Hennayake, M., Meedeniya, D., & Lim, G. (2025) · IEEE Access · doi:10.1109/access.2025.3606961 · Cited by 28 · see citing works
Prompt-Guided Generation of Structured Chest X-Ray Report Using a Pre-trained LLM
Li, H., Wang, H., Sun, X., He, H., & Feng, J. (2024) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme57554.2024.10687707 · Cited by 30 · see citing works
Wang, X., Wang, F., Li, Y., Ma, Q., Wang, S., Jiang, B., & Tang, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00483 · Cited by 30 · see citing works
Context-enhanced framework for medical image report generation using multimodal contexts
Li, H., Wang, H., Sun, X., He, H., & Feng, J. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112913 · Cited by 27 · see citing works
UniDCP: Unifying Multiple Medical Vision-Language Tasks via Dynamic Cross-Modal Learnable Prompts
Zhan, C., Zhang, Y., Lin, Y., Wang, G., & Wang, H. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3397191 · Cited by 25 · see citing works
CGFTrans: Cross-Modal Global Feature Fusion Transformer for Medical Report Generation
Xu, L., Tang, Q., Zheng, B., Lv, J., Li, W., & Zeng, X. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3414413 · Cited by 22 · see citing works
Attribute Prototype-Guided Iterative Scene Graph for Explainable Radiology Report Generation
Zhang, K., Yang, Y. Q., Yu, J., Jianping Fan, B., Jiang, H., Huang, Q., & Han, W. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3424505 · Cited by 20 · see citing works
DACG: Dual Attention and Context Guidance model for radiology report generation
Lang, W., Liu, Z., & Zhang, Y. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103377 · Cited by 20 · see citing works
Hou, X., Li, X., Liu, Z., Sang, S., Lu, M., & Zhang, Y. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126394 · Cited by 20 · see citing works
Improving radiology report generation with multi-grained abnormality prediction
Jin, Y., Chen, W., Tian, Y., Song, Y., & Yan, C. C. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128122 · Cited by 20 · see citing works
Unsupervised disease tags for automatic radiology report generation
Yi, X., Fu, Y., Hua, R., Liu, R., & Zhang, H. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105742 · Cited by 19 · see citing works
Contrastive Learning with Counterfactual Explanations for Radiology Report Generation
Li, M., Lin, H., Qiu, L., Liang, X., Chen, L., Saddik, A. E., & Chang, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72775-7_10 · Cited by 24 · see citing works
Surgical video workflow analysis via visual-language learning
Li, P., Shu, X., Feng, C.-M., Feng, Y., Zuo, W., & Tang, J. (2025) · npj Health Systems · doi:10.1038/s44401-024-00010-3 · Cited by 18 · see citing works
Chest radiology report generation based on cross-modal multi-scale feature fusion
Pan, Y., Liu, L., Yang, X.-B., Peng, W., & Huang, Q. (2024) · Journal of Radiation Research and Applied Sciences · doi:10.1016/j.jrras.2024.100823 · Cited by 19 · see citing works
Trust it or not: Confidence-guided automatic radiology report generation
Wang, Y., Lin, Z., Xu, Z., Dong, H., Luo, J., Tian, J., Shi, Z., Huang, L., Zhang, Y., Jianping Fan, B., et al. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127374 · Cited by 17 · see citing works
Hu, D., Jiang, Z., Shi, J., Xie, F., Wu, K., Tang, K., Cao, M. W., Huai, J., & Zheng, Y. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103163 · Cited by 17 · see citing works
Dual-modality visual feature flow for medical report generation
Tang, Q., Xu, L., Wang, Y., Zheng, B., Lv, J., Zeng, X., & Li, W. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103413 · Cited by 17 · see citing works
Automatic medical report generation based on deep learning: A state of the art survey
Liu, X., Xin, J., Shen, Q., Huang, Z., & Wang, Z. (2025) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2024.102486 · Cited by 19 · see citing works
Enhancing medical image report generation using a self-boosting multimodal alignment framework
Mir, A. N., Rizvi, D. R., & Nissar, I. (2025) · Health Information Science and Systems · doi:10.1007/s13755-025-00378-y · Cited by 16 · see citing works
PhraseAug: An Augmented Medical Report Generation Model With Phrasebook
Mei, X., Yang, L., Gao, D., Cai, X., Han, J., & Liu, T. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3416190 · Cited by 17 · see citing works
Automatic Medical Report Generation: Methods and Applications
Li, G., Tahir, A. M., Zhang, D., Wang, Z. J., & Ward, R. K. (2024) · APSIPA Transactions on Signal and Information Processing · doi:10.1561/116.20240044 · Cited by 19 · see citing works
A vision attention driven Language framework for medical report generation
Arısoy, M. V., Arısoy, A., & Uysal, İ. (2025) · Scientific Reports · doi:10.1038/s41598-025-95666-8 · Cited by 15 · see citing works
Weakly guided attention model with hierarchical interaction for brain CT report generation
Zhang, X., Yang, S., Shi, Y., Ji, J., Liu, Y., Wang, Z., & Xu, H. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107650 · Cited by 16 · see citing works
Self-citationShao, A., Liu, X., Shen, W., Li, Y., Wu, H., Pan, X., Yang, Z., Xu, Y., Zhu, T., Wang, Y., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01759-z · Cited by 16 · see citing works
“We also compared the InterpreFFA model with 3 baseline medical reportgenerationmodels,includingR2Gen,R2GenRL,andM2KT 22,29,32 .”
LLM-Driven Medical Report Generation via Communication-Efficient Heterogeneous Federated Learning
Che, H., Jin, H., Guo, Z., Lin, Y., Jin, C., & Chen, H. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3591185 · Cited by 16 · see citing works
Visual prior-based cross-modal alignment network for radiology report generation
Zhang, S., Zhou, C., Chen, L., Li, Z.-Y., Gao, Y., & Chen, Y. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107522 · Cited by 15 · see citing works
SumGPT: A Multimodal Framework for Radiology Report Summarization to Improve Clinical Performance
Sultan, T., Rony, M. A. T., Islam, M. S., AlShathri, S. I., & El‐Shafai, W. (2025) · IEEE Access · doi:10.1109/access.2025.3528335 · Cited by 16 · see citing works
Panayides, A. S., Chen, H., Filipović, N., Geroski, T., Hou, J., Lekadir, K., Kostas Marias, A., Matsopoulos, G. K., Papanastasiou, G., Sarder, P., et al. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3649496 · Cited by 13 · see citing works
Wang, D., Cheng, T., Wang, S., Chen, Y., & Yin, Y. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104297 · Cited by 13 · see citing works
Ouis, M. Y., & Akhloufi, M. A. (2024) · Frontiers in Imaging · doi:10.3389/fimag.2024.1373420 · Cited by 12 · see citing works
Parres, D., Albiol, A., & Paredes, R. (2024) · Bioengineering · doi:10.3390/bioengineering11040351 · Cited by 14 · see citing works
Label correlated contrastive learning for medical report generation
Liu, X., Xin, J., Dai, B. T., Shen, Q., Huang, Z., & Wang, Z. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108482 · Cited by 12 · see citing works
A review of medical text analysis: Theory and practice
Chen, Y., Zhang, C., Bai, R., Sun, T., Ding, W., & Wang, R. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103024 · Cited by 12 · see citing works
Xiang, Z., Cui, S., Shang, C., Jiang, J., & Zhang, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72086-4_28 · Cited by 11 · see citing works
Granularity Matters: Pathological Graph-driven Cross-modal Alignment for Brain CT Report Generation
Shi, Y., Ji, J., Zhang, X., Qu, L., & Liu, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.408 · Cited by 11 · see citing works
Mei, X., Yang, L., Gao, D., Cai, X., Han, J., & Liu, T. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3543101 · Cited by 11 · see citing works
ORID: Organ-Regional Information Driven Framework for Radiology Report Generation
Gu, T., Yang, K., An, X., Feng, Z., Lin, D., & Cai, W. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00047 · Cited by 10 · see citing works
Liu, K., Ma, Z., Kang, X., Zhong, Z., Jiao, Z., Baird, G. L., Bai, H. X., & Miao, Q. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72384-1_41 · Cited by 10 · see citing works
Song, S., Tang, H., Yang, H., & Li, X. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3591364 · Cited by 11 · see citing works
Activating Associative Disease-Aware Vision Token Memory for LLM-Based X-Ray Report Generation
Wang, X., Wang, F., Wang, H., Jiang, B., Li, C., Wang, Y., Tian, Y., & Tang, J. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3603416 · Cited by 12 · see citing works
Medical Report Generation via Multimodal Spatio-Temporal Fusion
Mei, X., Mao, R., Cai, X., Yang, L., & Cambria, E. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681377 · Cited by 8 · see citing works
Tsaniya, H., Fatichah, C., Suciati, N., Obi, T., & Lee, J. H. (2025) · IEEE Access · doi:10.1109/access.2025.3588344 · Cited by 8 · see citing works
Izhar, A., Idris, N., & Japar, N. B. (2025) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2025.103220 · Cited by 9 · see citing works
Li, Y., Kong, C., Zhao, G., & Zhao, Z. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11337-0 · Cited by 7 · see citing works
Jing, P., Lee, K., Zhang, Z., Zhou, H., Yuan, Z., Gao, Z., Zhu, L., Papanastasiou, G., Fang, Y., & Yang, G. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103910 · Cited by 11 · see citing works
Shahzadi, I., Madni, T. M., Janjua, U. I., Batool, G., Naz, B., & Ali, M. Q. (2024) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-024-01126-6 · Cited by 7 · see citing works
Dai, L., Lei, J., Ma, F., Sun, Z., Du, H., Zhang, H., Jiang, J., Wei, J., Wang, D., Tan, G., et al. (2024) · Radiology Artificial Intelligence · doi:10.1148/ryai.230520 · Cited by 7 · see citing works
MMSummary: Multimodal Summary Generation for Fetal Ultrasound Video
Guo, X., Men, Q., & Noble, J. A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72083-3_63 · Cited by 7 · see citing works
CLR2G: Cross modal Contrastive Learning on Radiology Report Generation
Xue, H., Ma, Q., Liu, G., Qu, J., Liu, Y., & Liu, A. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679668 · Cited by 8 · see citing works
Fang, J., Xing, S., Li, K., Guo, Z., Li, G., & Yu, C. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.107562 · Cited by 6 · see citing works
He, Z., Wong, A. N. N., & Yoo, J. S. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110625 · Cited by 7 · see citing works
Knowledge fusion in deep learning-based medical vision-language models: A review
Xu, D., Chen, Y., Chai, Z., Xiao, Y., Yan, Y., Ding, W., Wang, H., Jin, Z. J., Jiao, W., Yue, W., et al. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103455 · Cited by 6 · see citing works
Yi, X., Fu, Y., Bi, E., Liang, J., Zhang, H., Yu, J., Li, Q., Hua, R., & Wang, R. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.108102 · Cited by 6 · see citing works
Enhanced Knowledge Injection for Radiology Report Generation
Li, Q., Xu, J., Yuan, R., Chen, M., Zhang, Y., Feng, R., Zhang, X., & Gao, S. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385817 · Cited by 6 · see citing works
Factual serialization enhancement: A key innovation for chest X-ray report generation
Liu, K., Ma, Z., Liu, M., Jiao, Z., Kang, X., Miao, Q., & Kun, X. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.132550 · Cited by 6 · see citing works
Interactive dual-stream contrastive learning for radiology report generation
Zhang, Z., & Jiang, A. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104718 · Cited by 6 · see citing works
Abnormal-region-aware Multi-modal Feature Fusion for medical report generation
Gao, Y., Ni, Z., Liu, W., Ni, L., Xin, L., Hu, L., & Zhang, L. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113538 · Cited by 6 · see citing works
Automated Chest X-Ray Diagnosis Report Generation with Cross-Attention Mechanism
Zhao, J. M., Yao, W. H., Sun, L., Shi, L., Kuang, Z., Wu, C., & Han, Q. (2025) · Applied Sciences · doi:10.3390/app15010343 · Cited by 5 · see citing works
Qingyong, S., Feng, C., Shi, G., Wang, B., & Zhuang, Y. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103240 · Cited by 5 · see citing works
MRGAgents: A Multi-Agent Framework for Improved Medical Report Generation with Med-LVLMs
Wang, P., Ye, S., Naseem, U., & Kim, J. (2025) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta68720.2025.11302496 · Cited by 7 · see citing works
Zhong, Z., & Xie, X. (2024) · BJR|Artificial Intelligence · doi:10.1093/bjrai/ubae012 · Cited by 6 · see citing works
Thought graph traversal for test-time scaling in chest X-ray VLLMs
Yao, Y., Wen, Z., Tong, Y., Tian, X., Li, X., Ma, X., Xu, D., & Gedeon, T. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.113639 · Cited by 4 · see citing works
Haggag, M., Amira, A., Kurugöllü, F., Zaidi, H., & Soudan, B. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.131946 · Cited by 4 · see citing works
Diagnostic Captioning by Cooperative Task Interactions and Sample-Graph Consistency
Wang, Z., Wang, L., Li, X., & Zhou, L. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3562866 · Cited by 4 · see citing works
Natural language generation in healthcare: A review of methods and applications
Lyu, M., Li, X., Chen, Z., Pan, J., Peng, C., Talankar, S., & Wu, Y. (2026) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2026.104997 · Cited by 10 · see citing works
MvKeTR: Chest CT Report Generation With Multi-View Perception and Knowledge Enhancement
Deng, X., He, X., Bao, J., Zhou, Y., Cai, S., Cai, C., & Chen, Z. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3581289 · Cited by 6 · see citing works
Hybrid framework for automated generation of mammography radiology reports
Godoy, E., Mellado, D., de Ferrari, J., Querales, M., Saez, A., Chabert, S., Parra, D., & Rodrigo Salas, F. (2025) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.07.018 · Cited by 3 · see citing works
Akbar, W., Li, S., Abdullah, A., Hussain, A., Attar, R. W., Hussain, T., Alhazmi, A. H., & Alsagri, R. (2025) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001425400051 · Cited by 3 · see citing works
Wang, J., Zhu, L., Bhalerao, A. H., & He, Y. (2025) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2025.102644 · Cited by 5 · see citing works
Automatic medical report generation: a comprehensive review of methodologies and applications
Yan, T., He, H. W., Chan, I. N., Qin, Y. Y., Li, Z., & Wong, P. K. (2026) · PeerJ Computer Science · doi:10.7717/peerj-cs.3474 · Cited by 3 · see citing works
Shen, Y., Chen, Q., He, X., Agrawal, R. V., Grzybowski, A. E., Jin, K., & Ye, X. (2026) · Ophthalmology and Therapy · doi:10.1007/s40123-026-01316-1 · Cited by 3 · see citing works
MG-3D: Multi-grained knowledge-enhanced vision-language pre-training for 3D medical image analysis
Ni, X., Wu, L., Zhuang, J., Wang, Q., Wu, M., Vardhanabhuti, V., Zhang, L., Gao, H., & Chen, H. (2026) · Medical Image Analysis · doi:10.1016/j.media.2026.104027 · Cited by 5 · see citing works
Zhang, H., Liu, J., Liu, W., Chen, H., Yu, Z., Yuan, Y., Wang, P., & Qin, J. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3376462 · Cited by 3 · see citing works
Deep Disease Label-guided Graph Convolutional Network for Medical Report Generation
Xu, L., Wang, Y., He, C., Tang, Q., Zeng, X., & Lv, J. (2025) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3722226 · Cited by 3 · see citing works
MedKit: Multi-level feature distillation with knowledge injection for radiology report generation
Su, Z., Song, H., Lin, Y., Wu, Y., Weng, X., Mao, Z., Liu, B., Yin, H., & Yang, J. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129003 · Cited by 4 · see citing works
A Multi-Modal Feature Fusion-Based Approach for Chest X-Ray Report Generation
Cheddi, F., Habbani, A., & Nait‐Charif, H. (2024) · International Conference on Wireless Networks and Mobile Communications (WINCOM) · doi:10.1109/wincom62286.2024.10657835 · Cited by 3 · see citing works
Optimizing medical image report generation through a discrete diffusion framework
Sun, S., Su, Z., Meizhou, J., Feng, Y., Hu, Q., Luo, J., Hu, K., & Yang, Z. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-07111-2 · Cited by 4 · see citing works
Das, J., & Garai, P. (2025) · International Conference on Intelligent Systems, Advanced Computing and Communication (ISACC) · doi:10.1109/isacc65211.2025.10969237 · Cited by 3 · see citing works
UniCrossAdapter: Multimodal Adaptation of CLIP for Radiology Report Generation
Chen, Y., Du, C., Li, C., Hu, J., Shi, Y., Xiong, S., Zhu, X. X., & Mou, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73471-7_12 · Cited by 3 · see citing works
Song, T., Lin, Q., Zeng, X., Cao, Y., Man, Z., Liu, C., Cai, Z., Gui, C., & Huang, X. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.108359 · Cited by 2 · see citing works
Liu, H., Wei, D., Peng, Q., Huang, Y., Wu, X., Zheng, Y., & Wang, L. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-96625-5_15 · Cited by 3 · see citing works
Self-citationPseudo-labeling and knowledge-guided contrastive learning for radiology report generation
Ye, F., Hu, X. Z., Ding, Y., & Liu, F. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104941 · Cited by 2 · see citing works
MRI-based classification of lumbar spinal stenosis using a hybrid quantum and deep learning model
AYATA, F., Seyyarer, E., & Genç, H. (2025) · The European Physical Journal Plus · doi:10.1140/epjp/s13360-025-07148-5 · Cited by 2 · see citing works
Feng, J., Cao, X., Fang, J., Cai, L., Wang, Q., & Zhang, J. (2026) · Academic Radiology · doi:10.1016/j.acra.2026.03.017 · Cited by 2 · see citing works
PeFoMed: Parameter efficient fine-tuning of multimodal large language models for medical CXR
Liu, G., Tang, X., He, J., Li, P., Chen, Z., & Zhong, S. (2026) · Scientific Reports · doi:10.1038/s41598-026-47871-2 · Cited by 45 · see citing works
- Authors
- Yang, S., Wu, X., Ge, S., Zheng, Z., Zhou, S. K., & Xiao, L.
- Affiliations
- Chinese Academy of Sciences (CN)
- Tencent (China) (CN)
- Beijing Tsinghua Chang Gung Hospital (CN)
- University of Science and Technology of China (CN)
- Beijing University of Posts and Telecommunications (CN)
- Corresponding author
- Xiao, Li
- Venue
- Medical Image Analysis
- Publisher
- Elsevier BV
- Published
- 2023-05
- Volume
- 86
- Pages
- 102798
- ISSN
- 1361-8415 (ISSN-L)
- OpenAlex
- W4360604432
- Cited by
- 199
- References
- 47
- Citations by year
- 2026: 58 · 2025: 89 · 2024: 41 · 2023: 11
- FWCI
- 18.03
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Medical Image Analysis (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Tencent
- National Natural Science Foundation of China (31900979)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- radiology report generation, chest radiography, knowledge distillation, multimodal alignment, text embeddings, semantic alignment, natural language generation
- MeSH terms
- Humans, Learning, Radiography, Radiology, Benchmarking, Knowledge Bases
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
