The DOI matches its Crossref record; 27 of the 28 references it cites were found in a registry or an index.
On the Automatic Generation of Medical Imaging Reports
Open accessAbstract
Medical imaging is widely used in clinical practice for diagnosis and treatment.Report-writing can be error-prone for unexperienced physicians, and timeconsuming and tedious for experienced physicians.To address these issues, we study the automatic generation of medical imaging reports.This task presents several challenges.First, a complete report contains multiple heterogeneous forms of information, including findings and tags.Second, abnormal regions in medical images are difficult to identify.Third, the reports are typically long, containing multiple sentences.To cope with these challenges, we (1) build a multi-task learning framework which jointly performs the prediction of tags and the generation of paragraphs, (2) propose a co-attention mechanism to localize regions containing abnormalities and generate narrations for them, (3) develop a hierarchical LSTM model to generate long paragraphs.We demonstrate the effectiveness of the proposed methods on two publicly available datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recently, deep learning methods have also been used for automated generation of radiology reports , typically incorporating long-short-term-memory (LSTM) network models to generate the textual paragraphs [314, 315, 316, 317], and also to identify findings in radiology reports [318, 319, 320].”
From An overview of deep learning in medical imaging focusing on MRI (2019) · cites this paper as [314]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 5×
“Results in Table 3 show that Ours-CoAttention and VGG-19 [17] network performs very similar for tag prediction.”
BackgroundMethodInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“than adopting a single-layer LSTM (Hochreiter and Schmidhuber, 1997), which is less capable of modeling long word sequences, we leverage the compositional nature of the report and adopt a hierarchical LSTM to produce long texts.”
MethodAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“We used the Adam (Kingma and Ba, 2014) optimizer for parameter learning.”
MethodBLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318 · cited 7×
“Note that the only difference between Ours-NoAttention and CNN-RNN [18] in Table 1 is that Ours-NoAttention adopts a hierarchical LSTM decoder while CNNRNN [18] only adopts a single-layer LSTM.”
BackgroundMethodInfluentialLong-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
CIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845 · cited 2×
“We report the paragraph generation (upper part of Table 1) and one sentence generation (lower part of Table 1) results using the following captioning evaluation tools: BLEU [13], METEOR [4], ROUGE [12] and CIDER [17].”
MethodMeteor Universal: Language Specific Translation Evaluation for Any Target Language
Denkowski, M., & Lavie, A. (2014) · WMT@ACL · doi:10.3115/v1/w14-3348 · Cited by 2,108 · cited 2×
“We report the paragraph generation (upper part of Table 1) and one sentence generation (lower part of Table 1) results using the following captioning evaluation tools: BLEU [13], METEOR [4], ROUGE [12] and CIDER [17].”
MethodImage Captioning with Semantic Attention
You, Q., Jin, H., Wang, Z., Chen, F., & Luo, J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.503 · Cited by 1,869 · cited 6×
“We first compare our method with several state-of-the-art image captioning models: CNN-RNN [19], LRCN [5], Soft ATT [20], ATT-RK [21].”
BackgroundMethodInfluential
Research works printed but not linked: 2
- Jimmy Ba, Volodymyr Mnih, and Koray Kavukcuoglu. 2015. Multiple object recognition with visual atten- tion. ICLR.
- Chin-Yew Lin. 2004. Rouge: A package for auto- matic evaluation of summaries. In Text summariza- tion branches out: Proceedings of the ACL-04 work- shop, volume 8. Barcelona, Spain.
Cited by
Sorted by citations
Items 101–200 of 429
Tripathi, S., Augustin, A. I., Sukumaran, R., Dheer, S., & Kim, E. (2022) · Artificial Intelligence in the Life Sciences · doi:10.1016/j.ailsci.2022.100043 · Cited by 32
Deep Context-Encoding Network For Retinal Image Captioning
Huang, J.-H., Wu, T.-W., Yang, C.-H. H., & Worring, M. (2021) · Proceedings - International Conference on Image Processing · doi:10.1109/icip42928.2021.9506803 · Cited by 34
Variational Topic Inference for Chest X-Ray Report Generation
Najdenkoska, I., Zhen, X., Worring, M., & Shao, L. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87199-4_59 · Cited by 33
“Successful chest X-ray report generation methods mainly follow the neural encoder-decoder architecture [11,17,19,33,32,31,10,3,20], where a convolutional neural network (CNN) encodes the image into a fixed-size representation and then, sentence by sentence, a recurrent neural network decodes the representation into a…”
BackgroundCross Encoder-Decoder Transformer with Global-Local Visual Extractor for Medical Image Captioning
Lee, H., Cho, H., Park, J., Chae, J., & Kim, J. (2022) · Sensors · doi:10.3390/s22041429 · Cited by 29
“[11] introduced a co‐attention (CoATT) mechanism for localizing sub‐regions in the Figure 1.”
MethodAn Inclusive Task-Aware Framework for Radiology Report Generation
Wang, L., Ning, M., Lu, D., Wei, D., Zheng, Y., & Chen, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16452-1_54 · Cited by 31
Multimodal contrastive learning for radiology report generation
Wu, X., Li, J., Wang, J., & Qian, Q. (2022) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-022-04398-4 · Cited by 26
“Down (Anderson et al. 2018), HRNN (Krause et al. 2017), CoAtt (Jing et al. 2018), MRA (Xue et al. 2018), HGRG-Agent (Li et al. 2018), SentSAT+KG (Zhang et al. 2020b), Transformer (Chen et al 2020), CDGPT2 (Alfarghaly et al. 2021), and CMCL (Liu et al. 2021).”
Ultrasound Report Generation With Cross-Modality Feature Alignment via Unsupervised Guidance
Li, J., Su, T., Zhao, B., Lv, F., Wang, Q., Navab, N., Hu, Y., & Jiang, Z. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3424978 · Cited by 38
“For instance, Jing et al. [23] employed a CNN to classify features extracted from radiology images, promoting the model to discriminate disease types.”
MethodMedSkip: Medical Report Generation Using Skip Connections and Integrated Attention
Pahwa, E., Mehta, D., Kapadia, S., Jain, D., & Luthra, A. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00380 · Cited by 27
“For PEIR Gross, as compared to [12] CoAtt module formed by combining both visual and semantic features, we extract visual features alone from each down-sampled branch which helps in accomodating significant areas in the images of the diverse range of affected body parts.”
BackgroundMethodInfluentialZhang, B., Vakanski, A., & Xian, M. (2021) · Machine learning for signal processing ... · doi:10.1109/mlsp52302.2021.9596314 · Cited by 28
Integrating domain knowledge for biomedical text analysis into deep learning: A survey
Cai, L., Li, J., Lv, H., Liu, W., Niu, H., & Wang, Z. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104418 · Cited by 25
Deep Learning for Chest Radiology: A Review
Feng, Y., Teh, H. S., & Cai, Y. (2019) · Current Radiology Reports · doi:10.1007/s40134-019-0333-9 · Cited by 25
Multimodal Approaches for Alzheimer’s Detection Using Patients’ Speech and Transcript
Cai, H., Huang, X., Liu, Z., Liao, W., Dai, H., Wu, Z., Zhu, D., Ren, H., Li, Q., Liu, T., et al. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-43075-6_34 · Cited by 22
Improving Medical X-ray Report Generation by Using Knowledge Graph
Zhang, D., Ren, A., Liang, J., Liu, Q., Wang, H., & Ma, Y. (2022) · Applied Sciences · doi:10.3390/app122111111 · Cited by 23
Expert-defined Keywords Improve Interpretability of Retinal Image Captioning
Wu, T.-W., Huang, J.-H., Lin, J. L., & Worring, M. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00190 · Cited by 26
“• CoAtt[26] adopts co-attention mechanism to produce joint context vectors for generating medical descriptions based on topics.”
BackgroundStyle-Aware Radiology Report Generation with RadGraph and Few-Shot Prompting
Yan, B., Liu, R., Kuo, D., Adithan, S., Reis, E., Kwak, S., Venugopal, V., O’Connell, C., Saenz, A., Rajpurkar, P., et al. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.977 · Cited by 22
Automatic Disease Detection and Report Generation for Gastrointestinal Tract Examination
Harzig, P., Einfalt, M., & Lienhart, R. W. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3356066 · Cited by 21
“Then techniques like hierarchical LSTM networks [3, 6, 8] could be used to automatically generate text paragraphs in natural language.”
MethodCross-Modal Augmented Transformer for Automated Medical Report Generation
Tang, Y., Yuan, Y., Tao, F., & Tang, M. (2025) · IEEE Journal of Translational Engineering in Health and Medicine · doi:10.1109/jtehm.2025.3536441 · Cited by 25
“Numerous existing methodologies, such as those proposed in [13], [14], [15], and [16], closely align with standard image captioning approaches and effectively leverage the prevalent encoder-decoder framework.”
Show, tell and summarise: learning to generate and summarise radiology findings from medical images
Singh, S., Karimi, S., Ho‐Shon, K., & Hamey, L. (2021) · Neural Computing and Applications · doi:10.1007/s00521-021-05943-6 · Cited by 23
“Table 1 shows the split of our dataset following the same distribution of previous studies [5, 72].”
BackgroundResultTranSQ: Transformer-Based Semantic Query for Medical Report Generation
Kong, M. C., Huang, Z., Kuang, K., Zhu, Q., & Wu, F. Q. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16452-1_58 · Cited by 26
Wang, H., Jin, Y., & Zhu, L. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160647 · Cited by 21
Improving radiology report generation with multi-grained abnormality prediction
Jin, Y., Chen, W., Tian, Y., Song, Y., & Yan, C. C. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128122 · Cited by 23
Automatic Generation of Chest X-ray Reports Using a Transformer-based Deep Learning Model
Amjoud, A. B., & Amrouch, M. (2021) · International Conference on Intelligent Computing in Data Sciences (ICDS) · doi:10.1109/icds53782.2021.9626725 · Cited by 25
Weakly Guided Hierarchical Encoder-Decoder Network for Brain CT Report Generation
Yang, S., Ji, J., Zhang, X., Liu, Y., & Wang, Z. (2021) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm52615.2021.9669626 · Cited by 20
Kaur, N., & Mittal, A. (2022) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-022-04454-z · Cited by 19
“Jing et al. (2017) concluded that the co-attention mechanism that leverages both the visual and semantic features performs better than applying semantic and visual attention individually.”
BackgroundMethodInfluentialIn-Context Learning for Zero-shot Medical Report Generation
Liu, R., Li, M., Zhao, S., Chen, L., Chang, X., & Yao, L. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680760 · Cited by 22
“Initial efforts saw the deployment of LSTM [10] networks, including two-level LSTM configurations [16, 19], as decoders for generating reports.”
BackgroundUrsuleanu, T. F., Luca, A. R., Gheorghe, L., Grigorovici, R., Iancu, S., Hlusneac, M., Preda, C., & Grigorovici, A. (2021) · Diagnostics · doi:10.3390/diagnostics11081373 · Cited by 20
Non-local Attention Improves Description Generation for Retinal Images
Huang, J.-H., Wu, T.-W., Yang, C.-H. H., Shi, Z., Lin, I.-H., Tegner, J. N., & Worring, M. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00331 · Cited by 19
“In [19], the authors propose a multi-task learning framework to predict tags and generate captions at the same time.”
BackgroundMethodInfluentialMATNet: Exploiting Multi-Modal Features for Radiology Report Generation
Shang, C., Cui, S., Li, T., Wang, X., Li, Y., & Jiang, J. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3229844 · Cited by 18
“We follow the previous work [9], [11], [19] by concatenating the impressions and findings sections as the target report.”
MethodClinically Correct Report Generation from Chest X-Rays Using Templates
Pino, P., Parra, D., Besa, C., & Lagos, C. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-87589-3_67 · Cited by 21
“Most works use common Convolutional Neural Networks (CNNs) as encoder (e.g. Densenet [6]), and as decoder: a single LSTM to generate word by word [2]; two LSTMs arranged hierarchically to generate sentences and words [9,10,16,31]; or a Transformer-based network [3,17,29].”
BackgroundMethodResultInfluentialAutomatic Medical Image Report Generation with Multi-view and Multi-modal Attention Mechanism
Yang, S., Niu, J., Wu, J., & Liu, X. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-60248-2_48 · Cited by 17
Contrastive Learning with Counterfactual Explanations for Radiology Report Generation
Li, M., Lin, H., Qiu, L., Liang, X., Chen, L., Saddik, A. E., & Chang, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72775-7_10 · Cited by 31
HERGen: Elevating Radiology Report Generation with Longitudinal Data
Wang, F., Du, S., & Yu, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73001-6_11 · Cited by 33
A vision attention driven Language framework for medical report generation
Arısoy, M. V., Arısoy, A., & Uysal, İ. (2025) · Scientific Reports · doi:10.1038/s41598-025-95666-8 · Cited by 15
“The same way of separating the data as in 57 was adopted here and 10% reports were randomly selected for testing.”
Enhancing medical image report generation using a self-boosting multimodal alignment framework
Mir, A. N., Rizvi, D. R., & Nissar, I. (2025) · Health Information Science and Systems · doi:10.1007/s13755-025-00378-y · Cited by 16
“MIC methods employing encoder-decoder architectures [8, 9], attention mechanisms, and external knowledge integration have demonstrated promising results in radiology, histopathology, and ophthalmology.”
Chen, J., Dai, X., Guo, Y., Zhu, J., Mei, X., Deng, M., & Sun, G. (2023) · Remote Sensing · doi:10.3390/rs15051436 · Cited by 17
Domain-specific image captioning: a comprehensive review
Sharma, H., & Padha, D. (2024) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-024-00328-6 · Cited by 19
Siam, K., Faruk, M. J. H., He, B., Cheng, J. Q., & Gu, H. (2025) · Information · doi:10.3390/info16110971 · Cited by 18
“The high-dimensional features extracted from X-rays contribute to improved predictions when combined with clinical notes and lab data [22,23].”
Automated Radiology Report Generation Using Transformers
Nimalsiri, W., Hennayake, M., Rathnayake, K. D. I. S., Ambegoda, T. D., & Meedeniya, D. (2023) · International Conference on Advanced Research in Computing (ICARC) · doi:10.1109/icarc57651.2023.10145699 · Cited by 19
“[4] have utilized a hierarchical LSTM decoder to generate medical reports.”
MethodStudy on Medical Image Report Generation Based on Improved Encoding-Decoding Method
Li, S., Wang, W., Li, J., et al. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-26763-6_66 · Cited by 15
Automatic Report Generation for Chest X-Ray Images: A Multilevel Multi-attention Approach
Gajbhiye, G. O., Nandedkar, A. V., & Faye, I. (2020) · Communications in computer and information science · doi:10.1007/978-981-15-4015-8_15 · Cited by 17
Automatic Radiology Report Generation by Learning with Increasingly Hard Negatives
Voutharoja, B. P., Wang, L., & Zhou, L. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230545 · Cited by 16
Weakly guided attention model with hierarchical interaction for brain CT report generation
Zhang, X., Yang, S., Shi, Y., Ji, J., Liu, Y., Wang, Z., & Xu, H. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107650 · Cited by 16
Automatic Caption Generation of Retinal Diseases with Self-trained RNN Merge Model
Mishra, S., & Banerjee, M. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-981-15-2930-6_1 · Cited by 14
KdTNet: Medical Image Report Generation via Knowledge-Driven Transformer
Cao, Y., Cui, L., Yu, F., Zhang, L., Li, Z., Liu, N., & Xu, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-00129-1_8 · Cited by 18
CoMT: Chain-of-Medical-Thought Reduces Hallucination in Medical Report Generation
Jiang, Y., Chen, J., Yang, D., Li, M., Wang, S., Wu, T., Li, K., & Zhang, L. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887699 · Cited by 21
“Some approaches [2], [3], [10] utilize disease classification to learn discriminative features better.”
GREN: Graph-Regularized Embedding Network for Weakly-Supervised Disease Localization in X-Ray Images
Qi, B., Zhao, G., Wei, X., Du, C., Pan, C., Yu, Y., & Li, J. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3193108 · Cited by 16
Self Adaptive Global-Local Feature Enhancement for Radiology Report Generation
Wang, Y., Wang, K., Liu, X., Gao, T., Zhang, J., & Wang, G. (2023) · Proceedings - International Conference on Image Processing · doi:10.1109/icip49359.2023.10222405 · Cited by 19
“Most frameworks adopt the encoder-decoder architecture [1, 2, 3], e.g., a VIT [4] image encoder and then a report decoder based on transformer decoder [5].”
BackgroundMethodLLM-Driven Medical Report Generation via Communication-Efficient Heterogeneous Federated Learning
Che, H., Jin, H., Guo, Z., Lin, Y., Jin, C., & Chen, H. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3591185 · Cited by 19
“However, the advancement of LLM-driven MRG faces substantial challenges, primarily due to the fragmented and privacy-restricted nature of medical report data across healthcare institutions [12]–[14].”
Parres, D., Albiol, A., & Paredes, R. (2024) · Bioengineering · doi:10.3390/bioengineering11040351 · Cited by 18
“The interest in RRG has grown, led by impactful models like TieNet [4], as well as innovative contributions from Li et al. [3], Jing et al. [19], and Jing et al. [19], each leveraging advanced techniques such as pretrained networks, coattention mechanisms, and retrieval policy modules for efficient…”
MethodGIT-CXR: End-to-End Transformer for Chest X-Ray Report Generation
Sîrbu, I., Sîrbu, I.-R., Bogojeska, J., & Rebedea, T. E. (2025) · Information · doi:10.3390/info16070524 · Cited by 13
TMRGM: A Template-Based Multi-Attention Model for X-Ray Imaging Report Generation
Wang, X., Zhang, Y., Guo, Z. H., & Li, J. (2021) · Journal of Artificial Intelligence for Medical Sciences · doi:10.2991/jaims.d.210428.002 · Cited by 12
“For chest X-ray report generation, we compare our proposed method with state-of-the-art method: TieNet [28], CoAtt [29], and Adapt-Att [34].”
BackgroundMethodSurgical Instruction Generation with Transformers
Zhang, J., Nie, Y., Chang, J. M., & Zhang, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87202-1_28 · Cited by 17
“The most related research topic to us is medical report generation [12,7,6], which describes the impression , findings, tags, etc.”
BackgroundAIMNet: Adaptive Image-Tag Merging Network For Automatic Medical Report Generation
Shi, J., Wang, S., Wang, R., & Ma, S. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747702 · Cited by 16
SGT: Scene Graph-Guided Transformer for Surgical Report Generation
Lin, C., Zheng, S., Liu, Z., Li, Y., Zhu, Z., & Zhao, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16449-1_48 · Cited by 15
“Existing studies on medical image caption more focus on radiological images, such as chest X-rays [9, 23].”
BackgroundAutomated Enriched Medical Concept Generation for Chest X-ray Images
Gasimova, A. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33850-3_10 · Cited by 14
“Recently, we have seen supervised learning approaches that aim to take advantage of past radiological exams containing reports in order to either auto-generate the reports [17,9,23], or to assist in classification tasks [16,21,20,24,22].”
BackgroundYang, Z., & Liu, Q. (2020) · IEEE Access · doi:10.1109/access.2020.2980578 · Cited by 11
“Secondly, image captioning provides a medical assistance diagnosis for inexperienced doctors [7].”
BackgroundImproving Radiology Report Generation with D2-Net: When Diffusion Meets Discriminator
Jin, Y., Chen, W., Tian, Y., Song, Y., Yan, C. C., & Mao, Z. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448326 · Cited by 12
“Recent years, we have seen a surge in research on automated radiology report generation models [2; 3; 4; 5; 6].”
BackgroundFDT − Dr2T: a unified Dense Radiology Report Generation Transformer framework for X-ray images
Sharma, D., Dhiman, C., & Kumar, D. (2024) · Machine Vision and Applications · doi:10.1007/s00138-024-01544-0 · Cited by 10
“These methods though generated fluent reports still suffer from data deviation which leads to the generation of general reports without prominent abnormalities [2, 17–19].”
InfluentialIzhar, A., Idris, N. B., & Japar, N. B. (2025) · Cognitive Computation · doi:10.1007/s12559-025-10404-6 · Cited by 11
“• CoAtt [59] addresses the challenges of automatic medical imaging report generation by introducing a multi-task learning framework that simultaneously predicts tags and generates paragraphs.”
Mei, X., Yang, L., Gao, D., Cai, X., Han, J., & Liu, T. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3543101 · Cited by 13
“Moreover, the implementation of co-attention mechanisms and hierarchical LSTM models Co-Attention [32] aims to localize abnormalities more accurately and produce detailed, comprehensive reports.”
Gu, Y., Li, R., Wang, X., & Zhou, Z. (2023) · Bioengineering · doi:10.3390/bioengineering10080966 · Cited by 10
“Jing et al. [10] embedded the MTI tags predicted by the encoder in order to gener-Bioengineering ate diagnostic reports.”
BackgroundMethodInfluentialShetty, S., Ananthanarayana, V. S., & Mahale, A. R. (2022) · Mathematics · doi:10.3390/math10193646 · Cited by 10
Multi-Level Objective Alignment Transformer for Fine-Grained Oral Panoramic X-Ray Report Generation
Gao, N., Yao, R., Liang, R., Chen, P., Liu, T., & Dang, Y. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3368922 · Cited by 9
“Most existing methods [9], [10], [11] follow standard image captioning meth-ods and rely on holistic methods to generate medical reports.”
MethodTan, J., Kim, S., Kim, E., Lee, S. H., Ahn, S., & Jeong, W. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72083-3_3 · Cited by 13
Act Like a Radiologist: Radiology Report Generation Across Anatomical Regions
Chen, Q., Xie, Y., Wu, B., Chen, X., Ang, J. S. K., To, M., Chang, X., & Wu, Q. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0960-4_3 · Cited by 15
“In this part, we provide more discussions, including the effect of different feature extractors in Section A.1, the impact of hyper-parameter λ in Section A.2, and whether feeding image tags into the model would cause information leakage in Section A.3.”
Aligning, Autoencoding and Prompting Large Language Models for Novel Disease Reporting
Liu, F., Wu, X., Huang, J., Yang, B., Branson, K., Schwab, P., Clifton, L., Zhang, P., Luo, J., Zheng, Y., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3534586 · Cited by 13
“Given the annotated radiology report for the input radiology image, existing works train the model by minimizing a supervised training loss, e.g., cross-entropy loss [1], [8].”
Autocompletion of Chief Complaints in the Electronic Health Records using Large Language Models
Islam, K. M. S., Nipu, A. S., Madiraju, P., & Deshpande, P. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386778 · Cited by 11
“Recent improvement in NLG is revolutionizing EHR generation in different fields such as report generation from medical images [21], medical note generation from table data [22], medical topic to text generation [23], and so on.”
BackgroundTransformers Are All You Need to Generate Automatic Report from Chest X-ray Images
Mondal, C., Pham, D.-S., Tan, T., Gedeon, T., & Gupta, A. (2023) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta60407.2023.00060 · Cited by 9
“However, our model demonstrates the best performance for the Cider metric, surpassing the method of [10] by a significant margin of 20 .”
BackgroundMethodInfluentialVision to Language: Methods, Metrics and Datasets
Sharif, N., Nadeem, U., Shah, S. A. A., Bennamoun, M., & Liu, W. (2020) · Learning and Analytics in Intelligent Systems · doi:10.1007/978-3-030-49724-8_2 · Cited by 9
ORID: Organ-Regional Information Driven Framework for Radiology Report Generation
Gu, T., Yang, K., An, X., Feng, Z., Lin, D., & Cai, W. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00047 · Cited by 11
“As a result, diverse RRG methodologies have been introduced [6, 7, 16, 20, 21, 25, 34, 43, 47, 62], each tackling the issue from distinct angles and showcasing notable enhancements in performance.”
Guo, K., Zheng, S., Huang, R., & Gao, R. (2023) · IEEE Access · doi:10.1109/access.2023.3322425 · Cited by 10
Medical Report Generation Based on Segment-Enhanced Contrastive Representation Learning
Zhao, R., Wang, X., Dai, H., Gao, P., & Li, P. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44696-2_65 · Cited by 9
“Currently, the mainstream approach to medical report generation is to use a deep-learning-based encoder-decoder architecture [25,9,26].”
BackgroundMethodInfluentialMulti-Modality Regional Alignment Network for Covid X-Ray Survival Prediction and Report Generation
Zhong, Z., Li, J., Sollee, J., Collins, S., Bai, H. X., Zhang, P. J., Healey, T. T., Atalay, M. K., Gao, X., & Jiao, Z. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3417849 · Cited by 10
“This global representation is then incorporated into the set of region features, f 30 ROI = f G , therefore j belongs to the range [1, ..., 29, 30].”
Generating Medical Report via Joint Probability Graph Reasoning
Zhang, J., Cheng, M., Li, X., Shen, X., Liu, Y., & Wan, Y. (2025) · Tsinghua Science & Technology · doi:10.26599/tst.2024.9010058 · Cited by 9
A Study on Automatic Generation of Chinese Discharge Summary
Xiong, Y., Tang, B., Chen, Q., Wang, X., & Yan, J. (2019) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm47256.2019.8983293 · Cited by 9
“[21] proposed a deep leaming method to generate medical imaging reports.”
MethodMedical Image Tagging by Deep Learning and Retrieval
Kougia, V., Pavlopoulos, J., & Androutsopoulos, I. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58219-7_14 · Cited by 8
Wang, H., Niu, J., Liu, X., & Wang, Y. (2022) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm55620.2022.9995003 · Cited by 8
“To alleviate the data bias problem, the CoAtt [5] proposes a co-attention to incorporate disease tags into the generation process, then utilizes a hierarchical LSTM to generate disease-related reports.”
MethodInfluentialLi, Y., Kong, C., Zhao, G., & Zhao, Z. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11337-0 · Cited by 7
“…predicted labels or semantic concepts and fuse this encoded information with visual features to guide and refine the report generation process (Jing et al. 2018; Yin et al. 2019; Yuan et al. 2019; Syeda-Mahmood et al. 2020; Alfarghaly et al. 2021; Zhou et al. 2021; Jia et al. 2022; Wang et…”
InfluentialA vision-language model with multi-granular knowledge fusion in medical imaging
Chen, K., Li, Y., Zhu, X., Zhang, W., & Hu, B. (2024) · World Wide Web · doi:10.1007/s11280-024-01315-x · Cited by 8
“Jing et al. [13] and Yuan et al. [14] enhanced synchronization between image and text through co-attention and sentence-level attention mechanisms, leading to more accurate radiology reports.”
Multilabel Classification of Radiology Image Concepts Using Deep Learning
Santamato, V., & Marengo, A. (2025) · Applied Sciences · doi:10.3390/app15095140 · Cited by 7
Vision‐Language Models for Automated Chest X‐ray Interpretation: Leveraging ViT and GPT‐2
Islam, M. R., Hossain, M. Z., Ahmed, M., & Samu, M. S. S. (2025) · Engineering Reports · doi:10.1002/eng2.70220 · Cited by 10
Izhar, A., Idris, N., & Japar, N. B. (2025) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2025.103220 · Cited by 11
Clinically Coherent Radiology Report Generation with Imbalanced Chest X-rays
Yu, H., & Zhang, Q. (2022) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm55620.2022.9994871 · Cited by 10
“…or image captioning: (1) CNN-RNN, where we use a CNN for visual feature extraction and a LSTM with attention for report generation; (2) CoATT [21]; (3) Memory-driven Generation (R2Gen) [9]; (4) Cross-modal Generation (R2GenCMN) [22]; (5) Clinically Accurate Generation (CAGen) [23]; (6) Image…”
MethodIIHT: Medical Report Generation with Image-to-Indicator Hierarchical Transformer
Fan, K., Cai, X., & Niranjan, M. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-8076-5_5 · Cited by 11
Diffusion Networks with Task-Specific Noise Control for Radiology Report Generation
Tian, Y., Xia, F., & Song, Y. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681476 · Cited by 10
“We follow the convention of previous studies [3, 4, 13, 16, 39] by only preserving the “Findings” sections for both datasets.”
BackgroundMethodInfluentialAI-assisted Method for Efficiently Generating Breast Ultrasound Screening Reports
Zhang, H., Yuan, K., Ge, S., et al. (2022) · Current Medical Imaging Formerly Current Medical Imaging Reviews · doi:10.2174/1573405618666220329092537 · Cited by 7
“As a result, doctors need 5 to 10 minutes to complete the ultrasound report, which takes up a lot of time([8]).”
BackgroundBabar, Z., van Laarhoven, T., & Marchiori, E. (2021) · PLoS ONE · doi:10.1371/journal.pone.0259639 · Cited by 7
Keyword-augmented and semi-automatic generation of FESS reports: a proof-of-concept study
Kunz, V., Wildfeuer, V., Bieck, R., Sorge, M., Zebralla, V., Dietz, A., Neumuth, T., & Pirlich, M. (2022) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-022-02791-0 · Cited by 7
“Through taking over repetitive and time-consuming tasks with these applications, such as writing reports [11], physicians can potentially shift valuable time to patient treatment [5, 12–14].”
BackgroundBridging the Gap: Cross-modal Knowledge Driven Network for Radiology Report Generation
Kang, B., Xiong, Y., Jiao, J., Zhang, Y., Jia, X., & Li, J. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385967 · Cited by 8
Interactive dual-stream contrastive learning for radiology report generation
Zhang, Z., & Jiang, A. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104718 · Cited by 7
Naseem, U., Thapa, S., & Masood, A. (2024) · JMIR Medical Informatics · doi:10.2196/56627 · Cited by 6
“Furthermore, the task of the automatic generation of medical image reports, introduced by Jing et al [9], aimed to streamline the reporting process for physicians, enhancing both efficiency and accuracy.”
Shahzadi, I., Madni, T. M., Janjua, U. I., Batool, G., Naz, B., & Ali, M. Q. (2024) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-024-01126-6 · Cited by 7
“These reports may contain occasional mistakes, repetitions, or omissions of mentioning less common yet significant abnormalities [24, 25].”
Co-Occurrence Relationship Driven Hierarchical Attention Network for Brain CT Report Generation
Zhang, X., Dou, S., Ji, J., Liu, Y., & Wang, Z. (2024) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3413002 · Cited by 7
“Jing et al. [9] propose a joint attention mechanism for Chest X-ray report generation, which exploits the co-attention to learn visual and semantic features.”
BackgroundInfluentialEnd-to-End Clustering Enhanced Contrastive Learning for Radiology Reports Generation
Liu, X., Xin, J., Shen, Q., Li, C., Huang, Z., & Wang, Z. (2024) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3449876 · Cited by 7
“Most of the existing methods use cross-entropy loss to train models for CNN-HRNN architecture [4], [5], [6], [7], [8].”
A Metaphoric Design of Electronic Medical Record (EMR) for Periodic Health Examination Reports
Alamer, S. A., Ilyas, Q. M., Ahmad, M., & Irfan, D. (2021) · International Journal of Cloud Applications and Computing · doi:10.4018/ijcac.2022010110 · Cited by 6
Automatic Medical Report Generation via Latent Space Conditioning and Transformers
Adornetto, C., Guzzo, A., & Vasile, A. (2023) · IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech) · doi:10.1109/dasc/picom/cbdcom/cy59711.2023.10361320 · Cited by 6
“Pre-trained models [9] are also being used effectively, eliminating the need for specific vocabularies and enabling faster learning from large-scale datasets.”
MethodBetter Understanding: Stylized Image Captioning with Style Attention and Adversarial Training
Yang, Z., Liu, Q., & Liu, G. (2020) · Symmetry · doi:10.3390/sym12121978 · Cited by 6
“Jing et al. [5] utilized the method of deep learning to generate medical reports.”
A Pathology Image Diagnosis Network with Visual Interpretability and Structured Diagnostic Report
Ma, K., Wu, K., Cheng, H., Gu, C., Xu, R., & Guan, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-04224-0_24 · Cited by 6
Deep-Learning and HPC to Boost Biomedical Applications for Health (DeepHealth)
Caballero, M., Gómez, J. A., & Bantouna, A. (2019) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms.2019.00040 · Cited by 6
“Recent developments in Deep Learning models allowed to automatically generate natural text from features extracted with a multimodal approach, where images and related text are combined in a joint semantic description [3,4].”
BackgroundArtificial Intelligence in Medicine
Strümke, I., Hicks, S. A., Thambawita, V. L., Jha, D., Parasa, S., Riegler, M. A., & Halvorsen, P. (2021) · Springer eBooks · doi:10.1007/978-3-030-58080-3_163-1 · Cited by 5
Metadata
- Authors
- Jing, B., Xie, P., & Xing, E.
- Affiliations
- Carnegie Mellon University (US)
- Venue
- Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2018
- Pages
- 2577-2586
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2770165365
- Cited by
- 657
- References
- 28
- Citations by year
- 2026: 57 · 2025: 123 · 2024: 115 · 2023: 111 · 2022: 80 · 2021: 81 · 2020: 45 · 2019: 34 · 2018: 11
- FWCI
- 12.69
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Keywords
- multi-task learning, co-attention, hierarchical LSTM, tag prediction, clinical report writing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
