METransformer: Radiology Report Generation by Transformer with Multiple Learnable Expert Tokens
Open accessAbstract
In clinical scenarios, multi-specialist consultation could significantly benefit the diagnosis, especially for intricate cases. This inspires us to explore a “multi-expert joint diagnosis” mechanism to upgrade the existing “single expert” framework commonly seen in the current literature. To this end, we propose METransformer, a method to realize this idea with a transformer-based backbone. The key design of our method is the introduction of multiple learnable “expert” tokens into both the transformer encoder and decoder. In the encoder, each expert token interacts with both vision tokens and other expert tokens to learn to attend different image regions for image representation. These expert tokens are encouraged to capture complementary information by an orthogonal loss that minimizes their over-lap. In the decoder, each attended expert token guides the cross-attention between input words and visual tokens, thus influencing the generated report. A metrics-based expert voting strategy is further developed to generate the final report. By the multi-experts concept, our model enjoys the merits of an ensemble-based approach but through a manner that is computationally more efficient and supports more sophisticated interactions among experts. Experimental results demonstrate the promising performance of our proposed model on two widely used benchmarks. Last but not least, the framework-level innovation makes our work ready to incorporate advances on existing “single-expert” models to further improve its performance.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
References to prior studies can be removed to reduce hallucinations when prior studies are not available.
“Such references can be removed to reduce hallucinations when prior studies are not available, 9,14,18 or used in conjunction with prior images to enable descriptions of change.17,19,20”
From MAIRA-2: Grounded Radiology Report Generation (2024) · cites this paper as [14]
The Ablation Study (§4.4; page unavailable) discusses expert tokens and report-generation performance, not removing references to prior studies. Searches for hallucinations, prior images, previous examinations, and removal of references found no passage addressing the claimed practice or its effect. (LLM, conf=0.72)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · cited 14× · see citing works
“Specifically, we compare METransformer with 5 state-of-the-art (SOTA) image captioning methods, including Show-tell [32], AdaAtt [24], Att2in [1], Transformer [7] and M2transformer [6].”
MethodInfluentialSqueeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667 · cited 1× · see citing works
“A squeeze-excitation operation [10] to B mid is performed to obtain channel-wise attention Bilinear Encoder Layer.”
MethodPyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473 · cited 1× · see citing works
“We implement our model using Pytorch [27] and Pytorch-lightning library 2 with two NVIDIA GeForce RTX 3090 GPU cards.”
MethodAn Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 21,793 · cited 3× · see citing works
“1, our encoder comprises a vision transformer (ViT) [9] encoder and an expert bilinear transformer encoder.”
MethodInfluentialLayer NormalizationPreprint
Ba, J., Kiros, J., & Hinton, G. E. (2016) · arXiv.org · doi:10.48550/arxiv.1607.06450 · Cited by 13,745 · cited 2× · see citing works
“Layer-norm(LN) [2] is applied before every block and residual connections are applied after every block.”
BackgroundMethod(2004) · Cited by 9,718 · see citing works
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741 · see citing works
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · cited 4× · see citing works
“As shown in Table 1, on both datasets, our METransformer consistently outperforms those “single-expert” based models, including attention-based baselines (Att2in [37], AdaAtt [24]), memory-augmented meth-”
BackgroundMethodInfluentialShow and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318 · cited 4× · see citing works
“A broad collection of methods was proposed in the last few years [1,6,24,25,28,32,37,42,45,48] and have achieved great success in advancing the stateof-the-art.”
BackgroundMethodInfluentialBottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S. J., & Zhang, L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00636 · Cited by 5,117 · cited 7× · see citing works
“Specifically, we compare METransformer with 5 state-of-the-art (SOTA) image captioning methods, including Show-tell [32], AdaAtt [24], Att2in [1], Transformer [7] and M2transformer [6].”
BackgroundMethodInfluentialCIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845 · see citing works
(2005) · Cited by 4,152 · see citing works
Bleu: a method for automatic evaluation of machine translation
Papineni, K. (2024) · doi:10.57702/ljteqmrb · Cited by 3,682 · see citing works
Bilinear CNN Models for Fine-Grained Visual Recognition
Lin, T., RoyChowdhury, A., & Maji, S. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.170 · Cited by 2,207 · cited 2× · see citing works
“Further, in the bilinear transformer encoder, to enable each “expert” to capture fine-grained image information, we compute higher-order attention between expert tokens and visual tokens, which has proved to be effective in fine-grained classification tasks [20].”
BackgroundMethodSelf-Critical Sequence Training for Image Captioning
Rennie, S. J., Marcheret, E., Mroueh, Y., Ross, J., & Goel, V. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.131 · Cited by 2,177 · see citing works
Image Captioning with Semantic Attention
You, Q., Jin, H., Wang, Z., Chen, F., & Luo, J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.503 · Cited by 1,859 · cited 1× · see citing works
“A broad collection of methods was proposed in the last few years [1,6,24,25,28,32,37,42,45,48] and have achieved great success in advancing the stateof-the-art.”
BackgroundSelf-citationKnowing When to Look: Adaptive Attention via a Visual Sentinel for Image Captioning
Lu, J., Xiong, C., Parikh, D., & Socher, R. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.345 · Cited by 1,724 · cited 5× · see citing works
“Specifically, we compare METransformer with 5 state-of-the-art (SOTA) image captioning methods, including Show-tell [32], AdaAtt [24], Att2in [1], Transformer [7] and M2transformer [6].”
BackgroundMethodInfluentialOn the Variance of the Adaptive Learning Rate and BeyondPreprint
Liu, L., Jiang, H., He, P., Chen, W., Liu, X., Gao, J., & Han, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.03265 · Cited by 1,529 · cited 1× · see citing works
“Our model is trained using Radam optimizer [23] with a mini-batch size of 16.”
MethodSelf-citationMeshed-Memory Transformer for Image Captioning
Cornia, M., Stefanini, M., Baraldi, L., & Cucchiara, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01059 · Cited by 1,259 · cited 2× · see citing works
“A broad collection of methods was proposed in the last few years [1,6,24,25,28,32,37,42,45,48] and have achieved great success in advancing the stateof-the-art.”
BackgroundPreparing a collection of radiology examinations for distribution and retrieval
Demner-Fushman, D., Kohli, M. D., Rosenman, M. B., Shooshan, S. E., Rodriguez, L., Antani, S., Thoma, G. R., & McDonald, C. J. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv080 · Cited by 1,164 · cited 2× · see citing works
“In this experiment, two datasets are used for the performance evaluation, i.e., a widely-used benchmark IU-Xray [8] and the currently largest dataset MIMIC-CXR [13] for medical report generation.”
BackgroundMethodVL-BERT: Pre-training of Generic Visual-Linguistic Representations
Su, W., Zhu, X., Cao, Y., Li, B., Lu, L., Wei, F., & Dai, J. (2019) · arXiv · doi:10.48550/arxiv.1908.08530 · Cited by 950 · cited 1× · see citing works
“We introduce two types of the segment, “[0]” and “[1]”, to separate the input tokens from different sources [29], i.”
BackgroundX-Linear Attention Networks for Image Captioning
Pan, Y., Yao, T., Li, Y., & Mei, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01098 · Cited by 720 · cited 4× · see citing works
“Most of them adopt the conventional encoder-decoder architecture, with convolutional neural networks (CNNs) as the encoder and recurrent (e.g., LSTM/GRU) or non-recurrent networks (e.g., Transformer) as the decoder with a carefully designed attention module [24, 25, 37] to produce image description.”
BackgroundMethodInfluentialGenerating Radiology Reports via Memory-driven Transformer
Chen, Z., Song, Y., Chang, T.-H., & Wan, X. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.112 · Cited by 693 · see citing works
On the Automatic Generation of Medical Imaging Reports
Jing, B., Xie, P., & Xing, E. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1240 · Cited by 657 · cited 8× · see citing works
“Moreover, eight SOTA medical report generation models are involved in the comparison, including CoAtt [12], HGRNAgent [17], R2Gen [5], R2GenCMN [4], PPKED [21], GSK [41] and MSAT [34].”
BackgroundMethodResultInfluentialHadamard Product for Low-rank Bilinear PoolingPreprint
Kim, J.-H., On, K.-W., Lim, W., Kim, J., Ha, J.-W., & Zhang, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.04325 · Cited by 604 · see citing works
Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generation
Liu, F., Wu, X., Ge, S., Fan, W., & Zou, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01354 · Cited by 396 · cited 6× · see citing works
“Moreover, eight SOTA medical report generation models are involved in the comparison, including CoAtt [12], HGRNAgent [17], R2Gen [5], R2GenCMN [4], PPKED [21], GSK [41] and MSAT [34].”
BackgroundMethodInfluentialCheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301590 · Cited by 392 · cited 1× · see citing works
“For this purpose, the CheXpert [11] is applied to labeling the generated reports and the results are compared with ground truths in 14 different categories related to thoracic diseases and support devices.”
MethodCross-modal Memory Networks for Radiology Report Generation
Chen, Z., Shen, Y., Song, Y., & Wan, X. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.459 · Cited by 369 · see citing works
When Radiology Report Generation Meets Knowledge Graph
Zhang, Y., Wang, X., Xu, Z., Yu, Q., Yuille, A., & Xu, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6989 · Cited by 348 · cited 2× · see citing works
“Most recently, many works [16, 21, 22, 40, 41, 47] attempt to integrate knowledge graphs into medical report generation pipeline to improve the quality of the generated reports.”
BackgroundKnowledge-Driven Encode, Retrieve, Paraphrase for Medical Image Report Generation
Li, C. Y., Liang, X., Hu, Z., & Xing, E. P. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016666 · Cited by 316 · cited 2× · see citing works
“[12] and HGRG-Agent [17] on IU-Xray are quoted from [15] while the other results are quoted from their respective papers.”
BackgroundHybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation
Li, C. Y., Liang, X., Hu, Z., & Xing, E. P. (2018) · arXiv · doi:10.48550/arxiv.1805.08298 · Cited by 271 · cited 4× · see citing works
“CoAtt [12] and HGRNAgent [17] only report results on IU-Xray in their original paper.”
BackgroundMethodInfluentialMultimodal Recurrent Model with Attention for Automated Radiology Report Generation
Xue, Y., Xu, T., Long, L. R., Xue, Z., Antani, S., Thoma, G. R., & Huang, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00928-1_52 · Cited by 241 · see citing works
Review Networks for Caption GenerationPreprint
Yang, Z., Yuan, Y., Wu, Y., Salakhutdinov, R., & Cohen, W. W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07912 · Cited by 230 · see citing works
Knowledge matters: Chest radiology report generation with general and specific knowledge
Yang, S., Wu, X., Ge, S., Zhou, S. K., & Xiao, L. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102510 · Cited by 225 · see citing works
Yuan, J., Liao, H., Luo, R., & Luo, J. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32226-7_80 · Cited by 215 · cited 1× · see citing works
“For example, some works [12,35,38,39,43, 46] utilize a hierarchically structured LSTM network to better handle the long narrative nature of medical reports.”
BackgroundRadiology report generation with a learned knowledge base and multi-modal alignment
Yang, S., Wu, X., Ge, S., Zheng, Z., Zhou, S. K., & Xiao, L. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102798 · Cited by 199 · cited 1× · see citing works
“Most recently, many works [16, 21, 22, 40, 41, 47] attempt to integrate knowledge graphs into medical report generation pipeline to improve the quality of the generated reports.”
BackgroundMIMIC-CXR: A large publicly available database of labeled chest radiographsPreprint
Johnson, A. E. W., Pollard, T., Berkowitz, S. J., Greenbaum, N. R., Lungren, M. P., Deng, C.-Y., Mark, R. G., & Horng, S. (2019) · arXiv (Cornell University) · Cited by 182 · see citing works
You, D., Liu, F., Ge, S., Xie, X., Zhang, J., & Wu, X. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87199-4_7 · Cited by 170 · cited 1× · see citing works
“Another group of works [34, 44] utilizes disease tags to facilitate report generation.”
BackgroundAuxiliary signal-guided knowledge encoder-decoder for medical report generation
Li, M., Liu, R., Wang, F., Chang, X., & Liang, X. (2022) · World Wide Web · doi:10.1007/s11280-022-01013-6 · Cited by 149 · cited 2× · see citing works
“Due to the fine-grained nature of radiology images, it is hard to focus on the correct image regions throughout the report generation procedure despite different attentions developed in recent works [16, 21].”
BackgroundMore Grounded Image Captioning by Distilling Image-Text Matching Model
Zhou, Y., Wang, M., Liu, D., Hu, Z., & Zhang, H. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00483 · Cited by 144 · see citing works
Automatic Generation of Medical Imaging Diagnostic Report with Hierarchical Recurrent Neural Network
Yin, C., Qian, B., Wei, J., Li, X., Zhang, X., Li, Y., & Zheng, Q. (2019) · Proceedings · doi:10.1109/icdm.2019.00083 · Cited by 116 · cited 1× · see citing works
“For example, some works [12,35,38,39,43, 46] utilize a hierarchically structured LSTM network to better handle the long narrative nature of medical reports.”
BackgroundAutomated Radiographic Report Generation Purely on Transformer: A Multicriteria Supervised Approach
Wang, Z., Han, H., Wang, L., Li, X., & Zhou, L. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3171661 · Cited by 112 · cited 1× · see citing works
“To further improve performance, some works [4, 5, 33] employ a transformer instead of LSTM as the report decoder, which has achieved good results.”
MethodSelf-citationA Self-boosting Framework for Automated Radiographic Report Generation
Wang, Z., Zhou, L., Wang, L., & Li, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00246 · Cited by 92 · cited 2× · see citing works
“In addition, Wang et al [35] introduced an image-report matching network to bridge the domain gap between image and text for reducing the difficulty of report generation.”
BackgroundSelf-citationA Medical Semantic-Assisted Transformer for Radiographic Report Generation
Wang, Z., Tang, M., Wang, L., Li, X., & Zhou, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16437-8_63 · Cited by 81 · see citing works
Self-citationAuto-Encoding Knowledge Graph for Unsupervised Medical Report GenerationPreprint
Liu, F., You, C., Wu, X., Ge, S., Wang, S., & Sun, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.04318 · Cited by 80 · cited 2× · see citing works
“To measure the accuracy of descriptions for clinical abnormalities, we follow [4, 5, 22] and further report clinical efficacy metrics.”
BackgroundMethodImproved Disease Classification in Chest X-Rays with Transferred Features from Report Generation
Xue, Y., & Huang, X. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20351-1_10 · Cited by 29 · cited 2× · see citing works
“For example, some works [12,35,38,39,43, 46] utilize a hierarchically structured LSTM network to better handle the long narrative nature of medical reports.”
BackgroundLi, Y., Pan, Y., Chen, J., Yao, T., & Mei, T. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3478331 · Cited by 26 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
1 of the 51 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- su, 2019, Vl-bert: Pre-training of generic visuallinguistic representations, ArXiv Preprint
Cited by
Sorted by citations
R2GenGPT: Radiology Report Generation with frozen LLMs
Wang, Z., Liu, L., Wang, L., & Zhou, L. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100033 · Cited by 205 · see citing works
Self-citationExplainable Deep Learning Methods in Medical Image Classification: A Survey
Patrício, C., Neves, J. C., & Teixeira, L. F. (2023) · ACM Computing Surveys · doi:10.1145/3625287 · Cited by 190 · see citing works
“Table 3 conveys the results in terms of a set of NLP evaluation metrics (BLEU score, ROUGE, METEOR, and CIDEr) for each of the selected methods.”
BackgroundPromptMRG: Diagnosis-Driven Prompts for Medical Report Generation
Jin, H., Che, H., Lin, Y., & Chen, H. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i3.28038 · Cited by 151 · see citing works
Automated Radiology Report Generation: A Review of Recent Advances
Sloan, P., Clatworthy, P. L., Simpson, E., & Mirmehdi, M. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2024.3408456 · Cited by 98 · see citing works
“Chen et al. [51]’s method has been utilised to assess an ARRG models performance by various researchers [11], [34], [49], [50], [67], [103], [123], [129].”
InfluentialVision-Language Models in medical image analysis: From simple fusion to general large models
Li, X., Li, L., Jiang, Y., Wang, H., Qiao, X., Feng, T., Luo, H., & Zhao, Y. J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102995 · Cited by 88 · see citing works
Reale-Nosei, G., Amador-Domínguez, E., & Serrano, E. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103264 · Cited by 68 · see citing works
TSGET: Two-Stage Global Enhanced Transformer for Automatic Radiology Report Generation
Yi, X., Fu, Y., Liu, R., Zhang, H., & Hua, R. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3350077 · Cited by 61 · see citing works
“Recently, Transformer is widely used in current automatic radiology report generation methods [17], [18].”
MethodComplex Organ Mask Guided Radiology Report Generation
Gu, T., Liu, D., Li, Z., & Cai, W. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00781 · Cited by 54 · see citing works
“More experiment information has been provided in the Supplementary Material.”
BackgroundMethodInfluentialToken-Mixer: Bind Image and Text in One Embedding Space for Medical Image Reporting
Yang, Y. Q., Yu, J., Fu, Z., Zhang, K., Yu, T., Wang, X., Jiang, H., Lv, J., Huang, Q., & Han, W. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3412402 · Cited by 42 · see citing works
“METransformer [27] proposes learnable expert tokens for Transformer-based encoder and decoder.”
MethodA survey of deep-learning-based radiology report generation using multimodal inputs
Wang, X., Figueredo, G. P., Li, R., Zhang, W. E., Chen, W., & Chen, X. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103627 · Cited by 50 · see citing works
Yi, X., Fu, Y., Yu, J., Liu, R., Zhang, H., & Hua, R. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3507073 · Cited by 35 · see citing works
“In the work [34], Wang et al. introduce the ME-Transformer model, which demonstrates the capability to simulate the process of multi-expert joint diagnosis.”
A Comprehensive Study of GPT-4V’s Multimodal Capabilities in Medical ImagingPreprint
Li, Y., Liu, Y., Wang, Z., Liang, X., Liu, L., Wang, L., Cui, L., Tu, Z., Wang, L., & Zhou, L. (2023) · medRxiv · doi:10.1101/2023.11.03.23298067 · Cited by 28 · see citing works
“The rapid advancements in artificial intelligence, particularly in the domains of computer vision and natural language processing, have made automated medical report generation a feasible reality (e.g., Chen et al., 2020; 2021; Liu et al., 2021; Wang et al., 2023a).”
BackgroundMethodSelf-citationPrompt-Guided Generation of Structured Chest X-Ray Report Using a Pre-trained LLM
Li, H., Wang, H., Sun, X., He, H., & Feng, J. (2024) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme57554.2024.10687707 · Cited by 30 · see citing works
“We conducted a comprehensive comparison of our model with recent methods, as referenced in [3], [7]–[9], [11], [12], using NLG and CE metrics (Tab.”
BackgroundMethodInfluentialChest X-Ray Report Generation Using Abnormality Guided Vision Language Model
Edirisinghe, D., Nimalsiri, W., Hennayake, M., Meedeniya, D., & Lim, G. (2025) · IEEE Access · doi:10.1109/access.2025.3606961 · Cited by 28 · see citing works
“Additionally, we evaluate its performance against other SOTA medical report generation models, such as METransformer [37] and Kiut [45]. encoding of clinical entities and relationships.”
InfluentialKARGEN: Knowledge-Enhanced Automated Radiology Report Generation Using Large Language Models
Li, Y., Wang, Z., Liu, Y., Wang, L., Liu, L., & Zhou, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72086-4_36 · Cited by 27 · see citing works
Self-citationA systematic evaluation of GPT-4V's multimodal capability for chest X-ray image analysis
Liu, Y., Li, Y., Wang, Z., Liang, X., Liu, L., Wang, L., Cui, L., Tu, Z., Wang, L., & Zhou, L. (2024) · Meta-Radiology · doi:10.1016/j.metrad.2024.100099 · Cited by 25 · see citing works
Self-citationCGFTrans: Cross-Modal Global Feature Fusion Transformer for Medical Report Generation
Xu, L., Tang, Q., Zheng, B., Lv, J., Li, W., & Zeng, X. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3414413 · Cited by 22 · see citing works
Cross-Modal Augmented Transformer for Automated Medical Report Generation
Tang, Y., Yuan, Y., Tao, F., & Tang, M. (2025) · IEEE Journal of Translational Engineering in Health and Medicine · doi:10.1109/jtehm.2025.3536441 · Cited by 25 · see citing works
“Furthermore, the proposed CAT demonstrates a significant enhancement over the state-of-the-art model, METransformer [46], with the CIDEr score increasing from 43.5% to 45.7% and from 10.5% to 11.8% on the two datasets, respectively.”
InfluentialAttribute Prototype-Guided Iterative Scene Graph for Explainable Radiology Report Generation
Zhang, K., Yang, Y. Q., Yu, J., Jianping Fan, B., Jiang, H., Huang, Q., & Han, W. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3424505 · Cited by 20 · see citing works
“…[10], PPKED [11], M 2 TR. PROGRESSIVE [12], Contrastive Attention [13], Align-Transformer [14], M 2 Trans w/ NLL [15], M 2 Trans w/ NLL + BS + fc E [15], M 2 Trans w/ NLL + BS + fc EN [15], ITA [18], CvT-21 2DistilGPT2 [19], RAMT-U [8], RGRG [7], UAR [20], KiUT [21], and METransformer [22].”
In-Context Learning for Zero-shot Medical Report Generation
Liu, R., Li, M., Zhao, S., Chen, L., Chang, X., & Yao, L. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680760 · Cited by 22 · see citing works
Improving radiology report generation with multi-grained abnormality prediction
Jin, Y., Chen, W., Tian, Y., Song, Y., & Yan, C. C. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128122 · Cited by 20 · see citing works
HERGen: Elevating Radiology Report Generation with Longitudinal Data
Wang, F., Du, S., & Yu, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73001-6_11 · Cited by 23 · see citing works
Spatio-Temporal and Retrieval-Augmented Modeling for Chest X-Ray Report Generation
Yang, Y. Q., You, X., Zhang, K., Fu, Z., Wang, X., Ding, J., Sun, J., Zhou, Y., Huang, Q., Han, W., et al. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3554498 · Cited by 22 · see citing works
Advancements in Radiology Report Generation: A Comprehensive Analysis
Mamdouh, D., Attia, M., Mohamed, O., Mohamed, N., Lotfy, A., Arafa, T., Rashed, E. A., & Khoriba, G. A. (2025) · Bioengineering · doi:10.3390/bioengineering12070693 · Cited by 20 · see citing works
“The METransformer, proposed by Wang et al., adopts a Vision Transformer (ViT) in its architecture to encode radiographic images using several learnable expert tokens [20].”
RECAP: Towards Precise Radiology Report Generation via Dynamic Disease Progression Reasoning
Hou, W., Cheng, Y., Xu, K., Li, W., & Liu, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.140 · Cited by 19 · see citing works
Unsupervised disease tags for automatic radiology report generation
Yi, X., Fu, Y., Hua, R., Liu, R., & Zhang, H. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105742 · Cited by 19 · see citing works
Contrastive Learning with Counterfactual Explanations for Radiology Report Generation
Li, M., Lin, H., Qiu, L., Liang, X., Chen, L., Saddik, A. E., & Chang, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72775-7_10 · Cited by 24 · see citing works
Adapter-Enhanced Hierarchical Cross-Modal Pre-Training for Lightweight Medical Report Generation
Yu, T., Lu, W., Yang, Y. Q., Han, W., Huang, Q., Yu, J., & Zhang, K. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3535699 · Cited by 18 · see citing works
MAIRA-2: Grounded Radiology Report Generation
Bannur, S., Bouzid, K., Castro, D. C., Schwaighofer, A., Thieme, A., Bond-Taylor, S., Ilse, M., Pérez-García, F., Salvatelli, V., Sharma, H., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.04449 · Cited by 19 · see citing works
Automatic Medical Report Generation: Methods and Applications
Li, G., Tahir, A. M., Zhang, D., Wang, Z. J., & Ward, R. K. (2024) · APSIPA Transactions on Signal and Information Processing · doi:10.1561/116.20240044 · Cited by 19 · see citing works
Self-citationA vision attention driven Language framework for medical report generation
Arısoy, M. V., Arısoy, A., & Uysal, İ. (2025) · Scientific Reports · doi:10.1038/s41598-025-95666-8 · Cited by 15 · see citing works
Automatic medical report generation based on deep learning: A state of the art survey
Liu, X., Xin, J., Shen, Q., Huang, Z., & Wang, Z. (2025) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2024.102486 · Cited by 19 · see citing works
Self-citationPark, J., Kim, S., Yoon, B., Hyun, J., & Choi, K. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3587687 · Cited by 21 · see citing works
“To evaluate single-image MRG performance, we compared our model with state-of-the-art MRG models, including LLM-CXR, RaDialog, METransformer [57], DCL [58], PromptMRG, LM-RRG [2], CheXagent, MAIRA-1 [3], Med-PaLM M, and LLaVA-Rad [4].”
SumGPT: A Multimodal Framework for Radiology Report Summarization to Improve Clinical Performance
Sultan, T., Rony, M. A. T., Islam, M. S., AlShathri, S. I., & El‐Shafai, W. (2025) · IEEE Access · doi:10.1109/access.2025.3528335 · Cited by 16 · see citing works
“For instance, the research in [8] demonstrated the effectiveness of the METransformer model, achieving a ROUGE-L score of 38.0% and a BLEU score of 17.2%.”
InfluentialEnhancing ECG Report Generation With Domain-Specific Tokenization for Improved Medical NLP Accuracy
Ashfaq, F., Jhanjhi, N. Z., Khan, N. A., Javed, D., Masud, M., & Shorfuzzaman, M. (2025) · IEEE Access · doi:10.1109/access.2025.3567566 · Cited by 15 · see citing works
“The authors in [13] and [14] uses a transformer based architecture for generating medical reports, specifically in the context of Chest X-ray images.”
LLM-Driven Medical Report Generation via Communication-Efficient Heterogeneous Federated Learning
Che, H., Jin, H., Guo, Z., Lin, Y., Jin, C., & Chen, H. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3591185 · Cited by 16 · see citing works
“Recognizing its critical role in healthcare, researchers have developed advanced approaches spanning memory modules [26]–[28], knowledge graphs [29]– [32], multi-task learning frameworks [12], [33]–[37], and mix-of-expert architectures [38].”
Tang, Y., Wang, D., Zhang, L., & Yuan, Y. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105651 · Cited by 14 · see citing works
Advancement in medical report generation: current practices, challenges, and future directions
Rehman, M., Shafi, I., Ahmad, J., García, C., Barrera, A. E. P., & Ashraf, I. (2024) · Medical & Biological Engineering & Computing · doi:10.1007/s11517-024-03265-y · Cited by 13 · see citing works
Automatic medical report generation combining contrastive learning and feature difference
Lyu, C., Qiu, C., Han, K., Li, S., Sheng, V. S., Rong, H., Song, Y., Liu, Y., & Qiu, C. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112630 · Cited by 11 · see citing works
Izhar, A., Idris, N. B., & Japar, N. B. (2025) · Cognitive Computation · doi:10.1007/s12559-025-10404-6 · Cited by 11 · see citing works
“Additionally, METransformer [28] incorporated an expert bi-linear attention mechanism, which was integrated into the ViT image encoder and transformer decoder to focus on various important regions within the radiology image.”
Pre-training on high-resolution X-ray images: an experimental study
Wang, X., Li, Y., Wu, W., Jin, J., Rong, Y., Jiang, B., Li, C., & Tang, J. (2025) · Visual Intelligence · doi:10.1007/s44267-025-00080-3 · Cited by 13 · see citing works
“In this paper, we focus on handling two representative tasks, including medical report generation [44–51] and disease classification [52–57].”
ChEX: Interactive Localization and Region Description in Chest X-Rays
Müller, P., Kaissis, G. A., & Rueckert, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72664-4_6 · Cited by 12 · see citing works
Act Like a Radiologist: Radiology Report Generation Across Anatomical Regions
Chen, Q., Xie, Y., Wu, B., Chen, X., Ang, J. S. K., To, M., Chang, X., & Wu, Q. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0960-4_3 · Cited by 12 · see citing works
Mei, X., Yang, L., Gao, D., Cai, X., Han, J., & Liu, T. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3543101 · Cited by 11 · see citing works
Enabling Multi-modal Conversational Interface for Clinical Imaging
Dayanandan, K., & Lall, B. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650805 · Cited by 9 · see citing works
“…answering for medical imaging are technically similar to conversational interfaces as they generate textual information from images [7, 30, 67, 73, 85], but are not optimized for factual correctness [49, 85, 86], and suffer from hallucination [31, 82, 88], which is unacceptable in clinical…”
Liu, K., Ma, Z., Kang, X., Zhong, Z., Jiao, Z., Baird, G. L., Bai, H. X., & Miao, Q. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72384-1_41 · Cited by 10 · see citing works
“Currently, there is a considerable body of studies on medical report generation [5,32,10,25,26], contributing significantly to the improvement of clinical effectiveness and linguistic fluency in reports.”
HKRG: Hierarchical knowledge integration for radiology report generation
Wang, B., Teng, P., Zhang, H., Yang, F., Wang, Z., Yi, X., Zhang, T., Wang, C., Tavares, A., & Xu, H. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126622 · Cited by 9 · see citing works
Self-citationORID: Organ-Regional Information Driven Framework for Radiology Report Generation
Gu, T., Yang, K., An, X., Feng, Z., Lin, D., & Cai, W. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00047 · Cited by 10 · see citing works
“Consistent with the methodology outlined in [6], both datasets were partitioned into training, validation, and testing sets in a 7:2:1 ratio. https://openi.nlm.nih.gov/faq https://physionet.org/content/mimic-cxr/2.0.0/”
InfluentialSong, S., Tang, H., Yang, H., & Li, X. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3591364 · Cited by 11 · see citing works
“To achieve it, some studies improved the encoder-decoder framework by designing new networks and modules [11], [12], [13], [14].”
Activating Associative Disease-Aware Vision Token Memory for LLM-Based X-Ray Report Generation
Wang, X., Wang, F., Wang, H., Jiang, B., Li, C., Wang, Y., Tian, Y., & Tang, J. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3603416 · Cited by 12 · see citing works
Xiao, L., Zhang, Y. Q., Wang, D., Zhong, H., Li, R., & Wang, Q. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681201 · Cited by 9 · see citing works
Visual-Textual Cross-Modal Interaction Network for Radiology Report Generation
Zhang, W., Cai, B., Hu, J., Qin, Q., & Xie, K. (2024) · IEEE Signal Processing Letters · doi:10.1109/lsp.2024.3379005 · Cited by 10 · see citing works
“To address this problem, [10] introduced memory vectors to align image and text features, while [6] introduced multiple expert tokens to focus on different areas.”
BackgroundLi, Y., Kong, C., Zhao, G., & Zhao, Z. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11337-0 · Cited by 7 · see citing works
“Wang et al. (2023) paired the X-Linear design with expert tokens, enabling consistent attention allocation across diverse image regions for robust image representation.”
InfluentialCherukuri, T. K., Shaik, N. S., Bodapati, J. D., & Ye, D. H. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890310 · Cited by 8 · see citing works
“The ME Transformer [18] introduced learnable expert tokens for better modality adaptation, yet its high computational cost limits practical deployment.”
Izhar, A., Idris, N., & Japar, N. B. (2025) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2025.103220 · Cited by 9 · see citing works
Diffusion Networks with Task-Specific Noise Control for Radiology Report Generation
Tian, Y., Xia, F., & Song, Y. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681476 · Cited by 7 · see citing works
“They leverage a visual encoder to capture visual features and use a text decoder to produce reports, where advanced architectures such as convolutional neural networks and Transformers are used.”
Interactive dual-stream contrastive learning for radiology report generation
Zhang, Z., & Jiang, A. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104718 · Cited by 6 · see citing works
CLR2G: Cross modal Contrastive Learning on Radiology Report Generation
Xue, H., Ma, Q., Liu, G., Qu, J., Liu, Y., & Liu, A. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679668 · Cited by 8 · see citing works
Topicwise Separable Sentence Retrieval for Medical Report Generation
Zhao, J., Yang, Z., Chen, Z., Fu, H., & Wan, L. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3507076 · Cited by 9 · see citing works
“…LSTM-based model like CA [49], (2) generation-based models, i.e. R2Gen [6], PPKED [11], Multicrite-ria [9], KnowMT [19], ICT [50], METransformer [52]), PromptMRG [54] (3) knowledge-graph-enhanced models such as KGAE [10], DCL [51], RAMT [53], (4) retrieval-based model like TranSQ [15], (5)…”
MethodRahman, Z. U., Lee, J.-H., Vu, D. T., Murtza, I., & Kim, J. Y. (2025) · IEEE Access · doi:10.1109/access.2025.3538325 · Cited by 6 · see citing works
He, Z., Wong, A. N. N., & Yoo, J. S. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110625 · Cited by 7 · see citing works
MRScore: Evaluating Medical Report with LLM-Based Reward System
Liu, Y., Wang, Z., Li, Y., Liang, X., Liu, L., Wang, L., & Zhou, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72384-1_27 · Cited by 13 · see citing works
Self-citationAbnormal-region-aware Multi-modal Feature Fusion for medical report generation
Gao, Y., Ni, Z., Liu, W., Ni, L., Xin, L., Hu, L., & Zhang, L. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113538 · Cited by 6 · see citing works
Contrastive Knowledge-Guided Large Language Models for Medical Report Generation
Sha, Y., Pan, H., Meng, W., & Li, K. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04978-0_11 · Cited by 5 · see citing works
Gao, W., Li, W., Fang, D., Gong, Z., Chen, C., Deng, Z., Rong, F., Chen, L., Feng, L., Huang, C., et al. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102877 · Cited by 5 · see citing works
MeFD-Net: multi-expert fusion diagnostic network for generating radiology image reports
Ran, R., Pan, R., Yang, W., Deng, Y. X., Zhang, W., Hu, W., & Qing, Q. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05680-y · Cited by 6 · see citing works
“Additionally, METransformer [9] bears some similarities to our proposed approach.”
Wang, Y., Sun, Y., Tan, T., Hao, C., Cui, Y., Su, X., Xie, W., Shen, L., & Yu, Z. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04981-0_61 · Cited by 4 · see citing works
Enhancing Radiology Report Generation via Multi-Phased Supervision
Chen, Z., Li, Y., Wang, Z., Gao, P., Barthélemy, J., Zhou, L., & Wang, L. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3580659 · Cited by 5 · see citing works
“Efforts have been made in the literature by introducing additional structures or procedures such as object detection [6], [7] and feature alignment [8], [9], [10] to enhance feature extraction from radiology images.”
Self-citationR2GenCSR: Mining Contextual and Residual Information for LLMs-Based Radiology Report Generation
Wang, X., Li, Y., Wang, F., Wang, S., Li, C., & Jiang, B. (2026) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2026.3669539 · Cited by 15 · see citing works
“ME-Transformer [55] simulates multi-expert joint diagnostics by introducing multiple learnable “expert” tokens.”
Zhu, Z., Cheng, X., Zhang, Y., Chen, Z., Long, Q., Li, H., Huang, Z., Wu, X., & Zheng, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72384-1_29 · Cited by 4 · see citing works
A Doctors Behavior Aware and Domain Knowledge Driven Model for Medical Report Generation
Wang, H., Niu, J., Liu, X., & Wang, Y. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385596 · Cited by 4 · see citing works
“Precision Recall F1 AdaAtt [29] 0.268 0.186 0.181 Att2in [23] 0.322 0.239 0.249 Up-Down [24] 0.320 0.231 0.238 R2Gen [3] 0.333 0.273 0.276 CMN [13] 0.334 0.275 0.278 METransformer [22] 0.364 0.309 0.311 DEKG 0.412 0.292 0.306 We refer to [3], [13] to preprocess the above 4 datasets.”
InfluentialReport is a mixture of topics: Topic-guided radiology report generation
Li, G., Huang, C., Zhou, X., Ji, D., & Zhang, H. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103586 · Cited by 3 · see citing works
Wang, J., Zhu, L., Bhalerao, A. H., & He, Y. (2025) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2025.102644 · Cited by 5 · see citing works
Noh, W.-J., Pi, S.-W., & Lee, B.-D. (2026) · Scientific Reports · doi:10.1038/s41598-025-34799-2 · Cited by 5 · see citing works
“METransformer 8 enhances representation learning by utilizing multiple parameter sets and orthogonal loss, encouraging diverse feature learning without requiring external annotations.”
ADCNet: Anomaly-Driven Cross-Modal Contrastive Network for Medical Report Generation
Liu, Y., Zhang, J., Liu, K., & Tan, L. (2025) · Electronics · doi:10.3390/electronics14030532 · Cited by 3 · see citing works
MedKit: Multi-level feature distillation with knowledge injection for radiology report generation
Su, Z., Song, H., Lin, Y., Wu, Y., Weng, X., Mao, Z., Liu, B., Yin, H., & Yang, J. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129003 · Cited by 4 · see citing works
ARDGen: Augmentation Regularization for Domain-Generalized Medical Report Generation
Ahsan, S. B., Ikhalas, M., Khan, M. M., et al. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00649 · Cited by 3 · see citing works
“We compare ARDGen with state-of-the-art methods on seen MIMIC-CXR and unseen IU X-Ray datasets Table 1 [4, 11, 13, 25, 31, 34, 41, 44].”
Label knowledge guided transformer for automatic radiology report generation
Wang, R., & Liang, J. (2025) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2025.108877 · Cited by 3 · see citing works
Song, T., Lin, Q., Zeng, X., Cao, Y., Man, Z., Liu, C., Cai, Z., Gui, C., & Huang, X. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.108359 · Cited by 2 · see citing works
A medical report generation method based on local visual modeling and image-text co-enhancement
Zhou, Y., Zhang, S., Wang, X., Sun, Y., Qin, Y., & Wang, H. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.108527 · Cited by 2 · see citing works
Pseudo-labeling and knowledge-guided contrastive learning for radiology report generation
Ye, F., Hu, X. Z., Ding, Y., & Liu, F. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104941 · Cited by 2 · see citing works
VTAG: Visual-Textual Association Guided Radiology Reports Generation
Su, Z., Lin, Y., Song, H., Jian, R., Liu, B., & Yang, J. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3623915 · Cited by 2 · see citing works
“Specifically, we compared the metrics of VTAG with several state-of-the-art (SOTA) modeling methods, including Transformer [11], M2Transformer [73], R2Gen [33], PPKED [53], CvT-212DistilGPT2 [71], KiUT [72], METransformer [73], RGRG [31], LLaVA-Med [29], PromptMRG [55], LLM-CXR [28], R2GenGPT [26],…”
Zhang, R., Huang, P., Ding, T., Chen, Y., Tian, X., Cao, Y., Chen, W., Chen, X., Chen, Q., & Zhu, F. (2025) · Oral Diseases · doi:10.1111/odi.70148 · Cited by 2 · see citing works
“SegORG achieved an F1-score of 0.695, surpassing all baseline models, including the previous top performer, METransformer (0.677).”
InfluentialLee, K., Yoon, S., & Lim, H. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.130633 · Cited by 2 · see citing works
Text-image alignment for ILD imaging: linking CXR evidence to CT quantification
Gao, J., Ren, Y., Yang, F., Hu, X., Changbo, S. U. N., Wang, S., & Chen, C. (2026) · npj Digital Medicine · doi:10.1038/s41746-025-02292-9 · Cited by 2 · see citing works
“Recent systems such as METransformer and KiUT report strong overall performance by introducing expert tokens and knowledge-injected U-shapedinteractions,respectively 40,41 .”
Ramesh, S., & Umapathy, S. (2026) · The Egyptian Journal of Neurology Psychiatry and Neurosurgery · doi:10.1186/s41983-026-01091-7 · Cited by 2 · see citing works
Khatoon, S., & Mahmood, A. (2026) · Journal of Imaging · doi:10.3390/jimaging12020068 · Cited by 3 · see citing works
“However its performance on ensemble-based metrics like CIDEr lagged behind models like METransformer [22] and R2Gen [23].”
LLM-Driven Chest X-Ray Report Generation With a Modular, Reduced-Size Architecture
Vargas, T. V., Pedrini, H., & Santanchè, A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-79032-4_14 · Cited by 2 · see citing works
Su, J., Chen, C., Lin, Y., Song, Z., Li, M., & Cao, W. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.107805 · Cited by 2 · see citing works
Yang, H., Tang, H., Song, S., & Li, X. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3589103 · Cited by 4 · see citing works
“More recent models like HReMRG-MR [61] and METransformer [62] explore high-order interactions or multidimensional attention to improve alignment.”
Allaberdiev, S., Chen, X., Mamatov, D., Abdusalomov, A., Ubaydullaev, U., & Khan, A. (2026) · IET Software · doi:10.1049/sfw2/5529107 · Cited by 1 · see citing works
Sun, J., Wei, D., Xu, Z., Lu, D., Liu, H., Wang, H., Tsaftaris, S. A., McDonagh, S., Zheng, Y., & Wang, L. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3516954 · Cited by 1 · see citing works
“For example, some studies [33], [71] focused on improving the accuracy of abnormality detection without considering report generation, whereas others [7], [72] ignored the rich information in the detection task while improving the generated reports.”
Self-citationRMPT: Reinforced Memory-Driven Pure Transformer for Automatic Chest X-Ray Report Generation
Qin, C., Xiong, Y., Chen, W., & Li, Y. (2025) · Mathematics · doi:10.3390/math13091492 · Cited by 1 · see citing works
Joint Imbalance Adaptation for Radiology Report Generation
Li, W., Han, G., Wu, Y., Huang, I., & Huang, X. (2025) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-025-00205-9 · Cited by 1 · see citing works
S-RRG-Bench: Structured radiology report generation with fine-grained evaluation framework
Li, Y., Liu, Y., Wang, Z., Liang, X., Liu, L., Wang, L., & Zhou, L. (2025) · Meta-Radiology · doi:10.1016/j.metrad.2025.100171 · Cited by 1 · see citing works
“METransformer [28] advanced this line of work by introducing multiple learnable “expert” tokens into both the encoder and decoder of a Transformer architecture.”
Self-citationGraph-Driven Medical Report Generation with Adaptive Knowledge Distillation
Chen, J., Huang, X., Jiang, M., Li, Y., Zou, Z., & Qian, D. (2025) · Applied Sciences · doi:10.3390/app152010974 · Cited by 1 · see citing works
Zheng, Z., Zhang, X., Zhang, Y., Liang, E., Weng, Z., & Chai, J. (2025) · IEEE International Conference on Signal Processing and Machine Learning (SPML) · doi:10.1109/spml66318.2025.11199804 · Cited by 1 · see citing works
“As a result, some researchers proposed the transformer model [5,6,7,8,9,10].”
DS-AKDGen: Dual-Stream Alignment-aware Knowledge Distillation for Medical Report Generation
Deng, Y. X., Liu, N., & Ni, J. (2025) · International Conference on Internet of Things, Automation and Artificial Intelligence (IoTAAI) · doi:10.1109/iotaai66837.2025.11213335 · Cited by 1 · see citing works
“BL-1 BL-2 BL-3 BL-4 MTR RG-L ATT2IN [10] 0.325 0.203 0.136 0.096 0.134 0.276 ADAATT [11] 0.299 0.185 0.124 0.088 0.118 0.266 CMCL [12] 0.344 0.217 0.140 0.097 0.133 0.281 R2GenCMN [13] 0.353 0.218 0.148 0.106 0.142 0.278 RAMT [14] 0.362 0.229 0.157 0.0.113 0.153 0.283 METtransformer [15]”
InfluentialChen, K., Zhu, X., Zhang, W., Jiang, S., & Chen, Q. (2025) · Health Information Science and Systems · doi:10.1007/s13755-025-00389-9 · Cited by 2 · see citing works
“– Knowledge Distillation and Alignment Models : PPKED [21], AlignTrans [38], and METransformer [33] employ techniques such as knowledge distilla-Page tion and alignment strategies to improve the interaction between image regions and textual descriptions, ensuring better semantic understanding and…”
Rahman, Z. U., Yu, G.-H., Lee, J., & Kim, J.-Y. (2026) · Applied Sciences · doi:10.3390/app16010470 · Cited by 1 · see citing works
- Authors
- Wang, Z., Liu, L., Wang, L., & Zhou, L.
- Affiliations
- The University of Sydney (AU)
- The University of Adelaide (AU)
- University of Wollongong (AU)
- Venue
- 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2023-06
- Pages
- 11558-11567
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4386076536
- Semantic Scholar
- 4d90382730eefbe8639e64f6855f206657758cc7
- Cited by
- 184
- References
- 51
- Citations by year
- 2026: 58 · 2025: 82 · 2024: 37 · 2023: 7
- FWCI
- 19.62
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
- Research Online (University of Wollongong) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- radiology report generation, vision-language transformers, cross-attention
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
