Clinical-BERT: Vision-Language Pre-training for Radiograph Diagnosis and Reports Generation

Yan, Bin; Pei, Mingtao

doi:10.1609/aaai.v36i3.20204

Cited by 94Multimodal Machine Learning ApplicationsArtificial Intelligence in Healthcare and EducationCOVID-19 diagnosis using AI

Clinical-BERT: Vision-Language Pre-training for Radiograph Diagnosis and Reports Generation

Open access

Yan, Bin, Pei, Mingtao·2022·Proceedings of the AAAI Conference on Artificial Intelligence

DOI 10.1609/aaai.v36i3.20204

Open this paper in CoChat — free account

Abstract

In this paper, we propose a vision-language pre-training model, Clinical-BERT, for the medical domain, and devise three domain-specific tasks: Clinical Diagnosis (CD), Masked MeSH Modeling (MMM), Image-MeSH Matching (IMM), together with one general pre-training task: Masked Language Modeling (MLM), to pre-train the model. The CD task helps the model to learn medical domain knowledge by predicting disease from radiographs. Medical Subject Headings (MeSH) words are important semantic components in radiograph reports, and the MMM task helps the model focus on the prediction of MeSH words. The IMM task helps the model learn the alignment of MeSH words with radiographs by matching scores obtained by a two-level sparse attention: region sparse attention and word sparse attention. Region sparse attention generates corresponding visual features for each word, and word sparse attention enhances the contribution of images-MeSH matching to the matching scores. To the best of our knowledge, this is the first attempt to learn domain knowledge during pre-training for the medical domain. We evaluate the pre-training model on Radiograph Diagnosis and Reports Generation tasks across four challenging datasets: MIMIC-CXR, IU X-Ray, COV-CTR, and NIH, and achieve state-of-the-art results for all the tasks, which demonstrates the effectiveness of our pre-training model.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1609/aaai.v36i3.20204

Cites

58 of 58 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Vision language models can perform joint embedding of radiographs and clinical notes, enabling effective learning of features from different modes.
“With the use of vision language models, it is possible to perform joint embedding of radiographs and clinical notes, thus enabling them to learn the features of different modes effectively48.”
From K-STAMM: a knowledge-enhanced spatial - temporal attention model with multimodal fusion for pneumonia prediction (2026) · cites this paper as [48]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Attention Is All You Need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
MizAR 60 for Mizar 50
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522 · see citing works
Untitled work
(2004) · Cited by 9,718 · see citing works
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · see citing works
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318 · see citing works
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S. J., & Zhang, L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00636 · Cited by 5,117 · see citing works
CIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845 · see citing works
ChestX-Ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., & Summers, R. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.369 · Cited by 4,616 · see citing works
Bleu: a method for automatic evaluation of machine translation
Papineni, K. (2024) · doi:10.57702/ljteqmrb · Cited by 3,682 · see citing works
CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep LearningPreprint
Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D. Y., Bagul, A., Langlotz, C. P., Shpanskaya, K. S., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.05225 · Cited by 2,283 · see citing works
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
Lu, J., Batra, D., Parikh, D., & Lee, S. (2019) · arXiv · doi:10.48550/arxiv.1908.02265 · Cited by 2,109 · see citing works
Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Li, X., Yin, X. P., Li, C., Zhang, P., Hu, X., Zhang, L., Wang, L., Hu, H., Li, D., Wei, F., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58577-8_8 · Cited by 1,736 · see citing works
VisualBERT: A Simple and Performant Baseline for Vision and Language
Li, L. H., Yatskar, M., Yin, D., Hsieh, C.-J., & Chang, K.-W. (2019) · arXiv · doi:10.48550/arxiv.1908.03557 · Cited by 1,198 · see citing works
Preparing a collection of radiology examinations for distribution and retrieval
Demner-Fushman, D., Kohli, M. D., Rosenman, M. B., Shooshan, S. E., Rodriguez, L., Antani, S., Thoma, G. R., & McDonald, C. J. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv080 · Cited by 1,164 · see citing works
VL-BERT: Pre-training of Generic Visual-Linguistic Representations
Su, W., Zhu, X., Cao, Y., Li, B., Lu, L., Wei, F., & Dai, J. (2019) · arXiv · doi:10.48550/arxiv.1908.08530 · Cited by 950 · see citing works
Unified Vision-Language Pre-Training for Image Captioning and VQA
Zhou, L., Palangi, H., Zhang, L., Hu, H., Corso, J., & Gao, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.7005 · Cited by 864 · see citing works
Unicoder-VL: A Universal Encoder for Vision and Language by Cross-Modal Pre-Training
Li, G., Duan, N., Fang, Y., Gong, M., & Jiang, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6795 · Cited by 795 · see citing works
Medical Subject Headings (MeSH).
(2000) · Europe PMC (PubMed Central) · Cited by 789 · see citing works
Generating Radiology Reports via Memory-driven Transformer
Chen, Z., Song, Y., Chang, T.-H., & Wan, X. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.112 · Cited by 693 · see citing works
On the Automatic Generation of Medical Imaging Reports
Jing, B., Xie, P., & Xing, E. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1240 · Cited by 657 · see citing works
TieNet: Text-Image Embedding Network for Common Thorax Disease Classification and Reporting in Chest X-Rays
Wang, X., Peng, Y., Lü, L., Lu, Z., & Summers, R. M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00943 · Cited by 574 · see citing works
Contrastive Learning of Medical Visual Representations from Paired Images and TextPreprint
Zhang, Y., Jiang, H., Miura, Y., Manning, C. D., & Langlotz, C. P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.00747 · Cited by 465 · see citing works
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301590 · Cited by 392 · see citing works
ChestX-ray: Hospital-Scale Chest X-ray Database and Benchmarks on Weakly Supervised Classification and Localization of Common Thorax Diseases
Wang, X., Peng, Y., Lü, L., Lu, Z., Bagheri, M. H., & Summers, R. M. (2019) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-030-13969-8_18 · Cited by 377 · see citing works
Cross-modal Memory Networks for Radiology Report Generation
Chen, Z., Shen, Y., Song, Y., & Wan, X. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.459 · Cited by 369 · see citing works
Cross-Modal Contrastive Learning for Text-to-Image Generation
Zhang, H., Koh, J. Y., Baldridge, J., Lee, H., & Yang, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00089 · Cited by 359 · see citing works
When Radiology Report Generation Meets Knowledge Graph
Zhang, Y., Wang, X., Xu, Z., Yu, Q., Yuille, A., & Xu, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6989 · Cited by 348 · see citing works
Meteor 1.3: Automatic Metric for Reliable Optimization and Evaluation of Machine Translation Systems
Denkowski, M., & Lavie, A. (2011) · Cited by 344 · see citing works
Knowledge-Driven Encode, Retrieve, Paraphrase for Medical Image Report Generation
Li, C. Y., Liang, X., Hu, Z., & Xing, E. P. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016666 · Cited by 316 · see citing works
Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning
Huang, Z., Zeng, Z., Huang, Y., Liu, B., Fu, D., & Fu, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01278 · Cited by 231 · see citing works
Multi-Modal Understanding and Generation for Medical Images and Text via Vision-Language Pre-Training
Moon, J. H., Lee, H., Shin, W., Kim, Y., & Choi, E. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3207502 · Cited by 221 · see citing works
Clinically Accurate Chest X-Ray Report GenerationPreprint
Liu, G., Hsu, T.-M. H., McDermott, M. B. A., Boag, W., Weng, W., Peter Szolovits, B., & Ghassemi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.02633 · Cited by 212 · see citing works
MIMIC-CXR: A large publicly available database of labeled chest radiographsPreprint
Johnson, A. E. W., Pollard, T., Berkowitz, S. J., Greenbaum, N. R., Lungren, M. P., Deng, C.-Y., Mark, R. G., & Horng, S. (2019) · arXiv (Cornell University) · Cited by 182 · see citing works
Auxiliary signal-guided knowledge encoder-decoder for medical report generation
Li, M., Liu, R., Wang, F., Chang, X., & Liang, X. (2022) · World Wide Web · doi:10.1007/s11280-022-01013-6 · Cited by 149 · see citing works
Competence-based Multimodal Curriculum Learning for Medical Report Generation
Liu, F., Ge, S., & Wu, X. (2021) · doi:10.18653/v1/2021.acl-long.234 · Cited by 145 · see citing works
Show, Describe and Conclude: On Exploiting the Structure Information of Chest X-ray Reports
Jing, B., Wang, Z., & Xing, E. (2019) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/p19-1657 · Cited by 131 · see citing works
Oscar: Object-Semantics Aligned Pre-training for Vision-Language TasksPreprint
Li, X., Yin, X., Li, C., Zhang, P., Hu, X., Zhang, L., Wang, L., Hu, H., Li, D., Wei, F., et al. (2020) · arXiv (Cornell University) · doi:10.4230/oasics.ldk.2021.36 · Cited by 129 · see citing works
Kaleido-BERT: Vision-Language Pre-training on Fashion Domain
Zhuge, M., Gao, D., Fan, D.-P., Jin, L., Chen, B., Zhou, H., Qiu, M., & Shao, L. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01246 · Cited by 128 · see citing works
Weakly Supervised Deep Learning for Thoracic Disease Classification and Localization on Chest X-rays
Yan, C., Yao, J., Li, R., Xu, Z., & Huang, J. (2018) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3233547.3233573 · Cited by 121 · see citing works
Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-trainingPreprint
Li, G., Duan, N., Fang, Y., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.06066 · Cited by 115 · see citing works
Deep Mining External Imperfect Data for Chest X-Ray Disease Screening
Luo, L., Yu, L., Chen, H., Liu, Q., Wang, X., Xu, J., & Heng, P. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.3000949 · Cited by 95 · see citing works
Lesion Location Attention Guided Network for Multi-Label Thoracic Disease Classification in Chest X-Rays
Chen, B., Li, J., Lu, G., & Zhang, D. D. (2019) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2019.2952597 · Cited by 84 · see citing works
Unified Vision-Language Pre-Training for Image Captioning and VQAPreprint
Zhou, L., Palangi, H., Zhang, L., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.11059 · Cited by 74 · see citing works
VIVO: Visual Vocabulary Pre-Training for Novel Object Captioning
Hu, X., Yin, X. P., Lin, K., Zhang, L., Gao, J., Wang, L., & Liu, Z. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16249 · Cited by 73 · see citing works
Thoracic Disease Identification and Localization with Limited Supervision
Li, Z., Wang, C., Han, M., Xue, Y., Wei, W., Li, L.-J., & Fei-Fei, L. (2019) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-030-13969-8_7 · Cited by 67 · see citing works
TieNet: Text-Image Embedding Network for Common Thorax Disease Classification and Reporting in Chest X-raysPreprint
Wang, X., Peng, Y., Lü, L., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.04334 · Cited by 57 · see citing works
Writing by Memorizing: Hierarchical Retrieval-based Medical Report Generation
Yang, X., Ye, M., You, Q., & Ma, F. (2021) · doi:10.18653/v1/2021.acl-long.387 · Cited by 49 · see citing works
Learning Visual-Semantic Embeddings for Reporting Abnormal Findings on Chest X-rays
Ni, J., Hsu, C., Gentili, A., & McAuley, J. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.176 · Cited by 41 · see citing works
Generating Radiology Reports via Memory-driven TransformerPreprint
Chen, Z., Song, Y., Chang, T., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.16056 · Cited by 26 · see citing works
Cross-modal Memory Networks for Radiology Report Generation
Chen, Z., Shen, Y., Song, Y., & Wan, X. (2022) · arXiv · doi:10.48550/arxiv.2204.13258 · Cited by 24 · see citing works
Exploring and Distilling Posterior and Prior Knowledge for Radiology Report GenerationPreprint
Liu, F., Wu, X., Ge, S., Fan, W., & Zou, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.06963 · Cited by 20 · see citing works
Competence-based Multimodal Curriculum Learning for Medical Report GenerationPreprint
Liu, F., Ge, S., Zou, Y., & Wu, X. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.14579 · Cited by 19 · see citing works
VIVO: Visual Vocabulary Pre-Training for Novel Object CaptioningPreprint
Hu, X., Yin, X. P., Lin, K., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.13682 · Cited by 16 · see citing works
Deep Mining External Imperfect Data for Chest X-ray Disease ScreeningPreprint
Luo, L., Yu, L., Chen, H., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03796 · Cited by 7 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805

Flagged first, then by citations

Research works printed but not linked: 3

Chen, Z.; Shen, Y.; Song, Y.; and Wan, X. 2021. Crossmodal Memory Networks for Radiology Report Generation. In ACL/IJCNLP .
Murahari, V. S.; Batra, D.; Parikh, D.; and Das, A. 2020. Large-scale Pretraining for Visual Dialog: A Simple Stateof-the-Art Baseline. In ECCV .
Zhuge, M.; Gao, D.; Fan, D.-P.; Jin, L.; Chen, B.; Zhou, H.; Qiu, M.; and Shao, L. 2021. Kaleido-BERT: VisionLanguage Pre-training on Fashion Domain. arXiv preprint , arXiv:2103.16110. · arXiv

Sorted by citations

Large-scale Multi-modal Pre-trained Models: A Comprehensive Survey
Wang, X., Chen, G., Qian, G., Gao, P., Wei, X.-Y., Wang, Y., Tian, Y., & Gao, W. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1410-8 · Cited by 245 · see citing works
Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing
Bannur, S., Hyland, S., Liu, Q., Pérez-García, F., Ilse, M., Castro, D. C., Boecking, B., Sharma, H., Bouzid, K., Thieme, A., et al. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01442 · Cited by 227 · see citing works
Revolutionizing Digital Pathology With the Power of Generative Artificial Intelligence and Foundation Models
Waqas, A., Bui, M. M., Glassy, E. F., Naqa, I. E., Borkowski, P., Borkowski, A., & Rasool, G. (2023) · Laboratory Investigation · doi:10.1016/j.labinv.2023.100255 · Cited by 149 · see citing works
PromptMRG: Diagnosis-Driven Prompts for Medical Report Generation
Jin, H., Che, H., Lin, Y., & Chen, H. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i3.28038 · Cited by 151 · see citing works
Foundation Model for Advancing Healthcare: Challenges, Opportunities and Future Directions
He, Y., Huang, F., Jiang, X., Nie, Y., Wang, M., Wang, J., & Chen, H. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2024.3496744 · Cited by 130 · see citing works
Automated Radiology Report Generation: A Review of Recent Advances
Sloan, P., Clatworthy, P. L., Simpson, E., & Mirmehdi, M. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2024.3408456 · Cited by 98 · see citing works
Vision-language foundation models for medical imaging: a review of current practices and innovations
Ryu, J. S., Kang, H., Chu, Y., & Yang, S. (2025) · Biomedical Engineering Letters · doi:10.1007/s13534-025-00484-6 · Cited by 99 · see citing works
Vision-Language Models in medical image analysis: From simple fusion to general large models
Li, X., Li, L., Jiang, Y., Wang, H., Qiao, X., Feng, T., Luo, H., & Zhao, Y. J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102995 · Cited by 88 · see citing works
A scoping review on multimodal deep learning in biomedical images and texts
Sun, Z., Lin, M., Zhu, Q., Xie, Q., Wang, F., Lu, Z., & Peng, Y. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104482 · Cited by 69 · see citing works
From explainable to interpretable deep learning for natural language processing in healthcare: How far from reality?
Huang, G., Li, Y., Jameel, S., Long, Y., & Papanastasiou, G. (2024) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2024.05.004 · Cited by 74 · see citing works
Towards long-tailed, multi-label disease classification from chest X-ray: Overview of the CXR-LT challenge
Holste, G., Zhou, Y., Wang, S., Jaiswal, A. K., Lin, M., Zhuge, S., Yang, Y., Kim, D., Nguyen-Mau, T.-H., Tran, M., et al. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103224 · Cited by 56 · see citing works
Multi-Grained Radiology Report Generation With Sentence-Level Image-Language Contrastive Learning
Liu, A., Guo, Y., Yong, J., & Xu, F. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3372638 · Cited by 55 · see citing works
Multi-task paired masking with alignment modeling for medical vision-language pre-training
Zhang, K., Yang, Y., Yu, J., Jiang, H., Fan, J., Huang, Q., & Han, W. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3325965 · Cited by 51 · see citing works
Scaling-up medical vision-and-language representation learning with federated learning
Lu, S., Liu, Z., Liu, T., & Zhou, W. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107037 · Cited by 47 · see citing works
Advancing medical imaging with language models: featuring a spotlight on ChatGPT
Hu, M., Qian, J. Y., Pan, S., Li, Y., Qiu, R. L. J., & Yang, X. (2024) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ad387d · Cited by 74 · see citing works
Universal and extensible language-vision models for organ segmentation and tumor detection from abdominal computed tomography
Liu, J., Zhang, Y., Wang, K., Yavuz, M. C., Chen, X., Yuan, Y., Li, H., Yang, Y., Yuille, A., Tang, Y., et al. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103226 · Cited by 44 · see citing works
Token-Mixer: Bind Image and Text in One Embedding Space for Medical Image Reporting
Yang, Y. Q., Yu, J., Fu, Z., Zhang, K., Yu, T., Wang, X., Jiang, H., Lv, J., Huang, Q., & Han, W. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3412402 · Cited by 42 · see citing works
Continual Self-Supervised Learning: Towards Universal Multi-Modal Medical Data Representation Learning
Ye, Y., Xie, Y., Zhang, J., Chen, Z., Wu, Q., & Xia, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01057 · Cited by 40 · see citing works
Multi-modal large language models in radiology: principles, applications, and potential
Shen, Y., Xu, Y., Ma, J., Rui, W., Zhao, C., Heacock, L. C., & Huang, C. (2024) · Abdominal Radiology · doi:10.1007/s00261-024-04708-8 · Cited by 31 · see citing works
Pre-training in Medical Data: A Survey
Qiu, Y., Lin, F., Chen, W., & Xu, M. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1382-8 · Cited by 30 · see citing works
MDF-Net for abnormality detection by fusing X-rays with clinical data
Hsieh, C., Nobre, I. B., Sousa, S. C., Ouyang, C., Brereton, M. F., Nascimento, J. C., Jorge, J. A., & Moreira, C. (2023) · Scientific Reports · doi:10.1038/s41598-023-41463-0 · Cited by 28 · see citing works
Rethinking masked image modelling for medical image representation
Xie, Y., Gu, L., Harada, T., Zhang, J., Xia, Y., & Wu, Q. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103304 · Cited by 26 · see citing works
S3-Net: A Self-Supervised Dual-Stream Network for Radiology Report Generation
Pan, R., Ran, R., Hu, W., Zhang, W., Qin, Q., & Cui, S. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3345932 · Cited by 26 · see citing works
UniDCP: Unifying Multiple Medical Vision-Language Tasks via Dynamic Cross-Modal Learnable Prompts
Zhan, C., Zhang, Y., Lin, Y., Wang, G., & Wang, H. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3397191 · Cited by 25 · see citing works
Self-supervised multi-modal training from uncurated images and reports enables monitoring AI in radiology
Park, S. J., Lee, E. S., Shin, K. S., Lee, J. E., & Ye, J. C. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.103021 · Cited by 25 · see citing works
Recalibrated cross-modal alignment network for radiology report generation with weakly supervised contrastive learning
Hou, X., Li, X., Liu, Z., Sang, S., Lu, M., & Zhang, Y. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126394 · Cited by 20 · see citing works
Integrating language into medical visual recognition and reasoning: A survey
Lu, Y., & Wang, A. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103514 · Cited by 26 · see citing works
Spatio-Temporal and Retrieval-Augmented Modeling for Chest X-Ray Report Generation
Yang, Y. Q., You, X., Zhang, K., Fu, Z., Wang, X., Ding, J., Sun, J., Zhou, Y., Huang, Q., Han, W., et al. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3554498 · Cited by 22 · see citing works
Visual-language foundation models in medical imaging: A systematic review and meta-analysis of diagnostic and analytical applications
Sun, Y., Wen, X., Zhang, Y., Jin, L., Yang, C., Zhang, Q., Jiang, M. C., Xu, Z., Guo, W., Su, J., et al. (2025) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2025.108870 · Cited by 23 · see citing works
Rapid review: Growing usage of Multimodal Large Language Models in healthcare
Gupta, P., Zhang, Z., Song, M., Michalowski, M., Hu, X., Štiglic, G., & Topaz, M. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104875 · Cited by 23 · see citing works
Visual–language foundation models in medicine
Liu, C.-Y., Jin, Y.-X., Guan, Z., Li, T.-Y., Qin, Y.-M., Qian, B., Jiang, Z.-H., Wu, Y.-L., Wang, X.-N., Zheng, Y., et al. (2024) · The Visual Computer · doi:10.1007/s00371-024-03579-w · Cited by 36 · see citing works
CoMT: Chain-of-Medical-Thought Reduces Hallucination in Medical Report Generation
Jiang, Y., Chen, J., Yang, D., Li, M., Wang, S., Wu, T., Li, K., & Zhang, L. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887699 · Cited by 21 · see citing works
Adapter-Enhanced Hierarchical Cross-Modal Pre-Training for Lightweight Medical Report Generation
Yu, T., Lu, W., Yang, Y. Q., Han, W., Huang, Q., Yu, J., & Zhang, K. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3535699 · Cited by 18 · see citing works
Bridging multi-level gaps: Bidirectional reciprocal cycle framework for text-guided label-efficient segmentation in echocardiography
Zhang, Z., Zhang, H., Zeng, T., Yang, G., Shi, Z., & Gao, Z. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103536 · Cited by 17 · see citing works
Improving Medical Speech-to-Text Accuracy using Vision-Language Pre-training Models
Huh, J., Park, S. J., Lee, J. E., & Ye, J. C. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3345897 · Cited by 21 · see citing works
LLM-Driven Medical Report Generation via Communication-Efficient Heterogeneous Federated Learning
Che, H., Jin, H., Guo, Z., Lin, Y., Jin, C., & Chen, H. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3591185 · Cited by 16 · see citing works
SIA-Net: Sparse Interactive Attention Network for Multimodal Emotion Recognition
Li, S., Zhang, T., & Chen, C. L. P. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3409715 · Cited by 14 · see citing works
Alifuse: Aligning and Fusing Multimodal Medical Data for Computer-Aided Diagnosis
Chen, Q., & Yi, H. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10821851 · Cited by 15 · see citing works
CXR-LT 2024: A MICCAI challenge on long-tailed, multi-label, and zero-shot disease classification from chest X-ray
Lin, M., Holste, G., Wang, S., Zhou, Y., Wei, Y., Banerjee, I., Chen, P., Dai, T., Du, Y., Dvornek, N. C., et al. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103739 · Cited by 12 · see citing works
Self-Supervised Forecasting in Electronic Health Records With Attention-Free Models
Kumar, Y., Ilin, A., Salo, H., Kulathinal, S., Leinonen, M. K., & Marttinen, P. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3353164 · Cited by 11 · see citing works
Advancing Clinical Text Summarization through Extractive Methods using BERT-Based Models on the NBME Dataset
Sudarshan, R., Sasikala, D., & Kalavathi, S. (2023) · International Conference on Automation, Computing and Renewable Systems (ICACRS) · doi:10.1109/icacrs58579.2023.10404906 · Cited by 9 · see citing works
Framework for automation of short answer grading based on domain-specific pre-training
Bonthu, S., Sree, S. R., & Prasad, M. H. M. K. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109163 · Cited by 11 · see citing works
HKRG: Hierarchical knowledge integration for radiology report generation
Wang, B., Teng, P., Zhang, H., Yang, F., Wang, Z., Yi, X., Zhang, T., Wang, C., Tavares, A., & Xu, H. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126622 · Cited by 9 · see citing works
PairAug: What Can Augmented Image-Text Pairs Do for Radiology?
Xie, Y., Chen, Q., Wang, S., To, M., Lee, I., Khoo, E. W., Hendy, K., Koh, D., Xia, Y., & Wu, Q. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01107 · Cited by 9 · see citing works
Automatic radiology report generation with deep learning: a comprehensive review of methods and advances
Li, Y., Kong, C., Zhao, G., & Zhao, Z. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11337-0 · Cited by 7 · see citing works
Artificial Intelligence in Pathology: Advancing Large Models for Scalable Applications
Xiao, Z., Feng, B., Yang, J., Sun, G., Shen, Y., Xu, S., Yang, L., Xu, H., Zhang, M., & Wang, S. (2025) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-103123-095814 · Cited by 9 · see citing works
Clothing Purification with Causality Meets Vision-Language Pretraining Models
Yang, Z., Zhu, H., Lei, N., Fernando, B., & Wang, Z. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02548-7 · Cited by 7 · see citing works
Reason like a radiologist: Chain-of-thought and reinforcement learning for verifiable report generation
Jing, P., Lee, K., Zhang, Z., Zhou, H., Yuan, Z., Gao, Z., Zhu, L., Papanastasiou, G., Fang, Y., & Yang, G. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103910 · Cited by 11 · see citing works
Knowledge fusion in deep learning-based medical vision-language models: A review
Xu, D., Chen, Y., Chai, Z., Xiao, Y., Yan, Y., Ding, W., Wang, H., Jin, Z. J., Jiao, W., Yue, W., et al. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103455 · Cited by 6 · see citing works
MedScreenDental: Automated structured dental record generation via multimodal language model integration
Jin, W., Sun, Y., Ji, K., Jiang, X., Hu, Y., Wang, J., & Liu, J. (2025) · Displays · doi:10.1016/j.displa.2025.103119 · Cited by 5 · see citing works
SMART: Self-Weighted Multimodal Fusion for Diagnostics of Neurodegenerative Disorders
Chen, Q., & Hong, Y. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681140 · Cited by 5 · see citing works
Dual Graph Convolutional Networks for Chest X-ray Report Generation
Liu, Y., ZHANG, G., Chun, M. A., & Gu, Z. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.10.170 · Cited by 4 · see citing works
R2GenCSR: Mining Contextual and Residual Information for LLMs-Based Radiology Report Generation
Wang, X., Li, Y., Wang, F., Wang, S., Li, C., & Jiang, B. (2026) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2026.3669539 · Cited by 15 · see citing works
Knowledge Distillation and Transformer-Based Framework for Automatic Spine CT Report Generation
Batool, H., Mukhtar, A., Khawaja, S. G., Alghamdi, N. S., Khan, A. M., Qayyum, A., Adil, R., Khan, Z., Akram, M. U., Akbar, M. U., et al. (2025) · IEEE Access · doi:10.1109/access.2025.3546131 · Cited by 4 · see citing works
Possibilities and Pitfalls of Generative Pre-Trained Transformers in Healthcare
Kumar, T., Kait, R., Ankita, & Rani, S. (2023) · doi:10.1109/icacctech61146.2023.00016 · Cited by 4 · see citing works
KU-DMIS-MSRA at RadSum23: Pre-trained Vision-Language Model for Radiology Report Summarization
Kim, G., Kim, H., Ji, L., Bae, S., Kim, C., Sung, M., Kim, H., Yan, K., Chang, E., & Kang, J. (2023) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2023.bionlp-1.59 · Cited by 3 · see citing works
Artificial intelligence for abdominopelvic trauma imaging: trends, gaps, and future directions
Dreizin, D., Cheng, C., Liao, C., Jindal, A., & Colak, E. (2025) · Abdominal Radiology · doi:10.1007/s00261-025-04816-z · Cited by 2 · see citing works
Knowledge-Grounded Medical Dialogue Generation
Kambhamettu, H., Huang, Y., Johnson, K. B., & Bradbury, A. R. (2024) · Studies in computational intelligence · doi:10.1007/978-3-031-63592-2_16 · Cited by 2 · see citing works
K-STAMM: a knowledge-enhanced spatial - temporal attention model with multimodal fusion for pneumonia prediction
Anbukkarasi, S., Hemalatha, S., Balakrishnan, A., Varadhaganapathy, S., & Easwaramoorthy, S. V. (2026) · Scientific reports · doi:10.1038/s41598-026-47146-w · Cited by 1 · see citing works
“With the use of vision language models, it is possible to perform joint embedding of radiographs and clinical notes, thus enabling them to learn the features of different modes effectively 48 .”
ExpertODE: Continuous Diagnosis Prediction with Expert Enhanced Neural Ordinary Differential Equations
Zhang, H., Lv, H., Tan, Y., Ma, G., Wang, F., & Yang, C. (2024) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme57554.2024.10687504 · Cited by 1 · see citing works
Machine Learning for Personalized Prediction of Electrocardiogram (EKG) Use in Emergency Care
Wang, H., & Zhang, X. (2025) · Journal of Personalized Medicine · doi:10.3390/jpm15080358 · Cited by 1 · see citing works
Metapath-Enhanced Language Model Pretraining on Text-Attributed Heterogeneous Graphs
Chen, S., Fang, Q., Qian, S., & Xu, C. (2025) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3763241 · Cited by 1 · see citing works
Comparative Evaluation of Deep Learning and Foundation Model Embeddings for Osteoarthritis Feature Classification in Knee Radiographs
Chavoshi, M., Trivedi, H., Newsome, J. M., Mansuri, A., Li, F., Dapamede, T., Khosravi, B., & Gichoya, J. W. (2025) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-025-01636-x · Cited by 1 · see citing works
ECG Report Generation with Diagnostic Knowledge Enhanced Prompt Learning
Fan, P., Yue, X., Chen, Y., Wei, Z., Shi, J., & Xu, Z. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05479-1_2 · Cited by 1 · see citing works
Observe, align, and enhance: a hierarchical retrieval-augmented vision-language model for generating radiology reports
Chen, K., Zhu, X., Zhang, W., Jiang, S., & Chen, Q. (2025) · Health Information Science and Systems · doi:10.1007/s13755-025-00389-9 · Cited by 2 · see citing works
CyclicAligner: Knowledge-Enhanced Cyclical Alignment for Chest X-Ray Report Generation
Sun, J., Wang, P., Zhang, K., Tan, X., Fu, Z., & Yang, Y. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11356622 · Cited by 1 · see citing works
Integrating Clinical Knowledge into Radiology Report Generation to Enhance Diagnosis
Cao, Z., Yue, X., Chen, Y., Xu, Z., Wei, Z., & Li, Z. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11355985 · Cited by 1 · see citing works
Multimodal AI in healthcare: Review of vision–language foundation models for real-world medical applications
Razzaq, T., Taj, M., & Iqbal, A. (2026) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2026.105075 · Cited by 1 · see citing works
Enhancing Medical Report Generation in Multi-Slice Fusion Scenarios
Bu, S., Li, T., & Dai, Z. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385537 · Cited by 1 · see citing works
MambaXray-CTL: Multi-Stage Contrastive Training for Medical Report Generation with a Mamba-Based Multi-Modal Large Model
Feng, W., Lu, Y., Li, X., Shi, S., & Qi, Y. (2025) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc58881.2025.11343574 · Cited by 1 · see citing works
SANSformers: Self-Supervised Forecasting in Electronic Health Records with Attention-Free ModelsPreprint
Kumar, Y., Ilin, A., Salo, H., Kulathinal, S., Leinonen, M. K., & Marttinen, P. (2023) · doi:10.36227/techrxiv.23911365.v1
SANSformers: Self-Supervised Forecasting in Electronic Health Records with Attention-Free ModelsPreprint
Kumar, Y., Ilin, A., Salo, H., Kulathinal, S., Leinonen, M. K., & Marttinen, P. (2023) · doi:10.36227/techrxiv.23911365
Enhancing Medical Image Report Generation Through Regional and Global Feature Interaction
Teng, P., Yang, H., & Zhang, Q. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-1907-8_4
Look, Imitate and Refine: A Hierarchical Multimodel Retrieval Augmented Vision-Language Model for Radiology Report Generation
Chen, K., Zhu, X., Zhang, W., Jiang, S., & Chen, Q. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-3755-3_10
UniMRG: Refining Medical Semantic Understanding Across Modalities via LLM-Orchestrated Synergistic Evolution
Xu, H., Sowmya, A., Katz, I. A., & Wang, D. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04971-1_60
Retrieval from Dynamic Phrases: Generating Radiograph Reports with Phrase-Level Template and Dynamic Memory Bank
Chen, H., Yan, B., Shen, H., & Pei, M. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228627
Self-citation
Focusing on Abnormal: Visual Contrastive Classification and Semantic Enhancement for Medical Report Generation
Chen, H., Pei, M., & Nie, Z. (2025) · Communications in computer and information science · doi:10.1007/978-981-95-4100-3_10
Self-citation
Synthetic Chest X-Ray Augmentation via Generative Variational Autoencoding for Pneumonia Detection
Hartomo, K. D., Arthur, C., Handoko, A. B., & Chiong, R. (2025) · International Conference on Image and Vision Computing New Zealand (IVCNZ) · doi:10.1109/ivcnz67716.2025.11281874
Clinical-BERT: Vision-Language Pre-Training for Radiograph Diagnosis and Reports Generation
Association for Artificial Intelligence 2022, Pei, M., & Yan, B. (2022) · Underline Science Inc. · doi:10.48448/320c-z592
Self-citation
Align Modalities: Advancing Medical Report Generation with Unified Encoder and Inter-Case Contrastive Learning
Chen, H., Pei, M., & Nie, Z. (2025) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc58881.2025.11342904
Self-citation
Severity-Aware Radiology Report Generation: Knowledge Graph Expansion and Momentum-Guided Classification Enhancement
Yu, C., You, J., Huang, S., Yan, Z., Xu, W., & Lin, Z. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11356157
Knowledge-Driven Query Network with Adaptive Cross-View Attention for Structured Radiology Report Generation
Hou, X., Li, Y., & Wang, S. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00133
Fog/Edge-Aware State Space Models for Multi-Task Chest X-Ray Report Generation and Lesion Detection
Feng, W., Lü, Y., Li, X., Yung, K. L., & Ip, W. H. (2026) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2026.3667969
Knowledge enhancement with cross-modal fusion network for radiological report generation
LIANG, Z., Hou, X., Song, X., & Zhang, Y. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02291-0
Chest X-Ray Report Automation with Vision Transformer and GPT: A Case Study on the IU-Xray Dataset and Web-Based Solution
Laila, A., Islam, M. D., Haider, M. S., Habib, S. M. M., & Prottoy, A. A. (2025) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit68739.2025.11491591
Category-Guided Vision-Language Model for IoMT-Enabled Medical Report Generation Under Heterogeneous Imaging Data
Liu, X., Jin, Q., Zhang, T.-T., Tang, Z., Chen, Y., & Liu, Z. H. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3699691
Transformer-based deep learning models for disease prediction: A systematic review
Mwogosi, A. (2026) · Intelligence-Based Medicine · doi:10.1016/j.ibmed.2026.100416
MedXplainQA: Context-Aware Chest X-Ray Reporting with Question Answering Support
Venugopalan, G., Lavanya, P., V, Viju, N., Surendran, S., & Thomas, S. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.05.168
REVA-PO: Stabilizing Reinforcement Learning for Chest X-Ray Report Generation
Guo, L., Tahir, A. M., & Wang, Z. J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37321-2_31
An innovative multimodal AKI prediction with integrated model based on Pre-trained BERT and TFT network
Zeng, H., Xu, R., Lu, A., Yu, R., Zhang, A., Fang, H., & Li, T. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-01189-1
MPDRL: difficulty-aware representation learning with medical priors for vision–language pre-training
Liu, J., Tan, Z., Chen, W., & Liu, X. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1906686
Prototype-Guided Visual Recalibration for LLM-based Medical Report GenerationPreprint
Li, L., Shi, Y., An, K., Zhu, X., Yun, X., & Liu, J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7584765
MeMix: Multi-Encoder Mixture Framework for Medical Report Generation
Cao, Y., Cui, L., & Shen, Z. (2026) · doi:10.1109/cvprf71909.2026.00526
Clinical-context guidance and hyperbolic prototype learning for long-tailed radiology report generation
Yang, F., Li, G., He, Z., Wu, R., Luo, G., Yue, Y., Ji, D., Wu, J., & Zhang, H. (2026) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2026.102834

MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training in RadiologyPreprint
Wu, C., Zhang, X., Zhang, Y., Wang, Y., & Xie, W. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.02228 · Cited by 6 · see citing works
MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training for X-ray Diagnosis
Wu, C., Zhang, X., Zhang, Y., Wang, Y., & Xie, W. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01954 · Cited by 122 · see citing works
Multi-Modal Understanding and Generation for Medical Images and Text via Vision-Language Pre-Training
Moon, J. H., Lee, H., Shin, W., Kim, Y., & Choi, E. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3207502 · Cited by 221 · see citing works
MedKLIP: Medical Knowledge Enhanced Language-Image Pre-TrainingPreprint
Wu, C., Zhang, X., Zhang, Y., Wang, Y., & Xie, W. (2023) · medRxiv · doi:10.1101/2023.01.10.23284412 · Cited by 80 · see citing works
Knowledge-enhanced visual-language pre-training on chest radiology images
Zhang, X., Wu, C., Zhang, Y., Xie, W., & Wang, Y. (2023) · Nature Communications · doi:10.1038/s41467-023-40260-7 · Cited by 207 · see citing works

Authors: Yan, B., & Pei, M.
Affiliations: Beijing Institute of Technology (CN)
ORCID: Yan, Bin
Pei, Mingtao
Venue: Proceedings of the AAAI Conference on Artificial Intelligence
Publisher: Association for the Advancement of Artificial Intelligence (AAAI)
Published: 2022-06-28
Volume: 36
Issue: 3
Pages: 2982-2990
ISSN: 2159-5399 (ISSN-L)
2374-3468
DOI: 10.1609/aaai.v36i3.20204
OpenAlex: W4283821415
Cited by: 94
References: 53
Citations by year: 2026: 14 · 2025: 39 · 2024: 23 · 2023: 17 · 2022: 1
FWCI: 7.25
Type: conference paper
Language: en
Open access: diamond
doi.org
Locations: Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
Keywords: vision-language pre-training, radiology report generation, Medical Subject Headings, masked language modeling, chest X-ray classification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Clinical-BERT: Vision-Language Pre-training for Radiograph Diagnosis and Reports Generation

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Self-check

Cites

Cited by

Related

Record

About these checks