Knowledge-enhanced visual-language pre-training on chest radiology images
Open accessAbstract
While multi-modal foundation models pre-trained on large-scale data have been successful in natural language understanding and vision recognition, their use in medical domains is still limited due to the fine-grained nature of medical tasks and the high demand for domain knowledge. To address this challenge, we propose an approach called Knowledge-enhanced Auto Diagnosis (KAD) which leverages existing medical domain knowledge to guide vision-language pre-training using paired chest X-rays and radiology reports. We evaluate KAD on four external X-ray datasets and demonstrate that its zero-shot performance is not only comparable to that of fully supervised models but also superior to the average of three expert radiologists for three (out of five) pathologies with statistical significance. Moreover, when few-shot annotation is available, KAD outperforms all existing approaches in fine-tuning settings, demonstrating its potential for application in different clinical scenarios.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
By leveraging encoders pre-trained on large unimodal datasets and occasionally fine-tuning them on available multimodal data, researchers can achieve robust performance despite the relative scarcity of multimodal data.
“By leveraging encoders pre-trained on large unimodal datasets and occasionally fine-tuning them on the available multimodal data, researchers can achieve robust performance despite the relative scarcity of multimodal data (Wang et al., 2023a; Zhang et al., 2023).”
From Navigating the landscape of multimodal AI in medicine: A scoping review on technical challenges and clinical applications (2025) · cites this paper as Zhang et al., 2023
Methods (Implementation details; Algorithm overview, page not available) describe initializing a text encoder from PubMedBERT and then training a visual-language model on paired chest X-rays and reports. The Discussion reports strong performance with limited labeled data for downstream fine-tuning. These passages do not establish that both encoders were pre-trained on large unimodal datasets or that the reported performance was achieved despite scarce multimodal data; Methods (Datasets, page not available) instead describes a substantial paired image–report pre-training dataset. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Self-citationVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 21,793 · see citing works
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · see citing works
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., & Summers, R. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.369 · Cited by 4,616 · see citing works
The unified medical language system (UMLS): integrating biomedical terminology
Bodenreider, O. (2004) · Nucleic Acids Research · doi:10.1093/nar/gkh061 · Cited by 4,356 · see citing works
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
Self-citationCheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep LearningPreprint
Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D. Y., Bagul, A., Langlotz, C. P., Shpanskaya, K. S., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.05225 · Cited by 2,283 · see citing works
Domain-specific language model pretraining for biomedical natural language processing
Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J., & Poon, H. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3458754 · Cited by 2,170 · see citing works
MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports
Johnson, A. E. W., Pollard, T. J., Berkowitz, S. J., Greenbaum, N. R., Lungren, M. P., Deng, C.-Y., Mark, R. G., & Horng, S. (2019) · Scientific Data · doi:10.1038/s41597-019-0322-0 · Cited by 1,816 · see citing works
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555 · see citing works
ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing
Neumann, M., King, D., Beltagy, I., & Ammar, W. (2019) · BioNLP Workshop and Shared Task (BioNLP) · doi:10.18653/v1/w19-5034 · Cited by 772 · see citing works
Top-Down Neural Attention by Excitation Backprop
Zhang, J., Bargal, S. A., Lin, Z., Brandt, J., Shen, X., & Sclaroff, S. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-017-1059-x · Cited by 769 · see citing works
PadChest: A large chest x-ray image dataset with multi-label annotated reports
Bustos, A., Pertusa, A., Salinas, J.-M., & de la Iglesia-Vayá, M. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101797 · Cited by 691 · see citing works
SNOMED-CT: The advanced terminology and coding system for eHealth.
Donnelly, K. (2006) · PubMed · Cited by 541 · see citing works
Huang, S.-C., Shen, L., Lungren, M. P., & Yeung, S. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00391 · Cited by 489 · see citing works
Dale, R. (2020) · Natural Language Engineering · doi:10.1017/s1351324920000601 · Cited by 461 · see citing works
Tiu, E., Talius, E., Patel, P., Langlotz, C. P., Ng, A. Y., & Rajpurkar, P. (2022) · Nature Biomedical Engineering · doi:10.1038/s41551-022-00936-9 · Cited by 442 · see citing works
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301590 · Cited by 392 · see citing works
Zhou, Z., Sodha, V., Pang, J., Gotway, M. B., & Liang, J. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101840 · Cited by 353 · see citing works
Making the Most of Text Semantics to Improve Biomedical Vision-Language Processing
Boecking, B., Usuyama, N., Bannur, S., Castro, D. C., Schwaighofer, A., Hyland, S., Wetscherek, M., Naumann, T., Nori, A., Alvarez-Valle, J., et al. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20059-5_1 · Cited by 317 · cited 11× · see citing works
“Specifically, ConVIRT [64] simply uses chest X-rays and the associated raw report for self-supervised training; GLoRIA [25] learns global and localized representations by contrasting attention-weighted image sub-regions and words in the reports; BioVIL [6] introduces a pre-trained radiology-specific text encoder CXR-B…”
BackgroundMethodInfluentialHow Much Can CLIP Benefit Vision-and-Language Tasks?Preprint
Shen, S., Li, L. H., Tan, H. X., Bansal, M., Rohrbach, A., Chang, K.-W., Yao, Z., & Keutzer, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.06383 · Cited by 272 · see citing works
Zhou, H.-Y., Chen, X., Zhang, Y., Luo, R., Wang, L., & Yu, Y. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-021-00425-9 · Cited by 126 · see citing works
Self-citationZhou, H.-Y., Yu, S., Bian, C., Hu, Y., Ma, K., & Zheng, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59710-8_39 · Cited by 100 · see citing works
Align, Reason and Learn: Enhancing Medical Vision-and-Language Pre-training with Knowledge
Chen, Z., Li, G., & Wan, X. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3547948 · Cited by 86 · see citing works
MedKLIP: Medical Knowledge Enhanced Language-Image Pre-TrainingPreprint
Wu, C., Zhang, X., Zhang, Y., Wang, Y., & Xie, W. (2023) · medRxiv · doi:10.1101/2023.01.10.23284412 · Cited by 80 · see citing works
Self-citationRadGraph: Extracting Clinical Entities and Relations from Radiology ReportsPreprint
Jain, S., Agrawal, A., Saporta, A., Truong, S. Q., Nguyen Duong, D., Bui, T., Chambon, P., Lungren, M. P., Ng, A. Y., Langlotz, C. P., et al. (2021) · arXiv (Cornell University) · doi:10.13026/2f5n-pb92 · Cited by 36 · see citing works
RadGraph: Extracting Clinical Entities and Relations from Radiology Reports
Jain, S., Agrawal, A., Saporta, A., Truong, S. Q., Nguyen Duong, D., Bui, T., Chambon, P., Lungren, M., Ng, A., Langlotz, C., et al. (2021) · PhysioNet · doi:10.13026/hm87-5p47 · Cited by 34 · see citing works
Rajpurkar, P., Joshi, A., Pareek, A., Ng, A. Y., & Lungren, M. P. (2021) · ACM Conference on Health, Inference, and Learning (CHIL) · doi:10.1145/3450439.3451876 · Cited by 18 · see citing works
Joint Learning of Localized Representations from Medical Images and Reports
Müller, P., Kaissis, G. A., Zou, C., & Rückert, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19809-0_39 · Cited by 16 · see citing works
ChestX-Det10: Chest X-ray Dataset on Detection of Thoracic AbnormalitiesPreprint
Liu, J., Lian, J., & Yu, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.10550 · Cited by 14 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- [2]Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 4171–4186 (ACL, 2019).
- [5]Ma, C., Yang, Y., Wang, Y., Zhang, Y. & Xie, W. Open-vocabulary semantic segmentation with frozen vision-language models. British Machine Vision Conference (2022).
- [24]Rajpurkar, P., Joshi, A., Pareek, A., Ng, A. Y. & Lungren, M. P. Chexternal: generalization of deep learning models for chest x-ray interpretation to photos of chest x-rays and external clinical settings. In Proc. Conference on Health, Inference, and Learning, 125–132 (2021).
- [34]Loshchilov, I. & Hutter, F. Decoupled weight decay regularization. In Proc. International Conference on Learning Representations (2019).
Cited by
Sorted by citations
A generalist medical language model for disease diagnosis assistance
Liu, X., Liu, H., Yang, G., Jiang, Z., Cui, S., Zhang, Z., Wang, H., Tao, L., Sun, Y., Song, Z., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03416-6 · Cited by 183 · see citing works
Towards generalist foundation model for radiology by leveraging web-scale 2D&3D medical data
Wu, C., Zhang, X., Zhang, Y., Hui, H., Wang, Y., & Xie, W. (2025) · Nature Communications · doi:10.1038/s41467-025-62385-7 · Cited by 273 · see citing works
Self-citationHe, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., & Cambria, E. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102963 · Cited by 147 · see citing works
Large language models for disease diagnosis: a scoping review
Zhou, S., Xu, Z., Zhang, M., Xu, C., Guo, Y., Zhan, Z., Fang, Y., Ding, S., Wang, J., Xu, K., et al. (2025) · npj Artificial Intelligence · doi:10.1038/s44387-025-00011-z · Cited by 143 · see citing works
CLIP in medical imaging: A survey
Zhao, Z., Liu, Y., Wu, H., Wang, M., Li, Y., Wang, S., Teng, L., Liu, D., Cui, Z., Wang, Q., et al. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103551 · Cited by 175 · see citing works
Foundation Model for Advancing Healthcare: Challenges, Opportunities and Future Directions
He, Y., Huang, F., Jiang, X., Nie, Y., Wang, M., Wang, J., & Chen, H. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2024.3496744 · Cited by 130 · see citing works
Schouten, D., Nicoletti, G., Dille, B., Chia, C., Vendittelli, P., Schuurmans, M., Litjens, G., & Khalili, N. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103621 · Cited by 108 · see citing works
Has multimodal learning delivered universal intelligence in healthcare? A comprehensive survey
Lin, Q., Zhu, Y., Mei, X., Huang, L., Ma, J., He, K., Peng, Z., Cambria, E., & Feng, M. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102795 · Cited by 76 · see citing works
He, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., & Cambria, E. (2024) · doi:10.2139/ssrn.4809363 · Cited by 69 · see citing works
Demographic bias of expert-level vision-language foundation models in medical imaging
Yang, Y., Liu, Y., Liu, X., Gulhane, A., Mastrodicasa, D., Wu, W., Wang, E. J., Sahani, D. V., & Patel, S. (2025) · Science Advances · doi:10.1126/sciadv.adq0305 · Cited by 67 · see citing works
Huang, W., Li, C., Zhou, H.-Y., Yang, H., Liu, J., Liang, Y., Zheng, H., Zhang, S., & Wang, S. (2024) · Nature Communications · doi:10.1038/s41467-024-51749-0 · Cited by 62 · see citing works
Buess, L., Keicher, M., Navab, N., Maier, A., & Arasteh, S. T. (2025) · Biomedical Engineering Letters · doi:10.1007/s13534-025-00497-1 · Cited by 71 · see citing works
Large-vocabulary segmentation for medical images with text prompts
Zhao, Z., Zhang, Y., Wu, C., Zhang, X., Zhou, X., Zhang, Y., Wang, Y., & Xie, W. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01964-w · Cited by 81 · see citing works
Self-citationHolste, G., Zhou, Y., Wang, S., Jaiswal, A. K., Lin, M., Zhuge, S., Yang, Y., Kim, D., Nguyen-Mau, T.-H., Tran, M., et al. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103224 · Cited by 56 · see citing works
Sun, K., Xue, S., Sun, F., Sun, H., Luo, Y., Wang, L., Wang, S., Guo, N., Liu, L., Zhao, T., et al. (2025) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2025.103265 · Cited by 61 · see citing works
BrainMass: Advancing Brain Network Analysis for Diagnosis With Large-Scale Self-Supervised Learning
Yang, Y., Ye, C., Su, G., Zhang, Z., Chang, Z., Chen, H., Chan, P., Yu, Y., & Ma, T. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3414476 · Cited by 45 · see citing works
A fully open AI foundation model applied to chest radiography
Ma, D., Pang, J., Gotway, M. B., & Liang, J. (2025) · Nature · doi:10.1038/s41586-025-09079-8 · Cited by 48 · see citing works
Wang, X., Yue, Q., & Liu, X. (2025) · Automation in Construction · doi:10.1016/j.autcon.2025.105995 · Cited by 41 · see citing works
Enhancing Medical Image Reclamation for Chest Samples Using B-Coefficients, DT-CWT and EPS Algorithm
Kumar, B. P. P., Rangaiah, P. K. B., & Augustine, R. (2023) · IEEE Access · doi:10.1109/access.2023.3322205 · Cited by 40 · see citing works
Ye, Y., Xie, Y., Zhang, J., Chen, Z., Wu, Q., & Xia, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01057 · Cited by 40 · see citing works
A data-efficient strategy for building high-performing medical foundation models
Sun, Y., Tan, W., Gu, Z., He, R., Chen, S., Pang, M., & Yan, B. (2025) · Nature Biomedical Engineering · doi:10.1038/s41551-025-01365-0 · Cited by 42 · see citing works
Foundation model of ECG diagnosis: Diagnostics and explanations of any form and rhythm on ECG
Tian, Y., Li, Z., Jin, Y., Wang, M., Wei, X., Zhao, L., Liu, Y., Liu, J., & Liu, C. (2024) · Cell Reports Medicine · doi:10.1016/j.xcrm.2024.101875 · Cited by 41 · see citing works
Graph Artificial Intelligence in Medicine
Johnson, R., Li, M. M., Noori, A., Queen, O., & Žitnik, M. (2024) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-110723-024625 · Cited by 40 · see citing works
CARZero: Cross-Attention Alignment for Radiology Zero-Shot Classification
Lai, H., Yao, Q., Jiang, Z., Wang, R., He, Z., Tao, X., & Zhou, S. K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01059 · Cited by 33 · see citing works
Empowering Generalist Material Intelligence with Large Language Models
Yuan, W., Chen, G., Wang, Z., & You, F. (2025) · Advanced Materials · doi:10.1002/adma.202502771 · Cited by 34 · see citing works
Wu, J., He, K., Mao, R., Shang, X., & Cambria, E. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103283 · Cited by 28 · see citing works
Uncertainty-aware large language models for explainable disease diagnosis
Zhou, S., Wang, J., Xu, Z., Wang, S., Brauer, D., Welton, L., Cogan, J., Chung, Y.-H., Tian, L., Zhan, Z., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02071-6 · Cited by 27 · see citing works
Rajendran, P., Yang, Y., Niedermayr, T. R., Gensheimer, M. F., Beadle, B. M., Le, Q., Xing, L., & Dai, X. (2024) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2025.110740 · Cited by 27 · see citing works
IMITATE: Clinical Prior Guided Hierarchical Vision-Language Pre-Training
Liu, C., Cheng, S. L., Shi, M., Shah, A., Bai, W., & Arcucci, R. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3449690 · Cited by 30 · see citing works
Cross- and Intra-Image Prototypical Learning for Multi-Label Disease Diagnosis and Interpretation
Wang, C., Liu, F., Chen, Y., Frazer, H. M. L., & Carneiro, G. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3541830 · Cited by 24 · see citing works
UniDCP: Unifying Multiple Medical Vision-Language Tasks via Dynamic Cross-Modal Learnable Prompts
Zhan, C., Zhang, Y., Lin, Y., Wang, G., & Wang, H. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3397191 · Cited by 25 · see citing works
Self-citationLarge-scale long-tailed disease diagnosis on radiology images
Zheng, Q., Zhao, W., Wu, C., Zhang, X., Dai, L., Guan, H., Li, Y., Zhang, Y., Wang, Y., & Xie, W. (2024) · Nature Communications · doi:10.1038/s41467-024-54424-6 · Cited by 23 · see citing works
Self-citationUniChest: Conquer-and-Divide Pre-Training for Multi-Source Chest X-Ray Classification
Dai, T., Zhang, R., Hong, F., Yao, J., Zhang, Y., & Wang, Y. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3381123 · Cited by 23 · see citing works
Self-citationA concept-based interpretable model for the diagnosis of choroid neoplasias using multimodal data
Wu, Y., Liu, Y., Yang, Y., Yao, M. S., Yang, W., Shi, X., Yang, L., Li, D., Liu, Y., Yin, S., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-58801-7 · Cited by 21 · see citing works
Concept-Based Lesion Aware Transformer for Interpretable Retinal Disease Diagnosis
Wen, C., Ye, M., He, L., Chen, T., & Xiao, X. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3429148 · Cited by 20 · see citing works
Integrating language into medical visual recognition and reasoning: A survey
Lu, Y., & Wang, A. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103514 · Cited by 26 · see citing works
Sun, Y., Wen, X., Zhang, Y., Jin, L., Yang, C., Zhang, Q., Jiang, M. C., Xu, Z., Guo, W., Su, J., et al. (2025) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2025.108870 · Cited by 23 · see citing works
Self-citationA systematic review of multi-modal large language models on domain-specific applications
Li, S., Wong, K. W., Wang, G., & Duong, T.-T. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11398-1 · Cited by 21 · see citing works
Anatomical Structure-Guided Medical Vision-Language Pre-training
Li, Q., Yan, X., Xu, J., Yuan, R., Zhang, Y., Feng, R., Shen, Q., Zhang, X., & Wang, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72120-5_8 · Cited by 20 · see citing works
Self-citationChatGPT-4's Accuracy in Estimating Thyroid Nodule Features and Cancer Risk From Ultrasound Images
Cabezas, E., Toro-Tobón, D., Johnson, T., Álvarez, M., Azadi, J. R., Gonzalez-Velazquez, C. D., Ospina, N. S., Ponce, O. J., Branda, M. E., & Brito, J. P. (2025) · Endocrine Practice · doi:10.1016/j.eprac.2025.03.008 · Cited by 19 · see citing works
Visual–language foundation models in medicine
Liu, C.-Y., Jin, Y.-X., Guan, Z., Li, T.-Y., Qin, Y.-M., Qian, B., Jiang, Z.-H., Wu, Y.-L., Wang, X.-N., Zheng, Y., et al. (2024) · The Visual Computer · doi:10.1007/s00371-024-03579-w · Cited by 36 · see citing works
Gefter, W. B., Prokop, M., Seo, J. B., Raoof, S., Langlotz, C. P., & Hatabu, H. (2024) · Radiology · doi:10.1148/radiol.232778 · Cited by 18 · see citing works
Bootstrapping Chest CT Image Understanding by Distilling Knowledge from X-Ray Expert Models
Cao, W., Zhang, J., Xia, Y., Mok, T. C. W., Li, Z., Ye, X., Lü, L., Zheng, J., Tang, Y., & Zhang, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01068 · Cited by 18 · see citing works
UrFound: Towards Universal Retinal Foundation Models via Knowledge-Guided Masked Modeling
Yu, K., Zhou, Y., Bai, Y., Soh, Z. D., Xu, X., Goh, R. S. M., Cheng, C.-Y., & Liu, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72390-2_70 · Cited by 17 · see citing works
Adapter-Enhanced Hierarchical Cross-Modal Pre-Training for Lightweight Medical Report Generation
Yu, T., Lu, W., Yang, Y. Q., Han, W., Huang, Q., Yu, J., & Zhang, K. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3535699 · Cited by 18 · see citing works
Large Language Model-Informed ECG Dual Attention Network for Heart Failure Risk Prediction
Chen, C., Li, L., Beetz, M., Banerjee, A., Gupta, R., & Grau, V. (2025) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2025.3536922 · Cited by 17 · see citing works
Huang, W., Li, C., Yang, H., Liu, J., Liang, Y., Zheng, H., & Wang, S. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103299 · Cited by 16 · see citing works
Panayides, A. S., Chen, H., Filipović, N., Geroski, T., Hou, J., Lekadir, K., Kostas Marias, A., Matsopoulos, G. K., Papanastasiou, G., Sarder, P., et al. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3649496 · Cited by 13 · see citing works
Intelligent generation of 3D disassembly processes driven by multimodal large language models
Zhao, Y., Chen, L., Yu, H., & Zheng, Y. (2026) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2026.04.032 · Cited by 13 · see citing works
Knowledge-Enhanced Visual-Language Pretraining for Computational Pathology
Zhou, X., Zhang, X., Wu, C., Zhang, Y., Xie, W., & Wang, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72943-0_20 · Cited by 15 · see citing works
Self-citationPre-training on high-resolution X-ray images: an experimental study
Wang, X., Li, Y., Wu, W., Jin, J., Rong, Y., Jiang, B., Li, C., & Tang, J. (2025) · Visual Intelligence · doi:10.1007/s44267-025-00080-3 · Cited by 13 · see citing works
Silva-Rodríguez, J., Dolz, J., & Ayed, I. B. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103596 · Cited by 12 · see citing works
Yao, J., & Han, T. (2025) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2025.116527 · Cited by 10 · see citing works
ECAMP: Entity-centered Context-aware Medical Vision Language Pre-training
Wang, R., Yao, Q., Jiang, Z., Lai, H., He, Z., Tao, X., & Zhou, S. K. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103690 · Cited by 16 · see citing works
Enabling Multi-modal Conversational Interface for Clinical Imaging
Dayanandan, K., & Lall, B. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650805 · Cited by 9 · see citing works
Recent Advances in Medical Image Classification
Dao, L., & Ly, N. Q. (2024) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2024.0150727 · Cited by 12 · see citing works
Liu, K., Ma, Z., Kang, X., Zhong, Z., Jiao, Z., Baird, G. L., Bai, H. X., & Miao, Q. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72384-1_41 · Cited by 10 · see citing works
Cross-modal contrastive learning for unified placenta analysis using photographs
Pan, Y., Mehta, M., Goldstein, J. A., Ngonzi, J. N. J., Bebell, L. M., Roberts, D. J., Carreon, C. K., Gallagher, K., Walker, R. E., Gernand, A. D., et al. (2024) · Patterns · doi:10.1016/j.patter.2024.101097 · Cited by 9 · see citing works
HKRG: Hierarchical knowledge integration for radiology report generation
Wang, B., Teng, P., Zhang, H., Yang, F., Wang, Z., Yi, X., Zhang, T., Wang, C., Tavares, A., & Xu, H. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126622 · Cited by 9 · see citing works
Foundation Models Meet Medical Image Interpretation
Jiao, L., Hao, J., Li, R., Li, L., Liu, X., Liu, F., Ma, W., Chen, P., Huang, Z., Yang, J., et al. (2026) · Research · doi:10.34133/research.1024 · Cited by 9 · see citing works
Zheng, J., Chen, M., Yuan, H., Luo, J., Wang, Y., Nai, J., Tao, X., & Liu, Y. (2023) · Chinese Chemical Letters · doi:10.1016/j.cclet.2023.108812 · Cited by 8 · see citing works
Self-citationEnhancing Medical Vision-Language Contrastive Learning via Inter-Matching Relation Modeling
Li, M., Meng, M., FULHAM, M. J., Feng, D. D., Bi, L., & Kim, J. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3534436 · Cited by 9 · see citing works
Wang, J., Song, M., Fan, D.-P., Yang, J., Zhang, S., Yang, J., Liu, J., Wang, C., & Wang, B. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127173 · Cited by 8 · see citing works
Medical Knowledge Intervention Prompt Tuning for Medical Image Classification
Du, Y., Yu, N., & Wang, S. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3584841 · Cited by 9 · see citing works
Li, Y., Kong, C., Zhao, G., & Zhao, Z. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11337-0 · Cited by 7 · see citing works
Zou, L., Li, J., Chen, H., Liang, M., Jia, K., Zhong, Y., & Chen, J. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126660 · Cited by 7 · see citing works
Generalized domain prompt learning for accessible scientific vision-language models
Cao, Q., Chen, Y., Lu, L., Sun, H., Zeng, Z., Yang, X., & Zhang, D. (2025) · Nexus · doi:10.1016/j.ynexs.2025.100069 · Cited by 7 · see citing works
Wang, M., Li, Z., Tian, Y., Wei, X., Jin, Y., & Liu, C. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103302 · Cited by 7 · see citing works
Zhong, W., Sun, M.-Y., Yao, S., Liu, Y., Peng, D., Liu, Y., Yang, K., Gao, H., Yan, H., Hao, W., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/73233 · Cited by 7 · see citing works
Dai, L., Lei, J., Ma, F., Sun, Z., Du, H., Zhang, H., Jiang, J., Wei, J., Wang, D., Tan, G., et al. (2024) · Radiology Artificial Intelligence · doi:10.1148/ryai.230520 · Cited by 7 · see citing works
Self-citationRobust and explainable framework to address data scarcity in diagnostic imaging
Zhao, Z., Alzubaidi, L., Zhang, J., Duan, Y., Naseem, U., & Gu, Y. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.111052 · Cited by 10 · see citing works
HuBERT-ECG as a self-supervised foundation model for broad and scalable cardiac applicationsPreprint
Coppola, E., Savardi, M., Massussi, M., Adamo, M., Metra, M., & Signoroni, A. (2024) · medRxiv · doi:10.1101/2024.11.14.24317328 · Cited by 8 · see citing works
CLR2G: Cross modal Contrastive Learning on Radiology Report Generation
Xue, H., Ma, Q., Liu, G., Qu, J., Liu, Y., & Liu, A. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679668 · Cited by 8 · see citing works
Zhang, M., Xia, X., Wang, Q., Pan, Y., Zhang, G., & Wang, Z. (2025) · Environmental Health and Preventive Medicine · doi:10.1265/ehpm.24-00270 · Cited by 7 · see citing works
Wang, S., Zhu, Y., Yang, Z., Luo, X., Zhang, Y., Fu, P., Wang, H., Wang, M., Song, Z., Li, Q., et al. (2025) · Nature Biomedical Engineering · doi:10.1038/s41551-025-01500-x · Cited by 13 · see citing works
Self-citationA vision–language pretrained transformer for versatile clinical respiratory disease applications
Ma, L., Liang, H., He, Y., Wang, W., Yan, Z., Li, W., Wang, R., Li, Y., Lizhu, Y., Liu, Y., et al. (2025) · Nature Biomedical Engineering · doi:10.1038/s41551-025-01544-z · Cited by 7 · see citing works
Factual serialization enhancement: A key innovation for chest X-ray report generation
Liu, K., Ma, Z., Liu, M., Jiao, Z., Kang, X., Miao, Q., & Kun, X. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.132550 · Cited by 6 · see citing works
Interactive dual-stream contrastive learning for radiology report generation
Zhang, Z., & Jiang, A. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104718 · Cited by 6 · see citing works
Wang, Z. Y., Jiang, X., Gao, C., Dong, F., Dai, W., Wang, B., Yan, B., Chen, Q., Huang, W., Zhang, T., et al. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822152 · Cited by 6 · see citing works
T3D: Advancing 3D Medical Vision-Language Pre-Training by Learning Multi-View Visual Consistency
Liu, C., Ouyang, C., Chen, Y., Quilodrán-Casas, C., Ма, Л., Fu, J., Guo, Y., Shah, A., Bai, W., & Arcucci, R. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00698 · Cited by 12 · see citing works
A Review of Visual Grounding on Remote Sensing Images
Wang, Z., Liu, L., Wan, G., Zhang, W., Zhong, B., Chang, H., Li, X., Liu, X., & Sun, G. (2025) · Electronics · doi:10.3390/electronics14142815 · Cited by 5 · see citing works
Vision-language model-based semantic-guided imaging biomarker for lung nodule malignancy prediction
Zhuang, L., Tabatabaei, S. M. H., Salehi‐Rad, R., Tran, L. M., Aberle, D. R., Prosper, A. E., & Hsu, W. W. Y. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104947 · Cited by 10 · see citing works
Asif, S., Hadi, F., Ain, Q. U., Yan, Y., Wang, V. Y., & Xu, D. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100847 · Cited by 5 · see citing works
Unifying the Electron Microscopy Multiverse through a Large-scale Foundation ModelPreprint
He, L., Shi, R., Wang, W., Fang, G., Cai, Y., & Ма, Л. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.04.13.648639 · Cited by 5 · see citing works
Grounded knowledge-enhanced medical vision–language pre-training for chest X-ray
Deng, Q., Huang, Z., Wang, Y., Wang, Z., Wang, Z., Zhang, X., Dou, Q., Hui, Y. Y., & Hui, E. S. K. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.108416 · Cited by 5 · see citing works
Self-citationPromises and challenges of applying large language models in the healthcare domain
Wang, Q., Gong, Z., Lai, Z., Bu, L., Dahlweid, F.-M., & Sun, H. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1772274 · Cited by 4 · see citing works
Zhao, Z., Guo, S., Han, L., Zhou, G., & Jia, J. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.107742 · Cited by 4 · see citing works
Domain Adaptation and Generalization Using Foundation Models in Healthcare ImagingPreprint
Ying, H., Lia, Y., & Fu, Z. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5345726 · Cited by 5 · see citing works
Scaling Chest X-Ray Foundation Models From Mixed Supervisions for Dense Prediction
Wang, F., & Yu, L. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3589928 · Cited by 4 · see citing works
Zhong, D., Li, X., Zhiyong, H., Wang, S., Yu, Z. W., Hou, M., Yan, Y., & Liu, Y.-S. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.108318 · Cited by 3 · see citing works
Ma, D., Pang, J., Velan, S. S., Gotway, M. B., & Liang, J. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103828 · Cited by 5 · see citing works
EchoGraph system for automated quality assessment of echocardiography reports
Chao, C., Delbrouck, J.-B., Asadi, M., Banerjee, I., Farina, J., Galasso, F. I., Mahmoud, A., Abbas, M. T., Wang, Y.-C., Arsanjani, R., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02140-w · Cited by 2 · see citing works
Self-citationLearning with less supervision: A survey of label-efficient learning for medical image analysis
Jin, C., Guo, Z., Lin, Y., Luo, L., & Chen, H. (2026) · Medical Image Analysis · doi:10.1016/j.media.2026.104062 · Cited by 31 · see citing works
DxDirector: an agentic large language model driving the full-process clinical diagnosis
Xu, S., Huang, X., Wei, Z., Pang, L., Shen, H., & Cheng, X. (2026) · Nature Communications · doi:10.1038/s41467-026-71928-5 · Cited by 4 · see citing works
A foundation model unlocks unified biomedical image analysis
Huang, Y., Dou, H., & Ni, D. (2024) · Nature Methods · doi:10.1038/s41592-024-02519-9 · Cited by 3 · see citing works
Deep Disease Label-guided Graph Convolutional Network for Medical Report Generation
Xu, L., Wang, Y., He, C., Tang, Q., Zeng, X., & Lv, J. (2025) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3722226 · Cited by 3 · see citing works
Self-citationSuo, J., Dong, K., He, K., He, K., & Qionghai, D. (2025) · Nature Biomedical Engineering · doi:10.1038/s41551-025-01456-y · Cited by 3 · see citing works
MedKit: Multi-level feature distillation with knowledge injection for radiology report generation
Su, Z., Song, H., Lin, Y., Wu, Y., Weng, X., Mao, Z., Liu, B., Yin, H., & Yang, J. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129003 · Cited by 4 · see citing works
Tessler, I., Yamin, T., Peeri, H., Alon, E. E., Zimlichman, E., Glicksberg, B. S., & Klang, E. (2024) · Future Medicine AI · doi:10.2217/fmai-2024-0001 · Cited by 3 · see citing works
Report-Concept Textual-Prompt Learning for Enhancing X-ray Diagnosis
Zhao, X., Liu, Z., Лю, Ф., Li, G., Dou, Y., & Peng, S. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681568 · Cited by 3 · see citing works
- Authors
- Zhang, X., Wu, C., Zhang, Y., Xie, W., & Wang, Y.
- Affiliations
- Shanghai Jiao Tong University (CN)
- Corresponding author
- Xie, Weidi
- Wang, Yanfeng
- Venue
- Nature Communications
- Published
- 2023-07-28
- Volume
- 14
- Issue
- 1
- Pages
- 4542-4542
- ISSN
- 2041-1723 (ISSN-L)
- PMC
- PMC10382552
- OpenAlex
- W4385347692
- Cited by
- 207
- References
- 30
- Citations by year
- 2026: 54 · 2025: 104 · 2024: 47 · 2023: 2
- FWCI
- 18.75
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Nature Communications (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Funders
- Science and Technology Commission of Shanghai Municipality (21DZ1100100, 22511106101, BP0719010, 18DZ2270700)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
- Keywords
- vision-language pre-training, chest radiography, radiology reports, medical foundation models, few-shot fine-tuning
- MeSH terms
- Radiologists, Humans, Language, Radiography, Radiology, Knowledge
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
