The DOI matches its Crossref record; 63 of the 68 references it cites were found in a registry or an index.
Making the Most of Text Semantics to Improve Biomedical Vision-Language Processing
Open accessAbstract
Multi-modal data abounds in biomedicine, such as radiology images and reports. Interpreting this data at scale is essential for improving clinical care and accelerating clinical research. Biomedical text with its complex semantics poses additional challenges in vision--language modelling compared to the general domain, and previous work has used insufficiently adapted models that lack domain-specific language understanding. In this paper, we show that principled textual semantic modelling can substantially improve contrastive learning in self-supervised vision--language processing. We release a language model that achieves state-of-the-art results in radiology natural language inference through its improved vocabulary and novel language pretraining objective leveraging semantics and discourse characteristics in radiology reports. Further, we propose a self-supervised joint vision--language approach with a focus on better text modelling. It establishes new state of the art results on a wide range of publicly available benchmarks, in part by leveraging our new domain-specific language model. We release a new dataset with locally-aligned phrase grounding annotations by radiologists to facilitate the study of complex semantic modelling in biomedical vision--language processing. A broad evaluation, including on this new dataset, shows that our contrastive learning approach, aided by textual-semantic modelling, outperforms prior methods in segmentation tasks, despite only using a global-alignment objective.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“We note that the official MIMIC-CXR test split is highly enriched for abnormal cases, 26 hence prior studies are more common (Table 1).”
From MAIRA-2: Grounded Radiology Report Generation (2024) · cites this paper as [26]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“By inviting a domain expert to review and adapt individual phrase outputs, our approach to qualitative evaluation of report generation differs from most existing works that primarily ask experts to rate overall report quality via a (Likert-) scale; 59 provide total error counts/ categorisation; 48,60 or engage in comparison rankings across different report candidates.61”
From MAIRA-2: Grounded Radiology Report Generation (2024) · cites this paper as [48]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 4×
“We use a ResNet-50 [28] architecture as our image encoder and pretrain the model on MIMIC-CXR images using SimCLR [6] with domain-specific augmentations as detailed in Section 4.”
BackgroundMethodInfluentialVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · cited 2×
“[64] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 4×
“[16] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.”
BackgroundMethodInfluentialLearning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · cited 5×
“A variety of self-supervised VLP approaches have been proposed towards jointly learning visual and textual representations of paired data without supervision, such as frameworks using contrastive objectives [26, 42, 60], approaches based on joint transformer architectures [40, 41, 51, 69], self-supervised VLP with wor…”
BackgroundMethodInfluentialGoldberger, A. L., Amaral, L. A. N., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G., Mietus, J. E., Moody, G. B., Peng, C.-K., & Stanley, H. E. (2000) · Circulation · doi:10.1161/01.cir.101.23.e215 · Cited by 14,809
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · cited 6×
“network (CNN) [37] image encoder Eimg, our CXR-BERT text encoder Etxt, and projection models Pimg and Ptxt to learn representations in a joint space.”
BackgroundMethodInfluentialMIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141 · cited 10×
“The phases proceed as follows: (I) First, we construct a custom WordPiece [80] vocabulary of 30k to-kens from PubMed abstracts 4 (15 GB), MIMIC-III [35] clinical notes (3.5 GB), and MIMIC-CXR radiology reports (0.1 GB).”
BackgroundMethodInfluentialBioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · cited 5×
“Related medical VLP work commonly uses publicly available contextual word embedding models including BioBERT [39], ClinicalBERT [2], BioClini-calBERT [2], or PubMedBERT [26].”
BackgroundMethodInfluential
Research works printed but not linked: 21
- Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: III, H.D., Singh, A. (eds.) Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13–18 July 2020, Virtual Event. Proceedings of Machine Learning Research, vol. 119, pp. 1597–1607. PMLR (13–18 Jul 2020), http://proceedings.mlr.press/v119/chen20j.html
- Graf, B., et al.: Pneumothorax and chest tube classification on chest X-rays for detection of missed pneumothorax. Machine Learning for Health (ML4H) NeurIPS Workshop: Extended Abstract (2020). https://arxiv.org/abs/2011.07353 · arXiv
- Hayat, N., Lashen, H., Shamout, F.E.: Multi-label generalized zero shot learning for the classiffcation of disease in chest radiographs. In: Machine Learning for Healthcare Conference, pp. 461–477. PMLR (2021)
- Hsu, T.M.H., Weng, W.H., Boag, W., McDermott, M., Szolovits, P.: Unsupervised multimodal representation learning across medical images and reports. Machine Learning for Health (ML4H) NeurIPS Workshop (2018). https://arxiv.org/abs/1811.08615 · arXiv
- Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456. PMLR (2015)
- Li, L.H., Yatskar, M., Yin, D., Hsieh, C.J., Chang, K.W.: VisualBERT: a simple and performant baseline for vision and language. arXiv preprint arXiv:1908.03557 (2019) · arXiv
- Li, Y., et al.: Supervision exists everywhere: a data efficient contrastive language-image pre-training paradigm. arXiv preprint arXiv:2110.05208 (2021) · arXiv
- Liu, X., et al.: Adversarial training for large neural language models. arXiv preprint arXiv:2004.08994 (2020) · arXiv
- Liu, Y., et al.: RoBERTa: a robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019) · arXiv
- Logeswaran, L., Lee, H.: An efficient framework for learning sentence representations. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, 30 April - 3 May 2018, Conference Track Proceedings. OpenReview.net (2018). https://openreview.net/forum?id=rJvJXZb0W
- Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: International Conference on Learning Representations (2018). https://openreview.net/forum?id=Bkg6RiCqY7
- Lu, J., Batra, D., Parikh, D., Lee, S.: ViLBERT: pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. In: Advances in Neural Information Processing Systems, vol. 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019 (December), pp. 8–14 (2019). Vancouver, BC, Canada, pp. 13–23 (2019). https://proceedings.neurips.cc/paper/2019/hash/c74d97b01eae257e44aa9d5bade97baf-Abstract.html
- Oord, A.v.d., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018) · arXiv
- Preechakul, K., Piansaddhayanon, C., Naowarat, B., Khandhawit, T., Sriswasdi, S., Chuangsuwanich, E.: Set prediction in the latent space. In: Advances in Neural Information Processing Systems, vol. 34 (2021)
- Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement. arXiv preprint arXiv:1804.02767 (2018) · arXiv
- Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: Towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, vol. 28: Annual Conference on Neural Information Processing Systems 2015(December), pp. 7–12, 2015. Montreal, Quebec, Canada 28, pp. 91–99 (2015). https://proceedings.neurips.cc/paper/2015/hash/14bfa6bb14875e45bba028a21ed38046-Abstract.html
- Simard, P., Steinkraus, D., Platt, J.: Best practices for convolutional neural networks applied to visual document analysis. In: Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings, pp. 958–963. IEEE (2003)
- Wolf, T., et al.: Huggingface’s transformers: state-of-the-art natural language processing. arXiv preprint arXiv:1910.03771 (2019) · arXiv
- You, Y., Gitman, I., Ginsburg, B.: Large batch training of convolutional networks. arXiv preprint arXiv:1708.03888 (2017) · arXiv
- Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. In: Bengio, Y., LeCun, Y. (eds.) 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, 2–4 May 2016, Conference Track Proceedings (2016). http://arxiv.org/abs/1511.07122 · arXiv
- Zhang, Y., Jiang, H., Miura, Y., Manning, C.D., Langlotz, C.P.: Contrastive learning of medical visual representations from paired images and text. arXiv preprint arXiv:2010.00747 (2020) · arXiv
Cited by
Sorted by citations
A multimodal generative AI copilot for human pathology
Lu, M. Y., Chen, B., Williamson, D. F. K., Chen, R. J., Zhao, M., Chow, A. K., Ikemura, K., Kim, A., Pouli, D., Patel, A., et al. (2024) · Nature · doi:10.1038/s41586-024-07618-3 · Cited by 461
The shaky foundations of large language models and foundation models for electronic health records
Wornow, M., Xu, Y., Thapa, R., Patel, B., Steinberg, E., Fleming, S., Pfeffer, M. A., Fries, J., & Shah, N. H. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00879-8 · Cited by 324
A Multimodal Biomedical Foundation Model Trained from Fifteen Million Image–Text Pairs
Zhang, S., Xu, Y., Usuyama, N., Xu, H., Bagga, J., Tinn, R., Preston, S., Rao, R., Wei, M., Valluri, N., et al. (2025) · NEJM AI · doi:10.1056/aioa2400640 · Cited by 283
Self-citationLLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day
Li, C., Wong, C., Zhang, S., Usuyama, N., Liu, H., Yang, J., Naumann, T., Poon, H., & Gao, J. (2023) · arXiv · doi:10.48550/arxiv.2306.00890 · Cited by 590
“Domain-specific pretraining has been shown to be effective for biomedical natural language processing (NLP) applications [18, 15, 11, 31] and biomedical vision-language (VL) tasks [16, 7, 41, 51, 9].”
BackgroundSelf-citationLearning to Exploit Temporal Structure for Biomedical Vision-Language Processing
Bannur, S., Hyland, S., Liu, Q., Pérez-García, F., Ilse, M., Castro, D. C., Boecking, B., Sharma, H., Bouzid, K., Thieme, A., et al. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01442 · Cited by 227
“Despite this, the existing VLP work to date considers alignment between only single images and reports [9,32,46,81], going so far as to remove temporal content from reports in training data to prevent ‘hallucinations’ in downstream report generation [54].”
BackgroundMethodInfluentialSelf-citationKnowledge-enhanced visual-language pre-training on chest radiology images
Zhang, X., Wu, C., Zhang, Y., Xie, W., & Wang, Y. (2023) · Nature Communications · doi:10.1038/s41467-023-40260-7 · Cited by 207
“Specifically, ConVIRT [64] simply uses chest X-rays and the associated raw report for self-supervised training; GLoRIA [25] learns global and localized representations by contrasting attention-weighted image sub-regions and words in the reports; BioVIL [6] introduces a pre-trained radiology-specific text encoder CXR-B…”
BackgroundMethodInfluentialPre-trained Language Models in Biomedical Domain: A Systematic Survey
Wang, B., Xie, Q., Pei, J., Chen, Z., Tiwari, P., Li, Z., & Fu, J. (2023) · ACM Computing Surveys · doi:10.1145/3611651 · Cited by 238
“12 Dual-Encoder ResNet ClinicalBERT MIMIC-CXR COVID-Rural, NIH-CXR, Object CXR, SIIM BioViL [26] 2022.”
A medical multimodal large language model for future pandemics
Liu, F., Zhu, T., Wu, X., Yang, B., You, C., Wang, C., Lu, L., Liu, Z., Zheng, Y., Sun, X., et al. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00952-2 · Cited by 155
“In particular, when transferring our approach trained on patient data from Spain to China, we observe an encouraging performance, i.e., 90.1% AUC, which is competitive with the region-speci fi c results of previous works CLIP (80.7% AUC) and BioViL (90.4% AUC), which were obtained by training and testing on the data co…”
InfluentialCLIP in medical imaging: A survey
Zhao, Z., Liu, Y., Wu, H., Wang, M., Li, Y., Wang, S., Teng, L., Liu, D., Cui, Z., Wang, Q., et al. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103551 · Cited by 248
MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training for X-ray Diagnosis
Wu, C., Zhang, X., Zhang, Y., Wang, Y., & Xie, W. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01954 · Cited by 148
CXR-CLIP: Toward Large Scale Chest X-ray Language-Image Pre-training
You, K., Gu, J., Ham, J., Park, B., Kim, J., Hong, E. K., Baek, W., & Roh, B. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43895-0_10 · Cited by 109
“BioVIL [2] enhanced the text encoder by matching two sections during language pre-training.”
MethodFoundation Model for Endoscopy Video Analysis via Large-Scale Self-supervised Pre-train
Wang, Z., Liu, C., Zhang, S., et al. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43996-4_10 · Cited by 93
Exploring scalable medical image encoders beyond text supervision
Pérez-García, F., Sharma, H., Bond-Taylor, S., Bouzid, K., Salvatelli, V., Ilse, M., Bannur, S., Castro, D. C., Schwaighofer, A., Lungren, M. P., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-024-00965-w · Cited by 115
“In the evolving landscape of vision–language deep learning, the prevalent use of textual supervision [1, 2] has been a cornerstone in learning novel visual descriptors for downstream applications [2, 3], including biomedical domains [4, 5, 6, 7].”
Self-citationA Comprehensive Survey of Foundation Models in Medicine
Khan, W., Leem, S., See, K. B., Wong, J. K., Zhang, S., & Fang, R. (2025) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2025.3531360 · Cited by 137
Automated Radiology Report Generation: A Review of Recent Advances
Sloan, P., Clatworthy, P. L., Simpson, E., & Mirmehdi, M. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2024.3408456 · Cited by 106
“Other papers of significance where various approaches to ARRG were introduced by way of pretrained language models are [37], [56], [83], [119], [120], [124], [131].”
Vision-language foundation models for medical imaging: a review of current practices and innovations
Ryu, J. S., Kang, H., Chu, Y., & Yang, S. (2025) · Biomedical Engineering Letters · doi:10.1007/s13534-025-00484-6 · Cited by 109
“Their extensive pretraining allows them to generalize across applications, thereby minimizing man - ual efforts and enhancing accuracy (ACC) [ 10].”
Yildirim, N., Richardson, H., Wetscherek, M. T., Bajwa, J., Jacob, J., Pinnock, M. A., Harris, S., Coelho De Castro, D., Bannur, S., Hyland, S., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642013 · Cited by 82
“T) [8, 17], ELIXR [136], Med-PaLM M [127]) invite high expectations of how the use of AI may transform clinical practice through efficiency and quality gains [120]; and improved overall patient care.”
BackgroundMethodInfluentialSelf-citationPRIOR: Prototype Representation Joint Learning from Medical Images and Reports
Cheng, P., Lin, L., Lyu, J., Huang, Y., Luo, W., & Tang, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01953 · Cited by 93
MedKLIP: Medical Knowledge Enhanced Language-Image Pre-TrainingPreprint
Wu, C., Zhang, X., Zhang, Y., Wang, Y., & Xie, W. (2023) · medRxiv · doi:10.1101/2023.01.10.23284412 · Cited by 80
“BioViL notices the language pattern in reports is different from other natural texts and re-designs the language model used for medical VLP [7].”
BackgroundMethodResultInfluentialZhong, Y., Xu, M., Liang, K., Chen, K., & Wu, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43901-8_69 · Cited by 73
Has multimodal learning delivered universal intelligence in healthcare? A comprehensive survey
Lin, Q., Zhu, Y., Mei, X., Huang, L., Ma, J., He, K., Peng, Z., Cambria, E., & Feng, M. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102795 · Cited by 86
A scoping review on multimodal deep learning in biomedical images and texts
Sun, Z., Lin, M., Zhu, Q., Xie, Q., Wang, F., Lu, Z., & Peng, Y. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104482 · Cited by 69
“Boecking et al. [8] have recently proposed the MS-CXR dataset, which comprises 1153 image-sentence pairs with annotated bounding boxes and corresponding phrases validated by radiologists.”
CAMANet: Class Activation Map Guided Attention Network for Radiology Report Generation
Wang, J., Bhalerao, A. H., Yin, T., See, S. C. W., & He, Y. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3354712 · Cited by 72
“Recently, several large language models (LLMs) [28]– [30] have been proposed for the radiology domain.”
BackgroundHuang, W., Li, C., Zhou, H.-Y., Yang, H., Liu, J., Liang, Y., Zheng, H., Zhang, S., & Wang, S. (2024) · Nature Communications · doi:10.1038/s41467-024-51749-0 · Cited by 62
Open challenges and opportunities in federated foundation models towards biomedical healthcare
Li, X., Peng, L., Wang, Y., & Zhang, W. (2025) · BioData Mining · doi:10.1186/s13040-024-00414-9 · Cited by 84
Buess, L., Keicher, M., Navab, N., Maier, A., & Arasteh, S. T. (2025) · Biomedical Engineering Letters · doi:10.1007/s13534-025-00497-1 · Cited by 79
“…videos, Labels, QA-pairs 6K videos, 6K labels, 3K QA Medical videos MIMIC-CXR [8] X-ray, Reports 377K images, 227K reports Chest X-ray MS-CXR [83] X-ray, Descriptions, Bounding-boxes 1K image-sentence pairs, Bounding-boxes Chest X-ray OmniMedVQA [137] Medical images, QA 118K images, 127K…”
MedSyn: Text-Guided Anatomy-Aware Synthesis of High-Fidelity 3-D CT Images
Xu, Y., Sun, L., Peng, W., Jia, S., Morrison, K., Perer, A., Zandifar, A., Visweswaran, S., Eslami, M., & Batmanghelich, K. N. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3415032 · Cited by 68
“To enable the text-guided medical image generation that utilize radiology reports as the guidance, we train a text encoder (Medical BERT [16]) that encodes natural language from radiology reports into high-dimensional vectors ( f text ).”
MethodMulti-task paired masking with alignment modeling for medical vision-language pre-training
Zhang, K., Yang, Y., Yu, J., Jiang, H., Fan, J., Huang, Q., & Han, W. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3325965 · Cited by 51
“The report paired with the image is fed into the pre-trained text encoder (CXRBERT [37]) to obtain global report representations t ∈ RM×K corresponding to the report.”
MethodImproving Medical Vision-Language Contrastive Pretraining With Semantics-Aware Triage
Liu, B., Lu, D., Wei, D., Wu, X., Wang, Y., Zhang, Y., & Zheng, Y. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3294980 · Cited by 56
“For the retrieval-based scheme [4], [19], [31], [44], the pretraining objective is to align visual and textual features in the embedding space [14], [26].”
MethodCT2Rep: Automated Radiology Report Generation for 3D Medical Imaging
Hamamci, I. E., Er, S., & Menze, B. (2024) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-72390-2_45 · Cited by 47
Liu, C., Cheng, S. L., Chen, C., Qiao, M., Zhang, W., Shah, A., Bai, W., & Arcucci, R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43907-0_61 · Cited by 54
A multimodal multidomain multilingual medical foundation model for zero shot clinical diagnosis
Liu, F., Zheng, L., Yin, Q., Huang, J., Luo, J., Thakur, A., Branson, K., Schwab, P., Yin, B., Wu, X., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-024-01339-7 · Cited by 50
Xplainer: From X-Ray Observations to Explainable Zero-Shot Diagnosis
Pellegrini, C., Keicher, M., Özsoy, E., Jiraskova, P., Braren, R., & Navab, N. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43904-9_41 · Cited by 32
CARZero: Cross-Attention Alignment for Radiology Zero-Shot Classification
Lai, H., Yao, Q., Jiang, Z., Wang, R., He, Z., Tao, X., & Zhou, S. K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01059 · Cited by 42
Med3DVLM: An Efficient Vision-Language Model for 3D Medical Image Analysis
Xin, Y., Ates, G. C., Gong, K., & Shao, W. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3604595 · Cited by 73
“Similarly, BioViL [4] leveraged large-scale biomedical datasets to refine multi-modal representations, significantly outperforming previous supervised methods.”
Harnessing Text Insights With Visual Alignment for Medical Image Segmentation
Zeng, Q., Luo, H., Lu, Z., Xie, Y., Wang, Z., Zhang, Y., & Xia, Y. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3601359 · Cited by 33
“This included LMs such as BioClinicalBERT [59], RadBERT [48] and CXR-BERT [58], as well as text encoders from pre-trained VLMs like CLIP and CXR-CLIP.”
Medical Phrase Grounding with Region-Phrase Context Contrastive Alignment
Chen, Z., Zhou, Y., Tran, A., Zhao, J., Wan, L., Ooi, G. S. K., Cheng, L. T.-E., Thng, C. H., Xu, X., Liu, Y., et al. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-43990-2_35 · Cited by 26
PadChest-GR: A Bilingual Chest X-Ray Dataset for Grounded Radiology Report Generation
Castro, D. C., Bustos, A., Bannur, S., Hyland, S. L., Bouzid, K., Wetscherek, M. T., Sánchez-Valverde, M. D., Jaques-Pérez, L., Pérez-Rodríguez, L., Takeda, K., et al. (2025) · NEJM AI · doi:10.1056/aidbp2401120 · Cited by 29
Self-citationJang, J., Kyung, D., Kim, S. H., Lee, H., Bae, K., & Choi, E. (2024) · Scientific Reports · doi:10.1038/s41598-024-73695-z · Cited by 34
A Comprehensive Study of GPT-4V’s Multimodal Capabilities in Medical ImagingPreprint
Li, Y., Liu, Y., Wang, Z., Liang, X., Liu, L., Wang, L., Cui, L., Tu, Z., Wang, L., & Zhou, L. (2023) · medRxiv · doi:10.1101/2023.11.03.23298067 · Cited by 28
“From the case study, it appears that the GPT-4V has the potential to generate bounding boxes, but notably, its accuracy performs rather poorly.”
BackgroundLongitudinal data and a semantic similarity reward for chest X-ray report generation
Nicolson, A., Dowling, J., Anderson, D., & Koopman, B. (2024) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2024.101585 · Cited by 36
IMITATE: Clinical Prior Guided Hierarchical Vision-Language Pre-Training
Liu, C., Cheng, S. L., Shi, M., Shah, A., Bai, W., & Arcucci, R. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3449690 · Cited by 30
“5, IMITATE boasts an impressive capability to accurately delineate the clinical regions of concern in the CXR images, outperforming other compared methods [3], [4], [27], [28], [52].”
Phan, V. M. H., Xie, Y., Qi, Y., Liu, L., Liu, L., Zhang, B., Liao, Z., Wu, Q., To, M., & Verjans, J. W. H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01092 · Cited by 26
“The supervised head in our network uses Transformer-based fusion layers to extract discriminative features.”
InfluentialCommon Vision-Language Attention for Text-Guided Medical Image Segmentation of Pneumonia
Guo, Y., Zeng, X., Zeng, P., Fei, Y., Lu, W. S., Zhou, J., & Wang, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72114-4_19 · Cited by 24
A systematic evaluation of GPT-4V's multimodal capability for chest X-ray image analysis
Liu, Y., Li, Y., Wang, Z., Liang, X., Liu, L., Wang, L., Cui, L., Tu, Z., Wang, L., & Zhou, L. (2024) · Meta-Radiology · doi:10.1016/j.metrad.2024.100099 · Cited by 25
Leveraging foundation and large language models in medical artificial intelligence
Wong, I. N., Monteiro, O., Baptista‐Hon, D. T., Wang, K., Lu, W.-Y., Sun, Z., Nie, S., & Yin, Y. (2024) · Chinese Medical Journal · doi:10.1097/cm9.0000000000003302 · Cited by 27
Contrastive Masked Image-Text Modeling for Medical Visual Representation Learning
Chen, C., Zhong, A., Wu, D., Luo, J., & Li, Q. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43904-9_48 · Cited by 23
Park, S. J., Lee, E. S., Shin, K. S., Lee, J. E., & Ye, J. C. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.103021 · Cited by 27
Visual-Textual Matching Attention for Lesion Segmentation in Chest Images
Bui, P.-N., Le, D.-T., & Choo, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72114-4_67 · Cited by 23
Turgut, Ö., Müller, P., Hager, P., Shit, S., Starck, S., Menten, M. J., Martens, E., & Rueckert, D. (2025) · Medical Image Analysis · doi:10.1016/j.media.2024.103451 · Cited by 22
Residual-based Language Models are Free Boosters for Biomedical Imaging Tasks
Lai, Z., Wu, J., Chen, S., Zhou, Y., & Hovakimyan, N. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00515 · Cited by 42
“Utilizing these specialized datasets, significant advancements have been made in applying general-domain vision-language models to biomedical imaging [6, 23, 54].”
Large language model for medical images: a survey of taxonomy, systematic review, and future trends
Wang, P., Lu, W., Lu, C., Zhou, R., Li, M., & Qin, L. (2025) · Big Data Mining and Analytics · doi:10.26599/bdma.2024.9020090 · Cited by 21
UniChest: Conquer-and-Divide Pre-Training for Multi-Source Chest X-Ray Classification
Dai, T., Zhang, R., Hong, F., Yao, J., Zhang, Y., & Wang, Y. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3381123 · Cited by 31
“BioVIL [4] focused on the representation of radiological reports and proposed a radiology-specific text encoder along with the classical VLP paradigm.”
MethodEVA-X: a foundation model for general chest x-ray analysis with self-supervised learning
Yao, J., Wang, X., Song, Y., Zhao, H., Ma, J., Chen, Y., Liu, W., & Wang, B. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02032-z · Cited by 36
“Specifically, the mean standard deviation of EVA-X on both datasets is 0.03, which is lower than all other methods including Medical MAE [15] (0.045), MGCA [9] (0.055), BioViL [10] (0.135), etc.”
InfluentialSun, Y., Wen, X., Zhang, Y., Jin, L., Yang, C., Zhang, Q., Jiang, M. C., Xu, Z., Guo, W., Su, J., et al. (2025) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2025.108870 · Cited by 27
MAdapter: A Better Interaction Between Image and Language for Medical Image Segmentation
Zhang, X., Ni, B., Yang, Y., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72114-4_41 · Cited by 19
Chen, W., Liu, J., Liu, T., & Yuan, Y. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02246-w · Cited by 24
“Several vision-language models [29, 32, 5, 6, 31, 22, 3, 30] have been proposed for medical computer vision tasks.”
HERGen: Elevating Radiology Report Generation with Longitudinal Data
Wang, F., Du, S., & Yu, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73001-6_11 · Cited by 33
Integrating language into medical visual recognition and reasoning: A survey
Lu, Y., & Wang, A. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103514 · Cited by 32
Advancements in Radiology Report Generation: A Comprehensive Analysis
Mamdouh, D., Attia, M., Mohamed, O., Mohamed, N., Lotfy, A., Arafa, T., Rashed, E. A., & Khoriba, G. A. (2025) · Bioengineering · doi:10.3390/bioengineering12070693 · Cited by 20
Large Language Models in Medical Image Analysis: A Systematic Survey and Future Directions
Urooj, B., Fayaz, M., Ali, S., Dang, L. M., & Kim, K. W. (2025) · Bioengineering · doi:10.3390/bioengineering12080818 · Cited by 26
“, the model provides an answer based on the visual content [23–25].”
Uncertainty-Aware Medical Diagnostic Phrase Identification and Grounding
Zou, K., Bai, Y., Liu, B., Chen, Y., Chen, Z., Zhou, Y., Yuan, X., Wang, M., Shen, X., Cao, X., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3596878 · Cited by 36
“In the medical domain, visual-language pretraining frameworks like MedKLIP [25] and BioViL [26] incorporated clinical knowledge to improve grounding performance, enabling fine-grained contextual understanding critical for downstream tasks.”
A systematic review of multi-modal large language models on domain-specific applications
Li, S., Wong, K. W., Wang, G., & Duong, T.-T. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11398-1 · Cited by 24
LIMITR: Leveraging Local Information for Medical Image-Text Representation
Dawidowicz, G., Hirsch, E., & Tal, A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01935 · Cited by 18
“Specifically, given an image, the goal of [3, 8, 23, 25, 27] is to classify it into pre-defined diagnoses.”
BackgroundMethodInfluentialBootstrapping Chest CT Image Understanding by Distilling Knowledge from X-Ray Expert Models
Cao, W., Zhang, J., Xia, Y., Mok, T. C. W., Li, Z., Ye, X., Lü, L., Zheng, J., Tang, Y., & Zhang, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01068 · Cited by 23
“This progress has notably impacted the medical imaging field, particularly in chest X-ray zero-shot diagnosis [3, 7, 15, 31].”
BackgroundYu, X., Elazab, A., Ge, R., Zhu, J., Zhang, L., Jia, G., Wu, Q., Wan, X., Li, L., & Wang, C. (2025) · Neural Networks · doi:10.1016/j.neunet.2024.107096 · Cited by 18
He, J., Li, P., Liu, G., & Zhong, S. (2025) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi60581.2025.10981029 · Cited by 20
“To enhance semantic understanding, some studies align textual words with local image regions [5,6].”
MAIRA-2: Grounded Radiology Report Generation
Bannur, S., Bouzid, K., Castro, D. C., Schwaighofer, A., Thieme, A., Bond-Taylor, S., Ilse, M., Pérez-García, F., Salvatelli, V., Sharma, H., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.04449 · Cited by 19
“Table 4 presents the mean intersection over union (mIoU) of pixel masks from generated vs ground-truth boxes on our test split of the MS-CXR dataset (Boecking et al., 2022).”
BackgroundMethodInfluentialSelf-citationViLLA: Fine-Grained Vision-Language Representation Learning from Real-World Data
Varma, M., Delbrouck, J.-B., Hooper, S., Chaudhari, A. S., & Langlotz, C. P. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02031 · Cited by 17
“Learning from Real-World Multimodal Data: Our work relates closely to prior studies that have developed VLMs for medical [42, 40, 2, 15] and product datasets [1, 5, 41].”
MethodLanguage model and its interpretability in biomedicine: A scoping review
Lyu, D., Wang, X., Chen, Y., & Wang, F. (2024) · iScience · doi:10.1016/j.isci.2024.109334 · Cited by 17
“21,24,25,28,30,52,57,61,62,76 Since NLI requires sentence-level semantics, it is particularly useful in taskslikeparaphraseminingandinformationretrievalin thegeneraldomainandmedicalconceptnormalization, semantic relatedness,and question answering in the biomedical domain.”
BackgroundHuang, W., Li, C., Yang, H., Liu, J., Liang, Y., Zheng, H., & Wang, S. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103299 · Cited by 19
“…which measures the difference between the similarity values of inner and outer bounding box regions without relying on hard threshold values, is a more objective metric that could be more clinically relevant, particularly when heatmap visualizations are needed instead of discrete segmentation [10].”
BackgroundMethodInfluentialMedFILIP: Medical Fine-Grained Language-Image Pre-Training
Liang, X., Li, X., Li, F., Jiang, J., Dong, Q., Wang, W., Wang, K., Dong, S., Luo, G., & Li, S. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3528196 · Cited by 20
Park, J., Kim, S., Yoon, B., Hyun, J., & Choi, K. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3587687 · Cited by 29
“For CXR, the MS-CXR benchmark dataset [25] was released for phrase grounding, and MedRPG [26] specializes in identifying bounding boxes for phrases in CXR images.”
R-LLaVA: Improving Med-VQA Understanding through Visual Region of Interest
Chen, X., Lai, Z., Ruan, K., Chen, S., Liu, J., & Liu, Z. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11227685 · Cited by 15
“This approach integrates simple physician annotations, such as bounding boxes, as prior knowledge, directly infusing these visual cues into the image space via CLIP.”
Trade-Offs in Fine-Tuned Diffusion Models between Accuracy and Interpretability
Dombrowski, M., Reynaud, H., Müller, J. P., Baugh, M., & Kainz, B. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i19.30095 · Cited by 20
Automatic medical report generation based on deep learning: A state of the art survey
Liu, X., Xin, J., Shen, Q., Huang, Z., & Wang, Z. (2025) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2024.102486 · Cited by 22
UniBrain: Universal Brain MRI diagnosis with hierarchical knowledge-enhanced pre-training
Lei, J., Dai, L., Jiang, H., Wu, C., Zhang, X., Zhang, Y., Yao, J., Xie, W., Zhang, Y., Li, Y., et al. (2025) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2025.102516 · Cited by 17
Frequency-Domain Multi-modal Fusion for Language-Guided Medical Image Segmentation
Yu, B., Yang, J., Du, Z., et al. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05114-1_27 · Cited by 13
“Language Encoder: For the input clinical text T ∈ R L , we follow prior works [8,12,13] to adopt CXR-BERT [19] as the language encoder to extract word-level linguistic features F T ∈ R L × C .”
Dutta, N., Bose, K., Syailendra, E. A., Chu, L. C. H., & Gupta, P. K. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106227 · Cited by 19
Panayides, A. S., Chen, H., Filipović, N., Geroski, T., Hou, J., Lekadir, K., Kostas Marias, A., Matsopoulos, G. K., Papanastasiou, G., Sarder, P., et al. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3649496 · Cited by 14
Zeng, Q., Luo, H., Ma, X., et al. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3674365 · Cited by 13
“…of TeViD on the choice of text encoder, we evaluated three representative models with di ff erent pretraining objectives and domain coverage: CXR-BERT [48], a radiology-specific language model; BioBERT [52], a general biomedical text encoder; and MedCLIP [53], a multimodal vision–language model.”
Privacy Distillation: Reducing Re-identification Risk of Diffusion Models
Fernández, V., Sanchez, P., Pinaya, W. H. L., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-53767-7_1 · Cited by 13
Multimodal Foundation Models for Medical Imaging - A Systematic Review and Implementation GuidelinesPreprint
Huang, S.-C., Jensen, M., Yeung, S. Y., Lungren, M. P., Poon, H., & Chaudhari, A. S. (2024) · medRxiv · doi:10.1101/2024.10.23.24316003 · Cited by 15
Self-citationLin, M., Holste, G., Wang, S., Zhou, Y., Wei, Y., Banerjee, I., Chen, P., Dai, T., Du, Y., Dvornek, N. C., et al. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103739 · Cited by 22
Zeng, Q., Luo, H., Ma, X., et al. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04978-0_22 · Cited by 12
Gene-Induced Multimodal Pre-training for Image-Omic Classification
Jin, T., Xie, X., Wan, R., Li, Q., & Wang, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43987-2_49 · Cited by 12
Qin, J., Liu, C., Cheng, S. L., Guo, Y., & Arcucci, R. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447326 · Cited by 17
“…we evaluated its performance using different combinations among three pre-trained image encoders, namely ResNet autoencoder [15, 16], DINOv2-small and DINOv2-base [17], and five pre-trained language models, which are BERT [18], BioBERT [19], ClinicalBERT [20], CXR-BERT [12] and PubMedBERT [21].”
BackgroundMethodABP: Asymmetric Bilateral Prompting for Text-Guided Medical Image Segmentation
Zeng, X., Zeng, P., Cui, J., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72114-4_6 · Cited by 11
Automatic medical report generation combining contrastive learning and feature difference
Lyu, C., Qiu, C., Han, K., Li, S., Sheng, V. S., Rong, H., Song, Y., Liu, Y., & Qiu, C. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112630 · Cited by 11
MDFormer: Transformer-Based Multimodal Fusion for Robust Chest Disease Diagnosis
Liu, X., Pan, F., Song, H., Cao, S., Li, C., & Li, T. (2025) · Electronics · doi:10.3390/electronics14101926 · Cited by 14
“Specifically, for the j -th modality data of the i -th patient, after being processed by the encoder, we obtain the feature representation e ( j ) i , where j ∈ { img , txt , ts } .”
Aligning Human Knowledge with Visual Concepts Towards Explainable Medical Image Classification
Gao, Y., Gu, D., Zhou, M., & Metaxas, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72117-5_5 · Cited by 10
ChEX: Interactive Localization and Region Description in Chest X-Rays
Müller, P., Kaissis, G. A., & Rueckert, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72664-4_6 · Cited by 19
Aligning, Autoencoding and Prompting Large Language Models for Novel Disease Reporting
Liu, F., Wu, X., Huang, J., Yang, B., Branson, K., Schwab, P., Clifton, L., Zhang, P., Luo, J., Zheng, Y., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3534586 · Cited by 13
“Recently, few-shot learning has received growing research interest across many AI domains [34], [42], [65].”
InfluentialECAMP: Entity-centered Context-aware Medical Vision Language Pre-training
Wang, R., Yao, Q., Jiang, Z., Lai, H., He, Z., Tao, X., & Zhou, S. K. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103690 · Cited by 17
Mlip: Medical Language-Image Pre-Training With Masked Local Representation Learning
Liu, J., Zhou, H.-Y., Li, C., Huang, W., Yang, H., Liang, Y., Shi, G., Zheng, H., & Wang, S. (2024) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi56570.2024.10635357 · Cited by 10
“We select 10 state-of-the-arts methods on different tasks including ConVIRT[18], GLoRIA[9], BioVIL[19], Table 2 .”
MethodEnabling Multi-modal Conversational Interface for Clinical Imaging
Dayanandan, K., & Lall, B. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650805 · Cited by 9
“…contains 3,77,110 chest x-rays and associated semi-structured free-text radiology reports from 227,835 imaging studies conducted on 65,379 patients at the Beth Israel Deaconess Medical Center Emergency Department in Boston [33] and has many derivatives with additional information [6, 40, 48, 66].”
Uslu, E. E., Sezer, E., & Güven, Z. A. (2024) · IEEE Access · doi:10.1109/access.2024.3400007 · Cited by 11
“Six different BERT-based language models were used in this study, including BERT [4], BioBERT [5], ClinicalBERT [6], [7], CXR-BERT generalized and specialized editions [8], and S-PubMedBert [9].”
MethodCross-modal contrastive learning for unified placenta analysis using photographs
Pan, Y., Mehta, M., Goldstein, J. A., Ngonzi, J. N. J., Bebell, L. M., Roberts, D. J., Carreon, C. K., Gallagher, K., Walker, R. E., Gernand, A. D., et al. (2024) · Patterns · doi:10.1016/j.patter.2024.101097 · Cited by 9
“47 Research in this area has focused on improving VLC methodologies through innovations in model architectures, 48,49 visual representations, 50–52 textual representations, 45,53 and loss functions 54,55 as well as sampling strategies, 56,57 training strategies, 58 and classifier performance.”
RadAlign: Advancing Radiology Report Generation with Vision-Language Concept Alignment
Gu, D., Gao, Y., Zhou, Y., Zhou, M., & Metaxas, D. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04981-0_46 · Cited by 11
Fine-grained Medical Vision-Language Representation Learning for Radiology Report Generation
Wang, S., Peng, B., Liu, Y., & Peng, Q. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.989 · Cited by 10
Metadata
- Authors
- Boecking, B., Usuyama, N., Bannur, S., Castro, D. C., Schwaighofer, A., Hyland, S., Wetscherek, M., Naumann, T., Nori, A., Alvarez-Valle, J., Poon, H., & Oktay, O.
- Affiliations
- Microsoft (United States) (US)
- Corresponding author
- Oktay, Ozan
- ORCID
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer Nature Switzerland
- Published
- 2022
- Pages
- 1-21
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- OpenAlex
- W4312533035
- Semantic Scholar
- a1aeb5442e31276b197696f49b3243112a4049ce
- Cited by
- 317
- References
- 68
- Citations by year
- 2026: 88 · 2025: 122 · 2024: 73 · 2023: 34
- FWCI
- 20.76
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Lecture notes in computer science (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- radiology reports, contrastive learning, domain-specific language models, natural language inference, phrase grounding, multimodal medical data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
