Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing
Open accessAbstract
Self-supervised learning in vision-language processing (VLP) exploits semantic alignment between imaging and text modalities. Prior work in biomedical VLP has mostly relied on the alignment of single image and report pairs even though clinical notes commonly refer to prior images. This does not only introduce poor alignment between the modalities but also a missed opportunity to exploit rich self-supervision through existing temporal content in the data. In this work, we explicitly account for prior images and reports when available during both training and fine-tuning. Our approach, named BioViL-T, uses a CNN-Transformer hybrid multi-image encoder trained jointly with a text model. It is designed to be versatile to arising challenges such as pose variations and missing input images across time. The resulting model excels on downstream tasks both in single- and multi-image setups, achieving state-of-the-art (SOTA) performance on (I) progression classification, (II) phrase grounding, and (III) report generation, whilst offering consistent improvements on disease classification and sentence-similarity tasks. We release a novel multi-modal temporal benchmark dataset, MS-CXR-T, to quantify the quality of vision-language representations in terms of temporal semantics. Our experimental results show the advantages of incorporating prior images and reports to make most use of the data.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Comparison to previous imaging studies is crucial for tracking the development of disease or impact of treatment.
“Comparison to previous imaging studies is crucial for tracking the development of disease or impact of treatment, and references to prior studies are frequent in radiology reporting. 16,17”
From MAIRA-2: Grounded Radiology Report Generation (2024) · cites this paper as [17]
Appendix B says radiologists compare images from different studies to assess pathology progression. Appendix A.1 also describes how prior-study information helps identify improvement or stability. Our copy does not explicitly address using those comparisons to assess the impact of treatment. (LLM, conf=0.82)
Experimental — accuracy not yet measuredReferences to prior studies can be used in conjunction with prior images to enable descriptions of change.
“Such references can be removed to reduce hallucinations when prior studies are not available, 9,14,18 or used in conjunction with prior images to enable descriptions of change.17,19,20”
From MAIRA-2: Grounded Radiology Report Generation (2024) · cites this paper as [17]
References to prior studies are frequent in radiology reporting.
“Comparison to previous imaging studies is crucial for tracking the development of disease or impact of treatment, and references to prior studies are frequent in radiology reporting. 16,17”
From MAIRA-2: Grounded Radiology Report Generation (2024) · cites this paper as [17]
It is not possible to quantitatively compare MAIRA-2 to models trained to generate other sections, such as Impression or both Findings and Impression together.
“It is not possible to quantitatively compare to models trained to generate other sections, such as Impression 17 or both Findings and Impression together.15,58”
From MAIRA-2: Grounded Radiology Report Generation (2024) · cites this paper as [17]
It is not possible to quantitatively compare MAIRA-2 to models trained to generate other sections, such as Impression or both Findings and Impression together.
“It is not possible to quantitatively compare to models trained to generate other sections, such as Impression 17 or both Findings and Impression together.15,58”
From MAIRA-2: Grounded Radiology Report Generation (2024) · cites this paper as [58]
In the MIMIC-CXR dataset, when the Comparison section is equivalent to 'No comparison available', references to prior studies are rarely observed, in contrast to the background rate of 40% in the full MIMIC-CXR dataset.
“Empirically, in the MIMIC-CXR dataset, when the Comparison section is equivalent to 'No comparison available', references to prior studies are rarely observed, in contrast to the background rate of 40% in the full MIMIC-CXR dataset.17”
From MAIRA-2: Grounded Radiology Report Generation (2024) · cites this paper as [17]
Appendix B, “Temporal aspects of the MIMIC-CXR v.2 dataset” (page unavailable), says that around 40% of reports have a progression label; it does not give a 40% rate of references to prior studies. Searches for “No comparison available” and references to prior studies in the Comparison section found no analysis of whether such references are rare under that condition. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 21,793 · see citing works
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285 · see citing works
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · see citing works
Goldberger, A. L., Amaral, L. A. N., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G., Mietus, J. E., Moody, G. B., Peng, C.-K., & Stanley, H. E. (2000) · Circulation · doi:10.1161/01.cir.101.23.e215 · Cited by 14,809 · see citing works
Layer NormalizationPreprint
Ba, J., Kiros, J., & Hinton, G. E. (2016) · arXiv.org · doi:10.48550/arxiv.1607.06450 · Cited by 13,762 · see citing works
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778 · see citing works
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522 · see citing works
(2004) · Cited by 9,718 · see citing works
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242 · cited 3× · see citing works
“…from image–text pairs has enabled the development of flexible general-purpose vision–language models both in the general domain [40, 53, 77] and for specialised domains such as biomedicine and radiology [9, 32, 81] often use only a single image for contrastive learning (e.g., InfoNCE [49]).”
BackgroundAvants, B., Epstein, C. L., Grossman, M., & Gee, J. C. (2007) · Medical Image Analysis · doi:10.1016/j.media.2007.06.004 · Cited by 5,618 · see citing works
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., & Summers, R. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.369 · Cited by 4,616 · see citing works
Bleu: a method for automatic evaluation of machine translation
Papineni, K. (2024) · doi:10.57702/ljteqmrb · Cited by 3,682 · see citing works
Domain-specific language model pretraining for biomedical natural language processing
Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J., & Poon, H. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3458754 · Cited by 2,170 · see citing works
UNITER: UNiversal Image-TExt Representation Learning
Chen, Y.-C., Li, L., Yu, L., El Kholy, A., Ahmed, F., Gan, Z., Cheng, Y., & Liu, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58577-8_7 · Cited by 2,055 · see citing works
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
Jia, C., Yang, Y., Xia, Y., Chen, Y.-T., Parekh, Z., Pham, H., Le, Q. V., Sung, Y., Li, Z., & Duerig, T. (2021) · arXiv · doi:10.48550/arxiv.2102.05918 · Cited by 1,977 · see citing works
CoCa: Contrastive Captioners are Image-Text Foundation ModelsPreprint
Yu, J.-H., Wang, Z., Vasudevan, V., Yeung, L., Seyedhosseini, M., & Wu, Y.-H. (2022) · Trans. Mach. Learn. Res. · doi:10.48550/arxiv.2205.01917 · Cited by 1,899 · see citing works
Fully Convolutional Siamese Networks for Change Detection
Daudt, R. C., Saux, B. L., & Boulch, A. (2018) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2018.8451652 · Cited by 1,872 · see citing works
Li, J., Li, D., Xiong, C., & Hoi, S. (2022) · arXiv · doi:10.48550/arxiv.2201.12086 · Cited by 1,741 · see citing works
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555 · see citing works
Preparing a collection of radiology examinations for distribution and retrieval
Demner-Fushman, D., Kohli, M. D., Rosenman, M. B., Shooshan, S. E., Rodriguez, L., Antani, S., Thoma, G. R., & McDonald, C. J. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv080 · Cited by 1,164 · see citing works
Finetuned Language Models Are Zero-Shot Learners
Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2021) · arXiv · doi:10.48550/arxiv.2109.01652 · Cited by 1,101 · see citing works
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
Li, J., Selvaraju, R. R., Gotmare, A. D., Joty, S., Xiong, C., & Hoi, S. (2021) · arXiv · doi:10.48550/arxiv.2107.07651 · Cited by 1,078 · see citing works
Quantifying Attention Flow in Transformers
Abnar, S., & Zuidema, W. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.385 · Cited by 995 · see citing works
End-to-End Change Detection for High Resolution Satellite Images Using Improved UNet++
Peng, D., Zhang, Y., & Guan, H. (2019) · Remote Sensing · doi:10.3390/rs11111382 · Cited by 805 · see citing works
VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-ExpertsPreprint
Bao, H., Wang, W.-H., Dong, L., Liu, Q., Mohammed, O., Aggarwal, K., Som, S., Piao, S.-H., & Wei, F. (2021) · Neural Information Processing Systems · doi:10.48550/arxiv.2111.02358 · Cited by 803 · see citing works
Self-citationLowekamp, B., Chen, D. T., Ibáñez, L., & Blezek, D. J. (2013) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2013.00045 · Cited by 797 · see citing works
ConViT: improving vision transformers with soft convolutional inductive biases
d’Ascoli, S., Touvron, H., Leavitt, M. L., Morcos, A. S., Biroli, G., & Sagun, L. (2022) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/ac9830 · Cited by 782 · see citing works
Optimization of mutual information for multiresolution image registration
Unser, M. A., & Thévenaz, P. (2000) · IEEE Transactions on Image Processing · doi:10.1109/83.887976 · Cited by 694 · see citing works
Generating Radiology Reports via Memory-driven Transformer
Chen, Z., Song, Y., Chang, T.-H., & Wan, X. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.112 · Cited by 693 · see citing works
Deep Learning Predicts Lung Cancer Treatment Response from Serial Medical Imaging
Xu, Y., Hosny, A., Zeleznik, R., Parmar, C., Coroller, T. P., Franco, I. “., Mak, R. H., & Aerts, H. J. W. L. (2019) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-18-2495 · Cited by 635 · see citing works
Change Detection Based on Artificial Intelligence: State-of-the-Art and Challenges
Shi, W., Zhang, M., Zhang, R., Chen, S., & Zhan, Z. (2020) · Remote Sensing · doi:10.3390/rs12101688 · Cited by 599 · see citing works
FLAVA: A Foundational Language And Vision Alignment Model
Singh, A., Hu, R., Goswami, V., Couairon, G., Galuba, W., Rohrbach, M., & Kiela, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01519 · Cited by 597 · see citing works
Generating Wikipedia by Summarizing Long SequencesPreprint
Liu, P. J., Saleh, M., Pot, E., Goodrich, B., Sepassi, R., Kaiser, Ł., & Shazeer, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.10198 · Cited by 578 · see citing works
Huang, S.-C., Shen, L., Lungren, M. P., & Yeung, S. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00391 · Cited by 489 · see citing works
Self-citationContrastive Learning of Medical Visual Representations from Paired Images and TextPreprint
Zhang, Y., Jiang, H., Miura, Y., Manning, C. D., & Langlotz, C. P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.00747 · Cited by 465 · see citing works
Debiased Contrastive LearningPreprint
Chuang, C.-Y., Robinson, J., Lin, Y.-C., Torralba, A. B., & Jegelka, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.00224 · Cited by 395 · see citing works
Rethinking and Improving Relative Position Encoding for Vision Transformer
Wu, K., Peng, H., Chen, M., Fu, J., & Chao, H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00988 · Cited by 393 · see citing works
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301590 · Cited by 392 · see citing works
Self-citationFILIP: Fine-grained Interactive Language-Image Pre-TrainingPreprint
Yao, L., Huang, R., Hou, L., Lu, G., Niu, M., Xu, H., Liang, X., Li, Z., Jiang, X., & Xu, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.07783 · Cited by 388 · see citing works
An Empirical Study of Training End-to-End Vision-and-Language Transformers
Dou, Z.-Y., Xu, Y., Gan, Z., Wang, J., Wang, S., Wang, L., Zhu, C., Zhang, P., Yuan, L., Peng, N., et al. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01763 · Cited by 366 · see citing works
How Do Vision Transformers Work?
Park, N., & Kim, S. (2022) · arXiv · doi:10.48550/arxiv.2202.06709 · Cited by 356 · cited 18× · see citing works
“Table 6 shows that the proposed framework greatly increases the sensitivity of sentence embeddings to temporal content whilst better capturing the static content (RadNLI).”
BackgroundInfluentialVirTex: Learning Visual Representations from Textual Annotations
Desai, K., & Johnson, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01101 · Cited by 350 · see citing works
Shih, G., Wu, C. C., Halabi, S. S., Kohli, M. D., Prevedello, L. M. S., Cook, T. S., Sharma, A., Amorosa, J. K., Arteaga, V. A., Galperin-Aizenberg, M., et al. (2019) · Radiology Artificial Intelligence · doi:10.1148/ryai.2019180041 · Cited by 341 · see citing works
VLMo: Unified Vision-Language Pre-Training with Mixture-Of-Modality-Experts
Bao, H., Wang, W., Dong, L., Liu, Q., Mohammed, O. K., Aggarwal, K., Som, S., Piao, S., & Wei, F. (2022) · neural information processing systems · doi:10.52202/068431-2384 · Cited by 336 · see citing works
Self-citationMaking the Most of Text Semantics to Improve Biomedical Vision-Language Processing
Boecking, B., Usuyama, N., Bannur, S., Castro, D. C., Schwaighofer, A., Hyland, S., Wetscherek, M., Naumann, T., Nori, A., Alvarez-Valle, J., et al. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20059-5_1 · Cited by 317 · cited 10× · see citing works
“Despite this, the existing VLP work to date considers alignment between only single images and reports [9,32,46,81], going so far as to remove temporal content from reports in training data to prevent ‘hallucinations’ in downstream report generation [54].”
BackgroundMethodInfluentialSelf-citationSelf-supervised Co-training for Video Representation Learning
Han, T., Xie, W., & Zisserman, A. (2020) · arXiv · doi:10.48550/arxiv.2010.09709 · Cited by 292 · see citing works
Smit, A., Jain, S., Rajpurkar, P., Pareek, A., Ng, A., & Lungren, M. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.117 · Cited by 262 · see citing works
Self-citationMoon, J. H., Lee, H., Shin, W., Kim, Y., & Choi, E. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3207502 · Cited by 221 · see citing works
Chest x-ray findings and temporal lung changes in patients with COVID-19 pneumonia
Rousan, L. A., El-Obeid, E., Karrar, M., & Khader, Y. S. (2020) · BMC Pulmonary Medicine · doi:10.1186/s12890-020-01286-5 · Cited by 210 · see citing works
Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation
Miura, Y., Zhang, Y., Tsai, E., Langlotz, C., & Jurafsky, D. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.416 · Cited by 149 · see citing works
Exploring the Limits of Out-of-Distribution Detection
Fort, S., Ren, J., & Lakshminarayanan, B. (2021) · arXiv · doi:10.48550/arxiv.2106.03004 · Cited by 142 · see citing works
Durrleman, S., Pennec, X., Trouvé, A., Braga, J., Gerig, G., & Ayache, N. (2012) · International Journal of Computer Vision · doi:10.1007/s11263-012-0592-x · Cited by 125 · see citing works
Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIP
Esmaeilpour, S., Liu, B., Robertson, E., & Shu, L. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i6.20610 · Cited by 122 · see citing works
Biomedical and clinical English model packages for the Stanza Python NLP library
Zhang, Y., Zhang, Y., Qi, P., Manning, C. D., & Langlotz, C. P. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab090 · Cited by 112 · see citing works
Smit, A., Jain, S., Rajpurkar, P., Pareek, A., Ng, A. Y., & Lungren, M. P. (2020) · arXiv · doi:10.48550/arxiv.2004.09167 · Cited by 103 · see citing works
Self-citationClinical-BERT: Vision-Language Pre-training for Radiograph Diagnosis and Reports Generation
Yan, B., & Pei, M. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20204 · Cited by 94 · see citing works
Broaden Your Views for Self-Supervised Video Learning
Recasens, A., Luc, P., Alayrac, J.-B., Wang, L., Strub, F., Tallec, C., Malinowski, M., Pătrăucean, V., Altché, F., Valko, M., et al. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00129 · Cited by 90 · see citing works
Robust Contrastive Learning against Noisy Views
Chuang, C.-Y., Hjelm, R. D., Wang, X., Vineet, V., Joshi, N., Torralba, A. B., Jegelka, S., & Song, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01617 · Cited by 79 · see citing works
RadGraph: Extracting Clinical Entities and Relations from Radiology Reports
Jain, S., Agrawal, A., Saporta, A., Truong, S. Q., Duong, D. N., Bui, T., Chambon, P., Zhang, Y., Lungren, M. P., Ng, A. Y., et al. (2021) · arXiv · doi:10.48550/arxiv.2106.14463 · Cited by 66 · see citing works
Self-citationInfluence of prior radiologic information on the interpretation of radiographic examinations
Aideyan, U. O., Berbaum, K. S., & Smith, W. L. (1995) · Academic Radiology · doi:10.1016/s1076-6332(05)80165-5 · Cited by 66 · see citing works
Vu, Y. N. T., Wang, R., Balachandar, N., Liu, C., Ng, A. Y., & Rajpurkar, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.10663 · Cited by 63 · see citing works
COVID-19 Prognosis via Self-Supervised Representation Learning and Multi-Image PredictionPreprint
Sriram, A., Muckley, M. J., Sinha, K., Shamout, F. E., Pineau, J., Geras, K. J., Azour, L., Aphinyanaphongs, Y., Yakubova, N., & Moore, W. H. (2021) · PubMed · doi:10.48550/arxiv.2101.04909 · Cited by 50 · see citing works
Wang, C., Rimner, A., Hu, Y., Tyagi, N., Jiang, J., Yorke, E. D., Riyahi, S., Mageras, G. S., Deasy, J. O., & Zhang, P. (2019) · Medical Physics · doi:10.1002/mp.13765 · Cited by 48 · see citing works
Longitudinal Detection of Radiological Abnormalities with Time-Modulated LSTM
Santeramo, R., Withey, S., & Montana, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00889-5_37 · Cited by 35 · see citing works
Long Short View Feature Decomposition via Contrastive Video Representation Learning
Behrmann, N., Fayyaz, M., Gall, J., & Noroozi, M. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00911 · Cited by 30 · see citing works
Contrastive Learning of Global and Local Video Representations
Ma, S., Zeng, Z., McDuff, D., & Song, Y. (2021) · neural information processing systems · Cited by 23 · see citing works
CheXRelNet: An Anatomy-Aware Model for Tracking Longitudinal Relationships Between Chest X-Rays
Karwande, G., Mbakwe, A. B., Wu, J. T.-Y., Celi, L. A., Moradi, M., & Lourentzou, I. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16431-6_55 · Cited by 23 · see citing works
Data Efficient Masked Language Modeling for Vision and Language
Bitton, Y., Elhadad, M., Stanovsky, G., & Schwartz, R. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.259 · Cited by 18 · see citing works
A nonparametric-based rib suppression method for chest radiographs
Lee, J.-S., Wang, J.-W., Wu, H.-H., & Yuan, M.-Z. (2012) · Computers & Mathematics with Applications · doi:10.1016/j.camwa.2012.03.084 · Cited by 18 · see citing works
Ramesh, V., Chi, N. A., & Rajpurkar, P. (2022) · arXiv · doi:10.48550/arxiv.2210.06340 · Cited by 17 · see citing works
Longitudinal Change Detection on Chest X-rays Using Geometric Correlation Maps
Oh, D. Y., Kim, J., & Lee, K. J. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32226-7_83 · Cited by 14 · see citing works
Time Is MattEr: Temporal Self-supervision for Video TransformersPreprint
Yun, S., Kim, J., Han, D., Song, H., Ha, J.-W., & Shin, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.09067 · Cited by 9 · see citing works
Leveraging Time Irreversibility with Order-Contrastive Pre-trainingPreprint
Agrawal, M., Lang, H., Offin, M. D., Gazit, L., & Sontag, D. A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.02599 · Cited by 4 · see citing works
Zeng, D., Kheir, J. N., Zeng, P., & Shi, Y. (2021) · International Conference on Computer Aided Design · doi:10.1109/iccad51958.2021.9643443 · Cited by 4 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
1 of the 80 references couldn’t be matched to a known paper.
Research works printed but not linked: 4
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Lan- guage models are few-shot learners. Advances in neural in- formation processing systems, 33:1877-1901, 2020. 5, 19
- Mark Endo, Rayan Krishnan, Viswesh Krishna, Andrew Y Ng, and Pranav Rajpurkar. Retrieval-based chest x-ray report generation using a pre-trained contrastive language-image model. In Machine Learning for Health, pages 209-219. PMLR, 2021. 5, 6, 18
- Ilya Loshchilov and Frank Hutter. Decoupled weight de- cay regularization. In International Conference on Learning Representations, 2018. 17, 19
- Zhaoyang Zeng, Daniel McDuff, Yale Song, et al. Con- trastive learning of global and local video representa- tions. Advances in Neural Information Processing Systems, 34:7025-7040, 2021. 3
Cited by
Sorted by citations
Advances in medical image analysis with vision Transformers: A comprehensive review
Azad, R., Kazerouni, A., Heidari, M., Aghdam, E. K., Molaei, A., Jia, Y., Jose, A., Roy, R., & Merhof, D. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.103000 · Cited by 523 · see citing works
Towards Generalist Biomedical AI
Tu, T., Azizi, S., Driess, D., Schaekermann, M., Amin, M., Chang, P.-C., Carroll, A., Lau, C., Tanno, R., Ktena, I., et al. (2024) · NEJM AI · doi:10.1056/aioa2300138 · Cited by 483 · see citing works
The current and future state of AI interpretation of medical images
Rajpurkar, P., & Lungren, M. P. (2023) · New England Journal of Medicine · doi:10.1056/nejmra2301725 · Cited by 443 · see citing works
Self-citationVision-language models for medical report generation and visual question answering: a review
Hartsock, I., & Rasool, G. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1430984 · Cited by 257 · see citing works
Collaboration between clinicians and vision–language models in radiology report generation
Tanno, R., Barrett, D. G. T., Sellergren, A., Ghaisas, S., Dathathri, S., See, A., Welbl, J., Lau, C., Tu, T., Azizi, S., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03302-1 · Cited by 186 · see citing works
CLIP in medical imaging: A survey
Zhao, Z., Liu, Y., Wu, H., Wang, M., Li, Y., Wang, S., Teng, L., Liu, D., Cui, Z., Wang, Q., et al. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103551 · Cited by 175 · see citing works
Multimodal generative AI for medical image interpretation
Rao, V. M., Hla, M., Moor, M., Adithan, S., Kwak, S., Topol, E. J., & Rajpurkar, P. (2025) · Nature · doi:10.1038/s41586-025-08675-y · Cited by 128 · see citing works
Multimodal Large Language Models in Medical Imaging: Current State and Future Directions
Nam, Y., Kim, D., Kyung, S., Seo, J., Song, J. M., Kwon, J., Kim, J., Jo, W., Park, H. J., Sung, J., et al. (2025) · Korean Journal of Radiology · doi:10.3348/kjr.2025.0599 · Cited by 106 · see citing works
Exploring scalable medical image encoders beyond text supervision
Pérez-García, F., Sharma, H., Bond-Taylor, S., Bouzid, K., Salvatelli, V., Ilse, M., Bannur, S., Castro, D. C., Schwaighofer, A., Lungren, M. P., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-024-00965-w · Cited by 115 · see citing works
Self-citationAutomated Radiology Report Generation: A Review of Recent Advances
Sloan, P., Clatworthy, P. L., Simpson, E., & Mirmehdi, M. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2024.3408456 · Cited by 98 · see citing works
Yildirim, N., Richardson, H., Wetscherek, M. T., Bajwa, J., Jacob, J., Pinnock, M. A., Harris, S., Coelho De Castro, D., Bannur, S., Hyland, S., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642013 · Cited by 82 · see citing works
Self-citationArtificial Intelligence in Cardiovascular Care—Part 2: Applications
Jain, S. S., Elias, P., Poterucha, T., Randazzo, M., Lopez Jimenez, F., Khera, R., Perez, M., Ouyang, D., Pirruccello, J., Salerno, M., et al. (2024) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2024.03.401 · Cited by 74 · see citing works
Has multimodal learning delivered universal intelligence in healthcare? A comprehensive survey
Lin, Q., Zhu, Y., Mei, X., Huang, L., Ma, J., He, K., Peng, Z., Cambria, E., & Feng, M. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102795 · Cited by 76 · see citing works
Reale-Nosei, G., Amador-Domínguez, E., & Serrano, E. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103264 · Cited by 68 · see citing works
Huang, W., Li, C., Zhou, H.-Y., Yang, H., Liu, J., Liang, Y., Zheng, H., Zhang, S., & Wang, S. (2024) · Nature Communications · doi:10.1038/s41467-024-51749-0 · Cited by 62 · see citing works
Semi-Supervised Medical Report Generation via Graph-Guided Hybrid Feature Consistency
Zhang, K., Jiang, H., Zhang, J., Huang, Q., Jianping Fan, B., Yu, J., & Han, W. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3273390 · Cited by 63 · see citing works
Medical multimodal multitask foundation model for lung cancer screening
Niu, C., Lyu, Q., Carothers, C. D., Kaviani, P., Tan, J., Yan, P., Kalra, M. K., Whitlow, C. T., & Wang, G. (2025) · Nature Communications · doi:10.1038/s41467-025-56822-w · Cited by 60 · see citing works
Buess, L., Keicher, M., Navab, N., Maier, A., & Arasteh, S. T. (2025) · Biomedical Engineering Letters · doi:10.1007/s13534-025-00497-1 · Cited by 71 · see citing works
BrainSegFounder: Towards 3D foundation models for neuroimage segmentation
Cox, J., Liu, P., Stolte, S. E., Yang, Y., Liu, K., See, K. B., Ju, H., & Fang, R. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103301 · Cited by 56 · see citing works
Efficiency and Quality of Generative AI–Assisted Radiograph Reporting
Huang, J., Wittbrodt, M. T., Teague, C. N., Karl, E., Galal, G., Thompson, M., Chapa, A. R., Chiu, M.-L., Herynk, B. A., Linchangco, R., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.13921 · Cited by 54 · see citing works
LongFormer: Longitudinal Transformer for Alzheimer’s Disease Classification with Structural MRIs
Chen, Q., Fu, Q., Bai, H., & Hong, Y. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00354 · Cited by 52 · see citing works
MedBLIP: Bootstrapping Language-Image Pretraining from 3D Medical Images and Texts
Chen, Q., & Hong, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0908-6_6 · Cited by 57 · see citing works
Sellergren, A., Kazemzadeh, S., Jaroensri, T., Kiraly, A., Traverse, M., Kohlberger, T., Xu, S., Jamil, F., Hughes, C., Lau, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2507.05201 · Cited by 49 · see citing works
Sun, K., Xue, S., Sun, F., Sun, H., Luo, Y., Wang, L., Wang, S., Guo, N., Liu, L., Zhao, T., et al. (2025) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2025.103265 · Cited by 61 · see citing works
Zambrano Chaves, J. M., Huang, S.-C., Xu, Y., Xu, H., Usuyama, N., Zhang, S., Wang, F., Xie, Y., Khademi, M., Yang, Z., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-58344-x · Cited by 45 · see citing works
“For synthetic reports without reference ground truths, we use a similarity comparison (cosine similarity) between the embeddings of corresponding CXR images and synthetic reports, as determined by BiomedCLIP-CXR and BioViL-T 68 .”
Self-citationToken-Mixer: Bind Image and Text in One Embedding Space for Medical Image Reporting
Yang, Y. Q., Yu, J., Fu, Z., Zhang, K., Yu, T., Wang, X., Jiang, H., Lv, J., Huang, Q., & Han, W. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3412402 · Cited by 42 · see citing works
Radiology and multi-scale data integration for precision oncology
Paverd, H., Zormpas-Petridis, K., Clayton, H., Burge, S., & Crispin-Ortuzar, M. (2024) · npj Precision Oncology · doi:10.1038/s41698-024-00656-0 · Cited by 41 · see citing works
A survey of deep-learning-based radiology report generation using multimodal inputs
Wang, X., Figueredo, G. P., Li, R., Zhang, W. E., Chen, W., & Chen, X. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103627 · Cited by 50 · see citing works
Ardic, N., & Dinc, R. (2025) · Health Informatics Journal · doi:10.1177/14604582251366141 · Cited by 32 · see citing works
Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts
Chen, Z., Diao, S., Wang, B., Li, G., & Wan, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02139 · Cited by 34 · see citing works
CARZero: Cross-Attention Alignment for Radiology Zero-Shot Classification
Lai, H., Yao, Q., Jiang, Z., Wang, R., He, Z., Tao, X., & Zhou, S. K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01059 · Cited by 33 · see citing works
A Comprehensive Study of GPT-4V’s Multimodal Capabilities in Medical ImagingPreprint
Li, Y., Liu, Y., Wang, Z., Liang, X., Liu, L., Wang, L., Cui, L., Tu, Z., Wang, L., & Zhou, L. (2023) · medRxiv · doi:10.1101/2023.11.03.23298067 · Cited by 28 · see citing works
Towards Generalist Biomedical AI
Tu, T., Azizi, S., Driess, D., Schaekermann, M., Amin, M., Chang, P.-C., Carroll, A., Lau, C., Tanno, R., Ktena, I., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.14334 · Cited by 483 · see citing works
Longitudinal data and a semantic similarity reward for chest X-ray report generation
Nicolson, A., Dowling, J., Anderson, D., & Koopman, B. (2024) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2024.101585 · Cited by 33 · see citing works
Wu, J., He, K., Mao, R., Shang, X., & Cambria, E. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103283 · Cited by 28 · see citing works
Chest X-Ray Report Generation Using Abnormality Guided Vision Language Model
Edirisinghe, D., Nimalsiri, W., Hennayake, M., Meedeniya, D., & Lim, G. (2025) · IEEE Access · doi:10.1109/access.2025.3606961 · Cited by 28 · see citing works
A Comprehensive Review of Multimodal Large Language Models for Medical Imaging and Omics Data
Vavekanand, R. (2026) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-026-10504-y · Cited by 29 · see citing works
Phan, V. M. H., Xie, Y., Qi, Y., Liu, L., Liu, L., Zhang, B., Liao, Z., Wu, Q., To, M., & Verjans, J. W. H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01092 · Cited by 26 · see citing works
A comprehensive survey of AI agents in healthcare
Xu, G., Li, X., Chen, Y., Duan, Y., Wu, S., Yu, H., Chiu, C.-H., Ni, J., Tang, N., Li, T. J.-J., et al. (2026) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2026.105045 · Cited by 32 · see citing works
A systematic evaluation of GPT-4V's multimodal capability for chest X-ray image analysis
Liu, Y., Li, Y., Wang, Z., Liang, X., Liu, L., Wang, L., Cui, L., Tu, Z., Wang, L., & Zhou, L. (2024) · Meta-Radiology · doi:10.1016/j.metrad.2024.100099 · Cited by 25 · see citing works
Park, S. J., Lee, E. S., Shin, K. S., Lee, J. E., & Ye, J. C. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.103021 · Cited by 25 · see citing works
Exploring the Boundaries of GPT-4 in Radiology
Liu, Q., Hyland, S. L., Bannur, S., Bouzid, K., Castro, D. C., Wetscherek, M. T., Tinn, R., Sharma, H., Pérez‐García, F., Schwaighofer, A., et al. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.891 · Cited by 29 · see citing works
Self-citationIntegrating language into medical visual recognition and reasoning: A survey
Lu, Y., & Wang, A. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103514 · Cited by 26 · see citing works
Spatio-Temporal and Retrieval-Augmented Modeling for Chest X-Ray Report Generation
Yang, Y. Q., You, X., Zhang, K., Fu, Z., Wang, X., Ding, J., Sun, J., Zhou, Y., Huang, Q., Han, W., et al. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3554498 · Cited by 23 · see citing works
Gubbala, S. (2026) · Learning and analytics in intelligent systems · doi:10.1007/978-3-032-29501-9_39 · Cited by 20 · see citing works
Anatomical Structure-Guided Medical Vision-Language Pre-training
Li, Q., Yan, X., Xu, J., Yuan, R., Zhang, Y., Feng, R., Shen, Q., Zhang, X., & Wang, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72120-5_8 · Cited by 20 · see citing works
HERGen: Elevating Radiology Report Generation with Longitudinal Data
Wang, F., Du, S., & Yu, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73001-6_11 · Cited by 23 · see citing works
EchoFM: Foundation Model for Generalizable Echocardiogram Analysis
Kim, S., Jin, P., Song, S., Chen, C., Li, Y., Ren, H., Li, X., Liu, T., & Li, Q. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3580713 · Cited by 22 · see citing works
RECAP: Towards Precise Radiology Report Generation via Dynamic Disease Progression Reasoning
Hou, W., Cheng, Y., Xu, K., Li, W., & Liu, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.140 · Cited by 19 · see citing works
Controllable Chest X-Ray Report Generation from Longitudinal Representations
Serra, F., Wang, C., Deligianni, F., Dalton, J., & O’Neil, A. Q. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.325 · Cited by 20 · see citing works
MAIRA-2: Grounded Radiology Report Generation
Bannur, S., Bouzid, K., Castro, D. C., Schwaighofer, A., Thieme, A., Bond-Taylor, S., Ilse, M., Pérez-García, F., Salvatelli, V., Sharma, H., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.04449 · Cited by 19 · see citing works
“MAIRA-2 is built with a similar architecture to MAIRA-1 (Hyland et al., 2023), based on the LLaVA framework (Liu et al., 2023b;a).”
BackgroundMethodResultInfluentialSelf-citationA Review of Non-Fully Supervised Deep Learning for Medical Image Segmentation
Zhang, X., Wang, J., Wei, J., Yuan, X., & Wu, M. (2025) · Information · doi:10.3390/info16060433 · Cited by 16 · see citing works
Automatic Medical Report Generation: Methods and Applications
Li, G., Tahir, A. M., Zhang, D., Wang, Z. J., & Ward, R. K. (2024) · APSIPA Transactions on Signal and Information Processing · doi:10.1561/116.20240044 · Cited by 19 · see citing works
RadEdit: Stress-Testing Biomedical Vision Models via Diffusion Image Editing
Pérez‐García, F., Bond-Taylor, S., Sanchez, P. P., van Breugel, B., Castro, D. C., Sharma, H., Salvatelli, V., Wetscherek, M. T., Richardson, H., Lungren, M. P., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73254-6_21 · Cited by 16 · see citing works
Self-citationTowards Interpretable Radiology Report Generation via Concept Bottlenecks Using a Multi-agentic RAG
Alam, H., Srivastav, D., Kadir, M. A., & Sonntag, D. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88714-7_18 · Cited by 19 · see citing works
Angeloni, M., Rizzi, D., Schoen, S., Caputo, A., Merolla, F., Hartmann, A., Ferrazzi, F., & Fraggetta, F. (2025) · Genome Medicine · doi:10.1186/s13073-025-01484-y · Cited by 16 · see citing works
Medical multimodal large language models: A systematic review
Hu, Y., Xu, C., Lin, B., Yang, W., & Tang, Y. Y. (2025) · Intelligent Oncology · doi:10.1016/j.intonc.2025.09.005 · Cited by 14 · see citing works
A topic modeling‐based bibliometric exploration of automatic summarization research
Chen, X., Xie, H., Tao, X., Xu, L., Wang, J., Dai, H., & Wang, F. L. (2024) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1540 · Cited by 14 · see citing works
A Systematic Review and Implementation Guidelines of Multimodal Foundation Models in Medical ImagingPreprint
Huang, S.-C., Jensen, M., Yeung, S. Y., Lungren, M. P., Poon, H., & Chaudhari, A. S. (2025) · Research Square · doi:10.21203/rs.3.rs-5537908/v1 · Cited by 17 · see citing works
Self-citationLearning neuroimaging models from health system-scale data
Lyu, Y., Harake, E. S., Chowdury, A., Banerjee, S., Gologorsky, R., Liu, S., Meißner, A.-K., Rao, A., Zhao, C., Kondepudi, A., et al. (2026) · Nature Biomedical Engineering · doi:10.1038/s41551-025-01608-0 · Cited by 14 · see citing works
Multimodal Foundation Models for Medical Imaging - A Systematic Review and Implementation GuidelinesPreprint
Huang, S.-C., Jensen, M., Yeung, S. Y., Lungren, M. P., Poon, H., & Chaudhari, A. S. (2024) · medRxiv · doi:10.1101/2024.10.23.24316003 · Cited by 15 · see citing works
Self-citationAI-Driven Digital Pathology: Deep Learning and Multimodal Integration for Precision Oncology
Jang, H., & Lee, S. H. (2025) · International Journal of Molecular Sciences · doi:10.3390/ijms27010379 · Cited by 12 · see citing works
ChEX: Interactive Localization and Region Description in Chest X-Rays
Müller, P., Kaissis, G. A., & Rueckert, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72664-4_6 · Cited by 12 · see citing works
Thieme, A., Rajamohan, A. M., Cooper, B., Groombridge, H., Simister, R. J., Wong, B., Woznitza, N., Pinnock, M. A., Wetscherek, M. T., Morrison, C. P., et al. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3716500 · Cited by 12 · see citing works
Self-citationECAMP: Entity-centered Context-aware Medical Vision Language Pre-training
Wang, R., Yao, Q., Jiang, Z., Lai, H., He, Z., Tao, X., & Zhou, S. K. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103690 · Cited by 16 · see citing works
Enabling Multi-modal Conversational Interface for Clinical Imaging
Dayanandan, K., & Lall, B. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650805 · Cited by 9 · see citing works
Recent Advances in Medical Image Classification
Dao, L., & Ly, N. Q. (2024) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2024.0150727 · Cited by 12 · see citing works
Song, S., Tang, H., Yang, H., & Li, X. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3591364 · Cited by 12 · see citing works
Foundation Models Meet Medical Image Interpretation
Jiao, L., Hao, J., Li, R., Li, L., Liu, X., Liu, F., Ma, W., Chen, P., Huang, Z., Yang, J., et al. (2026) · Research · doi:10.34133/research.1024 · Cited by 9 · see citing works
Vavekanand, R., & Kumar, T. (2026) · Signal Image and Video Processing · doi:10.1007/s11760-026-05214-8 · Cited by 9 · see citing works
Al-Jebrni, A. H., Jiang, Z., Qin, Y., Qiu, Y., Jiang, N., Li, G., Zheng, Y., Wong, T. Y., Kim, S., Jung, Y., et al. (2026) · Computational Visual Media · doi:10.26599/cvm.2026.9450530 · Cited by 9 · see citing works
MAGDA: Multi-agent Guideline-Driven Diagnostic Assistance
Bani-Harouni, D., Navab, N., & Keicher, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73471-7_17 · Cited by 8 · see citing works
Medical Report Generation via Multimodal Spatio-Temporal Fusion
Mei, X., Mao, R., Cai, X., Yang, L., & Cambria, E. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681377 · Cited by 8 · see citing works
Li, Y., Kong, C., Zhao, G., & Zhao, Z. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11337-0 · Cited by 7 · see citing works
Multimodal transformers for joint diagnosis and retrieval from chest X-rays and radiology reports
Jahanian, M., Karimi, A., Eraghi, N. O., & Zarafshan, F. (2025) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2025.101672 · Cited by 7 · see citing works
Kim, H., Karaman, B. K., Zhao, Q., Wang, A. Q., Sabuncu, M. R., Initiative, A. D. N., Weiner, M. W., Aisen, P. S. S., Petersen, R. C., Jack, C. R., et al. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2411492122 · Cited by 7 · see citing works
Artificial Intelligence in Pathology: Advancing Large Models for Scalable Applications
Xiao, Z., Feng, B., Yang, J., Sun, G., Shen, Y., Xu, S., Yang, L., Xu, H., Zhang, M., & Wang, S. (2025) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-103123-095814 · Cited by 9 · see citing works
Clothing Purification with Causality Meets Vision-Language Pretraining Models
Yang, Z., Zhu, H., Lei, N., Fernando, B., & Wang, Z. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02548-7 · Cited by 7 · see citing works
NeoCLIP: a self-supervised foundation model for the interpretation of neonatal radiographs
Huang, Y., Sharma, P. K., Palepu, A. K., Greenbaum, N. R., Beam, A. L., & Beam, K. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01922-6 · Cited by 7 · see citing works
Intelligent analysis of chest X-ray based on multi-modal instruction tuning
Yao, J., Wang, J., Xiao, Z., Hao, X., & Jiang, X. (2025) · Meta-Radiology · doi:10.1016/j.metrad.2025.100172 · Cited by 6 · see citing works
Generative AI and foundation models in medical image
Oda, M. (2025) · Radiological Physics and Technology · doi:10.1007/s12194-025-00968-1 · Cited by 6 · see citing works
Systematic Literature Review on Medical Image Captioning Using CNN-LSTM and Transformer-Based Models
Fadhilah, H., & Utama, N. P. (2025) · JURNAL MASYARAKAT INFORMATIKA · doi:10.14710/jmasif.16.1.73127 · Cited by 9 · see citing works
Islam, K., Paul, R., Wang, S., Zhao, Y., Adhikary, P. P., Li, Q., Qin, X., & Liu, Y. (2025) · Microsystems & Nanoengineering · doi:10.1038/s41378-025-00874-x · Cited by 5 · see citing works
Medical Large Vision Language Models with Multi-image Visual Ability
Yang, X., Miao, J., Yuan, Y., Wang, J., Dou, Q., Li, J., & Heng, P. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04971-1_38 · Cited by 6 · see citing works
Phenotyping of Histology Imaging Data with Histomics
Neha, F., Bhati, D., & Shukla, D. K. (2026) · AI · doi:10.3390/ai7060228 · Cited by 5 · see citing works
MemControl: Mitigating Memorization in Diffusion Models via Automated Parameter Selection
Dutt, R., Bohdal, O., Sanchez, P. P., Tsaftaris, S. A., & Hospedales, T. M. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00441 · Cited by 5 · see citing works
Lu, Z., Xu, N., Tian, H., Wang, L., & Liu, A.-A. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3602970 · Cited by 5 · see citing works
Yang, Z., & Shen, L. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00454 · Cited by 4 · see citing works
Yan, J., Gu, X., & Chen, Y. (2025) · Remote Sensing · doi:10.3390/rs17142481 · Cited by 6 · see citing works
Ren, G., Hou, M., Yang, X., Huang, Y., Li, D., Zhang, N., Liu, P., Ye, H., Chen, Z., & Chen, H. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02044-9 · Cited by 4 · see citing works
Helmy, H., Hosseini, A., Ibrahim, A., Mirza, A. B., Sadek, A., & Serag, A. (2026) · Applied Artificial Intelligence · doi:10.1080/08839514.2026.2626117 · Cited by 4 · see citing works
Automated Spinal MRI Labelling from Reports Using a Large Language Model
Park, R. Y., Windsor, R., Jamaludin, A., & Zisserman, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72086-4_10 · Cited by 4 · see citing works
Chen, K., Kuo, M., Lee, C., Liao, H.-C., Tsai, D., Lin, S.-A., Hsiang, C.-W., Chang, C.-K., Ko, K., Hsu, Y., et al. (2025) · Journal of Medical Systems · doi:10.1007/s10916-025-02263-3 · Cited by 4 · see citing works
Application of Vision-Language Models for Assessing Osteoarthritis
Felfeliyan, B., Zhou, Y., Ghosh, S., Küpper, J., Liu, S., Hareendranathan, A. R., & Jaremko, J. L. (2024) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi56570.2024.10635386 · Cited by 4 · see citing works
Unified Medical Image Pre-training in Language-Guided Common Semantic Space
He, X., Yang, Y., Jiang, X., Luo, X., Hu, H. R., Zhao, S., Li, D., Yang, Y., & Qiu, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73004-7_8 · Cited by 5 · see citing works
Scaling Chest X-Ray Foundation Models From Mixed Supervisions for Dense Prediction
Wang, F., & Yu, L. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3589928 · Cited by 4 · see citing works
Generative AI for developing foundation models in radiology and imaging: engineering perspectives
Lee, J., Kyung, S., & Kim, N. (2025) · Biomedical Engineering Letters · doi:10.1007/s13534-025-00517-0 · Cited by 3 · see citing works
Noh, W.-J., Pi, S.-W., & Lee, B.-D. (2026) · Scientific Reports · doi:10.1038/s41598-025-34799-2 · Cited by 5 · see citing works
Kulsoom, U., Glavin, F. G., & Bendechache, M. (2025) · IEEE Access · doi:10.1109/access.2025.3582728 · Cited by 5 · see citing works
MM-CLIP:Multi-granularity Mammography CLIP for Breast Cancer Diagnosis
Wei, H., & Fan, H. (2025) · IEEE International Conference on Digital Signal Processing (DSP) · doi:10.1145/3725988.3726002 · Cited by 3 · see citing works
- Authors
- Bannur, S., Hyland, S., Liu, Q., Pérez-García, F., Ilse, M., Castro, D. C., Boecking, B., Sharma, H., Bouzid, K., Thieme, A., Schwaighofer, A., Wetscherek, M., Lungren, M. P., Nori, A., Alvarez-Valle, J., & Oktay, O.
- Affiliations
- Microsoft (United States) (US)
- Microsoft Health Futures
- ORCID
- Venue
- 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2023-06
- Pages
- 15016-15027
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4386075916
- Cited by
- 227
- References
- 80
- Citations by year
- 2026: 93 · 2025: 85 · 2024: 40 · 2023: 9
- FWCI
- 24.28
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Keywords
- temporal structure, self-supervised learning, clinical reports, phrase grounding, report generation, chest radiography
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
