Cited by 493Multimodal Machine Learning ApplicationsImage Retrieval and Classification TechniquesDomain Adaptation and Few-Shot Learning

The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.

GLoRIA: A Multimodal Global-Local Representation Learning Framework for Label-efficient Medical Image Recognition

Huang, Shih-Cheng, Shen, Liyue, Lungren, Matthew P., Yeung, Serena·2021·2021 IEEE/CVF International Conference on Computer Vision (ICCV)

DOI 10.1109/iccv48922.2021.00391

Open this paper in CoChat — free account

Abstract

In recent years, the growing utilization of medical imaging is placing an increasing burden on radiologists. Deep learning provides a promising solution for automatic medical image analysis and clinical decision support. However, large-scale manually labeled datasets required for training deep neural networks are difficult and expensive to obtain for medical images. The purpose of this work is to develop label-efficient multimodal medical imaging representations by leveraging radiology reports. We propose an attention-based framework for learning global and local representations by contrasting image sub-regions and words in the paired report. In addition, we propose methods to leverage the learned representations for various downstream medical image recognition tasks with limited labels. Our results demonstrate high-performance and label-efficiency for image-text retrieval, classification (finetuning and zerosshot settings), and segmentation on different datasets.

Integrity signals

Registry

Crossref · 10.1109/iccv48922.2021.00391

Cites

41 of 41 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141
Self-citation
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
ChestX-Ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., & Summers, R. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.369 · Cited by 4,616
VQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487
AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks
Xu, T., Zhang, P., Huang, Q., Zhang, H., Gan, Z., Huang, X., & He, X. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00143 · Cited by 1,923
Learning to detect unseen object classes by between-class attribute transfer
Lampert, C. H., Nickisch, H., & Harmeling, S. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206594 · Cited by 1,849

Show all 41 references

Research works printed but not linked: 1

frome, 2013, DeViSE A deep visual-semantic embedding model [J]

Sorted by citations

Items 101–200 of 493

LIMITR: Leveraging Local Information for Medical Image-Text Representation
Dawidowicz, G., Hirsch, E., & Tal, A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01935 · Cited by 18
Bootstrapping Chest CT Image Understanding by Distilling Knowledge from X-Ray Expert Models
Cao, W., Zhang, J., Xia, Y., Mok, T. C. W., Li, Z., Ye, X., Lü, L., Zheng, J., Tang, Y., & Zhang, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01068 · Cited by 23
CausalCLIPSeg: Unlocking CLIP’s Potential in Referring Medical Image Segmentation with Causal Intervention
Chen, Y., Wei, M., Zheng, Z., Hu, J., Shi, Y., Xiong, S., Zhu, X. X., & Mou, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72384-1_8 · Cited by 17
Adapter-Enhanced Hierarchical Cross-Modal Pre-Training for Lightweight Medical Report Generation
Yu, T., Lu, W., Yang, Y. Q., Han, W., Huang, Q., Yu, J., & Zhang, K. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3535699 · Cited by 18
Bridging multi-level gaps: Bidirectional reciprocal cycle framework for text-guided label-efficient segmentation in echocardiography
Zhang, Z., Zhang, H., Zeng, T., Yang, G., Shi, Z., & Gao, Z. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103536 · Cited by 17
Chest X-Ray Foundation Model With Global and Local Representations Integration
Yang, Z., Xu, X., Zhang, J., Wang, G., Kalra, M. K., & Yan, P. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3581907 · Cited by 17
Radio DINO: A foundation model for advanced radiomics and AI-driven medical imaging analysis
Zedda, L., Loddo, A., & Ruberto, C. D. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110583 · Cited by 23
Fusion of X-Ray Images and Clinical Data for a Multimodal Deep Learning Prediction Model of Osteoporosis: Algorithm Development and Validation Study
Tang, J., Yin, X. Y., Lai, J., Luo, K., & Wu, D. (2025) · JMIR Medical Informatics · doi:10.2196/70738 · Cited by 17
Controllable Text-to-Image Synthesis for Multi-Modality MR Images
Kim, K., Na, Y., Ye, S., Lee, J., Ahn, S. S., Park, J. E., & Kim, H. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00775 · Cited by 22
UniBrain: Universal Brain MRI diagnosis with hierarchical knowledge-enhanced pre-training
Lei, J., Dai, L., Jiang, H., Wu, C., Zhang, X., Zhang, Y., Yao, J., Xie, W., Zhang, Y., Li, Y., et al. (2025) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2025.102516 · Cited by 17
A Systematic Review and Implementation Guidelines of Multimodal Foundation Models in Medical ImagingPreprint
Huang, S.-C., Jensen, M., Yeung, S. Y., Lungren, M. P., Poon, H., & Chaudhari, A. S. (2025) · Research Square · doi:10.21203/rs.3.rs-5537908/v1 · Cited by 17
Self-citation
Echo-Vision-FM: a pre-training and fine-tuning framework for echocardiogram video vision foundation model
Zhang, Z., Wu, Q., Ding, S., Wang, X., & Ye, J. (2025) · Nature Communications · doi:10.1038/s41467-025-66340-4 · Cited by 19
ViLLA: Fine-Grained Vision-Language Representation Learning from Real-World Data
Varma, M., Delbrouck, J.-B., Hooper, S., Chaudhari, A. S., & Langlotz, C. P. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02031 · Cited by 17
Enhancing the vision–language foundation model with key semantic knowledge-emphasized report refinement
Huang, W., Li, C., Yang, H., Liu, J., Liang, Y., Zheng, H., & Wang, S. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103299 · Cited by 19
ECAMP: Entity-centered Context-aware Medical Vision Language Pre-training
Wang, R., Yao, Q., Jiang, Z., Lai, H., He, Z., Tao, X., & Zhou, S. K. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103690 · Cited by 17
MedFILIP: Medical Fine-Grained Language-Image Pre-Training
Liang, X., Li, X., Li, F., Jiang, J., Dong, Q., Wang, W., Wang, K., Dong, S., Luo, G., & Li, S. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3528196 · Cited by 20
Exploring the Transfer Learning Capabilities of CLIP in Domain Generalization for Diabetic Retinopathy
Baliah, S., Maani, F. A., Sanjeev, S., & Khan, M. H. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-45673-2_44 · Cited by 15
Freeze the Backbones: a Parameter-Efficient Contrastive Approach to Robust Medical Vision-Language Pre-Training
Qin, J., Liu, C., Cheng, S. L., Guo, Y., & Arcucci, R. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447326 · Cited by 17
Enhancing Label-Efficient Medical Image Segmentation with Text-Guided Diffusion Models
Feng, C.-M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72111-3_24 · Cited by 15
Knowledge-Enhanced Visual-Language Pretraining for Computational Pathology
Zhou, X., Zhang, X., Wu, C., Zhang, Y., Xie, W., & Wang, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72943-0_20 · Cited by 20
Multimodal Foundation Models for Medical Imaging - A Systematic Review and Implementation GuidelinesPreprint
Huang, S.-C., Jensen, M., Yeung, S. Y., Lungren, M. P., Poon, H., & Chaudhari, A. S. (2024) · medRxiv · doi:10.1101/2024.10.23.24316003 · Cited by 15
Self-citation
RadCLIP: Enhancing Radiologic Image Analysis Through Contrastive Language–Image Pretraining
Lu, Z., Li, H., Parikh, N. A., Dillman, J. R., & He, L. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3568036 · Cited by 19
Trade-Offs in Fine-Tuned Diffusion Models between Accuracy and Interpretability
Dombrowski, M., Reynaud, H., Müller, J. P., Baugh, M., & Kainz, B. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i19.30095 · Cited by 20
Foundation Models: From Current Developments, Challenges, and Risks to Future Opportunities
Hussain, A., Ali, S., Farwa, U. E., Mozumder, M. A. I., & Kim, H.-C. (2025) · International Conference on Advanced Communications Technology (ICACT) · doi:10.23919/icact63878.2025.10936649 · Cited by 16
Vision-language foundation model for generalizable nasal disease diagnosis using unlabeled endoscopic records
Liu, X., Gong, W., Chen, X., Li, Z., Li, Z., Wang, L., Liu, Q., Sun, X., Liu, X., Chen, X., et al. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111646 · Cited by 14
Beyond Medical Imaging - A Review of Multimodal Deep Learning in Radiology
Heiliger, L., Sekuboyina, A., Menze, B., Egger, J., & Kleesiek, J. (2022) · doi:10.36227/techrxiv.19103432 · Cited by 13
Knowledge Boosting: Rethinking Medical Contrastive Vision-Language Pre-training
Chen, X., He, Y., Xue, C. Q., Ge, R., Li, S., & Yang, G. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43907-0_39 · Cited by 13
Pre-training on high-resolution X-ray images: an experimental study
Wang, X., Li, Y., Wu, W., Jin, J., Rong, Y., Jiang, B., Li, C., & Tang, J. (2025) · Visual Intelligence · doi:10.1007/s44267-025-00080-3 · Cited by 16
Recent Advances in Medical Image Classification
Dao, L., & Ly, N. Q. (2024) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2024.0150727 · Cited by 14
Missing Modality Prediction for Unpaired Multimodal Learning via Joint Embedding of Unimodal Models
Kim, D., & Kim, T. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73016-0_11 · Cited by 12
ChEX: Interactive Localization and Region Description in Chest X-Rays
Müller, P., Kaissis, G. A., & Rueckert, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72664-4_6 · Cited by 19
Overcoming the challenges of multi-modal medical image sharing: A novel data distillation strategy via contrastive learning
Du, T., Li, W., Wang, Z., Yang, F., Teng, P., Yi, X., Chen, H., Wang, Z., Zhang, P., & Zhang, T. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129043 · Cited by 12
LVR: A language and vision fusion method for rice diseases segmentation under complex environment
Zhao, T., Zhou, H., Yan, M., Zhou, G., Hu, Y., Hu, Y., Pan, M., Pan, M., Yu, Y., & Liu, Y. (2025) · European Journal of Agronomy · doi:10.1016/j.eja.2025.127599 · Cited by 12
Multimodal contrastive learning for enhanced explainability in pediatric brain tumor molecular diagnosis
Ketabi, S., Wagner, M. W., Hawkins, C. E., Tabori, U. Y., Ertl-Wagner, B. B., & Khalvati, F. (2025) · Scientific Reports · doi:10.1038/s41598-025-94806-4 · Cited by 12
MIT-SAM: Medical Image-Text SAM With Mutually Enhanced Heterogeneous Features Fusion for Medical Image Segmentation
Zhou, X., Yan, L., Ding, R., Atabansi, C. C., Nie, J., Chen, L., Feng, Y., & Liu, H. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3561425 · Cited by 12
Multi-view and Multi-scale Alignment for Contrastive Language-Image Pre-training in Mammography
Du, Y., Onofrey, J. A., & Dvornek, N. C. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-96625-5_17 · Cited by 17
T3D: Advancing 3D Medical Vision-Language Pre-Training by Learning Multi-View Visual Consistency
Liu, C., Ouyang, C., Chen, Y., Quilodrán-Casas, C., Ма, Л., Fu, J., Guo, Y., Shah, A., Bai, W., & Arcucci, R. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00698 · Cited by 12
Aligning, Autoencoding and Prompting Large Language Models for Novel Disease Reporting
Liu, F., Wu, X., Huang, J., Yang, B., Branson, K., Schwab, P., Clifton, L., Zhang, P., Luo, J., Zheng, Y., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3534586 · Cited by 13
3D foundation model for generalizable disease detection in head computed tomography
Zhu, W., Huang, H., Tang, H., Musthyala, R., Yu, B., Chen, L., Vega, E., O’Donnell, T., Hayek, R., Kuohn, L. R., et al. (2026) · Nature Biomedical Engineering · doi:10.1038/s41551-026-01668-w · Cited by 22
Text-Guided Face Recognition using Multi-Granularity Cross-Modal Contrastive Learning
Hasan, M. M., Sami, S. M., & Nasrabadi, N. M. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00568 · Cited by 10
Mlip: Medical Language-Image Pre-Training With Masked Local Representation Learning
Liu, J., Zhou, H.-Y., Li, C., Huang, W., Yang, H., Liang, Y., Shi, G., Zheng, H., & Wang, S. (2024) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi56570.2024.10635357 · Cited by 10
UMSCS: A Novel Unpaired Multimodal Image Segmentation Method Via Cross-Modality Generative and Semi-supervised Learning
Yang, F., Li, X., Wang, B., Teng, P., & Liu, G. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02389-4 · Cited by 11
Multi-modal learning methods in medical imaging area: A survey
Sun, Y., Chen, W., & Sun, Z. (2025) · Digital Signal Processing · doi:10.1016/j.dsp.2025.105441 · Cited by 10
Multimodal Vision–Language Models in Medical Imaging: A Survey of Retrieval, Interpretability, and Trust
Imran, M., & Lee, Y. (2026) · IEEE Access · doi:10.1109/access.2026.3656368 · Cited by 11
MedIM: Boost Medical Image Representation via Radiology Report-Guided Masking
Xie, Y., Gu, L., Harada, T., Zhang, J., Xia, Y., & Wu, Q. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43907-0_2 · Cited by 10
Radio-pathomic approaches in pediatric neuro-oncology: Opportunities and challenges
Familiar, A., Mahtabfar, A., Kazerooni, A. F., Kiani, M., Vossough, A., Viaene, A. N., Storm, P. B., Resnick, A. C., & Nabavizadeh, A. (2023) · Neuro-Oncology Advances · doi:10.1093/noajnl/vdad119 · Cited by 12
Envisioning Medclip: A Deep Dive into Explainability for Medical Vision-Language Models
Hashmi, A. U. R., Mahapatra, D., & Yaqub, M. K. (2024) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi56570.2024.10635843 · Cited by 9
Cross-modal contrastive learning for unified placenta analysis using photographs
Pan, Y., Mehta, M., Goldstein, J. A., Ngonzi, J. N. J., Bebell, L. M., Roberts, D. J., Carreon, C. K., Gallagher, K., Walker, R. E., Gernand, A. D., et al. (2024) · Patterns · doi:10.1016/j.patter.2024.101097 · Cited by 9
Topicwise Separable Sentence Retrieval for Medical Report Generation
Zhao, J., Yang, Z., Chen, Z., Fu, H., & Wan, L. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3507076 · Cited by 9
HKRG: Hierarchical knowledge integration for radiology report generation
Wang, B., Teng, P., Zhang, H., Yang, F., Wang, Z., Yi, X., Zhang, T., Wang, C., Tavares, A., & Xu, H. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126622 · Cited by 10
Text-Assisted Vision Model for Medical Image Segmentation
Rahman, M. M., Rahman, S., Bhatt, S., & Faezipour, M. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3569491 · Cited by 10
AMF: A multi-modal framework for crop leaf diseases segmentation
Hu, Y., Wang, G., Liu, E., Zhu, J., & He, M. (2025) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2025.110550 · Cited by 9
Enhancing Free-Hand 3-D Photoacoustic and Ultrasound Reconstruction Using Deep Learning
Lee, S., Kim, S., Seo, M., Park, S., Imrus, S., Ashok, K., Lee, D., Park, C., Lee, S., Kim, J., et al. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3579454 · Cited by 9
Medical radiology report generation: A systematic review of current deep learning methods, trends, and future directions
Izhar, A., Idris, N., & Japar, N. B. (2025) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2025.103220 · Cited by 11
Artificial Intelligence in Pathology: Advancing Large Models for Scalable Applications
Xiao, Z., Feng, B., Yang, J., Sun, G., Shen, Y., Xu, S., Yang, L., Xu, H., Zhang, M., & Wang, S. (2025) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-103123-095814 · Cited by 9
A multimodal vision–language model for generalizable annotation-free pathology localization
Yang, H., Zhou, H.-Y., Liu, J., Huang, W., Li, C., Li, Z., Gao, Y., Liu, Q., Liang, Y., Yang, Q., et al. (2026) · Nature Biomedical Engineering · doi:10.1038/s41551-025-01574-7 · Cited by 11
Foundation Models Meet Medical Image Interpretation
Jiao, L., Hao, J., Li, R., Li, L., Liu, X., Liu, F., Ma, W., Chen, P., Huang, Z., Yang, J., et al. (2026) · Research · doi:10.34133/research.1024 · Cited by 12
Visual-Language Foundation Models and Artificial Intelligence Agents in Healthcare: Bridging from Technological Innovation to Clinical Impact
Al-Jebrni, A. H., Jiang, Z., Qin, Y., Qiu, Y., Jiang, N., Li, G., Zheng, Y., Wong, T. Y., Kim, S., Jung, Y., et al. (2026) · Computational Visual Media · doi:10.26599/cvm.2026.9450530 · Cited by 9
Fine-grained Medical Vision-Language Representation Learning for Radiology Report Generation
Wang, S., Peng, B., Liu, Y., & Peng, Q. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.989 · Cited by 10
UMCL: Unified Medical Image-Text-Label Contrastive Learning With Continuous Prompt
Wang, Y., & Wang, G. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10386034 · Cited by 9
PairAug: What Can Augmented Image-Text Pairs Do for Radiology?
Xie, Y., Chen, Q., Wang, S., To, M., Lee, I., Khoo, E. W., Hendy, K., Koh, D., Xia, Y., & Wu, Q. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01107 · Cited by 14
Enhancing Medical Vision-Language Contrastive Learning via Inter-Matching Relation Modeling
Li, M., Meng, M., FULHAM, M. J., Feng, D. D., Bi, L., & Kim, J. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3534436 · Cited by 9
Multi-domain improves classification in out-of-distribution and data-limited scenarios for medical image analysis
Özkan, E., & Boix, X. (2024) · Scientific Reports · doi:10.1038/s41598-024-73561-y · Cited by 8
CLR2G: Cross modal Contrastive Learning on Radiology Report Generation
Xue, H., Ma, Q., Liu, G., Qu, J., Liu, Y., & Liu, A. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679668 · Cited by 8
AMVLM: Alignment-Multiplicity Aware Vision-Language Model for Semi-Supervised Medical Image Segmentation
Pan, Q., Li, Z., Qiao, W., Lou, J., Yang, Q., Yang, G., & Ji, B. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3573018 · Cited by 8
Medical Report Generation With Knowledge Distillation and Multi-Stage Hierarchical Attention in Vision Transformer Encoder and GPT-2 Decoder
Tsaniya, H., Fatichah, C., Suciati, N., Obi, T., & Lee, J. H. (2025) · IEEE Access · doi:10.1109/access.2025.3588344 · Cited by 8
Unsupervised Cross-Modality Domain Adaptation for Vestibular Schwannoma Segmentation and Koos Grade Prediction Based on Semi-supervised Contrastive Learning
Han, L., Huang, Y., Tan, T., & Mann, R. M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44153-0_5 · Cited by 8
Artificial intelligence powered advancements in upper extremity joint MRI: A review
Chen, W., Lim, L. J. R., Lim, R. Q. R., Yi, Z., Huang, J., He, J., Yang, G., & Liu, B. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e28731 · Cited by 8
Towards Medical Vision-Language Contrastive Pre-training via Study-Oriented Semantic Exploration
Liu, B., Lu, Z., & Wang, Y. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681531 · Cited by 8
Automatic radiology report generation with deep learning: a comprehensive review of methods and advances
Li, Y., Kong, C., Zhao, G., & Zhao, Z. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11337-0 · Cited by 7
Multimodal transformers for joint diagnosis and retrieval from chest X-rays and radiology reports
Jahanian, M., Karimi, A., Eraghi, N. O., & Zarafshan, F. (2025) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2025.101672 · Cited by 7
Multimodal Self-Supervised Learning for Lesion Localization
Yang, H., Zhou, H.-Y., Li, C., Huang, W., Liu, J., Liang, Y., Shi, G., Zheng, H., Liu, Q., & Wang, S. (2024) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi56570.2024.10635268 · Cited by 7
Hybrid cross-modality fusion network for medical image segmentation with contrastive learning
Zhou, X., Song, Q., Nie, J., Feng, Y., Liu, H., Liang, F., Chen, L., & Xie, J. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110073 · Cited by 8
MCG-Net: Medical Chief Complaint-guided Multi-modal Masked Content Pre-training for chest image classification
Zou, L., Li, J., Chen, H., Liang, M., Jia, K., Zhong, Y., & Chen, J. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126660 · Cited by 7
Driven by textual knowledge: A Text-View Enhanced Knowledge Transfer Network for lung infection region segmentation
Fang, L., Li, X., Xu, Y., Zhang, F., & Zhang, C. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103625 · Cited by 7
Radiology report generation using automatic keyword adaptation, frequency-based multi-label classification and text-to-text large language models
He, Z., Wong, A. N. N., & Yoo, J. S. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110625 · Cited by 9
C2M-DoT: Cross-modal consistent multi-view medical report generation with domain transfer network
Wang, R., Xu, Z., Wang, X., Liu, W., & Lukasiewicz, T. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103442 · Cited by 7
KBNet: A Language and Vision Fusion Multi-Modal Framework for Rice Disease Segmentation
Yan, X., Zhou, H., Zhu, J., He, M., Zhao, T., Tan, X., & Zeng, J. (2025) · Plants · doi:10.3390/plants14162465 · Cited by 7
Self-supervised learning leads to improved performance in biparametric prostate MRI classification
de Almeida, J. G., Verde, A. S. C., Gaivão, A. M., Bilreiro, C., Santiago, I., Ip, J., Belião, S., Matos, C., Tsiknakis, M., Kostas Marias, A., et al. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.111262 · Cited by 7
Towards good practice for convolution and attention with PANs in federated medical image classification
Makhanov, N., Nhan, H. T., Wong, K., & Tu, N. A. (2024) · The Journal of Supercomputing · doi:10.1007/s11227-024-06476-0 · Cited by 7
MEDBind: Unifying Language and Multimodal Medical Data Embeddings
Gao, Y., Kim, S., Austin, D. E., & McIntosh, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72390-2_21 · Cited by 9
Assessing severity of pediatric pneumonia using multimodal transformers with multi-task learning
Li, J., Nan, Z.-A., Qi, G., Cai, J., Zhao, X., Li, X., Liu, S., Wang, Y., Wu, Y., Miao, X., et al. (2024) · Digital Health · doi:10.1177/20552076241305168 · Cited by 7
TexDC: Text-Driven Disease-Aware 4D Cardiac Cine MRI Images Generation
Liu, C., Yuan, X., Yu, Z., & Wang, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0901-7_12 · Cited by 7
Self-Supervised Foundation Model for Template Matching
Hristov, A., Dimov, D., & Nisheva-Pavlova, M. M. (2025) · Big Data and Cognitive Computing · doi:10.3390/bdcc9020038 · Cited by 6
ViTexNet: Vision-Text Guided Dynamic Convolution Network for Medical Image Segmentation
Bhardwaj, R., Tambe, U. Y., & Neog, D. R. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04981-0_65 · Cited by 6
MG-UNet: A Memory-Guided UNet for Lesion Segmentation in Chest Images
Ding, S., Li, M., & Wang, C. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04927-8_34 · Cited by 6
Chestxgen: Dynamic Memory-Augmented Vision-Language Transformer with Context-Aware Gating for Radiology Report Generation
Allaberdiev, S., Khan, A., Mamarasulov, S., & Chen, X. (2025) · Journal of Artificial Intelligence and Soft Computing Research · doi:10.2478/jaiscr-2026-0003 · Cited by 6
EviVLM: When Evidential Learning Meets Vision Language Model for Medical Image Segmentation
Pan, Q.-W., Li, Z., Yang, G., Yang, Q., & Ji, B. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3622492 · Cited by 6
An Alignment and Imputation Network (AINet) for Breast Cancer Diagnosis With Multimodal Multi-View Ultrasound Images
Chen, H., Li, Y., Zhang, J., Yang, L., Sun, Y., Chen, Y., Zhou, S., Li, Z., Qian, X., Xu, Q., et al. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3625254 · Cited by 6
Learning with less: A survey of deep learning in medical imaging under varying supervision levels
Kumari, S., & Singh, P. P. (2026) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2026.103375 · Cited by 7
Cross modal global local representation learning from radiology reports and x-ray chest images
Hadjiyski, N., Vosoughi, A., & Wismüller, A. (2023) · SPIE Medical Imaging: Computer-Aided Diagnosis · doi:10.1117/12.2654520 · Cited by 6
Disease-Informed Adaptation of Vision-Language Models
Zhang, J., Wang, G., Kalra, M. K., & Yan, P. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3484294 · Cited by 7
EyeGraphGPT: Knowledge Graph Enhanced Multimodal Large Language Model for Ophthalmic Report Generation
Wang, Z. Y., Jiang, X., Gao, C., Dong, F., Dai, W., Wang, B., Yan, B., Chen, Q., Huang, W., Zhang, T., et al. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822152 · Cited by 6
Advancing Medical Radiograph Representation Learning: A Hybrid Pre-training Paradigm with Multilevel Semantic Granularity
Jiang, H., Hao, X., Huang, Y., Ma, C., Zhang, J., Pan, Y. F., & Zhang, R. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-91721-9_2 · Cited by 6
Knowledge fusion in deep learning-based medical vision-language models: A review
Xu, D., Chen, Y., Chai, Z., Xiao, Y., Yan, Y., Ding, W., Wang, H., Jin, Z. J., Jiao, W., Yue, W., et al. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103455 · Cited by 6
Comparative analysis of popular predictors for difficult laryngoscopy using hybrid intelligent detection methods
Liu, X., Flanagan, C., Fang, J., Lei, Y., McGrath, L., Wang, J., Guo, X., Guo, J., McGrath, H., & Han, Y. (2022) · Heliyon · doi:10.1016/j.heliyon.2022.e11761 · Cited by 6
Improved Multimodal Fusion for Small Datasets with Auxiliary Supervision
Holste, G., van der Wal, D., Pinckaers, H., Yamashita, R., Mitani, A., & Esteva, A. (2023) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi53787.2023.10230356 · Cited by 6
Enhancing Gait Video Analysis in Neurodegenerative Diseases by Knowledge Augmentation in Vision Language Model
Wang, D., Yuan, K., Muller, C., Blanc, F., Padoy, N., & Seo, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72086-4_24 · Cited by 6
MOSMOS: Multi-organ segmentation facilitated by medical report supervision
Tian, W., Huang, X., Hou, J., Ren, C., Jiang, L., Zhao, R.-W., Jin, G., Zhang, Y., & Geng, D. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.107743 · Cited by 6
A multi-scale linear cross-modal fusion architecture for tomato leaf disease segmentation
Yang, Z., Sun, L., Deng, J., Cai, A., Liu, Z., Liu, G., Zhang, J., Zhou, G., Hu, Y., & Li, L. (2025) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2025.111155 · Cited by 5

Authors: Huang, S.-C., Shen, L., Lungren, M. P., & Yeung, S.
Affiliations: Stanford University (US)
ORCID: Huang, Shih-Cheng
Shen, Liyue
Lungren, Matthew P.
Yeung, Serena
Venue: 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher: IEEE
Published: 2021-10
Pages: 3922-3931
DOI: 10.1109/iccv48922.2021.00391
OpenAlex: W3201906559
Cited by: 489
References: 41
Citations by year: 2026: 122 · 2025: 195 · 2024: 106 · 2023: 51 · 2022: 14 · 2021: 1
FWCI: 14.04
Type: conference paper
Language: en
Open access: closed
Licence: doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords: multimodal representation learning, label-efficient learning, medical image recognition, radiology reports, image-text retrieval, zero-shot classification, medical image segmentation, limited labeled data

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

GLoRIA: A Multimodal Global-Local Representation Learning Framework for Label-efficient Medical Image Recognition

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks