Reducing Hallucinations of Medical Multimodal Large Language Models with Visual Retrieval-Augmented Generation
Open accessAbstract
Multimodal Large Language Models (MLLMs) have shown impressive performance in vision and text tasks. However, hallucination remains a major challenge, especially in fields like healthcare where details are critical. In this work, we show how MLLMs may be enhanced to support Visual RAG (V-RAG), a retrieval-augmented generation framework that incorporates both text and visual data from retrieved images. On the MIMIC-CXR chest X-ray report generation and Multicare medical image caption generation datasets, we show that Visual RAG improves the accuracy of entity probing, which asks whether a medical entities is grounded by an image. We show that the improvements extend both to frequent and rare entities, the latter of which may have less positive training data. Downstream, we apply V-RAG with entity probing to correct hallucinations and generate more clinically accurate X-ray reports, obtaining a higher RadGraph-F1 score.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Retrieval-augmented generation (RAG) has become prominent for mitigating hallucinations.
“This strategy, known as retrieval-augmented generation (RAG), has become prominent for mitigating hallucinations (Chu et al., 2025).”
From Trustworthy Medical Question Answering: An Evaluation-Centric Survey (2025) · cites this paper as Chu et al., 2025
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 26
- Bai, Z.; Wang, P.; Xiao, T.; He, T.; Han, Z.; Zhang, Z.; and Shou, M. Z. 2024. Hallucination of Multimodal Large Language Models: A Survey. ArXiv , abs/2404.18930. · arXiv
- Chen, Z.; Varma, M.; Delbrouck, J.-B.; Paschali, M.; Blankemeier, L.; Veen, D. V.; Valanarasu, J. M. J.; Youssef, A.; Cohen, J. P.; Reis, E. P.; Tsai, E. B.; Johnston, A.; Olsen, C.; Abraham, T. M.; Gatidis, S.; Chaudhari, A. S.; and Langlotz, C. P. 2024. CheXagent: Towards a Foundation Model for Chest X-Ray Interpretation. ArXiv , abs/2401.12208. · arXiv
- Delbrouck, J.-B.; Chambon, P.; Chen, Z.; Varma, M.; Johnston, A.; Blankemeier, L.; Van Veen, D.; Bui, T.; Truong, S.; and Langlotz, C. 2024. RadGraph-XL: A Large-Scale Expert-Annotated Dataset for Entity and Relation Extraction from Radiology Reports. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Findings of the Association for Computational Linguistics ACL 2024 , 12902-12915. Bangkok, Thailand and virtual meeting: Association for Computational Linguistics.
- Guu, K.; Lee, K.; Tung, Z.; Pasupat, P.; and Chang, M.-W. 2020. REALM: Retrieval-Augmented Language Model PreTraining. ArXiv , abs/2002.08909. · arXiv
- Hu, J. E.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y .; Wang, S.; and Chen, W. 2021. LoRA: Low-Rank Adaptation of Large Language Models. ArXiv , abs/2106.09685. · arXiv
- Huang, W.; Liu, H.; Guo, M.; and Gong, N. Z. 2024. Visual Hallucinations of Multi-modal Large Language Models. ArXiv , abs/2402.14683. · arXiv
- Irvin, J. A.; Rajpurkar, P.; Ko, M.; Yu, Y.; Ciurea-Ilcus, S.; Chute, C.; Marklund, H.; Haghgoo, B.; Ball, R. L.; Shpanskaya, K. S.; Seekins, J.; Mong, D. A.; Halabi, S. S.; Sandberg, J. K.; Jones, R.; Larson, D. B.; Langlotz, C.; Patel, B. N.; Lungren, M. P.; and Ng, A. 2019. CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison. In AAAI Conference on Artificial Intelligence .
- Lewis, P.; Perez, E.; Piktus, A.; Petroni, F.; Karpukhin, V.; Goyal, N.; Kuttler, H.; Lewis, M.; tau Yih, W.; Rockt¨ aschel, T.; Riedel, S.; and Kiela, D. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. ArXiv , abs/2005.11401. · arXiv
- Li, C.; Wong, C.; Zhang, S.; Usuyama, N.; Liu, H.; Yang, J.; Naumann, T.; Poon, H.; and Gao, J. 2023a. LLaVAMed: Training a Large Language-and-Vision Assistant for Biomedicine in One Day. ArXiv , abs/2306.00890. · arXiv
- Li, Y.; Du, Y.; Zhou, K.; Wang, J.; Zhao, W. X.; and rong Wen, J. 2023b. Evaluating Object Hallucination in Large Vision-Language Models. In Conference on Empirical Methods in Natural Language Processing .
- Liu, H.; Li, C.; Li, Y.; and Lee, Y. J. 2023a. Improved Baselines with Visual Instruction Tuning. ArXiv , abs/2310.03744. · arXiv
- Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2023b. Visual Instruction Tuning. ArXiv , abs/2304.08485. · arXiv
- Liu, H.; Son, K.; Yang, J.; Liu, C.; Gao, J.; Lee, Y. J.; and Li, C. 2023c. Learning Customized Visual Models with Retrieval-Augmented Knowledge. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 15148-15158.
- Liu, Z.; Sun, Z.; Zang, Y.; Li, W.; Zhang, P.; wen Dong, X.; Xiong, Y.; Lin, D.; and Wang, J. 2024. RAR: Retrieving And Ranking Augmented MLLMs for Visual Recognition. ArXiv , abs/2403.13805. · arXiv
- Malkov, Y.; and Yashunin, D. A. 2016. Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence , 42: 824-836.
- Moor, M.; Huang, Q.; Wu, S.; Yasunaga, M.; Zakka, C.; Dalmia, Y.; Reis, E. P.; Rajpurkar, P.; and Leskovec, J. 2023. Med-Flamingo: a Multimodal Medical Few-shot Learner. ArXiv , abs/2307.15189. · arXiv
- Pal, A.; and Sankarasubbu, M. 2024. Gemini Goes to Med School: Exploring the Capabilities of Multimodal Large Language Models on Medical Challenge Problems & Hallucinations. ArXiv , abs/2402.07023. · arXiv
- Qu, X.; Chen, Q.; Wei, W.; Sun, J.; and Dong, J. 2024b. Alleviating Hallucination in Large Vision-Language Models with Active Retrieval Augmentation. arXiv:2408.00555. · arXiv
- Sarto, S.; Cornia, M.; Baraldi, L.; Nicolosi, A.; and Cucchiara, R. 2024. Towards Retrieval-Augmented Architectures for Image Captioning. ACM Transactions on Multimedia Computing, Communications and Applications .
- Shahul, E.; James, J.; Anke, L. E.; and Schockaert, S. 2023. RAGAs: Automated Evaluation of Retrieval Augmented Generation. In Conference of the European Chapter of the Association for Computational Linguistics .
- Siriwardhana, S.; Weerasekera, R.; Wen, E.; Kaluarachchi, T.; Rana, R. K.; and Nanayakkara, S. 2022. Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering. Transactions of the Association for Computational Linguistics , 11: 1-17.
- Wu, C.; Zhang, X.; Zhang, Y.; Wang, Y.; and Xie, W. 2023. Towards Generalist Foundation Model for Radiology. ArXiv , abs/2308.02463. · arXiv
- Wu, J.; Kim, Y.; and Wu, H. 2024. Hallucination Benchmark in Medical Visual Question Answering. arXiv:2401.05827. · arXiv
- Zhang, X.; Wu, C.; Zhao, Z.; Lin, W.; Zhang, Y.; Wang, Y.; and Xie, W. 2023b. PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering. ArXiv , abs/2305.10415. · arXiv
- Zhang, Y.; Zhang, Y.; Qi, P.; Manning, C. D.; and Langlotz, C. P. 2021. Biomedical and clinical English model packages for the Stanza Python NLP library. Journal of the American Medical Informatics Association .
- Zhou, Z.; Zhang, J.; Guan, Z.; Hu, M.; Lao, N.; Mu, L.; Li, S.; and Mai, G. 2024. Img2Loc: Revisiting Image Geolocalization using Multi-modality Foundation Models and Image-based Retrieval-Augmented Generation. ArXiv , abs/2403.19584. · arXiv
Cited by
Sorted by citations
Trustworthy Medical Question Answering: An Evaluation-Centric Survey
Wang, Y., Wang, B., Mercer, R. E., Rudzicz, F., Roy, S. S., Ren, P., Chen, Z., & Wang, X. (2025) · arXiv · doi:10.48550/arxiv.2506.03659
- Authors
- Chu, Y.-W., Zhang, K., Malon, C., & Min, M. R.
- Affiliations
- Purdue University West Lafayette (US)
- Lehigh University (US)
- NEC (United States) (US)
- ORCID
- Min, Martin Renqiang
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2502.15040
- OpenAlex
- W6929472636
- Cited by
- 13
- References
- 0
- Citations by year
- 2026: 8 · 2025: 5
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, other-oa)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- hallucination mitigation, multimodal large language models, chest X-ray report generation, medical image captioning, entity grounding, MIMIC-CXR
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
