The DOI matches its DataCite record.
A Comprehensive Survey on the Trustworthiness of Large Language Models in Healthcare
Open accessAbstract
The application of large language models (LLMs) in healthcare holds significant promise for enhancing clinical decision-making, medical research, and patient care. However, their integration into real-world clinical settings raises critical concerns around trustworthiness, particularly around dimensions of truthfulness, privacy, safety, robustness, fairness, and explainability. These dimensions are essential for ensuring that LLMs generate reliable, unbiased, and ethically sound outputs. While researchers have recently begun developing benchmarks and evaluation frameworks to assess LLM trustworthiness, the trustworthiness of LLMs in healthcare remains underexplored, lacking a systematic review that provides a comprehensive understanding and future insights. This survey addresses that gap by providing a comprehensive review of current methodologies and solutions aimed at mitigating risks across key trust dimensions. We analyze how each dimension affects the reliability and ethical deployment of healthcare LLMs, synthesize ongoing research efforts, and identify critical gaps in existing approaches. We also identify emerging challenges posed by evolving paradigms, such as multi-agent collaboration, multi-modal reasoning, and the development of small open-source medical models. Our goal is to guide future research toward more trustworthy, transparent, and clinically viable LLMs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Recent research explored integrating uncertainty quantification into LLMs to improve calibration, enhancing the correlation between confidence and correctness.
“Recent research explored integrating uncertainty quantification into LLMs to improve calibration, enhancing the correlation between confidence and correctness (Aljohani et al., 2025).”
From Trustworthy Medical Question Answering: An Evaluation-Centric Survey (2025) · cites this paper as Aljohani et al., 2025
The survey’s Truthfulness findings discuss uncertainty-quantification methods that flag low-confidence responses, and its Robustness findings say such methods can identify cases where models are likely to err. Neither passage says these methods improve calibration or strengthen the correlation between confidence and correctness; those outcomes are not supported by our copy. (LLM, conf=0.84)
Experimental — accuracy not yet measuredWhen deployed in clinical settings, LLM-based medical QA models continue to exhibit critical trust failures including hallucinated medical facts, unjustified overconfidence, and occasional biased or unsafe recommendations.
“when deployed in clinical settings, these models continue to exhibit critical trust failures: hallucinated medical facts, unjustified overconfidence, and occasional biased or unsafe recommendations (Aljohani et al., 2025).”
From Trustworthy Medical Question Answering: An Evaluation-Centric Survey (2025) · cites this paper as Aljohani et al., 2025
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
No references could be resolved yet.
Research works printed but not linked: 88
- Hammaad Adam, Aparna Balagopalan, Emily Alsentzer, Fotini Christia, and Marzyeh Ghassemi. 2022. Just following ai orders: When unbiased people are influ- enced by biased ai. In Workshop on Trustworthy and Socially Responsible Machine Learning, NeurIPS 2022.
- Muhammad Aurangzeb Ahmad, Ilker Yaramis, and Taposh Dutta Roy. 2023. Creating trustworthy llms: Dealing with hallucinations in healthcare ai. arXiv preprint arXiv:2311.01463. · arXiv
- Mubashara Akhtar, Oana Cocarascu, and Elena Simperl. 2022. PubHealthTab: A public health table-based dataset for evidence-based fact checking. In Find- ings of the Association for Computational Linguistics: NAACL 2022, pages 1-16, Seattle, United States. As- sociation for Computational Linguistics. · DOI
- Ian L Alberts, Lorenzo Mercolli, Thomas Pyka, George Prenosil, Kuangyu Shi, Axel Rominger, and Ali Afshar-Oromieh. 2023. Large language models (llm) and chatgpt: what will the impact on nuclear medicine be? European journal of nuclear medicine and molecular imaging, 50(6):1549-1552.
- Bayan Altalla', Sameera Abdalla, Ahmad Altamimi, Layla Bitar, Amal Al Omari, Ramiz Kardan, and Iyad Sultan. 2025. Evaluating gpt models for clinical note de-identification. Scientific Reports, 15(1):3852.
- Elham Asgari, Nina Montaña-Brown, Magda Dubois, Saleh Khalil, Jasmine Balloch, Joshua Au Yeung, and Dominic Pimenta. 2025. A framework to assess clin- ical safety and hallucination rates of llms for medical text summarisation. npj Digital Medicine, 8(1):274.
- Zhenyu Bi, Sajib Acharjee Dip, Daniel Hajialigol, Sind- hura Kommu, Hanwen Liu, Meng Lu, and Xuan Wang. 2024. Ai for biomedicine in the era of large language models. arXiv preprint arXiv:2403.15673. · arXiv
- Kuluhan Binici, Abhinav Ramesh Kashyap, Viktor Schlegel, Andy T. Liu, Vijay Prakash Dwivedi, Thanh-Tung Nguyen, Xiaoxue Gao, Nancy F. Chen, and Stefan Winkler. 2025. MEDSAGE: Enhancing robustness of medical dialogue summarization to ASR errors with LLM-generated synthetic dialogues. In AI4X 2025 International Conference.
- Alberto Mario Ceballos-Arroyo, Monica Munnangi, Ji- uding Sun, Karen Zhang, Jered McInerney, Byron C. Wallace, and Silvio Amir. 2024. Open (clinical) LLMs are sensitive to instruction phrasings. In Pro- ceedings of the 23rd Workshop on Biomedical Nat- ural Language Processing, pages 50-71, Bangkok, Thailand. Association for Computational Linguistics. · DOI
- Yan Chen and Pouyan Esmaeilzadeh. 2024. Generative ai in medical practice: in-depth exploration of privacy and security challenges. Journal of Medical Internet Research, 26:e53008.
- Roi Cohen, May Hamri, Mor Geva, and Amir Glober- son. 2023. LM vs LM: Detecting factual errors via cross examination. In The 2023 Conference on Em- pirical Methods in Natural Language Processing.
- Leonardo Cotta and Chris J Maddison. 2024. Out-of- context prompting boosts fairness and robustness in large language model predictions. In ICML 2024 Workshop on Foundation Models in the Wild.
- Badhan Chandra Das, M Hadi Amini, and Yanzhao Wu. 2024. Security and privacy challenges of large language models: A survey. arXiv preprint arXiv:2402.00888. · arXiv
- Souvik Das and Rohini K Srihari. 2024. Uniwiz: A unified large language model orchestrated wizard for safe knowledge grounded conversations. In Findings of the Association for Computational Linguistics ACL 2024, pages 1749-1762.
- Biaoyan Fang, Xiang Dai, and Sarvnaz Karimi. 2024. Understanding faithfulness and reasoning of large language models on plain biomedical summaries. In Findings of the Association for Computational Lin- guistics: EMNLP 2024, pages 9890-9911, Miami, Florida, USA. Association for Computational Lin- guistics. · DOI · Paper page
- Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. 2024. Detecting hallucinations in large language models using semantic entropy. Nature, 630(8017):625-630.
- Xiaotang Gai, Chenyi Zhou, Jiaxiang Liu, Yang Feng, Jian Wu, and Zuozhu Liu. 2025. Medthink: A rationale-guided framework for explaining medical visual question answering. In Findings of the Associ- ation for Computational Linguistics: NAACL 2025, pages 7438-7450.
- Jack Gallifant, Shan Chen, Pedro José Ferreira Mor- eira, Nikolaj Munch, Mingye Gao, Jackson Pond, Leo Anthony Celi, Hugo Aerts, Thomas Hartvigsen, and Danielle Bitterman. 2024. Language models are surprisingly fragile to drug names in biomedical benchmarks. In Findings of the Association for Com- putational Linguistics: EMNLP 2024, pages 12448- 12465, Miami, Florida, USA. Association for Com- putational Linguistics. · DOI · Paper page
- Shantanu Ghosh, Ke Yu, Forough Arabshahi, and kay- han Batmanghelich. 2023. Bridging the gap: From post hoc explanations to inherently interpretable mod- els for medical imaging. In ICML 3rd Workshop on Interpretable Machine Learning in Healthcare (IMLH).
- Zhibin Gou, Zhihong Shao, Yeyun Gong, yelong shen, Yujiu Yang, Nan Duan, and Weizhu Chen. 2024. CRITIC: Large language models can self-correct with tool-interactive critiquing. In The Twelfth Inter- national Conference on Learning Representations.
- Jiatong Han, Jannik Kossen, Muhammed Razzak, Lisa Schut, Shreshth A Malik, and Yarin Gal. 2024a. Se- mantic entropy probes: Robust and cheap hallucina- tion detection in llms. In ICML 2024 Workshop on Foundation Models in the Wild.
- Rujun Han, Peng Qi, Yuhao Zhang, Lan Liu, Juliette Burger, William Yang Wang, Zhiheng Huang, Bing Xiang, and Dan Roth. 2023. RobustQA: Benchmark- ing the robustness of domain adaptation for open- domain question answering. In Findings of the As- sociation for Computational Linguistics: ACL 2023, pages 4294-4311, Toronto, Canada. Association for Computational Linguistics. · DOI
- Tessa Han, Aounon Kumar, Chirag Agarwal, and Himabindu Lakkaraju. 2024b. Medsafetybench: Evaluating and improving the medical safety of large language models. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track.
- Tessa Han, Aounon Kumar, Chirag Agarwal, and Himabindu Lakkaraju. 2024c. Towards safe large lan- guage models for medicine. In ICML 2024 Workshop on Models of Human Feedback for AI Alignment.
- Tianyu Han, Lisa C. Adams, Jens-Michalis Papaioan- nou, Paul Grundmann, Tom Oberhauser, Alexei Figueroa, Alexander Löser, Daniel Truhn, and Keno K. Bressem. 2025. Medalpaca -an open- source collection of medical conversational ai models and training data. Preprint, arXiv:2304.08247. · arXiv
- Tianyu Han, Sven Nebelung, Firas Khader, Tianci Wang, Gustav Müller-Franzes, Christiane Kuhl, Sebastian Försch, Jens Kleesiek, Christoph Haarburger, Keno K Bressem, et al. 2024d. Medical large language mod- els are susceptible to targeted misinformation attacks. NPJ Digital Medicine, 7(1):288.
- Xiang Han, Qi Zhang, Kai Wang, Yitong Zhang, Chenyu Guo, Dongdong Chen, Xinyang Liu, and James Zou. 2024e. Medical large language mod- els are vulnerable to data-poisoning attacks. Nature Medicine. · DOI · Paper page
- Yinhan He, Zaiyi Zheng, Patrick Soga, Yaochen Zhu, Yushun Dong, and Jundong Li. 2024. Explaining graph neural networks with large language models: A counterfactual perspective on molecule graphs. In Findings of the Association for Computational Lin- guistics: EMNLP 2024, pages 7079-7096, Miami, Florida, USA. Association for Computational Lin- guistics. · DOI
- Lianting Hu, Dantong Li, Huazhang Liu, Xuanhui Chen, Yunfei Gao, Shuai Huang, Xiaoting Peng, Xueli Zhang, Xiaohe Bai, Huan Yang, et al. 2024. En- hancing fairness in ai-enabled medical systems with the attribute neutral framework. Nature Communica- tions, 15(1):8767.
- Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, et al. 2024. Position: Trustllm: Trustworthiness in large language models. In International Conference on Machine Learning, pages 20166-20270.
- PMLR.
- Yixin Ji, Kaixin Wu, Juntao Li, Wei Chen, Mingjie Zhong, Xu Jia, and Min Zhang. 2024. Retrieval and reasoning on KGs: Integrate knowledge graphs into large language models for complex question answer- ing. In Findings of the Association for Computa- tional Linguistics: EMNLP 2024, pages 7598-7610, Miami, Florida, USA. Association for Computational Linguistics. · DOI
- Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung. 2023. Towards mitigating LLM hallucination via self reflection. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1827-1843, Singapore. Associ- ation for Computational Linguistics. · DOI
- Ruinan Jin, Zikang Xu, Yuan Zhong, Qingsong Yao, Qi Dou, S Kevin Zhou, and Xiaoxiao Li. 2024. FairmedFM: Fairness benchmarking for medical imaging foundation models. In The Thirty-eight Con- ference on Neural Information Processing Systems Datasets and Benchmarks Track.
- Jitendra Jonnagaddala and Zoie Shui-Yee Wong. 2025. Privacy preserving strategies for electronic health records in the era of large language models. npj Digital Medicine, 8(1):34.
- Praveen K Kanithi, Clément Christophe, Marco AF Pi- mentel, Tathagata Raha, Nada Saadi, Hamza Javed, Svetlana Maslenkova, Nasir Hayat, Ronnie Rajan, and Shadab Khan. 2024a. Medic: Towards a com- prehensive framework for evaluating llms in clinical applications. arXiv preprint arXiv:2409.07314. · arXiv
- Praveen K Kanithi, Clément Christophe, Marco AF Pi- mentel, Tathagata Raha, Nada Saadi, Hamza Javed, Svetlana Maslenkova, Nasir Hayat, Ronnie Rajan, and Shadab Khan. 2024b. Medic: Towards a com- prehensive framework for evaluating llms in clinical applications. Preprint, arXiv:2409.07314. · arXiv
- Yuhe Ke, Rui Yang, Sui An Lie, Taylor Xin Yi Lim, Yilin Ning, Irene Li, Hairil Rizal Abdullah, Daniel Shu Wei Ting, and Nan Liu. 2024. Mitigating cogni- tive biases in clinical decision-making through multi- agent conversations using large language models: simulation study. Journal of Medical Internet Re- search, 26:e59439.
- Michelle Kim, Junghwan Kim, and Kristen Johnson. 2023. Race, gender, and age biases in biomedical masked language models. In Findings of the Asso- ciation for Computational Linguistics: ACL 2023, pages 11806-11815, Toronto, Canada. Association for Computational Linguistics. · DOI
- Yunsoo Kim, Jinge Wu, Yusuf Abdulle, and Honghan Wu. 2024. MedExQA: Medical question answer- ing benchmark with multiple explanations. In Pro- ceedings of the 23rd Workshop on Biomedical Natu- ral Language Processing, pages 167-181, Bangkok, Thailand. Association for Computational Linguistics. · DOI
- Hannah Rose Kirk, Bertie Vidgen, Paul Röttger, and Scott A Hale. 2024. The benefits, risks and bounds of personalizing the alignment of large language models to individuals. Nature Machine Intelligence, 6(4):383-392.
- Tobias Leemann, Bardh Prenkaj, and Gjergji Kasneci. 2024. Is my data safe? predicting instance-level membership inference success for white-box and black-box attacks. In ICML 2024 Next Generation of AI Safety Workshop.
- Songda Li, Yunqi Zhang, Chunyuan Deng, Yake Niu, and Hui Zhao. 2024. Better late than never: Model- agnostic hallucination post-processing framework to- wards clinical text summarization. In Findings of the Association for Computational Linguistics: ACL 2024, pages 995-1011, Bangkok, Thailand. Associa- tion for Computational Linguistics. · DOI
- Ruixi Lin and Hwee Tou Ng. 2023. Mind the bi- ases: Quantifying cognitive biases in language model prompting. In Findings of the Association for Com- putational Linguistics: ACL 2023, pages 5269-5281, Toronto, Canada. Association for Computational Lin- guistics. · DOI
- Jiaxiang Liu, Tianxiang Hu, Yan Zhang, Xiaotang Gai, YANG FENG, and Zuozhu Liu. 2023a. A chatGPT aided explainable framework for zero-shot medical image diagnosis. In ICML 3rd Workshop on Inter- pretable Machine Learning in Healthcare (IMLH).
- Lei Liu, Xiaoyan Yang, Junchi Lei, Xiaoyang Liu, Yue Shen, Zhiqiang Zhang, Peng Wei, Jinjie Gu, Zhixuan Chu, Zhan Qin, et al. 2024a. A survey on medi- cal large language models: Technology, application, trustworthiness, and future directions. arXiv preprint arXiv:2406.03712. · arXiv
- Tianyu Liu, Yijia Xiao, Xiao Luo, Hua Xu, Wenjin Zheng, and Hongyu Zhao. 2024b. Geneverse: A collection of open-source multimodal large language models for genomic and proteomic research. In Find- ings of the Association for Computational Linguistics: EMNLP 2024, pages 4819-4836, Miami, Florida, USA. Association for Computational Linguistics. · DOI
- Zhengliang Liu, Yue Huang, Xiaowei Yu, Lu Zhang, Zihao Wu, Chao Cao, Haixing Dai, Lin Zhao, Yiwei Li, Peng Shu, et al. 2023b. Deid-gpt: Zero-shot med- ical text de-identification by gpt-4. arXiv preprint arXiv:2303.11032. · arXiv
- Meng Lu, Brandon Ho, Dennis Ren, and Xuan Wang. 2024. TriageAgent: Towards better multi-agents col- laborations for large language model-based clinical triage. In Findings of the Association for Computa- tional Linguistics: EMNLP 2024, pages 5747-5764, Miami, Florida, USA. Association for Computational Linguistics. · DOI
- Sazan Mahbub, Caleb Ellington, Sina Alinejad, Kevin Wen, Yingtao Luo, Ben Lengerich, and Eric P. Xing. 2024. From one to zero: RAG-IM adapts language models for interpretable zero-shot predictions on clin- ical tabular data. In NeurIPS 2024 Third Table Rep- resentation Learning Workshop.
- Shahrad Mohammadzadeh, Juan David Guerra, Marco Bonizzato, Reihaneh Rabbany, and Golnoosh Far- nadi. 2024. Hallucination detox: Sensitive neuron dropout (send) for large language model training. In Neurips Safe Generative AI Workshop 2024.
- Kyle O'Brien, Nathan Ng, Isha Puri, Jorge Mendez, Hamid Palangi, Yoon Kim, Marzyeh Ghassemi, and Thomas Hartvigsen. 2024. Improving black-box ro- bustness with in-context rewriting. arXiv preprint arXiv:2402.08225. · arXiv
- Jesutofunmi A Omiye, Jenna C Lester, Simon Spichak, Veronica Rotemberg, and Roxana Daneshjou. 2023. Large language models propagate race-based medicine. NPJ Digital Medicine, 6(1):195.
- Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. 2023. Med-HALT: Medical domain hallucination test for large language models. In Pro- ceedings of the 27th Conference on Computational Natural Language Learning (CoNLL), pages 314- 334, Singapore. Association for Computational Lin- guistics. · DOI · Paper page
- Xudong Pan, Mi Zhang, Shouling Ji, and Min Yang. 2020. Privacy risks of general-purpose language models. In 2020 IEEE Symposium on Security and Privacy (SP), pages 1314-1331. IEEE.
- Parth Patel, Nafise Moosavi, and Leon Derczynski. 2024. Evaluating biases in context-dependent sexual and reproductive health questions. In Findings of the Association for Computational Linguistics: ACL 2024.
- Stephen R Pfohl, Heather Cole-Lewis, Rory Sayres, Darlene Neal, Mercy Asiedu, Awa Dieng, Nenad Tomasev, Qazi Mamunur Rashid, Shekoofeh Azizi, Negar Rostamzadeh, et al. 2024. A toolbox for sur- facing health equity harms and biases in large lan- guage models. Nature Medicine, 30(12):3590-3600.
- Maxx Richard Rahman, Ruoxuan Liu, and Wolfgang Maass. 2024. Incorporating metabolic information into LLMs for anomaly detection in clinical time- series. In NeurIPS Workshop on Time Series in the Age of Large Models.
- Mourad Sarrouti, Asma Ben Abacha, Yassine Mrabet, and Dina Demner-Fushman. 2021. Evidence-based fact-checking of health-related claims. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3499-3512, Punta Cana, Do- minican Republic. Association for Computational Linguistics. · DOI · Paper page
- Samuel Schmidgall, Carl Harris, Ime Essien, Daniel Olshvang, Tawsifur Rahman, Ji Woong Kim, Rojin Ziaei, Jason Eshraghian, Peter Abadir, and Rama Chellappa. 2024. Evaluation and mitigation of cogni- tive biases in medical language models. npj Digital Medicine, 7(1):295.
- Amir Hassan Shariatmadari, Sikun Guo, Sneha Srini- vasan, and Aidong Zhang. 2024. Harnessing the power of knowledge graphs to enhance llm explain- ability in the biomedical domain,(2024). IJACSA) International Journal of Advanced Computer Science and Applications.
- Tanmay Singh, Harshvardhan Aditya, Vijay K Madis- etti, and Arshdeep Bahga. 2024. Whispered tuning: Data privacy preservation in fine-tuning llms through differential privacy. Journal of Software Engineering and Applications, 17(1):1-22.
- Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mah- davi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. 2023. Large language models encode clinical knowledge. Nature, 620(7972):172-180.
- Andries Petrus Smit, Paul Duckworth, Nathan Grinsz- tajn, Kale-ab Tessera, Thomas D Barrett, and Arnu Pretorius. 2023. Are we going mad? benchmark- ing multi-agent debate between language models for medical q&a. In Deep Generative Models for Health Workshop NeurIPS 2023.
- Yanqi Song, Ruiheng Liu, Shu Chen, Qianhao Ren, Yu Zhang, and Yongqi Yu. 2024. SecureSQL: Evalu- ating data leakage of large language models as natural language interfaces to databases. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 5975-5990, Miami, Florida, USA. Asso- ciation for Computational Linguistics. · DOI
- Akshay Swaminathan, Sid Salvi, Philip Chung, Alison Callahan, Suhana Bedi, Alyssa Unell, Mehr Kashyap, Roxana Daneshjou, Nigam Shah, and Dev Dash. 2024. Feasibility of automatically detecting practice of race-based medicine by large language models. In AAAI 2024 Spring Symposium on Clinical Founda- tion Models.
- Neşet Özkan Tan, Niket Tandon, David Wadden, Oyvind Tafjord, Mark Gahegan, and Michael Wit- brock. 2024. Faithful reasoning over scientific claims. In Proceedings of the AAAI Symposium Series, vol- ume 3, pages 263-272.
- Ruixiang Tang, Yu-Neng Chuang, Xuanting Cai, Meng- nan Du, and Xia Hu. 2024. Secure your model: An effective key prompt protection mechanism for large language models. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 4061-4073, Mexico City, Mexico. Association for Computational Linguistics. · DOI
- Tao Tu, Mike Schaekermann, Anil Palepu, Khaled Saab, Jan Freyberg, Ryutaro Tanno, Amy Wang, Brenna Li, Mohamed Amin, Yong Cheng, et al. 2025. To- wards conversational diagnostic artificial intelligence. Nature, pages 1-9.
- Tao Wang, Linhai Zhang, Chenchen Ye, Junxi Liu, and Deyu Zhou. 2022. A novel framework based on med- ical concept driven attention for explainable medical code prediction via external knowledge. In Find- ings of the Association for Computational Linguis- tics: ACL 2022, pages 1407-1416, Dublin, Ireland. Association for Computational Linguistics. · DOI
- Yichen Wang, Shangbin Feng, Abe Bohan Hou, Xiao Pu, Chao Shen, Xiaoming Liu, Yulia Tsvetkov, and Tianxing He. 2024. Stumbling blocks: Stress testing the robustness of machine-generated text detectors under attacks. arXiv preprint arXiv:2402.11638. · arXiv
- Qiyao Wei, Alex James Chan, Lea Goetz, David Watson, and Mihaela van der Schaar. 2024. Actions speak louder than words: Superficial fairness alignment in LLMs. In ICLR 2024 Workshop on Reliable and Responsible Foundation Models.
- Xinwei Wu, Weilong Dong, Shaoyang Xu, and Deyi Xiong. 2024. Mitigating privacy seesaw in large language models: Augmented privacy neuron edit- ing via activation patching. In Findings of the As- sociation for Computational Linguistics: ACL 2024, pages 5319-5332, Bangkok, Thailand. Association for Computational Linguistics. · DOI
- Sean Xie, Saeed Hassanpour, and Soroush Vosoughi. 2024. Addressing healthcare-related racial and LGBTQ+ biases in pretrained language models. In Findings of the Association for Computational Lin- guistics: NAACL 2024, pages 4451-4464, Mexico City, Mexico. Association for Computational Lin- guistics. · DOI
- Rui Xin, Niloofar Mireshghallah, Shuyue Stella Li, Michael Duan, Hyunwoo Kim, Yejin Choi, Yulia Tsvetkov, Sewoong Oh, and Pang Wei Koh. 2024. A false sense of privacy: Evaluating textual data san- itization beyond surface-level privacy leakage. In Neurips Safe Generative AI Workshop 2024.
- Qianqi Yan, Xuehai He, and Xin Eric Wang. 2024. Med- hvl: Automatic medical domain hallucination evalua- tion for large vision-language models. In AAAI 2024 Spring Symposium on Clinical Foundation Models.
- Junwei Yang, Zequn Liu, Ming Zhang, and Sheng Wang. 2022. Pathway2Text: Dataset and method for biomedical pathway description generation. In Findings of the Association for Computational Lin- guistics: NAACL 2022, pages 1441-1454, Seattle, United States. Association for Computational Lin- guistics. · DOI
- Xinyu Yang, Zichen Wen, Wenjie Qu, Zhaorun Chen, Zhiying Xiang, Beidi Chen, and Huaxiu Yao. 2024a. Memorization and privacy risks in domain-specific large language models. In ICLR 2024 Workshop on Reliable and Responsible Foundation Models.
- Yifan Yang, Xiaoyu Liu, Qiao Jin, Furong Huang, and Zhiyong Lu. 2024b. Unmasking and quantifying racial bias of large language models in medical report generation. Communications Medicine, 4(1). · DOI
- Lifan Yuan, Yangyi Chen, Ganqu Cui, Hongcheng Gao, Fangyuan Zou, Xingyi Cheng, Heng Ji, Zhiyuan Liu, and Maosong Sun. 2023. Revisiting out-of- distribution robustness in nlp: Benchmarks, analysis, and llms evaluations. Advances in Neural Informa- tion Processing Systems, 36:58478-58507.
- Travis Zack, Eric Lehman, Mirac Suzgun, Jorge A Ro- driguez, Leo Anthony Celi, Judy Gichoya, Dan Ju- rafsky, Peter Szolovits, David W Bates, Raja-Elie E Abdulnour, et al. 2024. Assessing the potential of gpt-4 to perpetuate racial and gender biases in health care: a model evaluation study. The Lancet Digital Health, 6(1):e12-e22.
- Gongbo Zhang, Qiao Jin, Denis Jered McInerney, Yong Chen, Fei Wang, Curtis L Cole, Qian Yang, Yan- shan Wang, Bradley A Malin, Mor Peleg, et al. 2024. Leveraging generative ai for clinical evidence syn- thesis needs to ensure trustworthiness. Journal of Biomedical Informatics, 153:104640.
- Haoran Zhang, Amy X Lu, Mohamed Abdalla, Matthew McDermott, and Marzyeh Ghassemi. 2020. Hurtful words: quantifying biases in clinical contextual word embeddings. In proceedings of the ACM Conference on Health, Inference, and Learning, pages 110-120.
- Jujia Zhao, Wenjie Wang, Chen Xu, Zhaochun Ren, See-Kiong Ng, and Tat-Seng Chua. 2024a. Llm- based federated recommendation. arXiv preprint arXiv:2402.09959. · arXiv
- Yutian Zhao, Huimin Wang, Yuqi Liu, Wu Suhuang, Xian Wu, and Yefeng Zheng. 2024b. Can LLMs replace clinical doctors? exploring bias in disease di- agnosis by large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 13914-13935, Miami, Florida, USA. Association for Computational Linguistics. · DOI
- Ge Zheng, Bin Yang, Jiajin Tang, Hong-Yu Zhou, and Sibei Yang. 2023. Ddcot: Duty-distinct chain-of- thought prompting for multimodal reasoning in lan- guage models. Advances in Neural Information Pro- cessing Systems, 36:5168-5191.
- Jiazheng Zhu, Shaojuan Wu, Xiaowang Zhang, Yuexian Hou, and Zhiyong Feng. 2023. Causal intervention for mitigating name bias in machine reading compre- hension. In Findings of the Association for Computa- tional Linguistics: ACL 2023, pages 12837-12852.
- Li Zhui, Li Fenghe, Wang Xuehu, Fu Qining, and Ren Wei. 2024. Ethical considerations and fundamen- tal principles of large language models in medical education. Journal of Medical Internet Research, 26:e60083.
Cited by
Sorted by citations
Trustworthy Medical Question Answering: An Evaluation-Centric Survey
Wang, Y., Wang, B., Mercer, R. E., Rudzicz, F., Roy, S. S., Ren, P., Chen, Z., & Wang, X. (2025) · arXiv · doi:10.48550/arxiv.2506.03659
“Yet, when deployed in clinical settings, these models continue to exhibit critical trust failures: hallucinated medical facts, unjustified over-confidence, and occasional biased or unsafe recommendations (Aljohani et al., 2025).”
Self-citation
Metadata
- Authors
- Aljohani, M., Hou, J., Kommu, S., & Wang, X.
- Affiliations
- Virginia Tech (US)
- ORCID
- Hou, Jun
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- Pages
- 6720-6748
- arXiv
- 2502.15871
- OpenAlex
- W4416035162
- Cited by
- 33
- References
- 0
- Citations by year
- 2026: 22 · 2025: 11
- FWCI
- 26.70
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (2442253)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- large language models, trustworthiness, honesty, privacy, safety, robustness, fairness, explainability, clinical decision support, LLM benchmark, agent cooperation, multimodal reasoning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review