Benchmarking the Confidence of Large Language Models in Clinical Questions
Open accessAbstract
Background and Aim: The capabilities of large language models (LLMs) to self-assess their own confidence in answering questions in the biomedical realm remain underexplored. This study evaluates the confidence levels of 12 LLMs across five medical specialties to assess their ability to accurately judge their responses. Methods: We used 1,965 multiple-choice questions assessing clinical knowledge from internal medicine, obstetrics and gynecology, psychiatry, pediatrics, and general surgery areas. Models were prompted to provide answers and to also provide their confidence for the correct answer (0–100). The confidence rates and the correlation between accuracy and confidence were analyzed. Results: There was an inverse correlation (r=-0.40, p=0.001) between confidence and accuracy, where worse performing models showed paradoxically higher confidence. For instance, a top performing model, GPT4o had a mean accuracy of 74% with a mean confidence of 63%, compared to a least performant model, Qwen-2-7B, which showed mean accuracy 46% but mean confidence 76%. The mean difference in confidence between correct and incorrect responses was low for all models, ranging from 0.6% to 5.4%, with GPT4o having the highest differentiation of 5.4%. Conclusion: Better performing LLMs show more aligned overall confidence levels. However, even the most accurate models still show minimal variation in confidence between right and wrong answers. This underscores an important limitation in current LLMs’ self-assessment mechanisms, highlighting the need for further research before integration into clinical settings.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Omar et al. (2024) have shown that across multiple specialties, current LLMs frequently assign high confidence to incorrect answers, revealing poor calibration in clinical settings.
“Studies such as Omar et al. (2024) have shown that across multiple specialties, current LLMs frequently assign high confidence to incorrect answers, revealing poor calibration in clinical settings.”
From Trustworthy Medical Question Answering: An Evaluation-Centric Survey (2025) · cites this paper as Omar et al., 2024
Verbalized confidence is known to be overconfident in clinical question answering.
“While verbalized confidence is known to be overconfident in clinical question answering [38], our results contradict recent work that suggests that consistency-based methods alleviate some of these concerns [39].”
From Identifying Deprescribing Opportunities With Large Language Models in Older Adults: Retrospective Cohort Study (2025) · cites this paper as [38]
Cites
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · see citing works
Large language models in medicine
Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., & Ting, D. S. W. (2023) · Nature Medicine · doi:10.1038/s41591-023-02448-8 · Cited by 3,944 · see citing works
The future landscape of large language models in medicine
Clusmann, J., Kolbinger, F. R., Muti, H. S., Carrero, Z. I., Eckardt, J.-N., Laleh, N. G., Löffler, C. M. L., Schwarzkopf, S.-C., Unger, M., Veldhuizen, G. P., et al. (2023) · Communications Medicine · doi:10.1038/s43856-023-00370-1 · Cited by 1,045 · see citing works
Sabiston Textbook of Surgery: The Biological Basis of Modern Surgical Practice
Townsend, C. M. (2007) · Andalas University Repository (Andalas University) · Cited by 807 · see citing works
Yang, G., Ye, Q., & Xia, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.07.016 · Cited by 717 · see citing works
Johnson, D., Goodman, R., Patrinely, J., Stone, C., Zimmerman, E., Donald, R., Chang, S., Berkowitz, S., Finn, A., Jahangir, E., et al. (2023) · doi:10.21203/rs.3.rs-2566942/v1 · Cited by 598 · see citing works
Ensemble Learning for Disease Prediction: A Review
Mahajan, P., Uddin, S., Hajati, F., & Moni, M. A. (2023) · Healthcare · doi:10.3390/healthcare11121808 · Cited by 460 · see citing works
Accuracy of a Generative Artificial Intelligence Model in a Complex Diagnostic Challenge
Kanjee, Z., Crowe, B., & Rodman, A. (2023) · JAMA · doi:10.1001/jama.2023.8288 · Cited by 449 · see citing works
Large language models and the perils of their hallucinations
Azamfirei, R., Kudchadkar, S. R., & Fackler, J. C. (2023) · Critical Care · doi:10.1186/s13054-023-04393-x · Cited by 324 · see citing works
Sezgın, E. (2023) · Digital Health · doi:10.1177/20552076231186520 · Cited by 279 · see citing works
Opening the black box of AI‐Medicine
Poon, A. I. F., & Sung, J. J. �. (2021) · Journal of Gastroenterology and Hepatology · doi:10.1111/jgh.15384 · Cited by 223 · see citing works
Hoch, C. C., Wollenberg, B., Lüers, J.-C., Knoedler, S., Knoedler, L., Frank, K., Cotofana, S., & Alfertshofer, M. (2023) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-023-08051-4 · Cited by 154 · see citing works
Large Language Models Are Poor Medical Coders — Benchmarking of Medical Code Querying
Soroush, A., Glicksberg, B. S., Zimlichman, E., Barash, Y., Freeman, R., Charney, A. W., Nadkarni, G. N., & Klang, E. (2024) · NEJM AI · doi:10.1056/aidbp2300040 · Cited by 137 · see citing works
Self-citationGPT versus Resident Physicians — A Benchmark Based on Official Board Scores
Katz, U., Cohen, E., Shachar, E., Somer, J., Fink, A., Morse, E., Shreiber, B., & Wolf, I. (2024) · NEJM AI · doi:10.1056/aidbp2300192 · Cited by 135 · see citing works
Black Box Warning: Large Language Models and the Future of Infectious Diseases Consultation
Schwartz, I. S., Link, K. E., Daneshjou, R., & Cortés-Penfield, N. (2024) · Clinical Infectious Diseases · doi:10.1093/cid/ciad633 · Cited by 131 · see citing works
Straw, I. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.101965 · Cited by 86 · see citing works
Giannos, P. (2023) · BMJ Neurology Open · doi:10.1136/bmjno-2023-000451 · Cited by 84 · see citing works
Glicksberg, B. S., Timsina, P., Patel, D., Sawant, A., Vaid, A., Raut, G., Charney, A. W., Apakama, D., Carr, B. G., Freeman, R., et al. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae103 · Cited by 82 · see citing works
Self-citationArtificial Intelligence-Based Ensemble Learning Model for Prediction of Hepatitis C Disease
Onyema, E. M., Dalal, S., Dhaou, I. B., Agubosim, C. C., Umoke, C. C., Richard-Nnabu, N. E., & Dahiya, N. (2022) · Frontiers in Public Health · doi:10.3389/fpubh.2022.892371 · Cited by 80 · see citing works
A Call for Artificial Intelligence Implementation Science Centers to Evaluate Clinical Effectiveness
Longhurst, C., Singh, K., Chopra, A., Atreja, A., & Brownstein, J. S. (2024) · NEJM AI · doi:10.1056/aip2400223 · Cited by 67 · see citing works
Yang, H., Li, M., Zhou, H., Xiao, Y., Fang, Q., Zhou, S., & Zhang, R. (2025) · Journal of Medical Internet Research · doi:10.2196/70080 · Cited by 64 · see citing works
Čartolovni, A., Malešević, A., & Poslon, L. (2023) · Digital Health · doi:10.1177/20552076231220833 · Cited by 51 · see citing works
Omar, M., Nassar, S., Hijazi, K., Glicksberg, B. S., Nadkarni, G. N., & Klang, E. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.109545 · Cited by 43 · see citing works
Self-citationOne LLM is not Enough: Harnessing the Power of Ensemble Learning for Medical Question AnsweringPreprint
Yang, H., Li, M., Zhou, H., et al. (2023) · medRxiv · doi:10.1101/2023.12.21.23300380 · Cited by 38 · see citing works
Paraphrasing to improve the performance of Electronic Health Records Question Answering.
Soni, S., & Roberts, K. E. (2020) · PubMed · Cited by 9 · see citing works
Omar, M., Nassar, S., Hijaze, K., et al. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4780940 · Cited by 3 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
A Brief Review on Benchmarking for Large Language Models Evaluation in Healthcare
Cilar, L., Chen, H., Chen, A., Topaz, M., Tam, W., Bian, J., & Štiglic, G. (2025) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70010 · Cited by 24 · see citing works
Socrates, V., Wright, D. S., Huang, T., Fereydooni, S., Dien, C., Chi, L., Albano, J., Patterson, B., Sasidhar Kanaparthy, N., Wright, C. X., et al. (2025) · JMIR Aging · doi:10.2196/69504 · Cited by 17 · see citing works
Visual-textual integration in LLMs for medical diagnosis: A preliminary quantitative analysis
Agbareia, R., Omar, M., Soffer, S., Glicksberg, B. S., Nadkarni, G. N., & Klang, E. (2024) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2024.12.019 · Cited by 17 · see citing works
Self-citationPristoupil, J., Oleaga, L., Junquero, V., Merino, C., Özbek, S. S., Lambert, L., & European Society of Radiology (ESR) (2025) · European Radiology Experimental · doi:10.1186/s41747-025-00591-0 · Cited by 6 · see citing works
Naderi, N., Safavi‐Naini, S. A. A., Savage, T., Khalafi, M. A., Lewis, P. R., Atf, Z., Nadkarni, G. N., & Soroush, A. (2026) · npj gut and liver. · doi:10.1038/s44355-026-00053-3 · Cited by 8 · see citing works
Self-citationHan, T., Shin, J., Lee, J. H., & Gu, K. (2026) · Journal of Medical Internet Research · doi:10.2196/86498 · Cited by 2 · see citing works
Probing Hidden States for Calibrated, Alignment-Resistant Predictions in LLMsPreprint
Berkowitz, J. S., Kivelson, S., Srinivasan, A., Gisladottir, U., Tsang, K. K., Cortina, J. M. A., Kuchi, A., Patock, J. R., Czarny, R., & Tatonetti, N. P. (2025) · medRxiv · doi:10.1101/2025.09.17.25336018 · Cited by 1 · see citing works
Trustworthy Medical Question Answering: An Evaluation-Centric Survey
Wang, Y., Wang, B., Mercer, R. E., Rudzicz, F., Roy, S. S., Ren, P., Chen, Z., & Wang, X. (2025) · arXiv · doi:10.48550/arxiv.2506.03659
“Studies such as Omar et al. (2024) have shown that across multiple specialties, current LLMs frequently assign high confidence to incorrect answers, revealing poor calibration in clinical settings.”
Han, T., Shin, J., Lee, J. H., & Gu, K. (2025) · doi:10.2196/preprints.86498
Benchmarking general and medical LLM performance in Parkinson's disease management
Shechter, Y., Klevor, R., Kouchache, T., Bouhadoun, S., & Postuma, R. B. (2026) · Clinical Parkinsonism & Related Disorders · doi:10.64898/2026.05.13.26353021
- Authors
- Omar, M., Agbareia, R., Glicksberg, B. S., Nadkarni, G. N., & Klang, E.
- Affiliations
- Icahn School of Medicine at Mount Sinai (US)
- Corresponding author
- Omar, Mahmud
- Venue
- medRxiv
- Publisher
- openRxiv
- Published
- 2024-08-11
- OpenAlex
- W4401498253
- Cited by
- 10
- References
- 24
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 1
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- medRxiv (acceptedVersion, cc-by-nc-nd)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- large language models, confidence calibration, self-assessment, clinical knowledge, multiple-choice questions, medical specialties, GPT-4o, benchmarking, clinical decision support
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
