Toward expert-level medical question answering with large language models
Open accessThe DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Abstract
Large language models (LLMs) have shown promise in medical question answering, with Med-PaLM being the first to exceed a 'passing' score in United States Medical Licensing Examination style questions. However, challenges remain in long-form medical question answering and handling real-world workflows. Here, we present Med-PaLM 2, which bridges these gaps with a combination of base LLM improvements, medical domain fine-tuning and new strategies for improving reasoning and grounding through ensemble refinement and chain of retrieval. Med-PaLM 2 scores up to 86.5% on the MedQA dataset, improving upon Med-PaLM by over 19%, and demonstrates dramatic performance increases across MedMCQA, PubMedQA and MMLU clinical topics datasets. Our detailed human evaluations framework shows that physicians prefer Med-PaLM 2 answers to those from other physicians on eight of nine clinical axes. Med-PaLM 2 also demonstrates significant improvements over its predecessor across all evaluation metrics, particularly on new adversarial datasets designed to probe LLM limitations (P < 0.001). In a pilot study using real-world medical questions, specialists preferred Med-PaLM 2 answers to generalist physician answers 65% of the time. While specialist answers were still preferred overall, both specialists and generalists rated Med-PaLM 2 to be as safe as physician answers, demonstrating its growing potential in real-world medical applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These models were chosen as they represent current leading LLMs3,36,40,41 as of February 2024.”
From An automated framework for assessing how well LLMs cite relevant medical references (2025) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“To account for the inherent stochasticity of LLMs, we report results from 10 independent runs per benchmark as mean accuracy with 95% confidence intervals (CIs), moving beyond the single-run point estimates common in prior works.2,3,4”
From Model confrontation and collaboration: A debate intelligence framework for enhancing medical reasoning in large language models (2026) · cites this paper as [2]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“medical domain-specific LLMs such as Medical Pathways Language Model (Med-PaLM) [2] and Biomedical Generative Pretrained Transformer (BioGPT) [3] have shown improved performance across various clinical natural language processing (NLP) tasks, including question answering, summarization, entity recognition, relation extraction, document classification, and literature mining.”
From Retrieval-Augmented Generation (RAG) in Healthcare: A Comprehensive Review (2025) · cites this paper as [2]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs have demonstrated strong performance across a wide range of biomedical knowledge and reasoning tasks, accelerating their adoption in health care workflows [1-6].”
From Retrieve-Then-Verify for Evaluating Evidence Support and Hallucination in Large Language Model–Generated Medical Information: Empirical Study (2026) · cites this paper as [4]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Large language models (LLMs) like OpenAI’s GPT-5 and knowledge-specific systems, such as OpenEvidence, have reported the ability to generate patient-specific, evidence-based recommendations.10”
From Large Language Model–Generated Electronic Consultations: A Blinded Comparative Study of OpenEvidence and Gastroenterologist Electronic Consultations (2026) · cites this paper as [10]
Experimental — accuracy not yet measured
9 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“The advent of transformers 4 and LLMs 5,6 has renewed interest in the possibilities of AI for medical question-answering tasks—a long-standing ‘grand challenge’ 7–9 .”
THE USE OF CONFIDENCE OR FIDUCIAL LIMITS ILLUSTRATED IN THE CASE OF THE BINOMIAL
CLOPPER, C. J., & PEARSON, E. S. (1934) · Biometrika · doi:10.1093/biomet/26.4.404 · Cited by 5,003
New creatinine- and cystatin C-based equations to estimate GFR without race
Inker, L. A., Eneanya, N. D., Coresh, J., Tighiouart, H., Wang, D., Sang, Y., Crews, D. C., Doria, A., Estrella, M. M., Froissart, M., et al. (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2102953 · Cited by 4,695
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · cited 1×
“This progress is exemplified by recent iterations within prominent LLM families like GPT (GPT-4, GPT-4o, GPT-4o1) 37 , Gemini (Gemini 1.0, Gemini 1.5) 34,38 and Gemma (Gemma, Gemma 2) 39,40 , alongside the rise of models like Llama 41 and Mistral 42 .”
InfluentialChain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263 · cited 1×
“55, involves augmenting each few-shot example in a prompt with a step-by-step explanation toward the final answer.”
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
Self-citationLarge language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083
Self-citation
Research works printed but not linked: 4
- 5. Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. Bert: pre-training of deep bidirectional transformers for language understanding. In Proc. NAACL-HLT Vol. 1 (eds Burstein, J. et al.) 4171-4186 (Association for Computational Linguistics, 2019).
- 10. Bobrow, D. G.) 167-180 (MIT Press, 1994).
- 38. Achiam, J. et al. GPT-4 technical report. Preprint at https://arxiv. org/abs/2303.08774 (2023).
- 63. Randolph, Justus J. 2005 "Free-Marginal Multirater Kappa (multirater K [free]): An Alternative to Fleiss' Fixed-Marginal Multirater Kappa." Presented at the Joensuu Learning and Instruction Symposium, vol. 2005 https://eric.ed.gov/?id=ED490661
Cited by
Items 401–500 of 1,031
A review of medical text analysis: Theory and practice
Chen, Y., Zhang, C., Bai, R., Sun, T., Ding, W., & Wang, R. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103024 · Cited by 13
Ghorbian, M., Ghobaei‐Arani, M., & Ghorbian, S. (2025) · Methods · doi:10.1016/j.ymeth.2025.04.001 · Cited by 13
CMedRAGBot: A Chinese Medical Chatbot Based on Graph RAG and Large Language Models
Zhang, D., Du, H., Wang, X., Zhu, M., Pang, X., Wei, D., & Wang, X. (2025) · Interdisciplinary Sciences Computational Life Sciences · doi:10.1007/s12539-025-00715-5 · Cited by 19
MID-LLM: Enhancing Medical Image Diagnostics With LLMs in a Blockchain AI Framework
Kumar, R., Rao, Y., Kumar, J., Mawuli, C. B., Ali, W., & Zeng, S. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3583408 · Cited by 14
“5) PreTrained Initialization [35]: Uses models like Med-PaLM to minimize training effort.”
Wang, B., Liu, Y., Tian, H., Hua, R., Chang, K., Xia, J., Dai, X., Gao, Z., Liu, S., Wang, R., et al. (2025) · Tsinghua Science & Technology · doi:10.26599/tst.2024.9010125 · Cited by 14
Self-citationPr????mpt: Sanitizing Sensitive Prompts for LLMs
Chowdhury, A. R., Glukhov, D., Anshumaan, D., Chalasani, P., Papernot, N., Jha, S., & Bellare, M. (2026) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2026.231277 · Cited by 37
Szabó, A., & Laein, G. D. (2025) · Scientific Reports · doi:10.1038/s41598-025-17571-4 · Cited by 17
Owais, M., Zubair, M., Kareem, D. N. A., Naseer, S. A., Alam, M., Ahmad, M., & Hussain, I. (2026) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-026-10738-w · Cited by 21
Scaffolding for success: Blending learning with and about Generative AI in medical education
Boman, M., Jhun, P., & Schaekermann, M. (2025) · Medical Teacher · doi:10.1080/0142159x.2025.2571041 · Cited by 18
“…be a valuable resource in this evolving landscape, as we continue to see remarkable advances in GenAI and its ability to encode clinical knowledge [6,7], and to engage in clinical reasoning tasks, such as interactive history-taking and differential diagnosis generation, akin to human experts [8,9].”
Self-citationHolistic AI in medicine; improved performance and explainability
Petridis, P., Margaritis, G., Stoumpou, V., & Bertsimas, D. (2026) · npj Digital Medicine · doi:10.1038/s41746-025-02298-3 · Cited by 17
Gong, E. J., Bang, C. S., & Shin, Y. S. (2026) · Bioengineering · doi:10.3390/bioengineering13030365 · Cited by 19
“Specialized medical models trained on biomedical literature show promise for improved domain-specific performance, though validation frameworks and bias assessment remain essential [122].”
Rahit, K. M. T. H., Avramović, V., Chong, J. X., & Tarailo‐Graovac, M. (2024) · BMC Bioinformatics · doi:10.1186/s12859-024-05693-x · Cited by 13
Zhao, W., Deng, Z., Yadav, S., & Yu, P. S. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589335.3651941 · Cited by 18
Elek, A., Ekizalioğlu, D. D., & Güler, E. (2024) · Diagnostic and Interventional Radiology · doi:10.4274/dir.2024.232680 · Cited by 18
Yuan, Z., Liu, K., Peng, R., Li, S., Leybourne, D. J., Musa, N., Huang, H., & Yang, P. (2024) · IEEE Internet of Things Magazine · doi:10.1109/iotm.001.2400036 · Cited by 21
Fukushima, T., Manabe, M., Yada, S., Wakamiya, S., Yoshida, A., Urakawa, Y., Maeda, A., Kan, S., Takahashi, M., & Aramaki, E. (2024) · JMIR Medical Informatics · doi:10.2196/65047 · Cited by 13
Momand, Z., Mongkolnam, P., Chan, J. H., & Charoenkitkarn, N. (2025) · Sensors and Materials · doi:10.18494/sam5387 · Cited by 13
He, Y., Zhu, X., Li, D., & Wang, H. (2025) · Electronics · doi:10.3390/electronics14101961 · Cited by 30
“Med-PaLM demonstrated near-expert performance on medical QA benchmarks like MedQA [23], illustrating the potential of LLMs in vertical applications.”
Dennstädt, F., Windisch, P. Y., Filchenko, I., Zink, J., Putora, P. M., Shaheen, A., Gaio, R., Čihorić, N., Wosny, M., Aeppli, S., et al. (2025) · Oncology · doi:10.1159/000546946 · Cited by 16
Enhancing Ultrahigh Resolution Remote Sensing Imagery Analysis With ImageRAG: A new framework
Zhang, Z., Shen, H., Zhao, T., Guan, Z., Chen, B., Wang, Y., Xu, J. W., Cai, Y., Shang, Y., & Yin, J. (2025) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2025.3574742 · Cited by 22
“6B in real-world deployment scenarios or domain-specific LLMs such as Med-PaLM 2 [65] when dealing with UHR medical images.”
AI Veterinary Assistance: Enhancing Clinical Decision-Making in Animal Healthcare
Jin, Y.-G., Wu, G., Seo, J. Y., Park, S.-J., Hur, S.-H., Aliyeva, D., Park, J.-H., & Kim, K.-M. (2025) · IEEE Access · doi:10.1109/access.2025.3587787 · Cited by 14
“Recent developments in large language models (LLMs) [27], such as ChatGPT, have accelerated the creation of AI technologies capable of analyzing medical data and assisting in consultations [28], [29].”
Reviewing clinical knowledge in medical large language models: Training and beyond
Li, Q. Y., Liu, H., Guo, C., Gao, C., Chen, D., Wang, M., Gao, F., van Harmelen, F., & Gu, J. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114215 · Cited by 20
A review of large language models and the recommendation task
Munson, J., Cuezze, T., Nesar, M. S. B., & Zosso, D. P. (2025) · Discover Artificial Intelligence · doi:10.1007/s44163-025-00334-5 · Cited by 16
Nakaura, T., Uetani, H., Yoshida, N., Kobayashi, N., Nagayama, Y., Kidoh, M., Kuroda, J., Mukasa, A., & Hirai, T. (2025) · European Radiology · doi:10.1007/s00330-025-11924-3 · Cited by 12
Gao, Y., Yuan, Y., Wang, K., Wang, Y., Gao, T., Yang, Y., Ma, L., Li, R., Wang, G., & Liu, X. (2025) · Patterns · doi:10.1016/j.patter.2025.101347 · Cited by 15
“Med-PaLM 2, 113,114 a medi-cal-specific LLM, could assist clinicians in making informed decisions by analyzing patient data and medical literature, providing evidence-based recommendations for personalized IVF treatment plans.”
Large Language Models in Lung Cancer: Systematic Review
Zhong, R., Chen, S., Li, Z., Gao, T., Su, Y., Zhang, W., Liu, D., Gao, L., & Hu, K. (2025) · Journal of Medical Internet Research · doi:10.2196/74177 · Cited by 14
“For example, Med-PaLM 2 [68], which excels at lengthy medical question-answering; BioBERT [69], which specializes in biomedical texts; and ClinicalBERT [70], which focuses on clinical texts.”
Anh, N. V., Vuong, T. Q. T., & Nguyen, V. H. (2025) · PLoS ONE · doi:10.1371/journal.pone.0335775 · Cited by 15
“At the frontier, domain-aligned models have reached expert-level performance on medical question answering [17].”
Embeddings of clinical codes enable knowledge-grounded AI in medicine
Johnson, R., Gottlieb, U., Shaham, G., Eisen, L., Waxman, J. G., Devons-Sberro, S., Ginder, C. R., Hong, P. J., Sayeed, R., Su, X., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02664-9 · Cited by 17
“ClinVec embeddings can be incorporated into foundation models, such as LLMs, through knowledge injection 76 – 79 (Fig.”
Zsidai, B., Kaarre, J., Hilkert, A., Narup, E., Senorski, E. H., Grassi, A., Ayeni, O. R., Musahl, V., Ley, C., Herbst, E., et al. (2023) · Journal of Experimental Orthopaedics · doi:10.1186/s40634-023-00662-4 · Cited by 14
MEDITRON: Open Medical Foundation Models Adapted for Clinical PracticePreprint
Chen, Z., Romanou, A., Bonnet, A., Hernández-Cano, A., AlKhamissi, B., Matoba, K., Salvi, F., Pagliardini, M., Fan, S., Köpf, A., et al. (2024) · Research Square · doi:10.21203/rs.3.rs-4139743/v1 · Cited by 31
Large Language Model Uncertainty Measurement and Calibration for Medical Diagnosis and Treatment
Savage, T., Wang, J., Gallo, R., Boukil, A., Patel, V., Ahmad Safavi-Naini, S. A., Soroush, A., & Chen, J. H. (2024) · doi:10.1101/2024.06.06.24308399 · Cited by 11
Self-citationMondal, H., Dash, I., Mondal, S., Varikasuvu, S. R., Gayen, R. K., Sharma, S., & Biri, S. K. (2025) · Expert Review of Clinical Pharmacology · doi:10.1080/17512433.2025.2568090 · Cited by 14
“These models have demonstrated capabilities in medical question answering [5], clinical decision support [6], and biomedical knowledge retrieval [7].”
Large Language Models in Healthcare: A Review
Zou, S., & He, J. (2023) · International Symposium on Computer Science and Intelligent Control (ISCSIC) · doi:10.1109/iscsic60498.2023.00038 · Cited by 15
Comparing Vision-Capable Models, GPT-4 and Gemini, With GPT-3.5 on Taiwan’s Pulmonologist Exam
Chen, C., Hsieh, K.-Y., Huang, K.-E., & Lai, H.-Y. (2024) · Cureus · doi:10.7759/cureus.67641 · Cited by 13
Evaluating Large Language Models for Health-related Queries with Presuppositions
Kaur, N., Choudhury, M., & Pruthi, D. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.850 · Cited by 32
Cao, Y., Lü, W., Shi, R., Liu, F., Liu, S. H., Xu, X., Yang, J., Rong, G., Xin, C., Zhou, X., et al. (2024) · Advances in Ophthalmology Practice and Research · doi:10.1016/j.aopr.2024.12.002 · Cited by 12
Nazar, W., Nazar, G., Kamińska, A., & Daniłowicz‐Szymanowicz, L. (2025) · Journal of Medical Internet Research · doi:10.2196/70481 · Cited by 20
“Moreover, incorporating evidence-based medical information is crucial for patient safety [11,21,22].”
InfluentialPrompting Medical Large Vision-Language Models to Diagnose Pathologies by Visual Question Answering
Guo, D., & Terzopoulos, D. (2025) · The Journal of Machine Learning for Biomedical Imaging · doi:10.59275/j.melba.2025-1a8b · Cited by 20
Contrastive learning with large language models for medical code prediction
Wu, Y., Zhang, J., Chen, X., Yao, X., & Chen, Z. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127241 · Cited by 18
Li, H., Cheng, X., & Xiao-ping, Z. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713526 · Cited by 19
Pawelczyk, J., Kraus, M., Voigtlaender, S., Siebenlist, S., & Rupp, M. (2025) · HSS Journal® The Musculoskeletal Journal of Hospital for Special Surgery · doi:10.1177/15563316251341321 · Cited by 21
“Recent breakthroughs in the generalist medical foundation model domain, such as Google’s MedPaLM 2 [57], offer an exciting glimpse into a future where healthcare is augmented by AI models with generalist medical capabilities approaching those of human physicians across several dimensions such as…”
Ahn, J., Kang, B. G., Chang, M. Y., & Yoon, S. (2025) · Clinical and Experimental Otorhinolaryngology · doi:10.21053/ceo.2025-00121 · Cited by 10
Zheng, W., & Arakawa, Y. (2025) · IEEE Access · doi:10.1109/access.2025.3598088 · Cited by 13
“Driven by recent progress in LLMs, several health robots have been developed [184], [185], [186], [187].”
Li, B., Huang, Y., Mao, W., Liu, J., & Ma, Q. (2025) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s548258 · Cited by 13
“67,68 They may also enhance patient self-management through interactive health education and psychological support.”
Cheng, P., Zhang, K., Lyu, M., Liu, H., Sun, L., & Wu, Y. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104946 · Cited by 19
Gao, S., Zhao, X., Chen, L., Yu, J., Tian, S., Zhou, H., Chen, J., Long, S., He, Q., & Feng, X. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02139-3 · Cited by 15
Self-citationChoi, J. Y., & Yoo, T. K. (2026) · Discover Health Systems · doi:10.1007/s44250-026-00343-w · Cited by 14
“Based on the PaLM-2 architecture, Med-PaLM 2 achieved 86.5% accuracy on the MedQA benchmark, significantly surpassing earlier models [32].”
InfluentialNatural language generation in healthcare: A review of methods and applications
Lyu, M., Li, X., Chen, Z., Pan, J., Peng, C., Talankar, S., & Wu, Y. (2026) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2026.104997 · Cited by 19
A survey on memory-efficient transformer-based model training in AI for science
Tian, K., Qiao, L., Liu, B., Jiang, G., Li, S., & Li, D. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-025-50302-6 · Cited by 16
Current knowledge and availability of machine learning across the spectrum of trauma science
Gauss, T., Perkins, Z., & Tjardes, T. (2023) · Current Opinion in Critical Care · doi:10.1097/mcc.0000000000001104 · Cited by 13
ERD: A Framework for Improving LLM Reasoning for Cognitive Distortion Classification
Lim, S., Kim, Y., Choi, C.-H., Sohn, J.-Y., & Kim, B.-H. (2024) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2024.clinicalnlp-1.25 · Cited by 28
Designing Retrieval-Augmented Language Models for Clinical Decision Support
Quigley, K., Koker, T., Taylor, J., Mancuso, V., & Brattain, L. J. (2024) · Studies in computational intelligence · doi:10.1007/978-3-031-63592-2_13 · Cited by 8
Explanatory argument extraction of correct answers in resident medical exams
Goenaga, I., Atutxa, A., Gojenola, K., Oronoz, M., & Agerri, R. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102985 · Cited by 17
Rawat, R., McBride, H., Nirmal, D. C., Ghosh, R., Moon, J. H., Alamuri, D. K., & Zhu, K. (2024) · Workshop on NLP for Positive Impact (NLP4PI) · doi:10.18653/v1/2024.nlp4pi-1.29 · Cited by 28
Container-Based Data Pipelines on the Computing Continuum for Remote Patient Monitoring
Nikolov, N., Solberg, A., Prodan, R., Soylu, A., Matskin, M., & Roman, D. (2023) · Computer · doi:10.1109/mc.2023.3285414 · Cited by 14
Thieme, A., Rajamohan, A., Cooper, B., Groombridge, H., Simister, R., Wong, B., Woznitza, N., Pinnock, M. A., Wetscherek, M. T., Morrison, C., et al. (2024) · arXiv · doi:10.48550/arxiv.2405.05299 · Cited by 10
Gender Bias in Decision-Making with Large Language Models: A Study of Relationship Conflicts
Levy, S., Adler, W. D., Karver, T. S., Dredze, M. H., & Kaufman, M. R. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.331 · Cited by 31
A Novel RAG Framework with Knowledge-Enhancement for Biomedical Question Answering
Du, Y., Wang, Z., Wang, B., Jin, X., & Yu, P. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822837 · Cited by 11
Comparing dentist and chatbot answers to dental questions for quality and empathy
Maruska, E. E., Elgreatly, A., Madaio, W., Razoky, K., Bay, R. C., & Mahrous, A. (2025) · JADA Foundational Science · doi:10.1016/j.jfscie.2025.100044 · Cited by 17
Du, R. F., Carbonell, E. L., Huang, J., Liu, S., Wang, X., Shen, D. G., & Ke, J. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3551913 · Cited by 19
Generative artificial intelligence powered chatbots in urology
Khawaja, Z., Adhoni, M. Z. U., & Byrnes, K. G. (2025) · Current Opinion in Urology · doi:10.1097/mou.0000000000001280 · Cited by 10
Generative AI and the profession of genetic counseling
Meekins‐Doherty, L., Dive, L. L., McEwen, A., & Sexton, A. (2025) · Journal of Genetic Counseling · doi:10.1002/jgc4.2009 · Cited by 13
LightLLM: A Versatile Large Language Model for Predictive Light Sensing
Hu, J., Jia, H. G., Hassan, M., Yao, L., Kusý, B., & Hu, W. S. (2025) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3715014.3722067 · Cited by 20
Li, X., Hou, X., Ravi, N., Huang, Z., & Gan, Y. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127833 · Cited by 18
Large language models for Alzheimer's disease drug discovery
Alkam, T., Tarshizi, E., & Benschoten, A. H. V. (2025) · Journal of Alzheimer s Disease · doi:10.1177/13872877251346890 · Cited by 11
“56 When using real-world medical questions, specialists preferred fi ne-tuned LLM answers to generalist physician answers 65% of the time.”
Chandra, M., Sriraman, S., Verma, G., Khanuja, H. S., Campayo, J. S., Li, Z., Birnbaum, M. L., & Choudhury, M. D. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.553 · Cited by 27
Miranda, J., Pereira-Silva, R., Guichard, J., Meneses, J., Carreira, A. N., & Seixas, D. (2025) · Bioengineering · doi:10.3390/bioengineering12060653 · Cited by 15
“Furthermore, expert-level medical question-answering models such as Med-PaLM 2 have exhibited performance on par with practising clinicians across multiple medical question benchmarks, and their answers were frequently preferred by both physicians and laypeople [10].”
Üçdal, M. T., Bakhshandehpour, A., Durak, M. B., Balaban, Y. H., Kekilli, M., & Şimşek, C. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14124347 · Cited by 9
“…Med-PaLM, a PaLM-based LLM fine-tuned on medical data, was the first to exceed a passing score on the MedQA benchmark (67.6% accuracy) and has been further improved in Med-PaLM 2, which attains ~86.5% accuracy on MedQA and ~72.3% on the MedMCQA dataset of Indian medical licensing questions [15].”
InfluentialVerification is All You Need: Prompting Large Language Models for Zero-Shot Clinical Coding
Li, S., Zheng, C., Wu, J., Xu, Q., Xu, X., Wang, H., Sun, Y., Bai, Z., Xu, Y., Zhu, L., et al. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3593028 · Cited by 13
Shemtob, L., Nouri, A. G., Harvey-Sullivan, A., Qiu, C. S., Martin, J., Martin, M., Noden, S., Rob, T., Neves, A. L., Majeed, A., et al. (2025) · JAMIA Open · doi:10.1093/jamiaopen/ooaf082 · Cited by 12
Aligning large language models for cognitive behavioral therapy: a proof-of-concept study
Kim, Y., Choi, C.-H., Cho, S., Sohn, J.-Y., & Kim, B.-H. (2025) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2025.1583739 · Cited by 22
A survey of large language models for data challenges in graphs
Li, M., Zhang, P., Xing, W., Zheng, Y., Zaporojets, K., Chen, J., Zhang, R., Zhang, Y., Gong, S., Hu, J. S., et al. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129643 · Cited by 20
Self-citationBhadra, S., Pundir, C., Mukherjee, M., Kar, A., Banerjee, S., Charoensup, R., Duangyod, T., & Mukherjee, P. K. (2025) · Pharmacological Research · doi:10.1016/j.phrs.2025.108002 · Cited by 14
Alexandrou, M., Kumar, S., Mahtani, A. U., Strepkos, D., Carvalho, P. E. P., Mutlu, D., Ser, Ö. S., Rempakos, A., Mastrodemos, O. C., Rangan, B. V., et al. (2025) · JACC: Cardiovascular Interventions · doi:10.1016/j.jcin.2025.08.019 · Cited by 11
Xie, S., Cai, H., Sun, Y., & Lv, X. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104951 · Cited by 13
Šimunović, I., Rezić, K., Franić, N., Boduljak, G., Batinić, M., Jukić, I. K. D., Jelovina, I., Biočić, J., Pogorelić, Z., & Markić, J. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03258-3 · Cited by 9
“Prior studies provide strong evidence that medical large language models possess substantial clinical knowledge [16, 17].”
Artificial intelligence‐enabled digital biomedical engineering
Song, P., Tang, X., Lv, X., Reis, R. L., Chen, X., Bai, L., & Su, J. (2025) · BMEMat · doi:10.1002/bmm2.70049 · Cited by 10
Salehin, I., Sajib, M. T. A., Badhon, N. H., Rifat, M. S. H., Amin, N., & Moon, N. N. (2025) · Engineering Reports · doi:10.1002/eng2.70365 · Cited by 13
Boie, S. D., Giesa, N., Sekutowicz, M., Zhumagambetov, R., Haufe, S., Grünewald, E., & Balzer, F. (2026) · Frontiers in Medicine · doi:10.3389/fmed.2026.1746867 · Cited by 11
Prompt mechanisms in medical imaging: A comprehensive survey
Yang, H., Liang, X., Li, Z., Sun, Y., Hu, Z., Xie, X., Dashtbozorg, B., Huang, J., Zhu, S., Han, L., et al. (2026) · The Innovation · doi:10.1016/j.xinn.2026.101271 · Cited by 9
“Key examples include large language models (LLMs) such as GPT-4 20,21 and Med-PaLM, 22,23 which are applied to tasks such as radiological report generation and clinical question answering.”
Han, C., Yang, G., Li, H., Zhu, L., & (11284364), M. F. (2026) · Chinese Medicine · doi:10.1186/s13020-026-01346-8 · Cited by 12
“Second, severe multimodal data scarcity exists—studies rely predominantly on text despite TCM’s requirement for integrated diagnostic inputs including tongue images, pulse recordings, and facial features [71].”
Evaluating large language models for evidence-based clinical question answering
Wang, C., & Chen, Y. T. (2026) · Patterns · doi:10.1016/j.patter.2026.101519 · Cited by 13
“Large language models (LLMs) have demonstrated strong capabilities in open-domain and medical question answering (QA) and reasoning, 1,2 but their performance in complex, evidence-based clinical domains remains an active area of exploration.”
Yang, Y., Wang, Z.-Y., Liu, Q., Sun, S., Wang, K., Chellappa, R., Zhou, Z., Yuille, A., Zhu, L., Zhang, Y.-D., et al. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00779 · Cited by 28
“However, recent studies [26] show that LLMs, even those specifically tailored for medicine [13, 54, 55], diagnose significantly worse than clinicians and make less informed treatment decisions.”
Self-citationTutorial: guidance on the use of large language models for medical research
Jin, Q., Wan, N., Leaman, R., Tian, S., Wang, Z., Yang, Y., Wang, Z., Xiong, G., Lai, P.-T., Zhu, Q. L., et al. (2026) · Nature Protocols · doi:10.1038/s41596-026-01408-z · Cited by 16
Quality of ChatGPT Responses to Frequently Asked Questions in Carpal Tunnel Release Surgery
Amen, T. B., Torabian, K. A., Subramanian, T., Yang, B. W., Liimakka, A. P., & Fufa, D. T. (2024) · Plastic & Reconstructive Surgery Global Open · doi:10.1097/gox.0000000000005822 · Cited by 13
ChatASD: LLM-Based AI Therapist for ASD
Ren, X., Bai, Y., Duan, H., Fan, L., Fei, E., Wu, G., RAY, P. K., Hu, M., Yan, C., & Zhai, G. (2024) · Communications in computer and information science · doi:10.1007/978-981-97-3626-3_23 · Cited by 14
Large Language Models Based Stemming for Information Retrieval: Promises, Pitfalls and Failures
Wang, S., Zhuang, S., & Zuccon, G. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657949 · Cited by 11
Survey of Large Language Models for Answering Questions Across Various Fields
Shailendra, P., Ghosh, R. C., Kumar, R., & Sharma, N. (2024) · International Conference on Advanced Computing and Communication Systems (ICACCS) · doi:10.1109/icaccs60874.2024.10717078 · Cited by 24
Huntsman, D. D., & Bulaj, G. W. (2025) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph22020225 · Cited by 11
“The integration of e-commerce and digital health platforms that utilize AI/LLM technologies can facilitate personalized health communications, but such technologies face challenges such as fairness, bias and trust [342,343].”
Exploring Large Language Models for Specialist-Level Oncology Care
Palepu, A. K., Dhillon, V., Niravath, P. A., Weng, W., Prasad, P., Saab, K. K., Tanno, R., Cheng, Y., Mai, H., Burns, E. A., et al. (2025) · NEJM AI · doi:10.1056/aics2500025 · Cited by 18
Self-citationGenetic Discovery Enabled by A Large Language ModelPreprint
Tu, T., Fang, Z., Cheng, Z., Spasić, S., Palepu, A. K., Stanković, K. M., Natarajan, V., & Peltz, G. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.11.09.566468 · Cited by 12
Self-citationThaker, N. G., Redjal, N., Loaiza‐Bonilla, A., Penberthy, D. R., Showalter, T., Choudhri, A., Williamson, S. K., Thaker, G. H., Shah, C., Ward, M. C., et al. (2024) · AI in Precision Oncology · doi:10.1089/aipo.2023.0007 · Cited by 9
Addressing the Productivity Paradox in Healthcare with Retrieval Augmented Generative AI Chatbots
Ranasinghe, S., Silva, D. D., Mills, N., Alahakoon, D., Manic, M., Lim, Y., & Ranasinghe, W. K. B. (2024) · IEEE International Conference on Industrial Technology (ICIT) · doi:10.1109/icit58233.2024.10540818 · Cited by 10
Özkan, E., & Boix, X. (2024) · Scientific Reports · doi:10.1038/s41598-024-73561-y · Cited by 11
Zhang, S., Qi, C., Li, Y., Liu, J., Wang, J., Yan, C., Liu, W., Wang, Y., Zhao, C., Zhang, X., et al. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1442975 · Cited by 7
Rani, M., Mishra, B. K., Thakker, D., Babar, M. A., Jones, W., & Din, A. (2024) · International Conference on IT and Industrial Technologies (ICIT) · doi:10.1109/icit63607.2024.10859641 · Cited by 10
Exploring finetuned audio-LLM on heart murmur features
Florea, A. N., Jiang, X., Mesgarani, N., & Jiang, X. (2025) · Smart Health · doi:10.1016/j.smhl.2025.100557 · Cited by 16
Natural and artificial intelligence – the psychotechnical agenda of the 21st century
Webster, C. S. (2025) · Journal of Psychology and AI · doi:10.1080/29974100.2025.2491445 · Cited by 11
Explainable medical visual question answering via chain of evidence
Qiu, C., Huang, K., Xie, Z., Liu, M., Gu, J., & Zong, X. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113672 · Cited by 13
A survey for wearable sensors empowering smart healthcare in the era of large language models
Liu, Y., Li, J., Lian, C., Zhang, X., Gong, J., Wang, Y., & Zhao, Y. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103409 · Cited by 12
Self-citation
Metadata
- Authors
- Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Amin, M., Hou, L., Clark, K., Pfohl, S. R., Cole-Lewis, H., Neal, D., Rashid, Q. M., Schaekermann, M., Wang, A., Dash, D., Chen, J. H., Shah, N. H., Lachgar, S., Mansfield, P. A., Prakash, S., Green, B., Dominowska, E., Agüera y Arcas, B., Tomašev, N., Liu, Y., Wong, R., Semturs, C., Mahdavi, S. S., Barral, J. K., Webster, D. R., Corrado, G. S., Matias, Y., Azizi, S., Karthikesalingam, A., & Natarajan, V.
- Affiliations
- Google (United States) (US)
- Stanford Medicine (US)
- Stanford Health Care (US)
- Corresponding author
- Azizi, Shekoofeh
- Karthikesalingam, Alan
- Natarajan, Vivek
- ORCID
- Singhal, Karan
- Tu, Tao
- Gottweis, Juraj
- Sayres, Rory
- Amin, Mohamed
- Clark, Kevin
- Pfohl, Stephen R.
- Cole-Lewis, Heather
- Rashid, Qazi Mamunur
- Schaekermann, Mike
- Wang, Amy
- Dash, Dev
- Chen, Jonathan H.
- Shah, Nigam H.
- Lachgar, Sami
- Mansfield, Philip Andrew
- Prakash, Sushant
- Green, Bradley
- Dominowska, Ewa
- Tomašev, Nenad
- Liu, Yun
- Wong, Renee
- Semturs, Christopher
- Mahdavi, S. Sara
- Barral, Joelle K.
- Webster, Dale R.
- Corrado, Greg S.
- Matias, Yossi
- Azizi, Shekoofeh
- Karthikesalingam, Alan
- Natarajan, Vivek
- Venue
- Nature Medicine
- Published
- 2025-01-08
- Volume
- 31
- Issue
- 3
- Pages
- 943-950
- ISSN
- 1078-8956 (ISSN-L)
- 1546-170X
- PMC
- PMC11922739
- OpenAlex
- W4406152279
- Semantic Scholar
- b34c0b0169925a6c7f14e3de9764ed9505f30de3
- Cited by
- 1,734
- References
- 57
- Citations by year
- 2026: 771 · 2025: 591
- FWCI
- 277.36
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-nd
- creativecommons.org/licenses/by-nc-nd/4.0 (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0 (vor)
- Locations
- Nature Medicine (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Gordon and Betty Moore Foundation (#12409)
- American Heart Association
- National Institutes of Health (UM1TR004921)
- Stanford Bio-X
- National Institute on Drug Abuse (UG1DA015815)
- National Institute of Allergy and Infectious Diseases (R01AI178121)
- National Center for Advancing Translational Sciences
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- medical question answering, large language models, ensemble refinement, clinical reasoning, grounding, USMLE, MedQA, physician preferences
- MeSH terms
- Large Language Models, Humans, Physicians, Pilot Projects, United States
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.