Toward expert-level medical question answering with large language models
Open accessThe DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Abstract
Large language models (LLMs) have shown promise in medical question answering, with Med-PaLM being the first to exceed a 'passing' score in United States Medical Licensing Examination style questions. However, challenges remain in long-form medical question answering and handling real-world workflows. Here, we present Med-PaLM 2, which bridges these gaps with a combination of base LLM improvements, medical domain fine-tuning and new strategies for improving reasoning and grounding through ensemble refinement and chain of retrieval. Med-PaLM 2 scores up to 86.5% on the MedQA dataset, improving upon Med-PaLM by over 19%, and demonstrates dramatic performance increases across MedMCQA, PubMedQA and MMLU clinical topics datasets. Our detailed human evaluations framework shows that physicians prefer Med-PaLM 2 answers to those from other physicians on eight of nine clinical axes. Med-PaLM 2 also demonstrates significant improvements over its predecessor across all evaluation metrics, particularly on new adversarial datasets designed to probe LLM limitations (P < 0.001). In a pilot study using real-world medical questions, specialists preferred Med-PaLM 2 answers to generalist physician answers 65% of the time. While specialist answers were still preferred overall, both specialists and generalists rated Med-PaLM 2 to be as safe as physician answers, demonstrating its growing potential in real-world medical applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These models were chosen as they represent current leading LLMs3,36,40,41 as of February 2024.”
From An automated framework for assessing how well LLMs cite relevant medical references (2025) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“To account for the inherent stochasticity of LLMs, we report results from 10 independent runs per benchmark as mean accuracy with 95% confidence intervals (CIs), moving beyond the single-run point estimates common in prior works.2,3,4”
From Model confrontation and collaboration: A debate intelligence framework for enhancing medical reasoning in large language models (2026) · cites this paper as [2]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“medical domain-specific LLMs such as Medical Pathways Language Model (Med-PaLM) [2] and Biomedical Generative Pretrained Transformer (BioGPT) [3] have shown improved performance across various clinical natural language processing (NLP) tasks, including question answering, summarization, entity recognition, relation extraction, document classification, and literature mining.”
From Retrieval-Augmented Generation (RAG) in Healthcare: A Comprehensive Review (2025) · cites this paper as [2]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs have demonstrated strong performance across a wide range of biomedical knowledge and reasoning tasks, accelerating their adoption in health care workflows [1-6].”
From Retrieve-Then-Verify for Evaluating Evidence Support and Hallucination in Large Language Model–Generated Medical Information: Empirical Study (2026) · cites this paper as [4]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Large language models (LLMs) like OpenAI’s GPT-5 and knowledge-specific systems, such as OpenEvidence, have reported the ability to generate patient-specific, evidence-based recommendations.10”
From Large Language Model–Generated Electronic Consultations: A Blinded Comparative Study of OpenEvidence and Gastroenterologist Electronic Consultations (2026) · cites this paper as [10]
Experimental — accuracy not yet measured
9 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“The advent of transformers 4 and LLMs 5,6 has renewed interest in the possibilities of AI for medical question-answering tasks—a long-standing ‘grand challenge’ 7–9 .”
THE USE OF CONFIDENCE OR FIDUCIAL LIMITS ILLUSTRATED IN THE CASE OF THE BINOMIAL
CLOPPER, C. J., & PEARSON, E. S. (1934) · Biometrika · doi:10.1093/biomet/26.4.404 · Cited by 5,003
New creatinine- and cystatin C-based equations to estimate GFR without race
Inker, L. A., Eneanya, N. D., Coresh, J., Tighiouart, H., Wang, D., Sang, Y., Crews, D. C., Doria, A., Estrella, M. M., Froissart, M., et al. (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2102953 · Cited by 4,695
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · cited 1×
“This progress is exemplified by recent iterations within prominent LLM families like GPT (GPT-4, GPT-4o, GPT-4o1) 37 , Gemini (Gemini 1.0, Gemini 1.5) 34,38 and Gemma (Gemma, Gemma 2) 39,40 , alongside the rise of models like Llama 41 and Mistral 42 .”
InfluentialChain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263 · cited 1×
“55, involves augmenting each few-shot example in a prompt with a step-by-step explanation toward the final answer.”
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
Self-citationLarge language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083
Self-citation
Research works printed but not linked: 4
- 5. Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. Bert: pre-training of deep bidirectional transformers for language understanding. In Proc. NAACL-HLT Vol. 1 (eds Burstein, J. et al.) 4171-4186 (Association for Computational Linguistics, 2019).
- 10. Bobrow, D. G.) 167-180 (MIT Press, 1994).
- 38. Achiam, J. et al. GPT-4 technical report. Preprint at https://arxiv. org/abs/2303.08774 (2023).
- 63. Randolph, Justus J. 2005 "Free-Marginal Multirater Kappa (multirater K [free]): An Alternative to Fleiss' Fixed-Marginal Multirater Kappa." Presented at the Joensuu Learning and Instruction Symposium, vol. 2005 https://eric.ed.gov/?id=ED490661
Cited by
Items 701–800 of 1,031
Xin, W., Yi, H., Song, J., Tian, Z., Chen, H., & Tan, S. (2025) · Asian Journal of Psychiatry · doi:10.1016/j.ajp.2025.104707 · Cited by 4
Carrillo-Larco, R. M., Lovón-Melgarejo, J., Castillo‐Cara, M., & Bravo-Rocca, G. (2026) · Medical Science Educator · doi:10.1007/s40670-026-02692-w · Cited by 5
Generative AI for developing foundation models in radiology and imaging: engineering perspectives
Lee, J., Kyung, S., & Kim, N. (2025) · Biomedical Engineering Letters · doi:10.1007/s13534-025-00517-0 · Cited by 4
“For instance, models such as LLaVA-Med [59 ] and Med-PaLM [ 60 ] extended existing LLMs through domain-specific multimodal training, thereby improving performance in VQA and diagnostic assistance.”
Explainable Bilingual Medical-Question-Answering Model Using Ensemble Learning Technique
Sait, A. R. W., & Alkhurayyif, Y. (2025) · Electronics · doi:10.3390/electronics14204128 · Cited by 4
Instruction Tuning and CoT Prompting for Contextual Medical QA with LLMs
Le, C., Gong, Z., Wang, C., Ni, H., Li, P., & Chen, X. (2025) · International Conference on Artificial Intelligence, Human-Computer Interaction and Natural Language Processing (ICAHN) · doi:10.1109/icahn67688.2025.00016 · Cited by 14
“Prior work (e.g., Med-PaLM [13]) has shown the promise of such techniques in improving factuality and safety in medical QA, though primarily in closed or proprietary models.”
Efficient Dynamic Clustering-Based Document Compression for Retrieval-Augmented-Generation
Li, W., Zhang, X., Liu, K., Lei, X., Ma, W., & Liu, Y. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.522 · Cited by 8
Self-citationEvaluating the Performance of AI-Powered Browser Extensions in Online Pharmacy Exams
Ricco, M. H., Anderson, H. D., Linnebur, S. A., Saseen, J. J., & Trujillo, J. M. (2025) · American Journal of Pharmaceutical Education · doi:10.1016/j.ajpe.2025.101897 · Cited by 4
Zhu, S., Xie, Y., Tang, Y., Zhou, Y., Zhao, R., & Dong, X. (2025) · European Journal of Pediatrics · doi:10.1007/s00431-025-06602-x · Cited by 4
Artificial intelligence in healthcare research: Research ethics perspective
Hey, C. Y., & Kulkarni, S. (2025) · British Journal of Clinical Pharmacology · doi:10.1002/bcp.70395 · Cited by 3
Language proficiency assessment of autistic children using large language models
Qin, S., Liu, M., Wei, T., & Liu, Q. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129712 · Cited by 4
Zhou, Y., Wang, W., Wang, P., & Hu, K. (2026) · Frontiers in Medicine · doi:10.3389/fmed.2025.1709413 · Cited by 4
Valencia, J. M. C., Martínez-Rolán, X., & Seoane, J. P. (2026) · EDU REVIEW International Education and Learning Review / Revista Internacional de Educación y Aprendizaje · doi:10.62701/revedu.v13.5514 · Cited by 6
Zhang, Z., Shang, Y., Li, X., & Cheng, Z. (2026) · Energy storage materials · doi:10.1016/j.ensm.2026.104892 · Cited by 3
You, J., & Shin, H. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf230 · Cited by 5
Li, W., Zhang, Y., Wang, C. R., Li, Y., He, X., Wang, A. L., Xu, M., Zhang, F., Sun, H., Wang, K., et al. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.131476 · Cited by 4
Self-citationThe role of large language models in emergency care: a comprehensive benchmarking study
Naderi, B., Liu, L., Ghandehari, A., Khoshons, D., Taylor, R. A., Bhavsar, N., Balasubramanian, S., Tanouye, R., Creech, N., Davidson, C., et al. (2026) · npj Artificial Intelligence · doi:10.1038/s44387-026-00078-2 · Cited by 4
“Prior work has shown that domain-speci fi c fi ne-tuning on datasets such as PubMedQA or MedMCQA substantially enhances performance on knowledge-intensive tasks by optimizing attention weights for factual accuracy and structured reasoning sequences 13 .”
Zhou, C., Gong, Q., Luan, H., 詹文东, Zhu, J., & Zhang, Q. (2026) · Scientific Reports · doi:10.1038/s41598-026-38959-w · Cited by 5
“To clearly define the scope of our empirical study, we emphasize that all comparative experiments in this work are conducted against open-weight and locally deployable baselines (ChatGLM, BERT, CNN and their variants), rather than proprietary frontier medical LLMs such as GPT-4 or Med-PaLM.”
Cai, H., Wang, C., Zhang, Y., Ding, H., Hong, W., Zhao, Y., Cheng, S., & Wang, Y. (2026) · Diagnostics · doi:10.3390/diagnostics16040616 · Cited by 3
“For instance, Med-PaLM 2, an advanced LLM, has demonstrated diagnostic and treatment capabilities comparable to those of physicians, highlighting its significant potential in the medical field [11].”
Skjuve, M. B., Følstad, A., & Brandtzæg, P. B. (2026) · Digital Health · doi:10.1177/20552076261427447 · Cited by 5
“34,35,36,37 Ayers et al. 35 found that medical professionals preferred AI-generated medical advice over the advice of doctors, stating that the AI-generated output provided higher-quality advice and appeared more empathetic.”
InfluentialZhang, K., Feng, Y., & Zheng, J. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3ib59n936jrz · Cited by 10
Jacobs, C., Johnson, H., Brownlie, K., Joiner, R., & Thompson, T. (2026) · JMIR Formative Research · doi:10.2196/88905 · Cited by 4
“More sophisticated domain-specific models such as Med-PaLM2 have achieved expert-level performance on complex medical questions, indicating their potential for specialized health care applications [5,6].”
Systematic review of foundation models for structured electronic health records
Guo, L. L., Arciniegas, S. E., Yan, A. P., Fries, J., TOMLINSON, G. A., & Sung, L. (2026) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocag033 · Cited by 6
Ali, U., Shafi, I., Ahmad, J., Caceres, A. Z., Montero, T. C. C., Rehman, H. M. R. U., & Ashraf, I. (2026) · Computational and Structural Biotechnology Journal · doi:10.34133/csbj.0023 · Cited by 4
“Such multiscale visual embed-dings are used in frameworks such as PathAlign [ 57 ], which facilitates region-aware grounding, in which the generated clinical text is aligned with particular cellular morphologies.”
AI-assisted tumor board decision-making in pancreatic oncology
Mergen, M., Busch, F., Schwarberg, B., Koldeweihe, P., Jungwirth, D., Sydlik, J., Maurer, H. C., Makowski, M. R., Spitzl, D., & Gassert, F. T. (2026) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-026-03444-x · Cited by 3
“Studies utilizing models like GPT-4 and Med-PaLM have demonstrated promising performance in question answering and clinical reasoning tasks but often rely on synthetic or benchmark datasets [32, 33].”
Zehra, I., Ali, S., & Lee, S. W. (2026) · Journal for Healthcare Quality · doi:10.1097/jhq.0000000000000524 · Cited by 3
Soubh, N., Rasenack, E. C. L., Haarmann, H., Wiedmann, F., Zabel, M., Schmidt, C., Suliman, R., & Bergau, L. (2026) · Journal of Medical Internet Research · doi:10.2196/86692 · Cited by 4
“For example, Google’s Med-PaLM and Med-PaLM 2 achieved scores exceeding the passing threshold for the USMLE (United States Medical Licensing Examination) [1].”
A domain-adapted large language model to support clinicians in psychiatric clinical practice
Wang, R., Liu, S., Zhang, L., Zhu, X., Yang, Z., Wang, H., Huang, J., Wang, Y., Yang, X., Wu, F., et al. (2026) · Nature Machine Intelligence · doi:10.1038/s42256-026-01224-w · Cited by 9
Zuo, Y., Wan, Q., & Wang, S. (2026) · Journal of Medical Internet Research · doi:10.2196/85770 · Cited by 3
“Previous work has shown that lowering the decoding temperature in language models can improve accuracy by reducing output randomness [35,36].”
Li, J., Yuan, H., Luo, L., Lv, T., Jiang, Y., Wang, F., Zhang, P., Lv, H., Wang, J., Sun, Y., et al. (2026) · Communications in computer and information science · doi:10.1007/978-981-95-7299-1_29 · Cited by 5
“For example, Med-PaLM 2 [19] integrates large-scale, high-quality medical data with multi-stage instruction fine-tuning to enhance capabilities in medical question answering, clinical reasoning, medical knowledge retrieval, and professional text generation.”
RC-RAG: A Reliability-Centered Retrieval-Augmented LLM Framework for PHM Question Answering
Su, H., Wang, R., Ji, D.-Y., & Lee, J. (2026) · IEEE Transactions on Reliability · doi:10.1109/tr.2026.3702056 · Cited by 4
“Recently,advancesinLLM-basedquestionanswering(QA) systemsofferapromisingdirectionforaccessingknowledge throughnaturallanguageinteraction[13],[14],[15],[16].”
Ahmad, P. N., Anwar, M. S., Barahona, I., Rahman, A. U., Nisar, H., & Burhan, U. (2026) · Future Internet · doi:10.3390/fi18060324 · Cited by 3
Zhan, Z., Zhou, S., & Zhang, R. (2026) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocag042 · Cited by 4
“Many studies have demonstrated that CoT improves LLM accuracy on QA tasks [21, 13].”
A survey on medical multimodal retrieval-augmented generation (RAG)
Wang, R., Wang, J., Huang, Y., Guo, H., & Pang, J. (2026) · High-Confidence Computing · doi:10.1016/j.hcc.2026.100415 · Cited by 3
Zhu, T., Howson, J. M. M., & Nevado‐Holgado, A. J. (2026) · EBioMedicine · doi:10.1016/j.ebiom.2026.106343 · Cited by 4
“19,20 LLMs also demonstrate state-of-the-art performance in time series applications 21,22 through learnt token transitions and language-based embeddings.”
Yuan, T. (2026) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2026.106628 · Cited by 3
Noori, G. H., Bibi, S., & Lee, S. W. (2026) · INQUIRY The Journal of Health Care Organization Provision and Financing · doi:10.1177/00469580261478793 · Cited by 3
“15,16 Multimodal AI systems those that combine multiple types of data within a single analytical framework now integrate clinical language, genomic sequences, medical images, and structured electronic records.”
Are we ready for personalized CAR‐T therapy?
Strzelec, A., & Helbig, G. (2023) · European Journal Of Haematology · doi:10.1111/ejh.14039 · Cited by 3
Choaib, M., Garouani, M., Bouneffa, M., & Mohanna, Y. (2024) · International Conference on Control, Decision and Information Technologies (CoDIT) · doi:10.1109/codit62066.2024.10708357 · Cited by 3
Xinyu: An Efficient LLM-based System for Commentary Generation
Wu, Y., Tang, B., Xi, C., Yu, Y., Wang, P., Liu, Y., Kuang, K., Deng, H., Li, Z., Xiong, F., et al. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671537 · Cited by 6
Self-citationCan GPT Redefine Medical Understanding? Evaluating GPT on Biomedical Machine Reading Comprehension
Vatsal, S., & Singh, A. (2024) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2024.bionlp-1.20 · Cited by 7
Identifying Critical Tokens for Accurate Predictions in Transformer-Based Medical Imaging Models
Kang, S., Vankerschaver, J., & Özbulak, U. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73290-4_17 · Cited by 4
MedCare: Advancing Medical LLMs through Decoupling Clinical Alignment and Knowledge Aggregation
Liao, Y., Jiang, S., Chen, Z., Wang, Y., & Wang, Y. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.619 · Cited by 10
Khoylyan, A., Salvato, J. S., Vazquez, F., Girgis, M., Tang, A., & Chen, T. (2024) · North American Spine Society Journal (NASSJ) · doi:10.1016/j.xnsj.2024.100580 · Cited by 7
Large language model and its application in the questioning-answering system of minerals
Ji, X., Liu, C., Yang, M., He, M., Zhang, Z., Shan, Z. E. N. G., & Wang, Y. (2025) · Bulletin of Mineralogy Petrology and Geochemistry · doi:10.3724/j.issn.1007-2802.20240155 · Cited by 7
A LLMs-Assisted Framework for Parkinson's Disease Assessment Based on PPMI Dataset
Gao, Z., Ni, Q., Liu, W., & Zhang, L. (2024) · International Conference on Algorithms, Computing and Artificial Intelligence (ACAI) · doi:10.1109/acai63924.2024.10899636 · Cited by 6
Is multimodality part of the solution to prevent another AI winter?
Úbeda, P. L., Martín‐Noguerol, T., & Luna, A. (2025) · European Journal of Radiology Artificial Intelligence · doi:10.1016/j.ejrai.2025.100022 · Cited by 3
Sağlık Çalışanlarının Yapay Zekâ Uygulamalarına Yönelik Tutum Analizi
Şirin, H. B., & ALARÇİN, E. Y. (2025) · Anadolu Üniversitesi Sosyal Bilimler Dergisi · doi:10.18037/ausbd.1529145 · Cited by 3
Zedda, L., Loddo, A., & Ruberto, C. D. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131192 · Cited by 5
The virtual doctor prescribing the future: Diagnostics with interactive clinical decision support
Ruhland, J. B., Wichmann, J., Degtyar, D., Martin, R., Fehse, L., Klau, J. H., Papenbrock, T., Sowa, J., Canbay, A. E., Freisleben, B., et al. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110968 · Cited by 4
Zong, C. F., Wan, J., Tang, S., & Zhang, L. (2025) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2025.103246 · Cited by 6
Tinajero, C. A. C. (2025) · European Journal of Pediatric Surgery · doi:10.1055/a-2722-3871 · Cited by 6
Yang, S., Jing, M., Wang, S., Huang, Z., Wang, J., Kou, J., Shi, M., Xia, Z., Wei, Q., Xing, W., et al. (2025) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2025.103310 · Cited by 5
Zhang, J., Ubuzima, P., Huang, G., Lee, E.-H., Wang, Y., Xu, H., Xia, L., & Wu, T. (2025) · BMC Oral Health · doi:10.1186/s12903-025-07540-8 · Cited by 3
“Future research could explore targeted medical domain-specific fine-tuning and chain of retrieval approaches to further boost performance [33].”
Precision oncology: from large language models to multi-agent systems
Guo, X., Chen, J., Zhang, Y., Zhang, X., Lu, J., Qin, Q., Chen, J., Gao, X., & Wang, J. (2026) · Frontiers in Oncology · doi:10.3389/fonc.2026.1828507 · Cited by 5
“Despite the excellence of LLMs in speci fi c tasks such as medical examinations and documentation (27, 28), and their emerging clinical utility (29), they face limitations in real-world oncology scenarios characterized by high complexity, multiple stages, and reliance on external tools and…”
Self-citationGómez, F. J. D., & Galende, J. N. (2024) · Revista Tecnología Ciencia y Educación · doi:10.51302/tce.2024.19081 · Cited by 6
RAPID DEVELOPMENT OF CHATBOT FOR TOURISM PROMOTION IN LATGALE
Kodors, S., Kaņepe, G., Zeps, D., Zarembo, I., & Litavniece, L. (2024) · Environment Technology Resources Proceedings of the International Scientific and Practical Conference · doi:10.17770/etr2024vol2.8060 · Cited by 3
Kandiyana, A., Mouton, P. R., Hall, L., & Goldgof, D. B. (2024) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms61543.2024.00042 · Cited by 3
ModelLock: Locking Your Model With a Spell
Gao, Y., Sun, Y., Ma, X., Wu, Z., & Jiang, Y. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3685507 · Cited by 5
Knowledge Distillation in Mixture of Experts for Multi-Modal Medical LLMs
Nathani, M., Soni, R., & Mishra, R. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825733 · Cited by 4
Society for Scientific Data Processing mbH Gottingen (GWDG), G., & Kunkel, J. M. (2024) · Journal of Artificial Intelligence & Robotics · doi:10.52768/3067-7947/1017 · Cited by 7
Alsulami, M. M. (2026) · PeerJ Computer Science · doi:10.7717/peerj-cs.3446 · Cited by 4
Knowledge Graphs and Large Language Models
Dvoichenkov, D. D. (2023) · Control Systems and Computers · doi:10.15407/csc.2023.03.054 · Cited by 3
ChatGPT: An Artificial Intelligence-Based Approach to Enhance Medical Applications
Chaddad, A., He, C., & Jiang, Y. (2023) · IEEE International Conference on Bioinformatics and Bioengineering (BIBE) · doi:10.1109/bibe60311.2023.00078 · Cited by 3
Comparison of Large Language Models’ Performance on Neurosurgical Board Examination QuestionsPreprint
Andrade, N. S., & Donty, S. V. (2025) · medRxiv · doi:10.1101/2025.02.20.25322623 · Cited by 3
Evaluating Large Reasoning Model Performance on Complex Medical Scenarios In The MMLU-Pro BenchmarkPreprint
Hoyt, R. E., Knight, D. R. T., Haider, M., & Bajwa, M. (2025) · medRxiv · doi:10.1101/2025.04.07.25325385 · Cited by 4
“[37] We wrote our prompt concisely so that DeepSeek could only respond to the listed answers.”
RiskAgent: Autonomous Medical AI Copilot for Generalist Risk PredictionPreprint
Liu, F., Wu, J., Zhou, H., Gu, X., Molaei, S., Thakur, A., Clifton, L., Wu, H., & Clifton, D. A. (2025) · medRxiv · doi:10.1101/2025.04.03.25323489 · Cited by 5
Thomas, J., Mudgal, A., Liu, W., Tahiraj, N., Mohammed, Z., & Diddi, D. (2025) · medRxiv · doi:10.1101/2025.07.01.25330679 · Cited by 5
“Google's Med-PaLM and Med-PaLM 2 demonstrated that LLMs could achieve physician-level performance on medical question-answering tasks [8].”
Ahn, S. (2025) · Annals of Laboratory Medicine · doi:10.3343/alm.2025.0422 · Cited by 4
“Med-PaLM 2 is a combination of an improved base model and medical fine-tuning, enabling it to achieve 86.5% accuracy and surpass physician performance [14].”
InfluentialDevelopment of a RAG-based Expert LLM for Clinical Support in Radiation OncologyPreprint
Liu, T., Wang, X., Inkman, M. J., Hong, J. C., Waters, M. R., & Zhang, J. (2025) · medRxiv · doi:10.1101/2025.09.16.25335813 · Cited by 7
Neuro-symbolic LLM Integration in Clinical Medicine: A Systematic Review and TaxonomyPreprint
Gorenshtein, A., Omar, M., Barash, Y., Nadkarni, G. N., & Klang, E. (2026) · Research Square · doi:10.21203/rs.3.rs-8902022/v1 · Cited by 4
“LLM advances 1 – 4 accelerate efforts to deploy such systems in clinical workflows.”
Shao, Y., Tang, X., Sohn, J., Chen, J., Liao, Y., Zhang, J., Xiang, J., Wu, F., Zhao, Y., Wu, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.07459 · Cited by 3
Self-citationZhu, Y., Zhang, J., Sun, Y., Wen, J., Liu, Z., Liu, X., Xu, S., Wu, N., Zhang, Y., Zhou, G., et al. (2026) · DIGITAL HEALTH · doi:10.1177/20552076261428234 · Cited by 3
Mobility functional status ascertainment in electronic health records using large language models
Liu, X., Garg, M., Jia, H., Sauver, J. S., Pagali, S. R., & Sohn, S. (2026) · Scientific Reports · doi:10.1038/s41598-026-37025-9 · Cited by 2
“Recent advances in LLM architecture and training have demonstrated remarkable performance on a wide range of NLP tasks, including information extraction, question answering, and text summarization, overcoming many of the limitations of prior NLP approaches 23–25 .”
Huang, H., DeLucia, A., Tiyyala, V. M., & Dredze, M. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.693 · Cited by 2
Yamagishi, Y., Nakamura, Y., Kikuchi, T., Sonoda, Y., Hirakawa, H., Kano, S., Nakamura, S., Hanaoka, S., Yoshikawa, T., & Abe, O. (2025) · JMIR Medical Informatics · doi:10.2196/71137 · Cited by 4
Ke, Y., Lin, H., Ruan, Y., Tang, J., & Li, L. (2025) · Mathematics · doi:10.3390/math13091538 · Cited by 8
Full-Parameter Fine-Tuning Method of LLMs for Sports Injury Prevention and Treatment
Zhu, X., Gao, Z., & Wang, X. A. (2025) · International Journal of Mobile Computing and Multimedia Communications · doi:10.4018/ijmcmc.376486 · Cited by 5
Chain-of-Thought for Cancer Pathology Interpretation and Summarization
Mali, A., Kohut, A., & Templeton, J. M. (2025) · IEEE International Conference on Smart Computing · doi:10.1109/smartcomp65954.2025.00081 · Cited by 2
SPSleepNet: enhancing EEG-based sleep staging for OSA patients via sleep position integration
Zhang, Y., Qiu, X., Ma, G., Yang, L., Jing, T., & Huang, J. (2025) · Complex & Intelligent Systems · doi:10.1007/s40747-025-02030-4 · Cited by 2
Frydman-Gani, C., Arias, A., Vallejo, M. P., Martínez, J. D. L., Valencia‐Echeverry, J., Castano, M., Bui, A. A. T., Freimer, N. B., López‐Jaramillo, C., & Loohuis, L. M. O. (2026) · Journal of Medical Internet Research · doi:10.2196/93328 · Cited by 3
Basu, A., Gupta, A., Das, S., & Shamsolmoali, P. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.07.130 · Cited by 2
A Review of Critical Deep Learning-Based Image Guidance Technologies for Surgical Robots
Cao, Y., Zhang, J., Li, H., Ren, B., Zhao, Y. H., Gao, C., & Zhou, Y. (2025) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2025.3584339 · Cited by 5
“Recently, Google released Med-PaLM2, which enhances the per-formanceofclinicalQ&AthroughLLMfine-tuningandprompt-ingstrategies[231].”
Evaluating Psychological Competency via Chinese Q&A in Large Language Models
Gao, F., He, Y., Chen, Q., & Liu, F. (2025) · Applied Sciences · doi:10.3390/app15169089 · Cited by 3
Big Loop and Atomization: A Holistic Review on the Expansion Capabilities of Large Language Models
Hu, Z., Huang, Y., Feng, J., & Deng, C. (2025) · Applied Sciences · doi:10.3390/app15179466 · Cited by 4
Evaluating Expertise of Concrete Pavement Construction in Modern Large Language Models
Smetana, M., Sukharev, I., & Khazanovich, L. (2025) · Transportation Research Record Journal of the Transportation Research Board · doi:10.1177/03611981251350643 · Cited by 3
Large language models driven reliable clinical decision-making: Framework and application
Du, J., Li, X., Liu, Y., Lv, T., Liu, H., & Yin, H. (2025) · Informatics and Health · doi:10.1016/j.infoh.2025.08.002 · Cited by 3
Self-citationGanti, B., Balasubramanian, K., & Pati, S. B. B. (2025) · IEEE Access · doi:10.1109/access.2025.3614148 · Cited by 3
“LLM-integrated EEG assistants have been proposed for real-time triage, differential diagnosis, and seizure report generation [15], [16], [140].”
Clinical knowledge enhanced medical image classification
Xu, Z., Liang, J., Wei, Z., Yue, X., & Li, D. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.112414 · Cited by 4
Bielick, C. G., Awwad, A., Ellen, J., Jalilian, L., McCoy, L. G., Mishra, V., Osmanlliu, E., Pfohl, S. R., & Celi, L. A. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001115 · Cited by 8
“GPT-4 demonstrated 80–90% accuracy across all USMLE steps without specialized training [1], while Google’s Med-PaLM 2 reached 86.5% accuracy on similar questions [2].”
InfluentialSelf-citationQuality assessment of patient-facing urologic telesurgery content using validated tools
Davuluri, T. S., Gabriel, P., Wainstein, M. D., & Ekwenna, O. (2025) · Journal of Robotic Surgery · doi:10.1007/s11701-025-02871-8 · Cited by 2
Foundation Model for Medical Imaging: A Comprehensive Review
Jiao, L., Yang, J., Li, R., Liu, F., Liu, X., Chen, P., Guo, Y., Li, L., Shang, R., & Ma, W. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3618796 · Cited by 11
ChatGPT and other large language models for childhood asthma
Drummond, D., Girault, A., & Gonsard, A. (2025) · Paediatric Respiratory Reviews · doi:10.1016/j.prrv.2025.09.002 · Cited by 5
Hu, L., Xu, X., Zhuang, Y., Lin, Y., Xu, M., Wu, X., & Li, B. (2025) · Scientific Reports · doi:10.1038/s41598-025-22315-5 · Cited by 2
Souza, G. D. P., Melo, G., & Schneider, D. (2025) · Informatics · doi:10.3390/informatics12040113 · Cited by 3
“Similarly, the development of domain-specific medical LLMs, such as Med-PaLM2 [33], exemplifies this approach, achieving state-of-the-art results on benchmarks like MedQA through extensive fine-tuning, as highlighted by Kumar et al. and Fang et al. [9,21].”
Mwaniki, P., Musau, W., Isaaka, L., Wanyama, C. W., Menon, V., Denniston, A. K., Liu, X., Emmanuel-Fabula, M., Williams, G., Mateen, B. A., et al. (2026) · BMJ Digital Health & AI · doi:10.1136/bmjdh-2026-000081 · Cited by 3
Boczkowski, D., Dolata, T., Radej, D., Sawina, P., Suleiman, R., Latkowska, A., Kowalczyk, A., Loson-Kawalec, M., Jaworski, W., Wielochowska, A., et al. (2025) · Cureus · doi:10.7759/cureus.95581 · Cited by 2
Physician Perspectives on Large Language Models in Health Care: A Cross-Sectional Survey Study
Hong, H. J., Shah, N. H., Pfeffer, M. A., & Lehmann, L. S. (2025) · Applied Clinical Informatics · doi:10.1055/a-2735-0527 · Cited by 5
Self-citationAI-Powered Automation in Healthcare: A Multi-agent Approach with LLM
de Almeida, M. P., Machado, J. M., Sousa, R., & Peixoto, H. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-05925-3_11 · Cited by 2
Pupo, B. D., Costa, D. G., Immich, R., von Wangenheim, A., Pinto, A. S. R., & de Macedo, D. D. J. (2025) · Applied Sciences · doi:10.3390/app152111732 · Cited by 2
Kumari, M., Chauhan, R., Jain, R., & Garg, P. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103902 · Cited by 6
Metadata
- Authors
- Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Amin, M., Hou, L., Clark, K., Pfohl, S. R., Cole-Lewis, H., Neal, D., Rashid, Q. M., Schaekermann, M., Wang, A., Dash, D., Chen, J. H., Shah, N. H., Lachgar, S., Mansfield, P. A., Prakash, S., Green, B., Dominowska, E., Agüera y Arcas, B., Tomašev, N., Liu, Y., Wong, R., Semturs, C., Mahdavi, S. S., Barral, J. K., Webster, D. R., Corrado, G. S., Matias, Y., Azizi, S., Karthikesalingam, A., & Natarajan, V.
- Affiliations
- Google (United States) (US)
- Stanford Medicine (US)
- Stanford Health Care (US)
- Corresponding author
- Azizi, Shekoofeh
- Karthikesalingam, Alan
- Natarajan, Vivek
- ORCID
- Singhal, Karan
- Tu, Tao
- Gottweis, Juraj
- Sayres, Rory
- Amin, Mohamed
- Clark, Kevin
- Pfohl, Stephen R.
- Cole-Lewis, Heather
- Rashid, Qazi Mamunur
- Schaekermann, Mike
- Wang, Amy
- Dash, Dev
- Chen, Jonathan H.
- Shah, Nigam H.
- Lachgar, Sami
- Mansfield, Philip Andrew
- Prakash, Sushant
- Green, Bradley
- Dominowska, Ewa
- Tomašev, Nenad
- Liu, Yun
- Wong, Renee
- Semturs, Christopher
- Mahdavi, S. Sara
- Barral, Joelle K.
- Webster, Dale R.
- Corrado, Greg S.
- Matias, Yossi
- Azizi, Shekoofeh
- Karthikesalingam, Alan
- Natarajan, Vivek
- Venue
- Nature Medicine
- Published
- 2025-01-08
- Volume
- 31
- Issue
- 3
- Pages
- 943-950
- ISSN
- 1078-8956 (ISSN-L)
- 1546-170X
- PMC
- PMC11922739
- OpenAlex
- W4406152279
- Semantic Scholar
- b34c0b0169925a6c7f14e3de9764ed9505f30de3
- Cited by
- 1,734
- References
- 57
- Citations by year
- 2026: 771 · 2025: 591
- FWCI
- 277.36
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-nd
- creativecommons.org/licenses/by-nc-nd/4.0 (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0 (vor)
- Locations
- Nature Medicine (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Gordon and Betty Moore Foundation (#12409)
- American Heart Association
- National Institutes of Health (UM1TR004921)
- Stanford Bio-X
- National Institute on Drug Abuse (UG1DA015815)
- National Institute of Allergy and Infectious Diseases (R01AI178121)
- National Center for Advancing Translational Sciences
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- medical question answering, large language models, ensemble refinement, clinical reasoning, grounding, USMLE, MedQA, physician preferences
- MeSH terms
- Large Language Models, Humans, Physicians, Pilot Projects, United States
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.