Toward expert-level medical question answering with large language models
Open accessThe DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Abstract
Large language models (LLMs) have shown promise in medical question answering, with Med-PaLM being the first to exceed a 'passing' score in United States Medical Licensing Examination style questions. However, challenges remain in long-form medical question answering and handling real-world workflows. Here, we present Med-PaLM 2, which bridges these gaps with a combination of base LLM improvements, medical domain fine-tuning and new strategies for improving reasoning and grounding through ensemble refinement and chain of retrieval. Med-PaLM 2 scores up to 86.5% on the MedQA dataset, improving upon Med-PaLM by over 19%, and demonstrates dramatic performance increases across MedMCQA, PubMedQA and MMLU clinical topics datasets. Our detailed human evaluations framework shows that physicians prefer Med-PaLM 2 answers to those from other physicians on eight of nine clinical axes. Med-PaLM 2 also demonstrates significant improvements over its predecessor across all evaluation metrics, particularly on new adversarial datasets designed to probe LLM limitations (P < 0.001). In a pilot study using real-world medical questions, specialists preferred Med-PaLM 2 answers to generalist physician answers 65% of the time. While specialist answers were still preferred overall, both specialists and generalists rated Med-PaLM 2 to be as safe as physician answers, demonstrating its growing potential in real-world medical applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These models were chosen as they represent current leading LLMs3,36,40,41 as of February 2024.”
From An automated framework for assessing how well LLMs cite relevant medical references (2025) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“To account for the inherent stochasticity of LLMs, we report results from 10 independent runs per benchmark as mean accuracy with 95% confidence intervals (CIs), moving beyond the single-run point estimates common in prior works.2,3,4”
From Model confrontation and collaboration: A debate intelligence framework for enhancing medical reasoning in large language models (2026) · cites this paper as [2]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“medical domain-specific LLMs such as Medical Pathways Language Model (Med-PaLM) [2] and Biomedical Generative Pretrained Transformer (BioGPT) [3] have shown improved performance across various clinical natural language processing (NLP) tasks, including question answering, summarization, entity recognition, relation extraction, document classification, and literature mining.”
From Retrieval-Augmented Generation (RAG) in Healthcare: A Comprehensive Review (2025) · cites this paper as [2]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs have demonstrated strong performance across a wide range of biomedical knowledge and reasoning tasks, accelerating their adoption in health care workflows [1-6].”
From Retrieve-Then-Verify for Evaluating Evidence Support and Hallucination in Large Language Model–Generated Medical Information: Empirical Study (2026) · cites this paper as [4]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Large language models (LLMs) like OpenAI’s GPT-5 and knowledge-specific systems, such as OpenEvidence, have reported the ability to generate patient-specific, evidence-based recommendations.10”
From Large Language Model–Generated Electronic Consultations: A Blinded Comparative Study of OpenEvidence and Gastroenterologist Electronic Consultations (2026) · cites this paper as [10]
Experimental — accuracy not yet measured
9 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“The advent of transformers 4 and LLMs 5,6 has renewed interest in the possibilities of AI for medical question-answering tasks—a long-standing ‘grand challenge’ 7–9 .”
THE USE OF CONFIDENCE OR FIDUCIAL LIMITS ILLUSTRATED IN THE CASE OF THE BINOMIAL
CLOPPER, C. J., & PEARSON, E. S. (1934) · Biometrika · doi:10.1093/biomet/26.4.404 · Cited by 5,003
New creatinine- and cystatin C-based equations to estimate GFR without race
Inker, L. A., Eneanya, N. D., Coresh, J., Tighiouart, H., Wang, D., Sang, Y., Crews, D. C., Doria, A., Estrella, M. M., Froissart, M., et al. (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2102953 · Cited by 4,695
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · cited 1×
“This progress is exemplified by recent iterations within prominent LLM families like GPT (GPT-4, GPT-4o, GPT-4o1) 37 , Gemini (Gemini 1.0, Gemini 1.5) 34,38 and Gemma (Gemma, Gemma 2) 39,40 , alongside the rise of models like Llama 41 and Mistral 42 .”
InfluentialChain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263 · cited 1×
“55, involves augmenting each few-shot example in a prompt with a step-by-step explanation toward the final answer.”
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
Self-citationLarge language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083
Self-citation
Research works printed but not linked: 4
- 5. Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. Bert: pre-training of deep bidirectional transformers for language understanding. In Proc. NAACL-HLT Vol. 1 (eds Burstein, J. et al.) 4171-4186 (Association for Computational Linguistics, 2019).
- 10. Bobrow, D. G.) 167-180 (MIT Press, 1994).
- 38. Achiam, J. et al. GPT-4 technical report. Preprint at https://arxiv. org/abs/2303.08774 (2023).
- 63. Randolph, Justus J. 2005 "Free-Marginal Multirater Kappa (multirater K [free]): An Alternative to Fleiss' Fixed-Marginal Multirater Kappa." Presented at the Joensuu Learning and Instruction Symposium, vol. 2005 https://eric.ed.gov/?id=ED490661
Cited by
Items 901–1,000 of 1,031
MedSentry: Understanding and Mitigating Safety Risks in Medical LLM Multi-Agent Systems
Chen, K., Zhen, T., Wang, H., Liu, K., Li, X., Huo, J., Yang, T., Xu, J., Dong, W., & Gao, Y. (2025) · arXiv · doi:10.48550/arxiv.2505.20824 · Cited by 32
“The evaluation in medical scenarios has primarily focused on general medical problems such as MedQA [30], PubMedQA [31], and MultiMedQA [32].”
MARCH: Multi-Agent Radiology Clinical Hierarchy for CT Report Generation
Lin, Y., Ding, Y., Wu, Y., & Peng, Y. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2026.acl-short.24 · Cited by 5
A Scoping Review of LLM-as-a-Judge in Healthcare and the MedJUDGE Framework
Li, C., Akhtar, Z., Kwak, M., Ji, Y., Zhang, H., Obi, T., Ren, Y., Wu, X., Sivarajkumar, S., Lehmann, H. P., et al. (2026) · arXiv · doi:10.48550/arxiv.2604.25933 · Cited by 1
Performance of Large Language Models on the Radiation and Cancer Biology Practice Exam
Bertschmann, J., Xu, Y., Bayley, C., Abdellatif, A., & Lee, S. L. (2024) · Frontiers in Oncology · doi:10.1016/j.ijrobp.2024.07.1347 · Cited by 1
Revolutionizing Medical Inquiry Responses Harnessing Advanced Language Models for Enhanced Precision
Keerthana, S. M., Kirthidevi, S., & Nandhana, V. (2024) · International Conference on Emerging Research in Computational Science (ICERCS) · doi:10.1109/icercs63125.2024.10895183 · Cited by 1
Friedman, M., Mor, L., Segman, I., Mizrachi, Y., Shoshan, N. B., Eisenberg, H., Shieldkrot, T., Weiner, E., & Barda, G. (2025) · American Journal of Obstetrics & Gynecology MFM · doi:10.1016/j.ajogmf.2025.101667 · Cited by 1
Enhancing Patient Intake Process in Mental Health Consultations Using RAG-Driven Chatbot
Shayaninasab, M., Zahoor, M., & Yalçın, Ö. N. (2024) · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) · doi:10.1109/aciiw63320.2024.00053 · Cited by 5
A Retrieval-Augmented Dialogue Framework for Multimodal Medical Consultation
Zhang, X., & Zhang, Y. (2024) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat62293.2024.00099 · Cited by 1
Fine-Tuning LLMs for Anesthesiology via Compositional Data Generation
Li, Y., Zhan, Y., Yu, B., Wang, Z., Zhang, C., Zhu, D., Zhou, C., Zhou, B., Ding, L., Liu, W., et al. (2025) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2025.3567602 · Cited by 1
“Google’s MedPaLM [49] and MedPalM-2 [50] have ex-celled in medical Q&A, image interpretation, and genomics.”
Conversational AI in Medicine: Redefining Diagnostics with AMIE
Lin, Z., Du, Y., Yang, W., Qian, Y., & Li, J. J. (2025) · BIO Integration · doi:10.15212/bioi-2025-0085 · Cited by 6
A Large Language Model Based on the Retrieval-Augmented Generation and Prompt-Tuning Framework
Wang, S., Wang, K., & Zhan, Y. (2025) · International Symposium on Big Data and Applied Statistics (ISBDAS) · doi:10.1109/isbdas64762.2025.11117135 · Cited by 1
Lilamand, M., Decaix, T., Gourraud, P., Dufour, R., Gavazzi, G., Boaddaert, J., Baudron, C. R., Vallet, H., Annweiler, C., & Chapelet, G. (2025) · European Geriatric Medicine · doi:10.1007/s41999-025-01334-5 · Cited by 1
Momin, E. A., & Mansoor, H. (2025) · Cureus · doi:10.7759/cureus.95083 · Cited by 1
Generative AI: A Disruptor to Health Professions Learner Assessment
Morjaria, L., Gandhi, B., Burns, L., & Sibbald, M. (2025) · Cureus · doi:10.7759/cureus.96034 · Cited by 2
“In fact, answers to consumer medical questions generated by a GenAI model, Med-PaLM 2, were preferred over physician answers by a panel of physicians [29].”
CURE: Confidence-Driven Unified Reasoning Ensemble Framework for Medical Question Answering
Elshaer, Z., & Rashed, E. A. (2025) · Big Data and Cognitive Computing · doi:10.3390/bdcc9120299 · Cited by 2
MedStructGen: A Two-Stage Method for Medical Record Generation Using Large Language Models
Ma, Z., Liu, R., & Wang, Y. (2025) · IEEE International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/icpads67057.2025.11323208 · Cited by 1
“In the medical field, for example, Google’s Med-PaLM [11] has achieved near-professional doctor-level performance in medical question answering via domain-specific fine-tuning.”
Yang, N., Li, K., Liu, B., Chen, X., Jiang, A., Chang, Q., Gan, W., Zhu, L., Mou, W., Zeng, D., et al. (2026) · Intelligent Oncology · doi:10.1016/j.intonc.2026.100039 · Cited by 1
AASQ-GPT: Accurate Activation Static Quantization Towards Generative Pre-Trained Transformer
Wang, Z. (2025) · IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC) · doi:10.1109/imcec66174.2025.11331709 · Cited by 3
“State-of-the-art open-source models ( e.g. LLaMA[2], Deepseek[3], etc.) have been applied in numerous fields such as general-purpose chat assistants, medical question answering[4], and finance[5].”
LLM-Assisted Keyword Curation for Early Disease Outbreak Detection Using Search Trends
Jerik Brahmos, A., & Princess, P. J. B. (2025) · International Conference on Applied Artificial Intelligence and Computing (ICAAIC) · doi:10.1109/icaaic64647.2025.11330303 · Cited by 1
“In their investigation of LLM performance in answering medical questions, Singhal et al. [17] showed that they were highly capable of comprehending medical context and terminology, achieving expert-level performance on medical licensing exams.”
A Survey on AI-Driven Techniques for Automated Medical Diagnosis and Consultation System
Kumar, A. D., Gupta, S., Birwatkar, A., Mokal, S., & Patil, B. (2025) · International Conference on Future Technologies (ICFT) · doi:10.1109/icft66708.2025.11336729 · Cited by 1
Mahadik, R., Mandal, A. K., Tripathi, S., & Jaiswal, S. (2025) · International Conference on Responsible, Generative and Explainable AI (ResGenXAI) · doi:10.1109/resgenxai64788.2025.11344029 · Cited by 2
Large Language Models Triage of Retina Patient Emergency Telephone Calls: A Pilot Study
Chahal, R., Beca, F. A., Chau, V. Q., Kiryakoza, L. C., Abousy, M., Fan, K. C., Wykoff, C. C., & Al-khersan, H. (2026) · Journal of VitreoRetinal Diseases · doi:10.1177/24741264251414097 · Cited by 2
Large Language Models in the Assessment and Care of Internet Gaming Disorder
Kranas, A., & Verykios, V. S. (2026) · AI in Medicine · doi:10.3390/aimed1010006 · Cited by 1
“Domain-adapted medical LLMs achieve state-of-the-art accuracy on MedQA (e.g., Med-PaLM 2) and, in human evaluations, physicians often prefer its answers on multiple clinical axes, while specialists still outperform the model—highlighting promise and limits for clinical use [35].”
EagleCare: Augmenting ICU Monitoring with LLMs and Knowledge Graph Reasoning
Guo, J., Gao, Y., Liu, Z., Rao, Y., Hu, J., & Wang, H. (2025) · IEEE International Conference on Knowledge Graph (ICKG) · doi:10.1109/ickg66886.2025.00024 · Cited by 2
“The adaptation of LLMs to medical domains has largely relied on task-specific fine-tuning, as seen in models like Med-PaLM [8], Med-PaLM2 [14], and GPT-4-Med [15], which leverage extensive medical corpora for clinical report generation and decision support.”
Srivastava, A. K., Levin, J. M., Derrico, A., & Dames, P. (2026) · IEEE/SICE International Symposium on System Integration (SII) · doi:10.1109/sii64115.2026.11404461 · Cited by 1
Transforming Medical Machine Translation with Next-Generation Large Language Models
Suram, P., & Patro, P. (2025) · OITS International Conference on Information Technology (OCIT) · doi:10.1109/ocit66168.2025.11400016 · Cited by 1
“Medical Machine Translation (MMT) signifies a particular application of natural language processing attentive on translating patient records, k-sheets, medical documents, clinical notes, and healthcare materials diagonally languages though continuing language consistency also medical accuracy [25].”
Clinical usability of generative artificial intelligence for MR safety advice
Rose, H., Thorpe, J. C., Panek, R., Goncalves, E., & Morgan, P. S. (2026) · European Journal of Radiology Artificial Intelligence · doi:10.1016/j.ejrai.2026.100083 · Cited by 1
Advancing the science of qualitative patient preference assessment using large language models
Grover, T., Krebs, E., Weymann, D., Ehman, M., & Regier, D. A. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001263 · Cited by 3
“While both scientific outcome prediction and qualitative analysis are forward-looking contexts where the future outcome is initially unknown, scientific outcome prediction always requires clear, objective criteria for determining the correctness of a generated prediction [17,18,54].”
Dominić, I., S, M. K. H., Nandakumar, R. G., & Regunathan, R. (2025) · International Conference on Signal Processing, Computation, Electronics, Power and Telecommunication (IConSCEPT) · doi:10.1109/iconscept66142.2025.11436852 · Cited by 1
“Overlapping speech, medical terminology, and hallucinations were problems for early ASR and LLM systems [1]–[3].”
Singh, A. P., Sharma, S. P., & Sharma, V. K. (2025) · International Conference on Data, Energy and Communication Networks (DECoN) · doi:10.1109/decon67170.2025.11447870 · Cited by 1
Fung, A. L., Abdalla, M., Cheng, Y. B., Elsayed, M., Chojecki, D., Mitchell, J. R., & Ye, C. (2026) · Frontiers in Health Services · doi:10.3389/frhs.2026.1797583 · Cited by 1
“Med-PaLM 2 achieved 86.5% accuracy on MedQA (US Medical Licensing Exam-style questions), with physicians preferring its answers to those from other physicians on 8 of 9 clinical axes (28).”
InfluentialGuo, Y., Meng, X., Yu, E., Zhang, W., Yang, Y., Ma, H., Shao, C., Wang, W., Wang, R., Wang, H., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02644-z · Cited by 3
He, N., Yang, L., Hu, X., & He, Y. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1785443 · Cited by 1
“While prior medical QA studies have largely emphasized answer correctness and reasoning quality [3, 4], our work introduces a broader evaluation lens by incorporating deployment-oriented metrics such as P95 latency and first-attempt connection reliability.”
InfluentialLiu, Z., Ding, Y., Chen, J., Yan, Z., Cheng, X., Zhou, W., Fu, P., & Wang, Z. (2026) · JMIR Medical Informatics · doi:10.2196/97520 · Cited by 1
Self-citationKasa, K., Yoshimaru, D., Ohta, H., Ohki, T., & Okano, H. J. (2026) · General Thoracic and Cardiovascular Surgery · doi:10.1007/s11748-026-02304-9 · Cited by 1
“Domain-adapted systems such as Med-PaLM 2 have demonstrated physician-level accuracy in medical question answering while maintaining comparable safety standards [2].”
Screening Routine Clinical Notes for Epilepsy Surgery Candidates Using Large Language Models
Fennig, U., Amir, N., Schiller, M., Loebenstein, R., Nissan, J., Boxer, M., Gofrit, S. G., Goshen, G., Beniczky, S., & Maggio, N. (2026) · Annals of Clinical and Translational Neurology · doi:10.1002/acn3.70427 · Cited by 1
“…advances in LLM development have also led to the emergence of domain-specific models trained or fine-tuned for medical reasoning and diagnosis, such as Med-PaLM 2 and MedFound, which demonstrate strong performance on medical question-answering benchmarks and clinical diagnostic tasks [12, 22].”
Artificial Intelligence Across the Radiology Workflow: A Nine-Stage Narrative Review
Rai, M. C. E., Far, A. B., Darweesh, M., Dhou, S., Aburaed, N., El, S., Elkhazendar, M., & Ellahham, S. (2026) · Diagnostics · doi:10.3390/diagnostics16101485 · Cited by 1
“Ensuring explainability and traceability of model outputs remains an active area of investigation in clinical contexts [181].”
Batra, P., Puttaswamy, E., Hurkadle, J., Shastri, D., Sharma, V., Raza, M., & Jakhal, M. (2026) · European Journal of Orthodontics · doi:10.1093/ejo/cjag024 · Cited by 1
A Comparative Evaluation of Large Language Models on Pediatric Board-Style Examinations
Schönberg, N., Deschler, D. J. P., Hauer, J., & Zeumer, M. (2026) · Hospital Pediatrics · doi:10.1542/hpeds.2025-009153 · Cited by 1
Luo, J., Wang, Y.-L., Zhao, M.-J., Yin, J.-L., Ding, D.-F., & Wu, X.-B. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1811701 · Cited by 1
Léon, M., Feng, R., Quiroz-Flores, M., Pelletier, G. J., Bethencourt, D. M., Shibata, M., He, H., & Ruaengsri, C. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1769467 · Cited by 2
Sun, X., & Wang, T. (2026) · Advanced Science · doi:10.1002/advs.75819 · Cited by 2
“Building upon these, the recent developments of LLMs, such as BioGPT [ 143 ] and Med-PaLM [ 144 ], offer great potential for the data-mining paradigm.”
Behl, A., & Sharma, K. K. (2026) · Probiotics and Antimicrobial Proteins · doi:10.1007/s12602-026-11077-9 · Cited by 3
Karataş, S. Ş., & Öner, S. F. (2026) · Health Policy and Technology · doi:10.1016/j.hlpt.2026.101281 · Cited by 1
Zhan, X., Yu, W., Cai, J., & Chen, J. (2026) · PLoS ONE · doi:10.1371/journal.pone.0353059 · Cited by 1
“These achievements have ignited intense scholarly discourse regarding the potential for artificial intelligence to transform medical education paradigms and professional credentialing processes [7].”
Self-citationZhang, W., Xu, J., Dong, T., Hao, X., Yang, Q., Zhang, J., & Han, Y. (2026) · Exploratory Research in Clinical and Social Pharmacy · doi:10.1016/j.rcsop.2026.100821 · Cited by 2
“25 Notably, a recent international study also showed that LLMs performed well, even comparable to general practitioners, in assisting antibiotic decision-making in primary care settings regarding diagnostic accuracy and the judgment of whether to use antibiotics.”
Razzaq, T., Taj, M., & Iqbal, A. (2026) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2026.105075 · Cited by 2
Language-Guided Segmentation of Medical Images: A Review of Foundation Models
Qamar, S. (2026) · Bioengineering · doi:10.3390/bioengineering13070803 · Cited by 3
“Med-Gemini reached expert-level performance on several clinical benchmarks [44,56,57], clinician-VLM collaboration improved radiology report generation [58], and CT-CLIP brought contrastive pretraining to volumetric chest CT paired with structured reports [59].”
Wojtas, J., Pak, G., & Górski, W. J. (2026) · Cureus · doi:10.7759/cureus.112868 · Cited by 1
Large language models for interpretation of health checkup results
You, J., & Shin, H. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02467-y · Cited by 1
Palermi, S., Saglietto, A., D’Ascenzo, F., Ferrari, G. M. D., Bruno, F., Tse, G., Kizilkilic, S. E., Tjong, F. V. Y., Ruipérez-Campillo, S., Keser, N., et al. (2026) · Progress in Cardiovascular Diseases · doi:10.1016/j.pcad.2026.07.005 · Cited by 2
Oberoi, K., Hassan, S., Woon, D. T. S., & Sethi, K. (2026) · Société Internationale d’Urologie Journal · doi:10.3390/siuj7040047 · Cited by 1
“Specialist preference for AI chatbot responses over physician answers has similarly been documented in oncological contexts [5].”
CGX: OCR-enhanced knowledge graph retrieval for explainable heart failure analysis
Nguyen, D. T., Le, A. N., Trinh, B. T., Vu, D. B., & Nguyen, H. N. (2026) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2026.105089 · Cited by 2
Enhancing Power Grid Model Generation With Large Language Models via Progressive Fine-Tuning
Deng, K., Zhou, Y., Yang, Y., Yang, X., Dong, W., & Guo, Q. (2026) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2026.3723876 · Cited by 1
“In specific downstream tasks, LLMs typically require fine-tuning to achieve effective performance, such as in mathematics [20] and healthcare [21].”
Barbosa, G. P., Carvalho, E., Oliveira, D. A., Moura, A. R., Guerra, A., Pinheiro, F., Correia, R., Santos, A. M. D., Ramião, N., & Mascarenhas, M. (2026) · Diagnostics · doi:10.3390/diagnostics16193107 · Cited by 1
Digitalisierung in der Urologie – Herausforderung und Chance
AK Informationstechnologie und Dokumentation (IT@DOC) (DGU), A. V., (DGU), A. I. U. D. (., AK Versorgungsforschung, Q. U. Ö. (., AK Bildgebende Systeme (DGU), S. U. (., Witzsch, U. K. F., Borkowetz, A., Enzmann, T., Rodler, S., Leyh‐Bannurah, S., Loch, T., et al. (2023) · Die Urologie · doi:10.1007/s00120-023-02154-w · Cited by 2
T-Agent: A Term-Aware Agent for Medical Dialogue Generation
Hu, Z., Zhao, H., Zhao, Y., Xu, S., & Xu, B. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650649 · Cited by 5
Khan, S. A., & Gunasekera, C. D. (2025) · Eye · doi:10.1038/s41433-025-03605-8 · Cited by 3
Afzaal, U., Su, Z., Sajjad, U., Stack, T., Rezapour, M., Lü, H., Niu, S., Gürcan, M. N., Frankel, W. L., Chen, W., et al. (2025) · SPIE Medical Imaging: Digital and Computational Pathology · doi:10.1117/12.3047352 · Cited by 1
Belligoli, P., Bitterman, D. S., & Miller, T. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-95841-0_6 · Cited by 1
RDLoRA:Rank-Decomposed Low-Rank Adaptation for Multi-task Medical Applications
He, Q., & Le, S. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-9994-0_29 · Cited by 1
Kim, E.-J., Joe, C. H., Yun, M., & Jeong, Y.-S. (2025) · Applied Sciences · doi:10.3390/app15158136 · Cited by 1
He, Y. V., Chowdhury, M., Joselowitz, J., Higham, A., & Lim, E. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.857 · Cited by 6
GRAF: Graph Retrieval Augmented by Facts for Romanian Legal Multi-Choice Question Answering
Crăciun, C.-G., Smădu, R.-A., Cercel, D.-C., & Cercel, M.-C. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.659 · Cited by 7
Mapping 1,000+ Language Models via the Log-Likelihood Vector
Oyama, M., Yamagiwa, H., Takase, Y., & Shimodaira, H. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1584 · Cited by 11
SudoLM: Learning Access Control of Parametric Knowledge with Authorization Alignment
Liu, Q., Wang, F., Xiao, C., & Chen, M. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1318 · Cited by 13
Impact of LLM assistance on physician decision-making: a multi-country randomized controlled trial
Rounding, N., Arif, L. S., Berg, J., Cals, J., de Boer, D. P., de Bont, E. G. P. M., Dijksman, S., Findyartini, A., Fouarge, D. J. A. G., Fregin, M.-C., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-03111-5 · Cited by 3
The Expanding Landscape of Neural Architectures and Their Impact in Biomedicine
Zhang, Z. F., Zhan, H., Wu, T., Burns, R., Hundal, J., & Costa, H. A. (2025) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-103023-050856 · Cited by 1
Evaluation of ophthalmic large language models: quantitative vs. qualitative methods
Tan, T. F., Thirunavukarasu, A. J., Quek, C. W. N., & Ting, D. S. W. (2025) · Current Opinion in Ophthalmology · doi:10.1097/icu.0000000000001171 · Cited by 1
Technological Evolution and Research Trends of Intelligent Question-Answering Systems in Healthcare
Lei, B., & Yin, P. (2025) · Healthcare · doi:10.3390/healthcare13182269 · Cited by 3
“MedicationQA [32] Contains QA data on drug information, medication recommendations, drug interactions, etc., centered around drug-related questions.”
LLM-driven medical document analysis: Enhancing trustworthy pathology and differential diagnosis
Kang, L., Fu, X., Terrades, O. R., Vazquez-Corral, J., Valveny, E., & Karatzas, D. (2026) · Lecture Notes in Computer Science · doi:10.1007/978-3-032-04624-6_36 · Cited by 1
Jiang, S., Chen, Y., Song, S., Zhang, Y., Jin, Y., Yang, F., Wu, J., & Liu, Z. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05141-7_32 · Cited by 6
Language as Data: A Survey of Natural Language Processing for Economics and Finance
Lucchetti, A. H., & Cajueiro, D. O. (2025) · Journal of Economic Surveys · doi:10.1111/joes.70014 · Cited by 2
BRMDS: an LLM-based multi-dimensional summary generation approach for bug reports
Zhang, Y., Li, Y., Fang, M., Yuan, X., & Du, J. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00553-1 · Cited by 4
Enhanced Fault Diagnosis Using Large Language Models and Probabilistic Label Fusion
Chen, Q., Yao, S., Wang, P., Shi, J., Qin, W., Li, J., Wang, J., & He, W. (2025) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2025.09.401 · Cited by 2
Kryvenko, I., & Chalyy, K. A. (2025) · Medical Informatics and Engineering · doi:10.11603/mie.1996-1960.2024.3-4.15462 · Cited by 1
Jun, T. J. (2025) · Korean Journal of Medicine · doi:10.3904/kjm.2025.100.5.219 · Cited by 1
Towards High-Capacity Provably Secure Steganography via Cascade Sampling
Lv, M., Fu, H., Yi, X., Huang, H., Cao, Y., & Liu, C. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-3537-8_29 · Cited by 1
Castano-Villegas, N., Llano, I., Villa, M. C., Zea, J., & Velásquez, L. (2026) · Intelligence-Based Medicine · doi:10.1016/j.ibmed.2026.100447 · Cited by 3
ViDove: A Translation Agent System with Multimodal Context and Memory-Augmented Reasoning
Lu, Y., Dai, W., Liu, J., Kwok, C. W., Wu, Z., Xiao, X., Sun, A., Fu, S., Zhan, J., Wang, Y., et al. (2025) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2025.emnlp-demos.17 · Cited by 5
Sayed, E. A., & Mosaad, S. M. (2025) · Sustainable Machine Intelligence Journal · doi:10.63689/3005-3617.1066 · Cited by 1
Evaluation of Question-Answering Systems Based on Semantic Similarity
Hoa, D. M., Bảy, V. Đ., & Tran, T. K. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-95-1746-6_53 · Cited by 1
Optimal Transport Depth Up-Scaling
Cao, M., Wang, X., & Αλέτρας, Ν. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.520 · Cited by 6
Detecting and Correcting Hallucinations in LLMs via Substantive Uncertainty and Iterative Validation
Chen, Z., Cheng, Y., & Gao, Y. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-3352-7_14 · Cited by 1
Enabling doctor-centric medical AI with LLMs through workflow-aligned tasks and benchmarks
Xie, W., Xiao, Q., Zheng, Y., Wang, X., Chen, J., Ji, K., Gao, A., Tiwari, P., Wan, X., Jiang, F., et al. (2025) · npj Health Systems · doi:10.1038/s44401-025-00038-z · Cited by 1
“To further validate the universality of the tasks de fi ned in the focus group discussions and gain deeper insights into doctors ’ needs for medical LLM assistance,weconductedasurveywithdoctorsfrom13tertiaryhospitals.”
Self-citationShao, M., & Zhang, H. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02146-4 · Cited by 4
MediMind: A Domain-Adaptive Medical LLM Leveraging Fine-Tuning and Retrieval-Augmented Generation
Hashemi, S. M. H., Kashani, H. B., & Fatemi, S. (2025) · International Conference on the Internet of Things (IoT) · doi:10.1109/iot69654.2025.11297589 · Cited by 2
“The evolution continued with Med-PaLM 2, which achieved expert-level performance with 86.5% accuracy on MedQA USMLE [10].”
InfluentialLiu, D., Long, Y., Zuoqiu, S., Liu, D., Li, K., Xie, K., Yang, R., Lin, Y., Liu, H., Yin, R., et al. (2026) · Journal of Medical Internet Research · doi:10.2196/97221 · Cited by 1
Friedman, M., Slouk, A., Gonen, N., Guzy, L., Paz, Y. G., Sacks, K. N., Rottenstreich, A., Weiner, E., Gluck, O., & Kleiner, I. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm15010117 · Cited by 1
“…this study align with what was already previously identified and reported suggesting that Large Language Models such as ChatGPT-4 perform well in structured and straightforward medical tasks [15,16] but still face challenges in scenarios requiring deep clinical reasoning or interpretation [17,18].”
Wang, R., Wang, J., Wang, Y., Zheng, C., Huang, S., Liu, X., Tan, G., Zhao, W., Wang, Z., Peng, S., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02285-8 · Cited by 1
Towards More Accurate App Review Response Generation with Fine-Grained Filtering
Li, Z., Li, Y., Zhang, T., Gao, C., & Ding, Z. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3786790 · Cited by 1
Lu, J., Chiu, K. W.-H., Chan, C., Cheng, H.-M., Zhou, J., Justin Christopher, N. G., Tang, F. F. Y., Kan, W. K., Yu, P. L. H., & Seto, W. (2026) · European Journal of Radiology · doi:10.1016/j.ejrad.2026.112689 · Cited by 1
Ahmed, H., & Sadka, A. H. (2026) · IEEE Access · doi:10.1109/access.2026.3656396 · Cited by 2
A Context Adaptive Instruction Tuning Framework for Diverse Clinical NLP Tasks
Mukalazi, S. H., Ouyang, J., & Katende, J. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11356497 · Cited by 1
“Prior work indicates that instruction sensitivity varies across task types [40], yet existing tuning strategies rarely adapt to such differences [14], [41].”
A data-efficient 3D medical vision-language model using only a 2D encoder
Lian, Y., Xie, Y., Jiang, Y., Wang, L., & Yu, H. (2026) · Scientific Reports · doi:10.1038/s41598-026-39526-z · Cited by 2
Gupta, S., Gal, Z. T., Touray, J., Luiselli, G. A., Ceesay, A., Manneh, E. K., Cham, M., Rolston, J. D., Chrenek, R., Bah, M. G., et al. (2026) · Neurosurgery · doi:10.1227/neu.0000000000003963 · Cited by 1
Assessing the Proficiency of Large Language Models in the Metal Casting: An Empirical Study
Yu, H., Yin, Z., Jin, Z., Wilk-Kołodziejczyk, D., Duan, S., Yu, S., Huo, Y., & Bao, C. (2026) · International Journal of Metalcasting · doi:10.1007/s40962-026-01888-5 · Cited by 1
Hao, K., & Wang, B. (2026) · Digital Medicine · doi:10.1097/dm-2025-00015 · Cited by 4
COMFORT: A Continual Fine-Tuning Framework for Foundation Models Targeted at Consumer Healthcare
Li, C., & Jha, N. (2026) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3797955 · Cited by 1
Leveraging Online Data to Enhance Medical Knowledge in a Small Persian Language Model
Ghassabi, M., Rostami, P., Kashani, H. B., Poursina, A., Kazemi, Z., & Tavakoli, M. (2025) · National and International Iranian Conference on Biomedical Engineering (ICBME) · doi:10.1109/icbme68496.2025.11392505 · Cited by 4
“Despite the vast amount of research and development dedicated to English medical language models, such as Med-Palm [2] [3] and others, there remains a significant disparity in resources available for non-English languages, particularly Persian.”
Metadata
- Authors
- Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Amin, M., Hou, L., Clark, K., Pfohl, S. R., Cole-Lewis, H., Neal, D., Rashid, Q. M., Schaekermann, M., Wang, A., Dash, D., Chen, J. H., Shah, N. H., Lachgar, S., Mansfield, P. A., Prakash, S., Green, B., Dominowska, E., Agüera y Arcas, B., Tomašev, N., Liu, Y., Wong, R., Semturs, C., Mahdavi, S. S., Barral, J. K., Webster, D. R., Corrado, G. S., Matias, Y., Azizi, S., Karthikesalingam, A., & Natarajan, V.
- Affiliations
- Google (United States) (US)
- Stanford Medicine (US)
- Stanford Health Care (US)
- Corresponding author
- Azizi, Shekoofeh
- Karthikesalingam, Alan
- Natarajan, Vivek
- ORCID
- Singhal, Karan
- Tu, Tao
- Gottweis, Juraj
- Sayres, Rory
- Amin, Mohamed
- Clark, Kevin
- Pfohl, Stephen R.
- Cole-Lewis, Heather
- Rashid, Qazi Mamunur
- Schaekermann, Mike
- Wang, Amy
- Dash, Dev
- Chen, Jonathan H.
- Shah, Nigam H.
- Lachgar, Sami
- Mansfield, Philip Andrew
- Prakash, Sushant
- Green, Bradley
- Dominowska, Ewa
- Tomašev, Nenad
- Liu, Yun
- Wong, Renee
- Semturs, Christopher
- Mahdavi, S. Sara
- Barral, Joelle K.
- Webster, Dale R.
- Corrado, Greg S.
- Matias, Yossi
- Azizi, Shekoofeh
- Karthikesalingam, Alan
- Natarajan, Vivek
- Venue
- Nature Medicine
- Published
- 2025-01-08
- Volume
- 31
- Issue
- 3
- Pages
- 943-950
- ISSN
- 1078-8956 (ISSN-L)
- 1546-170X
- PMC
- PMC11922739
- OpenAlex
- W4406152279
- Semantic Scholar
- b34c0b0169925a6c7f14e3de9764ed9505f30de3
- Cited by
- 1,734
- References
- 57
- Citations by year
- 2026: 771 · 2025: 591
- FWCI
- 277.36
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-nd
- creativecommons.org/licenses/by-nc-nd/4.0 (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0 (vor)
- Locations
- Nature Medicine (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Gordon and Betty Moore Foundation (#12409)
- American Heart Association
- National Institutes of Health (UM1TR004921)
- Stanford Bio-X
- National Institute on Drug Abuse (UG1DA015815)
- National Institute of Allergy and Infectious Diseases (R01AI178121)
- National Center for Advancing Translational Sciences
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- medical question answering, large language models, ensemble refinement, clinical reasoning, grounding, USMLE, MedQA, physician preferences
- MeSH terms
- Large Language Models, Humans, Physicians, Pilot Projects, United States
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.