The DOI matches its DataCite record; 11 of the 11 references it cites were found in a registry or an index.
Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine
Open accessAbstract
Generalist foundation models such as GPT-4 have displayed surprising capabilities in a wide variety of domains and tasks. Yet, there is a prevalent assumption that they cannot match specialist capabilities of fine-tuned models. For example, most explorations to date on medical competency benchmarks have leveraged domain-specific training, as exemplified by efforts on BioGPT and Med-PaLM. We build on a prior study of GPT-4's capabilities on medical challenge benchmarks in the absence of special training. Rather than using simple prompting to highlight the model's out-of-the-box capabilities, we perform a systematic exploration of prompt engineering. We find that prompting innovation can unlock deeper specialist capabilities and show that GPT-4 easily tops prior leading results for medical benchmarks. The prompting methods we explore are general purpose, and make no specific use of domain expertise, removing the need for expert-curated content. Our experimental design carefully controls for overfitting during the prompt engineering process. We introduce Medprompt, based on a composition of several prompting strategies. With Medprompt, GPT-4 achieves state-of-the-art results on all nine of the benchmark datasets in the MultiMedQA suite. The method outperforms leading specialist models such as Med-PaLM 2 by a significant margin with an order of magnitude fewer calls to the model. Steering GPT-4 with Medprompt achieves a 27% reduction in error rate on the MedQA dataset over the best methods to date achieved with specialist models and surpasses a score of 90% for the first time. Beyond medical problems, we show the power of Medprompt to generalize to other domains and provide evidence for the broad applicability of the approach via studies of the strategy on exams in electrical engineering, machine learning, philosophy, accounting, law, nursing, and clinical psychology.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs have quickly surpassed the average human score on the United States Medical Licensing Exam (USMLE) in a short amount of time, from 38.1% in September 20212 to 90.2% in November 20233 (human passing score is 60%, human expert score is 87%4).”
From AgentClinic: a multimodal benchmark for tool-using clinical AI agents (2026) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“To account for the inherent stochasticity of LLMs, we report results from 10 independent runs per benchmark as mean accuracy with 95% confidence intervals (CIs), moving beyond the single-run point estimates common in prior works.2,3,4”
From Model confrontation and collaboration: A debate intelligence framework for enhancing medical reasoning in large language models (2026) · cites this paper as [4]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Our study included 6 cases that could feasibly be completed within a single study session while remaining comparable to standard practices in national licensing and objective structured clinical examinations to use a small, but broad sample of clinical cases.6,46,47,48,49”
From Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial (2024) · cites this paper as [47]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“More recently, LLMs have surpassed human experts on medical exams (LLMs: 90.2% (Nori et al. 2023), human experts: 87% (Li´ evin et al. 2022), passing score: 60%).”
From Human-Clinical AI Agent Collaboration (2025) · cites this paper as Nori et al. 2023
The cited source says what the claim attributes to it.
In the citing paper
“However, the recently introduced Medprompt strategy (Nori et al., 2023), which does not use debate, performs the best overall.”
From Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs (2023) · cites this paper as Nori et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Recently, LLMs have shown the ability to encode clinical knowledge7,8, retrieve relevant medical texts9, and perform accurate single-turn medical question-answering3,10–12.”
From AgentClinic: a multimodal benchmark for tool-using clinical AI agents (2026) · cites this paper as [3]
The cited source says what the claim attributes to it.
In the citing paper
“reported multiple choice evaluations where specialist QAs typically have lower performance3”
From AgentClinic: a multimodal benchmark for tool-using clinical AI agents (2026) · cites this paper as [3]
The cited source says what the claim attributes to it.
In the citing paper
“with GPT-4 showing superior performance compared to GPT-3.5 in knowledge assessments3,43.”
From AgentClinic: a multimodal benchmark for tool-using clinical AI agents (2026) · cites this paper as [3]
The cited source says what the claim attributes to it.
In the citing paper
“LLMs have advanced text generation capability and extensive domain-specific knowledge [8,9].”
From Mitigating cognitive biases in clinical decision-making through multi-agent conversations using large language models: simulation study (2024) · cites this paper as [8]
The cited source says what the claim attributes to it.
In the citing paper
“In recent years, large language models (LLMs) tailored for healthcare have achieved state-of-the-art (SOTA) performance on a range of medical benchmarks.2,3,4”
From Model confrontation and collaboration: A debate intelligence framework for enhancing medical reasoning in large language models (2026) · cites this paper as [4]
The cited source says what the claim attributes to it.
In the citing paper
“This may be explained by the sensitivity of LLM output to prompt formulation.32”
From Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial (2024) · cites this paper as [32]
The cited source says what the claim attributes to it.
In the citing paper
“where LLMs score well above passing8–10,17–20”
From Evaluation and mitigation of the limitations of large language models in clinical decision-making (2024) · cites this paper as [20]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 17
- Niels J. Blunch. Position bias in multiple-choice questions. Journal of Marketing Research, 21(2):216-220, 1984.
- Rich Caruana, Alexandru Niculescu-Mizil, Geoff Crew, and Alex Ksikes. Ensemble selection from libraries of models. In Proceedings of the twenty-first international conference on Machine learning, page 18, 2004.
- Matthias Feurer and Frank Hutter. Hyperparameter optimization. Automated machine learning: Methods, systems, challenges, pages 3-33, 2019.
- Zelalem Gero, Chandan Singh, Hao Cheng, Tristan Naumann, Michel Galley, Jianfeng Gao, and Hoifung Poon. Self-verification improves few-shot clinical information extraction. In ICML 3rd Workshop on Interpretable Machine Learning in Healthcare (IMLH), 2023.
- Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Nau- mann, Jianfeng Gao, and Hoifung Poon. Domain-specific language model pretraining for biomed- ical natural language processing. ACM Transactions on Computing for Healthcare (HEALTH), 3(1):1-23, 2021.
- Ayanna M. Howard, Cha Zhang, and Eric Horvitz. Addressing bias in machine learning algo- rithms: A pilot study on emotion recognition for intelligent systems. 2017 IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO), pages 1-7, 2017.
- Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14):6421, 2021.
- Miyoung Ko, Jinhyuk Lee, Hyunjae Kim, Gangwoo Kim, and Jaewoo Kang. Look at the first sentence: Position bias in question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1109-1121, Online, November 2020. Association for Computational Linguistics.
- Tiffany H Kung, Morgan Cheatham, Arielle Medenilla, Czarina Sillos, Lorie De Leon, Camille Elepaño, Maria Madriaga, Rimel Aggabao, Giezel Diaz-Candido, James Maningo, et al. Per- formance of chatgpt on usmle: Potential for ai-assisted medical education using large language models. PLoS digital health, 2(2):e0000198, 2023.
- Michael Madaio, Lisa Egede, Hariharan Subramonyam, Jennifer Wortman Vaughan, and Hanna Wallach. Assessing the fairness of ai systems: AI practitioners' processes, challenges, and needs for support. In 25th ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW 2022), February 2022.
- John McCarthy, Marvin L Minsky, Nathaniel Rochester, and Claude E Shannon. A proposal for the Dartmouth summer research project on artificial intelligence, August 31, 1955. AI magazine, 27(4):12-12, 2006.
- Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. In Conference on Health, Inference, and Learning, pages 248-260. PMLR, 2022.
- Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. Advances in neural information processing systems, 27, 2014.
- Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. Emergent abilities of large language models. Transactions on Machine Learning Research, 2022. Survey Certification.
- Michihiro Yasunaga, Antoine Bosselut, Hongyu Ren, Xikun Zhang, Christopher D. Manning, Percy Liang, and Jure Leskovec. Deep bidirectional language-knowledge graph pretraining. In Neural Information Processing Systems (NeurIPS), 2022.
- Michihiro Yasunaga, Jure Leskovec, and Percy Liang. Linkbert: Pretraining language models with document links. In Association for Computational Linguistics (ACL), 2022.
- Travis Zack, Eric Lehman, Mirac Suzgun, Jorge A. Rodriguez, Leo Anthony Celi, Judy Gichoya, Dan Jurafsky, Peter Szolovits, David W. Bates, Raja-Elie E. Abdulnour, Atul J. Butte, and Emily Alsentzer. Coding inequity: Assessing gpt-4's potential for perpetuating racial and gender biases in healthcare. medRxiv, 2023.
Cited by
Sorted by citations
Items 101–200 of 200
Wang, Z., Zhang, Y., Min, W., Guan, Q., & Yu, W. (2025) · Journal of Geography in Higher Education · doi:10.1080/03098265.2025.2549784 · Cited by 9
Jarrett, P. C., Hill, J., Howell, M., Moore, K. G., Thoppil, J. J., Ortiz, L. V., Parnell, S. T., Courtney, D. M., McDonald, S., Diercks, D., et al. (2025) · Cureus · doi:10.7759/cureus.94476 · Cited by 8
Zhang, Y., Huang, B., Yuan, W., Jiang, Z., Peng, L., Chen, S., & Tan‐Soo, J. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104090 · Cited by 7
Ting, D. S. W., Ong, J. C. L., Jin, L., Elangovan, K., Lim, G. Y. S., Lim, D. Y. Z., Sng, G. G. R., Ke, Y., Tung, J. Y. M., Zhong, R. J., et al. (2024) · Research Square · doi:10.21203/rs.3.rs-4023142/v1 · Cited by 7
Liu, N., Chen, L., Tian, X., Zou, W., Chen, K., & Cui, M. (2024) · arXiv · doi:10.48550/arxiv.2401.02777 · Cited by 7
A Self-Evolving Framework for Multi-Agent Medical Consultation Based on Large Language Models
Chen, K., Qi, J., Huo, J., Tian, P., Meng, F., Yang, X., & Gao, Y. (2025) · ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp49660.2025.10889517 · Cited by 6
Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs
Smit, A., Duckworth, P., Grinsztajn, N., Barrett, T. D., & Pretorius, A. (2023) · arXiv · doi:10.48550/arxiv.2311.17371 · Cited by 73
Njuguna, G. N., & Qingfei, M. (2025) · Technology in Society · doi:10.1016/j.techsoc.2025.103080 · Cited by 7
Wang, T.-H., Jheng, J.-C., Tseng, Y.-T., Chen, L., & Chen, Y. (2025) · Journal of the Chinese Medical Association · doi:10.1097/jcma.0000000000001273 · Cited by 5
Griewing, S., Gremke, N., Wagner, U. A. G., Wallwiener, M., Kühn, S., & Commission Digital Medicine of the German Society for Gynecology and Obstetrics (2024) · Geburtshilfe und Frauenheilkunde · doi:10.1055/a-2335-6122 · Cited by 6
Sawamura, S., Kohiyama, K., Takenaka, T., Sera, T., Inoue, T., & Nagai, T. (2025) · Cureus · doi:10.7759/cureus.76989 · Cited by 5
Systematic analysis of hepatotoxicity: combining literature mining and AI language models
Bauer, C., Dang, L. T. D., van den Beucken, T., Schuchhardt, J., & Herwig, R. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1561292 · Cited by 4
Quo Vadis, AI-Empowered Doctor?
Takahashi, G., von Liechti, L., & Tarshizi, E. (2025) · JMIR Medical Education · doi:10.2196/70079 · Cited by 4
Deep Learning for Computational Heterogeneous Catalysis: Fundamentals and Applications
Deshmukh, G., Ghanekar, P. G., & Greeley, J. (2025) · Journal of the Indian Institute of Science · doi:10.1007/s41745-025-00484-6 · Cited by 6
Improving LLM Prompting with Ensemble of Instructions: A Case Study on Sentiment Analysis
Tran, V., & Matsui, T. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-3076-6_21 · Cited by 4
Automated Spinal MRI Labelling from Reports Using a Large Language Model
Park, R. Y., Windsor, R., Jamaludin, A., & Zisserman, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72086-4_10 · Cited by 4
Noda, M., Koshu, R., Tsunoda, R., Ogihara, H., Kamo, T., Itō, M., & Fushiki, H. (2025) · JMIR Formative Research · doi:10.2196/70070 · Cited by 5
Moser, D., Bender, M., & Sariyar, M. (2025) · Applied Artificial Intelligence · doi:10.1080/08839514.2025.2519169 · Cited by 5
Survey and improvement strategies for gene prioritization with large language models
Neeley, M. B., Qi, G., Wang, G., Tang, R., Mao, D., Liu, C., Pasupuleti, S., Yuan, B., Xia, F., Liu, P., et al. (2024) · Bioinformatics Advances · doi:10.1093/bioadv/vbaf148 · Cited by 9
Comments on Contemporary Uses of Machine Learning for Electronic Health Records
Bryan, J., & Li, D. (2024) · North Carolina Medical Journal · doi:10.18043/001c.120570 · Cited by 4
Trippas, J. R. (2024) · ACM eBooks · doi:10.1145/3674127.3674135 · Cited by 5
Large language models: game-changers in the healthcare industry
Dong, B., Zhang, L., Yuan, J., Chen, Y., Li, Q., & Shen, L. (2024) · 中国科学通报：英文版 · doi:10.1016/j.scib.2024.11.031 · Cited by 4
Samaan, J. S., Margolis, S., Srinivasan, N., Srinivasan, A., Yeo, Y. H., Anand, R., Samaan, F. S., Mirocha, J., Safavi‐Naini, S. A. A., Kurdi, B. E., et al. (2024) · medRxiv · doi:10.1101/2024.07.27.24310809 · Cited by 4
Hänscheid, T., Carrasco, J., & Grobusch, M. P. (2025) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2025.08.006 · Cited by 4
Lau, J., Bisht, S., Horton, R., Crisan, A., Jones, J. H., Gantotti, S., & DeSantis, E. R. H. (2024) · JMIR AI · doi:10.2196/55277 · Cited by 6
Arkangel AI: A conversational agent for real-time, evidence-based medical question-answering
Villa, M. C., Castano-Villegas, N., Llano, I., Martinez, J., Guevara, M. F., Zea, J., & Velásquez, L. (2025) · Intelligence-Based Medicine · doi:10.1016/j.ibmed.2025.100274 · Cited by 4
Kadi, H., Abdellatif, A., Njamen, D. I. K., & Pereme, F. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1598979 · Cited by 5
Ba, H., Du, H., Cheng, C., Zhang, Y., Ruan, L., & Chen, Z. (2025) · JMIR Formative Research · doi:10.2196/70602 · Cited by 3
Miao, B. Y., Williams, C. Y. K., Chinedu-Eneh, E., Zack, T. I., Alsentzer, E., Butte, A. J., & Chen, I. Y. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01615-0 · Cited by 14
Volokhovskyi, V. (2024) · Innovative technologies and scientific solutions for industries · doi:10.30837/2522-9818.2024.2.048 · Cited by 4
Matching Human Expertise: ChatGPT’s Performance on Hand Surgery Examinations
Kirschenbaum, Z. A., Han, Y., Vrindten, K. L., Wang, H., Cody, R. P., Katt, B. M., & Kirschenbaum, D. (2025) · Hand · doi:10.1177/15589447251322914 · Cited by 3
Mensah, P. B., Quao, N. S. A., & Dagadu, S. (2024) · medRxiv · doi:10.1101/2024.04.17.24305971 · Cited by 4
Murphy, K. B., Schilder, B. M., & Skene, N. (2024) · medRxiv · doi:10.1101/2024.06.10.24308475 · Cited by 3
Lai, S. M., Malik, A. M., Sathe, T. S., Silvestri, C. J., Manji, G. A., & Kluger, M. D. (2024) · medRxiv · doi:10.1101/2024.09.20.24314053 · Cited by 3
RiskAgent: Autonomous Medical AI Copilot for Generalist Risk PredictionPreprint
Liu, F., Wu, J., Zhou, H., Gu, X., Molaei, S., Thakur, A., Clifton, L., Wu, H., & Clifton, D. A. (2025) · medRxiv · doi:10.1101/2025.04.03.25323489 · Cited by 3
Jin, R., Guo, Y., Qu, Y., Yang, M., Shang, C., Yang, Q., Chao, L., Zhou, Y., Xu, R. M., Xu, Z., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.09.03.674085 · Cited by 5
Temporal Extraction and Retrieval
Alonso, O., & Spaniol, M. (2024) · ACM eBooks · doi:10.1145/3674127.3674137 · Cited by 2
Tonellotto, N. (2024) · ACM eBooks · doi:10.1145/3674127.3674130 · Cited by 3
Maruyama, J., Maki, S., Furuya, T., Nagashima, Y., Kitagawa, K., Toki, Y., Iwata, S., Yazaki, M., Kitamura, T., Gushiken, S., et al. (2025) · Journal of Orthopaedic Science · doi:10.1016/j.jos.2025.03.003 · Cited by 2
Wang, Y., Jiang, Y., Jin, W., Xu, Y., Lin, W., Wang, J., Song, Q., & Fang, Z. (2026) · Frontiers in Medicine · doi:10.3389/fmed.2025.1716122 · Cited by 2
Harnessing Large Language Models for Satellite Ground Tests
Connolly, B. J., & Anderson, K. M. (2024) · AIAA SciTech Forum · doi:10.2514/6.2024-0916 · Cited by 2
Generating simulated data with a large language model
Kerley, J., Anderson, D. T., Buck, A. R., & Alvey, B. J. (2024) · Synthetic Data for Artificial Intelligence and Machine Learning: Tools, Techniques, and Applications · doi:10.1117/12.3013460 · Cited by 2
Are You Being Tracked? Discover the Power of Zero-Shot Trajectory Tracing with LLMs!
Yang, H., Ji, S., Wu, R., & Xu, W. (2024) · doi:10.1109/cscaiot62585.2024.00008 · Cited by 15
Ghassel, A., Zhu, X., & Thomas, S. W. (2024) · Conference proceedings - Canadian Conference on Electrical and Computer Engineering · doi:10.1109/ccece59415.2024.10667232 · Cited by 3
Verberne, S. (2024) · ACM eBooks · doi:10.1145/3674127.3674141 · Cited by 5
Ferro, N., & Maistro, M. (2024) · ACM eBooks · doi:10.1145/3674127.3674132 · Cited by 3
Privacy in Information Retrieval
Soboroff, I. M. (2024) · ACM eBooks · doi:10.1145/3674127.3674139 · Cited by 2
Sun, X., Hong, Q., Zhang, M., Li, Y., Chen, T., Huang, Z., Liang, G., Tang, W., Xu, S., Ni, X., et al. (2026) · Cell Reports Medicine · doi:10.1016/j.xcrm.2025.102547 · Cited by 3
Self-citationMedSentry: Understanding and Mitigating Safety Risks in Medical LLM Multi-Agent Systems
Chen, K., Zhen, T., Wang, H., Liu, K., Li, X., Huo, J., Yang, T., Xu, J., Dong, W., & Gao, Y. (2025) · arXiv · doi:10.48550/arxiv.2505.20824 · Cited by 32
“We also include two state-of-the-art baselines, Medprompt [65] and Multi-expert Prompting [66].”
The Illusion of Readiness in Health AI
Gu, Y., Fu, J., Liu, X., Valanarasu, J. M. J., Codella, N. C., Tan, R., Liu, Q., Jin, Y., Zhang, S., Wang, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2509.18234 · Cited by 1
Self-citationMDTeamGPT: Mitigating Context Collapse and Enabling Self-Evolution in Medical Multi-Agent Reasoning
Chen, K., Li, X., Yang, T., Wang, H., Yang, G., Huo, J., & Gao, Y. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.1427 · Cited by 2
Mtfineval: a multi-domain chinese financial question-answering benchmark
Liu, X., & Jin, K. (2024) · International Conference on Big Data and Digital Management (ICBDDM) · doi:10.1145/3696500.3696552 · Cited by 1
eCommerce Search and Discovery
Kallumadi, S. T., & King, T. H. (2024) · ACM eBooks · doi:10.1145/3674127.3674140 · Cited by 1
Test for the Best: Using ChatGPT to Create Effective Ad Taglines
Redden, J. P. (2025) · Journal of Current Issues & Research in Advertising · doi:10.1080/10641734.2025.2457663 · Cited by 1
An AI‐Enabled Data Processing Pipeline for Ingesting Borehole Data in Peridotite Environments
Aiken, J. M., Dufornet, E., Amiri, H., Ternieten, L., & Plümper, O. (2025) · Journal of Geophysical Research Machine Learning and Computation · doi:10.1029/2025jh000666 · Cited by 2
Khalid, S. M., Wölker, T., Molano, L.-A. G., Graf, S., & Keller, A. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag089 · Cited by 1
Amplifying missing voices in healthcare research: an AI framework for co-production of PPIE
Steele, A., Strachan, F., Milligan, C., & Ibbotson, T. R. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1771729 · Cited by 1
The Doctor Will (Still) See You Now: On the Structural Limits of Agentic AI in Healthcare
Aránguiz-Dias, G., Meimandi, K. J., Griffith, A., Dias, C. A., Kim, G. R., Saadeddin, L., & Kochenderfer, M. J. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812224 · Cited by 4
Yin, H., Tang, J., Li, S., & Wang, T. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-9431-7_36 · Cited by 1
Galuščáková, P., Oard, D. W., & Nair, S. (2024) · ACM eBooks · doi:10.1145/3674127.3674136 · Cited by 2
Gómez, E., Cuesta, H., Gkiokas, A., Gómez-Cañón, J. S., Porcaro, L., & Yesiler, F. (2024) · ACM eBooks · doi:10.1145/3674127.3702959 · Cited by 1
Baeza-Yates, R. A., Murgai, L., & Dudy, S. (2024) · ACM eBooks · doi:10.1145/3674127.3674138 · Cited by 1
A framework for an AI pipeline for borehole dataPreprint
Aiken, J. M., Dufournet, E., Amiri, H., Ternieten, L., & Plümper, O. (2024) · doi:10.22541/au.171111390.00795107/v1 · Cited by 1
Shah, M., Kuo, E. J., Kuo, J. H., Hsu, S. Y., McManus, C. M., Liou, R., Lee, J. A., & Sathe, T. S. (2024) · medRxiv · doi:10.1101/2024.05.29.24308002 · Cited by 1
Exploring the Boundaries of Financial Statement Fraud Detection with Large Language ModelsPreprint
Kirkos, E., Boskou, G., Chatzipetrou, E., Tiakas, E., & Σπάθης, Χ. Τ. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4842962 · Cited by 5
Exploring the Boundaries of Financial Statement Fraud Detection with Large Language ModelsPreprint
Boskou, G., Chatzipetrou, E., Tiakas, E., Kirkos, E., & Spathis, C. T. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4897041 · Cited by 1
Do Language Models Think Like Doctors?Preprint
McCoy, L. G., Swamy, R., Sagar, N., Wang, M., Cao, J., Bacchi, S., Fong, J. M. N., Tan, N. C., Tan, K. C., Buckley, T. A., et al. (2025) · medRxiv · doi:10.1101/2025.02.11.25321822 · Cited by 2
Human-Clinical AI Agent Collaboration
Kadem, M., & Al-Khazraji, B. (2025) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v6i1.36060
“More recently, LLMs have surpassed human experts on medical exams (LLMs: 90.2% (Nori et al. 2023), human experts: 87% (Li´evin et al. 2022), passing score: 60%).”
InfluentialThe economics of accuracy for medical reasoning with large language models
Bhattacharyya, K., & Kamabattula, S. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001182 · Cited by 1
Wang, Y., Zhang, X., Yao, K., Zeng, Z., Song, K., Lin, J., & Shen, Z. (2026) · arXiv · doi:10.48550/arxiv.2606.05241
ScoreFlow: Mastering LLM Agent Workflows via Score-based Preference Optimization
Wang, Y., Yang, L., Li, G., Wang, M., & Aragam, B. (2025) · arXiv · doi:10.48550/arxiv.2502.04306
Alonso, O., & Baeza-Yates, R. (2024) · ACM eBooks · doi:10.1145/3674127.3674128
Searching Personal Collections
Bendersky, M., Metzler, D., Najork, M., & Wang, X. (2024) · ACM eBooks · doi:10.1145/3674127.3674142
Query-driven Search Functionality
Li, L. (2024) · ACM eBooks · doi:10.1145/3674127.3674133
Alonso, O., & Baeza-Yates, R. (2024) · ACM eBooks · doi:10.1145/3674127.3702962
Alonso, O., & Baeza-Yates, R. (2024) · ACM eBooks · doi:10.1145/3674127.3674129
Alonso, O., & Baeza-Yates, R. (2024) · ACM eBooks · doi:10.1145/3674127.3702961
Hersh, W., & Roberts, K. E. (2024) · ACM eBooks · doi:10.1145/3674127.3702960
Bast, H., Kalmbach, J., Klumpp, T., & Korzen, C. (2024) · ACM eBooks · doi:10.1145/3674127.3674134
Subramaniam, S., Rizvi, S., Ramesh, R., Sehgal, V., Gurusamy, B., Arif, H., Tran, J. S., Thamman, R., Anyanwu, E. C., Mastouri, R. A., et al. (2024) · medRxiv · doi:10.1101/2024.02.20.24302419
Few-Shot N-Ary Knowledge Inference Using Large Language Models
Lang, F., Song, W., Zhu, Q., & Zhu, S. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-0792-1_22
Zhao, Y. (2025) · Research Square · doi:10.21203/rs.3.rs-6124845/v1 · Cited by 1
Skin Disease Classification with LVLMs: An Empirical Study
Zeng, X., Wang, Z., Zhang, H., Luo, Y., & Wang, W. (2025) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd64889.2025.11033252
Eidam, B., & Bauer, D. (2025) · Springer eBooks · doi:10.1007/978-3-658-48158-2_7
A hybrid deployment model for generative artificial intelligence in hospitals
Griot, M., Hemptinne, C., Vanderdonckt, J. M., & Yüksel, D. (2025) · Machine Learning Health · doi:10.1088/3049-477x/addb51 · Cited by 1
Mendonca, R., Smedinghoff, S., Li, B. S., Gupta, K., Ruan, Y., Morris, S., Isabelli, P., Hattery, D. W., Ratevosian, J., & Mateen, B. A. (2025) · doi:10.12688/verixiv.1936.1
Radke, R., Diller, G., Reddy, R. G., Shivaram, P., Danford, D. A., & Kutty, S. (2025) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztaf111 · Cited by 1
Exploring Generalist Foundation Models for Time Series of Electrodermal Activity Data
Alchieri, L., Candian, L., Abdalazim, N., Alecci, L., de Felice, G., & Silvia Santini, B. (2025) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3714394.3756186
Application of Large Language Models in the Problem of Event Forecasting
Dakhnovich, A. D., Богина, В. М., & Makeeva, A. A. (2025) · Automatic Control and Computer Sciences · doi:10.3103/s0146411625701111
Advancing Item Writing Methods with Large Language Models and Subject Matter Experts
Rios, O., & Albano, T. (2026) · Applied Measurement in Education · doi:10.1080/08957347.2026.2655640
Li, C., Jia, W.-W., Ghanad, E., Gao, H., Abdelhadi, S., Reißfelder, C., Șandra-Petrescu, F., & Yang, C. (2026) · Digestive Diseases · doi:10.1159/000551433
Adapting Efficient LLMs for Container Logistics: A Case Study on Few-Shot Prompt Engineering
Ilin, M., & Pavlyuk, D. (2026) · Lecture notes in intelligent transportation and infrastructure · doi:10.1007/978-3-032-18771-0_38
Murphy, K. B., Schilder, B. M., & Skene, N. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1794934 · Cited by 4
Rajeev, M., & Narayan, A. (2026) · medRxiv · doi:10.64898/2026.06.06.26354838
Eryılmaz, B., Bahn, M., Damm, H., Chu, B., Bender, N., Kleesiek, J., & Nensa, F. (2026) · doi:10.2196/preprints.109219
Fu, R., Zhang, Y., Zhang, S., Wu, X., Gu, W., Wang, F., Che, Z., Liu, X., Sun, Y., & Zhang, Y. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3820774
Self-citationLakehouse‐Driven Surgical Databases: Scoping Review and Implementation of a Free Flap Database
Barrette, L., Velu, S. M., Troftgruben, M. H. S., DOOLEY, B., Farwell, D. G., & Brody, R. M. (2026) · The Laryngoscope · doi:10.1002/lary.70956
OLIVE: Output-Level Iterative Verification Engine
Jahromi, A., Gao, J., He, Z., & Mendoza, J. (2026) · ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software for Humanity (SPLASH) · doi:10.1145/3837729.3839568
Evaluating Gemma 3 27B for local undergraduate physics tutoring
Völschow, M., Buczek, P., Riefer, P., Jasko, A., & Steinbach, P. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-02296-8
Metadata
- Authors
- Nori, H., Lee, Y. T., Zhang, S., Carignan, D., Edgar, R., Fusi, N., King, N., Larson, J., Li, Y., Liu, W., Luo, R., McKinney, S. M., Ness, R. O., Poon, H., Qin, T., Usuyama, N., White, C., & Horvitz, E.
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2311.16452
- OpenAlex
- W4389156617
- Cited by
- 178
- References
- 0
- Citations by year
- 2026: 16 · 2025: 67 · 2024: 93 · 2023: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- GPT-4, prompt engineering, medical question answering, fine-tuned models, overfitting, domain generalization, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
