The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Evaluating large language models as agents in the clinic
Open accessAbstract
Recent developments in large language models (LLMs) have unlocked opportunities for healthcare, from information synthesis to clinical decision support. These LLMs are not just capable of modeling language, but can also act as intelligent “agents” that interact with stakeholders in open-ended conversations and even influence clinical decision-making. Rather than relying on benchmarks that measure a model’s ability to process clinical data or answer standardized test questions, LLM agents can be modeled in high-fidelity simulations of clinical settings and should be assessed for their impact on clinical workflows. These evaluation frameworks, which we refer to as “Artificial Intelligence Structured Clinical Examinations” (“AI-SCE”), can draw from comparable technologies where machines operate with varying degrees of self-governance, such as self-driving cars, in dynamic environments with multiple stakeholders. Developing these robust, real-world clinical evaluations will be crucial towards deploying LLM agents in medical settings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Agent-based modeling: Methods and techniques for simulating human systems
Bonabeau, E. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.082080899 · Cited by 4,500
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
MIMIC-IV, a freely accessible electronic health record dataset
Johnson, A. E. W., Bulgarelli, L., Shen, L., Gayles, A., Shammout, A., Horng, S., Pollard, T. J., Hao, S., Moody, B., Gow, B., et al. (2023) · Scientific Data · doi:10.1038/s41597-022-01899-x · Cited by 3,401
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · arXiv · doi:10.48550/arxiv.2304.03442 · Cited by 2,173
Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine
Lee, P., Bubeck, S., & Petro, J. (2023) · New England Journal of Medicine · doi:10.1056/nejmsr2214184 · Cited by 1,738
Toward expert-level medical question answering with large language models
Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Amin, M., Hou, L., Clark, K., Pfohl, S. R., Cole-Lewis, H., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03423-7 · Cited by 1,734
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.12712 · Cited by 1,526
Fagnant, D. J., & Kockelman, K. M. (2014) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2013.12.001 · Cited by 1,177
Ethical Machine Learning in Healthcare
Chen, I. Y., Pierson, E., Rose, S., Joshi, S., Ferryman, K., & Ghassemi, M. (2021) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-092820-114757 · Cited by 645
Capabilities of GPT-4 on Medical Challenge Problems
Nori, H., King, N., McKinney, S. M., Carignan, D., & Horvitz, E. (2023) · arXiv · doi:10.48550/arxiv.2303.13375 · Cited by 509
Agent-Based Modeling in Public Health: Current Applications and Future Directions
Tracy, M., Cerdá, M., & Keyes, K. M. (2018) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040617-014317 · Cited by 408
Large language models are few-shot clinical information extractors
Agrawal, M., Hegselmann, S., Lang, H., Kim, Y., & Sontag, D. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.130 · Cited by 358
The shaky foundations of large language models and foundation models for electronic health records
Wornow, M., Xu, Y., Thapa, R., Patel, B., Steinberg, E., Fleming, S., Pfeffer, M. A., Fries, J., & Shah, N. H. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00879-8 · Cited by 324
Objective structured clinical examination: the assessment of choice
Zayyan, M. (2011) · Oman Medical journal · doi:10.5001/omj.2011.55 · Cited by 295
Agent-based modeling: A revolution?
Bankes, S. C. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.072081299 · Cited by 295
Korngiebel, D. M., & Mooney, S. D. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00464-x · Cited by 259
NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails
Rebedea, T., Dinu, R., Sreedhar, M. N., Parisien, C., & Cohen, J. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2023.emnlp-demo.40 · Cited by 220
A Survey on Simulators for Testing Self-Driving Cars
Kaur, P., Taghavi, S., Tian, Z., & Shi, W. (2021) · International Conference on Connected and Autonomous Driving (MetroCAD) · doi:10.1109/metrocad51599.2021.00018 · Cited by 152
Towards Conversational Diagnostic AIPreprint
Tu, T., Palepu, A. K., Schaekermann, M., Saab, K. K., Freyberg, J., Tanno, R., Wang, A., Li, B., Amin, M., Tomašev, N., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.05654 · Cited by 114
The Diagnostic and Triage Accuracy of the GPT-3 Artificial Intelligence Model
Levine, D. M., Tuwani, R., Kompa, B., Varma, A., Finlayson, S. G., Mehrotra, A., & Beam, A. (2023) · doi:10.1101/2023.01.30.23285067 · Cited by 106
Six ways large language models are changing healthcare
Webster, P. C. (2023) · Nature Medicine · doi:10.1038/s41591-023-02700-1 · Cited by 58
Radhakrishnan, L., Schenk, G., Muenzen, K., Oskotsky, B., Choshali, H. A., Plunkett, T., Israni, S., & Butte, A. J. (2023) · JAMIA Open · doi:10.1093/jamiaopen/ooad045 · Cited by 54
Self-citationAuto-GPT for Online Decision Making: Benchmarks and Additional Opinions
Yang, H., Yue, S., & He, Y. (2023) · arXiv · doi:10.48550/arxiv.2306.02224 · Cited by 44
Evaluation of GPT-3.5 and GPT-4 for supporting real-world information needs in healthcare deliveryPreprint
Dash, D., Thapa, R., Banda, J. M., Swaminathan, A., Cheatham, M., Kashyap, M., Kotecha, N., Chen, J. H., Gombar, S., Downing, N. L., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.13714 · Cited by 33
Bedi, S., Fleming, S. L., Chiang, C., et al. (2023) · medRxiv · doi:10.1101/2023.04.25.23288588 · Cited by 20
MultiTurnCleanup: A Benchmark for Multi-Turn Spoken Conversational Transcript Cleanup
Shen, H., Zayats, V., Rocholl, J. C., et al. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.613 · Cited by 2
Flagged first, then by citations
Research works printed but not linked: 1
- Johri, S. et al. Testing the Limits of Language Models: A Conversational Framework for Medical AI Assessment. medRxiv https://www.medrxiv.org/content/10.1101/2023.09.12.23295399v2 (2023). · DOI
Cited by
Sorted by citations
Large Language Models and Empathy: Systematic Review
Sorin, V., Brin, D., Barash, Y., Konen, E., Charney, A., Nadkarni, G., & Klang, E. (2024) · Journal of Medical Internet Research · doi:10.2196/52597 · Cited by 240
Next-generation agentic AI for transforming healthcare
Karunanayake, N. (2025) · Informatics and Health · doi:10.1016/j.infoh.2025.03.001 · Cited by 165
Agentic Large Language Models, a Survey
Plaat, A., Van Duijn, M., Van Stein, N., Preuss, M., Van der Putten, P., & Batenburg, K. J. (2025) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.18675 · Cited by 75
Testing AI on language comprehension tasks reveals insensitivity to underlying meaning
Dentella, V., Günther, F., Murphy, E., Marcus, G., & Leivada, E. (2024) · Scientific Reports · doi:10.1038/s41598-024-79531-8 · Cited by 54
Weisman, D., Sugarman, A., Huang, Y. M., Gelberg, L., Ganz, P. A., & Comulada, W. S. (2025) · JMIR Formative Research · doi:10.2196/65670 · Cited by 37
BioAgents: Bridging the gap in bioinformatics analysis with multi-agent systems
Mehandru, N., Hall, A. K., Melnichenko, O., Dubinina, Y., Tsirulnikov, D., Bamman, D., Alaa, A., Saponas, S., & Malladi, V. S. (2025) · Scientific Reports · doi:10.1038/s41598-025-25919-z · Cited by 19
Self-citationYao, Z., Parashar, A., Zhou, H., Jang, W. S., Ouyang, F., Yang, Z., & Yu, H. (2025) · Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2025.naacl-long.538 · Cited by 27
Large Language Model Agent in Financial Trading: A Survey
Ding, H., Li, Y., Wang, J., Chen, H., Guo, D., & Zhang, Y. (2026) · Proceedings of the 2026 9th International Conference on Computers in Management and Business · doi:10.1145/3802463.3802475 · Cited by 10
Innovating global regulatory frameworks for generative AI in medical devices is an urgent priority
Ong, J. C. L., Ning, Y., Liu, M., Ma, Y., Zhao, L., Singh, K., Chang, R. T., Vogel, S., Lim, J. C. W., Tan, I. S. K., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02552-2 · Cited by 7
MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Servers
Wang, Z., Chang, Q., Patel, H., Biju, S., Wu, C.-E., Liu, Q., Ding, A., Rezazadeh, A., Shah, A., Bao, Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.20453 · Cited by 4
Automatic Interactive Evaluation for Large Language Models with State Aware Patient Simulator
Liao, Y., Meng, Y., Wang, Y., Liu, H., Wang, Y., & Wang, Y. (2024) · arXiv · doi:10.48550/arxiv.2403.08495 · Cited by 15
Metadata
- Authors
- Mehandru, N., Miao, B. Y., Almaraz, E. R., Sushil, M., Butte, A. J., & Alaa, A.
- Affiliations
- University of California, Berkeley (US)
- University of California, San Francisco (US)
- Corresponding author
- Alaa, Ahmed
- ORCID
- Venue
- npj Digital Medicine
- Published
- 2024-04-03
- Volume
- 7
- Issue
- 1
- Pages
- 84-84
- ISSN
- 2398-6352 (ISSN-L)
- PMC
- PMC10991271
- OpenAlex
- W4393867901
- Cited by
- 143
- References
- 29
- Citations by year
- 2026: 52 · 2025: 74 · 2024: 17
- FWCI
- 13.37
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- npj Digital Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Division of Intramural Research, National Institute of Allergy and Infectious Diseases (U01AI150741)
- U.S. Department of Health & Human Services | NIH | NIH Office of the Director (OT2OD024611)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- large language models, LLM agents, clinical decision support, AI evaluation framework, clinical simulation, clinical workflow, clinical implementation of AI, benchmarking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
