The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index; 22 of the 26 sources its claims cite were checked for support.
Mitigating Cognitive Biases in Clinical Decision-Making Through Multi-Agent Conversations Using Large Language Models: Simulation Study
Open accessPublished version of a preprint
Abstract
Background: Cognitive biases in clinical decision-making significantly contribute to errors in diagnosis and suboptimal patient outcomes. Addressing these biases presents a formidable challenge in the medical field. Objective: This study aimed to explore the role of large language models (LLMs) in mitigating these biases through the use of the multi-agent framework. We simulate the clinical decision-making processes through multi-agent conversation and evaluate its efficacy in improving diagnostic accuracy compared with humans. Methods: A total of 16 published and unpublished case reports where cognitive biases have resulted in misdiagnoses were identified from the literature. In the multi-agent framework, we leveraged GPT-4 (OpenAI) to facilitate interactions among different simulated agents to replicate clinical team dynamics. Each agent was assigned a distinct role: (1) making the final diagnosis after considering the discussions, (2) acting as a devil’s advocate to correct confirmation and anchoring biases, (3) serving as a field expert in the required medical subspecialty, (4) facilitating discussions to mitigate premature closure bias, and (5) recording and summarizing findings. We tested varying combinations of these agents within the framework to determine which configuration yielded the highest rate of correct final diagnoses. Each scenario was repeated 5 times for consistency. The accuracy of the initial diagnoses and the final differential diagnoses were evaluated, and comparisons with human-generated answers were made using the Fisher exact test. Results: A total of 240 responses were evaluated (3 different multi-agent frameworks). The initial diagnosis had an accuracy of 0% (0/80). However, following multi-agent discussions, the accuracy for the top 2 differential diagnoses increased to 76% (61/80) for the best-performing multi-agent framework (Framework 4-C). This was significantly higher compared with the accuracy achieved by human evaluators (odds ratio 3.49; P=.002). Conclusions: The multi-agent framework demonstrated an ability to re-evaluate and correct misconceptions, even in scenarios with misleading initial investigations. In addition, the LLM-driven, multi-agent conversation framework shows promise in enhancing diagnostic accuracy in diagnostically challenging medical scenarios.
Overview
CoChat Summary
TL;DR
This simulation study used GPT-4 Turbo and an AutoGen multi-agent framework to re-evaluate 16 clinical case reports in which cognitive biases had caused misdiagnosis. The best-performing 4-agent framework, which explicitly discussed cognitive biases, improved final top-2 differential diagnostic accuracy to 76% (61/80), significantly better than human evaluators (odds ratio 3.49; P=.002).
Key findings
- Initial diagnoses by the first-responder agent were correct in 0% (0/80) of responses across all three multi-agent frameworks, whereas human-generated initial diagnoses were correct in 27% (13/48).
- After multi-agent discussion, final diagnosis accuracy was 64% (51/80) for Framework 3, 69% (55/80) for Framework 4, and 76% (61/80) for Framework 4-C; only Framework 4 (odds ratio 2.39; P=.03) and Framework 4-C (odds ratio 3.49; P=.002) were significantly better than human answers.
- Human final diagnoses were correct in 48% (23/48).
- Across all three frameworks, multi-agent discussions led to the correct final diagnosis in 13 of 16 scenarios (81%), and identified anchoring bias, confirmation bias, availability bias, and premature closure. In case 12, Framework 4-C reached the correct diagnosis of pneumothorax in 100% (5/5) of simulations after an initially misleading chest radiograph interpretation.
Methods
- PubMed was searched for case reports of misdiagnosis attributed to cognitive bias; 162 reports were found, 37 were eligible, and 15 published cases were selected, plus 1 unpublished personal critical care case, for 16 scenarios.
- GPT-4 Turbo was accessed via API and used within the AutoGen multi-agent conversation framework. Three configurations were tested: Framework 3 (Junior Resident I, Junior Resident II, Recorder), Framework 4 (plus Professional Expert), and Framework 4-C (plus Senior Doctor tasked with discussing cognitive biases). Each scenario was repeated 5 times.
- Diagnostic accuracy was assessed for initial and final diagnoses. Two physicians graded answers, with consensus for discrepancies. Three doctors with at least 5 years of clinical experience provided human comparator differential diagnoses. Multi-agent and human answers were compared using the Fisher exact test.
Limitations
- The paper states that reliance on published case reports limits the breadth of clinical scenarios and may affect generalizability.
- The paper states that excluding visual data such as medical imaging confines the model to text-based information, omitting a critical component of clinical diagnosis.
- The paper states that inherent LLM biases from pre-training data and technical limitations in understanding complex medical terminology and nuances may limit applicability compared with experienced clinicians. Evident from the text: the study used only 16 simulated scenarios, did not test real-time clinical outcomes, and the human comparator involved only 3 doctors.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
15 Valid · 1 Not valid · 3 Not checked · 46 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Despite concerted efforts involving educational strategies, optimal work environments, and a culture promoting bias awareness and correction [5,6], the eradication of these biases remains an elusive goal.”
§ Mitigating Cognitive Biases in Clinical Decision-Making Through Multi-Agent Conversations Using Large Language Models: Simulation Study > Introduction
What the sources say
- Partly supportedcites [5] sourceExperimental — accuracy not yet measured
The Metacognition and Clinical Decision Making section recommends education and awareness to reduce bias and describes working conditions that make analytical thinking difficult. The Conclusion likewise advocates sustained training. The paper does not establish that concerted efforts have already provided optimal work environments or a culture of bias correction; the sentence's claim about elusive eradication follows the citation rather than being clearly attributed to it. (LLM, conf=0.82)
- Not checked — no full text of the sourcecites [6] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Human cognitive biases in clinical decision-making are increasingly recognized as a crucial factor in health care errors and suboptimal patient outcomes [1].”
§ Mitigating Cognitive Biases in Clinical Decision-Making Through Multi-Agent Conversations Using Large Language Models: Simulation Study > Introduction
What the sources say
- Partly supportedcites [1] sourceExperimental — accuracy not yet measured
The Results and Discussion report associations between cognitive biases and diagnostic inaccuracies or management errors, supporting the claim about health care errors. But the Abstract Conclusions say more research is needed to determine their impact on patient outcomes, so the claim presents that link more firmly than the review does. (LLM, conf=0.86)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Cognitive biases in clinical decision-making significantly contribute to errors in diagnosis and suboptimal patient outcomes.”
- Also stated in: § Introduction · boosted
“Human cognitive biases in clinical decision-making are increasingly recognized as a crucial factor in health care errors and suboptimal patient outcomes [1].”
§ Mitigating Cognitive Biases in Clinical Decision-Making Through Multi-Agent Conversations Using Large Language Models: Simulation Study > Abstract > Background
What the sources say
- Partly supportedcites [1] sourceExperimental — accuracy not yet measured
The Results section reports associations between cognitive biases and diagnostic inaccuracies, as well as potential effects on management errors. But the claim’s stronger wording, “significantly contribute,” exceeds the review’s cautious conclusions. The Abstract Results section says only two studies evaluated patient outcomes, with one finding an association involving tolerance to ambiguity; it does not establish a general effect of cognitive biases on patient outcomes. Page numbers are not provided in our copy. (LLM, conf=0.87)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Introduction · boosted
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This was significantly higher compared with the accuracy achieved by human evaluators (odds ratio 3.49; P=.002).”
- Also stated in: § Overall Performance of Multi-Agent Conversation Framework · plain
“which was significantly better than the human answers (odds ratio 3.49; P=.002)”
§ Mitigating Cognitive Biases in Clinical Decision-Making Through Multi-Agent Conversations Using Large Language Models: Simulation Study > Abstract > Results
plainstates an associationn = 80 ·- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Overall Performance of Multi-Agent Conversation Framework · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, following multi-agent discussions, the accuracy for the top 2 differential diagnoses increased to 76% (61/80) for the best-performing multi-agent framework (Framework 4-C).”
- Also stated in: § Overall Performance of Multi-Agent Conversation Framework · plain
“After the multi-agent discussions, the final diagnosis was correct in 76% (61/80) of cases in Framework 4-C”
§ Mitigating Cognitive Biases in Clinical Decision-Making Through Multi-Agent Conversations Using Large Language Models: Simulation Study > Abstract > Results
plainstates a causen = 80 ·- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Overall Performance of Multi-Agent Conversation Framework · plain
24 more claims
Create a free account to see all 65 claims and every source verdict.
- Probably not a substantial claim. 29
- It may state two separate claims as one. 13
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“As agents negotiate strategies to achieve clinical goals, they may exhibit emergent behaviors that were neither programmed nor anticipated during validation (38).”
From Ethical issues in multi-agent AI systems for healthcare: a narrative review (2026) · cites this paper as [38]
Experimental — accuracy not yet measured
Citations
Cites
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · cited 1×
“Notably, these models have demonstrated their proficiency by successfully passing advanced medical examinations [10] and scoring clinical risk gradings on par with experienced physicians [11].”
Kung, T. H., Cheatham, M., ChatGPT, Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., et al. (2022) · doi:10.1101/2022.12.19.22283643 · Cited by 4,115
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · cited 1×
“…while LLMs have demonstrated potential as a valuable clinical aid in correcting cognitive biases, the implementation of such technology in health care necessitates rigorous ethical and regulatory oversight [52] and should continue to augment rather than replace the human clinician’s expertise [14].”
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
Scientific discovery in the age of artificial intelligence.
Wang, H., Fu, T., Du, Y., Gao, W., Huang, K., Liu, Z., Chandak, P., Liu, S., Van Katwyk, P., Deac, A., et al. (2023) · Nature · doi:10.1038/s41586-023-06221-2 · Cited by 1,823
Teamwork in healthcare: Key discoveries enabling safer, high-quality care.
Rosen, M. A., DiazGranados, D., Dietz, A. S., Benishek, L. E., Thompson, D., Pronovost, P. J., & Weaver, S. J. (2018) · American Psychologist · doi:10.1037/amp0000298 · Cited by 1,283 · cited 1×
“In the absence of established literature recommending an optimal team size for mitigating cognitive biases in medical settings, we constructed a simulation using 3 to 4 different agents, representing a typical clinical team composition [31].”
The imperative for regulatory oversight of large language models (or generative AI) in healthcare
Meskó, B., & Topol, E. J. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00873-0 · Cited by 1,047
The future landscape of large language models in medicine
Clusmann, J., Kolbinger, F. R., Muti, H. S., Carrero, Z. I., Eckardt, J.-N., Laleh, N. G., Löffler, C. M. L., Schwarzkopf, S.-C., Unger, M., Veldhuizen, G. P., et al. (2023) · Communications Medicine · doi:10.1038/s43856-023-00370-1 · Cited by 1,045 · cited 1×
“While LLMs can process and analyze vast quantities of data far beyond human capacity [18], they lack the nuanced understanding, empathy, and ethical reasoning inherent to human practitioners [19].”
Cognitive biases associated with medical decisions: a systematic review
Saposnik, G., Redelmeier, D., Ruff, C. C., & Tobler, P. N. (2016) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-016-0377-1 · Cited by 1,001 · cited 1×
“Human cognitive biases in clinical decision-making are increasingly recognized as a crucial factor in health care errors and suboptimal patient outcomes [1].”
Understanding how discrimination can affect health
Williams, D. R., Lawrence, J. A., Davis, B. A., & Vu, C. (2019) · Health Services Research · doi:10.1111/1475-6773.13222 · Cited by 853
Cited by
Enhancing diagnostic capability with multi-agents conversational large language models
Chen, X., Yi, H., You, M., Liu, W., Wang, L., Li, H., Zhang, X., Guo, Y., Fan, L., Chen, G., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01550-0 · Cited by 147
Retrieval-augmented generation for generative artificial intelligence in health care
Yang, R., Ning, Y., Keppo, E., Liu, M., Hong, C., Bitterman, D. S., Ong, J. C. L., Ting, D. S. W., & Liu, N. (2025) · npj Health Systems · doi:10.1038/s44401-024-00004-1 · Cited by 269
Self-citationLarge Language Models in Medical Diagnostics: Scoping Review With Bibliometric Analysis
Su, H., Sun, Y., Li, R., Zhang, A., Yang, Y., Xiao, F., Duan, Z., Chen, J., Hu, Q., Yang, T., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/72062 · Cited by 79
Generative AI for decision-making: A multidisciplinary perspective
Albashrawi, M. A. (2025) · Journal of Innovation & Knowledge · doi:10.1016/j.jik.2025.100751 · Cited by 103
AI Agents in Clinical Medicine: A Systematic ReviewPreprint
Gorenshtein, A., Omar, M., Glicksberg, B. S., Nadkarni, G. N., & Klang, E. (2025) · medRxiv · doi:10.1101/2025.08.22.25334232 · Cited by 94
Benchmarking the diagnostic performance of open source LLMs in 1933 Eurorad case reports
Kim, S. H., Schramm, S., Adams, L. C., Braren, R., Bressem, K. K., Keicher, M., Platzek, P., Paprottka, K. J., Zimmer, C., Hedderich, D. M., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01488-3 · Cited by 64
“In contrast, LLMs could also play a role in reducing cognitive biases if they are intentionally utilized to provide different perspectives and uncover common fallacies 23 .”
Li, R., Wang, X., Berlowitz, D. R., Mez, J., Lin, H., & Yu, H. Z. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01940-4 · Cited by 90
“Multi-agent methods have shown promise in healthcare tasks such as medical question answering 35 andmitigatingcognitivebiasesinclinicaldecision-making 36 .”
Cognitive bias in clinical large language models
Mahajan, A., Obermeyer, Z., Daneshjou, R., Lester, J., & Powell, D. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01790-0 · Cited by 77
Learning to break: Knowledge-enhanced reasoning in multi-agent debate system
Wang, H., Du, X., Yu, W., Chen, Q., Zhu, K., Chu, Z., Yan, L., & Guan, Y. (2025) · Neurocomputing · doi:10.1016/j.neucom.2024.129063 · Cited by 36
AI-induced never-skilling in medical education
Ke, Y., Jin, L., Ong, J. C. L., Thirunavukarasu, A. J., Car, J., Cheung, C. Y., Tham, Y. C., Ting, D. S. W., Ong, M. E. H., Compton, S., et al. (2026) · Nature Medicine · doi:10.1038/s41591-026-04438-y · Cited by 71
“Multi-agentic architectures simulating competing clinical perspectives can broaden the range of error types students encounter 43 .”
Self-citationYao, Z., Duan, L., Xu, S., Chi, L., & Sheng, D. (2025) · JMIR Medical Informatics · doi:10.2196/69485 · Cited by 38
“Previous studies have suggested that using multiple LLMs for cross-verification can effectively mitigate AI hallucination [23,24].”
Large language models for clinical decision support in gastroenterology and hepatology
Wiest, I. C., Bhat, M., Clusmann, J., Schneider, C. V., Jiang, X., & Kather, J. N. (2025) · Nature Reviews Gastroenterology & Hepatology · doi:10.1038/s41575-025-01108-1 · Cited by 47
“For example, in diagnostic processes, multi-agent conversations can highlight overlooked data points or challenge flawed assumptions, improving diagnostic accuracy and ensuring safer decision making 213 .”
Agent-guided AI-powered interpretation and reporting of nerve conduction studies and EMG (INSPIRE)
Gorenshtein, A., Sorka, M., Khateb, M., Aran, D., & Shelly, S. (2025) · Clinical Neurophysiology · doi:10.1016/j.clinph.2025.2110792 · Cited by 21
AI agent in healthcare: applications, evaluations, and future directions
Zhao, L., Liu, S., Xin, T., Tan, J., Wang, X., Li, Y., Bian, Z., Chen, Y., Kong, F., Bian, J., et al. (2026) · npj Artificial Intelligence · doi:10.1038/s44387-026-00076-4 · Cited by 50
A comprehensive survey of AI agents in healthcare
Xu, G., Li, X., Chen, Y., Duan, Y., Wu, S., Yu, H., Chiu, C.-H., Ni, J., Tang, N., Li, T. J.-J., et al. (2026) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2026.105045 · Cited by 55
Theodorakopoulos, L., Theodoropoulou, A., & Halkiopoulos, C. (2025) · Electronics · doi:10.3390/electronics14193930 · Cited by 50
Transforming clinical reasoning—the role of AI in supporting human cognitive limitations
Greengrass, C. J. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1715440 · Cited by 30
Vatsal, S., Dubey, H., & Singh, A. (2026) · IEEE Access · doi:10.1109/access.2026.3651218 · Cited by 31
Yang, T., Xiao, Y., Bao, Z., Hao, J., & Peng, J. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf601 · Cited by 33
Large Language Model Agent for Managing Patients With Suspected Hypertension
Wang, Y., Tan, W., Cheng, S., Peng, C., Jin, P. K., Qin, F., Tang, L., Zhu, T., Wu, B., Liu, J., et al. (2025) · Hypertension · doi:10.1161/hypertensionaha.125.25305 · Cited by 21
From visual question answering to intelligent AI agents in ophthalmology
Chen, X., Chen, R., Xu, P., Wan, X., Zhang, W., Yan, B., Shang, X., He, M., & Shi, D. (2025) · British Journal of Ophthalmology · doi:10.1136/bjo-2024-326097 · Cited by 12
Beyond Chatbots: Moving Toward Multistep Modular AI Agents in Medical Education
Chow, M., & Ng, O. (2025) · JMIR Medical Education · doi:10.2196/76661 · Cited by 15
“Yet empirical data show that single-prompt workflows still trail agentic teams in diagnostic accuracy and bias mitigation [16-18].”
Ozgun, M. C., Pei, J., Hindriks, K., Donatelli, L., Liu, Q., & Wang, J. (2025) · Proceedings of the 34th ACM International Conference on Information and Knowledge Management · doi:10.1145/3746252.3761164 · Cited by 7
Ethical issues in multi-agent AI systems for healthcare: a narrative review
Xie, Z., Wang, H., Dai, L., Wang, Z., Song, H., & Qian, J. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1792627 · Cited by 11
“As a result, even when harmful outcomes are identified, it remains unclear which agent, developer, deployer, or clinical actor should be audited, sanctioned, or held legally responsible, revealing a structural mismatch between distributed AI agency and liability-based regulatory models (38).”
Chang, Y., Liu, Y., Huang, S., & Hsu, W. (2025) · Psychiatry Research · doi:10.1016/j.psychres.2025.116501 · Cited by 8
Large Language Models in Randomized Controlled Trials Design: Observational Study
Jin, L., Ong, J. C. L., Elangovan, K., Ke, Y., Pyle, A., Ting, D. S. W., & Liu, N. (2025) · Journal of Medical Internet Research · doi:10.2196/67469 · Cited by 8
Self-citationWu, X., Huang, Y., & He, Q. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106088 · Cited by 9
Kim, S., Zhou, Y., Guo, Y., Xiao, C., & Zheng, K. (2026) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocag045 · Cited by 8
Multi-Agent Debate Strategies: Survey, Taxonomy, and Challenges
Motger, Q., Oriol, M., Marco, J., & Franch, X. (2026) · arXiv · doi:10.48550/arxiv.2607.26212 · Cited by 5
Park, Y. S., Jeon, D., Shi, S., Sheu, E. G., Tavakkoli, A., Nimeri, A. A., & Han, A. (2025) · International Journal of Surgery · doi:10.1097/js9.0000000000003631 · Cited by 8
Large language model-driven agents in nursing practice: A scoping review
Zheng, X., Zou, H., Wu, L., Dong, P., Yuan, W., & Chen, Y. (2025) · International Journal of Nursing Sciences · doi:10.1016/j.ijnss.2025.10.007 · Cited by 13
“Of the 25 articles included in the literature, ten studies [17,18,21,25, 29 – 31, 35, 38 , 40 ] adopted a collaborative architecture.”
Bousquet, C., & Beltramin, D. (2024) · Journal of Medical Internet Research · doi:10.2196/69742 · Cited by 6
“Ke et al [1] evaluated the ability of LLMs to avoid reproducing certain biases that physicians experience when making a diagnosis.”
Lu, J., Huang, J., Guo, Y. H., Wu, Q., Jiang, Z., Yang, T., Bian, J., & Bo, L. (2026) · BMC Anesthesiology · doi:10.1186/s12871-025-03605-x · Cited by 6
“Finally, tracking real-world outcomes and leveraging multi-agent conversations will be critical for mitigating biases and ensuring safe deployment [29].”
Hung, M., Ward, C., Marx, J., Strickler, I., & Jensen, A. (2026) · AI and Ethics · doi:10.1007/s43681-026-01048-9 · Cited by 4
Precision oncology: from large language models to multi-agent systems
Guo, X., Chen, J., Zhang, Y., Zhang, X., Lu, J., Qin, Q., Chen, J., Gao, X., & Wang, J. (2026) · Frontiers in Oncology · doi:10.3389/fonc.2026.1828507 · Cited by 4
“Related studies have demonstrated promise in applications including Alzheimer ’ s disease prediction and neuro-oncology decision support (13 – 15).”
Incorporating Medical Assistants in eHealth Environments Using an Agentic RAG Approach
Kalathas, D., Menychtas, A., Maglogiannis, I. G., & Tsanakas, P. D. (2025) · IFIP advances in information and communication technology · doi:10.1007/978-3-031-96235-6_12 · Cited by 4
Ke, Y. H., Leong, Y. H., Jin, L., Elangovan, K., Abdullah, H. R., Sia, A. T. H., Ong, J. C. L., Shah, N. H., Wong, T. Y., & Ting, D. S. W. (2025) · British Journal of Anaesthesia · doi:10.1016/j.bja.2025.05.044 · Cited by 5
Self-citationWho Does What? Archetypes of Roles Assigned to LLMs During Human-AI Decision-Making
Chappidi, S., Singh, J., & Krauze, A. V. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791428 · Cited by 8
Decision-Making with Deliberation: Meta-reviewing as a Document-grounded Dialogue
Purkayastha, S., Dycke, N., Lauscher, A., & Gurevych, I. (2026) · Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.eacl-long.158 · Cited by 2
Xu, L., Cong, X., Wang, R., Li, N., Liu, X., Wang, R., & Cuiping, X. (2025) · JMIR Medical Informatics · doi:10.2196/78279 · Cited by 2
“It is well-established that human cognitive biases can be mitigated to some extent through increased awareness, education, and data-driven decision-making methods [34,35].”
Fusion-Augmented Large Language Models: Boosting Diagnostic Trustworthiness via Model Consensus
Siam, K., Faruk, M. J. H., Cheng, J., & Gu, H. (2025) · IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI ...) · doi:10.1109/bhi67747.2025.11269545 · Cited by 1
“Recent studies found a significant reduction in review time, error, and bias, demonstrating real-world efficiency gains from consensus-guided triage [32].”
Grounding large language models in clinical diagnostics
Chen, X., Zhou, H., Yi, H., You, M., Liu, W., Li, W., Qin, Z., Li, H., Zhang, X., Guo, Y., et al. (2026) · Nature Communications · doi:10.1038/s41467-026-70274-w · Cited by 18
“This resultstandsincontrasttoprevious fi ndings,includingourown,which demonstrated that multi-agent collaboration enhances performance instaticdiagnosticsettings 14,29 .”
Goyal, A., Konieczny, K., Plombon, S., Rodriguez, J. A., Leeson, M., Fiskio, J. M., Yoon, C. S., Lipsitz, S. R., & Dalal, A. K. (2026) · Journal of General Internal Medicine · doi:10.1007/s11606-026-10527-y · Cited by 1
Iacobucci, G. J., Leo, R. J., & Pristach, C. A. (2025) · Personalized Medicine in Psychiatry · doi:10.1016/j.pmip.2025.100164 · Cited by 2
MedSentry: Understanding and Mitigating Safety Risks in Medical LLM Multi-Agent Systems
Chen, K., Zhen, T., Wang, H., Liu, K., Li, X., Huo, J., Yang, T., Xu, J., Dong, W., & Gao, Y. (2025) · arXiv · doi:10.48550/arxiv.2505.20824 · Cited by 32
“…as well as medical-specific models like Meditron-70b [1] and Llama-3-Meditron [2], LLM-based medical agents have demonstrated broad applicability across various healthcare domains, including drug discovery [3], hospital simulation [4], report generation [5], and clinical decision support [6, 7].”
Prossinger, H., Boschetti, S., Hladký, T., Říha, D., Steltzer, H., & Binter, J. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-12764-8_4
Harada, R., Kimura, Y., & Inoshita, K. (2025) · IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech) · doi:10.1109/cyberscitech68397.2025.00101 · Cited by 3
“Kampman et al. [11] showed that multiple agents can collaboratively assist clinical professionals through response generation and summarization.”
Trindade, A. A., Neves, J. P., & de Andrade, T. L. (2025) · doi:10.5753/eri-mt.2025.17227
“…com fontes de dados distribuídas, onde esses sistemas melhoram a integração de dados ao facilitar a sincronização semântica, resolver conflitos informacionais e eliminar redundâncias funcionais entre agentes, otimizando assim a eficiência operacional, conforme descrito por KE, Yuhe et al. (2024).”
The evolving landscape of large language models and non-large language models in health care
Yang, R., Li, H., Wong, M. Y. H., Ke, Y., Li, X., Yu, K., Liao, J., Liew, J. C. K., Nair, S. V., Ong, J. C. L., et al. (2026) · npj Health Systems · doi:10.1038/s44401-026-00076-1
“In health care, LLMs have shown extensive potential in numerous areas, including consultation, diagnosis, management, education, and research 11,12 .”
Self-citationWhat will be the next step of LLMs in TCM? A narrative review
Chen, S., Zhong, R., Zhang, W., Li, Z., Su, Y., Gao, L., & Hu, K. (2026) · Science of Traditional Chinese Medicine · doi:10.1097/st9.0000000000000109 · Cited by 1
Gao, Z., Zhang, L., He, Y., & Zhou, D. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-0369-7_9
Responsible AI for safer opioid risk management in older adults
Bakshi, S., Gopal, R., & Yaraghi, N. (2026) · npj Health Systems · doi:10.1038/s44401-026-00092-1
“When grounded in domain-speci fi c knowledge, these models can synthesize medical literature, contextualize data patterns, and generate human-readable rationales that align with clinical reasoning 41 .”
Şen, O. (2026) · Cutaneous and Ocular Toxicology · doi:10.1080/15569527.2026.2692370 · Cited by 1
An LLM-based multi-agent system for patient discharge assessment
Gebreab, S. A., Musamih, A., & Salah, K. (2026) · Computer Methods and Programs in Biomedicine Update · doi:10.1016/j.cmpbup.2026.100261
Effect of Generational Differences on the Quality of Anaesthesia Care: A Narrative Review
Balasubramanian, A., & Selvam, S. (2026) · Cureus · doi:10.7759/cureus.111525
“Encouraging collaborative discussion and participatory decision-making allows clinicians across career stages to contribute their respective strengths, fostering a culture of shared learning and supporting the delivery of high-quality anaesthesia care [24].”
Generative AI for healthcare: fundamentals, challenges, and perspectives
Chen, G., Liu, C., Anne, O. G., Tan, M., Xie, Z., Yin, J., Yip, J. W. L., Zhang, W., ZHU, J., & Ooi, B. C. (2026) · Science China Information Sciences · doi:10.1007/s11432-025-4927-2 · Cited by 10
“Similarly, Ke et al. [74] develop a multi-agent framework that assigns diverse roles to LLM-driven agents to mitigate cognitive biases in clinical decision-making, significantly improving diagnostic accuracy in complex medical scenarios.”
Takhdat, K., Fadely, A. E., Mohamed, E., Ouaamr, A., & Adib, A. R. E. (2026) · Medical Science Educator · doi:10.1007/s40670-026-02804-6 · Cited by 1
Faulty Tools or Disruptive Teammates? A New Theory of Human-AI Conflict and Compromise
Matthews, G., Cumings, R., Lin, J., Mouloua, M., Chella, A., & Pipitone, A. (2026) · Theoretical and Applied Ergonomics · doi:10.3390/tae2030014
“For example, although most trust research has not differentiated performance and relation-based trust, evidence for trust being associated with activation of a positive tool-based mental model [29,32] presumably reflects performance-based trust.”
Can AI assist in reducing diagnostic error? A narrative review
Scott, I. A. (2026) · Diagnosis · doi:10.1515/dx-2026-0021
“Multi-agent frameworks where di ff erent LLMs adopt di ff erent roles (e.g., acting as a devil ’ s advocate to expose biases, serving as a subject matter expert) substantially improve diagnosis when applied to actual cases where human cognitive biases have led to misdiagnosis [131].”
Amoei, M., Fraser, K., & Poenaru, D. (2026) · doi:10.2196/preprints.107982 · Cited by 1
Shao, M., Shao, H., Zhang, H., & Zhang, H. (2026) · doi:10.2196/preprints.108814
Gao, J., Dong, L., & Hu, J. (2026) · doi:10.2196/preprints.109941
Xu, T., Yin, Y., Zhang, X., Wei, W., Yuan, J., Wang, H., Ding, G., Xue, W., Chen, Z., Xie, S., et al. (2026) · Journal of Medical Internet Research · doi:10.2196/98118
Perez, J., Gutierrez, A., & Lopez, C. (2024) · Journal of Health Informatics and Digital Systems · doi:10.68159/u033966980
Anderson, C. (2026) · medRxiv · doi:10.64898/2026.09.01.26361960
Yu, S., Gu, X., Gao, Y., Zhao, F., & Zhang, J. (2026) · Integrated Diagnostics · doi:10.1016/j.intdia.2026.100006
Metadata
- Authors
- Ke, Y., Yang, R., Lie, S. A., Lim, T. X. Y., Ning, Y., Li, I., Abdullah, H. R., Ting, D. S. W., & Liu, N.
- Affiliations
- Singapore General Hospital (SG)
- Duke-NUS Medical School (SG)
- The University of Tokyo (JP)
- Singapore National Eye Center (SG)
- National University of Singapore (SG)
- Corresponding author
- Liu, Nan
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2024-11-19
- Volume
- 26
- Pages
- e59439
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC11615553
- OpenAlex
- W4404518047
- Semantic Scholar
- dc185e9974367145c0f37d196f2c8ffdd23da4d5
- Cited by
- 64
- References
- 52
- Citations by year
- 2026: 35 · 2025: 27 · 2024: 2
- FWCI
- 5.95
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Duke-NUS Medical School
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- cognitive biases, clinical decision-making, large language models, multi-agent systems, GPT-4, diagnostic accuracy, misdiagnosis, confirmation bias, anchoring effect, devil's advocate, differential diagnosis, AI-assisted diagnosis
- MeSH terms
- Clinical Decision-Making, Cognition, Communication, Computer Simulation, Diagnostic Errors, Humans, Language, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.