The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index; 22 of the 26 sources its claims cite were checked for support.
Mitigating Cognitive Biases in Clinical Decision-Making Through Multi-Agent Conversations Using Large Language Models: Simulation Study
Open accessPublished version of a preprint
Abstract
Background: Cognitive biases in clinical decision-making significantly contribute to errors in diagnosis and suboptimal patient outcomes. Addressing these biases presents a formidable challenge in the medical field. Objective: This study aimed to explore the role of large language models (LLMs) in mitigating these biases through the use of the multi-agent framework. We simulate the clinical decision-making processes through multi-agent conversation and evaluate its efficacy in improving diagnostic accuracy compared with humans. Methods: A total of 16 published and unpublished case reports where cognitive biases have resulted in misdiagnoses were identified from the literature. In the multi-agent framework, we leveraged GPT-4 (OpenAI) to facilitate interactions among different simulated agents to replicate clinical team dynamics. Each agent was assigned a distinct role: (1) making the final diagnosis after considering the discussions, (2) acting as a devil’s advocate to correct confirmation and anchoring biases, (3) serving as a field expert in the required medical subspecialty, (4) facilitating discussions to mitigate premature closure bias, and (5) recording and summarizing findings. We tested varying combinations of these agents within the framework to determine which configuration yielded the highest rate of correct final diagnoses. Each scenario was repeated 5 times for consistency. The accuracy of the initial diagnoses and the final differential diagnoses were evaluated, and comparisons with human-generated answers were made using the Fisher exact test. Results: A total of 240 responses were evaluated (3 different multi-agent frameworks). The initial diagnosis had an accuracy of 0% (0/80). However, following multi-agent discussions, the accuracy for the top 2 differential diagnoses increased to 76% (61/80) for the best-performing multi-agent framework (Framework 4-C). This was significantly higher compared with the accuracy achieved by human evaluators (odds ratio 3.49; P=.002). Conclusions: The multi-agent framework demonstrated an ability to re-evaluate and correct misconceptions, even in scenarios with misleading initial investigations. In addition, the LLM-driven, multi-agent conversation framework shows promise in enhancing diagnostic accuracy in diagnostically challenging medical scenarios.
Overview
CoChat Summary
TL;DR
This simulation study used GPT-4 Turbo and an AutoGen multi-agent framework to re-evaluate 16 clinical case reports in which cognitive biases had caused misdiagnosis. The best-performing 4-agent framework, which explicitly discussed cognitive biases, improved final top-2 differential diagnostic accuracy to 76% (61/80), significantly better than human evaluators (odds ratio 3.49; P=.002).
Key findings
- Initial diagnoses by the first-responder agent were correct in 0% (0/80) of responses across all three multi-agent frameworks, whereas human-generated initial diagnoses were correct in 27% (13/48).
- After multi-agent discussion, final diagnosis accuracy was 64% (51/80) for Framework 3, 69% (55/80) for Framework 4, and 76% (61/80) for Framework 4-C; only Framework 4 (odds ratio 2.39; P=.03) and Framework 4-C (odds ratio 3.49; P=.002) were significantly better than human answers.
- Human final diagnoses were correct in 48% (23/48).
- Across all three frameworks, multi-agent discussions led to the correct final diagnosis in 13 of 16 scenarios (81%), and identified anchoring bias, confirmation bias, availability bias, and premature closure. In case 12, Framework 4-C reached the correct diagnosis of pneumothorax in 100% (5/5) of simulations after an initially misleading chest radiograph interpretation.
Methods
- PubMed was searched for case reports of misdiagnosis attributed to cognitive bias; 162 reports were found, 37 were eligible, and 15 published cases were selected, plus 1 unpublished personal critical care case, for 16 scenarios.
- GPT-4 Turbo was accessed via API and used within the AutoGen multi-agent conversation framework. Three configurations were tested: Framework 3 (Junior Resident I, Junior Resident II, Recorder), Framework 4 (plus Professional Expert), and Framework 4-C (plus Senior Doctor tasked with discussing cognitive biases). Each scenario was repeated 5 times.
- Diagnostic accuracy was assessed for initial and final diagnoses. Two physicians graded answers, with consensus for discrepancies. Three doctors with at least 5 years of clinical experience provided human comparator differential diagnoses. Multi-agent and human answers were compared using the Fisher exact test.
Limitations
- The paper states that reliance on published case reports limits the breadth of clinical scenarios and may affect generalizability.
- The paper states that excluding visual data such as medical imaging confines the model to text-based information, omitting a critical component of clinical diagnosis.
- The paper states that inherent LLM biases from pre-training data and technical limitations in understanding complex medical terminology and nuances may limit applicability compared with experienced clinicians. Evident from the text: the study used only 16 simulated scenarios, did not test real-time clinical outcomes, and the human comparator involved only 3 doctors.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
15 Valid · 1 Not valid · 3 Not checked · 46 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Despite concerted efforts involving educational strategies, optimal work environments, and a culture promoting bias awareness and correction [5,6], the eradication of these biases remains an elusive goal.”
§ Mitigating Cognitive Biases in Clinical Decision-Making Through Multi-Agent Conversations Using Large Language Models: Simulation Study > Introduction
What the sources say
- Partly supportedcites [5] sourceExperimental — accuracy not yet measured
The Metacognition and Clinical Decision Making section recommends education and awareness to reduce bias and describes working conditions that make analytical thinking difficult. The Conclusion likewise advocates sustained training. The paper does not establish that concerted efforts have already provided optimal work environments or a culture of bias correction; the sentence's claim about elusive eradication follows the citation rather than being clearly attributed to it. (LLM, conf=0.82)
- Not checked — no full text of the sourcecites [6] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Human cognitive biases in clinical decision-making are increasingly recognized as a crucial factor in health care errors and suboptimal patient outcomes [1].”
§ Mitigating Cognitive Biases in Clinical Decision-Making Through Multi-Agent Conversations Using Large Language Models: Simulation Study > Introduction
What the sources say
- Partly supportedcites [1] sourceExperimental — accuracy not yet measured
The Results and Discussion report associations between cognitive biases and diagnostic inaccuracies or management errors, supporting the claim about health care errors. But the Abstract Conclusions say more research is needed to determine their impact on patient outcomes, so the claim presents that link more firmly than the review does. (LLM, conf=0.86)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Cognitive biases in clinical decision-making significantly contribute to errors in diagnosis and suboptimal patient outcomes.”
- Also stated in: § Introduction · boosted
“Human cognitive biases in clinical decision-making are increasingly recognized as a crucial factor in health care errors and suboptimal patient outcomes [1].”
§ Mitigating Cognitive Biases in Clinical Decision-Making Through Multi-Agent Conversations Using Large Language Models: Simulation Study > Abstract > Background
What the sources say
- Partly supportedcites [1] sourceExperimental — accuracy not yet measured
The Results section reports associations between cognitive biases and diagnostic inaccuracies, as well as potential effects on management errors. But the claim’s stronger wording, “significantly contribute,” exceeds the review’s cautious conclusions. The Abstract Results section says only two studies evaluated patient outcomes, with one finding an association involving tolerance to ambiguity; it does not establish a general effect of cognitive biases on patient outcomes. Page numbers are not provided in our copy. (LLM, conf=0.87)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Introduction · boosted
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This was significantly higher compared with the accuracy achieved by human evaluators (odds ratio 3.49; P=.002).”
- Also stated in: § Overall Performance of Multi-Agent Conversation Framework · plain
“which was significantly better than the human answers (odds ratio 3.49; P=.002)”
§ Mitigating Cognitive Biases in Clinical Decision-Making Through Multi-Agent Conversations Using Large Language Models: Simulation Study > Abstract > Results
plainstates an associationn = 80 ·- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Overall Performance of Multi-Agent Conversation Framework · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, following multi-agent discussions, the accuracy for the top 2 differential diagnoses increased to 76% (61/80) for the best-performing multi-agent framework (Framework 4-C).”
- Also stated in: § Overall Performance of Multi-Agent Conversation Framework · plain
“After the multi-agent discussions, the final diagnosis was correct in 76% (61/80) of cases in Framework 4-C”
§ Mitigating Cognitive Biases in Clinical Decision-Making Through Multi-Agent Conversations Using Large Language Models: Simulation Study > Abstract > Results
plainstates a causen = 80 ·- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Overall Performance of Multi-Agent Conversation Framework · plain
24 more claims
Create a free account to see all 65 claims and every source verdict.
- Probably not a substantial claim. 29
- It may state two separate claims as one. 13
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“As agents negotiate strategies to achieve clinical goals, they may exhibit emergent behaviors that were neither programmed nor anticipated during validation (38).”
From Ethical issues in multi-agent AI systems for healthcare: a narrative review (2026) · cites this paper as [38]
Experimental — accuracy not yet measured
Citations
Cites
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · cited 1×
“Notably, these models have demonstrated their proficiency by successfully passing advanced medical examinations [10] and scoring clinical risk gradings on par with experienced physicians [11].”
Kung, T. H., Cheatham, M., ChatGPT, Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., et al. (2022) · doi:10.1101/2022.12.19.22283643 · Cited by 4,115
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · cited 1×
“…while LLMs have demonstrated potential as a valuable clinical aid in correcting cognitive biases, the implementation of such technology in health care necessitates rigorous ethical and regulatory oversight [52] and should continue to augment rather than replace the human clinician’s expertise [14].”
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
Scientific discovery in the age of artificial intelligence.
Wang, H., Fu, T., Du, Y., Gao, W., Huang, K., Liu, Z., Chandak, P., Liu, S., Van Katwyk, P., Deac, A., et al. (2023) · Nature · doi:10.1038/s41586-023-06221-2 · Cited by 1,823
Teamwork in healthcare: Key discoveries enabling safer, high-quality care.
Rosen, M. A., DiazGranados, D., Dietz, A. S., Benishek, L. E., Thompson, D., Pronovost, P. J., & Weaver, S. J. (2018) · American Psychologist · doi:10.1037/amp0000298 · Cited by 1,283 · cited 1×
“In the absence of established literature recommending an optimal team size for mitigating cognitive biases in medical settings, we constructed a simulation using 3 to 4 different agents, representing a typical clinical team composition [31].”
The imperative for regulatory oversight of large language models (or generative AI) in healthcare
Meskó, B., & Topol, E. J. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00873-0 · Cited by 1,047
The future landscape of large language models in medicine
Clusmann, J., Kolbinger, F. R., Muti, H. S., Carrero, Z. I., Eckardt, J.-N., Laleh, N. G., Löffler, C. M. L., Schwarzkopf, S.-C., Unger, M., Veldhuizen, G. P., et al. (2023) · Communications Medicine · doi:10.1038/s43856-023-00370-1 · Cited by 1,045 · cited 1×
“While LLMs can process and analyze vast quantities of data far beyond human capacity [18], they lack the nuanced understanding, empathy, and ethical reasoning inherent to human practitioners [19].”
Cognitive biases associated with medical decisions: a systematic review
Saposnik, G., Redelmeier, D., Ruff, C. C., & Tobler, P. N. (2016) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-016-0377-1 · Cited by 1,001 · cited 1×
“Human cognitive biases in clinical decision-making are increasingly recognized as a crucial factor in health care errors and suboptimal patient outcomes [1].”
Understanding how discrimination can affect health
Williams, D. R., Lawrence, J. A., Davis, B. A., & Vu, C. (2019) · Health Services Research · doi:10.1111/1475-6773.13222 · Cited by 853
Interventions for increasing the use of shared decision making by healthcare professionals
Légaré, F., Adekpedjou, R., Stacey, D., Turcotte, S., Kryworuchko, J., Graham, I. D., Lyddiatt, A., Politi, M. C., Thomson, R., Elwyn, G., et al. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006732.pub4 · Cited by 698 · cited 1×
“This facilitates learning in a controlled, educational environment [25,26].”
Role play with large language models
Shanahan, M., McDonell, K., & Reynolds, L. (2023) · Nature · doi:10.1038/s41586-023-06647-8 · Cited by 491 · cited 1×
“The multi-agent framework features dialogue agents with near-human performance and could introduce an innovative paradigm in health care [22-24].”
Role-Play with Large Language Models
Shanahan, M., McDonell, K., & Reynolds, L. (2023) · arXiv · doi:10.48550/arxiv.2305.16367 · Cited by 490
Harrer, S. (2023) · eBioMedicine · doi:10.1016/j.ebiom.2023.104512 · Cited by 480
Eliminating Explicit and Implicit Biases in Health Care: Evidence and Research Needs
Vela, M. B., Erondu, A. I., Smith, N. A., Peek, M. E., Woodruff, J. N., & Chin, M. H. (2022) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-052620-103528 · Cited by 426 · cited 1×
“This facilitates learning in a controlled, educational environment [25,26].”
Large language models in health care: Development, applications, and challenges
Yang, R., Tan, T. F., Lu, W., Thirunavukarasu, A. J., Ting, D. S. W., & Liu, N. (2023) · Health Care Science · doi:10.1002/hcs2.61 · Cited by 366 · cited 1×
“The integration of artificial intelligence (AI), and in particular, large language models (LLMs), into clinical medicine is on the horizon [7].”
Self-citationEffects of Size and Task Type on Group Performance and Member Reactions
Hackman, J. R., & Vidmar, N. (1970) · Sociometry · doi:10.2307/2786271 · Cited by 251 · cited 1×
“The suggested optimal group size to facilitate group discussion and performance has been proposed to be between 3 and 5 [30].”
Critical Thinking: The Development of an Essential Skill for Nursing Students
Papathanasiou, I., Kleisiaris, C., Fradelos, E., Kakou, K., & Kourkouta, L. (2014) · Acta Informatica Medica · doi:10.5455/aim.2014.22.283-286 · Cited by 236 · cited 1×
“The reflective process fostered by engaging with LLMs in diagnosing and revising cases not only cultivates an educational atmosphere conducive to developing critical thinking skills but also suggests practical applications in clinical settings [50].”
The anchoring bias reflects rational use of cognitive resources
Lieder, F., Griffiths, T. L., M. Huys, Q. J., & Goodman, N. D. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1286-8 · Cited by 230 · cited 1×
“This is in line with previous research emphasizing the importance of collaborative decision-making in health care to mitigate individual cognitive biases [47].”
Embracing Large Language Models for Medical Applications: Opportunities and Challenges
Karabacak, M., & Margetis, K. (2023) · Cureus · doi:10.7759/cureus.39305 · Cited by 204
AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation
Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.08155 · Cited by 193 · cited 2×
“The multi-agent framework features dialogue agents with near-human performance and could introduce an innovative paradigm in health care [22-24].”
Berthet, V. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.802439 · Cited by 191
Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine
Nori, H., Lee, Y. T., Zhang, S., Carignan, D., Edgar, R., Fusi, N., King, N., Larson, J., Li, Y., Liu, W., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.16452 · Cited by 178 · cited 1×
“LLMs have advanced text generation capability and extensive domain-specific knowledge [8,9].”
Cognitive bias and how to improve sustainable decision making
Korteling, J. E. (., Paradies, G. L., & Sassen-van Meer, J. P. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1129835 · Cited by 143
Consequences of attributing discrimination to implicit vs. explicit bias
Daumeyer, N. M., Onyeador, I. N., Brown, X., & Richeson, J. A. (2019) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2019.04.010 · Cited by 135
Self-Adaptive Large Language Model (LLM)-Based Multiagent Systems
do Nascimento, N. M., Alencar, P., & Cowan, D. D. (2023) · IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C) · doi:10.1109/acsos-c58168.2023.00048 · Cited by 128 · cited 1×
“Simulations of such discussions through LLM agents could provide a new solution to increase the accuracy of diagnosis [21].”
Interactive Natural Language ProcessingPreprint
Wang, Z., Zhang, G., Yang, K., Shi, N., Zhou, W., Hao, S., Xiong, G., Li, Y., Sim, M. Y., Chen, X., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.13246 · Cited by 112
ChatGPT and Generating a Differential Diagnosis Early in an Emergency Department Presentation
Berg, H. T., van Bakel, B., van de Wouw, L., Jie, K. E., Schipper, A., Jansen, H., O’Connor, R. D., van Ginneken, B., & Kurstjens, S. (2024) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2023.08.003 · Cited by 97 · cited 1×
“The accuracy of ChatGPT making the correct emergency medicine diagnosis is still limited to 77% to 83% [17].”
Environmentally sustainable development and use of artificial intelligence in health care
Richie, C. (2022) · Bioethics · doi:10.1111/bioe.13018 · Cited by 92 · cited 1×
“This strategy aligns with the current trajectory of AI development in health care, where the focus is on integrating and maximizing existing AI technologies to enhance clinical decision-making and focus on sustainable AI [48,49].”
A translational perspective towards clinical AI fairness
Liu, M., Ning, Y., Teixayavong, S., Mertens, M., Xu, J., Ting, D. S. W., Cheng, L. T.-E., Ong, J. C. L., Teo, Z. L., Tan, T. F., et al. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00918-4 · Cited by 82 · cited 1×
“This strategy aligns with the current trajectory of AI development in health care, where the focus is on integrating and maximizing existing AI technologies to enhance clinical decision-making and focus on sustainable AI [48,49].”
Self-citationBelieving in Overcoming Cognitive Biases
(2020) · AMA Journal of Ethics · doi:10.1001/amajethics.2020.773 · Cited by 80
Beldhuis, I. E., Marapin, R. S., Jiang, Y. Y., Simões de Souza, N. F., Georgiou, A., Kaufmann, T., Castela Forte, J., & van der Horst, I. C. C. (2021) · Journal of Critical Care · doi:10.1016/j.jcrc.2021.04.012 · Cited by 58 · cited 1×
“These biases stem from innate psychological tendencies and can potentially lead to misjudgments and suboptimal outcomes in patient care [2-4].”
Breaking Bias: The Role of Artificial Intelligence in Improving Clinical Decision-Making
Brown, C., Nazeer, R., Gibbs, A., Le Page, P., & Mitchell, A. R. (2023) · Cureus · doi:10.7759/cureus.36415 · Cited by 55 · cited 3×
“…Urinary tract infection 2022 [42] Superior vena cava syndrome Anaphylaxis secondary to henna 2022 [43] Pneumothorax Pulmonary embolism 2023 [33] Thiamine deficiency Diabetic ketoacidosis with infections 2023 [44] Ischemic bowel Endometritis 2023 [45] Acute myocarditis caused by invasive…”
InfluentialRecognizing and reducing cognitive bias in clinical and forensic neurology
Satya-Murti, S., & Lockhart, J. (2015) · Neurology Clinical Practice · doi:10.1212/cpj.0000000000000181 · Cited by 42 · cited 1×
“Human cognitive biases can be mitigated through a combination of awareness, education, and structured approaches [20].”
Understanding and addressing cognitive bias in medical education
Hershberger, P. J., Markert, R. J., Part, H. M., Cohen, S. M., & Finger, W. W. (1996) · Advances in Health Sciences Education · doi:10.1007/bf00162919 · Cited by 34 · cited 1×
“Despite concerted efforts involving educational strategies, optimal work environments, and a culture promoting bias awareness and correction [5,6], the eradication of these biases remains an elusive goal.”
Lim, D. Y. Z., Ke, Y. H., Sng, G. G. R., Tung, J. Y. M., Chai, J. X., & Abdullah, H. R. (2023) · British Journal of Anaesthesia · doi:10.1016/j.bja.2023.06.052 · Cited by 24
Self-citationKe, Y., Yang, R., & Liu, N. (2024) · Journal of Medical Internet Research · doi:10.2196/48330 · Cited by 22 · cited 1×
“While LLMs can process and analyze vast quantities of data far beyond human capacity [18], they lack the nuanced understanding, empathy, and ethical reasoning inherent to human practitioners [19].”
Self-citationYang, R., Zeng, Q., You, K., Qiao, Y., Huang, L., Hsieh, C.-C., Rosand, B., Goldwasser, J., Dave, A., Keenan, T., et al. (2024) · Journal of Medical Internet Research · doi:10.2196/60601 · Cited by 16 · cited 1×
“Meanwhile, the technical limitations inherent in LLMs, including their understanding of complex medical terminologies and nuances [51], may not match the expertise of experienced clinicians, possibly limiting the scope of applicability.”
Self-citationCognitive biases and knowledge deficits leading to delayed recognition of cryptococcal meningitis
Deming, M. E., Mark, A., Nyemba, V., Heil, E. L., Palmeiro, R., & Schmalzle, S. A. (2019) · IDCases · doi:10.1016/j.idcr.2019.e00588 · Cited by 15 · cited 1×
“…of inappropriate secretion of antidiuretic hormone 2018 [35] Cryptococcal meningitis Headache and neck pain—migraines and muscle strain 2019 [36] Left common and external iliac artery occlusion Complex regional pain syndrome flare 2020 [37] Atypical ectopic pregnancy Pelvic inflammatory…”
An elderly woman with ‘heart failure’: Cognitive biases and diagnostic error
Mull, N., Reilly, J. B., & Myers, J. S. (2015) · Cleveland Clinic Journal of Medicine · doi:10.3949/ccjm.82a.14087 · Cited by 9 · cited 2×
“Miliary tuberculosis Heart failure 2015 [32] Heart failure secondary to dilated cardiomyopathy Bronchial asthma triggered by bacterial pneumonia 2017 [34] Nonfunctioning macropituitary adenoma causing adrenal insufficiency Syndrome of inappropriate secretion of antidiuretic hormone 2018 [35]…”
Availability Bias and the COVID-19 Pandemic: A Case Study of Legionella Pneumonia
Kyere, K., Aremu, T. O., & Ajibola, O. A. (2022) · Cureus · doi:10.7759/cureus.25846 · Cited by 6 · cited 1×
“…2020 [37] Atypical ectopic pregnancy Pelvic inflammatory disease 2022 [38] Bacterial pneumonia (legionella pneumoniae) COVID-19 pneumoniae 2022 [39] Vesicointestinal fistula due to Crohn disease Urinary tract infection with complicated pyelonephritis 2022 [40] Syphilitic gumma and…”
Retrieval-Augmented Generation for Generative Artificial Intelligence in MedicinePreprint
Yang, R., Ning, Y., Keppo, E., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.12449 · Cited by 6 · cited 1×
“The use of AI in medical settings is not just a technological issue; it intersects with complex ethical, legal, and medical considerations [11-16].”
Self-citationKamegai, K., Yokoyama, S., Takakura, S., Takayama, Y., Shiiki, S., Koyama, H., & Narita, M. (2022) · Medicine · doi:10.1097/md.0000000000030733 · Cited by 5 · cited 1×
“…Crohn disease Urinary tract infection with complicated pyelonephritis 2022 [40] Syphilitic gumma and osteomyelitis Bone (sternum) tuberculosis 2022 [41] Vertebral osteomyelitis and bilateral psoas and retroperitoneal abscesses Urinary tract infection 2022 [42] Superior vena cava syndrome…”
Cognitive Bias in the Management of a Critically Ill 29-Year-Old Patient
Vittorelli, J., Cacchillo, J., McCool, M., & McCague, A. (2023) · Cureus · doi:10.7759/cureus.39314 · Cited by 5 · cited 1×
“…[43] Pneumothorax Pulmonary embolism 2023 [33] Thiamine deficiency Diabetic ketoacidosis with infections 2023 [44] Ischemic bowel Endometritis 2023 [45] Acute myocarditis caused by invasive bacterial infection Acute myocarditis likely due to MIS-C a 2023 [46] Malignancy Congestive cardiac failure…”
KG-Rank: Enhancing Large Language Models for Medical QA with Knowledge Graphs and Ranking Techniques
Yang, R., Liu, H., Marrese-Taylor, E., Zeng, Q., Ke, Y. H., Li, W., Cheng, L., Chen, Q., Caverlee, J., Matsuo, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.05881 · Cited by 5 · cited 1×
“LLMs have advanced text generation capability and extensive domain-specific knowledge [8,9].”
Self-citationCommon and external iliac artery occlusion in Behçet’s disease: a case of anchoring bias
Khawaja, H. R., & Font, C. M. (2020) · BMJ Case Reports · doi:10.1136/bcr-2020-236554 · Cited by 4 · cited 1×
“…and neck pain—migraines and muscle strain 2019 [36] Left common and external iliac artery occlusion Complex regional pain syndrome flare 2020 [37] Atypical ectopic pregnancy Pelvic inflammatory disease 2022 [38] Bacterial pneumonia (legionella pneumoniae) COVID-19 pneumoniae 2022 [39]…”
What Causes Diagnostic Errors? Referred Patients and Our Own Cognitive Biases: A Case Report
Miyagami, T., Nakayama, I., & Naito, T. (2022) · American Journal of Case Reports · doi:10.12659/ajcr.935163 · Cited by 4 · cited 1×
“…COVID-19 pneumoniae 2022 [39] Vesicointestinal fistula due to Crohn disease Urinary tract infection with complicated pyelonephritis 2022 [40] Syphilitic gumma and osteomyelitis Bone (sternum) tuberculosis 2022 [41] Vertebral osteomyelitis and bilateral psoas and retroperitoneal…”
Chehayeb, R. J., Ilagan-Ying, Y. C., & Sankey, C. (2023) · Journal of General Internal Medicine · doi:10.1007/s11606-023-08091-w · Cited by 4 · cited 2×
“…Anaphylaxis secondary to henna 2022 [43] Pneumothorax Pulmonary embolism 2023 [33] Thiamine deficiency Diabetic ketoacidosis with infections 2023 [44] Ischemic bowel Endometritis 2023 [45] Acute myocarditis caused by invasive bacterial infection Acute myocarditis likely due to MIS-C a 2023 [46]…”
Not Like the Textbook: An Atypical Case of Ectopic Pregnancy
Birch, E. M., Molina, M. T., & Oliver, J. J. (2022) · Cureus · doi:10.7759/cureus.29881 · Cited by 3 · cited 1×
“…and external iliac artery occlusion Complex regional pain syndrome flare 2020 [37] Atypical ectopic pregnancy Pelvic inflammatory disease 2022 [38] Bacterial pneumonia (legionella pneumoniae) COVID-19 pneumoniae 2022 [39] Vesicointestinal fistula due to Crohn disease Urinary tract infection…”
Heart failure in dilated cardiomyopathy mimicking asthma triggered by pneumonia
Tetsuhara, K., Tsuji, S., Nakano, K., et al. (2017) · BMJ Case Reports · doi:10.1136/bcr-2017-222082 · Cited by 2
Missed Opportunities for Diagnosing Vertebral Osteomyelitis Caused by Influential Cognitive Biases
Kawahigashi, T., Harada, Y., Watari, T., Harada, T., Miyagami, T., Shikino, K., & Inada, H. (2022) · American Journal of Case Reports · doi:10.12659/ajcr.936058 · Cited by 2 · cited 1×
“…Bone (sternum) tuberculosis 2022 [41] Vertebral osteomyelitis and bilateral psoas and retroperitoneal abscesses Urinary tract infection 2022 [42] Superior vena cava syndrome Anaphylaxis secondary to henna 2022 [43] Pneumothorax Pulmonary embolism 2023 [33] Thiamine deficiency Diabetic…”
Severe Bacterial Infection Initially Misdiagnosed as MIS-C: Caution Needed
Stanzelova, A., Debray, A., Allali, S., Belhadjer, Z., Taha, M., Cohen, J. F., & Toubiana, J. (2023) · The Pediatric Infectious Disease Journal · doi:10.1097/inf.0000000000003896 · Cited by 2 · cited 1×
“…with infections 2023 [44] Ischemic bowel Endometritis 2023 [45] Acute myocarditis caused by invasive bacterial infection Acute myocarditis likely due to MIS-C a 2023 [46] Malignancy Congestive cardiac failure — — b a MIS-C: multisystem inflammatory syndrome in children. b Not applicable.”
Adenoma mimicking hyponatremia of SIAD
Ilaiwy, A., Thompson, N. E., & Waheed, A. A. (2018) · BMJ Case Reports · doi:10.1136/bcr-2018-226458 · Cited by 1 · cited 1×
“…2017 [34] Nonfunctioning macropituitary adenoma causing adrenal insufficiency Syndrome of inappropriate secretion of antidiuretic hormone 2018 [35] Cryptococcal meningitis Headache and neck pain—migraines and muscle strain 2019 [36] Left common and external iliac artery occlusion Complex…”
Salama, M. E., Ukwade, P., Khan, A. R., et al. (2022) · Cureus · doi:10.7759/cureus.29678 · Cited by 1 · cited 1×
“…and bilateral psoas and retroperitoneal abscesses Urinary tract infection 2022 [42] Superior vena cava syndrome Anaphylaxis secondary to henna 2022 [43] Pneumothorax Pulmonary embolism 2023 [33] Thiamine deficiency Diabetic ketoacidosis with infections 2023 [44] Ischemic bowel Endometritis 2023…”
Cited by
Enhancing diagnostic capability with multi-agents conversational large language models
Chen, X., Yi, H., You, M., Liu, W., Wang, L., Li, H., Zhang, X., Guo, Y., Fan, L., Chen, G., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01550-0 · Cited by 147
Retrieval-augmented generation for generative artificial intelligence in health care
Yang, R., Ning, Y., Keppo, E., Liu, M., Hong, C., Bitterman, D. S., Ong, J. C. L., Ting, D. S. W., & Liu, N. (2025) · npj Health Systems · doi:10.1038/s44401-024-00004-1 · Cited by 269
Self-citationLarge Language Models in Medical Diagnostics: Scoping Review With Bibliometric Analysis
Su, H., Sun, Y., Li, R., Zhang, A., Yang, Y., Xiao, F., Duan, Z., Chen, J., Hu, Q., Yang, T., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/72062 · Cited by 79
Generative AI for decision-making: A multidisciplinary perspective
Albashrawi, M. A. (2025) · Journal of Innovation & Knowledge · doi:10.1016/j.jik.2025.100751 · Cited by 103
AI Agents in Clinical Medicine: A Systematic ReviewPreprint
Gorenshtein, A., Omar, M., Glicksberg, B. S., Nadkarni, G. N., & Klang, E. (2025) · medRxiv · doi:10.1101/2025.08.22.25334232 · Cited by 94
Metadata
- Authors
- Ke, Y., Yang, R., Lie, S. A., Lim, T. X. Y., Ning, Y., Li, I., Abdullah, H. R., Ting, D. S. W., & Liu, N.
- Affiliations
- Singapore General Hospital (SG)
- Duke-NUS Medical School (SG)
- The University of Tokyo (JP)
- Singapore National Eye Center (SG)
- National University of Singapore (SG)
- Corresponding author
- Liu, Nan
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2024-11-19
- Volume
- 26
- Pages
- e59439
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC11615553
- OpenAlex
- W4404518047
- Semantic Scholar
- dc185e9974367145c0f37d196f2c8ffdd23da4d5
- Cited by
- 64
- References
- 52
- Citations by year
- 2026: 35 · 2025: 27 · 2024: 2
- FWCI
- 5.95
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Duke-NUS Medical School
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- cognitive biases, clinical decision-making, large language models, multi-agent systems, GPT-4, diagnostic accuracy, misdiagnosis, confirmation bias, anchoring effect, devil's advocate, differential diagnosis, AI-assisted diagnosis
- MeSH terms
- Clinical Decision-Making, Cognition, Communication, Computer Simulation, Diagnostic Errors, Humans, Language, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.