Medical reasoning in LLMs: an in-depth analysis of DeepSeek R1
The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Introduction: The integration of large language models (LLMs) into healthcare holds immense promise, but also raises critical challenges, particularly regarding the interpretability and reliability of their reasoning processes. While models like DeepSeek R1-which incorporates explicit reasoning steps-show promise in enhancing performance and explainability, their alignment with domain-specific expert reasoning remains understudied. Methods: This paper evaluates the medical reasoning capabilities of DeepSeek R1, comparing its outputs to the reasoning patterns of medical domain experts. Results: Through qualitative and quantitative analyses of 100 diverse clinical cases from the MedQA dataset, we demonstrate that DeepSeek R1 achieves 93% diagnostic accuracy and shows patterns of medical reasoning. Analysis of the seven error cases revealed several recurring errors: anchoring bias, difficulty integrating conflicting data, limited consideration of alternative diagnoses, overthinking, incomplete knowledge, and prioritizing definitive treatment over crucial intermediate steps. Discussion: These findings highlight areas for improvement in LLM reasoning for medical applications. Notably the length of reasoning was important with longer responses having a higher probability for error. The marked disparity in reasoning length suggests that extended explanations may signal uncertainty or reflect attempts to rationalize incorrect conclusions. Shorter responses (e.g., under 5,000 characters) were strongly associated with accuracy, providing a practical threshold for assessing confidence in model-generated answers. Beyond observed reasoning errors, the LLM demonstrated sound clinical judgment by systematically evaluating patient information, forming a differential diagnosis, and selecting appropriate treatment based on established guidelines, drug efficacy, resistance patterns, and patient-specific factors. This ability to integrate complex information and apply clinical knowledge highlights the potential of LLMs for supporting medical decision-making through artificial medical reasoning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Reasoning through these requires causal inference, temporal sequencing, and some skepticism-capabilities not captured by endpoint accuracy 45,46 .”
From Structured reasoning failures compromise LLM interpretation of clinical oncology notes (2026) · cites this paper as [46]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
DeepSeek-AI, Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.12948 · Cited by 18,918
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · cited 1×
“• Dual-process theory : Mirrored in the coexistence of fast, implicit completions (Type 1: zero-shot or few-shot inference) and slow, explicit reasoning traces (Type 2: deliberate chains or tree-of-thought sampling).”
Medical error—the third leading cause of death in the US
Makary, M. A., & Daniel, M. (2016) · BMJ · doi:10.1136/bmj.i2139 · Cited by 3,264
Rajpurkar, P., Chen, E., Banerjee, O., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-021-01614-0 · Cited by 3,212 · cited 1×
“…hypothesis arguing that large language modules appear to learn the same representations independent of model (Huh et al., 2024): every clinical vignette is pulled toward a stable, cross-modal “ideal” embedding representing the prototypical presentation and guideline-recommended next steps.”
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101 · cited 4×
“• Situated/distributed cognition: Corresponds to retrieval-augmented generation (RAG) (Lewis et al., 2020) and mixture-of-experts (MoE) (Li and Zhou, 2024) .”
InfluentialDiagnostic error in internal medicine
Graber, M. L., Franklin, N., & Gordon, R. (2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.13.1493 · Cited by 1,549
Benjamens, S., Dhunnoo, P., & Meskó, B. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00324-0 · Cited by 1,215 · cited 1×
“• Situated/distributed cognition: Corresponds to retrieval-augmented generation (RAG) (Lewis et al., 2020) and mixture-of-experts (MoE) (Li and Zhou, 2024) .”
Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial
Goh, E., Gallo, R., Hom, J., Strong, E., Weng, Y., Kerman, H., Cool, J. A., Kanjee, Z., Parsons, A. S., Ahuja, N., et al. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.40969 · Cited by 724
Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., & Szolovits, P. (2021) · Applied Sciences · doi:10.3390/app11146421 · Cited by 718
Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., & Szolovits, P. (2021) · doi:10.20944/preprints202105.0498.v1 · Cited by 718
HealthBench: Evaluating Large Language Models Towards Improved Human Health
Arora, R. K., Wei, J., Hicks, R. S., Bowman, P., Quiñonero-Candela, J., Tsimpourlas, F., Sharman, M., Shah, M., Vallone, A., Beutel, A., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.08775 · Cited by 654
Capabilities of GPT-4 on Medical Challenge Problems
Nori, H., King, N., McKinney, S. M., Carignan, D., & Horvitz, E. (2023) · arXiv · doi:10.48550/arxiv.2303.13375 · Cited by 509
Reasoning with large language models for medical question answering
Lucas, M. M., Yang, J., Pomeroy, J. K., & Yang, C. C. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae131 · Cited by 66
Influence of a Large Language Model on Diagnostic Reasoning: A Randomized Clinical Vignette Study
Goh, E., Gallo, R., Hom, J., Strong, E., Weng, Y., Kerman, H., Cool, J., Kanjee, Z., Parsons, A. S., Ahuja, N., et al. (2024) · doi:10.1101/2024.03.12.24303785 · Cited by 38
The Platonic Representation Hypothesis
Huh, M., Cheung, B., Wang, T., & Isola, P. (2024) · arXiv · doi:10.48550/arxiv.2405.07987 · Cited by 35 · cited 1×
“…Maps onto the Platonic representation hypothesis arguing that large language modules appear to learn the same representations independent of model (Huh et al., 2024): every clinical vignette is pulled toward a stable, cross-modal “ideal” embedding representing the prototypical presentation and…”
Croxford, E., Gao, Y., First, E., Pellegrino, N., Schnier, M., Caskey, J., Oguss, M., Wills, G., Chen, G., Dligach, D., et al. (2025) · doi:10.1101/2025.04.22.25326219 · Cited by 25
MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning
Li, S., Balachandran, V., Feng, S., Ilgen, J., Pierson, E., Koh, P., & Tsvetkov, Y. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-0908 · Cited by 19
Research works printed but not linked: 39
- Barbosa Chaves, 2022, The use of deliberate reflection to reduce confirmation bias among orthopedic surgery residents, Sci. Med, 32, 42216, 10.15448/1980-6108.2022.1.42216 · DOI
- Borg, 2024, Enhancing clinical reasoning skills for medical students: a qualitative comparison of LLM-powered social robotic versus computer-based virtual patients within rheumatology, Rheumatol. Int, 44, 3041, 10.1007/s00296-024-05731-0 · DOI · Paper page
- Boushehri, 2015, Clinical reasoning assessment through medical expertise theories: past, present and future directions, Med. J. Islamic Repub. Iran, 29, 222
- Charlin, 2000, Scripts and medical diagnostic knowledge: theory and applications for clinical reasoning instruction and research, Acad. Med, 75, 182, 10.1097/00001888-200002000-00020 · DOI · Paper page
- Crescitelli, 2019, Opening the horizons of clinical reasoning to qualitative research, Acta Biomed, 90, 8
- Custers, 2013, Medical education and cognitive continuum theory: an alternative perspective on medical problem solving and clinical reasoning, Acad. Med, 88, 1074, 10.1097/ACM.0b013e31829a3b10 · DOI · Paper page
- Durning, 2011, Situativity theory: a perspective on how participants and the environment can interact: Amee guide no. 52, Med. Teach, 33, 188, 10.3109/0142159X.2011.550965 · DOI · Paper page
- Durning, 2024, Teaching clinical reasoning: principles from the literature to help improve instruction from the classroom to the bedside, Korean J. Med. Educ, 36, 145, 10.3946/kjme.2024.292 · DOI
- Ferreira, 2010, Clinical reasoning in the real world is mediated by bounded rationality: implications for diagnostic clinical practice guidelines, PLoS ONE, 5, e10265, 10.1371/journal.pone.0010265 · DOI
- Gee, 2017, Using theory to interpret how senior clinicians define, learn, and teach clinical reasoning, MedEdPublish, 6, 182, 10.15694/mep.2017.000182 · DOI
- Gold, 2022, Clinical reasoning education in the clerkship years: a cross-disciplinary national needs assessment, PLoS ONE, 17, e0273250, 10.1371/journal.pone.0273250 · DOI
- Jay, 2024, Clinical reasoning—the essentials for teaching medical students, trainees and non-medical healthcare professionals, Br. J. Hosp. Med, 85, 1, 10.12968/hmed.2024.0052 · DOI
- Kim, 2018, Understanding uncertainty in medicine: concepts and implications in medical education, Korean J. Med. Educ, 30, 181, 10.3946/kjme.2018.92 · DOI
- Lai, 2025, Med-r1: reinforcement learning for generalizable medical reasoning in vision-language models, arXiv, 10.48550/arXiv.2503.13939 · DOI
- Li, 2024, Your mixture-of-experts llm is secretly an embedding model for free, arXiv, 10.48550/arXiv.2410.10814 · DOI
- Lubarsky, 2015, Using script theory to cultivate illness script formation and clinical reasoning in health professions education, Can. Med. Educ. J, 6, e61, 10.36834/cmej.36631 · DOI · Paper page
- Maharana, 2025, Right prediction, wrong reasoning: uncovering llm misalignment in ra disease diagnosis, arXiv, 10.48550/arXiv.2504.06581 · DOI
- Mettarikanon, 2024, Analysis of patient information and differential diagnosis with clinical reasoning in pre-clinical medical students, Int. Med. Educ, 3, 23, 10.3390/ime3010003 · DOI
- Moroz, 2017, Clinical reasoning workshop: cervical spine and shoulder disorders, MedEdPORTAL, 13, 10560, 10.15766/mep_2374-8265.10560 · DOI
- Mukhopadhyay, 2024, Clinical reasoning skills among second-phase medical students in west bengal, india: an exploratory study, Cureus, 16, e68839, 10.7759/cureus.68839 · doi:10.7759/cureus.68839
- Nierenberg, 2020, Using the chief complaint driven medical history: theoretical background and practical steps for student clinicians, MedEdPublish, 9, 17, 10.15694/mep.2020.000017.1 · DOI
- Olupeliyawa, 2017, Clinical reasoning: implications and strategies for postgraduate medical education, J. Postgrad. Inst. Med, 4, 59, 10.4038/jpgim.8174 · doi:10.4038/jpgim.8174
- Pelaccia, 2011, An analysis of clinical reasoning through a recent and comprehensive approach: the dual-process theory, Med. Educ. Online, 16, 5890, 10.3402/meo.v16i0.5890 · DOI · Paper page
- Radović, 2022, An ontology-driven learning assessment using the script concordance test, Appl. Sci, 12, 1472, 10.3390/app12031472 · DOI
- Schiff, 2013, Primary care closed claims experience of Massachusetts malpractice insurers, JAMA Inter. Med, 173, 2063, 10.1001/jamainternmed.2013.11070 · DOI · Paper page
- Shimizu, 2012, Pivot and cluster strategy: a preventive measure against diagnostic errors, Int. J. Gen. Med, 5, 917, 10.2147/IJGM.S38805 · DOI
- Shimozono, 2020, A cognitive bias in diagnostic reasoning and its remediation by the “2-dimensional approach”, MedEdPublish, 9, 123, 10.15694/mep.2020.000123.1 · DOI
- Shin, 2019, Reasoning processes in clinical reasoning: from the perspective of cognitive psychology, Korean J. Med. Educ, 31, 299, 10.3946/kjme.2019.140 · DOI · Paper page
- Sudacka, 2023, Clinical reasoning: what do nurses, physicians, and students reason about, J. Interprof. Care, 37, 990, 10.1080/13561820.2023.2208605 · DOI
- Torre, 2020, Widening the lens on teaching and assessing clinical reasoning: from “in the head” to “out in the world”, Diagnosis, 7, 181, 10.1515/dx-2019-0098 · DOI
- Wang, 2024, JMLR: joint medical llm and retrieval training for enhancing reasoning and professional question answering capability, arXiv, 10.48550/arXiv.2402.17887 · DOI
- Wang, 2024, 10.18653/v1/2024.findings-emnlp.95 · DOI · Paper page
- Weinstein, 2017, Diagnosing and remediating clinical reasoning difficulties: a faculty development workshop, MedEdPORTAL, 13, 10650, 10.15766/mep_2374-8265.10650 · DOI
- Welch, 2017, Grounded theory - a lens to understanding clinical reasoning, MedEdPublish, 6, 2, 10.15694/mep.2017.000002 · DOI
- 2023, Health workforce snapshot
- Wu, 2025, Medreason: eliciting factual medical reasoning steps in llms via knowledge graphs, arXiv, 10.48550/arXiv.2504.00993 · DOI
- Yazdani, 2019, Five decades of research and theorization on clinical reasoning: a critical review, Adv. Med. Educ. Pract, 10, 703, 10.2147/AMEP.S213492 · DOI
- Yu, 2025, Finemedlm-o1: enhancing the medical reasoning ability of llm from supervised fine-tuning to test-time training, arXiv, 10.48550/arXiv.2501.09213 · DOI
- Zwaan, 2010, Patient record review of the incidence, consequences, and causes of diagnostic adverse events, Arch. Intern. Med, 170, 1015, 10.1001/archinternmed.2010.146 · DOI · Paper page
Cited by
Xu, X., & Sankar, R. (2025) · Information · doi:10.3390/info16100894 · Cited by 20
“First, advances in foundation models such as DeepSeek R1 [95] offer improved capabilities in long-context comprehension, instruction following, and nuanced medical reasoning.”
Structured reasoning failures compromise LLM interpretation of clinical oncology notes
Kenaston, M. W., Ayub, U., Parmar, M., Kumar, P., Anjum, M. U., Naqvi, S. A. A., Chaudhuri, A. A., Zakharia, Y., Heath, E. I., Bekaii-Saab, T. S., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02951-5 · Cited by 2
Metadata
- Authors
- Moëll, B., Sand Aronsson, F., & Akbar, S.
- Venue
- Frontiers in Artificial Intelligence
- Publisher
- Frontiers Media SA
- Published
- 2025-06-18
- Volume
- 8
- ISSN
- 2624-8212
- Cited by
- 49
- References
- 19
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.