CoChat
Sign Up
Cited by 30Artificial Intelligence in Healthcare and EducationMachine Learning in HealthcareArtificial Intelligence in Healthcare

Automating expert-level medical reasoning evaluation of large language models

Open access
Zhou, Shuang, Xie, Wenya, Li, Jiaxi, Zhan, Zaifu, Song, Meijia, Yang, Han, Espinoza, Cheyenna, Welton, Lindsay +11 more·2025·npj Digital Medicine
DOI 10.1038/s41746-025-02208-7PMC PMC12796170
Open this paper in CoChat — free accountOpen PDF

Abstract

As large language models (LLMs) become increasingly integrated into clinical decision-making, ensuring trustworthy reasoning is paramount. However, current evaluation strategies of LLMs’ medical reasoning capability either suffer from unsatisfactory assessment or poor scalability, and a rigorous benchmark remains absent. To address this, we present MedThink-Bench, a benchmark designed for rigorous and scalable assessment of LLMs’ medical reasoning. MedThink-Bench comprises 500 high-complexity questions spanning ten medical domains, accompanied by expert-authored, step-by-step rationales that elucidate intermediate reasoning processes. Further, we introduce LLM-w-Rationale, an evaluation framework that combines fine-grained rationale assessment with an LLM-as-a-Judge paradigm, enabling expert-level fidelity in evaluating reasoning quality while preserving scalability. Results show that LLM-w-Rationale correlates strongly with expert evaluation (Pearson coefficient up to 0.87) while requiring only 1.4% of the evaluation time. Overall, MedThink-Bench establishes a rigorous and scalable standard for evaluating medical reasoning in LLMs, advancing their safe and responsible deployment in clinical practice.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41746-025-02208-7

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Open-source alternatives now match or exceed proprietary models in reasoning quality.

    “they now match or exceed proprietary models in reasoning quality [82]”

    From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [82]

    The Discussion reports that some open-source models outperformed commercial models on MedThink-Bench’s medical-reasoning evaluation. The claim is broader than that finding: the paper does not establish that open-source alternatives generally match or exceed proprietary models across reasoning tasks. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Open-source models demonstrate competitive performance in reasoning quality.

    “The demonstrated competitive performance of open-source models in reasoning quality [82]”

    From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [82]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 55 · Cited by 30

Research works printed but not linked: 49

  1. [1]Zhou, S. et al. Large language models for disease diagnosis: a scoping review. npj Artif. Intell. 1, 9 (2025). 40607112 PMC12216946 · DOI
  2. [3]Truhn, D. et al. A pilot study on the efficacy of GPT-4 in providing orthopedic treatment recommendations from MRI reports. Sci. Rep. 13, 20159 (2023). 37978240 PMC10656559 · DOI · Paper page
  3. [4]Goh, E. et al. GPT-4 assistance for improvement of physician performance on patient care tasks: a randomized controlled trial. Nat. Med. 31, 1233–1238 (2025). 39910272 PMC12380382 · DOI
  4. [7]Li, J. et al. Fact or guesswork? Evaluating large language model’s medical knowledge with structured one-hop judgment. Preprint at arXiv10.48550/arXiv.2502.14275 (2025).
  5. [8]Zhou, S. et al. Uncertainty-aware large language models for explainable disease diagnosis. npj Digit. Med. 8, 690 (2025). 41254208 PMC12627459 · DOI · Paper page
  6. [12]Liu, W. et al. Mitigating hallucination through theory-consistent Symmetric Multimodal Preference Optimization. Preprint at arXiv10.48550/arXiv.2506.11712 (2025).
  7. [13]Mündler, N., He, J., Jenko, S. & Vechev, M. T. Self-contradictory hallucinations of large language models: evaluation, detection and mitigation. In Proc. International Conference on Learning Representations (ICLR, 2024).
  8. [14]Haltaufderheide, J. & Ranisch, R. The ethics of ChatGPT in medicine and healthcare: a systematic review on Large Language Models (LLMs). npj Digit. Med. 7, 183 (2024). 38977771 PMC11231310 · DOI · Paper page
  9. [16]Kim, H. et al. Small language models learn enhanced reasoning skills from medical textbooks. npj Digit. Med. 8, 240 (2025). 40316765 PMC12048634 · DOI
  10. [17]Wang, W. et al. Medical reasoning in the era of LLMs: a systematic review of enhancement techniques and applications. Preprint at arXiv10.48550/arXiv.2508.00669 (2025).
  11. [18]Peng, Q. et al. Aligning clinical needs and AI capabilities: a survey on LLMs for medical reasoning. Preprint at techrxiv 10.36227/techrxiv.175790907.73315176/v1 (2025). · DOI
  12. [19]Zhu, Y. et al. DiagnosisArena: benchmarking diagnostic reasoning for large language models. Preprint at arXiv10.48550/arXiv.2505.14107 (2025).
  13. [20]Tang, X. et al. MedAgentsBench: benchmarking thinking models and agent frameworks for complex medical reasoning. Preprint at arXiv10.48550/arXiv.2503.07459 (2025).
  14. [21]Zuo, Y. et al. MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding. In Proc.Forty-Second International Conference on Machine Learning, International Conference on Machine Learning (ICML, 2025).
  15. [22]Singhal, K. et al. Large language models encode clinical knowledge. Nature 620, 172–180 (2023). 37438534 PMC10396962 · DOI · Paper page
  16. [23]Yang, H. et al. Large language model Synergy for ensemble learning in Medical Question Answering: design and evaluation study. J. Med. Internet Res. 27, e70080 (2025). 40658884 PMC12337233 · DOI
  17. [24]Jin, Q. et al. Hidden flaws behind expert-level accuracy of multimodal GPT-4 vision in medicine. npj Digit. Med. 7, 190 (2024). 39043988 PMC11266508 · DOI · Paper page
  18. [26]Zhou, S. et al. Explainable differential diagnosis with dual-inference large language models. npj Health Syst. 2, 12 (2025). 40291514 PMC12021655 · DOI · Paper page
  19. [27]Kim, Y. et al. MedExQA: Medical Question Answering Benchmark with Multiple Explanations. In Proc.23rd Workshop on Biomedical Natural Language Processing, 167–181 (Association for Computational Linguistics, 2024).
  20. [28]Brodeur, P. G. et al. Superhuman performance of a large language model on the reasoning tasks of a physician. Preprint at arXiv10.48550/arXiv.2412.10849 (2024).
  21. [30]Li, D. et al. ExplainCPE: A Free-text Explanation Benchmark of Chinese Pharmacist Examination. Findings of the Association for Computational Linguistics: EMNLP 2023, 1922–40 (Association for Computational Linguistics, 2023).
  22. [31]Qiu, P. et al. Quantifying the reasoning abilities of LLMs on clinical cases. Nat. Commun. 16, 9799 (2025). 41198657 PMC12592457 · DOI · Paper page
  23. [32]Papineni, K., Roukos, S., Ward, T. & Zhu, W.-J. BLEU. In Proc. 40th Annual Meeting on Association for Computational Linguistics - ACL ’02. 10.3115/1073083.1073135 (Association for Computational Linguistics, 2001). · DOI · Paper page
  24. [33]Lin, C.-Y. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out. 74–81 (Association for Computational Linguistics, Barcelona, Spain, 2004).
  25. [34]Zhang, T. et al. BERTScore: Evaluating Text Generation with BERT. International Conference on Learning Representations (2020).
  26. [35]Liu, L. et al. Towards automatic evaluation for LLMs’ clinical capabilities: Metric, data, and algorithm. In Proc. 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining Vol. 614, 5466–5475 (ACM, 2024).
  27. [36]Croxford, E. et al. Current and future state of evaluation of large language models for medical summarization tasks. npj Health Syst. 2, 6 (2025). 40124388 PMC11928168 · DOI
  28. [37]Wu, K. et al. MedCaseReasoning: evaluating and learning diagnostic reasoning from clinical case reports. Preprint at arXiv10.48550/arXiv.2505.11733 (2025).
  29. [38]Ding, C. et al. Building a human-verified clinical reasoning dataset via a human LLM hybrid pipeline for trustworthy medical AI. Preprint at arXiv10.48550/arXiv.2505.06912 (2025).
  30. [39]Chen, X. et al. FFA-GPT: an automated pipeline for fundus fluorescein angiography interpretation and question-answer. npj Digit. Med. 7, 111 (2024). 38702471 PMC11068733 · DOI · Paper page
  31. [43]Zhang, H. et al. HuatuoGPT, Towards Taming Language Model to Be a Doctor. Findings of the Association for Computational Linguistics: EMNLP 2023, 10859–10885 (Association for Computational Linguistics, 2023).
  32. [45]Wei, J. et al. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Proc.36th International Conference on Neural Information Processing Systems (Curran Associates Inc., 2022).
  33. [46]Zhou, S. et al. Mitigating ethical issues for large language models in oncology: a systematic review. JCO Clin. Cancer Inform. 9, e2500076 (2025). 40991877 · DOI
  34. [47]Gu, J. et al. A survey on LLM-as-a-Judge. arXiv10.48550/arXiv.2411.15594 (2024).
  35. [48]Chen, H., Fang, Z., Singla, Y. & Dredze, M. Benchmarking large language models on answering and explaining challenging medical questions. In Proc. 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies(Volume 1: Long Papers) (eds Chiruzzo, L., Ritter, A. & Wang, L.) 3563–3599 (Association for Computational Linguistics, 2025).
  36. [50]Kim, Y., Wu, J., Abdulle, Y. & Wu, H. MedExQA: medical question answering benchmark with multiple explanations. In Proc. 23rd Workshop on Biomedical Natural Language Processing (eds Demner-Fushman, D., Ananiadou, S., Miwa, M., Roberts, K. & Tsujii, J.) 167–181 (Association for Computational Linguistics, 2024).
  37. [51]Phan, L. et al. Humanity’s last exam. Preprint at arXiv10.48550/arXiv.2501.14249 (2025).
  38. [53]Jin, Q., Dhingra, B., Liu, Z., Cohen, W. & Lu, X. PubMedQA: A Dataset for Biomedical Research Question Answering. In Proc. 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (eds Inui, K., Jiang, J., Ng, V. & Wan, X.) 2567–2577 (Association for Computational Linguistics, 2019).
  39. [55]Hendrycks, Dan, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring Massive Multitask Language Understanding. In Proc. International Conference on Learning Representations (ICLR, 2021).
  40. [56]Vilares, D. & Gómez-Rodríguez, C. HEAD-QA: a healthcare dataset for complex reasoning. In Proc. 57th Annual Meeting of the Association for Computational Linguistics (eds Korhonen, A., Traum, D. & Màrquez, L.) 960–966 (Association for Computational Linguistics, 2019).
  41. [57]Banerjee, S. & Alon, L. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proc.ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, 65–72 (Association for Computational Linguistics, 2005).
  42. [58]Sellam, T. et al. BLEURT: Learning Robust Metrics for Text Generation. In Proc. 58th Annual Meeting of the Association for Computational Linguistics, 7881-7892 (Association for Computational Linguistics, 2020).
  43. [59]Wei, H. et al. Systematic evaluation of LLM-as-a-judge in LLM alignment tasks: explainable metrics and diverse prompt templates. Preprint at arXiv10.48550/arXiv.2408.13006 (2024).
  44. [60]Open AI et al. GPT-4o System Card. Preprint at arXiv10.48550/arXiv.2408.13006 (2024).
  45. [62]Comanici, G. et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. 207, 06261 (2025).
  46. [63]Wang, B. et al. Baichuan-M1: pushing the medical capability of large language models. Preprint at arXiv10.48550/arXiv.2502.12671 (2025).
  47. [65]Christophe, C., Kanithi, P. K., Raha, T., Khan, S. & Pimentel, M. A. F. Med42-v2: A suite of clinical LLMs. Preprint at arXiv10.48550/arXiv.2408.06142 (2024).
  48. [66]Grattafiori, A. et al. The Llama 3 herd of models. Preprint at arXiv10.48550/arXiv.2407.21783 (2024).
  49. [69]Kruskal, W. H. & Wallis, W. A. Use of ranks in one-criterion variance analysis. J. Am. Stat. Assoc. 47, 583 (1952). · DOI · Paper page

Authors
Zhou, S., Xie, W., Li, J., Zhan, Z., Song, M., Yang, H., Espinoza, C., Welton, L., Mai, X., Jin, Y., Xu, Z., Chung, Y.-H., Xing, Y., Tsai, M.-H., Schaffer, E., Shi, Y., Liu, N., Liu, Z., & Zhang, R.
Affiliations
  • University of Minnesota (US)
  • University of Georgia (US)
  • University of Virginia (US)
  • Columbia University (US)
  • University of California, San Francisco (US)
  • University of Colorado Anschutz (US)
Corresponding author
Zhang, Rui
ORCID
Venue
npj Digital Medicine
Publisher
Springer Science and Business Media LLC
Published
2025-12-06
Volume
9
Issue
1
Pages
34-34
ISSN
2398-6352 (ISSN-L)
DOI
10.1038/s41746-025-02208-7
PMC
PMC12796170
OpenAlex
W4417075517
Cited by
30
References
55
Citations by year
2026: 29 · 2025: 1
FWCI
7.22
Type
journal article
Language
en
Open access
Licence
Locations
  • npj Digital Medicine (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University)
Funders
  • National Institutes of Health’s National Center (R01AT009457)
  • National Institute on Aging (R01AG078154)
  • National Cancer Institute (R01CA287413)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
large language models, medical reasoning, LLM evaluation, benchmark, LLM-as-a-judge, expert evaluation, clinical decision support

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review