CoChat
Sign Up
Cited by 20Explainable Artificial Intelligence (XAI)Artificial Intelligence in Healthcare and EducationMachine Learning in Healthcare

Explainable differential diagnosis with dual-inference large language models

Open access
Zhou, Shuang, Lin, Mingquan, Ding, Sirui, Wang, Jiashuo, Chen, Canyu, Melton, Genevieve B., Zou, James, Zhang, Rui·2025·npj Health Systems
DOI 10.1038/s44401-025-00015-6PMC PMC12021655
Open this paper in CoChat — free accountOpen PDF

Abstract

Automatic differential diagnosis (DDx) involves identifying potential conditions that could explain a patient’s symptoms and its accurate interpretation is of substantial significance. While large language models (LLMs) have demonstrated remarkable diagnostic accuracy, their capability to generate high-quality DDx explanations remains underexplored, largely due to the absence of specialized evaluation datasets and the inherent challenges of complex reasoning in LLMs. Therefore, building a tailored dataset and developing novel methods to elicit LLMs for generating precise DDx explanations are worth exploring. We developed the first publicly available DDx dataset, comprising expert-derived explanations for 570 clinical notes, to evaluate DDx explanations. Meanwhile, we proposed a novel framework, Dual-Inf, that could effectively harness LLMs to generate high-quality DDx explanations. To the best of our knowledge, it is the first study to tailor LLMs for DDx explanation and comprehensively evaluate their explainability. Overall, our study bridges a critical gap in DDx explanation, enhancing clinical decision-making.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s44401-025-00015-6

Cites

11 of 11 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Lexical overlap metrics were employed in studies with available gold-standard rationales.

    “These were employed in studies with available gold-standard rationales [65,69,72,98,124]”

    From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [65]

    Methods > Performance evaluation (page unavailable) says the study compared predicted interpretations with reference text and used METEOR, which measures unigram precision and recall. The Discussion (page unavailable) says experts manually generated the ground-truth explanations. But the paper describes its other text-comparison metrics as semantic measures, not lexical-overlap metrics, so it does not fully support the plural characterization. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Semantic similarity metrics such as BERTScore and domain-specific variants using ClinicalBERT or SciBERT embeddings assess meaning preservation beyond surface forms.

    “Semantic similarity metrics such as BERTScore [132] and domainspecific variants using ClinicalBERT [133] or SciBERT embeddings assess meaning preservation beyond surface forms [65,67,70,90,111].”

    From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [65]

    Methods > Performance evaluation (page unavailable) says BERTScore assesses semantic similarity between reference and generated text rather than relying solely on string matching. It does not describe domain-specific variants using ClinicalBERT or SciBERT embeddings; searches for those terms found no supporting passage. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Structured decoding approaches enforce interpretable reasoning pathways by construction, rather than hoping that prompted models will faithfully externalize their computations.

    “Structured decoding approaches [65,96,101,115,122] enforce interpretable reasoning pathways by construction, rather than hoping that prompted models will faithfully externalize their computations.”

    From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [65]

    The Methods > Model development section describes a defined forward-inference, backward-verification, and examination pathway for producing and checking explanations. But Methods > Implementation details says Dual-Inf incorporates chain-of-thought prompting into its LLM modules; the paper does not describe structured decoding or establish that its explanations faithfully externalize the models’ computations. Page numbers are unavailable in our copy. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Others expose intermediate reasoning steps such as agentic traces or structured decoding chains that allow clinicians to audit the logic.

    “Others expose intermediate reasoning steps such as agentic traces, structured decoding chains etc that allow clinicians to audit the logic , as in agentic systems [66,77,88,102,118] and structureddecoding approaches [65,101].”

    From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [65]

    The Methods (Model development; page not available) describe a multi-step diagnostic process that generates interpretations, and the Results (Interpretation performance; page not available) say clinicians evaluated those interpretations. Our copy does not say that the system exposes intermediate agentic traces or structured decoding chains for clinicians to audit; its examination of intermediate outputs is performed by another LLM. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Studies utilizing explanations for model improvement employed diverse techniques: rationale-guided training, iterative self-refinement, knowledge distillation, and feature engineering.

    “Studies utilizing explanations for model improvement employed diverse techniques: rationale-guided training where explanation losses improve learned representations [98,123,124]; iterative self-refinement through agent dialogue or bidirectional verification [65,83,109]; knowledge distillation from large to compact models via structured rationales [110]; and feature engineering where LLM-generated”

    From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [65]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 33

  1. [1]Adler-Milstein, J., Chen, J. H. & Dhaliwal, G. Next-generation artificial intelligence for diagnosis: from predicting diagnostic labels to “wayfinding. Jama 326, 2467–2468 (2021). 34882190 PMC12049696 · DOI
  2. [2]Fansi Tchango, A. et al. Towards trustworthy automatic diagnosis systems by emulating doctors’ reasoning with deep reinforcement learning. Adv. Neural Inf. Process. Syst. 35, 24502–24515 (2022).
  3. [3]Wu, L. et al. Differential diagnosis of secondary hypertension based on deep learning. Artif. Intell. Med. 141, 13 (2023). 37295898 · DOI
  4. [4]Levy, J., Álvarez, D., Del Campo, F. & Behar, J. A. Deep learning for obstructive sleep apnea diagnosis based on single channel oximetry. Nat. Commun. 14, 4881 (2023). 37573327 PMC10423260 · DOI
  5. [5]Zhang, Z. et al. Pathologist-level interpretable whole-slide cancer diagnosis with deep learning. Nat. Mach. Intell. 1, 236–245 (2019). · DOI · Paper page
  6. [7]Wu, S. et al. Benchmarking open-source large language models, GPT-4 and Claude 2 on multiple-choice questions in nephrology. NEJM AI 1, AIdbp2300092 (2024). · DOI
  7. [10]Tang, L. et al. Evaluating large language models on medical evidence summarization. NPJ Dig. Med. 6, 158 (2023). PMC10449915 37620423 · DOI · Paper page
  8. [11]Benary, M. et al. Leveraging Large Language Models for Decision Support in Personalized Oncology. JAMA Netw. Open 6, e2343689–e2343689 (2023). 37976064 PMC10656647 · DOI · Paper page
  9. [12]Kanjee, Z, Crowe, B & Rodman, A Accuracy of a generative artificial intelligence model in a complex diagnostic challenge. Jama 330, 78–80 (2023). 37318797 PMC10273128 · DOI · Paper page
  10. [13]Kwon, T. et al. Large language models are clinical reasoners: Reasoning-aware diagnosis framework with prompt-generated rationales. in Proceedings of the AAAI Conference on Artificial Intelligence. 18417–18425 (2024).
  11. [14]Hua, R. et al. Lingdan: enhancing encoding of traditional Chinese medicine knowledge for clinical reasoning tasks with large language models. J. Am. Med. Informat. Asso. 31, 2019–2029 (2024). PMC11339528 39038795 · DOI
  12. [16]Zhou, S. et al. Large Language Models for Disease Diagnosis: A Scoping Review. ArXiv, vol. abs/2409.00097, (2024). PMC12216946 40607112 · arXiv
  13. [17]Tu, T. et al. Towards conversational diagnostic AI. arXiv preprint arXiv:2401.05654, 2024. · arXiv
  14. [18]McDuff, D. et al. (2023, November 01, 2023). Towards Accurate Differential Diagnosis with Large Language Models. arXiv:2312.00164. Available: https://ui.adsabs.harvard.edu/abs/2023arXiv231200164M. · arXiv
  15. [20]Wei, J. et al. Chain-of-thought prompting elicits reasoning in large language models. Adv. neural Inf. Process. Syst. 35, 24824–24837 (2022).
  16. [21]Tchango, A. F., Goel, R., Wen, Z., Martel, J. & Ghosn, J. DDXPlus: A New Dataset For Automatic Medical Diagnosis. in Neural Information Processing Systems, (2022).
  17. [22]Wu, J. et al. Clinical Text Datasets for Medical Artificial Intelligence and Large Language Models — A Systematic Review. NEJM AI 1, AIra2400012 (2024). · DOI · Paper page
  18. [23]Ashwani, S. et al. Cause and effect: Can large language models truly understand causality? in Proceedings of the AAAI Symposium Series, 2–9 (2024).
  19. [24]Chi, H. et al. Unveiling Causal Reasoning in Large Language Models: Reality or Mirage? in The Thirty-eighth Annual Conference on Neural Information Processing Systems.
  20. [25]M. Parmar, et al. “LogicBench: Towards systematic evaluation of logical reasoning ability of large language models. in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024, 13679-13707.
  21. [28]Williams, C. Y., Miao, B. Y., Kornblith, A. E. & Butte, A. J. Evaluating the use of large language models to provide clinical recommendations in the Emergency Department. Nat. Commun. 15, 8236 (2024). 39379357 PMC11461960 · DOI · Paper page
  22. [29]Macherla, S., Luo, M., Parmar, M. & Baral, C. MDDial: A Multi-turn Differential Diagnosis Dialogue Dataset with Reliability Evaluation. arXiv preprint arXiv:2308.08147, (2023) · arXiv
  23. [30]Singhal, K. et al. Large language models encode clinical knowledge. Nature 620, 172–180 (2023). 37438534 PMC10396962 · DOI · Paper page
  24. [31]Chen, X. et al. FFA-GPT: an automated pipeline for fundus fluorescein angiography interpretation and question-answer. npj Digital Med. 7, 111 (2024). PMC11068733 38702471 · DOI · Paper page
  25. [33]Le, T., Bhushan, V., Sheikh-Ali, M. & Shahin F. A. First Aid for the USMLE Step 2 CS, Third Edition: McGraw-Hill Education, (2009).
  26. [34]Le, T. & Bechis, S.K. First Aid Q&A for the USMLE Step 1, Second Edition: McGraw-Hill Education, (2009).
  27. [36]Jiang, W. et al. Forward-backward reasoning in large language models for mathematical verification. in Findings of the Association for Computational Linguistics ACL 2024, 6647–6661 (2024).
  28. [38]Wang, X. et al. Self-Consistency Improves Chain of Thought Reasoning in Language Models. in The Eleventh International Conference on Learning Representations.
  29. [39]Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q. & Artzi, Y. BERTScore: Evaluating Text Generation with BERT. ArXiv, vol. abs/1904.09675, 2019. · arXiv
  30. [40]Reimers, N. & Gurevych, I. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Conference on Empirical Methods in Natural Language Processing, (2019).
  31. [41]Banerjee, S. & Lavie, A. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. in Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, 65–72 (2005).
  32. [43]Celikyilmaz, A., Clark, E. & Gao, J. Evaluation of Text Generation: A Survey. ArXiv, vol. abs/2006.14799, (2020). · arXiv
  33. [44]JDevlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. in North American Chapter of the Association for Computational Linguistics, (2019).

Cited by

Sorted by citations

Authors
Zhou, S., Lin, M., Ding, S., Wang, J., Chen, C., Melton, G. B., Zou, J., & Zhang, R.
Affiliations
  • University of Minnesota (US)
  • University of California, San Francisco (US)
  • University of Chicago (US)
  • Illinois Institute of Technology (US)
  • Stanford University (US)
Corresponding author
Zhang, Rui
ORCID
Venue
npj Health Systems
Publisher
Springer Science and Business Media LLC
Published
2025-04-24
Volume
2
Issue
1
Pages
12-12
ISSN
3005-1959 (ISSN-L)
DOI
10.1038/s44401-025-00015-6
PMC
PMC12021655
OpenAlex
W4409767213
Cited by
20
References
11
Citations by year
2026: 15 · 2025: 5
FWCI
8.02
Type
journal article
Language
en
Open access
Licence
Locations
  • npj Health Systems (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion, cc-by)
  • Knowledge@UChicago (University of Chicago) (submittedVersion)
  • University of Chicago (cc-by)
Funders
  • National Center for Complementary and Integrative Health (R01AT009457)
  • National Institute on Aging (R01AG078154)
  • National Cancer Institute (R01CA287413)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
differential diagnosis, large language models, explainable AI, clinical notes, clinical decision support, LLM evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review