CoChat
Sign Up
Cited by 27Artificial Intelligence in Healthcare and EducationArtificial Intelligence in HealthcareMachine Learning in Healthcare

Uncertainty-aware large language models for explainable disease diagnosis

Open access
Zhou, Shuang, Wang, Jiashuo, Xu, Zidu, Wang, Song, Brauer, David, Welton, Lindsay, Cogan, Jacob, Chung, Yuen-Hei +6 more·2025·npj Digital Medicine
DOI 10.1038/s41746-025-02071-6PMC PMC12627459
Open this paper in CoChat — free accountOpen PDF

Abstract

Explainable disease diagnosis, which leverages patient information (e.g., symptoms) and computational models to generate probable diagnoses and reasoning, holds strong clinical promise. Yet, when clinical notes lack sufficient evidence for a definitive diagnosis, such as the absence of definitive symptoms, diagnostic uncertainty commonly arises, increasing the risk of misdiagnosis. Despite its importance, the explicit identification and explanation of diagnostic uncertainty remain under-explored in artificial intelligence-driven systems. To fill this gap, we introduce ConfiDx, an uncertainty-aware large language model fine-tuned with diagnostic criteria. We formalized the task of uncertainty-aware diagnosis and curated richly annotated datasets that reflect varying degrees of diagnostic ambiguity. Evaluating on real-world datasets demonstrated that ConfiDx excelled in identifying diagnostic uncertainties, achieving superior diagnostic performance, and generating trustworthy explanations for diagnoses and uncertainties. Moreover, ConfiDx-assisted experts outperformed standalone experts by 10.7% in uncertainty recognition and 26% in uncertainty explanation, underscoring its substantial potential to improve clinical decision-making.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41746-025-02071-6

Cites

17 of 17 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Fifteen studies used only open-source models.

    “fifteen used only open-source models [67,72,73,96,98,113,129]”

    From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [67]

    The Methods (ConfiDx) say the study trained and evaluated four open-source models, but Methods (Comparison with other large-scale LLMs) also report testing commercial models, including GPT-4o, OpenAI-o1, Gemini-2.0, and Claude-3.7. This paper therefore did not use only open-source models. The source paper does not establish the citing document's count of fifteen studies. (LLM, conf=0.87)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • Zhou et al. [67] demonstrated that ConfiDx-assisted experts significantly outperform standalone experts in recognizing diagnostic uncertainty.

    “Zhou et al. [67] demonstrated that ConfiDx-assisted experts significantly outperform standalone experts in recognizing diagnostic uncertainty”

    From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [67]

    The Results section (AI-augmented clinician assessments; page unavailable) reports that ConfiDx-assisted experts improved uncertainty recognition by 10.7% compared with experts working independently. It does not report a statistical significance test for that comparison, so the claim of significant outperformance is not fully supported. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Rationale-supervised training represents a middle ground where models learn to generate explanations through supervised examples, potentially inheriting the reasoning patterns of expert annotators.

    “Rationale-supervised training [67,69,90,98,110,123,124] represents a middle ground where models learn to generate explanations through supervised examples, potentially inheriting the reasoning patterns of expert annotators.”

    From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [67]

    The Methods (ConfiDx; Data annotation and split) describe supervised fine-tuning on examples containing diagnostic explanations, with experts annotating evidence for the PMC dataset. The Discussion says detailed evidence-based explanations helped models associate diagnostic criteria with symptoms. The paper does not establish that the models inherit expert annotators’ reasoning patterns, and it says most annotations for the other datasets used a multi-agent framework. (LLM, conf=0.79)

    Experimental — accuracy not yet measured

  • Twelve studies (17.4%) engaged healthcare professionals to evaluate generated explanations.

    “Twelve studies (17.4%) engaged healthcare professionals to evaluate generated explanations”

    From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [67]

    The Methods section (“Manual evaluation”; page unavailable) says physicians graded generated explanations, and the Results section (“Explanation performance”; page unavailable) says clinicians assessed them. The paper describes its own evaluation, however; it does not report that twelve studies, or 17.4% of studies, engaged healthcare professionals. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • Semantic similarity metrics such as BERTScore and domain-specific variants using ClinicalBERT or SciBERT embeddings assess meaning preservation beyond surface forms.

    “Semantic similarity metrics such as BERTScore [132] and domainspecific variants using ClinicalBERT [133] or SciBERT embeddings assess meaning preservation beyond surface forms [65,67,70,90,111].”

    From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [67]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 52

  1. [3]Zhou, S. et al. Explainable differential diagnosis with dual-inference large language models. Npj Health Syst. 2, 12 (2025). 40291514 PMC12021655 · DOI · Paper page
  2. [4]Kresevic, S. et al. Optimization of hepatological clinical guidelines interpretation by large language models: a retrieval augmented generation-based framework. NPJ Digit. Med. 7, 102 (2024). 38654102 PMC11039454 · DOI · Paper page
  3. [6]Hu, Z. et al. Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in LLMs. In The Thirty-eighth Annual Conference on Neural Information Processing Systems (2024).
  4. [7]Smith, P. C. et al. Missing clinical information during primary care visits. JAMA 293, 565–571 (2005). 15687311 · DOI · Paper page
  5. [8]Szpakowicz, M. & Herd, A. Medically cleared”: how well are patients with psychiatric presentations examined by emergency physicians?. J. Emerg. Med. 35, 369–372 (2008). 18650052 · DOI
  6. [9]Burnett, S. J., Deelchand, V., Franklin, B. D., Moorthy, K. & Vincent, C. Missing clinical information in NHS hospital outpatient clinics: prevalence, causes and effects on patient care. BMC Health Serv. Res. 11, 114 (2011). 21605359 PMC3118108 · DOI
  7. [10]Dovey, S. M. et al. A preliminary taxonomy of medical errors in family practice. Qual. Saf. Health Care 11, 233–238 (2002). 12486987 PMC1743626 · DOI
  8. [11]Wilson, R. M., Harrison, B. T., Gibberd, R. W. & Hamilton, J. D. An analysis of the causes of adverse events from the Quality in Australian Health Care Study. Med. J. Aust. 170, 411–415 (1999). 10341771 · DOI
  9. [12]Antman, E. M. et al. ACC/AHA guidelines for the management of patients with ST-elevation myocardial infarction; A report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Committee to Revise the 1999 Guidelines for the Management of patients with acute myocardial infarction). J. Am. Coll. Cardiol. 44, E1–E211 (2004). 15358047 · DOI
  10. [13]Ghadri, J.-R. et al. International expert consensus document on takotsubo syndrome (part I): clinical characteristics, diagnostic criteria, and pathophysiology. Eur. Heart J. 39, 2032–2046 (2018). 29850871 PMC5991216 · DOI
  11. [14]Lyon, A. R. et al. Current state of knowledge on Takotsubo syndrome: a Position Statement from the Taskforce on Takotsubo Syndrome of the Heart Failure Association of the European Society of Cardiology: Current state of knowledge on Takotsubo syndrome. Eur. J. Heart Fail. 18, 8–27 (2016). 26548803 · DOI
  12. [15]Lin, M. et al. Improving model fairness in image-based computer-aided diagnosis. Nat. Commun. 14, 6261 (2023). 37803009 PMC10558498 · DOI
  13. [16]Zhou, S. et al. Open-world electrocardiogram classification via domain knowledge-driven contrastive learning. Neural Netw. 179, 106551 (2024). 39068675 · DOI
  14. [17]Zhou, S. et al. Large language models for disease diagnosis: A scoping review. npj Artificial Intelligence 1, 9 (2025). 40607112 PMC12216946 · DOI
  15. [18]Vazhentsev, A. et al. Uncertainty-aware abstention in medical diagnosis based on medical texts. Preprint at https://arxiv.org/abs/2502.18050 (2025). · arXiv
  16. [19]Ribeiro, A. H. et al. Automatic diagnosis of the 12-lead ECG using a deep neural network. Nat. Commun. 11, 1760 (2020). 32273514 PMC7145824 · DOI · Paper page
  17. [20]Ghorbani, A. et al. Deep learning interpretation of echocardiograms. NPJ Digit. Med. 3, 10 (2020). 31993508 PMC6981156 · DOI · Paper page
  18. [21]Barnett, A. J. et al. A case-based interpretable deep learning model for classification of mass lesions in digital mammography. Nat. Mach. Intell. 3, 1061–1070 (2021). · DOI
  19. [24]Kwon, T. et al. Large language models are clinical reasoners: reasoning-aware diagnosis framework with prompt-generated rationales. Proc. Conf. AAAI Artif. Intell. 38, 18417–18425 (2024).
  20. [25]Gao, Y. et al. Uncertainty estimation in diagnosis generation from large language models: next-word probability is not pre-test probability. JAMIA Open 8, ooae154 (2025). 39802674 PMC11723528 · DOI
  21. [27]OpenAI et al. GPT-4 Technical Report. Preprint at https://arxiv.org/abs/2303.08774 (2023). · arXiv
  22. [28]Touvron, H. et al. LLaMA: open and efficient foundation language models. Preprint at https://arxiv.org/abs/2302.13971 (2023). · arXiv
  23. [29]Krishna, S., Bhambra, N., Bleakney, R. & Bhayana, R. Evaluation of reliability, repeatability, robustness, and confidence of GPT-3.5 and GPT-4 on a radiology board-style examination. Radiology 311, e232715 (2024). 38771184 · DOI
  24. [32]Guyatt, G. H. et al. Users’ Guides to the Medical Literature: XXV. Evidence-based medicine: principles for applying the Users’ Guides to patient care. Evid. Based Med. Working Group. JAMA 284, 1290–1296 (2000). 10979117
  25. [33]Greenhalgh, T. Narrative based medicine: narrative based medicine in an evidence based world. BMJ 318, 323–325 (1999). 9924065 PMC1114786 · DOI · Paper page
  26. [34]Zhou, H. et al. A survey of large language models in medicine: progress, application, and challenge. Preprint at https://arxiv.org/abs/2311.05112 (2023). · arXiv
  27. [35]Wang, J. et al. A survey on large Language Models from general purpose to medical applications: datasets, methodologies, and evaluations. Preprint at https://arxiv.org/abs/2406.10303 (2024). · arXiv
  28. [36]Wang, B. et al. Pre-trained language models in biomedical domain: A systematic survey. ACM Computing Surveys 56, 1–52 (2023).
  29. [37]Xu, K., Cheng, Y., Hou, W., Tan, Q. & Li, W. Reasoning like a doctor: Improving medical dialogue systems via diagnostic reasoning process alignment. In Findings of the Association for Computational Linguistics ACL 2024 (eds. Ku, L.-W., Martins, A. & Srikumar, V.) 6796–6814 (Association for Computational Linguistics, 2024).
  30. [38]Cui, H. et al. Biomedical visual instruction tuning with clinician preference alignment. Adv. Neural Inf. Process. Syst. 37, 96449–96467 (2024). 40017809 PMC11867732 · DOI
  31. [39]Johnson, A. E. W. et al. MIMIC-IV, a freely accessible electronic health record dataset. Sci. Data10, 1 (2023). PMC9810617 36596836
  32. [40]Zhao, Z., Jin, Q., Chen, F., Peng, T. & Yu, S. A large-scale dataset of patient summaries for retrieval-based clinical decision support systems. Sci. Data 10, 909 (2023). 38110415 PMC10728216 · DOI · Paper page
  33. [41]Gemini Team et al. Gemini: a family of highly capable multimodal models. Preprint at https://arxiv.org/abs/2312.11805 (2023). · arXiv
  34. [42]Curtin, S., Tejada-Vera, B. & Bastian, B. Deaths: leading causes for 2022. CDC 10.15620/cdc/164020 (2024). · DOI
  35. [44]Wang, B. et al. DiReCT: Diagnostic reasoning for clinical notes via large language models. In Advances in Neural Information Processing Systems (NIPS, 2024).
  36. [46]Bhasuran, B. et al. Preliminary analysis of the impact of lab results on large language model generated differential diagnoses. NPJ Digit. Med. 8, 166 (2025). 40102561 PMC11920259 · DOI
  37. [47]Adler-Milstein, J., Chen, J. H. & Dhaliwal, G. Next-generation artificial intelligence for diagnosis: from predicting diagnostic labels to “wayfinding. JAMA 326, 2467–2468 (2021). 34882190 PMC12049696 · DOI
  38. [49]Zhan, Z., Zhou, S., Li, M. & Zhang, R. RAMIE: retrieval-augmented multi-task information extraction with large language models on dietary supplements. J. Am. Med. Inform. Assoc. 32, 545–554 (2025). 39798153 PMC11833482 · DOI
  39. [50]Han, T. et al. MedAlpaca -- an open-source collection of medical conversational AI models and training data. Preprint at https://arxiv.org/abs/2304.08247 (2023). · arXiv
  40. [51]Wu, C. et al. PMC-LLaMA: toward building open-source language models for medicine. J. Am. Med. Inform. Assoc. 31, 1833–1843 (2024). 38613821 PMC11639126 · DOI · Paper page
  41. [52]Zhou, J. et al. Pre-trained multimodal large language model enhances dermatological diagnosis using SkinGPT-4. Nat. Commun. 15, 5649 (2024). 38969632 PMC11226626 · DOI · Paper page
  42. [55]Thygesen, K. et al. Fourth universal definition of myocardial infarction (2018). Glob. Heart 13, 305–338 (2018). 30154043 · DOI
  43. [58]Sokolova, M. & Lapalme, G. A systematic analysis of performance measures for classification tasks. Inf. Process. Manag. 45, 427–437 (2009). · DOI · Paper page
  44. [59]Christophe, C. et al. Med42 - Evaluating Fine-Tuning Strategies for Medical LLMs: Full-Parameter vs. Parameter-Efficient Approaches. AAAI 2024 Spring Symposium on Clinical Foundation Models (2024).
  45. [60]Yang, R. et al. Retrieval-augmented generation for generative artificial intelligence in health care. npj Health Syst. 2, 1–5 (2025). PMC13354210 42527447 · DOI
  46. [61]Hu, E. J. et al. LoRA: Low-Rank Adaptation of Large Language Models. International Conference on Learning Representations (2022).
  47. [62]Yin, Z. et al. Do large language models know what they don’t know? In Findings of the Association for Computational Linguistics: ACL 2023 (Association for Computational Linguistics, 2023).
  48. [63]Amayuelas, A., Wong, K., Pan, L., Chen, W. & Wang, W. Y. Knowledge of knowledge: exploring known-unknowns uncertainty with large language models. In Findings of the Association for Computational Linguistics ACL 2024 (eds. Ku, L.-W., Martins, A. & Srikumar, V.) 6416–6432 (Association for Computational Linguistics, 2024).
  49. [64]Satanjeev Banerjee, A. L. METEOR An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization (Association for Computational Linguistics, 2005).
  50. [65]Zhang, T. et al. BERTScore: Evaluating Text Generation with BERT. International Conference on Learning Representations (2020).
  51. [66]Reimers, N. & Gurevych, I. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (Association for Computational Linguistics, 2019). 10.18653/v1/d19-1410. · DOI · Paper page
  52. [68]Croxford, E. et al. Current and future state of evaluation of large language models for medical summarization tasks. Npj Health Syst.2, 6 (2025). PMC11928168 40124388

Cited by

Sorted by citations

Authors
Zhou, S., Wang, J., Xu, Z., Wang, S., Brauer, D., Welton, L., Cogan, J., Chung, Y.-H., Tian, L., Zhan, Z., Hou, Y., Lin, M., Melton, G. B., & Zhang, R.
Affiliations
  • University of Minnesota (US)
  • University of Chicago (US)
  • Columbia University (US)
  • University of Virginia (US)
  • University of California, San Francisco (US)
  • Mayo Clinic (US)
Corresponding author
Zhang, Rui
ORCID
Venue
npj Digital Medicine
Publisher
Springer Science and Business Media LLC
Published
2025-11-18
Volume
8
Issue
1
Pages
690-690
ISSN
2398-6352 (ISSN-L)
DOI
10.1038/s41746-025-02071-6
PMC
PMC12627459
OpenAlex
W4416332205
Cited by
27
References
17
Citations by year
2026: 26 · 2025: 1
FWCI
6.50
Type
journal article
Language
en
Open access
Licence
Locations
  • npj Digital Medicine (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University)
Funders
  • National Center for Complementary and Integrative Health (R01AT009457)
  • National Institute on Aging (R01AG078154)
  • National Cancer Institute (R01CA287413)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
diagnostic uncertainty, clinical decision support, diagnostic reasoning, clinical notes

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review