CoChat

When silence is safer: a review and decision-theoretic framework for LLM abstention in healthcare

Presacan, Oriana, Nik, Alireza, Ojha, Jaya, Thambawita, Vajira, Ionescu, Bogdan, Riegler, Michael A. · 2026 · npj Digital Medicine

Published version of a preprint

Cited by 4

The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41746-026-02882-1

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 9 references

Research works printed but not linked: 50

  1. Singhal, K. et al. Toward expert-level medical question answering with large language models. Nat. Med. 31, 943–950 (2025). · DOI · Paper page
  2. Tu, T. et al. Towards conversational diagnostic artificial intelligence. Nature 642, 442–450 (2025). · DOI · Paper page
  3. Yang, X. et al. Multiple large language models versus experienced physicians in diagnosing challenging cases with gastrointestinal symptoms. NPJ Digit. Med. 8, 85 (2025). · DOI
  4. Van Veen, D. et al. Adapted large language models can outperform medical experts in clinical text summarization. Nat. Med. 30, 1134–1142 (2024). · DOI · Paper page
  5. Kim, J. et al. Large language models outperform mental and medical health care professionals in identifying obsessive-compulsive disorder. NPJ Digit. Med. 7, 193 (2024). · DOI · Paper page
  6. Ji, Z. et al. Hallucinations in Natural Language Generation: A Survey. ACM Computing Surveys (2023). · DOI · Paper page
  7. Xu, Z., Jain, S. & Kankanhalli, M. Hallucination is Inevitable: An Innate Limitation of Large Language Models. https://arxiv.org/abs/2401.11817. 2401.11817 (2025). · arXiv
  8. Kalai, A. T., Nachum, O., Vempala, S. S. & Zhang, E. Why Language Models Hallucinate. arXiv preprint https://arxiv.org/abs/2509.04664. ArXiv:2509.04664 [cs.CL], 2509.04664 (2025). · arXiv
  9. Asgari, S. et al. A framework to assess clinical safety and hallucination rates of llms for medical text summarisation. npj Digit. Med. 8, 45 (2025). · DOI
  10. Omar, M. et al. Multi-model assurance analysis showing large language models are highly vulnerable to adversarial hallucination attacks during clinical decision support. Commun. Med. 5, 330 (2025). · DOI · Paper page
  11. Bouguettaya, A., Stuart, E. M. & Aboujaoude, E. Racial bias in AI-mediated psychiatric diagnosis and treatment: A qualitative comparison of four large language models. NPJ Digit. Med. 8, 332 (2025). · DOI
  12. Wei, A., Haghtalab, N. & Steinhardt, J. Jailbroken: How does llm safety training fail? In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23 (Curran Associates Inc., 2023). · DOI · Paper page
  13. Zhang, H., Lou, Q. & Wang, Y. Towards Safe AI Clinicians: A Comprehensive Study on Large Language Model Jailbreaking in Healthcare. AMIA Annual Symposium Proceedings 2024, 1519–1528 (2025).
  14. Chen, Q., Yamaguchi, S. & Yamamoto, Y. LLM abuse prevention tool using GCG jailbreak attack detection and DistilBERT-based ethics judgment. Information 16, 204 (2025). · DOI
  15. Han, T., Kumar, A., Agarwal, C. & Lakkaraju, H. MedSafetyBench: Evaluating and Improving the Medical Safety of Large Language Models. In Advances in Neural Information Processing Systems 37, NeurIPS ’24, Datasets and Benchmarks Track 33423–33454 (Curran Associates Inc., 2024). · DOI
  16. Chen, S. et al. CARES: Comprehensive evaluation of safety and adversarial robustness in medical LLMs. In 39th Conference on Neural Information Processing Systems, NeurIPS ’25, Datasets and Benchmarks Track (2025). · DOI
  17. El-Yaniv, R. et al. On the foundations of noise-free selective classification. J. Mach. Learn. Res. 11, 1605–1641 (2010).
  18. Wen, B., Yao, J., Feng, S., Xu, C., Tsvetkov, Y., Howe, B. & Wang, L. L. Know your limits: A survey of abstention in large language models. Trans. Assoc. Comput. Linguist. 13, 529–556 (2025). · DOI · Paper page
  19. Adhikari, T. Learning to Say “I Don’t Know”: A Vision for Abstention in Large Language Models https://doi.org/10.36227/techrxiv.175682660.02761872/v1. (2025). · DOI
  20. Feng, S. et al. Don’t Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaboration. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. https://aclanthology.org/2024.acl-long.786/ (Association for Computational Linguistics, 2024). · DOI · Paper page
  21. Ghahramani, Z. Probabilistic machine learning and artificial intelligence. Nature 521, 452–459 (2015). · DOI · Paper page
  22. Kendall, A. & Gal, Y. What uncertainties do we need in Bayesian deep learning for computer vision? In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, 5580–5590 (Curran Associates Inc., Red Hook, NY, USA, 2017).
  23. Li, S. S. et al. MEDIQ: question-asking LLMs and a benchmark for reliable interactive clinical reasoning. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS ’24 (Curran Associates Inc., Red Hook, NY, USA, 2025).
  24. Li, X. et al. AT-CXR: Uncertainty-aware agentic triage for chest X-rays https://arxiv.org/abs/2508.19322. 2508.19322 (2025). · arXiv
  25. Strong, J., Men, Q. & Noble, A. Trustworthy and practical AI for healthcare: A guided deferral system with large language models https://arxiv.org/abs/2406.072122406.07212 (2025).
  26. Wen, B., Howe, B. & Wang, L. L. Characterizing LLM abstention behavior in science QA with context perturbations. In Findings of the Association for Computational Linguistics: EMNLP 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.197 (Association for Computational Linguistics, 2024). · DOI
  27. Thomas, J., Mishra, P., Sharma, D. & Krishnamurthy, P. LTRC-IIITH at EHRSQL 2024: Enhancing reliability of text-to-SQL systems through abstention and confidence thresholding. In Naumann, T., Ben Abacha, A., Bethard, S., Roberts, K. & Bitterman, D. (eds.) Proceedings of the 6th Clinical Natural Language Processing Workshop, 697–702. https://aclanthology.org/2024.clinicalnlp-1.66/ (Association for Computational Linguistics, Mexico City, Mexico, 2024). · doi:10.18653/v1/2024.clinicalnlp-1.66
  28. Xie, T. et al. SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal. In International Conference on Learning Representations, ICLR ’25 https://proceedings.iclr.cc/paper_files/paper/2025/hash/9622163c87b67fd5a4a0ec3247cf356e-Abstract-Conference.html (2025).
  29. Wang, P. et al. InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance https://arxiv.org/abs/2401.11206. 2401.11206 (2024). · arXiv
  30. Balakrishnan, S. et al. Evaluating artificial intelligence bias in nephrology: the role of diversity, equity, and inclusion in AI-driven decision-making and ethical regulation. Frontiers in Artificial Intelligence Vol 8, https://www.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2025.1525937 (2025). · DOI
  31. Kaur, N., Choudhury, M. & Pruthi, D. Evaluating large language models for health-related queries with presuppositions. In Ku, L.-W., Martins, A. & Srikumar, V. (eds.) Findings of the Association for Computational Linguistics: ACL 2024, 14308–14331. https://aclanthology.org/2024.findings-acl.850/ (Association for Computational Linguistics, Bangkok, Thailand, 2024). · DOI
  32. Qin, Z. et al. LMOD: A large multimodal ophthalmology dataset and benchmark for large vision-language models. In Chiruzzo, L., Ritter, A. & Wang, L. (eds.) Findings of the Association for Computational Linguistics: NAACL 2025, 2501–2522 (Association for Computational Linguistics, Albuquerque, New Mexico). https://aclanthology.org/2025.findings-naacl.135/ (2025). · DOI
  33. Lyu, K. et al. Keeping LLMs aligned after fine-tuning: The crucial role of prompt templates. In Advances in Neural Information Processing Systems 37, NeurIPS ’24, Main Conference Track https://doi.org/10.52202/079017-3766 (Curran Associates Inc., 2024). · DOI
  34. Mamun, M. A. A., Alouani, I. & Abu-Ghazaleh, N. Poison once, refuse forever: Weaponizing alignment for injecting bias in LLMs https://arxiv.org/abs/2508.20333. 2508.20333 (2025). · arXiv
  35. Avnat, E. et al. Performance of large language models in numerical versus semantic medical knowledge: Cross-sectional benchmarking study on evidence-based questions and answers. J. Med. Internet Res. 27, e64452 (2025). · DOI
  36. Farquhar, S. et al. Detecting hallucinations in large language models using semantic entropy. Nature 630, 625–630 (2024). · DOI · Paper page
  37. Wang, C., Zhou, W., Ghosh, S., Batmanghelich, K. & Li, W. Semantic consistency-based uncertainty quantification for factuality in radiology report generation. In Findings of the Association for Computational Linguistics: NAACL 2025 1739–1754 https://doi.org/10.18653/v1/2025.findings-naacl.95 (Association for Computational Linguistics, 2025). · DOI
  38. Socrates, V. et al. Identifying deprescribing opportunities with large language models in older adults: Retrospective cohort study. JMIR Aging 8, e69504 (2025). · DOI · Paper page
  39. Davies, R. H., Sanghvi, K., Nalam, R. & Ramnath, R. A Pre-Processing Framework for Securing LLM-RAG Interfaces Against Information Leakage. In Practice and Experience in Advanced Research Computing 2025: The Power of Collaboration. https://doi.org/10.1145/3708035.3736046 (Association for Computing Machinery, New York, NY, USA, 2025). · DOI
  40. Shrivastava, V., Liang, P. & Kumar, A. Llamas Know What GPTs Don’t Show: Surrogate Models for Confidence Estimation https://arxiv.org/abs/2311.08877. 2311.08877 (2023). · arXiv
  41. Shankar, R. et al. Energy landscapes enable reliable abstention in retrieval-augmented large language models for healthcare https://arxiv.org/abs/2509.04482. 2509.04482 (2025). · arXiv
  42. Li, H., He, Z. & Chen, X. et al. Smarter together: Combining large language models and small models for physiological signals visual inspection. J. Healthc. Inform. Res. 9, 656–685 (2025). · DOI
  43. Song, Y., Jeong, M. & Sung, M. Trustworthy Agents for Electronic Health Records through Confidence Estimation https://arxiv.org/abs/2508.19096. 2508.19096 (2025). · arXiv
  44. Berkowitz, J. et al. Probing Hidden States for Calibrated, Alignment-Resistant Predictions in LLMs. medRxivhttps://www.medrxiv.org/content/early/2025/09/27/2025.09.17.25336018 (2025).
  45. Cao, L. Learn to Refuse: Making Large Language Models More Controllable and Reliable through Knowledge Scope Limitation and Refusal Mechanism. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing 3628–3646 https://doi.org/10.18653/v1/2024.emnlp-main.212 (Association for Computational Linguistics, 2024). · DOI
  46. Qi, X. et al. Fine-tuning aligned language models compromises safety, even when users do not intend To! https://arxiv.org/abs/2310.03693. 2310.03693 (2023). · arXiv
  47. Johri, S., Jeong, J. et al. An evaluation framework for clinical use of large language models in patient interaction tasks. Nat. Med. 31, 77–86 (2025). · DOI · Paper page
  48. Chowdhury, M., He, Y. V., Joselowitz, J., Higham, A. & Lim, E. ASTRID – An automated and scalable TRIaD for the evaluation of RAG-based clinical question answering systems. In Findings of the Association for Computational Linguistics: ACL 2025 16700–16716 https://doi.org/10.18653/v1/2025.findings-acl.857 (Association for Computational Linguistics, 2025). · DOI
  49. Croxford, E. et al. Evaluating clinical AI summaries with large language models as judges. npj Digit. Med. 8, 640 (2025). · DOI
  50. Zheng, L. et al. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena https://arxiv.org/abs/2306.05685. 2306.05685 (2023). · arXiv

Metadata

Authors
Presacan, O., Nik, A., Ojha, J., Thambawita, V., Ionescu, B., & Riegler, M. A.
Venue
npj Digital Medicine
Published
2026-06-16
Volume
9
Issue
1
ISSN
2398-6352
Cited by
4
References
9
Language
en
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: When silence is safer: a review and decision-theoretic framework for LLM abstention in healthcare. https://app.cochat.ai/papers/10.1038/s41746-026-02882-1
The paper
Presacan, O., Nik, A., Ojha, J., Thambawita, V., Ionescu, B., & Riegler, M. A. (2026). When silence is safer: a review and decision-theoretic framework for LLM abstention in healthcare. npj Digital Medicine. https://doi.org/10.1038/s41746-026-02882-1

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.