Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 62 of the 88 references it cites were found in a registry or an index; 21 of the 30 sources its claims cite were checked for support.

The false hope of current approaches to explainable artificial intelligence in health care

Open access
Ghassemi, Marzyeh, Oakden-Rayner, Luke, Beam, Andrew L · 2021 · The Lancet Digital Health

Abstract

The black-box nature of current artificial intelligence (AI) has caused some to question whether AI must be explainable to be used in high-stakes scenarios such as medicine. It has been argued that explainable AI will engender trust with the health-care workforce, provide transparency into the AI decision making process, and potentially mitigate various kinds of bias. In this Viewpoint, we argue that this argument represents a false hope for explainable AI and that current explainability methods are unlikely to achieve these goals for patient-level decision support. We provide an overview of current explainability techniques and highlight how various failure cases can cause problems for decision making for individual patients. In the absence of suitable explainability methods, we advocate for rigorous internal and external validation of AI models as a more direct means of achieving the goals often associated with explainability, and we caution against having explainability be a requirement for clinically deployed models.

Overview

CoChat Summary

## TL;DR

This Viewpoint argues that current explainable AI methods in health care are unlikely to provide reliable, patient-level justifications, build trust, or justify clinical deployment decisions. The authors instead recommend rigorous validation and using explanations mainly for developer/auditor tasks such as troubleshooting, model performance improvement, and algorithmic audit.

## Key findings

  • The paper says current explainability techniques can produce broad descriptions of how a system works, but for individual decisions the explanations are “unreliable” or only “superficial,” and may be misleading.
  • The authors argue that explanations cannot yet “provide reassurance that an individual decision is correct, increase trust among users, nor justify the acceptance of AI recommendations in clinical practice.”
  • They note that explainability can still be useful for “model troubleshooting and systems audit,” including identifying common failure modes or biases.
  • For clinical deployment, they urge that AI systems be justified by “reliable and experimentally confirmed performance,” not by explainability-based rationalisations.

## Methods

  • This is a Viewpoint, not an empirical study; the paper gives a conceptual and argumentative overview of explainable AI in health care.
  • It discusses current explainability approaches, including inherent explainability and post-hoc explainability, with medical imaging heat maps, LIME, SHAP, feature visualisation, and prototypical explanations as illustrative examples.
  • The main analysis is a qualitative critique of how these methods can fail at the patient level, drawing on examples from medical imaging, language models, and prior literature on interpretability, automation bias, and model auditing.

## Limitations

  • The paper does not report original data, a study sample, or quantitative evaluation of explainability methods; it is a perspective piece.
  • It states that “precise technical definitions of explainability lack consensus,” which limits how uniformly the concept can be assessed.
  • Evident from the text: the argument is largely theoretical and literature-based, so its claims about real-world clinical impact are not supported by new empirical testing in this paper.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/s2589-7500(21)00208-9

Cites

62 of 62 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

19 Valid · 1 Not valid · 7 Not checked · 41 Own results and interpretations

33 more claims · 6 with their cited sources checked

Create a free account to see all 68 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 13
  • Probably not a substantial claim. 20

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

12 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 88 references

Research works printed but not linked: 1

  1. 2016, Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46, OJEU, 59, 294

Cited by

See all 1,724

Metadata

Authors
Ghassemi, M., Oakden-Rayner, L., & Beam, A. L.
Affiliations
  • Vector Institute (CA)
  • Harvard University (US)
Corresponding author
Beam, Andrew L
Venue
The Lancet Digital Health
Publisher
Elsevier BV
Published
2021-11
Volume
3
Issue
11
Pages
e745-e750
ISSN
2589-7500 (ISSN-L)
OpenAlex
W3209901185
Cited by
1,724
References
88
Citations by year
2026: 603 · 2025: 458 · 2024: 257 · 2023: 249 · 2022: 153 · 2021: 4
FWCI
11.99
Type
journal article
Language
en
Open access
Licence
Locations
  • The Lancet Digital Health (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
National Heart, Lung, and Blood Institute (7K01HL141771-02)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
explainable AI, health care AI, black-box models, clinical decision support, trust in AI, algorithmic bias, model validation, external validation, clinical implementation of AI
MeSH terms
Artificial Intelligence, Communication, Decision Making, Delivery of Health Care, Diagnostic Imaging, Health Personnel, Humans, Models, Biological, Bias, Comprehension, Dissent and Disputes, Trust

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The false hope of current approaches to explainable artificial intelligence in health care. https://app.cochat.ai/papers/10.1016/s2589-7500(21)00208-9
The paper
Ghassemi, M., Oakden-Rayner, L., & Beam, A. L. (2021). The false hope of current approaches to explainable artificial intelligence in health care. The Lancet Digital Health. https://doi.org/10.1016/s2589-7500(21)00208-9

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.