CoChat

The DOI matches its Crossref record; 46 of the 49 references it cites were found in a registry or an index; 8 of the 49 sources its claims cite were checked for support.

Explainability for artificial intelligence in healthcare: a multidisciplinary perspective

Open access
the Precise4Q consortium, Amann, Julia, Blasimme, Alessandro, Vayena, Effy, Frey, Dietmar, Madai, Vince I. · 2020 · BMC Medical Informatics and Decision Making

Abstract

Background: Explainability is one of the most heavily debated topics when it comes to the application of artificial intelligence (AI) in healthcare. Even though AI-driven systems have been shown to outperform humans in certain analytical tasks, the lack of explainability continues to spark criticism. Yet, explainability is not a purely technological issue, instead it invokes a host of medical, legal, ethical, and societal questions that require thorough exploration. This paper provides a comprehensive assessment of the role of explainability in medical AI and makes an ethical evaluation of what explainability means for the adoption of AI-driven tools into clinical practice. Methods: Taking AI-based clinical decision support systems as a case in point, we adopted a multidisciplinary approach to analyze the relevance of explainability for medical AI from the technological, legal, medical, and patient perspectives. Drawing on the findings of this conceptual analysis, we then conducted an ethical assessment using the “Principles of Biomedical Ethics” by Beauchamp and Childress (autonomy, beneficence, nonmaleficence, and justice) as an analytical framework to determine the need for explainability in medical AI. Results: Each of the domains highlights a different set of core considerations and values that are relevant for understanding the role of explainability in clinical practice. From the technological point of view, explainability has to be considered both in terms how it can be achieved and what is beneficial from a development perspective. When looking at the legal perspective we identified informed consent, certification and approval as medical devices, and liability as core touchpoints for explainability. Both the medical and patient perspectives emphasize the importance of considering the interplay between human actors and medical AI. We conclude that omitting explainability in clinical decision support systems poses a threat to core ethical values in medicine and may have detrimental consequences for individual and public health. Conclusions: To ensure that medical AI lives up to its promises, there is a need to sensitize developers, healthcare professionals, and legislators to the challenges and limitations of opaque algorithms in medical AI and to foster multidisciplinary collaboration moving forward.

Overview

CoChat Summary

TL;DR

This paper provides a multidisciplinary conceptual and ethical assessment of explainability in AI-based clinical decision support systems, covering technological, legal, medical, and patient perspectives. It concludes that omitting explainability threatens core ethical values in medicine and may have detrimental consequences for individual and public health.

Key findings

  • From the technological perspective, explainability can be inherent to an algorithm or approximated by other methods; inherent explainability is generally more accurate, but there is often a trade-off with performance because inherently explainable models are usually traditional methods such as linear or logistic regression. Explainability also helps developers detect “Clever Hans” errors, such as the Mount Sinai x-ray model that relied on hardware-related metadata rather than clinically relevant image features.
  • From the legal perspective, the paper identifies three core fields for explainability: informed consent, certification and approval as medical devices under FDA and MDR frameworks, and liability. It states that FDA and MDR currently require explainability only vaguely, and that GDPR’s implications remain debated.
  • From the medical perspective, the paper distinguishes first-level explainability (understanding how the system arrives at conclusions in general) from second-level explainability (identifying which features were important for an individual prediction). Explainability can help clinicians resolve disagreement with AI, identify false positives and false negatives, and support trust and quality assurance.
  • From the patient and ethical perspectives, “black-box medicine” is argued to conflict with patient-centered care and shared decision-making. Using Beauchamp and Childress’s four principles—autonomy, beneficence, nonmaleficence, and justice—the paper concludes that explainability is an ethical prerequisite for clinically applied AI, and that opaque systems may foster paternalism, defensive medicine, and unjust bias.

Methods

  • The paper performs a conceptual analysis of pertinent literature on explainable AI from the technological, legal, medical, and patient perspectives, taking AI-based clinical decision support systems as the case in point.
  • It then conducts an ethical assessment using the “Principles of Biomedical Ethics” by Beauchamp and Childress, structured around autonomy, beneficence, nonmaleficence, and justice.
  • No empirical data collection, patient or clinician sample, or quantitative analysis is reported.

Limitations

  • The paper states that the terminology of explainability is not well defined, that there exist only exploratory initial attempts to rank explainability methods quantitatively, that legal requirements remain vague or ambiguous, that liability questions lack clear answers, and that ethical consensus on disclosure of opaque AI has not yet emerged.
  • The paper also states that further work is needed to sensitize developers, healthcare professionals, and legislators and to foster multidisciplinary collaboration.
  • Evident from the text: this is a conceptual and normative analysis without empirical validation or systematic review methodology, and its legal discussion is explicitly framed from a Western legal point of view.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s12911-020-01332-6

Cites

46 of 46 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

7 Valid · 5 Not valid · 34 Not checked · 65 Own results and interpretations

39 more claims · 1 with their cited sources checked

Create a free account to see all 111 claims and to verify the 41 references still open.

See all claims — free account
  • It may state two separate claims as one. 34
  • Probably not a substantial claim. 47

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

8 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 49 references

Research works printed but not linked: 1

  1. Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academies Press (US); 2001. http://www.ncbi.nlm.nih.gov/books/NBK222274/. Accessed 21 May 2020.

Cited by

See all 2,203

Metadata

Authors
the Precise4Q consortium, Amann, J., Blasimme, A., Vayena, E., Frey, D., & Madai, V. I.
Affiliations
  • ETH Zurich (CH)
  • Charité – Universitätsmedizin Berlin (DE)
  • Birmingham City University (GB)
Corresponding author
Amann, Julia
Venue
BMC Medical Informatics and Decision Making
Published
2020-11-30
Volume
20
Issue
1
Pages
310-310
ISSN
1472-6947 (ISSN-L)
OpenAlex
W3109650690
Cited by
2,203
References
49
Citations by year
2026: 699 · 2025: 555 · 2024: 407 · 2023: 306 · 2022: 168 · 2021: 67
FWCI
53.13
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medical Informatics and Decision Making (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by)
  • ETH Zürich Research Collection (cc-by)
Funders
Horizon 2020 Research and Innovation Programme (777107)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
Keywords
explainability, artificial intelligence in healthcare, clinical decision support systems, biomedical ethics, informed consent, medical device regulation, liability, human-AI interaction, algorithmic opacity, multidisciplinary collaboration
MeSH terms
Artificial Intelligence, Delivery of Health Care, Health Facilities, Humans, Informed Consent, Decision Support Systems, Clinical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. https://app.cochat.ai/papers/10.1186/s12911-020-01332-6
The paper
the Precise4Q consortium, Amann, J., Blasimme, A., Vayena, E., Frey, D., & Madai, V. I. (2020). Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Medical Informatics and Decision Making. https://doi.org/10.1186/s12911-020-01332-6

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.