The DOI matches its Crossref record; 73 of the 73 references it cites were found in a registry or an index.
To explain or not to explain?—Artificial intelligence explainability in clinical decision support systems
Open accessAbstract
Explainability for artificial intelligence (AI) in medicine is a hotly debated topic. Our paper presents a review of the key arguments in favor and against explainability for AI-powered Clinical Decision Support System (CDSS) applied to a concrete use case, namely an AI-powered CDSS currently used in the emergency call setting to identify patients with life-threatening cardiac arrest. More specifically, we performed a normative analysis using socio-technical scenarios to provide a nuanced account of the role of explainability for CDSSs for the concrete use case, allowing for abstractions to a more general level. Our analysis focused on three layers: technical considerations, human factors, and the designated system role in decision-making. Our findings suggest that whether explainability can provide added value to CDSS depends on several key questions: technical feasibility, the level of validation in case of explainable algorithms, the characteristics of the context in which the system is implemented, the designated role in the decision-making process, and the key user group(s). Thus, each CDSS will require an individualized assessment of explainability needs and we provide an example of how such an assessment could look like in practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures

![Fig 2 Z-Inspection® process. This figure has been reproduced from Zicari RV, Brodersen J, Brusseau J, Düdder B, Eichhorn T, Ivanov T, et al. Z-Inspection®: A Process to Assess Trustworthy AI. IEEE Trans Technol Soc. 2021 Jun;2(2):83–97. [56].](/paper-figures/c71a414b2c984bfea375d14b7dd18a96/to-explain-or-not-to-explain-artificial-intelligence-explainability-in-clinical-figure-2.jpg)
![Fig 3 Ethical issue identified during the initial assessment of the use case [31].](/paper-figures/4314a4502f78427c86c49fe0ba120ef6/to-explain-or-not-to-explain-artificial-intelligence-explainability-in-clinical-figure-3.jpg)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While interpretability is increasingly recognized as essential for the clinical integration of AI, further prospective and real-world studies are needed to quantify its impact on patient outcomes and define implementation best practices429,430.”
From Artificial intelligence-driven multi-omics approaches in Alzheimer's disease: Progress, challenges, and future directions (2025) · cites this paper as [429]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In particular, the framework in [137] aims at avoiding situations that can have dire consequences for patients.”
From Connecting the dots in trustworthy Artificial Intelligence: From AI principles, ethics, and key requirements to responsible AI systems and regulation (2023) · cites this paper as [137]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The ongoing debate about explainability trade-offs [52,53], building on earlier work [43-46], underscores that selecting suitable XAI methods requires careful consideration of the context-balancing interpretability needs against performance constraints and workflow pressures.”
From Explainable AI for Clinical Decision Support Systems: Literature Review, Key Gaps, and Research Synthesis (2025) · cites this paper as [52]
The cited source says what the claim attributes to it.
In the citing paper
“as Amann et al. (2022) point out, transparency is not one-dimensional: “Explainability is but one measure of achieving transparency; other measures include detailed documentation of datasets and algorithms, as well as open communication of the system’s capabilities and limitations” (Amann et al., 2022).”
From An auditable and source-verified framework for clinical AI decision support: integrating retrieval-augmented generation with data provenance (2026) · cites this paper as Amann et al., 2022
The cited source says what the claim attributes to it.
In the citing paper
“leading studies have repeatedly identified that lack of transparency is a major barrier to AI-CDS adoption (Tun et al., 2025; Amann et al., 2022).”
From An auditable and source-verified framework for clinical AI decision support: integrating retrieval-augmented generation with data provenance (2026) · cites this paper as Amann et al., 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · cited 4×
“An explanation for a black-box model can then, for example, be obtained through an interpretable approximation (see, e.g., SHAP [32] or LIME [33]).”
MethodInfluentialGauriat, C.-M., Pencolé, Y., Ribot, P., & Brouillet, G. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.27 · Cited by 12,586
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 3×
“Due to these limitations in current techniques for generating explanations, some experts suggest the use of inherently transparent models such as regression, decision trees, or rule lists [45,50].”
BackgroundBarredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · cited 1×
“We aimed to capture the various disciplinary perspectives on explainability, ranging from philosophy, to medicine, to computer science, and law [14,17,18,59,60].”
BackgroundThe global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · cited 1×
“An often referenced way to foster trust is to increase the system’s transparency [12], and an important part of increasing transparency is the application of explainability [13].”
BackgroundLipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427 · cited 5×
“When referring to different algorithms that can achieve explainability, there are unfortunately a multitude of different terms in the field that are frequently used interchangeably and without a fixed definition, especially across disciplines [22,25,28].”
BackgroundInfluentialExplanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Research works printed but not linked: 4
- ZC Lipton, 2018, The mythos of model interpretability, Commun ACM, 61, 36, 10.1145/3233231 · doi:10.1145/3233231 · Paper page
- Karimi A-H, Barthe G, Balle B, Valera I. Model-Agnostic Counterfactual Explanations for Consequential Decisions. In: International Conference on Artificial Intelligence and Statistics [Internet]. PMLR; 2020 [cited 2021 Jun 15]. p. 895–905. Available from: http://proceedings.mlr.press/v108/karimi20a.html
- Aubakirova M, Bansal M. Interpreting Neural Networks to Improve Politeness Comprehension. ArXiv161002683 Cs [Internet]. 2016 Oct 9 [cited 2021 Apr 13]; Available from: http://arxiv.org/abs/1610.02683 · arXiv
- Karlekar S, Niu T, Bansal M. Detecting Linguistic Characteristics of Alzheimer’s Dementia by Interpreting Neural Models. ArXiv180406440 Cs [Internet]. 2018 Apr 17 [cited 2021 Apr 13]; Available from: http://arxiv.org/abs/1804.06440 · arXiv
Cited by
Díaz-Rodríguez, N., Del Ser, J., Coeckelbergh, M., López de Prado, M., Herrera-Viedma, E., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101896 · Cited by 800
A Scoping Review of Reporting Gaps in FDA-Approved AI Medical Devices
Muralidharan, V., Adewale, B. A., Huang, C. J., Nta, M. T., Ademiju, P. O., Pathmarajah, P., Hang, M. K., Adesanya, O., Abdullateef, R. O., Babatunde, A. O., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01270-x · Cited by 201
Gala, D., Behl, H., Shah, M., & Makaryus, A. N. (2024) · Healthcare · doi:10.3390/healthcare12040481 · Cited by 183
“Explainable AI methods aim to provide transparency in AI-driven decision making, offering insights into how algorithms arrive at specific conclusions [61].”
Solving the Explainable AI Conundrum by Bridging Clinicians’ Needs and Developers’ Goals
Bienefeld, N., Boss, J. M., Lüthy, R., Brodbeck, D., Azzati, J., Blaser, M., Willms, J., & Keller, E. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00837-4 · Cited by 159
Caruccio, L., Cirillo, S., Polese, G., Solimando, G., Sundaramurthy, S., & Tortora, G. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121186 · Cited by 173
Metadata
- Authors
- Amann, J., Vetter, D., Blomberg, S. N., Christensen, H. C., Coffee, M., Gerke, S., Gilbert, T. K., Hagendorff, T., Holm, S., Livne, M., Spezzatti, A., Strümke, I., Zicari, R. V., Madai, V. I., & initiative, O. B. O. T. Z.-I.
- Affiliations
- ETH Zurich (CH)
- Goethe University Frankfurt (DE)
- University of Copenhagen (DK)
- New York University (US)
- Pennsylvania State University (US)
- Cornell Tech (US)
- University of Tübingen (DE)
- Google (United Kingdom) (GB)
- University of California, Berkeley (US)
- Norwegian University of Science and Technology (NO)
- Seoul National University (KR)
- Birmingham City University (GB)
- Corresponding author
- Amann, Julia
- ORCID
- Venue
- PLOS Digital Health
- Publisher
- Public Library of Science (PLoS)
- Published
- 2022-02-17
- Volume
- 1
- Issue
- 2
- Pages
- e0000016
- ISSN
- 2767-3170 (ISSN-L)
- PMC
- PMC9931364
- OpenAlex
- W4212942200
- Cited by
- 251
- References
- 62
- Citations by year
- 2026: 75 · 2025: 70 · 2024: 55 · 2023: 35 · 2022: 16
- FWCI
- 14.49
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLOS Digital Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research at the University of Copenhagen (University of Copenhagen) (publishedVersion, cc-by)
- eYLS (Yale Law School) (submittedVersion, cc-by)
- eYLS (Yale Law School) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- NORA - Norwegian Open Research Archives (submittedVersion, cc-by)
- Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by)
- ETH Zürich Research Collection
- Funders
- Horizon 2020 Framework Programme (777107, 101016233, H2020-SC1-PHE-CORONAVIRUS-2020-2-RTD)
- Connecting Europe Facility (INEA/CEF/ICT/A2020/2276680, 2020-EU-IA-0098)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- clinical decision support systems, cardiac arrest prediction, normative analysis, human factors, explainable algorithms, user groups
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: To explain or not to explain?—Artificial intelligence explainability in clinical decision support systems. https://app.cochat.ai/papers/10.1371/journal.pdig.0000016
- The paper
- Amann, J., Vetter, D., Blomberg, S. N., Christensen, H. C., Coffee, M., Gerke, S., Gilbert, T. K., Hagendorff, T., Holm, S., Livne, M., Spezzatti, A., Strümke, I., Zicari, R. V., Madai, V. I., & initiative, O. B. O. T. Z.-I. (2022). To explain or not to explain?—Artificial intelligence explainability in clinical decision support systems. PLOS Digital Health. https://doi.org/10.1371/journal.pdig.0000016
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.