To explain or not to explain?—Artificial intelligence explainability in clinical decision support systems
Open accessThe DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.
Abstract
Explainability for artificial intelligence (AI) in medicine is a hotly debated topic. Our paper presents a review of the key arguments in favor and against explainability for AI-powered Clinical Decision Support System (CDSS) applied to a concrete use case, namely an AI-powered CDSS currently used in the emergency call setting to identify patients with life-threatening cardiac arrest. More specifically, we performed a normative analysis using socio-technical scenarios to provide a nuanced account of the role of explainability for CDSSs for the concrete use case, allowing for abstractions to a more general level. Our analysis focused on three layers: technical considerations, human factors, and the designated system role in decision-making. Our findings suggest that whether explainability can provide added value to CDSS depends on several key questions: technical feasibility, the level of validation in case of explainable algorithms, the characteristics of the context in which the system is implemented, the designated role in the decision-making process, and the key user group(s). Thus, each CDSS will require an individualized assessment of explainability needs and we provide an example of how such an assessment could look like in practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures

![Fig 2 Z-Inspection® process. This figure has been reproduced from Zicari RV, Brodersen J, Brusseau J, Düdder B, Eichhorn T, Ivanov T, et al. Z-Inspection®: A Process to Assess Trustworthy AI. IEEE Trans Technol Soc. 2021 Jun;2(2):83–97. [56].](https://figures.cochat.ai/paper-figures/c71a414b2c984bfea375d14b7dd18a96/to-explain-or-not-to-explain-artificial-intelligence-explainability-in-clinical-figure-2.jpg)
![Fig 3 Ethical issue identified during the initial assessment of the use case [31].](https://figures.cochat.ai/paper-figures/4314a4502f78427c86c49fe0ba120ef6/to-explain-or-not-to-explain-artificial-intelligence-explainability-in-clinical-figure-3.jpg)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While interpretability is increasingly recognized as essential for the clinical integration of AI, further prospective and real-world studies are needed to quantify its impact on patient outcomes and define implementation best practices429,430.”
From Artificial intelligence-driven multi-omics approaches in Alzheimer's disease: Progress, challenges, and future directions (2025) · cites this paper as [429]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In particular, the framework in [137] aims at avoiding situations that can have dire consequences for patients.”
From Connecting the dots in trustworthy Artificial Intelligence: From AI principles, ethics, and key requirements to responsible AI systems and regulation (2023) · cites this paper as [137]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The ongoing debate about explainability trade-offs [52,53], building on earlier work [43-46], underscores that selecting suitable XAI methods requires careful consideration of the context-balancing interpretability needs against performance constraints and workflow pressures.”
From Explainable AI for Clinical Decision Support Systems: Literature Review, Key Gaps, and Research Synthesis (2025) · cites this paper as [52]
The cited source says what the claim attributes to it.
In the citing paper
“as Amann et al. (2022) point out, transparency is not one-dimensional: “Explainability is but one measure of achieving transparency; other measures include detailed documentation of datasets and algorithms, as well as open communication of the system’s capabilities and limitations” (Amann et al., 2022).”
From An auditable and source-verified framework for clinical AI decision support: integrating retrieval-augmented generation with data provenance (2026) · cites this paper as Amann et al., 2022
The cited source says what the claim attributes to it.
In the citing paper
“leading studies have repeatedly identified that lack of transparency is a major barrier to AI-CDS adoption (Tun et al., 2025; Amann et al., 2022).”
From An auditable and source-verified framework for clinical AI decision support: integrating retrieval-augmented generation with data provenance (2026) · cites this paper as Amann et al., 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · cited 4×
“An explanation for a black-box model can then, for example, be obtained through an interpretable approximation (see, e.g., SHAP [32] or LIME [33]).”
MethodInfluentialGauriat, C.-M., Pencolé, Y., Ribot, P., & Brouillet, G. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.27 · Cited by 12,586
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 3×
“Due to these limitations in current techniques for generating explanations, some experts suggest the use of inherently transparent models such as regression, decision trees, or rule lists [45,50].”
BackgroundBarredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · cited 1×
“We aimed to capture the various disciplinary perspectives on explainability, ranging from philosophy, to medicine, to computer science, and law [14,17,18,59,60].”
BackgroundThe global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · cited 1×
“An often referenced way to foster trust is to increase the system’s transparency [12], and an important part of increasing transparency is the application of explainability [13].”
BackgroundLipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427 · cited 5×
“When referring to different algorithms that can achieve explainability, there are unfortunately a multitude of different terms in the field that are frequently used interchangeably and without a fixed definition, especially across disciplines [22,25,28].”
BackgroundInfluentialExplanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435 · cited 1×
“This is crucial in the case of no explainability since the dispatcher has no means to identify a wrong prediction when it occurs, and deep learning systems can make predictions with high confidence even when these are faulty [64,65].”
BackgroundTowards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · cited 2×
“Furthermore, explanations have the potential to reveal new insights regarding what the AI system learned from data, and better understand what the algorithm optimized for and the related trade-offs [37].”
BackgroundResultDeep learning for healthcare: review, opportunities and challenges
Miotto, R., Wang, F., Wang, S., Jiang, X., & Dudley, J. T. (2018) · Briefings in Bioinformatics · doi:10.1093/bib/bbx044 · Cited by 3,076
Clinically applicable deep learning for diagnosis and referral in retinal disease
De Fauw, J., Ledsam, J. R., Romera-Paredes, B., Nikolov, S., Tomasev, N., Blackwell, S., Askham, H., Glorot, X., O’Donoghue, B., Visentin, D., et al. (2018) · Nature Medicine · doi:10.1038/s41591-018-0107-6 · Cited by 2,639 · cited 1×
“In laboratory settings, proof-of-concept AI-based CDSSs show promising performance [2,3].”
BackgroundExplaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507
Opportunities and obstacles for deep learning in biology and medicine
Ching, T., Himmelstein, D. S., Beaulieu-Jones, B. K., Kalinin, A. A., Do, B. T., Way, G. P., Ferrero, E., Agapow, P.-M., Zietz, M., Hoffman, M. M., et al. (2018) · Journal of The Royal Society Interface · doi:10.1098/rsif.2017.0387 · Cited by 2,310
Explainability for artificial intelligence in healthcare: a multidisciplinary perspective
the Precise4Q consortium, Amann, J., Blasimme, A., Vayena, E., Frey, D., & Madai, V. I. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01332-6 · Cited by 2,203 · cited 6×
“Some authors even argue that explainability constitutes a normative requirement for medical applications [18].”
BackgroundResultInfluentialSelf-citationEuropean Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131
Causability and explainability of artificial intelligence in medicine
Holzinger, A., Langs, G., Denk, H., Zatloukal, K., & Müller, H. (2019) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1312 · Cited by 1,885 · cited 2×
“Furthermore, what constitutes a good explanation depends on who is interacting with it [18,19,21–24,37] and there is no quantitative way of determining the most useful interaction in advance [19,36,48,49].”
BackgroundLiu, X., Faes, L., Kale, A. U., Wagner, S. K., Fu, D. J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., et al. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30123-2 · Cited by 1,883 · cited 1×
“In laboratory settings, proof-of-concept AI-based CDSSs show promising performance [2,3].”
BackgroundLiu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., Denniston, A. K., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Chan, A.-W., Darzi, A., Holmes, C., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1034-x · Cited by 1,421
Explaining machine learning classifiers through diverse counterfactual explanations
Mothilal, R. K., Sharma, A., & Tan, C. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372850 · Cited by 1,233
Counterfactual explanations without opening the black box: automated decisions and the GDPR
Wachter, S., Mittelstadt, B., & Russell, C. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3063289 · Cited by 1,061
Perkins, G. D., Jacobs, I. G., Nadkarni, V. M., Berg, R. A., Bhanji, F., Biarent, D., Bossaert, L. L., Brett, S. J., Chamberlain, D., de Caen, A. R., et al. (2015) · Circulation · doi:10.1161/cir.0000000000000144 · Cited by 1,031
Human-computer collaboration for skin cancer recognition
Tschandl, P., Rinner, C., Apalla, Z., Argenziano, G., Codella, N., Halpern, A., Janda, M., Lallas, A., Longo, C., Malvehy, J., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0942-0 · Cited by 1,024 · cited 1×
“It has also been shown that providing HCPs with an AI system’s qualified second opinion can increase diagnostic accuracy over that of either the AI system or the HCP alone [38].”
BackgroundCan You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
Ovadia, Y., Fertig, E., Ren, J., Nado, Z., Sculley, D., Nowozin, S., Dillon, J. V., Lakshminarayanan, B., & Snoek, J. (2019) · arXiv · doi:10.48550/arxiv.1906.02530 · Cited by 989 · cited 1×
“This is crucial in the case of no explainability since the dispatcher has no means to identify a wrong prediction when it occurs, and deep learning systems can make predictions with high confidence even when these are faulty [64,65].”
BackgroundClinical decision support in the era of artificial intelligence
Shortliffe, E. H., & Sepúlveda, M. J. (2018) · JAMA · doi:10.1001/jama.2018.17163 · Cited by 920
Designing Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906 · cited 1×
“While there is some guidance on how to satisfy different explanatory needs [71] and attempts at metrics to compare generated explanations [72–74], there is no quantitative way to determine the best type of explanation for a given use case in advance [19,36,48,49].”
BackgroundFooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods
Slack, D., Hilgard, S., Jia, E., Singh, S., & Lakkaraju, H. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375830 · Cited by 900
Markus, A. F., Kors, J. A., & Rijnbeek, P. R. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103655 · Cited by 811
Artificial Intelligence and Black‐Box Medical Decisions: Accuracy versus Explainability
London, A. J. (2019) · Hastings Center Report · doi:10.1002/hast.973 · Cited by 801
Mittelstadt, B., Russell, C., & Wachter, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287574 · Cited by 794
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · cited 5×
“Explainability also enables users to verify that the system does not rely on artifacts or noise in the training data, and can be used to assess whether the system is fair in its decisions, particularly when training data may include a biased or incomplete picture of the population [36].”
BackgroundInfluentialLikelihood Ratios for Out-of-Distribution DetectionPreprint
Ren, J., Liu, P. J., Fertig, E., Snoek, J., Poplin, R. E., DePristo, M. A., Dillon, J. V., & Lakshminarayanan, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02845 · Cited by 727 · cited 1×
“If sufficient representation of minorities cannot be guaranteed, the system should at least detect when it encounters a not sufficiently understood dialect [66,67] to then notify the dispatchers whilst also refraining from making a prediction.”
BackgroundManipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Vaughan, J. W., & Wallach, H. (2018) · arXiv · doi:10.48550/arxiv.1802.07810 · Cited by 711
Langer, M., Oster, D., Speith, T., Hermanns, H., Kästner, L., Schmidt, E., Sesing, A., & Baum, K. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103473 · Cited by 652 · cited 1×
“We aimed to capture the various disciplinary perspectives on explainability, ranging from philosophy, to medicine, to computer science, and law [14,17,18,59,60].”
BackgroundWhat do we need to build explainable AI systems for the medical domain?
Holzinger, A., Biemann, C., Pattichis, C. S., & Kell, D. B. (2017) · arXiv · doi:10.48550/arxiv.1712.09923 · Cited by 636
Explainable machine learning in deployment
Bhatt, U., Xiang, A., Sharma, S., Weller, A., Taly, A., Jia, Y., Ghosh, J., Puri, R., Moura, J. M. F., & Eckersley, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375624 · Cited by 633 · cited 2×
“Explaining the inner workings of AI systems can facilitate finding potential flaws and identify the root causes of errors more efficiently [34].”
BackgroundA Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629
Beede, E., Baylor, E., Hersch, F., Iurchenko, A., Wilcox, L., Ruamviboonsuk, P., & Vardoulakis, L. M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376718 · Cited by 614
Cai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359206 · Cited by 594 · cited 1×
“When looking at the third layer, we consider that explainability can promote the use of the system as originally intended, namely as a CDSS that promotes algorithm-based decision making [75].”
BackgroundMurphy, D. J., Burrows, D., Santilli, S., Kemp, A. W., Tenner, S., Kreling, B., & Teno, J. (1994) · New England Journal of Medicine · doi:10.1056/nejm199402243300807 · Cited by 580 · cited 3×
“If OHCA is detected, the dispatcher instructs the caller to perform chest compressions (CPR), as every minute without CPR reduces the chances for successful resuscitation and drastically increases the risk for permanent complications, like brain damage.”
BackgroundOpportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey
Das, A., & Rad, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11371 · Cited by 504
Clinical Decision-Support Systems
Musen, M. A., Middleton, B., & Greenes, R. A. (2014) · Biomedical Informatics · doi:10.1007/978-1-4471-4474-8_22 · Cited by 483
Durán, J. M., & Jongsma, K. R. (2021) · Journal of Medical Ethics · doi:10.1136/medethics-2020-106820 · Cited by 446 · cited 3×
“While the decision process of these models is more interpretable for end-users than that of, for example, a deep neural network (DNN), these models can only be considered interpretable as long as they follow certain limitations, such as with respect to model complexity or number and type of input features, which begs…”
BackgroundRudin, C., & Radin, J. (2019) · Harvard Data Science Review · doi:10.1162/99608f92.5a8a3a3d · Cited by 432
Holzinger, A., Carrington, A., & Müller, H. (2020) · KI - Künstliche Intelligenz · doi:10.1007/s13218-020-00636-z · Cited by 406
Health information technology: fallacies and sober realities
Karsh, B.-T., Weinger, M. B., Abbott, P. A., & Wears, R. L. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.005637 · Cited by 365
A Benchmark for Interpretability Methods in Deep Neural Networks
Hooker, S., Erhan, D., Kindermans, P.-J., & Kim, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.10758 · Cited by 365
Kittler, H., Marghoob, A. A., Argenziano, G., Carrera, C., Curiel-Lewandrowski, C., Hofmann-Wellenhof, R., Malvehy, J., Menzies, S., Puig, S., Rabinovitz, H., et al. (2016) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2015.12.038 · Cited by 319 · cited 1×
“An additional argument against explainability is that, in some cases, a universal ground truth that HCPs agree on might not exist [24,41].”
BackgroundWhat Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use
Tonekaboni, S., Joshi, S., McCradden, M. D., & Goldenberg, A. (2019) · arXiv · doi:10.48550/arxiv.1905.05134 · Cited by 318 · cited 3×
“In such cases, a common proposition for fostering the users’ trust in the system is to include them more actively in the design and evaluation process [13,19,24,35].”
BackgroundBeware explanations from AI in health care
Babic, B., Gerke, S., Evgeniou, T., & Cohen, I. G. (2021) · Science · doi:10.1126/science.abg1834 · Cited by 252 · cited 1×
“In this case, it is necessary to carefully weigh the advantages and disadvantages of different techniques for generating explanations, as well as to analyze which of these techniques can be understood by end users [76].”
BackgroundSelf-citationYang, Q., Steinfeld, A., & Zimmerman, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300468 · Cited by 247
"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations
Lakkaraju, H., & Bastani, O. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375833 · Cited by 241
Fundamentals of Clinical Data Science
(2019) · Springer eBooks · doi:10.1007/978-3-319-99713-1 · Cited by 235
Efficient Search for Diverse Coherent Explanations
Russell, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287569 · Cited by 218
Miller, T., Howe, P., & Sonenberg, L. (2017) · arXiv · doi:10.48550/arxiv.1712.00547 · Cited by 218 · cited 2×
“Based on previous works [13,18,21,22,37] and the results of the current analysis, we argue that if well-validated and methodologically sound explanations that are intuitive and easy to grasp for users were readily available for a black-box system, there would be no valid argument against their inclusion.”
BackgroundResultMachine learning as a supportive tool to recognize cardiac arrest in emergency calls
Blomberg, S. N., Folke, F., Ersbøll, A. K., Christensen, H. C., Torp-Pedersen, C., Sayre, M. R., Counts, C. R., & Lippert, F. K. (2019) · Resuscitation · doi:10.1016/j.resuscitation.2019.01.015 · Cited by 209
Self-citation"The human body is a black box": supporting clinical decision-making with deep learning
Sendak, M., Elish, M. C., Gao, M., Futoma, J., Ratliff, W., Nichols, M., Bedoya, A., Balu, S., & O'Brien, C. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372827 · Cited by 196 · cited 4×
“An additional argument against explainability is that, in some cases, a universal ground truth that HCPs agree on might not exist [24,41].”
BackgroundInfluentialEvaluating and Aggregating Feature-based Model Explanations
Bhatt, U., Weller, A., & Moura, J. M. F. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/417 · Cited by 176
A Misdirected Principle with a Catch: Explicability for AI
Robbins, S. (2019) · Minds and Machines · doi:10.1007/s11023-019-09509-3 · Cited by 174
Visualizing and Understanding Neural Models in NLPPreprint
Li, J., Chen, X., Hovy, E. H., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.01066 · Cited by 165
The four dimensions of contestable AI diagnostics - A patient-centric approach to explainable AI
Ploug, T., & Holm, S. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.101901 · Cited by 148
Self-citationGrounds for Trust: Essential Epistemic Opacity and Computational Reliabilism
Durán, J. M., & Formanek, N. (2018) · Minds and Machines · doi:10.1007/s11023-018-9481-6 · Cited by 138
Blomberg, S. N., Christensen, H. C., Lippert, F., Ersbøll, A. K., Torp-Petersen, C., Sayre, M. R., Kudenchuk, P. J., & Folke, F. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.32320 · Cited by 138 · cited 5×
“In Copenhagen, OHCA related emergency calls are rare, comprising less than 1% of all calls to the 112 emergency line [58].”
BackgroundMethodInfluentialSelf-citationFour Principles of Explainable Artificial Intelligence
Phillips, P. J., Hahn, C. A., Fontana, P. C., Broniatowski, D. A., & Przybocki, M. A. (2020) · doi:10.6028/nist.ir.8312-draft · Cited by 136 · cited 5×
“It is also possible to craft misleading explanations that look trustworthy to the user and that, while they technically reflect the model’s decision process with high accuracy, omit parts of the explanation and ultimately decrease the level of trust in the model [42–44] by allowing vendors to use explainability to “fa…”
BackgroundResultInfluentialZ-Inspection ® : A Process to Assess Trustworthy AI
Zicari, R. V., Brodersen, J., Brusseau, J., Dudder, B., Eichhorn, T., Ivanov, T., Kararigas, G., Kringen, P., McCullough, M., Moslein, F., et al. (2021) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2021.3066209 · Cited by 134 · cited 2×
“), served as the basis for an interdisciplinary assessment of system trustworthiness following the Z-Inspection1 process [31,56], which is guided by the “Ethics Guidelines for Trustworthy AI” by the European Commission’s AI HLEG [13].”
BackgroundMethodSelf-citationRobust Out-of-distribution Detection for Neural NetworksPreprint
Chen, J., Li, Y., Wu, X., Liang, Y., & Jha, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.09711 · Cited by 88
Benchmarking Attribution Methods with Relative Feature Importance
Yang, M., & Kim, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.09701 · Cited by 83
Zicari, R. V., Brusseau, J., Blomberg, S. N. F., Christensen, H. C., Coffee, M. P., Ganapini, M. B., Gerke, S., Gilbert, T. K., Hickman, E., Hildt, E., et al. (2021) · Frontiers in Human Dynamics · doi:10.3389/fhumd.2021.673104 · Cited by 79 · cited 6×
“During the assessment, a lack of trust was identified as a possible cause [31].”
BackgroundMethodInfluentialSelf-citationDemystifying Black-box Models with Symbolic Metamodels
Alaa, A. M., & van der Schaar, M. (2019) · neural information processing systems · Cited by 55
Wnent, J., Masterson, S., Gräsner, J.-T., Böttiger, B. W., Herlitz, J., Koster, R. W., Rosell Ortiz, F., Tjelmeland, I., Maurer, H., & Bossaert, L. (2015) · Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine · doi:10.1186/s13049-015-0093-3 · Cited by 52 · cited 5×
“The chat protocol and transcripts of the workshop, as well as relevant publications [57,58] made available by the initiators of the technology (coauthors S.N.B., H.C.C.), served as the basis for an interdisciplinary assessment of system trustworthiness following the Z-Inspection 1 process [31,56], which is guided by t…”
BackgroundInfluentialMachine Learning Explainability for External Stakeholders
Bhatt, U., Andrus, M., Weller, A., & Xiang, A. (2020) · arXiv · doi:10.48550/arxiv.2007.05408 · Cited by 41 · cited 3×
“In such cases, a common proposition for fostering the users’ trust in the system is to include them more actively in the design and evaluation process [13,19,24,35].”
BackgroundOlsen, H. P., Slosser, J. L., Hildebrandt, T. T., & Wiesener, C. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3402974 · Cited by 28
Research works printed but not linked: 4
- ZC Lipton, 2018, The mythos of model interpretability, Commun ACM, 61, 36, 10.1145/3233231 · doi:10.1145/3233231 · Paper page
- Karimi A-H, Barthe G, Balle B, Valera I. Model-Agnostic Counterfactual Explanations for Consequential Decisions. In: International Conference on Artificial Intelligence and Statistics [Internet]. PMLR; 2020 [cited 2021 Jun 15]. p. 895–905. Available from: http://proceedings.mlr.press/v108/karimi20a.html
- Aubakirova M, Bansal M. Interpreting Neural Networks to Improve Politeness Comprehension. ArXiv161002683 Cs [Internet]. 2016 Oct 9 [cited 2021 Apr 13]; Available from: http://arxiv.org/abs/1610.02683 · arXiv
- Karlekar S, Niu T, Bansal M. Detecting Linguistic Characteristics of Alzheimer’s Dementia by Interpreting Neural Models. ArXiv180406440 Cs [Internet]. 2018 Apr 17 [cited 2021 Apr 13]; Available from: http://arxiv.org/abs/1804.06440 · arXiv
Cited by
Díaz-Rodríguez, N., Del Ser, J., Coeckelbergh, M., López de Prado, M., Herrera-Viedma, E., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101896 · Cited by 800
A Scoping Review of Reporting Gaps in FDA-Approved AI Medical Devices
Muralidharan, V., Adewale, B. A., Huang, C. J., Nta, M. T., Ademiju, P. O., Pathmarajah, P., Hang, M. K., Adesanya, O., Abdullateef, R. O., Babatunde, A. O., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01270-x · Cited by 201
Gala, D., Behl, H., Shah, M., & Makaryus, A. N. (2024) · Healthcare · doi:10.3390/healthcare12040481 · Cited by 183
“Explainable AI methods aim to provide transparency in AI-driven decision making, offering insights into how algorithms arrive at specific conclusions [61].”
Solving the Explainable AI Conundrum by Bridging Clinicians’ Needs and Developers’ Goals
Bienefeld, N., Boss, J. M., Lüthy, R., Brodbeck, D., Azzati, J., Blaser, M., Willms, J., & Keller, E. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00837-4 · Cited by 159
Caruccio, L., Cirillo, S., Polese, G., Solimando, G., Sundaramurthy, S., & Tortora, G. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121186 · Cited by 153
Metadata
- Authors
- Amann, J., Vetter, D., Blomberg, S. N., Christensen, H. C., Coffee, M., Gerke, S., Gilbert, T. K., Hagendorff, T., Holm, S., Livne, M., Spezzatti, A., Strümke, I., Zicari, R. V., Madai, V. I., & initiative, O. B. O. T. Z.-I.
- Affiliations
- ETH Zurich (CH)
- Goethe University Frankfurt (DE)
- University of Copenhagen (DK)
- New York University (US)
- Pennsylvania State University (US)
- Cornell Tech (US)
- University of Tübingen (DE)
- Google (United Kingdom) (GB)
- University of California, Berkeley (US)
- Norwegian University of Science and Technology (NO)
- Seoul National University (KR)
- Birmingham City University (GB)
- Corresponding author
- Amann, Julia
- ORCID
- Venue
- PLOS Digital Health
- Publisher
- Public Library of Science (PLoS)
- Published
- 2022-02-17
- Volume
- 1
- Issue
- 2
- Pages
- e0000016
- ISSN
- 2767-3170 (ISSN-L)
- PMC
- PMC9931364
- OpenAlex
- W4212942200
- Cited by
- 251
- References
- 62
- Citations by year
- 2026: 75 · 2025: 70 · 2024: 55 · 2023: 35 · 2022: 16
- FWCI
- 14.49
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLOS Digital Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research at the University of Copenhagen (University of Copenhagen) (publishedVersion, cc-by)
- eYLS (Yale Law School) (submittedVersion, cc-by)
- eYLS (Yale Law School) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- NORA - Norwegian Open Research Archives (submittedVersion, cc-by)
- Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by)
- ETH Zürich Research Collection
- Funders
- Horizon 2020 Framework Programme (777107, 101016233, H2020-SC1-PHE-CORONAVIRUS-2020-2-RTD)
- Connecting Europe Facility (INEA/CEF/ICT/A2020/2276680, 2020-EU-IA-0098)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- clinical decision support systems, cardiac arrest prediction, normative analysis, human factors, explainable algorithms, user groups
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.