The ethical requirement of explainability for AI-DSS in healthcare: a systematic review of reasons
Open accessThe DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Abstract
Background: Despite continuous performance improvements, especially in clinical contexts, a major challenge of Artificial Intelligence based Decision Support Systems (AI-DSS) remains their degree of epistemic opacity. The conditions of and the solutions for the justified use of the occasionally unexplainable technology in healthcare are an active field of research. In March 2024, the European Union agreed upon the Artificial Intelligence Act (AIA), requiring medical AI-DSS to be ad-hoc explainable or to use post-hoc explainability methods. The ethical debate does not seem to settle on this requirement yet. This systematic review aims to outline and categorize the positions and arguments in the ethical debate. Methods: We conducted a literature search on PubMed, BASE, and Scopus for English-speaking scientific peer-reviewed publications from 2016 to 2024. The inclusion criterion was to give explicit requirements of explainability for AI-DSS in healthcare and reason for it. Non-domain-specific documents, as well as surveys, reviews, and meta-analyses were excluded. The ethical requirements for explainability outlined in the documents were qualitatively analyzed with respect to arguments for the requirement of explainability and the required level of explainability. Results: The literature search resulted in 1662 documents; 44 documents were included in the review after eligibility screening of the remaining full texts. Our analysis showed that 17 records argue in favor of the requirement of explainable AI methods (xAI) or ad-hoc explainable models, providing 9 categories of arguments. The other 27 records argued against a general requirement, providing 11 categories of arguments. Also, we found that 14 works advocate the need for context-dependent levels of explainability, as opposed to 30 documents, arguing for context-independent, absolute standards. Conclusions: The systematic review of reasons shows no clear agreement on the requirement of post-hoc explainability methods or ad-hoc explainable models for AI-DSS in healthcare. The arguments found in the debate were referenced and responded to from different perspectives, demonstrating an interactive discourse. Policymakers and researchers should watch the development of the debate closely. Conversely, ethicists should be well informed by empirical and technical research, given the frequency of advancements in the field.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283
Rayyan-a web and mobile app for systematic reviews.
Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0384-4 · Cited by 26,092 · cited 1×
“We conducted the literature screening using Rayyan 2 for abstract screening and duplicate detection [16].”
MethodPage, M. J., Moher, D., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n160 · Cited by 13,262 · cited 2×
“We conducted a systematic review of reasons to characterize key presumptions and motivations in the debate and to identify the normative standards of explainability according to the PRISMA standards [13, 14].”
MethodFloridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., et al. (2018) · Minds and Machines · doi:10.1007/s11023-018-9482-5 · Cited by 4,774 · cited 1×
“While formerly mostly a matter of debate for the philosophy of science, the ethical debate around this concern most famously took off with the AI4People framework by Floridi et al. in 2018, introducing the principle of explicability in the AI-ethics context [8].”
BackgroundGusenbauer, M., & Haddaway, N. R. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1378 · Cited by 2,379 · cited 1×
“The literature databases used are the Bielefeld Academic Search Engine (BASE), PubMed, and Scopus, as they cover the relevant domains and offer transparent and reproducible search options and results [15].”
MethodExplainability for artificial intelligence in healthcare: a multidisciplinary perspective
the Precise4Q consortium, Amann, J., Blasimme, A., Vayena, E., Frey, D., & Madai, V. I. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01332-6 · Cited by 2,203
Liu, X., Faes, L., Kale, A. U., Wagner, S. K., Fu, D. J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., et al. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30123-2 · Cited by 1,883 · cited 1×
“Deep-learning-based AI-DSS impress with performance [3].”
Machine learning in medicine: Addressing ethical challenges
Vayena, E., Blasimme, A., & Cohen, I. G. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002689 · Cited by 912 · cited 1×
“…duty to explain risks and benefits is satisfiable by the disclosure of risks inherited by the AI-DSS at use [36, 38], or the rationale for adopting AI-DSS in general [39], contradicting Vayena, who claims that the communication meaningful details was “fundamental tenet of medical ethics” [62].”
BackgroundArtificial Intelligence and Black‐Box Medical Decisions: Accuracy versus Explainability
London, A. J. (2019) · Hastings Center Report · doi:10.1002/hast.973 · Cited by 801 · cited 6×
“With an outcome-oriented perspective in favor of clinical benefits, this is a reason for some to avoid explainability as a default requirement [1, 24, 31].”
BackgroundInfluentialMittelstadt, B., Russell, C., & Wachter, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287574 · Cited by 794 · cited 1×
“In the literature, different types of explanations in scientific context were elaborated by philosophers of science for centuries [18, 19].”
BackgroundDurán, J. M., & Jongsma, K. R. (2021) · Journal of Medical Ethics · doi:10.1136/medethics-2020-106820 · Cited by 446 · cited 6×
“…as well (double standard argument) [1, 23–25] Post-Hoc explainability methods add new levels of uncertainty and may cause false confidence [23, 26–30] There can be a trade-off between accuracy and explainability [1, 24, 31] Explainability is not required to resolve problems of responsibility…”
BackgroundInfluentialTransparency in Algorithmic and Human Decision-Making: Is There a Double Standard?
Zerilli, J., Knott, A., Maclaurin, J., & Gavaghan, C. (2019) · Philosophy & Technology · doi:10.1007/s13347-018-0330-6 · Cited by 420 · cited 1×
“Thus, the explainability of an AI-DSS is an attribute describing the ability of either the AI-DSS to give explanations for its decision-making or an agent to explain the decision-making of an AI-DSS to a certain degree [20, 21].”
BackgroundThe philosophical novelty of computer simulation methods
Humphreys, P. (2009) · Synthese · doi:10.1007/s11229-008-9435-2 · Cited by 362 · cited 2×
“Humphrey defines the epistemic opacity of a computational system as the inacces-sibility of its processes and attributes [7].”
BackgroundDefining the undefinable: the black box problem in healthcare artificial intelligence
Wadden, J. J. (2022) · Journal of Medical Ethics · doi:10.1136/medethics-2021-107529 · Cited by 224 · cited 3×
“By ensuring that patients fully comprehend the implications of AI-DSS recommendations, they can actively participate in the decision-making process [10, 50].”
BackgroundThe three ghosts of medical AI: Can the black-box present deliver?
Quinn, T. P., Jacobs, S., Senadeera, M., Le, V., & Coghlan, S. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2021.102158 · Cited by 215
Gartlehner, G., Affengruber, L., Titscher, V., Noel-Storr, A., Dooley, G., Ballarini, N., & König, F. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.01.005 · Cited by 206 · cited 1×
“A study by Gartlehner et al. from 2020 showed that single screening may miss up to 13% of relevant records [17].”
BackgroundHow to write a systematic review of reasons
Strech, D., & Sofaer, N. (2012) · Journal of Medical Ethics · doi:10.1136/medethics-2011-100096 · Cited by 168 · cited 2×
“As proposed by [12], we categorized the normative standards on the requirement of xAI and ad-hoc explain-ability made in the literature in conjunction with reasons.”
MethodThe four dimensions of contestable AI diagnostics - A patient-centric approach to explainable AI
Ploug, T., & Holm, S. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.101901 · Cited by 148 · cited 3×
“…methods add new levels of uncertainty and may cause false confidence [23, 26–30] There can be a trade-off between accuracy and explainability [1, 24, 31] Explainability is not required to resolve problems of responsibility [30, 32, 33] Explainability is not required and not sufficient for the…”
BackgroundScientific Explanation: Three Basic Conceptions
Salmon, W. C. (1984) · PSA Proceedings of the Biennial Meeting of the Philosophy of Science Association · doi:10.1086/psaprocbienmeetp.1984.2.192510 · Cited by 127 · cited 1×
“In the literature, different types of explanations in scientific context were elaborated by philosophers of science for centuries [18, 19].”
BackgroundGraziani, M., Dutkiewicz, L., Calvaresi, D., Amorim, J. P., Yordanova, K., Vered, M., Nair, R., Abreu, P. H., Blanke, T., Pulignano, V., et al. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10256-8 · Cited by 126 · cited 2×
“At the same time, the epistemic opacity of deep learning techniques motivates a vast and ongoing debate on the ethical permissibility of AI-DSS in healthcare, characterized by terminological incoherences across disciplines [10, 11].”
BackgroundWalsh, C. G., Chaudhry, B., Dua, P., Goodman, K. W., Kaplan, B., Kavuluru, R., Solomonides, A., & Subbian, V. (2020) · JAMIA Open · doi:10.1093/jamiaopen/ooz054 · Cited by 118 · cited 2×
“…AI-DSS require randomized clinical trials [36], interdisciplinary dialogues between developers and HCPs [36], monitoring public preferences [39], and additional technical training for HCPs [36] or users in general [37] on the general working of AI-DSS rather than explainable models [36].”
BackgroundMcCoy, L. G., Brenna, C. T. A., Chen, S. S., Vold, K., & Das, S. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.11.001 · Cited by 118 · cited 1×
“…for AI-DSS to be ethi-cally permissible in healthcare Medical decisions are commonly atheoretic as well (double standard argument) [1, 23–25] Post-Hoc explainability methods add new levels of uncertainty and may cause false confidence [23, 26–30] There can be a trade-off between…”
BackgroundAstromskė, K., Peičius, E., & Astromskis, P. (2020) · AI & Society · doi:10.1007/s00146-020-01008-9 · Cited by 116 · cited 4×
“Analogously, others argue that trustworthy AI-DSS require randomized clinical trials [36], interdisciplinary dialogues between developers and HCPs [36], monitoring public preferences [39], and additional technical training for HCPs [36] or users in general [37] on the general working of AI-DSS…”
BackgroundInfluentialMachine learning in medicine: should the pursuit of enhanced interpretability be abandoned?
Yoon, C. H., Torrance, R., & Scheinerman, N. (2022) · Journal of Medical Ethics · doi:10.1136/medethics-2020-107102 · Cited by 116 · cited 4×
“A widely accepted argument for the requirement of explainability is given by its value in finding and mitigating biases in AI-DSS [50, 52, 53, 57].”
BackgroundInfluentialInterpretable, not black-box, artificial intelligence should be used for embryo selection
Afnan, M. A. M., Liu, Y., Conitzer, V., Rudin, C., Mishra, A., Savulescu, J., & Afnan, M. (2021) · Human Reproduction Open · doi:10.1093/hropen/hoab040 · Cited by 112 · cited 6×
“Out of these, 3 explicitly advocate for ad-hoc explainable models [51, 57, 58].”
BackgroundMethodInfluentialMcCradden, M. D., Joshi, S., Anderson, J. A., Mazwi, M., Goldenberg, A., & Zlotnik Shaul, R. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa085 · Cited by 111 · cited 5×
“Moreover, while Theunissen and Browning as well as McCradden et al. acknowledge that the use of xAI methods may be valuable in the development of AI-DSS to avoid biases, in a dynamic clinical context they demand regular auditing and appropriate proxies to detect biases in practice [34, 35].”
BackgroundInfluentialAI in the hands of imperfect users
Kostick-Quenet, K. M., & Gerke, S. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00737-z · Cited by 101 · cited 2×
“…as well (double standard argument) [1, 23–25] Post-Hoc explainability methods add new levels of uncertainty and may cause false confidence [23, 26–30] There can be a trade-off between accuracy and explainability [1, 24, 31] Explainability is not required to resolve problems of responsibility…”
BackgroundMethodYour evidence? Machine learning algorithms for medical diagnosis and prediction
Heinrichs, B., & Eickhoff, S. B. (2020) · Human Brain Mapping · doi:10.1002/hbm.24886 · Cited by 99 · cited 4×
“They emphasize that shared decision-making is compromised by the HCPs’ and the patients’ inability to understand AI-DSS recommendations [51, 56, 60].”
BackgroundInfluentialArtificial intelligence in medicine and the disclosure of risks
Kiener, M. (2021) · AI & SOCIETY · doi:10.1007/s00146-020-01085-w · Cited by 97 · cited 3×
“Therefore, based on their perspective, AI-DSS could be implemented even without providing explanations for decision-making or allowing HCPs or patients to explain their decision-making [1, 23–38], sometimes depending on the context [2, 39–47].”
BackgroundEvidence, ethics and the promise of artificial intelligence in psychiatry
McCradden, M., Hui, K., & Buchman, D. Z. (2023) · Journal of Medical Ethics · doi:10.1136/jme-2022-108447 · Cited by 97 · cited 2×
“In other words, they produce a false sense of security, but in fact introduce new layers of uncertainty [30] and harbor the risk of amplifying automation biases [23, 27–29].”
Background“Just” accuracy? Procedural fairness demands explainability in AI-based medical resource allocations
Rueda, J., Delgado, J., Jounou, I. P., Hortal-Carmona, J., Ausín, T., & Rodríguez‐Arias, D. (2022) · AI & Society · doi:10.1007/s00146-022-01614-9 · Cited by 97 · cited 5×
“In 13 records, the authors argue that the required level of explainability depends on either the contemporary best practices [2, 39, 40], the normative reach of the decisions [40–45] and potential otherwise infeasible benefits [2, 46, 47, 61], or on the patients’ or HCPs’ values [40, 42, 61–63].”
BackgroundInfluentialWhen Doctors and AI Interact: on Human Responsibility for Artificial Risks
Verdicchio, M., & Perin, A. (2022) · Philosophy & Technology · doi:10.1007/s13347-022-00506-6 · Cited by 90
Kempt, H., & Nagel, S. K. (2022) · Journal of Medical Ethics · doi:10.1136/medethics-2021-107440 · Cited by 87 · cited 2×
“…can be a trade-off between accuracy and explainability [1, 24, 31] Explainability is not required to resolve problems of responsibility [30, 32, 33] Explainability is not required and not sufficient for the detection of biases [23, 34, 35] Trust, acceptance, and uptake are feasible by…”
BackgroundRe-focusing explainability in medicine
Arbelaez Ossa, L., Starke, G., Lorenzini, G., Vogt, J. E., Shaw, D. M., & Elger, B. S. (2022) · DIGITAL HEALTH · doi:10.1177/20552076221074488 · Cited by 87
Herington, J., McCradden, M. D., Creel, K., Boellaard, R., Jones, E. C., Jha, A. K., Rahmim, A., Scott, P. J. H., Sunderland, J. J., Wahl, R. L., et al. (2023) · Journal of Nuclear Medicine · doi:10.2967/jnumed.123.266080 · Cited by 87 · cited 1×
“Herington et al. posit that complete causal explanations are not necessary and not feasible for informed consent, as only a certain level of understanding may be presupposed by the average patient [27, 28].”
BackgroundEthics of artificial intelligence in global health: Explainability, algorithmic bias and trust
Kerasidou, A. (2021) · Journal of Oral Biology and Craniofacial Research · doi:10.1016/j.jobcr.2021.09.004 · Cited by 79 · cited 2×
“Therefore, based on their perspective, AI-DSS could be implemented even without providing explanations for decision-making or allowing HCPs or patients to explain their decision-making [1, 23–38], sometimes depending on the context [2, 39–47].”
BackgroundBlack-box assisted medical decisions: AI power vs. ethical physician care
Chan, B. (2023) · Medicine, Health Care and Philosophy · doi:10.1007/s11019-023-10153-z · Cited by 79
The EU passes the AI Act and its implications for digital medicine are unclear
Gilbert, S. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01116-6 · Cited by 68 · cited 1×
“The AIA defines all medical devices as defined in the medical device regulation or the in-vitro medical device regulation as ‘high-risk’ systems (cf. Annex II of the AIA [6] and Gilbert’s analysis [9]).”
BackgroundEthical Considerations for Artificial Intelligence in Medical Imaging: Deployment and Governance
Herington, J., McCradden, M. D., Creel, K., Boellaard, R., Jones, E. C., Jha, A. K., Rahmim, A., Scott, P. J. H., Sunderland, J. J., Wahl, R. L., et al. (2023) · Journal of Nuclear Medicine · doi:10.2967/jnumed.123.266110 · Cited by 61 · cited 2×
“In other words, they produce a false sense of security, but in fact introduce new layers of uncertainty [30] and harbor the risk of amplifying automation biases [23, 27–29].”
BackgroundThe Virtues of Interpretable Medical AI
Hatherley, J. J., Sparrow, R., & Howard, M. (2023) · Cambridge Quarterly of Healthcare Ethics · doi:10.1017/s0963180122000664 · Cited by 49
An extension of the theory of technology dominance: Capturing the underlying causal complexity
Sutton, S. G., Arnold, V., & Holt, M. (2023) · International Journal of Accounting Information Systems · doi:10.1016/j.accinf.2023.100626 · Cited by 46 · cited 1×
“In the extended theory of technology dominance, describing the effects of automation to deskilling, system transparency is taken to be an important decreasing factor [64].”
BackgroundDefending explicability as a principle for the ethics of artificial intelligence in medicine
Adams, J. (2023) · Medicine, Health Care and Philosophy · doi:10.1007/s11019-023-10175-7 · Cited by 42 · cited 5×
“…of AI-DSS is more likely to be confounded than for the analogies given [52] because HCPs’ decisions could be broadly explained by their social environment [48], or because other than opaque AI-DSS, HCPs could provide some useful information that that facilitate their decision-making [61].”
BackgroundInfluentialRelative explainability and double standards in medical decision-making
Kempt, H., Heilinger, J., & Nagel, S. K. (2022) · Ethics and Information Technology · doi:10.1007/s10676-022-09646-x · Cited by 40 · cited 2×
“Thus, the explainability of an AI-DSS is an attribute describing the ability of either the AI-DSS to give explanations for its decision-making or an agent to explain the decision-making of an AI-DSS to a certain degree [20, 21].”
BackgroundComing to Terms with the Black Box Problem: How to Justify AI Systems in Health Care
Felder, R. M. (2021) · The Hastings Center Report · doi:10.1002/hast.1248 · Cited by 38 · cited 2×
“…trade-off is only claimed but not substantiated [52, 58] Explainability is a requirement for accountability or the attribution of responsibility [45, 48, 51, 56, 59, 60] Explainability can help to find biases [50, 52, 53, 57] A lack of explainability threatens trust, acceptance, and uptake [34,…”
BackgroundPutting explainable AI in context: institutional explanations for medical AI
Theunissen, M., & Browning, J. (2022) · Ethics and Information Technology · doi:10.1007/s10676-022-09649-8 · Cited by 37 · cited 4×
“In terms of trust, Theunissen and Browning argue that informed consent is not or only partially based on adequately informing about risks, precautions, or benefits but foremost on trust in the institution of medicine [35].”
BackgroundInfluentialEthical issues in computational pathology
Sorell, T., Rajpoot, N., & Verrill, C. L. (2021) · Journal of Medical Ethics · doi:10.1136/medethics-2020-107024 · Cited by 35 · cited 2×
“…add new levels of uncertainty and may cause false confidence [23, 26–30] There can be a trade-off between accuracy and explainability [1, 24, 31] Explainability is not required to resolve problems of responsibility [30, 32, 33] Explainability is not required and not sufficient for the…”
BackgroundDiaz-Asper, C., Hauglid, M. K., Chandler, C., Cohen, A. S., Foltz, P. W., & Elvevåg, B. (2024) · American Psychologist · doi:10.1037/amp0001195 · Cited by 35 · cited 2×
“A major argument for that position was that the more nor-matively far-reaching the decisions for which the recommendations of an AI-DSS are used, the higher the need for justification and, thus, explanations [40–42 , 44].”
BackgroundFuner, F. (2022) · Medicine, Health Care and Philosophy · doi:10.1007/s11019-022-10076-1 · Cited by 34 · cited 1×
“A major argument for that position was that the more nor-matively far-reaching the decisions for which the recommendations of an AI-DSS are used, the higher the need for justification and, thus, explanations [40–42 , 44].”
BackgroundAn Overview of Explainable AI Methods, Forms and Frameworks
Kumar, D., & Mehta, M. A. (2022) · Intelligent systems reference library · doi:10.1007/978-3-031-12807-3_3 · Cited by 32
Ursin, F., Lindner, F., Ropinski, T., Salloch, S., & Timmermann, C. (2023) · Ethik in der Medizin · doi:10.1007/s00481-023-00761-x · Cited by 32 · cited 2×
“A major argument for that position was that the more nor-matively far-reaching the decisions for which the recommendations of an AI-DSS are used, the higher the need for justification and, thus, explanations [40–42 , 44].”
BackgroundHolm, S. (2023) · Cambridge Quarterly of Healthcare Ethics · doi:10.1017/s0963180123000294 · Cited by 26 · cited 4×
“Finally, it was argued that the acceptance of AI-DSS in healthcare by patients [58] or HCPs [56] requires explanations.”
BackgroundInfluentialJustice and the Normative Standards of Explainability in Healthcare
Kempt, H., Freyer, N., & Nagel, S. K. (2022) · Philosophy & Technology · doi:10.1007/s13347-022-00598-0 · Cited by 15 · cited 3×
“Using Artificial Intelligence based Decision Support Systems (AI-DSS) in healthcare applications appears valuable by increasing accessibility, precision, and speed of medical decision-making [1, 2] – provided that their use is critically reflected upon.”
BackgroundSelf-citationExplainability, Public Reason, and Medical Artificial Intelligence
Silva, M. D. (2023) · Ethical Theory and Moral Practice · doi:10.1007/s10677-023-10390-4 · Cited by 15 · cited 6×
“Therefore, based on their perspective, AI-DSS could be implemented even without providing explanations for decision-making or allowing HCPs or patients to explain their decision-making [1, 23–38], sometimes depending on the context [2, 39–47].”
BackgroundInfluentialWadden, J. J. (2021) · Canadian Journal of Bioethics · doi:10.7202/1077636ar · Cited by 13 · cited 3×
“…of AI-DSS is more likely to be confounded than for the analogies given [52] because HCPs’ decisions could be broadly explained by their social environment [48], or because other than opaque AI-DSS, HCPs could provide some useful information that that facilitate their decision-making [61].”
BackgroundNo-boundary thinking: a viable solution to ethical data-driven AI in precision medicine
Obafemi-Ajayi, T., Perkins, A., Nanduri, B., Wunsch, D. C., Foster, J. A., & Peckham, J. (2021) · AI and Ethics · doi:10.1007/s43681-021-00118-4 · Cited by 12
Research works printed but not linked: 6
- Kempt H, Heilinger JC, Nagel SK. I’m afraid I can’t let you do that, Doctor: meaningful disagreements with AI in medical contexts. AI Soc. 2022; https://www.scopus.com/inward/record.uri?eid=2-s2.0-85126268667&doi=10.1007%2fs00146-022-01418-x&partnerID=40&md5=1119b36454a9397335a9731ba4fe1b50 · DOI · Paper page
- Funer F. Accuracy and Interpretability: Struggling with the Epistemic Foundations of Machine Learning-Generated Medical Information and Their Practical Implications for the Doctor-Patient Relationship. Philos Technol. 2022;35(1). https://www.scopus.com/inward/record.uri?eid=2-s2.0-85124017187&doi=10.1007%2fs13347-022-00505-7&partnerID=40&md5=7537e359b13d6a783c60e4d4e5141902 · DOI
- Durán JM. Dissecting scientific explanation in AI (sXAI): A case for medicine and healthcare. Artif Intell. 2021;297. https://www.scopus.com/inward/record.uri?eid=2-s2.0-85102628246&doi=10.1016%2fj.artint.2021.103498&partnerID=40&md5=4019df5e11a4bb3dbb783e60842645cd · DOI · Paper page
- Grote T. Machine learning in healthcare and the methodological priority of epistemology over ethics. Inq U K. 2024; https://www.scopus.com/inward/record.uri?eid=2-s2.0-85184172601&doi=10.1080%2f0020174X.2024.2312207&partnerID=40&md5=a7294806595ee3e2c7bcbd73c98c4883 · DOI
- Riva G, Sajno E, Pupillo DEGASPARIS. C. Navigating the Ethical Crossroads: Bridging the gap between Predictive Power and Explanation in the use of Artificial Intelligence in Medicine. 2023; https://hdl.handle.net/10807/272879
- Herzog C. On the Ethical and Epistemological Utility of Explicable AI in Medicine. Philos Technol. 2022;35(2). https://www.scopus.com/inward/record.uri?eid=2-s2.0-85130962516&doi=10.1007%2fs13347-022-00546-y&partnerID=40&md5=1a6344516eecfe891b59b2a32d0095f0 · DOI
Cited by
Abbas, Q., Jeong, W., & Lee, S. W. (2025) · Healthcare · doi:10.3390/healthcare13172154 · Cited by 236
“Furthermore, explainability can improve model debugging and development, helping researchers and engineers identify biases, data quality issues, and unintended outcomes [21].”
Blackman, J. T., & Veerapen, R. J. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-02891-2 · Cited by 48
“However, a recent systematic review concluded that there is no definitive agreement on the requirement of explainability in the literature [12].”
Artificial intelligence-driven transformative applications in disease diagnosis technology
Zhou, J., Park, S., Dong, S., Tang, X., & Wei, X. (2025) · Medical Review · doi:10.1515/mr-2024-0097 · Cited by 56
“In March 2024, the European Union ’ s AI Act (AIA) mandated that medical AI decision-support systems must include explainability features to ensure that healthcare providers can understand the reasoning behind AI decisions [73].”
Ouhmouk, M., Baichoo, S., & Abık, M. (2025) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2025.101679 · Cited by 35
Shiddik, M. A. B. (2026) · Health Science Reports · doi:10.1002/hsr2.72172 · Cited by 25
Metadata
- Authors
- Freyer, N., Groß, D., & Lipprandt, M.
- Affiliations
- RWTH Aachen University (DE)
- Corresponding author
- Freyer, Nils
- Venue
- BMC Medical Ethics
- Published
- 2024-10-01
- Volume
- 25
- Issue
- 1
- Pages
- 104-104
- ISSN
- 1472-6939 (ISSN-L)
- PMC
- PMC11443763
- OpenAlex
- W4403039509
- Semantic Scholar
- 8135f7dd37df7cf4ad61756fc1152c792be3d821
- Cited by
- 67
- References
- 53
- Citations by year
- 2026: 48 · 2025: 19
- FWCI
- 2.27
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- BMC Medical Ethics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RWTH Publications (RWTH Aachen) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- RWTH Publications (RWTH Aachen)
- Funders
- Universitätsklinikum RWTH Aachen
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- explainable AI, AI decision support systems, health care, epistemic opacity, post-hoc explanation, EU AI Act, ethical requirements
- MeSH terms
- Artificial Intelligence, Delivery of Health Care, European Union, Humans, Decision Support Systems, Clinical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.