The Deception of Certainty: how Non-Interpretable Machine Learning Outcomes Challenge the Epistemic Authority of Physicians. A deliberative-relational Approach
Open accessThe DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Developments in Machine Learning (ML) have attracted attention in a wide range of healthcare fields to improve medical practice and the benefit of patients. Particularly, this should be achieved by providing more or less automated decision recommendations to the treating physician. However, some hopes placed in ML for healthcare seem to be disappointed, at least in part, by a lack of transparency or traceability. Skepticism exists primarily in the fact that the physician, as the person responsible for diagnosis, therapy, and care, has no or insufficient insight into how such recommendations are reached. The following paper aims to make understandable the specificity of the deliberative model of a physician-patient relationship that has been achieved over decades. By outlining the (social-)epistemic and inherently normative relationship between physicians and patients, I want to show how this relationship might be altered by non-traceable ML recommendations. With respect to some healthcare decisions, such changes in deliberative practice may create normatively far-reaching challenges. Therefore, in the future, a differentiation of decision-making situations in healthcare with respect to the necessary depth of insight into the process of outcome generation seems essential.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A guide to deep learning in healthcare
Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., Chou, K., Cui, C., Corrado, G., Thrun, S., & Dean, J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0316-z · Cited by 5,312 · cited 1×
“Connected with increasingly powerful processor technologies, such ML systems are already enabling improvements in accuracy and efficiency across several medical fields (Topol 2019; Esteva et al. 2019).”
BackgroundFloridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., et al. (2018) · Minds and Machines · doi:10.1007/s11023-018-9482-5 · Cited by 4,774 · cited 2×
“…trust in resulting decisions, and therefore demand from ML-based applications ‘explicable’ or ‘explainable’ generation processes and/or outcomes (cf. Floridi et al. 2018; Holzinger et al. 2020; Rudin and Radin 2019 ; Heinrichs and Eickhoff 2020; Coeckelbergh 2020; Bjerring and Busch 2021;…”
BackgroundUnintended consequences of machine learning in medicine
Cabitza, F., Rasoini, R., & Gensini, G. F. (2017) · JAMA · doi:10.1001/jama.2017.7797 · Cited by 1,066
Experts: Which Ones Should You Trust?
GOLDMAN, A. I. (2001) · Philosophy and Phenomenological Research · doi:10.1111/j.1933-1592.2001.tb00093.x · Cited by 911 · cited 3×
“This arises from some set of skills or techniques that constitute part of what it is to be an expert” (Goldman 2001).”
BackgroundArtificial Intelligence and Black‐Box Medical Decisions: Accuracy versus Explainability
London, A. J. (2019) · Hastings Center Report · doi:10.1002/hast.973 · Cited by 801
The impact of artificial intelligence in medicine on the future role of the physician
Ahuja, A. S. (2019) · PeerJ · doi:10.7717/peerj.7702 · Cited by 789 · cited 2×
“For these multi-layered networks can be hold, the larger the underlying data set, the better the outcome produced by the algorithm (Hinton 2007; Ahuja 2019).”
BackgroundArtificial intelligence in retina
Schmidt-Erfurth, U., Sadeghipour, A., Gerendas, B. S., Waldstein, S. M., & Bogunović, H. (2018) · Progress in Retinal and Eye Research · doi:10.1016/j.preteyeres.2018.07.004 · Cited by 779 · cited 1×
“Here non-rule-based algorithms ‘learn’, that is, generate insights, by recognizing certain patterns and regularities in a defined set of raw data (Hinton 2007; Schmidt-Erfurth et al. 2018; Ahuja 2019).”
BackgroundOn the ethics of algorithmic decision-making in healthcare
Grote, T., & Berens, P. (2020) · Journal of Medical Ethics · doi:10.1136/medethics-2019-105586 · Cited by 551 · cited 9×
“…complexities of many workaday consultations.” to the conclusion that the measure she recommends continues to deviate from that one of the ML system, she would have to decide whether to maintain her belief despite the divergent outcome, or to adjust it, or to abandon it (Grote and Berens 2020).”
BackgroundInfluentialCoeckelbergh, M. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-019-00146-8 · Cited by 475 · cited 2×
“…in resulting decisions, and therefore demand from ML-based applications ‘explicable’ or ‘explainable’ generation processes and/or outcomes (cf. Floridi et al. 2018; Holzinger et al. 2020; Rudin and Radin 2019 ; Heinrichs and Eickhoff 2020; Coeckelbergh 2020; Bjerring and Busch 2021; Funer 2022).”
BackgroundDurán, J. M., & Jongsma, K. R. (2021) · Journal of Medical Ethics · doi:10.1136/medethics-2020-106820 · Cited by 446
Cited by
Challenges of artificial intelligence in medicine and dermatology
Grzybowski, A., Jin, K., & Wu, H. (2024) · Clinics in Dermatology · doi:10.1016/j.clindermatol.2023.12.013 · Cited by 115
The ethical requirement of explainability for AI-DSS in healthcare: a systematic review of reasons
Freyer, N., Groß, D., & Lipprandt, M. (2024) · BMC Medical Ethics · doi:10.1186/s12910-024-01103-2 · Cited by 67
“A major argument for that position was that the more nor-matively far-reaching the decisions for which the recommendations of an AI-DSS are used, the higher the need for justification and, thus, explanations [40–42 , 44].”
BackgroundCallahan, A., McElfresh, D., Banda, J. M., Bunney, G., Char, D., Chen, J., Corbin, C. K., Dash, D., Downing, N. L., Jain, S. S., et al. (2024) · NEJM Catalyst · doi:10.1056/cat.24.0131 · Cited by 61
Tokgöz, P., Hafner, J., & Dockweiler, C. (2023) · Das Gesundheitswesen · doi:10.1055/a-2098-3108 · Cited by 5
“Potenziell problematisch an solchen Anwendungen ist, dass ihre Entscheidungsstruktur nicht oder nur teilweise und unter großem Aufwand von Menschen verstanden und bewertet werden kann [11, 12].”
Metadata
- Authors
- Funer, F.
- Venue
- Medicine, Health Care and Philosophy
- Published
- 2022-05-10
- Volume
- 25
- Issue
- 2
- Pages
- 167-178
- ISSN
- 1386-7423 (ISSN-L)
- 1572-8633
- OpenAlex
- W4280517080
- Cited by
- 34
- References
- 17
- FWCI
- 1.48
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Deception of Certainty: how Non-Interpretable Machine Learning Outcomes Challenge the Epistemic Authority of Physicians. A deliberative-relational Approach. https://app.cochat.ai/papers/10.1007/s11019-022-10076-1
- The paper
- Funer, F. (2022). The Deception of Certainty: how Non-Interpretable Machine Learning Outcomes Challenge the Epistemic Authority of Physicians. A deliberative-relational Approach. Medicine, Health Care and Philosophy. https://doi.org/10.1007/s11019-022-10076-1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.