Black-box assisted medical decisions: AI power vs. ethical physician care
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
I raise an ethical problem with physicians using “black box” medical AI algorithms, arguing that its use would compromise proper patient care. Even if AI results are reliable, my contention is that without being able to explain medical decisions to patients, physicians’ use of black box AIs would erode the effective and respectful care they provide patients. In addition, I argue that physicians should use AI black boxes only for patients in dire straits, or when physicians use AI as a “co-pilot” (analogous to a spellchecker) but can independently confirm its accuracy. My argument will be further sharpened when, lastly, I give important attention to Alex John London’s objection that physicians already sometimes prescribe treatment, such as lithium drugs, even though neither researchers nor doctors can explain why the treatment works.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450 · cited 1×
“Proponents may point out that AI will outperform human physicians, not only because of the sheer computing power AI possesses, but also because of the susceptibility of human clinicians to making diagnostic mistakes and suc-cumbing to cognitive bias (Topol 2019).”
BackgroundLipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
The Mythos of Model Interpretability
Lipton, Z. C. (2016) · arXiv · doi:10.48550/arxiv.1606.03490 · Cited by 5,422
Improving Diagnosis in Health Care
Committee on Diagnostic Error in Health Care, Board on Health Care Services, Institute of Medicine, & The National Academies of Sciences, E. (2015) · doi:10.17226/21794 · Cited by 1,376
Artificial Intelligence and Black‐Box Medical Decisions: Accuracy versus Explainability
London, A. J. (2019) · Hastings Center Report · doi:10.1002/hast.973 · Cited by 801
Mittelstadt, B., Russell, C., & Wachter, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287574 · Cited by 794
On the ethics of algorithmic decision-making in healthcare
Grote, T., & Berens, P. (2020) · Journal of Medical Ethics · doi:10.1136/medethics-2019-105586 · Cited by 551
Durán, J. M., & Jongsma, K. R. (2021) · Journal of Medical Ethics · doi:10.1136/medethics-2020-106820 · Cited by 446
Artificial Intelligence and Patient-Centered Decision-Making
Bjerring, J. C., & Busch, J. (2021) · Philosophy & Technology · doi:10.1007/s13347-019-00391-6 · Cited by 274 · cited 1×
“This would also take away some significant element of choice, at least informed choice, since the doctor would not be able to substantively advise the patient which treatment to choose (Grote and Berens 2020, 208 and Bjerring and Busch 2021, 363).”
BackgroundComputer knows best? The need for value-flexibility in medical AI
McDougall, R. J. (2019) · Journal of Medical Ethics · doi:10.1136/medethics-2018-105118 · Cited by 262
Cited by
Resnik, D. B., & Hosseini, M. (2025) · AI and Ethics · doi:10.1007/s43681-024-00493-8 · Cited by 228
“13 The opacity of AI systems is ethically problematic because one might argue that we should not use these devices if we cannot trust them, and we cannot trust them if even the best experts do not completely understand how they work [6, 7, 39, 47, 63, 186].”
Should AI models be explainable to clinicians?
Abgrall, G., Holder, A. L., Chelly Dagdia, Z., Zeitouni, K., & Monnet, X. (2024) · Critical Care · doi:10.1186/s13054-024-05005-y · Cited by 175
Large language models for generating medical examinations: systematic review
Artsi, Y., Sorin, V., Konen, E., Glicksberg, B. S., Nadkarni, G., & Klang, E. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05239-y · Cited by 128
The ethical requirement of explainability for AI-DSS in healthcare: a systematic review of reasons
Freyer, N., Groß, D., & Lipprandt, M. (2024) · BMC Medical Ethics · doi:10.1186/s12910-024-01103-2 · Cited by 67
Metadata
- Authors
- Chan, B.
- Venue
- Medicine, Health Care and Philosophy
- Published
- 2023-06-05
- Volume
- 26
- Issue
- 3
- Pages
- 285-292
- ISSN
- 1386-7423 (ISSN-L)
- 1572-8633
- OpenAlex
- W4379377098
- Cited by
- 79
- References
- 11
- FWCI
- 2.73
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Black-box assisted medical decisions: AI power vs. ethical physician care. https://app.cochat.ai/papers/10.1007/s11019-023-10153-z
- The paper
- Chan, B. (2023). Black-box assisted medical decisions: AI power vs. ethical physician care. Medicine, Health Care and Philosophy. https://doi.org/10.1007/s11019-023-10153-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.