The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Defining the undefinable: the black box problem in healthcare artificial intelligence
Abstract
The ‘black box problem’ is a long-standing talking point in debates about artificial intelligence (AI). This is a significant point of tension between ethicists, programmers, clinicians and anyone else working on developing AI for healthcare applications. However, the precise definition of these systems are often left undefined, vague, unclear or are assumed to be standardised within AI circles. This leads to situations where individuals working on AI talk over each other and has been invoked in numerous debates between opaque and explainable systems. This paper proposes a coherent and clear definition for the black box problem to assist in future discussions about AI in healthcare. This is accomplished by synthesising various definitions in the literature and examining several criteria that can be extrapolated from these definitions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Wadden defines”
From Positioning Paradata: A Conceptual Frame for AI Processual Documentation in Archives and Recordkeeping Contexts (2023) · cites this paper as [62]
In the citing paper
“Within AI tools, there is also the challenge of transparency and accountability with many AI algorithms, particularly deep learning models, operating as “black boxes” [54].”
From Reimagining Core Entrustable Professional Activities for Undergraduate Medical Education in the Era of Artificial Intelligence (2023) · cites this paper as [54]
Citations
Cites
Principles of Biomedical Ethics
(2012) · Thieves of Virtue · doi:10.7551/mitpress/9079.003.0009 · Cited by 13,453
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196
Medical image analysis using convolutional neural networks: a review
Anwar, S. M., Majid, M., Qayyum, A., Awais, M., Alnowami, M., & Khan, M. K. (2018) · Journal of Medical Systems · doi:10.1007/s10916-018-1088-1 · Cited by 1,431
If...Then: Algorithmic Power and Politics
Bucher, T. (2018) · doi:10.1093/oso/9780190493028.001.0001 · Cited by 1,223
(2019) · Cited by 986
Clinical applications of machine learning algorithms: beyond the black box
Watson, D. S., Krutzinna, J., Bruce, I. N., Griffiths, C. E., McInnes, I. B., Barnes, M. R., & Floridi, L. (2019) · BMJ · doi:10.1136/bmj.l886 · Cited by 450
Durán, J. M., & Jongsma, K. R. (2021) · Journal of Medical Ethics · doi:10.1136/medethics-2020-106820 · Cited by 446
Solving the Black Box Problem: A Normative Framework for Explainable Artificial Intelligence
Zednik, C. (2021) · Philosophy & Technology · doi:10.1007/s13347-019-00382-7 · Cited by 419
Cited by
One AI Does Not Fit All: A Cluster Analysis of the Laypeople’s Perception of AI Roles
Kim, T., Molina, M. D., Rheu, M. (., Zhan, E. S., & Peng, W. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581340 · Cited by 68
The ethical requirement of explainability for AI-DSS in healthcare: a systematic review of reasons
Freyer, N., Groß, D., & Lipprandt, M. (2024) · BMC Medical Ethics · doi:10.1186/s12910-024-01103-2 · Cited by 67
Jacobs, S. M., Lundy, N. N., Issenberg, S. B., & Chandran, L. (2023) · JMIR Medical Education · doi:10.2196/50903 · Cited by 33
Salvagno, M., Cassai, A. D., Zorzi, S., Zaccarelli, M., Pasetto, M., Sterchele, E. D., Chumachenko, D., Gerli, A. G., Azamfirei, R., & Taccone, F. S. (2024) · PLOS ONE · doi:10.1371/journal.pone.0309208 · Cited by 32
Cameron, S., Franks, P., & Hamidzadeh, B. (2023) · Journal on Computing and Cultural Heritage · doi:10.1145/3594728 · Cited by 25
Metadata
- Authors
- Wadden, J. J.
- Affiliations
- University of British Columbia (CA)
- Corresponding author
- Wadden, Jordan Joseph
- Venue
- Journal of Medical Ethics
- Publisher
- BMJ
- Published
- 2021-07-21
- Volume
- 48
- Issue
- 10
- Pages
- 764-768
- ISSN
- 0306-6800 (ISSN-L)
- 1473-4257
- OpenAlex
- W3184842396
- Cited by
- 224
- References
- 29
- Citations by year
- 2026: 54 · 2025: 67 · 2024: 52 · 2023: 32 · 2022: 18 · 2021: 1
- FWCI
- 9.05
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Medical Ethics (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Social Sciences and Humanities Research Council of Canada (Doctoral Fellowship 752-2020-2225)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- black box problem, artificial intelligence in healthcare, explainable AI, opacity, AI ethics, clinical decision support
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Defining the undefinable: the black box problem in healthcare artificial intelligence. https://app.cochat.ai/papers/10.1136/medethics-2021-107529
- The paper
- Wadden, J. J. (2022). Defining the undefinable: the black box problem in healthcare artificial intelligence. Journal of Medical Ethics. https://doi.org/10.1136/medethics-2021-107529
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.