The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Clinical AI: opacity, accountability, responsibility and liability
Open accessAbstract
The aim of this literature review was to compose a narrative review supported by a systematic approach to critically identify and examine concerns about accountability and the allocation of responsibility and legal liability as applied to the clinician and the technologist as applied the use of opaque AI-powered systems in clinical decision making. This review questions (a) if it is permissible for a clinician to use an opaque AI system (AIS) in clinical decision making and (b) if a patient was harmed as a result of using a clinician using an AIS’s suggestion, how would responsibility and legal liability be allocated? Literature was systematically searched, retrieved, and reviewed from nine databases, which also included items from three clinical professional regulators, as well as relevant grey literature from governmental and non-governmental organisations. This literature was subjected to inclusion/exclusion criteria; those items found relevant to this review underwent data extraction. This review found that there are multiple concerns about opacity, accountability, responsibility and liability when considering the stakeholders of technologists and clinicians in the creation and use of AIS in clinical decision making. Accountability is challenged when the AIS used is opaque, and allocation of responsibility is somewhat unclear. Legal analysis would help stakeholders to understand their obligations and prepare should an undesirable scenario of patient harm eventuate when AIS were used.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“While it aims to provide evidence-based recommendations, reports have highlighted instances where Watson for Oncology suggested treatments that contradicted medical guidelines [141].”
From Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond (2024) · cites this paper as [141]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Medical error—the third leading cause of death in the US
Makary, M. A., & Daniel, M. (2016) · BMJ · doi:10.1136/bmj.i2139 · Cited by 3,264
Five steps to conducting a systematic review
Khan, K. S., Kunz, R., Kleijnen, J., & Antes, G. (2003) · Journal of the Royal Society of Medicine · doi:10.1177/014107680309600304 · Cited by 1,873
Implementing Machine Learning in Health Care — Addressing Ethical Challenges
Char, D. S., Shah, N. H., & Magnus, D. (2018) · New England Journal of Medicine · doi:10.1056/nejmp1714229 · Cited by 1,857
Responsible Artificial Intelligence: How to Develop and Use AI in a Responsible Way
Dignum, V. (2019) · Artificial Intelligence: Foundations, Theory, and Algorithms · doi:10.1007/978-3-030-30371-6 · Cited by 1,197
Five steps to conducting a systematic review
Khan, K. S., Kunz, R., Kleijnen, J., et al. (2003) · Journal of the Royal Society of Medicine · doi:10.1258/jrsm.96.3.118 · Cited by 997
Computer Power and Human Reason: From Judgement to Calculation.
Crowther, J. F., & Weizenbaum, J. (1977) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2062802 · Cited by 726
Hengstler, M., Enkel, E., & Duelli, S. (2016) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2015.12.014 · Cited by 661
Recommendations for the ethical use and design of artificial intelligent care providers
Luxton, D. D. (2014) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2014.06.004 · Cited by 243
Opening the black box of machine learning
Medicine, T. L. R. (2018) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(18)30425-9 · Cited by 172
How to write a systematic review of reasons
Strech, D., & Sofaer, N. (2011) · Journal of Medical Ethics · doi:10.1136/medethics-2011-100096 · Cited by 168
Marsack, K. P., & Hollier, L. H. (2017) · Journal of Craniofacial Surgery · doi:10.1097/scs.0000000000003673 · Cited by 122
Computer Power and Human Reason: from Judgement to Calculation
Michie, D. (1976) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(76)80035-1 · Cited by 106
The Effects of Closed-Loop Medical Devices on the Autonomy and Accountability of Persons and Systems
Kellmeyer, P., Cochrane, T. I., Müller, O., MITCHELL, C. M., Tonio Ball, A., Fins, J. J., & Biller‐Andorno, N. (2016) · Cambridge Quarterly of Healthcare Ethics · doi:10.1017/s0963180116000359 · Cited by 87
If You Can't Beat It, Join It: Uncertainty and Trust in Medicine
Armstrong, K. (2018) · Annals of Internal Medicine · doi:10.7326/m18-0445 · Cited by 38
Kyte, D. G., Draper, H., Ives, J., Liles, C. R., Gheorghe, A., & Calvert, M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0060684 · Cited by 36
Oshana, M. A. L. (2004) · Philosophical Topics · doi:10.5840/philtopics2004321/22 · Cited by 36
Miles, A. (2007) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2007.00898.x · Cited by 28
Ethical and Technical Aspects of Emotions to Create Empathy in Medical Machines
Vallverdú, J., & Casacuberta, D. (2014) · International series on intelligent systems, control and automation: science and engineering · doi:10.1007/978-3-319-08108-3_20 · Cited by 25
Automating Medicine the Ethical Way
Whitby, B. (2014) · International series on intelligent systems, control and automation: science and engineering · doi:10.1007/978-3-319-08108-3_14 · Cited by 15
On the use of clinical psychology software: Practical, legal, and ethical concerns.
Hartman, D. E. (1986) · Professional Psychology Research and Practice · doi:10.1037/0735-7028.17.5.462 · Cited by 15
Having the Final Say: Machine Support of Ethical Decisions of Doctors
Inthorn, J., Tabacchi, M. E., & Seising, R. (2014) · International series on intelligent systems, control and automation: science and engineering · doi:10.1007/978-3-319-08108-3_12 · Cited by 9
Computer-Aided Diagnosis and Negligence
Bainbridge, D. (1991) · Medicine Science and the Law · doi:10.1177/002580249103100208 · Cited by 6
Neural Expert Systems in Medical Image Interpretation: Development, Use, and Ethical Issues
Pouloudi, A., & Magoulas, G. D. (2000) · Journal of Intelligent Systems · doi:10.1515/jisys.2000.10.5-6.451 · Cited by 5
Ethical and methodological aspects of medical computer data bases and knowledge bases
Doroszewski, J. (1988) · Metamedicine · doi:10.1007/bf00489406 · Cited by 5
To delegate or not to delegate: Care robots, moral agency and moral responsibility
van Wynsberghe, A. (2014) · University of Twente Research Information · Cited by 4
With a little help from AI friends
Sukel, K. (2017) · The New Scientist · doi:10.1016/s0262-4079(17)31376-3 · Cited by 4
(2019)
Research works printed but not linked: 1
- Association for computing machinery (2018) acm code of ethics and professional conduct. https://www.acm.org/code-of-ethics. Accessed Feb 2018
Cited by
Sorted by citations
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762
The Role of AI in Hospitals and Clinics: Transforming Healthcare in the 21st Century
Varnosfaderani, S. M., & Forouzanfar, M. (2024) · Bioengineering · doi:10.3390/bioengineering11040337 · Cited by 1,141
Legal and Ethical Consideration in Artificial Intelligence in Healthcare: Who Takes Responsibility?
Naik, N., Hameed, B. M. Z., Shetty, D. K., Swain, D., Shah, M., Paul, R., Aggarwal, K., Ibrahim, S., Patil, V., Smriti, K., et al. (2022) · Frontiers in Surgery · doi:10.3389/fsurg.2022.862322 · Cited by 931
Survey of Explainable AI Techniques in Healthcare
Chaddad, A., Peng, J., Xu, J., & Bouridane, A. (2023) · Sensors · doi:10.3390/s23020634 · Cited by 591
Schepman, A., & Rodway, P. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2085400 · Cited by 503
Metadata
- Authors
- Smith, H.
- Affiliations
- University of Bristol (GB)
- Corresponding author
- Smith, Helen
- ORCID
- Smith, Helen
- Venue
- AI & SOCIETY
- Published
- 2020-07-25
- Volume
- 36
- Issue
- 2
- Pages
- 535-545
- ISSN
- 0951-5666 (ISSN-L)
- 1435-5655
- OpenAlex
- W3045238219
- Cited by
- 245
- References
- 27
- Citations by year
- 2026: 73 · 2025: 62 · 2024: 53 · 2023: 35 · 2022: 15 · 2021: 6 · 2020: 1
- FWCI
- 4.20
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- AI & Society (publishedVersion, cc-by)
- Bristol Research (University of Bristol) (submittedVersion, cc-by)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Law › Digitalization, Law, and Regulation
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- clinical AI, algorithmic opacity, accountability, responsibility allocation, legal liability, clinical decision-making, patient harm, clinicians, technologists, algorithmic transparency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
