Cited by 76Artificial Intelligence in Healthcare and EducationEthics and Social Impacts of AIEthics and bioethics in healthcare

Evaluation of artificial intelligence clinical applications: Detailed case analyses show value of healthcare ethics approach in identifying patient care issues

Open access
Rogers, Wendy A., Draper, Heather, Carter, Stacy M.·2021·Bioethics

Abstract

This paper is one of the first to analyse the ethical implications of specific healthcare artificial intelligence (AI) applications, and the first to provide a detailed analysis of AI‐based systems for clinical decision support. AI is increasingly being deployed across multiple domains. In response, a plethora of ethical guidelines and principles for general AI use have been published, with some convergence about which ethical concepts are relevant to this new technology. However, few of these frameworks are healthcare‐specific, and there has been limited examination of actual AI applications in healthcare. Our ethical evaluation identifies context‐ and case‐specific healthcare ethical issues for two applications, and investigates the extent to which the general ethical principles for AI‐assisted healthcare expressed in existing frameworks capture what is most ethically relevant from the perspective of healthcare ethics. We provide a detailed description and analysis of two AI‐based systems for clinical decision support (Painchek® and IDx‐DR). Our results identify ethical challenges associated with potentially deceptive promissory claims, lack of patient and public involvement in healthcare AI development and deployment, and lack of attention to the impact of AIs on healthcare relationships. Our analysis also highlights the close connection between evaluation and technical development and reporting. Critical appraisal frameworks for healthcare AIs should include explicit ethical evaluation with benchmarks. However, each application will require scrutiny across the AI life‐cycle to identify ethical issues specific to healthcare. This level of analysis requires more attention to detail than is suggested by current ethical guidance or frameworks.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1111/bioe.12885

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 2

  1. 17 Abràmoff, M.D., Lavin, P.T., Birch, M., Shah, N., & Folk, J.C. (2018). Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. npj Digital Medicine . 1 (1), 39; Van Der Heijden, A.A., Abràmoff, M.D., Verbraak, F., Hecke, M.V., Liem, A., & Nijpels, G. (2018). Validation of automated screening for referable diabetic retinopathy with the IDx-DR device in the Hoorn Diabetes Care System. Acta Ophthalmologica . 96 (1), 63-68; Verbraak, F.D., Abràmoff, M.D., Bausch, G.C.F., Klaver, C., Nijpels, G., Schlingemann, ReinierO., & van der Heijden, A.A. (2019). Diagnostic Accuracy of a Device for the Automated Detection of Diabetic Retinopathy in a Primary Care Setting. Diabetes Care . 42 (4), 651-656.
  2. For Peer Review 15 Painchek ® , op cit . note 12. 16 Anon. (2019, December 1). The Non-Executive Chairman of PainChek Ltd. (ASX:PCK), John Murray, Just Sold 50% Of Their Holding. Simply Wall St . Retrieved from https://simplywall.st/stocks/au/healthcare/asx-pck/painchek-shares/news/the-nonexecutive-chairman-of-painchek-ltd-asxpck-john-murray-just-sold-50-of-their-holding/ Accessed 8 Feb 2021]. until they could be resolved. This produced the final list of issues for each use-case, presented in chronological order across the AI lifecycle (in Section 3).

Cited by

Sorted by citations

Authors
Rogers, W. A., Draper, H., & Carter, S. M.
Affiliations
  • Macquarie University (AU)
  • University of Warwick (GB)
  • University of Wollongong (AU)
Corresponding author
Rogers, Wendy A.
Venue
Bioethics
Publisher
Wiley
Published
2021-05-28
Volume
35
Issue
7
Pages
623-633
ISSN
  • 0269-9702 (ISSN-L)
  • 1467-8519
OpenAlex
W3164685439
Cited by
76
References
0
Citations by year
2026: 7 · 2025: 17 · 2024: 24 · 2023: 23 · 2022: 1 · 2021: 4
FWCI
3.76
Type
journal article
Language
en
Open access
Locations
  • Bioethics (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • Warwick Research Archive Portal (University of Warwick) (acceptedVersion)
  • Research Online (University of Wollongong) (submittedVersion)
  • Figshare (submittedVersion)
Funders
  • National Health and Medical Research Council (1181960)
  • Canadian Institute for Advanced Research
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics and bioethics in healthcare
Keywords
artificial intelligence in healthcare, clinical decision support, health care ethics, patient and public involvement, AI lifecycle
MeSH terms
Artificial Intelligence, Bioethics, Delivery of Health Care, Patient Care, Health Facilities, Humans

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review