Patient Consent and The Right to Notice and Explanation of AI Systems Used in Health Care
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Given the need for enforceable guardrails for artificial intelligence (AI) that protect the public and allow for innovation, the U.S. Government recently issued a Blueprint for an AI Bill of Rights which outlines five principles of safe AI design, use, and implementation. One in particular, the right to notice and explanation, requires accurately informing the public about the use of AI that impacts them in ways that are easy to understand. Yet, in the healthcare setting, it is unclear what goal the right to notice and explanation serves, and the moral importance of patient-level disclosure. We propose three normative functions of this right: (1) to notify patients about their care, (2) to educate patients and promote trust, and (3) to meet standards for informed consent. Additional clarity is needed to guide practices that respect the right to notice and explanation of AI in healthcare while providing meaningful benefits to patients.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452 · cited 1×
“…however, is that some individuals who are notified that the entity they are speaking with is a trained AI rather than a human are likely to respond with aversion (Lee and Lee 2023; McGuire et al. 2023; Mozafari, Weiger, and Hammerschmidt 2021; Schuetzler, Grimes, and Giboney 2019; Weizenbaum 1966).”
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · cited 1×
“…inadequate explanations for how an AI tool works may grow uncom-fortable with the use of AI tools in their care altogether, having somewhat of an opposite effect to the intended goal of increasing AI familiarity and enhancing trust (Starke and Ienca 2022; Dietvorst, Simmons, and Massey 2015).”
Precision medicine, AI, and the future of personalized health care
Johnson, K. B., Wei, W., Weeraratne, D., Frisse, M. E., Misulis, K., Rhee, K., Zhao, J., & Snowdon, J. L. (2021) · Clinical and Translational Science · doi:10.1111/cts.12884 · Cited by 1,854 · cited 1×
“The relevance of such a right is expected to grow as clinicians rapidly integrate AI into clinical practice, with applications in precision medicine, surgery, diagnostic radiology, and risk stratification (Johnson et al. 2021; Mun et al. 2020; Bohr and Memarzadeh 2020; Hashimoto et al. 2018).”
The false hope of current approaches to explainable artificial intelligence in health care
Ghassemi, M., Oakden-Rayner, L., & Beam, A. L. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00208-9 · Cited by 1,724 · cited 1×
“…on the ability to achieve this at all, uncertain that users of AI in healthcare or those affected by AI (healthcare workers and presumably patients, respectively) will be capable of judging the quality of an AI’s decision by receiving an explanation to it (Ghassemi, Oakden-Rayner, and Beam 2021).”
The rise of artificial intelligence in healthcare applications
Bohr, A., & Memarzadeh, K. (2020) · Artificial Intelligence in Healthcare · doi:10.1016/b978-0-12-818438-7.00002-2 · Cited by 1,636
Artificial intelligence in surgery: promises and perils
Hashimoto, D. A., Rosman, G., Rus, D., & Meireles, O. R. (2018) · Annals of Surgery · doi:10.1097/sla.0000000000002693 · Cited by 1,468
Markus, A. F., Kors, J. A., & Rijnbeek, P. R. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103655 · Cited by 811
Sharma, A., Lin, I. W., Miner, A. S., Atkins, D. C., & Althoff, T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-022-00593-2 · Cited by 500
Automation bias and verification complexity: a systematic review
Lyell, D., & Coiera, E. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw105 · Cited by 451 · cited 1×
“On the opposite side of this spectrum of trust, there is also evidence pointing to explanations increasing people’s confidence in an algorithmic decision to an unreason-able extent, exacerbating automation bias and over-trust of AI systems (Eiband et al. 2019; Lyell and Coiera 2017).”
Patient apprehensions about the use of artificial intelligence in healthcare
Richardson, J. P., Smith, C., Curtis, S., Watson, S., Zhu, X., Barry, B., & Sharp, R. R. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00509-1 · Cited by 424
Cited by
Wells, B. J., Nguyen, H. M., McWilliams, A., Pallini, M., Bovi, A., Kuzma, A., Kramer, J., Chou, S.-H., Hetherington, T., Corn, P., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01900-y · Cited by 90
“Notify patients when AI is being used and, when appropriate, to obtain their consent — particularly in sensitive or high-stakes situations 33,34 .”
Ethical Obligations to Inform Patients About Use of AI Tools
Mello, M. M., Char, D., & Xu, S. H. (2025) · JAMA · doi:10.1001/jama.2025.11417 · Cited by 57
Esmaeilzadeh, P. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03182-6 · Cited by 26
Metadata
- Authors
- Hurley, M. E., Lang, B. H., Kostick-Quenet, K. M., Smith, J. N., & Blumenthal-Barby, J.
- Venue
- The American Journal of Bioethics
- Publisher
- Informa UK Limited
- Published
- 2024-09-17
- Volume
- 25
- Issue
- 3
- Pages
- 102-114
- ISSN
- 1526-5161 (ISSN-L)
- 1536-0075
- OpenAlex
- W4402568239
- Cited by
- 63
- References
- 19
- FWCI
- 5.05
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Patient Consent and The Right to Notice and Explanation of AI Systems Used in Health Care. https://app.cochat.ai/papers/10.1080/15265161.2024.2399828
- The paper
- Hurley, M. E., Lang, B. H., Kostick-Quenet, K. M., Smith, J. N., & Blumenthal-Barby, J. (2025). Patient Consent and The Right to Notice and Explanation of AI Systems Used in Health Care. The American Journal of Bioethics. https://doi.org/10.1080/15265161.2024.2399828
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.