Computer knows best? The need for value-flexibility in medical AI
Abstract
Artificial intelligence (AI) is increasingly being developed for use in medicine, including for diagnosis and in treatment decision making. The use of AI in medical treatment raises many ethical issues that are yet to be explored in depth by bioethicists. In this paper, I focus specifically on the relationship between the ethical ideal of shared decision making and AI systems that generate treatment recommendations, using the example of IBM’s Watson for Oncology. I argue that use of this type of system creates both important risks and significant opportunities for promoting shared decision making. If value judgements are fixed and covert in AI systems, then we risk a shift back to more paternalistic medical care. However, if designed and used in an ethically informed way, AI could offer a potentially powerful way of supporting shared decision making. It could be used to incorporate explicit value reflection, promoting patient autonomy. In the context of medical treatment, we need value-flexible AI that can both respond to the values and treatment goals of individual patients and support clinicians to engage in shared decision making.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Computers recommend treatments based on specific parameters that may not actually best reflect the values and preferences of a particular patient.
“Computers recommend treatments based on specific parameters that may not actually best reflect the values and preferences of a particular patient [60].”
From Ethics and governance of trustworthy medical artificial intelligence (2023) · cites this paper as [60]
Cites
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362 · see citing works
Shared decision making: A model for clinical practice
Elwyn, G., Frosch, D., Thomson, R., Joseph-Williams, N., Lloyd, A., Kinnersley, P., Cording, E., Tomson, D., Dodd, C., Rollnick, S., et al. (2012) · Journal of General Internal Medicine · doi:10.1007/s11606-012-2077-6 · Cited by 4,405 · cited 1× · see citing works
“28 29 However, there are two ideas that are fundamental to the concept of shared decision making and particularly relevant in the context of thinking about AI in medicine.”
Charles, C., Gafni, A., & Whelan, T. (1997) · Social Science & Medicine · doi:10.1016/s0277-9536(96)00221-3 · Cited by 4,137 · cited 2× · see citing works
“25 26 It is seen as a well-justified combination of physician responsibility and patient autonomy, avoiding the extremes of either medical paternalism or giving patients sole decision-making power.”
Four Models of the Physician-Patient Relationship
Emanuel, E. J. (1992) · JAMA · doi:10.1001/jama.1992.03480160079038 · Cited by 2,453 · see citing works
Implementing Machine Learning in Health Care — Addressing Ethical Challenges
Char, D. S., Shah, N. H., & Magnus, D. (2018) · New England Journal of Medicine · doi:10.1056/nejmp1714229 · Cited by 1,857 · see citing works
Gottlieb, R. (1987) · The New Yorker · Cited by 1,309 · see citing works
Value Sensitive Design and Information Systems
Friedman, B., Kahn, P. H., & Borning, A. H. (2008) · Wiley eBooks · doi:10.1002/9780470281819.ch4 · Cited by 1,149 · see citing works
The ethics of artificial intelligence
Bostrom, N., & Yudkowsky, E. (2014) · The Cambridge Handbook of Artificial Intelligence · doi:10.1017/cbo9781139046855.020 · Cited by 918 · cited 1× · see citing works
“36 37 An important concept that has emerged from these discussions is ‘value sensitive design’.”
Computational intelligence: a logical approach
(1998) · Choice Reviews Online · doi:10.5860/choice.35-5701 · Cited by 726 · see citing works
Shared decision making in clinical medicine: past research and future directions
Frosch, D. L., & Kaplan, R. M. (1999) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(99)00097-5 · Cited by 721 · see citing works
Couët, N., Desroches, S., Robitaille, H., Vaillancourt, H., LeBlanc, A., Turcotte, S., Elwyn, G. J., & Légaré, F. (2013) · Health Expectations · doi:10.1111/hex.12054 · Cited by 519 · see citing works
Patient-centred medicine: Transforming the clinical method
Brown, J. B. (1998) · Patient Education and Counseling · doi:10.1016/s0738-3991(98)90096-9 · Cited by 456 · see citing works
Clinical Decision Support Systems
Berner, E. S. (2007) · Computers in health care · doi:10.1007/978-0-387-38319-4 · Cited by 276 · see citing works
The Handbook of Information and Computer Ethics
Himma, K. E. (2008) · Wiley eBooks · doi:10.1002/9780470281819 · Cited by 225 · see citing works
Shared Decision Making in the Medical Encounter: Are We All Talking about the Same Thing?
Moumjid, N., Gafni, A., Bremond, A. G., & Carrère, M. (2007) · Medical Decision Making · doi:10.1177/0272989x07306779 · Cited by 222 · see citing works
The Ideal of Shared Decision Making Between Physicians and Patients
Brock, D. W. (1991) · Kennedy Institute of Ethics journal · doi:10.1353/ken.0.0084 · Cited by 204 · see citing works
Patient-centred Medicine Transforming the clinical method
French, S. (2003) · Physiotherapy · doi:10.1016/s0031-9406(05)60508-1 · Cited by 188 · see citing works
The Cambridge handbook of artificial intelligence
(2015) · Choice Reviews Online · doi:10.5860/choice.187061 · Cited by 146 · see citing works
Doctor-patient relationship: from medical paternalism to enhanced autonomy.
Chin, J. J. (2002) · PubMed · Cited by 131 · see citing works
Dignum, V. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/655 · Cited by 120 · see citing works
Borning, A. H., Friedman, B., Davis, J., et al. (2006) · Springer eBooks · doi:10.1007/1-4020-4023-7_23 · Cited by 99 · see citing works
Healthcare Robots: Ethics, Design and Implementation
van Wynsberghe, A. (2015) · University of Twente Research Information · Cited by 82 · see citing works
Ruland, C. M., & Henry, S. B. (2002) · Journal of Biomedical Informatics · doi:10.1016/s1532-0464(03)00037-6 · Cited by 77 · see citing works
Watson Will See You Now: A Supercomputer to Help Clinicians Make Informed Treatment Decisions
Doyle-Lindrud, S. (2015) · Clinical journal of oncology nursing · doi:10.1188/15.cjon.31-32 · Cited by 62 · see citing works
Re-engineering shared decision-making
Gillick, M. R. (2015) · Journal of Medical Ethics · doi:10.1136/medethics-2014-102618 · Cited by 45 · see citing works
van Rysewyk, S. P., & Pontier, M. A. (2014) · International series on intelligent systems, control and automation: science and engineering · doi:10.1007/978-3-319-08108-3 · Cited by 43 · see citing works
Time to regenerate: the doctor in the age of artificial intelligence
Liu, X., Keane, P. A., & Denniston, A. K. (2018) · Journal of the Royal Society of Medicine · doi:10.1177/0141076818762648 · Cited by 38 · see citing works
From decision to shared-decision: Introducing patients’ preferences into clinical decision analysis
Sacchi, L., Rubrichi, S., Rognoni, C., Panzarasa, S., Parimbelli, E., Mazzanti, A., Napolitano, C., Priori, S. G., & Quaglini, S. (2014) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2014.10.004 · Cited by 37 · see citing works
Robotic Surgery: On the Cutting Edge of Ethics
SHARKEY, N. E., & SHARKEY, A. J. C. (2012) · Computer · doi:10.1109/mc.2012.424 · Cited by 29 · see citing works
Ethical Dilemmas in Pediatrics: Cases and Commentaries
Frankel, L. R., Goldworth, A., Rorty, M. V., et al. (2009) · DigitalGeorgetown (Georgetown University Library) · doi:10.1017/cbo9780511545504 · Cited by 19 · see citing works
Ethical considerations concerning computers in medicine in the 1980s.
de Dombal, F. T. (1987) · Journal of Medical Ethics · doi:10.1136/jme.13.4.179 · Cited by 19 · see citing works
Ethical and Legal Issues in Decision Support
Goodman, K. W. (2007) · Springer eBooks · doi:10.1007/978-0-387-38319-4_6 · Cited by 15 · see citing works
The Limits of Shared Decision-Making
Lashway, L. (1996) · Cited by 13 · see citing works
The computer may be assessing you now, but who decided its values?
Hodgkin, P. (2016) · BMJ · doi:10.1136/bmj.i6169 · Cited by 8 · see citing works
Near-drowning, futility, and the limits of shared decision making
Whitney, S. N. (2005) · Cambridge University Press eBooks · doi:10.1017/cbo9780511545504.015 · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Explainability for artificial intelligence in healthcare: a multidisciplinary perspective
the Precise4Q consortium, Amann, J., Blasimme, A., Vayena, E., Frey, D., & Madai, V. I. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01332-6 · Cited by 2,203 · see citing works
The ethics of AI in health care: A mapping review
Morley, J., Machado, C. C. V., Burr, C., Cowls, J., Joshi, I., Taddeo, M., & Floridi, L. (2020) · Social Science & Medicine · doi:10.1016/j.socscimed.2020.113172 · Cited by 1,105 · see citing works
On the ethics of algorithmic decision-making in healthcare
Grote, T., & Berens, P. (2020) · Journal of Medical Ethics · doi:10.1136/medethics-2019-105586 · Cited by 551 · see citing works
Ethics and governance of trustworthy medical artificial intelligence
Zhang, J., & Zhang, Z.-M. (2023) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-023-02103-9 · Cited by 507 · see citing works
“Computers recommend treatments based on specific parameters that may not actually best reflect the values and preferences of a particular patient [60].”
BackgroundDurán, J. M., & Jongsma, K. R. (2021) · Journal of Medical Ethics · doi:10.1136/medethics-2020-106820 · Cited by 446 · see citing works
Trust and medical AI: the challenges we face and the expertise needed to overcome them
Quinn, T. P., Senadeera, M., Jacobs, S., Coghlan, S., & Le, V. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa268 · Cited by 287 · see citing works
Laï, M.-C., Brian, M., & Mamzer, M.-F. (2020) · Journal of Translational Medicine · doi:10.1186/s12967-019-02204-y · Cited by 294 · see citing works
Artificial Intelligence and Patient-Centered Decision-Making
Bjerring, J. C., & Busch, J. (2021) · Philosophy & Technology · doi:10.1007/s13347-019-00391-6 · Cited by 274 · see citing works
Čartolovni, A., Tomičić, A., & Lazić Mosler, E. (2022) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2022.104738 · Cited by 267 · see citing works
Xu, H., & Shuttleworth, K. M. J. (2024) · Intelligent Medicine · doi:10.1016/j.imed.2023.08.001 · Cited by 196 · see citing works
Designing Futuristic Telemedicine Using Artificial Intelligence and Robotics in the COVID-19 Era
Bhaskar, S., Bradley, S., Sakhamuri, S., Moguilner, S., Chattu, V. K., Pandya, S., Schroeder, S., Ray, D., & Banach, M. (2020) · Frontiers in Public Health · doi:10.3389/fpubh.2020.556789 · Cited by 137 · see citing works
Zohny, H., McMillan, J., & King, M. (2023) · Journal of Medical Ethics · doi:10.1136/jme-2023-108909 · Cited by 130 · see citing works
McCradden, M. D., Baba, A., Saha, A., Ahmad, S., Boparai, K., Fadaiefard, P., & Cusimano, M. D. (2020) · CMAJ Open · doi:10.9778/cmajo.20190151 · Cited by 101 · see citing works
How to achieve trustworthy artificial intelligence for health.
Bærøe, K., Miyata-Sturm, A., & Henden, E. (2020) · Bulletin of the World Health Organization · doi:10.2471/blt.19.237289 · Cited by 103 · see citing works
The Ten Commandments of Ethical Medical AI
Muller, H., Mayrhofer, M. T., Van Veen, E.-B., & Holzinger, A. (2021) · Computer · doi:10.1109/mc.2021.3074263 · Cited by 102 · see citing works
Benzinger, L., Ursin, F., Balke, W.-T., Kacprowski, T., & Salloch, S. (2023) · BMC Medical Ethics · doi:10.1186/s12910-023-00929-6 · Cited by 95 · see citing works
The Debate on the Ethics of AI in Health Care: a Reconstruction and Critical Review
Morley, J., Machado, C., Burr, C., Cowls, J., Taddeo, M., & Floridi, L. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3486518 · Cited by 66 · see citing works
Ethics of artificial intelligence in medicine
Savulescu, J., Giubilini, A., Vandersluis, R., & Mishra, A. (2024) · Singapore Medical Journal · doi:10.4103/singaporemedj.smj-2023-279 · Cited by 68 · see citing works
AI support for ethical decision-making around resuscitation: proceed with care
Biller-Andorno, N., Ferrario, A., Joebges, S., Krones, T., Massini, F., Barth, P., Arampatzis, G., & Krauthammer, M. (2022) · Journal of Medical Ethics · doi:10.1136/medethics-2020-106786 · Cited by 67 · see citing works
Testimonial injustice in medical machine learning
Pozzi, G. (2023) · Journal of Medical Ethics · doi:10.1136/jme-2022-108630 · Cited by 62 · see citing works
Dlugatch, R., Georgieva, A., & Kerasidou, A. (2024) · BMC Medical Ethics · doi:10.1186/s12910-023-00990-1 · Cited by 52 · see citing works
Towards evidence-based practice 2.0: leveraging artificial intelligence in healthcare
Nilsen, P., Sundemo, D., Heintz, F., Neher, M., Nygren, J., Svedberg, P., & Petersson, L. (2024) · Frontiers in Health Services · doi:10.3389/frhs.2024.1368030 · Cited by 39 · see citing works
Froicu, E.-M., Creangă-Murariu, I., Afrăsânie, V.-A., Gafton, B., Alexa-Stratulat, T., Miron, L., Pușcașu, D. M., Poroch, V., Bacoanu, G., Radu, I., et al. (2025) · Current Oncology Reports · doi:10.1007/s11912-025-01698-8 · Cited by 38 · see citing works
“I’m afraid I can’t let you do that, Doctor”: meaningful disagreements with AI in medical contexts
Kempt, H., Heilinger, J.-C., & Nagel, S. K. (2023) · AI & SOCIETY · doi:10.1007/s00146-022-01418-x · Cited by 31 · see citing works
AI support for ethical decision-making around resuscitation: proceed with care
Biller-Andorno, N., Ferrario, A., Joebges, S., Krones, T., Massini, F., Barth, P., Arampatzis, G., & Krauthammer, M. (2020) · doi:10.1101/2020.08.17.20171769 · Cited by 67 · see citing works
Readying Medical Students for Medical AI: The Need to Embed AI Ethics Education
Quinn, T. P., & Coghlan, S. (2021) · arXiv · doi:10.48550/arxiv.2109.02866 · Cited by 13 · see citing works
- Authors
- McDougall, R. J.
- Affiliations
- The University of Melbourne (AU)
- Corresponding author
- McDougall, Rosalind J
- Venue
- Journal of Medical Ethics
- Publisher
- BMJ
- Published
- 2018-11-22
- Volume
- 45
- Issue
- 3
- Pages
- 156-160
- ISSN
- 0306-6800 (ISSN-L)
- 1473-4257
- OpenAlex
- W2901194671
- Cited by
- 262
- References
- 35
- Citations by year
- 2026: 35 · 2025: 50 · 2024: 54 · 2023: 27 · 2022: 37 · 2021: 27 · 2020: 22 · 2019: 10
- FWCI
- 4.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Medical Ethics (publishedVersion)
- PubMed (publishedVersion)
- HighWire Press Open Archive (submittedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics and bioethics in healthcare
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Keywords
- artificial intelligence in healthcare, shared decision-making, Watson for Oncology, treatment decision-making, paternalism, patient autonomy, value reflection, clinical decision support
- MeSH terms
- Decision Making, Shared, Artificial Intelligence, Decision Making, Computer-Assisted, Diagnosis, Computer-Assisted, Humans, Medical Oncology, Social Values, Paternalism
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
