Co-design and ethical artificial intelligence for health: An agenda for critical research and practice
Open accessThe DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
Applications of artificial intelligence/machine learning (AI/ML) in health care are dynamic and rapidly growing. One strategy for anticipating and addressing ethical challenges related to AI/ML for health care is patient and public involvement in the design of those technologies – often referred to as ‘co-design’. Co-design has a diverse intellectual and practical history, however, and has been conceptualized in many different ways. Moreover, AI/ML introduces challenges to co-design that are often underappreciated. Informed by perspectives from critical data studies and critical digital health studies, we review the research literature on involvement in health care, and involvement in design, and examine the extent to which co-design as commonly conceptualized is capable of addressing the range of normative issues raised by AI/ML for health care. We suggest that AI/ML technologies have amplified and modified existing challenges related to patient and public involvement, and created entirely new challenges. We outline three pitfalls associated with co-design for ethical AI/ML for health care and conclude with suggestions for addressing these practical and conceptual challenges.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reassembling the Social: An Introduction to Actor-Network-Theory
Latour, B. (2005) · doi:10.1093/oso/9780199256044.001.0001 · Cited by 16,778
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1×
“AI /ML has also been used in epidemiological modelling and forecasting (Lalmuanawma et al., 2020), clinical decision support (Montani and Striani, 2019), and health care operations and logistics (Obermeyer et al., 2019).”
MethodCo-creation and the new landscapes of design
Sanders, E. B.-N., & Stappers, P. J. (2008) · CoDesign · doi:10.1080/15710880701875068 · Cited by 5,561 · cited 2×
“It is important to note that co-design has been represented in the research literature in a variety of ways (Sanders and Stappers, 2008).”
Backgroundboyd, D., & Crawford, K. (2012) · Information, Communication & Society · doi:10.1080/1369118x.2012.678878 · Cited by 5,479
The Black Box Society: The Secret Algorithms That Control Money and Information
Pasquale, F. (2015) · doi:10.4159/harvard.9780674736061 · Cited by 3,796
D'Ignazio, C., & Klein, L. F. (2020) · doi:10.7551/mitpress/11805.001.0001 · Cited by 1,738
(2020) · doi:10.7551/mitpress/12255.001.0001 · Cited by 1,706
Fairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631 · cited 1×
“While humility in science and technology has been proposed by other commentators (Jasanoff, 2007; Selbst et al., 2019), this same consideration is at risk of being overlooked in co-design as a result of assuming that by letting patients or publics inform development, co-design is itself an…”
BackgroundThe Costs of Connection: How Data Is Colonizing Human Life and Appropriating It for Capitalism
Couldry, N., & Mejias, U. A. (2020) · doi:10.1515/9781503609754 · Cited by 1,246
Greenhalgh, T., Hinton, L., Finlay, T., Macfarlane, A., Fahy, N., Clyde, B., & Chant, A. (2019) · Health Expectations · doi:10.1111/hex.12888 · Cited by 1,059 · cited 1×
“…the world, it continues to hold interest in many areas of health research and practice, including health professions education (Rowland et al., 2018); health care research (Greenhalgh et al., 2019); health policy (Abelson et al., 2004); and quality improvement and innovation (Donetto et al., 2015).”
Background
Cited by
What factors contribute to the acceptance of artificial intelligence? A systematic review
Kelly, S., Kaye, S.-A., & Oviedo-Trespalacios, O. (2023) · Telematics and Informatics · doi:10.1016/j.tele.2022.101925 · Cited by 1,008
Power to the People? Opportunities and Challenges for Participatory AI
Birhane, A., Isaac, W., Prabhakaran, V., Diaz, M., Elish, M. C., Gabriel, I., & Mohamed, S. (2022) · Equity and Access in Algorithms Mechanisms and Optimization · doi:10.1145/3551624.3555290 · Cited by 330
“as well as the limits of participation in specific applications, such as healthcare [25].”
BackgroundPrakash, S., Balaji, J. N., Joshi, A., & Surapaneni, K. M. (2022) · Journal of Personalized Medicine · doi:10.3390/jpm12111914 · Cited by 159
Sankar, B. S., Gilliland, D., Rincon, J., Hermjakob, H., Yan, Y., Adam, I., Lemaster, G., Wang, D., Watson, K., Bui, A., et al. (2024) · Bioengineering · doi:10.3390/bioengineering11100984 · Cited by 26
Artificial intelligence and health equity in primary care: A qualitative study with key stakeholders
d’Elia, A., Gabbay, M., Frith, L., Rodgers, S., & Kierans, C. (2023) · doi:10.1101/2023.10.25.23297533 · Cited by 5
Metadata
- Authors
- Donia, J., & Shaw, J. A.
- Venue
- Big Data & Society
- Publisher
- SAGE Publications
- Published
- 2021-07
- Volume
- 8
- Issue
- 2
- ISSN
- 2053-9517 (ISSN-L)
- OpenAlex
- W4205467669
- Cited by
- 96
- References
- 23
- FWCI
- 2.27
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Co-design and ethical artificial intelligence for health: An agenda for critical research and practice. https://app.cochat.ai/papers/10.1177/20539517211065248
- The paper
- Donia, J., & Shaw, J. A. (2021). Co-design and ethical artificial intelligence for health: An agenda for critical research and practice. Big Data & Society. https://doi.org/10.1177/20539517211065248
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.