The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Patient safety and quality improvement: Ethical principles for a regulatory approach to bias in healthcare machine learning
Open accessAbstract
Accumulating evidence demonstrates the impact of bias that reflects social inequality on the performance of machine learning (ML) models in health care. Given their intended placement within healthcare decision making more broadly, ML tools require attention to adequately quantify the impact of bias and reduce its potential to exacerbate inequalities. We suggest that taking a patient safety and quality improvement approach to bias can support the quantification of bias-related effects on ML. Drawing from the ethical principles underpinning these approaches, we argue that patient safety and quality improvement lenses support the quantification of relevant performance metrics, in order to minimize harm while promoting accountability, justice, and transparency. We identify specific methods for operationalizing these principles with the goal of attending to bias to support better decision making in light of controllable and uncontrollable factors.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“It is advised that trial outcomes are also evaluated per clinically relevant user group12 or per affected non-user group (also in terms of algorithmic bias)12,24,91.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [91]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Performance analysis by population subgroups or specific problematic use cases is recommended to identify algorithmic bias6,7,23,24,26,42,61,72,79,91.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [91]
In the citing paper
“To further facilitate the implementation and consecutive monitoring, the literature recommends automatic AIPM deployment (moving software from testing to production environments with automated processes) and the facilitation of shadow deployment66,91, which enables prospective local validation (see phase 3) of new versions and updates19.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [91]
In the citing paper
“For example, end users may be made aware of the possibility of automation bias and trained to maintain vigilance22,26,57,87,91.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [91]
In the citing paper
“Second, the monitoring of dataset shift is also repeatedly mentioned in the literature5,10,22,72,79,91.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [91]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 2×
“7,8,15 When model errors are unevenly distributed then, so too are the risks.”
BackgroundUnequal treatment: confronting racial and ethnic disparities in health care
(2003) · Choice Reviews Online · doi:10.5860/choice.40-5843 · Cited by 7,043
Social determinants of health inequalities
Marmot, M. (2005) · The Lancet · doi:10.1016/s0140-6736(05)71146-6 · Cited by 5,086
Do no harm: a roadmap for responsible machine learning for health care
Wiens, J., Saria, S., Sendak, M., Ghassemi, M., Liu, V. X., Doshi-Velez, F., Jung, K., Heller, K., Kale, D., Saeed, M., et al. (2019) · Nature Medicine · doi:10.1038/s41591-019-0548-6 · Cited by 1,266
Self-citationWord embeddings quantify 100 years of gender and ethnic stereotypes
Garg, N., Schiebinger, L., Jurafsky, D., & Zou, J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1720347115 · Cited by 1,065 · cited 1×
“7 16 It is less clear how to deal with this challenge.”
BackgroundMachine Learning and Health Care Disparities in Dermatology
Adamson, A. S., & Smith, A. (2018) · JAMA Dermatology · doi:10.1001/jamadermatol.2018.2348 · Cited by 639 · cited 1×
“14 Attending to variations in decisional accuracy across subgroups can promote justice by encouraging due diligence with respect to appropriate model use across these groups.”
BackgroundHidden stratification causes clinically meaningful failures in machine learning for medical imaging
Oakden-Rayner, L., Dunnmon, J., Carneiro, G., & Re, C. (2020) · Proceedings of the ACM Conference on Health, Inference, and Learning · doi:10.1145/3368555.3384468 · Cited by 411
Hernandez-Boussard, T., Bozkurt, S., Ioannidis, J. P. A., & Shah, N. H. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa088 · Cited by 410
Can AI Help Reduce Disparities in General Medical and Mental Health Care?
(2019) · AMA Journal of Ethics · doi:10.1001/amajethics.2019.167 · Cited by 401
Assessing risk, automating racism
Benjamin, R. (2019) · Science · doi:10.1126/science.aaz3873 · Cited by 378 · cited 2×
“The problem of bias for healthcare ML Recent evidence indicates that model performance discrepancies, as a function of protected identities (meaning characteristics protected by human and civil rights legislation [eg, race, gender]), can have significant health implications for vulnerable groups, potentially worsening…”
Background
Cited by
Significance of machine learning in healthcare: Features, pillars and applications
Javaid, M., Haleem, A., Pratap Singh, R., Suman, R., & Rab, S. (2022) · International Journal of Intelligent Networks · doi:10.1016/j.ijin.2022.05.002 · Cited by 641
de Hond, A. A. H., Leeuwenberg, A. M., Hooft, L., Kant, I. M. J., Nijman, S. W. J., van Os, H. J. A., Aardoom, J. J., Debray, T. P. A., Schuit, E., van Smeden, M., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-021-00549-7 · Cited by 499
SHIFTing artificial intelligence to be responsible in healthcare: a systematic review
Siala, H., & Wang, Y. (2022) · Social Science & Medicine · doi:10.1016/j.socscimed.2022.114782 · Cited by 358
Chin, M. H., Afsar-Manesh, N., Bierman, A. S., Chang, C., Colón-Rodríguez, C. J., Dullabh, P., Duran, D. G., Fair, M., Hernandez-Boussard, T., Hightower, M., et al. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.45050 · Cited by 275
Responsible AI Pattern Catalogue: A Collection of Best Practices for AI Governance and Engineering
Lu, Q., Zhu, L., Xu, X., Whittle, J., Zowghi, D., & Jacquet, A. (2023) · ACM Computing Surveys · doi:10.1145/3626234 · Cited by 270
Metadata
- Authors
- McCradden, M. D., Joshi, S., Anderson, J. A., Mazwi, M., Goldenberg, A., & Zlotnik Shaul, R.
- Affiliations
- Hospital for Sick Children (CA)
- Vector Institute (CA)
- University of Toronto (CA)
- Canadian Institute for Advanced Research (CA)
- Bioethics Department, The Hospital for Sick Children, Toronto, Ontario, Canada
- Corresponding author
- McCradden, Melissa D
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2020-06-25
- Volume
- 27
- Issue
- 12
- Pages
- 2024-2027
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC7727331
- OpenAlex
- W3037637030
- Cited by
- 111
- References
- 14
- Citations by year
- 2026: 17 · 2025: 15 · 2024: 21 · 2023: 20 · 2022: 18 · 2021: 15 · 2020: 5
- FWCI
- 19.73
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model (vor)
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- algorithmic bias, machine learning in healthcare, patient safety, quality improvement, ethical principles, social inequality, health equity, accountability, justice, transparency, bias quantification, healthcare decision-making, regulatory approaches
- MeSH terms
- Artificial Intelligence, Data Collection, Humans, Prejudice, Government Regulation, Healthcare Disparities, Quality Improvement, Patient Safety, Social Determinants of Health
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Patient safety and quality improvement: Ethical principles for a regulatory approach to bias in healthcare machine learning. https://app.cochat.ai/papers/10.1093/jamia/ocaa085
- The paper
- McCradden, M. D., Joshi, S., Anderson, J. A., Mazwi, M., Goldenberg, A., & Zlotnik Shaul, R. (2020). Patient safety and quality improvement: Ethical principles for a regulatory approach to bias in healthcare machine learning. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocaa085
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.