The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Development and preliminary testing of Health Equity Across the AI Lifecycle (HEAAL): A framework for healthcare delivery organizations to mitigate the risk of AI solutions worsening health inequities
Open accessPublished version of a preprint
Abstract
The use of data-driven technologies such as Artificial Intelligence (AI) and Machine Learning (ML) is growing in healthcare. However, the proliferation of healthcare AI tools has outpaced regulatory frameworks, accountability measures, and governance standards to ensure safe, effective, and equitable use. To address these gaps and tackle a common challenge faced by healthcare delivery organizations, a case-based workshop was organized, and a framework was developed to evaluate the potential impact of implementing an AI solution on health equity. The Health Equity Across the AI Lifecycle (HEAAL) is co-designed with extensive engagement of clinical, operational, technical, and regulatory leaders across healthcare delivery organizations and ecosystem partners in the US. It assesses 5 equity assessment domains–accountability, fairness, fitness for purpose, reliability and validity, and transparency–across the span of eight key decision points in the AI adoption lifecycle. It is a process-oriented framework containing 37 step-by-step procedures for evaluating an existing AI solution and 34 procedures for evaluating a new AI solution in total. Within each procedure, it identifies relevant key stakeholders and data sources used to conduct the procedure. HEAAL guides how healthcare delivery organizations may mitigate the potential risk of AI solutions worsening health inequities. It also informs how much resources and support are required to assess the potential impact of AI solutions on health inequities.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Unchecked bias in health communications can also contribute to widening health disparities (10, 15, 16).”
From The Health Communications Assessment Checklist: a teaching tool for uncovering bias in the age of AI (2026) · cites this paper as [15]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Kim et al.30, HEAAL framework, is a comprehensive tool that evaluates AI across several domains: accountability, fairness, fitness to purpose, reliability and validity, and transparency throughout the lifecycle.”
From Advancing healthcare AI governance through a comprehensive maturity model based on systematic review (2026) · cites this paper as [30]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 2×
“…with unprecedented data [22], bias captured in specific data measurements [23,24], bias captured in unstructured text [25,26], bias embedded within outcome labels used to train models [11,12], and models learning shortcuts unrelated to disease process to generate diagnostic predictions [27,28].”
BackgroundRajkomar, A., Dean, J., & Kohane, I. (2019) · New England Journal of Medicine · doi:10.1056/nejmra1814259 · Cited by 4,579
The practical implementation of artificial intelligence technologies in medicine
He, J., Baxter, S. L., Xu, J., Xu, J., Zhou, X., & Zhang, K. (2019) · Nature Medicine · doi:10.1038/s41591-018-0307-0 · Cited by 2,246
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245
Self-citationRacial bias in pulse oximetry measurement
Sjoding, M. W., Dickson, R. P., Iwashyna, T. J., Gay, S. E., & Valley, T. S. (2020) · New England Journal of Medicine · doi:10.1056/nejmc2029240 · Cited by 1,114 · cited 1×
“…historic data to build an accurate model [21], an outlier event with unprecedented data [22], bias captured in specific data measurements [23,24], bias captured in unstructured text [25,26], bias embedded within outcome labels used to train models [11,12], and models learning shortcuts…”
BackgroundBias in artificial intelligence algorithms and recommendations for mitigation
Nazer, L. H., Zatarah, R., Waldrip, S., Ke, J. X. C., Moukheiber, M., Khanna, A. K., Hicklen, R. S., Moukheiber, L., Moukheiber, D., Ma, H., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000278 · Cited by 621
A Framework of Potential Sources of Harm Throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. V. (2021) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 576
Self-citationSources of bias in artificial intelligence that perpetuate healthcare disparities—A global review
Celi, L. A., Cellini, J., Charpignon, M.-L., Dee, E. C., Dernoncourt, F., Eber, R., Mitchell, W. G., Moukheiber, L., Schirmer, J., Situ, J., et al. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000022 · Cited by 576
Wu, E., Wu, K., Daneshjou, R., Ouyang, D., Ho, D. E., & Zou, J. (2021) · Nature Medicine · doi:10.1038/s41591-021-01312-x · Cited by 494
Research works printed but not linked: 1
- YT Tamer, 2022, Know thy patient: a novel approach and method for patient segmentation and clustering using machine learning to develop holistic, patient-centered programs and treatment plans, NEJM Catalyst Innovations in Care Delivery, 3
Cited by
Poon, E. G., Lemak, C. H., Rojas, J. C., Guptill, J., & Classen, D. C. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf065 · Cited by 230
Algorithmic bias in public health AI: a silent threat to equity in low-resource settings
Joseph, J. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1643180 · Cited by 142
Pal, A., Wangmo, T., Bharadia, T., Ahmed-Richards, M., Bhanderi, M., Kachhadiya, R., Allemann, S. S., & Elger, B. S. (2025) · Patient Preference and Adherence · doi:10.2147/ppa.s527922 · Cited by 63
Advancing healthcare AI governance through a comprehensive maturity model based on systematic review
Hussein, R., Zink, A., Ramadan, B., Howard, F. M., Hightower, M., Shah, S., & Beaulieu-Jones, B. K. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02418-7 · Cited by 96
Generative Artificial Intelligence in Medicine
Shanmugam, D., Agrawal, M., Movva, R., Chen, I. Y., Ghassemi, M., Jacobs, M. L., & Pierson, E. (2025) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-103123-095332 · Cited by 40
Metadata
- Authors
- Kim, J. Y., Hasan, A., Kellogg, K. C., Ratliff, W., Murray, S. G., Suresh, H., Valladares, A., Shaw, K., Tobey, D., Vidal, D. E., Lifson, M. A., Patel, M., Raji, I. D., Gao, M., Knechtle, W., Tang, L., Balu, S., & Sendak, M. P.
- Affiliations
- Duke University (US)
- Massachusetts Institute of Technology (US)
- University of California, San Francisco (US)
- Cornell University (US)
- United States Food and Drug Administration (US)
- Mayo Clinic (US)
- Duke Medical Center (US)
- University of California, Berkeley (US)
- Johns Hopkins University (US)
- Corresponding author
- Kim, Jee Young
- ORCID
- Venue
- PLOS Digital Health
- Publisher
- Public Library of Science (PLoS)
- Published
- 2024-05-09
- Volume
- 3
- Issue
- 5
- Pages
- e0000390
- ISSN
- 2767-3170 (ISSN-L)
- PMC
- PMC11081364
- OpenAlex
- W4396766072
- Cited by
- 61
- References
- 36
- Citations by year
- 2026: 26 · 2025: 21 · 2024: 13
- FWCI
- 5.61
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLOS Digital Health (publishedVersion, cc-by)
- medRxiv (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Gordon and Betty Moore Foundation (10849)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- health equity, artificial intelligence in healthcare, AI lifecycle, health disparities, equity assessment, accountability, algorithmic fairness, transparency, reliability and validity, AI governance, health care delivery organization, artificial intelligence adoption, risk mitigation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Development and preliminary testing of Health Equity Across the AI Lifecycle (HEAAL): A framework for healthcare delivery organizations to mitigate the risk of AI solutions worsening health inequities. https://app.cochat.ai/papers/10.1371/journal.pdig.0000390
- The paper
- Kim, J. Y., Hasan, A., Kellogg, K. C., Ratliff, W., Murray, S. G., Suresh, H., Valladares, A., Shaw, K., Tobey, D., Vidal, D. E., Lifson, M. A., Patel, M., Raji, I. D., Gao, M., Knechtle, W., Tang, L., Balu, S., & Sendak, M. P. (2024). Development and preliminary testing of Health Equity Across the AI Lifecycle (HEAAL): A framework for healthcare delivery organizations to mitigate the risk of AI solutions worsening health inequities. PLOS Digital Health. https://doi.org/10.1371/journal.pdig.0000390
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.