Development and preliminary testing of Health Equity Across the AI Lifecycle (HEAAL): A framework for healthcare delivery organizations to mitigate the risk of AI solutions worsening health inequities
Open accessPublished version of a preprint
The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
The use of data-driven technologies such as Artificial Intelligence (AI) and Machine Learning (ML) is growing in healthcare. However, the proliferation of healthcare AI tools has outpaced regulatory frameworks, accountability measures, and governance standards to ensure safe, effective, and equitable use. To address these gaps and tackle a common challenge faced by healthcare delivery organizations, a case-based workshop was organized, and a framework was developed to evaluate the potential impact of implementing an AI solution on health equity. The Health Equity Across the AI Lifecycle (HEAAL) is co-designed with extensive engagement of clinical, operational, technical, and regulatory leaders across healthcare delivery organizations and ecosystem partners in the US. It assesses 5 equity assessment domains–accountability, fairness, fitness for purpose, reliability and validity, and transparency–across the span of eight key decision points in the AI adoption lifecycle. It is a process-oriented framework containing 37 step-by-step procedures for evaluating an existing AI solution and 34 procedures for evaluating a new AI solution in total. Within each procedure, it identifies relevant key stakeholders and data sources used to conduct the procedure. HEAAL guides how healthcare delivery organizations may mitigate the potential risk of AI solutions worsening health inequities. It also informs how much resources and support are required to assess the potential impact of AI solutions on health inequities.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Unchecked bias in health communications can also contribute to widening health disparities (10, 15, 16).”
From The Health Communications Assessment Checklist: a teaching tool for uncovering bias in the age of AI (2026) · cites this paper as [15]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Kim et al.30, HEAAL framework, is a comprehensive tool that evaluates AI across several domains: accountability, fairness, fitness to purpose, reliability and validity, and transparency throughout the lifecycle.”
From Advancing healthcare AI governance through a comprehensive maturity model based on systematic review (2026) · cites this paper as [30]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 2×
“…with unprecedented data [22], bias captured in specific data measurements [23,24], bias captured in unstructured text [25,26], bias embedded within outcome labels used to train models [11,12], and models learning shortcuts unrelated to disease process to generate diagnostic predictions [27,28].”
BackgroundRajkomar, A., Dean, J., & Kohane, I. (2019) · New England Journal of Medicine · doi:10.1056/nejmra1814259 · Cited by 4,579
The practical implementation of artificial intelligence technologies in medicine
He, J., Baxter, S. L., Xu, J., Xu, J., Zhou, X., & Zhang, K. (2019) · Nature Medicine · doi:10.1038/s41591-018-0307-0 · Cited by 2,246
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245
Self-citationRacial bias in pulse oximetry measurement
Sjoding, M. W., Dickson, R. P., Iwashyna, T. J., Gay, S. E., & Valley, T. S. (2020) · New England Journal of Medicine · doi:10.1056/nejmc2029240 · Cited by 1,114 · cited 1×
“…historic data to build an accurate model [21], an outlier event with unprecedented data [22], bias captured in specific data measurements [23,24], bias captured in unstructured text [25,26], bias embedded within outcome labels used to train models [11,12], and models learning shortcuts…”
BackgroundBias in artificial intelligence algorithms and recommendations for mitigation
Nazer, L. H., Zatarah, R., Waldrip, S., Ke, J. X. C., Moukheiber, M., Khanna, A. K., Hicklen, R. S., Moukheiber, L., Moukheiber, D., Ma, H., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000278 · Cited by 621
A Framework of Potential Sources of Harm Throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. V. (2021) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 576
Self-citationSources of bias in artificial intelligence that perpetuate healthcare disparities—A global review
Celi, L. A., Cellini, J., Charpignon, M.-L., Dee, E. C., Dernoncourt, F., Eber, R., Mitchell, W. G., Moukheiber, L., Schirmer, J., Situ, J., et al. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000022 · Cited by 576
Wu, E., Wu, K., Daneshjou, R., Ouyang, D., Ho, D. E., & Zou, J. (2021) · Nature Medicine · doi:10.1038/s41591-021-01312-x · Cited by 494
Daneshjou, R., Smith, M. P., Sun, M. D., Rotemberg, V., & Zou, J. (2021) · JAMA Dermatology · doi:10.1001/jamadermatol.2021.3129 · Cited by 403
Artificial intelligence vs COVID-19: limitations, constraints and pitfalls
Naudé, W. (2020) · AI & SOCIETY · doi:10.1007/s00146-020-00978-0 · Cited by 368
Adams, R., Henry, K. E., Sridharan, A., Soleimani, H., Zhan, A., Rawat, N., Johnson, L., Hager, D. N., Cosgrove, S. E., Markowski, A., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01894-0 · Cited by 365
Considerations for addressing bias in artificial intelligence for health equity
Abràmoff, M. D., Tarver, M. E., Loyo-Berrios, N., Trujillo, S., Char, D., Obermeyer, Z., Eydelman, M. B., Foundational Principles of Ophthalmic Imaging and Algorithmic Interpretation Working Group of the Collaborative Community for Ophthalmic Imaging Foundation, W., & Maisel, W. H. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00913-9 · Cited by 349
Sun, M., Oliwa, T., Peek, M. E., & Tung, E. L. (2022) · Health Affairs · doi:10.1377/hlthaff.2021.01423 · Cited by 340 · cited 1×
“Numerous academic papers have surfaced potential causes of bias in AI products, including lack of representation and diversity in model training data [18-20], lack of sufficient historic data to build an accurate model [21], an outlier event with unprecedented data [22], bias captured in specific data measurements [23…”
BackgroundAutomated Identification of Adults at Risk for In-Hospital Clinical Deterioration
Escobar, G. J., Liu, V. X., Schuler, A., Lawson, B., Greene, J. D., & Kipnis, P. (2020) · New England Journal of Medicine · doi:10.1056/nejmsa2001090 · Cited by 314 · cited 1×
“For example, the integration of AI and ML in healthcare helps in the detection and management of sepsis [5], preventing unanticipated intensive care unit transfers [6], and automated calculation of left ventricular ejection fraction [7].”
MethodDeep learning predicts hip fracture using confounding patient and healthcare variables
Badgeley, M. A., Zech, J. R., Oakden-Rayner, L., Glicksberg, B. S., Liu, M., Gale, W., McConnell, M. V., Percha, B., Snyder, T. M., & Dudley, J. T. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0105-1 · Cited by 290
An algorithmic approach to reducing unexplained pain disparities in underserved populations
Pierson, E., Cutler, D. M., Leskovec, J., Mullainathan, S., & Obermeyer, Z. (2021) · Nature Medicine · doi:10.1038/s41591-020-01192-7 · Cited by 268
Blinded, randomized trial of sonographer versus AI cardiac function assessment
He, B., Kwan, A. C., Cho, J. H., Yuan, N., Pollick, C., Shiota, T., Ebinger, J., Bello, N. A., Wei, J., Josan, K., et al. (2023) · Nature · doi:10.1038/s41586-023-05947-3 · Cited by 256
Geographic Distribution of US Cohorts Used to Train Deep Learning Algorithms
Kaushal, A., Altman, R., & Langlotz, C. (2020) · JAMA · doi:10.1001/jama.2020.12067 · Cited by 239 · cited 1×
“Numerous academic papers have surfaced potential causes of bias in AI products, including lack of representation and diversity in model training data [18-20], lack of sufficient historic data to build an accurate model [21], an outlier event with unprecedented data [22], bias captured in specific data measurements [23…”
BackgroundEvaluation and Mitigation of Racial Bias in Clinical Machine Learning Models: Scoping Review
Huang, J., Galal, G., Etemadi, M., & Vaidyanathan, M. (2022) · JMIR Medical Informatics · doi:10.2196/36388 · Cited by 186 · cited 1×
“But to date, there has yet to be a comprehensive set of actionable procedures across the AI product lifecycle for healthcare delivery organization leaders to adopt internally to mitigate the risk of AI products worsening health inequities.”
BackgroundRaji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · arXiv · doi:10.48550/arxiv.2001.00973 · Cited by 179
Self-citationA Path for Translation of Machine Learning Products into Healthcare Delivery
(2020) · EMJ Innovations · doi:10.33590/emjinnov/19-00172 · Cited by 172 · cited 1×
“However, the proliferation of healthcare AI tools has outpaced regulatory frameworks, accountability measures, and governance standards to ensure safe, effective, and equitable use [3,9,10].”
BackgroundSelf-citationAddressing algorithmic bias and the perpetuation of health inequities: An AI bias aware framework
Agarwal, R., Bjarnadottir, M., Rhue, L., Dugas, M., Crowley, K., Clark, J., & Gao, G. (2023) · Health Policy and Technology · doi:10.1016/j.hlpt.2022.100702 · Cited by 170
Zhang, Y., Wang, S., Hermann, A., Joly, R., & Pathak, J. (2021) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.09.113 · Cited by 164
Wang, L., Zhou, Y., He, J., Zhu, B., Wang, F., Tang, L., Kleinsasser, M. J., Barker, D., Eisenberg, M. C., & Song, P. X. ‐. (2021) · Journal of Data Science · doi:10.6339/jds.202007_18(3).0003 · Cited by 149
Self-citationAn epidemiological forecast model and software assessing interventions on COVID-19 epidemic in ChinaPreprint
Wang, L., Zhou, Y., He, J., et al. (2020) · medRxiv · doi:10.1101/2020.02.29.20029421 · Cited by 128
A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. (2021) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3465416.3483305 · Cited by 82
Self-citationA Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. V. (2019) · arXiv · doi:10.48550/arxiv.1901.10002 · Cited by 82
Self-citationWang, H. E., Landers, M., Adams, R., Subbaswamy, A., Kharrazi, H., Gaskin, D. J., & Saria, S. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac065 · Cited by 63 · cited 1×
“But to date, there has yet to be a comprehensive set of actionable procedures across the AI product lifecycle for healthcare delivery organization leaders to adopt internally to mitigate the risk of AI products worsening health inequities.”
BackgroundOrganizational Governance of Emerging Technologies: AI Adoption in Healthcare
Kim, J. Y., Boag, W., Gulamali, F., Hasan, A., Hogg, H. D. J., Lifson, M., Mulligan, D., Patel, M., Raji, I. D., Sehgal, A., et al. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594089 · Cited by 55 · cited 3×
“We interviewed 89 individuals from 10 US healthcare delivery organizations and ecosystem partners [35].”
BackgroundMethodInfluentialSelf-citationBhavani, S. V., Wiley, Z., Verhoef, P. A., Coopersmith, C. M., & Ofotokun, I. (2022) · JAMA · doi:10.1001/jama.2022.12290 · Cited by 42
Adam, H., Yang, M. Y., Cato, K., Baldini, I., Senteio, C., Celi, L. A., Zeng, J., Singh, M., & Ghassemi, M. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534203 · Cited by 33
Mccradden, M., Odusi, O., Joshi, S., Akrout, I., Ndlovu, K., Glocker, B., Maicas, G., Liu, X., Mazwi, M., Garnett, T., et al. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3594096 · Cited by 33
Enabling collaborative governance of medical AI
Price, W. N., Sendak, M., Balu, S., & Singh, K. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00699-1 · Cited by 27
Self-citationConfounding variables can degrade generalization performance of radiological deep learning models
Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., & Oermann, E. K. (2018) · arXiv · doi:10.48550/arxiv.1807.00431 · Cited by 26
Discrimination By Artificial Intelligence In A Commercial Electronic Health Record—A Case Study
(2020) · Forefront Group · doi:10.1377/forefront.20200128.626576 · Cited by 17
Sendak, M., Kim, J. Y., Hasan, A., Ratliff, W., Lifson, M. A., Patel, M. R., Raji, I. D., Sehgal, A., Shaw, K., Tobey, D., et al. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000513 · Cited by 5
Self-citation
Research works printed but not linked: 1
- YT Tamer, 2022, Know thy patient: a novel approach and method for patient segmentation and clustering using machine learning to develop holistic, patient-centered programs and treatment plans, NEJM Catalyst Innovations in Care Delivery, 3
Cited by
Poon, E. G., Lemak, C. H., Rojas, J. C., Guptill, J., & Classen, D. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf065 · Cited by 143
Algorithmic bias in public health AI: a silent threat to equity in low-resource settings
Joseph, J. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1643180 · Cited by 142
Pal, A., Wangmo, T., Bharadia, T., Ahmed-Richards, M., Bhanderi, M., Kachhadiya, R., Allemann, S. S., & Elger, B. S. (2025) · Patient Preference and Adherence · doi:10.2147/ppa.s527922 · Cited by 63
Advancing healthcare AI governance through a comprehensive maturity model based on systematic review
Hussein, R., Zink, A., Ramadan, B., Howard, F. M., Hightower, M., Shah, S., & Beaulieu-Jones, B. K. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02418-7 · Cited by 96
Generative Artificial Intelligence in Medicine
Shanmugam, D., Agrawal, M., Movva, R., Chen, I. Y., Ghassemi, M., Jacobs, M. L., & Pierson, E. (2025) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-103123-095332 · Cited by 41
Metadata
- Authors
- Kim, J. Y., Hasan, A., Kellogg, K. C., Ratliff, W., Murray, S. G., Suresh, H., Valladares, A., Shaw, K., Tobey, D., Vidal, D. E., Lifson, M. A., Patel, M., Raji, I. D., Gao, M., Knechtle, W., Tang, L., Balu, S., & Sendak, M. P.
- Affiliations
- Duke University (US)
- Massachusetts Institute of Technology (US)
- University of California, San Francisco (US)
- Cornell University (US)
- United States Food and Drug Administration (US)
- Mayo Clinic (US)
- Duke Medical Center (US)
- University of California, Berkeley (US)
- Johns Hopkins University (US)
- Corresponding author
- Kim, Jee Young
- ORCID
- Venue
- PLOS Digital Health
- Publisher
- Public Library of Science (PLoS)
- Published
- 2024-05-09
- Volume
- 3
- Issue
- 5
- Pages
- e0000390
- ISSN
- 2767-3170 (ISSN-L)
- PMC
- PMC11081364
- OpenAlex
- W4396766072
- Cited by
- 61
- References
- 36
- Citations by year
- 2026: 26 · 2025: 21 · 2024: 13
- FWCI
- 5.61
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLOS Digital Health (publishedVersion, cc-by)
- medRxiv (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Gordon and Betty Moore Foundation (10849)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- health equity, artificial intelligence in healthcare, AI lifecycle, health disparities, equity assessment, accountability, algorithmic fairness, transparency, reliability and validity, AI governance, health care delivery organization, artificial intelligence adoption, risk mitigation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.