The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Human-Centered Design to Address Biases in Artificial Intelligence
Open accessAbstract
The potential of artificial intelligence (AI) to reduce health care disparities and inequities is recognized, but it can also exacerbate these issues if not implemented in an equitable manner. This perspective identifies potential biases in each stage of the AI life cycle, including data collection, annotation, machine learning model development, evaluation, deployment, operationalization, monitoring, and feedback integration. To mitigate these biases, we suggest involving a diverse group of stakeholders, using human-centered AI principles. Human-centered AI can help ensure that AI systems are designed and used in a way that benefits patients and society, which can reduce health disparities and inequities. By recognizing and addressing biases at each stage of the AI life cycle, AI can achieve its potential in health care.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 2×
“On the contrary, AI also has the potential to amplify disparities and inequalities in health care [11-13].”
Artificial intelligence in healthcare
Yu, K.-H., Beam, A. L., & Kohane, I. S. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0305-z · Cited by 2,817
Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy
Shneiderman, B. (2020) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1741118 · Cited by 1,743
Hidden in Plain Sight — Reconsidering the Use of Race Correction in Clinical Algorithms
Vyas, D. A., Eisenstein, L. G., & Jones, D. S. (2020) · New England Journal of Medicine · doi:10.1056/nejmms2004740 · Cited by 1,731
Health disparities and health equity: Concepts and measurement
Braveman, P. (2006) · Annual Review of Public Health · doi:10.1146/annurev.publhealth.27.021405.102103 · Cited by 1,570
An interpretable machine learning model for accurate prediction of sepsis in the ICU
Nemati, S., Holder, A., Razmi, F., Stanley, M. D., Clifford, G. D., & Buchman, T. G. (2018) · Critical Care Medicine · doi:10.1097/ccm.0000000000002936 · Cited by 832
Shneiderman, B. (2020) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3419764 · Cited by 768
Fleuren, L. M., Klausch, T. L. T., Zwager, C. L., Schoonmade, L. J., Guo, T., Roggeveen, L. F., Swart, E. L., Girbes, A. R. J., Thoral, P., Ercole, A., et al. (2020) · Intensive Care Medicine · doi:10.1007/s00134-019-05872-y · Cited by 755 · cited 1×
“This can lead to a scenario where a patient’s condition is not treated appropriately, such that their health status deteriorates [19].”
Inequality and the health-care system in the USA
Dickman, S. L., Himmelstein, D. U., & Woolhandler, S. (2017) · The Lancet · doi:10.1016/s0140-6736(17)30398-7 · Cited by 656
de Hond, A. A. H., Leeuwenberg, A. M., Hooft, L., Kant, I. M. J., Nijman, S. W. J., van Os, H. J. A., Aardoom, J. J., Debray, T. P. A., Schuit, E., van Smeden, M., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-021-00549-7 · Cited by 499
Causal inference and counterfactual prediction in machine learning for actionable healthcare
Prosperi, M., Guo, Y., Sperrin, M., Koopman, J. S., Min, J. S., He, X., Rich, S., Wang, M., Buchan, I. E., & Bian, J. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0197-y · Cited by 459 · cited 1×
“For instance, AI or ML practitioners can apply resampling or oversampling methods to balance the data [43] and counterfactual analysis, which reveals how decisions are made by the model to ensure that the AI system does not discriminate against certain groups of patients [44,45].”
Competencies for the Use of Artificial Intelligence–Based Tools by Health Care Professionals
Russell, R. G., Lovett Novak, L., Patel, M., Garvey, K. V., Craig, K. J. T., Jackson, G. P., Moore, D., & Miller, B. M. (2023) · Academic Medicine · doi:10.1097/acm.0000000000004963 · Cited by 320 · cited 1×
“• AI education or outreach specialists design and deliver education and outreach programs about AI for a variety of audiences [35-37].”
Xu, J., Yang, P., Xue, S., Sharma, B., Sanchez-Martin, M., Wang, F., Beaty, K. A., Dehan, E., & Parikh, B. (2019) · Human Genetics · doi:10.1007/s00439-019-01970-5 · Cited by 305
Toward robust mammography-based models for breast cancer risk
Yala, A., Mikhael, P. G., Strand, F., Lin, G., Smith, K., Wan, Y.-L., Lamb, L., Hughes, K., Lehman, C., & Barzilay, R. (2021) · Science Translational Medicine · doi:10.1126/scitranslmed.aba4373 · Cited by 287 · cited 1×
“As an illustration of how this can be achieved, AI researchers have been partnering with people from health organizations around the world to validate and develop strategies to implement a risk assessment algorithm for breast cancer for diverse populations [57].”
Desouza, K. C., Dawson, G. S., & Chenok, D. (2020) · Business Horizons · doi:10.1016/j.bushor.2019.11.004 · Cited by 271
An algorithmic approach to reducing unexplained pain disparities in underserved populations
Pierson, E., Cutler, D. M., Leskovec, J., Mullainathan, S., & Obermeyer, Z. (2021) · Nature Medicine · doi:10.1038/s41591-020-01192-7 · Cited by 268
Artificial Intelligence in Health Care: The Hope, the Hype, the Promise, the Peril
Series, T. L. H. S., & National Academy of Medicine (2019) · doi:10.17226/27111 · Cited by 227
Bias detectives: the researchers striving to make algorithms fair
Courtland, R. (2018) · Nature · doi:10.1038/d41586-018-05469-3 · Cited by 205 · cited 1×
“Notably, AI itself offers the potential to detect and mitigate biases in AI systems by involving open-source bias auditing tools [24].”
Explainable AI for Healthcare: From Black Box to Interpretable Models
Adadi, A., & Berrada, M. (2020) · Advances in Intelligent Systems and Computing · doi:10.1007/978-981-15-0947-6_31 · Cited by 137 · cited 1×
“Various studies show that the HCAI framework can be used to develop explainable AI methods to make the model more interpretable and to involve domain experts in the process of interpreting the model’s results and performances and providing feedback to improve the model [66,67].”
The Landscape and Gaps in Open Source Fairness Toolkits
Lee, M. S. A., & Singh, J. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445261 · Cited by 129
Walsh, C. G., Chaudhry, B., Dua, P., Goodman, K. W., Kaplan, B., Kavuluru, R., Solomonides, A., & Subbian, V. (2020) · JAMIA Open · doi:10.1093/jamiaopen/ooz054 · Cited by 118 · cited 2×
“Algorithmic bias refers to a term that transcends the technical definition of bias to encompass the broader societal meaning of prejudice and discrimination [54].”
The AI life cycle: a holistic approach to creating ethical AI for health decisions
Ng, M. Y., Kapur, S., Blizinsky, K. D., & Hernandez-Boussard, T. (2022) · Nature Medicine · doi:10.1038/s41591-022-01993-y · Cited by 71 · cited 1×
“The AI life cycle refers to the process of designing, developing, and deploying AI systems, which typically includes data collection and selection; data annotation; model development and evaluation; and model deployment, monitoring, and maintenance [21,22].”
Garvey, K. V., Thomas Craig, K. J., Russell, R., Novak, L. L., Moore, D., & Miller, B. M. (2022) · JMIR Medical Informatics · doi:10.2196/37478 · Cited by 62
Self-citationWalsh, C. G., McKillop, M. M., Lee, P., Harris, J. W., Simpson, C., & Novak, L. L. (2021) · JAMIA Open · doi:10.1093/jamiaopen/ooab092 · Cited by 26
Self-citationKazimzade, G., & Miceli, M. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375809 · Cited by 16
Cited by
Enhancing mental health with Artificial Intelligence: Current trends and future prospects
Olawade, D. B., Wada, O. Z., Odetayo, A., David-Olawade, A. C., Asaolu, F., & Eberhardt, J. (2024) · Journal of Medicine, Surgery, and Public Health · doi:10.1016/j.glmedi.2024.100099 · Cited by 514
Yelne, S., Chaudhary, M., Dod, K., Sayyad, A., & Sharma, R. (2023) · Cureus · doi:10.7759/cureus.49252 · Cited by 190
Koçak, B., Ponsiglione, A., Stanzione, A., Bluethgen, C., Santinha, J., Ugga, L., Huisman, M., Klontzas, M. E., Cannella, R., & Cuocolo, R. (2024) · Diagnostic and Interventional Radiology · doi:10.4274/dir.2024.242854 · Cited by 168
“132,133 To reflect diverse healthcare landscapes and disparities across countries and regions, initiatives to improve diversity and representativeness in datasets, possibly globally, may support this goal.”
Rawas, S. (2024) · Discover Artificial Intelligence · doi:10.1007/s44163-024-00118-3 · Cited by 119
“Furthermore, AI’s ability to process and interpret complex medical pictures, such as MRI and CT scans, has shown outstanding accuracy in detecting anomalies and assisting radiologists in spotting probable problems that the human eye may ignore [10].”
InfluentialCognitive imperialism in artificial intelligence: counteracting bias with indigenous epistemologies
Ofosu-Asare, Y. (2025) · AI & SOCIETY · doi:10.1007/s00146-024-02065-0 · Cited by 97
Metadata
- Authors
- Chen, Y., Clayton, E. W., Novak, L. L., Anders, S., & Malin, B.
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2023-03-24
- Volume
- 25
- Pages
- e43251
- ISSN
- 1438-8871 (ISSN-L)
- OpenAlex
- W4360811013
- Cited by
- 199
- References
- 25
- FWCI
- 48.59
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.