Assessing risk, automating racism
The DOI matches its Crossref record; 18 of the 19 references it cites were found in a registry or an index.
Abstract
A health care algorithm reflects underlying racial bias in society
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Digital media, particularly with artificial intelligence (AI) and algorithmic decision-making, often rely on historical data that may contain biases, leading to the reinforcement of existing prejudices.172,173”
From Digital Ecosystems, Children, and Adolescents: Technical Report (2026) · cites this paper as [172]
In the citing paper
“Racism in particular has been a core focus of this literature, with multiple empirical examples of how structural racism becomes embedded in both the data used to create these tools and the output they produce.5–7”
From Equity and AI Governance at Academic Medical Centers (2024) · cites this paper as [5]
In the citing paper
“This literature includes analysis of how AI/ML and predictive models can reflect, entrench, or create existing inequities.”
From Equity and AI Governance at Academic Medical Centers (2024) · cites this paper as [5]
In the citing paper
“Algorithmic management might perpetuate societal biases and reproduce discriminatory practices at work, whether the discrimination is built into the algorithms, or management's use of the algorithm, or as a result of customer's rating of workers (Noble 2018;Benjamin 2019;Kellogg et al. 2020;Akter et al. 2021;Zajko 2021;Heinrichs 2022).”
From Artificial intelligence and work: a critical review of recent research from the social sciences (2024) · cites this paper as Benjamin 2019
Citations
Cites
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137
Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor
Lebovits, H. (2019) · Public Integrity · doi:10.1080/10999922.2018.1511671 · Cited by 3,279
Algorithms of Oppression: How Search Engines Reinforce Racism
Noble, S. U. (2020) · doi:10.2307/j.ctt1pwt9w5 · Cited by 2,709
Merz, S. (2020) · Ethnic and Racial Studies · doi:10.1080/01419870.2020.1715454 · Cited by 1,870
Review of “Race After Technology: Abolitionist Tools for the New Jim Code”
Jipguep-Akhtar, M. (2020) · Social Forces · doi:10.1093/sf/soz162 · Cited by 1,473
(2007) · Choice Reviews Online · doi:10.5860/choice.44-5683 · Cited by 1,296
(2018) · Cited by 1,167
Miller, J. (2007) · Psychiatric Services · doi:10.1176/appi.ps.58.10.1380 · Cited by 649
Body and Soul: The Black Panther Party and the Fight against Medical Discrimination
Nelson, A. (2011) · University of Minnesota Press eBooks · doi:10.5749/minnesota/9780816676484.001.0001 · Cited by 487
Reproducing Race: An Ethnography of Pregnancy as a Site of Racialization
Bridges, K. (2019) · University of California Press eBooks · doi:10.1525/9780520949447 · Cited by 289
Cited by
Ethical Machine Learning in Healthcare
Chen, I. Y., Pierson, E., Rose, S., Joshi, S., Ferryman, K., & Ghassemi, M. (2021) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-092820-114757 · Cited by 645
“While there is scholarship addressing social implications and algorithmic fairness in general, there has been less work at the intersection of health, ML, and fairness (14, 15, 16), despite the potential life-or-death impacts of those models (8, 17).”
BackgroundOpening the black box: the promise and limitations of explainable machine learning in cardiology
Petch, J., Di, S., & Nelson, W. (2022) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2021.09.004 · Cited by 552
“10 Black boxes also limit the clinical actionability of model predictions, which further undermines their usefulness to clinicians.”
Algorithmic bias in data-driven innovation in the age of AI
Akter, S., McCarthy, G., Sajib, S., Michael, K., Dwivedi, Y. K., D’Ambra, J., & Shen, K. N. (2021) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2021.102387 · Cited by 440
Kickbusch, I., Piselli, D., Agrawal, A., Balicer, R., Banner, O., Adelhardt, M., Capobianco, E., Fabian, C., Singh Gill, A., Lupton, D., et al. (2021) · The Lancet · doi:10.1016/s0140-6736(21)01824-9 · Cited by 414
Power to the People? Opportunities and Challenges for Participatory AI
Birhane, A., Isaac, W., Prabhakaran, V., Diaz, M., Elish, M. C., Gabriel, I., & Mohamed, S. (2022) · Equity and Access in Algorithms Mechanisms and Optimization · doi:10.1145/3551624.3555290 · Cited by 330
“In parallel to the expansion of data enrichment and moderation, problematic applications of machine learning tools in high stakes domains such as criminal justice [10, 57, 73], healthcare [7, 9], and hiring [2, 71] have prompted both researchers, civil society, and regulators to increasingly urge greater use participa…”
BackgroundEthical limitations of algorithmic fairness solutions in health care machine learning
McCradden, M. D., Joshi, S., Mazwi, M., & Anderson, J. A. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30065-0 · Cited by 291
Foundation and large language models: fundamentals, challenges, opportunities, and social impacts
Myers, D., Mohawesh, R., Chellaboina, V. I., Sathvik, A. L., Venkatesh, P., Ho, Y.-H., Henshaw, H., Alhawawreh, M., Berdik, D., & Jararweh, Y. (2024) · Cluster Computing · doi:10.1007/s10586-023-04203-7 · Cited by 185
“Similarly, inherently biased labels can lead to disparate outcomes for margin-alised groups compared to majority groups [15, 17].”
Advancements in technology and digital media in tourism
van Nuenen, T., & Scarles, C. (2021) · Tourist Studies · doi:10.1177/1468797621990410 · Cited by 176
“In doing so, we should not forget that the existence, designs, and implementations of algorithmic systems are a product of social forces (Beer, 2017), integrating human biases at many levels (e.g. Benjamin, 2019; O’Neil, 2016).”
BackgroundAn empirical characterization of fair machine learning for clinical risk prediction
Pfohl, S. R., Foryciarz, A., & Shah, N. H. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103621 · Cited by 164
“In the absence of this context, a requirement that a predictive model satisfy a notion of group fairness provides little more than a “veneer of neutrality” [17, 101].”
BackgroundEthical machines: The human-centric use of artificial intelligence
Lepri, B., Oliver, N., & Pentland, A. (2021) · iScience · doi:10.1016/j.isci.2021.102249 · Cited by 149
“…of individuals’ privacy (Crawford and Schultz, 2014; de Montjoye et al., 2013a, 2015; Ohm, 2010), lack of transparency and accountability (Citron and Pasquale, 2014; Pasquale, 2015; Zarsky, 2016), and biases and discrimination (Barocas and Selbst, 2016; Eubanks, 2018; Noble, 2018; Benjamin, 2019).”
Considering Biased Data as Informative Artifacts in AI-Assisted Health Care
Ferryman, K., Mackintosh, M., & Ghassemi, M. (2023) · New England Journal of Medicine · doi:10.1056/nejmra2214964 · Cited by 130
Ensuring that biomedical AI benefits diverse populations
Zou, J., & Schiebinger, L. (2021) · EBioMedicine · doi:10.1016/j.ebiom.2021.103358 · Cited by 124
“In the U.S., Black patients have more health needs, but when cost is used as a proxy for need, these needs are not adequately identi fi ed [21].”
BackgroundMcCoy, L. G., Brenna, C. T. A., Chen, S. S., Vold, K., & Das, S. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.11.001 · Cited by 118
“In-deed, prominent case studies have demonstrated precisely how explainable (and not intentionally biased) models can nonetheless deliver biased results that are only apparent upon empirical analysis [ 57 , 58 ].”
BackgroundA Research Ethics Framework for the Clinical Translation of Healthcare Machine Learning
McCradden, M. D., Anderson, J. A., A. Stephenson, E., Drysdale, E., Erdman, L., Goldenberg, A., & Zlotnik Shaul, R. (2022) · The American Journal of Bioethics · doi:10.1080/15265161.2021.2013977 · Cited by 114
Artificial intelligence and work: a critical review of recent research from the social sciences
Deranty, J.-P., & Corbin, T. (2024) · AI & SOCIETY · doi:10.1007/s00146-022-01496-x · Cited by 118
“…biases and reproduce discriminatory practices at work, whether the discrimination is built into the algorithms, or management’s use of the algorithm, or as a result of customer’s rating of workers (Noble 2018; Benjamin 2019; Kellogg et al. 2020; Akter et al. 2021; Zajko 2021; Heinrichs 2022).”
Ng, F. Y. C., Thirunavukarasu, A. J., Cheng, H., Tan, T. F., Gutierrez, L., Lan, Y., Ong, J. C. L., Chong, Y. S., Ngiam, K. Y., Ho, D., et al. (2023) · Cell Reports Medicine · doi:10.1016/j.xcrm.2023.101230 · Cited by 113
McCradden, M. D., Joshi, S., Anderson, J. A., Mazwi, M., Goldenberg, A., & Zlotnik Shaul, R. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa085 · Cited by 111
“The problem of bias for healthcare ML Recent evidence indicates that model performance discrepancies, as a function of protected identities (meaning characteristics protected by human and civil rights legislation [eg, race, gender]), can have significant health implications for vulnerable groups, potentially worsening…”
BackgroundPritchett, M., Ala’i-Rosales, S., Cruz, A. R., & Cihon, T. M. (2021) · Behavior Analysis in Practice · doi:10.1007/s40617-021-00591-7 · Cited by 124
“Both issues present barriers to human rights–oriented data-based design and decision making dedicated to the protection and betterment of marginalized persons (Benjamin, 2019).”
BackgroundHealth Policy and Privacy Challenges Associated With Digital Technology
Grande, D., Marti, X. L., Feuerstein-Simon, R., Merchant, R. M., Asch, D. A., Lewson, A. B., & Cannuscio, C. C. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.8285 · Cited by 128
Health information technology to improve care for people with multiple chronic conditions
Samal, L., Fu, H. N. C., Camara, D. S., Wang, J., Bierman, A. S., & Dorr, D. A. (2021) · Health Services Research · doi:10.1111/1475-6773.13860 · Cited by 137
“123,125,126 Racial and ethnic minorities and low-income individuals have more chronic conditions and develop them at an earlier age so could be at greater risk of potential harms from algorithms developed to support the care of PLWMCC. Integrating algorithms into patient-centered workflows to…”
BackgroundEquity in essence: a call for operationalising fairness in machine learning for healthcare
Wawira Gichoya, J., McCoy, L. G., Celi, L. A., & Ghassemi, M. (2021) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2020-100289 · Cited by 104
Interactive Model Cards: A Human-Centered Approach to Model Documentation
Crisan, A., Drouhard, M., Vig, J., & Rajani, N. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533108 · Cited by 101
“Not only does this mode of interrogation leave many non-experts underserved, it also exacerbates the potential for harm once these models are deployed [5, 6, 11, 15].”
BackgroundEvidence, ethics and the promise of artificial intelligence in psychiatry
McCradden, M., Hui, K., & Buchman, D. Z. (2023) · Journal of Medical Ethics · doi:10.1136/jme-2022-108447 · Cited by 97
Hayes, A. S. (2025) · International Journal of Qualitative Methods · doi:10.1177/16094069251322346 · Cited by 184
“Unlike traditional computational tools that rely on pre-determined rules or statistical pattern matching, LLMs demonstrate a form of language “ understanding ” that, while fundamentally different from human cognition, enables remarkably nuanced analysis (Karimova, 2024; Marcus & Davis, 2019) that can grasp semantic re…”
InfluentialA Human-Centered Systematic Literature Review of Cyberbullying Detection Algorithms
Kim, S., Razi, A., Stringhini, G., Wisniewski, P. J., & De Choudhury, M. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476066 · Cited by 79
“Scholars in the evolving and emergent area of humancentered machine learning have therefore argued that machine learning needs to stay grounded in human needs [22], models need to be built in inclusive ways that adequately represent the diverse experiences of different individuals and minimize biases [122], and that m…”
BackgroundCan AI be racist? Color‐evasiveness in the application of machine learning to science assessments
Cheuk, T. (2021) · Science Education · doi:10.1002/sce.21671 · Cited by 72
Conceptualising fairness: three pillars for medical algorithms and health equity
Sikstrom, L., Maslej, M. M., Hui, K., Findlay, Z., Buchman, D. Z., & Hill, S. L. (2022) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2021-100459 · Cited by 76
“5 34 177 Thus, fairness is not an outcome of rigorous and thoughtful research, but the social and political process required to advance health equity.”
Birk, R. H., & Samuel, G. N. (2020) · Sociology of Health & Illness · doi:10.1111/1467-9566.13175 · Cited by 85
Chasing Ghosts: Race, Racism, and the Future of Microbiome Research
Wolfe, T. J. D., Arefin, M. R., Benezra, A., & Rebolleda‐Gómez, M. (2021) · mSystems · doi:10.1128/msystems.00604-21 · Cited by 71
Artificial Intelligence and Medical Humanities
Ostherr, K. (2022) · Journal of Medical Humanities · doi:10.1007/s10912-020-09636-4 · Cited by 58
“…show how racial bias is embedded in natural language processing programs (Bolukbasi et al. 2016; Caliskan, Bryson, and Narayanan 2017), this body of research demonstrates the disastrous consequences that can arise from the use of algorithms to mediate decision-making in healthcare (Benjamin 2019).”
BackgroundAutomation and Augmentation: Artificial Intelligence, Robots, and Work
Lei, Y.-W., & Kim, R. (2024) · Annual Review of Sociology · doi:10.1146/annurev-soc-090523-050708 · Cited by 95
Those designing healthcare algorithms must become actively anti-racist
Owens, K., & Walker, A. N. (2020) · Nature Medicine · doi:10.1038/s41591-020-1020-3 · Cited by 62
Racialized algorithms for kidney function: Erasing social experience
Braun, L., Wentz, A., Baker, R., Richardson, E., & Tsai, J. (2021) · Social Science & Medicine · doi:10.1016/j.socscimed.2020.113548 · Cited by 56
Ferryman, K. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa133 · Cited by 55
“10 This discovery could have led to adjustments to their input data, and scrutiny of the validity of the association between healthcare expenditure and illness before the model was developed.”
BackgroundMaterialized Oppression in Medical Tools and Technologies
Liao, S., & Carbonell, V. (2022) · The American Journal of Bioethics · doi:10.1080/15265161.2022.2044543 · Cited by 67
“One risk-assessment algorithm that uses expected healthcare cost as a proxy for risk, which is used by clinicians to decide who is enrolled into “high-risk care management” programs, can underpredict the health risks of Black patients (Benjamin 2019; Obermeyer et al. 2019).”
BackgroundBeyond ‘AI for Social Good’ (AI4SG): social transformations—not tech-fixes—for health equity
Holzmeyer, C. (2021) · Interdisciplinary Science Reviews · doi:10.1080/03080188.2020.1840221 · Cited by 51
“…of exclusion and oppression while also being more opaque and di ffi cult to challenge, as ‘ Jim Crow practices feed the ‘ New Jim Code ’ – automated systems that hide, speed and deepen racial discrimination behind a veneer of technical neutrality ’ (Benjamin 2019b, 422; also c.f. Benjamin 2019a).”
BackgroundInfluentialBirk, R. H., & Samuel, G. N. (2022) · Current Psychiatry Reports · doi:10.1007/s11920-022-01358-9 · Cited by 61
“Further, the data that feeds into machine learning reflects the world, and the world is not neutral, but given shape by structural injustices and inequalities, including for health [34, 35 • , 36].”
BackgroundAlgorithmic harms and digital ageism in the use of surveillance technologies in nursing homes
Berridge, C., & Grigorovich, A. (2022) · Frontiers in Sociology · doi:10.3389/fsoc.2022.957246 · Cited by 47
“Writing about this study, Benjamin (2019a) has further argued that while Obermeyer’s analysis identified a harmful bias, it also points to the limits of solely targeting bias in analyses of individual algorithmic harms rather than focusing on the broader structural inequity that is produced by…”
BackgroundExplanatory pragmatism: a context-sensitive framework for explainable medical AI
Nyrup, R., & Robinson, D. (2022) · Ethics and Information Technology · doi:10.1007/s10676-022-09632-3 · Cited by 84
“For example, in medical contexts, algorithms have been found to systematically underestimate the healthcare needs of Black patients relative to white patients (Benjamin, 2019; Obermeyer et al., 2019).”
BackgroundBias does not equal bias: a socio-technical typology of bias in data-based algorithmic systems
Lopez, P. (2021) · Internet Policy Review · doi:10.14763/2021.4.1598 · Cited by 82
“An example of this kind of thinking is Lawrence Mead, an academic “leading expert on poverty” who was very influential in the welfare reform in the USA in the mid-1990s (Ramesh, 2010).”
BackgroundInfluentialBorder security and the digitalisation of sovereignty: insights from EU borderwork
Martins, B. O., Lidén, K., & Jumbert, M. G. (2022) · European Security · doi:10.1080/09662839.2022.2101884 · Cited by 58
“Opening up to automated facial recognition technology would import all the problems regarding biases in facial recognition technology (Benjamin 2019, Birhane 2021) to an already challenging system.”
BackgroundBioethics, (Funding) Priorities, and the Perpetuation of Injustice
FABI, R. E., & Goldberg, D. S. (2021) · The American Journal of Bioethics · doi:10.1080/15265161.2020.1867934 · Cited by 47
“A broad and growing body of literature has examined, for instance, the role of machine learning algorithms in perpetuating bias and health inequities (Benjamin 2019; Gianfrancesco et al. 2018).”
BackgroundUncertain and under Quarantine: Toward a Sociology of Medical Ignorance
Whooley, O., & Barker, K. K. (2021) · Journal of Health and Social Behavior · doi:10.1177/00221465211009202 · Cited by 47
“Similar dynamics are evident in racial biases buried within medical algorithms (Benjamin 2019).”
BackgroundCulture and Digital Media in Adolescent Development
Manago, A. M., & McKenzie, J. (2022) · Cambridge University Press eBooks · doi:10.1017/9781108976237.010 · Cited by 57
Beyond the Front Yard: The Dehumanizing Message of Accent-Altering Technology
Payne, A. L., Austin, T., & Clemons, A. M. (2024) · Applied Linguistics · doi:10.1093/applin/amae002 · Cited by 41
Precision Medicine and Its Imprecise History
Phillips, C. J. (2020) · Harvard Data Science Review · doi:10.1162/99608f92.3e85b56a · Cited by 42
How Much Can Potential Jurors Tell Us About Liability for Medical Artificial Intelligence?
Price, W. N., Gerke, S., & Cohen, I. G. (2021) · Journal of Nuclear Medicine · doi:10.2967/jnumed.120.257196 · Cited by 40
“Among other things, AI can replicate biases that exist in the medical system ( 8 ), but in such a way that the tort system cannot identify that bias ( 7,9 ).”
Digital Technology, Politics, and Policy-Making
Gilardi, F. (2022) · Cambridge University Press eBooks · doi:10.1017/9781108887304 · Cited by 53
AI and Blackness: Toward Moving Beyond Bias and Representation
Dancy, C. L., & Saucier, P. K. (2021) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2021.3125998 · Cited by 50
“We might, for example, consider how the supposed agency and freedom in healthcare decisions by Black people is limited by current AI systems used in the automation of assessment [49] that exist within current healthcare systems and that rely on boundaries of the Human, and consider relations to the lack of agency by B…”
BackgroundLandau, A. Y., Ferrarello, S., Blanchard, A. M., Cato, K., Atkins, N., Salazar, S. N., Patton, D. U., & Topaz, M. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab286 · Cited by 40
Cognitive measures lacking in EHR prior to dementia or Alzheimer's disease diagnosis
Maserejian, N. N., Krzywy, H., Eaton, S., & Galvin, J. E. (2021) · Alzheimer s & Dementia · doi:10.1002/alz.12280 · Cited by 42
Epistemically violent biases in artificial intelligence design: the case of DALLE-E 2 and Starry AI
Mbalaka, B. (2023) · Digital Transformation and Society · doi:10.1108/dts-01-2023-0003 · Cited by 49
Kulkarni, V., Gawali, M., & Kharat, A. T. (2021) · JMIR Medical Informatics · doi:10.2196/28776 · Cited by 40
“Coercing technology to solve it can lead to automated systems that tick the right boxes for some arbitrary definition of fairness but eventually end up worsening social inequality and discrimination behind a veneer of technical neutrality [104].”
BackgroundAdministrative Errors and Race: Can Technology Mitigate Inequitable Administrative Outcomes?
Compton, M. E., Young, M., Bullock, J. B., & Greer, R. (2022) · Journal of Public Administration Research and Theory · doi:10.1093/jopart/muac036 · Cited by 49
Mccradden, M., Odusi, O., Joshi, S., Akrout, I., Ndlovu, K., Glocker, B., Maicas, G., Liu, X., Mazwi, M., Garnett, T., et al. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3594096 · Cited by 33
“Moreover, when so-called ‘ethical’ solutions to bias are employed without sufficient knowledge and attention to health equity literature and community engagement, disparities remain and can even worsen [6] [5].”
BackgroundInfluentialSpector‐Bagdady, K., Armoundas, A. A., Arnaout, R., Hall, J. L., McSwain, B. Y., Knowles, J. W., Price, W. N., Rawat, D. B., Riegel, B. J., Wang, T. Y., et al. (2023) · Circulation · doi:10.1161/cir.0000000000001173 · Cited by 37
“30,31 Accessibility is limited by access to the health care and research that generate health information in the first place, recruitment design, and different consent rates across communities.”
BackgroundWarr, M., & Heath, M. K. (2025) · Journal of Teacher Education · doi:10.1177/00224871251325073 · Cited by 35
“Benjamin (2019) termed this discriminatory design.”
InfluentialWho does Australia Lock Up? The Social Determinants of Justice
McCausland, R., & Baldry, E. (2023) · International Journal for Crime Justice and Social Democracy · doi:10.5204/ijcjsd.2504 · Cited by 56
Perrotta, C., & Pangrazio, L. (2023) · Education Policy Analysis Archives · doi:10.14507/epaa.31.7952 · Cited by 55
Artificial intelligence and its ‘slow violence’ to human rights
Teo, S. A. (2024) · AI and Ethics · doi:10.1007/s43681-024-00547-x · Cited by 65
“Thus, an AI system can perpetuate (biased) group-based representations, for example by ‘pre-dicting’ potential suspects for fraud [50], criminality [51, 52] and persistently stereotyping certain groups in society [46 ].”
Don’t stand so close to me: Public spaces, behavioral geography, and COVID-19
James, A. C. (2020) · Dialogues in Human Geography · doi:10.1177/2043820620935672 · Cited by 31
“In studying race and inequality, Benjamin (2016, 2019) notes that actions and choices can be influenced by power structures.”
BackgroundThe Pitfalls of Deep Eutectic Solvents in the Recycling of Lithium‐Ion Batteries
Neguse, S. M., Yoon, S., Lim, H., Jang, J., Baek, S., Jöckel, D. M., Widenmeyer, M., Balke‐Grünewald, B., & Weidenkaff, A. (2024) · Energy Technology · doi:10.1002/ente.202301213 · Cited by 30
Gisselbaek, M., Suppan, M., Minsart, L., Köselerli, E., Myatra, S. N., Matot, I., Chang, O. L. B., Saxena, S., & Berger‐Estilita, J. (2024) · Critical Care · doi:10.1186/s13054-024-05134-4 · Cited by 36
Multi-disciplinary fairness considerations in machine learning for clinical trials
Chien, I., Deliu, N., Turner, R. E., Weller, A., Villar, S. S., & Kilbertus, N. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533154 · Cited by 42
“ML, the path forward is to involve domain experts, affected stakeholders, and to recognize social context, including the potential for institutionalized discrimination, and examine each situation holistically [24, 194, 220].”
BackgroundMachine learning for healthcare that matters: Reorienting from technical novelty to equitable impact
Balagopalan, A., Baldini, I., Celi, L. A., Gichoya, J. W., McCoy, L. G., Naumann, T., Shalit, U., van der Schaar, M., & Wagstaff, K. L. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000474 · Cited by 42
A Systematic Review of ‘Fair’ AI Model Development for Image Classification and Prediction
Correa, R., Shaan, M., Trivedi, H., Patel, B. N., Celi, L. A., Gichoya, J. W., & Banerjee, I. (2022) · Journal of Medical and Biological Engineering · doi:10.1007/s40846-022-00754-z · Cited by 34
“Examples include differing allocation of healthcare resources based on patient demographics [8, 9], bias in language models developed on clinical notes [10], and melanoma detection models developed primarily on images of light-colored skin [11].”
BackgroundConjuring algorithms: Understanding the tech industry as stage magicians
Nagy, P., & Neff, G. (2024) · New Media & Society · doi:10.1177/14614448241251789 · Cited by 26
“…(e.g., Birhane, 2021), violate citizens’ privacy rights (e.g., Gebru et al., 2017), or exacerbate existing racial discriminatory practices (e.g., Benjamin, 2019), the “trick” of conjuring algorithms help shield them from accountability They rarely have to face consequences for the immense harm…”
Kühne, S., Jacobsen, J., Legewie, N. M., & Dollmann, J. (2025) · Journal of Medical Internet Research · doi:10.2196/70179 · Cited by 32
“…existing research has highlighted how systemic biases and inequalities regarding race and gender can shape the design, implementation, and use of AI technologies, with biased data sets, misapplication, and misplaced trust in AI fairness being major factors in this translation process [11-17].”
Toward Argument‐Based Fairness with an Application to AI‐Enhanced Educational Assessments
Huggins‐Manley, A. C., Booth, B. M., & D’Mello, S. K. (2022) · Journal of Educational Measurement · doi:10.1111/jedm.12334 · Cited by 36
Bias, bias everywhere: A response to Li et al. and Zhai and Nehm
Krist, C., & Kubsch, M. (2023) · Journal of Research in Science Teaching · doi:10.1002/tea.21913 · Cited by 26
Community-engaged artificial intelligence research: A scoping review
Loftus, T. J., Balch, J. A., Abbott, K. L., Hu, D., Ruppert, M. M., Shickel, B., Ozrazgat‐Baslanti, T., Efron, P. A., Tighe, P. J., Hogan, W. R., et al. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000561 · Cited by 27
Precision, Equity, and Public Health and Epidemiology Informatics – A Scoping Review
Buckeridge, D. L. (2020) · Yearbook of Medical Informatics · doi:10.1055/s-0040-1701989 · Cited by 29
“For example, using healthcare cost as an output, as opposed to a composite of cost and health status, can reinforce existing racial inequalities in the allocation of healthcare resources [37, 44].”
BackgroundReconciling big data and thick data to advance the new urban science and smart city governance
Hong, A., Baker, L., Prieto-Curiel, R., Duminy, J., Buswala, B., Guan, C., & Ravindranath, D. (2022) · Journal of Urban Affairs · doi:10.1080/07352166.2021.2021085 · Cited by 29
“…perspective there is the potential for uncritical applications of big data technologies, under a guise of technical neutrality, to condition urban governance approaches that reproduce social inequalities and biases (Bannister & O’Sullivan, 2021; Benjamin, 2019; Taylor, 2021; Williams et al., 2020).”
BackgroundThe Doctor and the Algorithm: Promise, Peril, and the Future of Health AI
Graham, S. S. (2022) · doi:10.1093/oso/9780197644461.001.0001 · Cited by 32
Can medical algorithms be fair? Three ethical quandaries and one dilemma
Bærøe, K., Gundersen, T., Henden, E., & Rommetveit, K. (2022) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2021-100445 · Cited by 22
“23 24 Space does not allow us to do justice to the vast literature on algorithm fairness and suggestions to mitigate algorithm biases.”
BackgroundTu, H., Shi, D., Liang, Z., Jiang, H., Kong, Z., Zhang, K., Shao, Y., Wu, Y., & Hao, X. (2021) · Journal of Materials Chemistry A · doi:10.1039/d1ta06379k · Cited by 22
Pfohl, S. R., Zhang, H., Xu, Y., Foryciarz, A., Ghassemi, M., & Shah, N. H. (2022) · Scientific Reports · doi:10.1038/s41598-022-07167-7 · Cited by 43
Pütten, A. M. R.-V. D., & Sach, A. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1416504 · Cited by 35
“…can lead to a failure to appropriately address such biases which may also be related to the supposed objectivity or neutrality of algorithms (Benjamin, 2019): People’s beliefs about the unbiased nature of algorithms may hinder the detection or flagging of biases because people tend to ignore…”
BackgroundThe commercialization of patient data in Canada: ethics, privacy and policy
Spithoff, S. M., Stockdale, J., Rowe, R., McPhail, B. M., & Persaud, N. (2022) · Canadian Medical Association Journal · doi:10.1503/cmaj.210455 · Cited by 21
Belle, A. B., Sutherland, C., Adesina, O., Kpodjedo, S., Ojöng, N., & Cole, L. (2023) · ACM Transactions on Computing Education · doi:10.1145/3617896 · Cited by 20
“In effect, as Ruha Benjamin argues in the context of algorithmic computations of health risk, digital technologies are increasingly automating racism, and this is sometimes a matter of life and death [101].”
BackgroundWang, H., & Blok, V. (2025) · Big Data & Society · doi:10.1177/20539517251340620 · Cited by 39
“…other studies explore how AI mirrors or perpetuates broader socio-political structures, such as capitalism (Zuboff, 2019), colonial-ism (Birhane, 2020; Couldry and Mejias, 2018; Muldoon and Wu, 2023), sexism (Rafanelli, 2022), racism (Benjamin, 2019), and caste systems (Sambasivan et al., 2021).”
Monroe‐White, T. (2021) · ACM SIGCPR Conference on Computer Personnel Research (SIGCPR) · doi:10.1145/3458026.3462161 · Cited by 24
“• Health care[71, 72] algorithms which reduce the number of Black patients identified for additional care by greater than 50% than equally ill white patients.”
BackgroundInfrastructural injustices in community‐driven afterschool STEAM
Shea, M., Jurow, A. S., Schiffer, J., Escudé, M., & Torres, A. (2023) · Journal of Research in Science Teaching · doi:10.1002/tea.21852 · Cited by 22
Social determinants of health in the Big Data mode of population health risk calculation
Rowe, R. (2021) · Big Data & Society · doi:10.1177/20539517211062881 · Cited by 20
“(2019) Risk calculation platforms are here presented as a-political and theory-free measurement tools to determine effective interventions, to differentiate between and curate fi nance for preventative programs.”
BackgroundDecoding Human Behavior with Big Data? Critical, Constructive Input from the Decision Sciences
Katsikopoulos, K. V., & Canellas, M. C. (2022) · AI Magazine · doi:10.1002/aaai.12034 · Cited by 25
Algorithmic agencing in platform markets
Wei, R., & Geiger, S. (2024) · Marketing Theory · doi:10.1177/14705931241275558 · Cited by 24
“While algorithms are susceptible to biases stemming from bugs or coding mistakes (Diakopoulos, 2013), biases present in training and input data for algorithms can also lead to systematic reinforcement of discriminations in their output (Benjamin, 2019).”
Hsu, A., Li, L., Schletz, M., & Yu, Z. (2023) · Environment and Planning B Urban Analytics and City Science · doi:10.1177/23998083231186622 · Cited by 25
“…make decisions, without proper oversight and scrutiny, can perpetuate existing biases present in the data used for training these systems, which can result in discriminatory outcomes that disproportionately impact certain groups or perpetuate systemic inequalities (Eubanks, 2018; Benjamin, 2019).”
BackgroundApplying anti-racist approaches to informatics: a new lens on traditional frames
Platt, J. E., Nong, P., Merid, B., Raj, M., Cope, E. L., Kardia, S. L. R., & Creary, M. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad123 · Cited by 20
Clinical algorithms, racism, and “fairness” in healthcare: A case of bounded justice
El‐Azab, S. A., & Nong, P. (2023) · Big Data & Society · doi:10.1177/20539517231213820 · Cited by 24
“Importantly, these datasets are products of human decision making that re fl ect efforts at aggregation and measurement within social contexts structured by historical medical racism and inequitable health policy (Benjamin, 2019; Cruz, 2022).”
BackgroundConceptualizing Algorithmic Stigmatization
Andalibi, N., Pyle, C., Barta, K., Xian, L., Jacobs, A. Z., & Ackerman, M. S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580970 · Cited by 33
“More broadly, algorithms can perpetuate existing social inequities and create new ones [20, 21, 97, 141].”
BackgroundToward Community-Based Natural Language Processing (CBNLP): Cocreating With Communities
Pillai, M., Griffin, A. C., Kronk, C. A., & McCall, T. (2023) · Journal of Medical Internet Research · doi:10.2196/48498 · Cited by 30
“Continuing to create digital tools without adequate representation of communities that will be affected by their use to identify risks and make decisions on resources can result in the automation of discrimination [33].”
Generative-AI-Generated Challenges for Health Data Research
Spector-Bagdady, K. (2023) · The American Journal of Bioethics · doi:10.1080/15265161.2023.2252311 · Cited by 18
“There have been many concerns raised about the biases and homogeny of databases used by AI which can and have resulted in inequitable results for historically marginalized patients (Benjamin 2019; Kaushal, Altman, and Langlotz 2020; Price 2019).”
An intentional approach to managing bias in general purpose embedding models
Weng, W., Sellergen, A., Kiraly, A. P., D’Amour, A., Park, J., Pilgrim, R., Pfohl, S., Lau, C. T., Natarajan, V., Azizi, S., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00227-3 · Cited by 29
Fiske, A., Blacker, S., Geneviève, L. D., Willem, T., Fritzsche, M.-C., Buyx, A., Celi, L. A., & McLennan, S. (2025) · The Lancet Digital Health · doi:10.1016/j.landig.2025.01.003 · Cited by 26
Ronquillo, C. E., Mitchell, J., Alhuwail, D., Peltonen, L.-M., Topaz, M., & Block, L. J. (2022) · Yearbook of Medical Informatics · doi:10.1055/s-0042-1742504 · Cited by 17
“As AI algorithms rely on historical data, it is also important to note the ways that the historical contexts of institutions and broader societal values shape the diversity of datasets [7].”
BackgroundOrigins of Racial and Ethnic Bias in Pulmonary Technologies
Sjoding, M. W., Ansari, S., & Valley, T. S. (2022) · Annual Review of Medicine · doi:10.1146/annurev-med-043021-024004 · Cited by 18
Navigating Privacy and Security in Telemedicine for Primary Care
Andreadis, K., Muellers, K., Lin, J. J., Mkuu, R. S., Horowitz, C. R., Kaushal, R., & Ancker, J. S. (2024) · The American Journal of Managed Care · doi:10.37765/ajmc.2024.89553 · Cited by 25
Karasaki, S., Morello-Frosch, R. A., & Callaway, D. S. (2024) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2024.175730 · Cited by 21
Reflections: Environmental History in the Era of COVID-19
Alagona, P. S., Carruthers, J., Chen, H., Dagenais, M., Silva, S. D. E., Fitzgerald, G., Hou, S., Jørgensen, D., Leal, C., McNeill, J. R., et al. (2020) · Environmental History · doi:10.1093/envhis/emaa053 · Cited by 23
Detecting discriminatory risk through data annotation based on Bayesian inferences
Beretta, E., Vetrò, A., Lepri, B., & Martin, J. C. D. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445940 · Cited by 27
“Several studies have shown evidence that many equity and discrimination issues are due to input data properties [5].”
Background
Metadata
- Authors
- Benjamin, R.
- Affiliations
- Princeton University (US)
- Corresponding author
- Benjamin, Ruha
- ORCID
- Benjamin, Ruha
- Venue
- Science
- Published
- 2019-10-25
- Volume
- 366
- Issue
- 6464
- Pages
- 421-422
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2981451251
- Semantic Scholar
- 0108b58175e0511476964f59a36701e8ca3a070f
- Cited by
- 378
- References
- 19
- Citations by year
- 2026: 61 · 2025: 43 · 2024: 61 · 2023: 61 · 2022: 86 · 2021: 40 · 2020: 25 · 2019: 1
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Anthropology › Race, Genetics, and Society
- Keywords
- algorithmic bias, racial bias, algorithmic risk assessment, automated decision-making, health disparities, algorithmic fairness
- MeSH terms
- Black or African American, Humans, Racism
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.