The DOI matches its Crossref record; 24 of the 26 references it cites were found in a registry or an index.
The Ugly Truth About Ourselves and Our Robot Creations: The Problem of Bias and Social Inequity
Abstract
Recently, there has been an upsurge of attention focused on bias and its impact on specialized artificial intelligence (AI) applications. Allegations of racism and sexism have permeated the conversation as stories surface about search engines delivering job postings for well-paying technical jobs to men and not women, or providing arrest mugshots when keywords such as "black teenagers" are entered. Learning algorithms are evolving; they are often created from parsing through large datasets of online information while having truth labels bestowed on them by crowd-sourced masses. These specialized AI algorithms have been liberated from the minds of researchers and startups, and released onto the public. Yet intelligent though they may be, these algorithms maintain some of the same biases that permeate society. They find patterns within datasets that reflect implicit biases and, in so doing, emphasize and reinforce these biases as global truth. This paper describes specific examples of how bias has infused itself into current AI and robotic systems, and how it may affect the future design of such systems. More specifically, we draw attention to how bias may affect the functioning of (1) a robot peacekeeper, (2) a self-driving car, and (3) a medical robot. We conclude with an overview of measures that could be taken to mitigate or halt bias from permeating robotic technology.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Human-induced bias is intentionally or unintentionally written by the developers because individuals are always influenced by their own moral perceptions and relevant interests, which affects data training”
From Ethics and governance of trustworthy medical artificial intelligence (2023) · cites this paper as [26]
In the citing paper
“Various high-profle controversies (e.g.[10,22,112]) have raised concerns over the technology's potential to both perpetuate existing societal inequalities and introduce new types of discrimination [52,67,116].”
From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [67]
In the citing paper
“AI applications released to the public are not free from racial and gender biases (Chen, Szolovits & Ghassemi, 2019; Howard & Borenstein, 2018).”
From Digital ageism: challenges and opportunities in artificial intelligence for older adults (2022) · cites this paper as Howard & Borenstein, 2018
Citations
Cites
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
Stereotype Threat and Women's Math Performance
Spencer, S. J., Steele, C. M., & Quinn, D. M. (1999) · Journal of Experimental Social Psychology · doi:10.1006/jesp.1998.1373 · Cited by 3,435
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536
Hall, W. J., Chapman, M. V., Lee, K. M., Merino, Y. M., Thomas, T. W., Payne, B. K., Eng, E., Day, S. H., & Coyne-Beasley, T. (2015) · American Journal of Public Health · doi:10.2105/ajph.2015.302903 · Cited by 2,176
Nwana, H. S. (1996) · The Knowledge Engineering Review · doi:10.1017/s026988890000789x · Cited by 1,588
Green, A. R., Carney, D. R., Pallin, D. J., Ngo, L. H., Raymond, K. L., Iezzoni, L. I., & Banaji, M. R. (2007) · Journal of General Internal Medicine · doi:10.1007/s11606-007-0258-5 · Cited by 1,460
Gender stereotypes about intellectual ability emerge early and influence children's interests
Bian, L., Leslie, S.-J., & Cimpian, A. (2017) · Science · doi:10.1126/science.aah6524 · Cited by 1,206
Handling imbalanced datasets: A review
Kotsiantis, S. B., Kanellopoulos, D. N., & Pintelas, P. (2006) · Cited by 920
Automated Experiments on Ad Privacy Settings: A Tale of Opacity, Choice, and Discrimination
Datta, A., Tschantz, M. C., & Datta, A. (2015) · Proceedings on Privacy Enhancing Technologies · doi:10.1515/popets-2015-0007 · Cited by 676
On the epistemic costs of implicit bias
Gendler, T. S. (2011) · Philosophical Studies · doi:10.1007/s11098-011-9801-7 · Cited by 517
Unequal Representation and Gender Stereotypes in Image Search Results for Occupations
Kay, M., Matuszek, C., & Munson, S. A. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702520 · Cited by 509
The evolution of error: error management, cognitive constraints, and adaptive decision-making biases
Johnson, D. D. P., Blumstein, D. T., Fowler, J. H., & Haselton, M. G. (2013) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2013.05.014 · Cited by 364
Responsibility for Implicit Bias
Holroyd, J. (2012) · Journal of Social Philosophy · doi:10.1111/j.1467-9833.2012.01565.x · Cited by 238
Artificial Intelligence and Life in 2030
Stone, P. (2024) · IEEE RESOURCE CENTERS · doi:10.17023/ta87-9545 · Cited by 202
An overview of approaches to qualitative model construction
Schut, C., & Bredeweg, B. (1996) · The Knowledge Engineering Review · doi:10.1017/s0269888900007657 · Cited by 147
Salary Differences Between Male and Female Registered Nurses in the United States
Muench, U., Sindelar, J. L., Busch, S. H., & Buerhaus, P. I. (2015) · JAMA · doi:10.1001/jama.2015.1487 · Cited by 104
Šabanović, S., Chang, W.-L., Bennett, C. C., Piatt, J. A., & Hakken, D. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-20892-3_11 · Cited by 100
Science mapping analysis characterizes 235 biases in biomedical research
Chavalarias, D., & Ioannidis, J. P. A. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.12.011 · Cited by 97
McHugh, C., & Way, J. (2016) · Philosophy and Phenomenological Research · doi:10.1111/phpr.12299 · Cited by 59
Rodger, J. A., & Pendharkar, P. C. (2003) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2003.09.005 · Cited by 56
Implicit Bias and Philosophy, Volume 1
Brownstein, M., & Saul, J. 1. (2016) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198713241.001.0001 · Cited by 48
Self-Driving Cars: Ethical Responsibilities of Design Engineers
Borenstein, J., Herkert, J. R., & Miller, K. (2017) · IEEE Technology and Society Magazine · doi:10.1109/mts.2017.2696600 · Cited by 47
Self-citationBorenstein, J., Howard, A., & Wagner, A. R. (2017) · Oxford University Press eBooks · doi:10.1093/oso/9780190652951.003.0009 · Cited by 12
Self-citation
Research works printed but not linked: 4
- Bolukbasi, T., Chang K-W, Zou, J., Saligrama, V., & Kalai, A. (2016). Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. https://arxiv.org/pdf/1607.06520.pdf . Accessed July 3, 2017. · arXiv
- Harpur, P. (2013). From universal exclusion to universal equality: Regulating ableism in a digital age. Northern Kentucky Law Review, 40(3), 529–565.
- IEEE. (2017). Ethically aligned design: A vision for prioritizing human wellbeing with artificial intelligence and autonomous systems. http://standards.ieee.org/develop/indconn/ec/ead_brochure.pdf . Accessed February 7, 2017.
- Steele, C. M. (2010). Whistling Vivaldi: And other clues to how stereotypes affect us (issues of our time). New York, NY: W. W. Norton and Co.
Cited by
Sorted by citations
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812
Emerging challenges in AI and the need for AI ethics education
Borenstein, J., & Howard, A. (2021) · AI and Ethics · doi:10.1007/s43681-020-00002-7 · Cited by 626
Self-citationTrustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612
“, underprivileged groups might experience systematic disadvantage in hiring decisions [49], or be disproportionally impacted in criminal risk profiling [104, 146, 161]).”
BackgroundParadoxes of artificial intelligence in consumer markets: Ethical challenges and opportunities
Du, S., & Xie, C. (2020) · Journal of Business Research · doi:10.1016/j.jbusres.2020.08.024 · Cited by 665
Ethics and governance of trustworthy medical artificial intelligence
Zhang, J., & Zhang, Z.-M. (2023) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-023-02103-9 · Cited by 507
“Human-induced bias is intentionally or unintentionally written by the developers because individuals are always influenced by their own moral perceptions and relevant interests, which affects data training [25, 26].”
Background
Metadata
- Authors
- Howard, A., & Borenstein, J.
- Affiliations
- Georgia Institute of Technology (US)
- Corresponding author
- Borenstein, Jason
- Venue
- Science and Engineering Ethics
- Published
- 2017-09-21
- Volume
- 24
- Issue
- 5
- Pages
- 1521-1536
- ISSN
- 1353-3452 (ISSN-L)
- 1471-5546
- OpenAlex
- W2760062370
- Cited by
- 337
- References
- 26
- Citations by year
- 2026: 29 · 2025: 64 · 2024: 53 · 2023: 59 · 2022: 38 · 2021: 49 · 2020: 27 · 2019: 13 · 2018: 4 · 2017: 1
- FWCI
- 28.19
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Science and Engineering Ethics (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Artificial Intelligence in Law
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- algorithmic bias, artificial intelligence, social inequities, implicit bias, racial bias, gender bias, autonomous vehicles, medical robots, bias mitigation, robot ethics
- MeSH terms
- Algorithms, Artificial Intelligence, Automobiles, Female, Humans, Learning, Male, Prejudice, Robotics, Social Justice, Bias, Biomedical Technology, Racism, Sexism, Datasets as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
