The Paradox of Automation as Anti-Bias Intervention
Open accessAbstract
A received wisdom is that automated decision-making serves as an anti-bias intervention. The conceit is that removing humans from the decision-making process will also eliminate human bias. The paradox, however, is that in some instances, automated decision-making has served to replicate and amplify bias. With a case study of the algorithmic capture of hiring as heuristic device, this Article provides a taxonomy of problematic features associated with algorithmic decision-making as anti-bias intervention and argues that those features are at odds with the fundamental principle of equal opportunity in employment. To examine these problematic features within the context of algorithmic hiring and to explore potential legal approaches to rectifying them, the Article brings together two streams of legal scholarship: law and technology studies and employment & labor law. Counterintuitively, the Article contends that the framing of algorithmic bias as a technical problem is misguided. Rather, the Article’s central claim is that bias is introduced in the hiring process, in large part, due to an American legal tradition of deference to employers, especially allowing for such nebulous hiring criterion as “cultural fit.” The Article observes the lack of legal frameworks that take into account the emerging technological capabilities of hiring tools which make it difficult to detect disparate impact. The Article thus argues for a re-thinking of legal frameworks that take into account both the liability of employers and those of the makers of algorithmic hiring systems who, as brokers, owe a fiduciary duty of care. Particularly related to Title VII, the Article proposes that in legal reasoning corollary to extant tort doctrines, an employer’s failure to audit and correct its automated hiring platforms for disparate impact could serve as prima facie evidence of discriminatory intent, leading to the development of the doctrine of discrimination per se. The article also considers other approaches separate from employment law such as establishing consumer legal protections for job applicants that would mandate their access to the dossier of information consulted by automated hiring systems in making the employment decision.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Hiring algorithms predict employment decisions or supervisory ratings, which are affected by race and gender biases.
“Hiring algorithms predict employment decisions or supervisory ratings, which are affected by race and gender biases ( 43 ).”
From Dissecting racial bias in an algorithm used to manage the health of populations (2019) · cites this paper as [43]
The Introduction describes an automated hiring system that disadvantaged women, and “A Legal Tradition of Employer Deference” warns that hiring systems can use proxies for race and gender. But the copy does not say that hiring algorithms predict employment decisions or supervisory ratings; “2. Hiring” instead describes predicting whether a candidate will accept an offer. No page numbers were provided for these passages. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 1
- 184 Miranda Katz, Amazon's Turker Crowd Has Had Enough , WIRED (Aug. 23, 2017, 6:55 AM), https://www.wired.com/story/amazons-turker-crowd-has-had-enough [https://perma.cc/RJ6EK7B7].
Cited by
Sorted by citations
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · see citing works
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
Algorithmic bias: review, synthesis, and future research directions
Kordzadeh, N., & Ghasemaghaei, M. (2022) · European Journal of Information Systems · doi:10.1080/0960085x.2021.1927212 · Cited by 770 · see citing works
“Another approach is to require companies to make their algorithm design and outcomes transparent, auditable to third parties, and explainable to individuals impacted by algorithmic decisions (Ajunwa, 2020; Domanski, 2019; Rossi, 2019).”
Towards a standard for identifying and managing bias in artificial intelligence
Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., & Hall, P. (2022) · doi:10.6028/nist.sp.1270 · Cited by 548 · see citing works
Wang, R., Harper, F. M., & Zhu, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376813 · Cited by 211 · see citing works
Algorithmic Management: Bright and Dark Sides, Practical Implications, and Research Opportunities
Benlian, A., Wiener, M., Cram, W. A., Krasnova, H., Maedche, A., Möhlmann, M., Recker, J., & Remus, U. (2022) · Business & Information Systems Engineering · doi:10.1007/s12599-022-00764-w · Cited by 180 · see citing works
“While Uber provides the current go-to use case for algorithmic management, we have already seen algorithms carry out many managerial functions such as analytics (Gal et al. 2020), hiring, firing, and promotion (Ajunwa 2020), or coaching and giving performance feedback (Luo et al. 2021).”
AI auditing: The Broken Bus on the Road to AI Accountability
Birhane, A., Steed, R., Ojewale, V., Vecchione, B., & Raji, I. D. (2024) · 2024 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml59370.2024.00037 · Cited by 96 · see citing works
Žliobaitė, I., & Custers, B. (2016) · Artificial Intelligence and Law · doi:10.1007/s10506-016-9182-5 · Cited by 125 · see citing works
The Impossibility of Automating Ambiguity
Birhane, A. (2021) · Artificial Life · doi:10.1162/artl_a_00336 · Cited by 121 · see citing works
Fairness and Bias in Algorithmic Hiring: A Multidisciplinary Survey
Fabris, A., Baranowska, N., Dennis, M. J., Graus, D., Hacker, P., Saldivar, J., Zuiderveen Borgesius, F., & Biega, A. J. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3696457 · Cited by 117 · see citing works
FairSquare: probabilistic verification of program fairness
Albarghouthi, A., D'Antoni, L., Drews, S., & Nori, A. V. (2017) · Proceedings of the ACM on Programming Languages · doi:10.1145/3133904 · Cited by 94 · see citing works
Auditing Work: Exploring the New York City algorithmic bias audit regime
Groves, L., Metcalf, J., Kennedy, A., Vecchione, B., & Strait, A. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658959 · Cited by 59 · see citing works
Vereschak, O., Alizadeh, F., Bailly, G., & Caramiaux, B. (2024) · International Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642018 · Cited by 58 · see citing works
FARE: Diagnostics for Fair Ranking using Pairwise Error Metrics
Kuhlman, C., VanValkenburg, M., & Rundensteiner, E. A. (2019) · The Web Conference · doi:10.1145/3308558.3313443 · Cited by 57 · see citing works
Operationalizing the Search for Less Discriminatory Alternatives in Fair Lending
Gillis, T. B., Meursault, V., & Ustun, B. (2024) · Conference on Fairness, Accountability and Transparency · doi:10.1145/3630106.3658912 · Cited by 30 · see citing works
The Unrealized Potential of Technology in Selection Assessment
Ryan, A. M., & Derous, E. (2019) · Journal of Work and Organizational Psychology · doi:10.5093/jwop2019a10 · Cited by 39 · see citing works
“…biases, the need to consider how technological advances may tap into those biases needs to be at the forefront of assessment design considerations, rather than assuming that the computer is a better judge (e.g., by training the algorithm to make nondiscriminatory decisions; Ajunwa, 2016).”
BackgroundBeyond Behaviorist Representational Harms: A Plan for Measurement and Mitigation
Chien, J., & Danks, D. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658946 · Cited by 22 · see citing works
Feature-based Learning for Diverse and Privacy-Preserving Counterfactual Explanations
Vo, V., Le, T., Nguyen, V., Zhao, H., Bonilla, E. V., Haffari, G., & Phung, D. Q. (2022) · Knowledge Discovery and Data Mining · doi:10.1145/3580305.3599343 · Cited by 22 · see citing works
Ghosh, B., Basu, D., & Meel, K. S. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3593983 · Cited by 8 · see citing works
Navigating Automated Hiring: Perceptions, Strategy Use, and Outcomes Among Young Job Seekers
Armstrong, L., & Metaxa, D. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711038 · Cited by 10 · see citing works
Encouraging self-blinding in hiring
Fath, S. B., Larrick, R. P., & Soll, J. B. (2023) · Behavioral Science & Policy · doi:10.1177/23794607231192721 · Cited by 17 · see citing works
Salminen, J. O., Jung, S., Chowdhury, S. A., & Jansen, B. J. J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-50334-5_6 · Cited by 7 · see citing works
The Societal Impacts of Algorithmic Decision-Making
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195 · Cited by 4 · see citing works
Hinov, N. L., & Ivanova, M. (2026) · AI · doi:10.3390/ai7030102 · Cited by 6 · see citing works
Hossain, M. P. (2024) · Journal of Cyber Policy · doi:10.1080/23738871.2024.2337129 · Cited by 7 · see citing works
Discrimination Exposed? On the Reliability of Explanations for Discrimination Detection
Skirzyński, J., Danks, D. J., & Ustun, B. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732167 · Cited by 1 · see citing works
Towards User Guided Actionable Recourse
Yetukuri, J., Hardy, I., & Liu, Y. (2023) · AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604708 · Cited by 5 · see citing works
Nodipha, O. G., & Maphala, B. P. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1833850 · Cited by 4 · see citing works
“Yet they also raise serious questions about fairness, transparency, employee autonomy, and the possible reproduction of existing inequalities (Ajunwa, 2020; Jarrahi and Sutherland, 2019; Rana and Awan, 2024).”
Salo-Pöntinen, H., & Saariluoma, P. O. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2551049 · Cited by 2 · see citing works
“…of the ability to make inferences on results of single appraisal criteria underlying the experience of valence by observing people ’ s facial micro-expressions (Aue et al., 2007; Aue & Scherer, 2008, 2011; Gentsch et al., 2015, 2020; Scherer et al., 2017; Scherer & Moors, 2019; Wu et al., 2012).”
Catch Me If You Can: The Dynamic Nature of Bias in Machine Learning Applications
Tarafdar, M., Rets, I., & Hu, Y. (2026) · Information Systems Journal · doi:10.1111/isj.70034 · Cited by 1 · see citing works
“Bias is considered prejudicial (Mehrabi et al. 2021; Van Giffen et al. 2022) when such disparities are based on individuals' ACs (Ajunwa 2019; Schwartz et al. 2022; Teodorescu et al. 2021) and when it involves differential treatment of, or impact on, an already-disadvantaged group based on ACs…”
InfluentialARTIFICIAL INTELLIGENCE AND GENDER EQUALITY: A SYSTEMATIC MAPPING STUDY
Olsson, E. (2022) · Proceedings of the 15th International Conference on ICT, Society and Human Beings (ICT 2022), the 19th International Conference Web Based Communities and Social Media (WBCSM 2022) and 14th International Conference on e-Health (EH 2022) · doi:10.33965/ict_wbc_eh2022_202204l015 · Cited by 2
AI Presents Both Problems and Opportunities for Minorities
Triana, M. D. C., & Upadhyay, A. (2026) · Journal of Management Studies · doi:10.1111/joms.70145
Vijaya Kumar, J. (2026) · Journal of Intelligent Decision Making and Information Science · doi:10.59543/jidmis.v3.2133 · Cited by 1 · see citing works
“Although AI is viewed as something that can reduce biases in humans, it can also introduce novel types of biases or conceal biases in seemingly objective systems [8].”
van Zyl, L. E. (2026) · The Journal of Positive Psychology · doi:10.1080/17439760.2026.2701096 · Cited by 2 · see citing works
The invisible layer: ethics, accountability, and algorithmic workforce governance
Castillo, L. D. (2026) · AI and Ethics · doi:10.1007/s43681-026-01130-2
“Ajunwa [37] analyzed the right of workers to know how algorithmic management systems operate, framing this not as a transparency courtesy but as a precondition for the exercise of workplace agency.”
Data Donation: A First Step Towards Improving Representativeness in Algorithmic Hiring Datasets
Saldivar, J., Wulf, J., & Castillo, C. (2026) · Social Science Computer Review · doi:10.1177/08944393261441840
The algorithmic conscience: reconciling AI efficiency with human dignity
Deniz, C., & Stiefenhofer, P. (2026) · Journal of Chinese Economic and Business Studies · doi:10.1080/14765284.2026.2640533 · Cited by 1 · see citing works
Mufti, H., Utama, J. P., & Zaman, M. A. N. (2026) · Journal of Artificial Intelligence and Engineering Applications (JAIEA) · doi:10.59934/jaiea.v5i2.2062 · Cited by 3 · see citing works
“However, as Ajunwa [1] explains, despite its various advantages, the use of AI in recruitment also presents a number of challenges and concerns.”
Machine Learning Models in Parole Decision-Making: A Pilot Study Comparing AI and Human Judgment
Farber, S. (2026) · Criminal Justice Policy Review · doi:10.1177/08874034251408304 · Cited by 1 · see citing works
“Paradoxically, while automation is often proposed as a solution to human bias, the technical limitations of AI models may perpetuate or even exacerbate existing discriminatory patterns (Ajunwa, 2020).”
La inteligencia artificial desde la mirada de los estudiantes universitarios
Rodríguez-Torres, Á.-F., & Boillos-Pereira, M. M. (2025) · European Public & Social Innovation Review · doi:10.31637/epsir-2026-1895 · Cited by 1 · see citing works
Individual and Group-level considerations of Actionable Recourse
Yetukuri, J. (2023) · AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604758 · Cited by 1
Díaz-Rodríguez, N., Binkytė-Sadauskienė, R., Bakkali, W., Bookseller, S., Tubaro, P., Bacevičius, A., & Chatila, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.14492 · Cited by 1 · see citing works
“Similarly, the Amazon hiring case shows that training employment prediction algorithms on historical data, where women in technology have been a minority, will not make fair decisions towards female candidates in the future [3].”
Algorithmic Management and a New Generation of Rights at Work
Atkinson, J., & Collins, P. (2024) · Social Science Research Network · doi:10.2139/ssrn.4853536 · Cited by 1
Algorithmic monoculture and social welfare
(2023) · The Societal Impacts of Algorithmic Decision-Making · doi:10.1145/3603195.3603202
From regulation to governance: The EU AI Act and HR governance along the employee lifecycle
Gegg, A., & Büttgen, M. (2026) · German Journal of Human Resource Management Zeitschrift für Personalforschung · doi:10.1177/23970022261488477
Demidont, A. C. (2026) · PLOS Digital Health · doi:10.64898/2026.02.22.26346836
“…empirical validation—that fairness interventions limited to the algorithmic layer (bias mitigation, audit requirements) would be expected to have limited population-level impact without simultaneous attention to data integration practices and institutional access to recourse mechanisms [12 ,13].”
Fear, Blame, and Deservingness: The Moral Economy of AI-Labor Anxiety on Reddit
Iordache, A., Rughiniş, C., Rughinis, R. V., & Țurcanu, D. (2026) · Social Sciences · doi:10.3390/socsci15100663
Azeez, G. K., & Rana, V. (2026) · AI & Society · doi:10.1007/s00146-026-03168-6
“…the use of AI for workplace surveillance and performance monitoring is eroding worker sovereignty, autonomy, dignity, and bargaining power, creating a new form of algorithmic taylorism that treats human body and labor-power as an instrument and mechanism to be pushed to extreme (Ajunwa 2020).”
Counterfactual Bias Testing for Application Tracking System
Yashwant, S., Bansal, S., Dubey, A., Chatterjee, S., Kumar, S., Gupta, S., & Thulsiram, G. (2026) · doi:10.48550/arxiv.2608.26899
“…and more recent audits of commercial hiring and screening tools have repeatedly found measurable disparate treatment along gender, race, age, and disability lines [Dastin, 2018, Bogen and Rieke, 2018, Raghavan et al., 2020, Ajunwa, 2020, Sánchez-Monedero et al., 2020, Wilson and Caliskan, 2024].”
Hukkeri, P., Pol, S., & Kuratti, S. S. (2026) · International Journal of Innovative Science and Research Technology · doi:10.38124/ijisrt/26aug500
“AI models learn from historical datasets, and if past hiring decisions favoured certain groups, AI may inadvertently reinforce those biases (Ajunwa, 2020).”
Algorithmic Bias in AI-Based Recruitment Systems: Implications for Fair Hiring Practices
Rakshitha, M. M. (2026) · International Journal of Economics and Business Management · doi:10.64823/ijebm.2601008
International Legal Mechanisms for Prohibiting Discrimination in the Field of Labor
Djambakieva, F. (2026) · Uzbek journal of law and digital policy. · doi:10.59022/ujldp.610
“More recently, studies have drawn attention to emerging forms of discrimination facilitated by digital technologies, including algorithmic recruitment tools that may perpetuate historical biases embedded in training data (Eubanks, 2018; Ajunwa, 2019).”
Ethical Leadership in the Age of Artificial Intelligence: A Systematic Literature Review
Hossain, M. (2026) · Digital Social Sciences · doi:10.69971/dss.2.2.2025.45
“In performance management, Ajunwa (2020) and Duggan et al. (2020) report the development of ‘algorithmic management’ (the monitoring, evaluation and disciplining of employees with the help of algorithms based on machine learning), which they describe as a type of ‘surveillance capitalism’ that…”
Moura, I., Powell, R. M., & Kaye, H. (2026) · Journal of Public Child Welfare · doi:10.1080/15548732.2026.2689958
AI Assistance for Human Review of Default Judgments
Worledge, T. T., Koraichi, O. B., Bernal, D., Caspi, A., Hashimoto, T. B., Guestrin, C., & Engstrom, D. (2026) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.48550/arxiv.2607.01256
“Second, prior works highlight the risk of allocational disparities arising from deployed AI that amplify historical bias, along identity-based attributes (Ajunwa 2019; Angwin et al. 2022).”
Value creation in online labor market projects: an information processing and learning framework
Alshalfan, A., Baveja, A., & Qi, L. (2026) · Cogent Business & Management · doi:10.1080/23311975.2026.2673284
Sharma, G. S., & Molamohammadi, M. (2026) · arXiv.org · doi:10.48550/arxiv.2604.22679
“Evidence shows these systems harm job seekers, particularly marginalized groups [13], and whether they reduce bias remains unclear [4, 14].”
Legal Challenges of AI And Automation in the Workplace
Alkhalifa, A. (2025) · Access to Justice in Eastern Europe · doi:10.33327/ajee-18-8.2-a000104
Harnessing SERM: Deep Neural Networks for Strategic Employee Attrition Prediction and Management
K, K., & Kumar, E. S. (2024) · 2024 International Conference on Communication, Computing and Energy Efficient Technologies (I3CEET) · doi:10.1109/i3ceet61722.2024.10994054
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603213
Discrimination in Modern Labor Markets: A Field Experiment
Nam, D. (2021) · Social Science Research Network · doi:10.2139/ssrn.3880688
- Authors
- Ajunwa, I.
- Affiliations
- University of North Carolina at Chapel Hill (US)
- Corresponding author
- Ajunwa, Ifeoma
- Venue
- SSRN Electronic Journal
- Publisher
- Elsevier BV
- Published
- 2016
- ISSN
- 1556-5068 (ISSN-L)
- OpenAlex
- W7061064965
- Cited by
- 122
- References
- 0
- Citations by year
- 2026: 8 · 2025: 18 · 2024: 14 · 2023: 41 · 2022: 10 · 2021: 12 · 2020: 10 · 2019: 7 · 2017: 1 · 2016: 1
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Locations
- SSRN Electronic Journal (acceptedVersion)
- University of North Carolina School of Law Scholarship Repository (University of North Carolina Hospitals) (submittedVersion)
- bepress Legal Repository (submittedVersion)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Law › Discrimination and Equality Law
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › AI and HR Technologies
- Keywords
- algorithmic bias, automated decision-making, automated hiring, equal employment opportunity, indirect discrimination, cultural fit, algorithmic accountability, Title VII, discriminatory intent, prima facie evidence, law and technology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
