Building and Auditing Fair Algorithms: A Case Study in Candidate Screening
The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Abstract
Academics, activists, and regulators are increasingly urging companies to develop and deploy sociotechnical systems that are fair and unbiased. Achieving this goal, however, is complex: the developer must (1) deeply engage with social and legal facets of "fairness" in a given context, (2) develop software that concretizes these values, and (3) undergo an independent algorithm audit to ensure technical correctness and social accountability of their algorithms. To date, there are few examples of companies that have transparently undertaken all three steps.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“First-party audits are now common at large tech companies, many of which have established auditing teams within their engineering departments.”
From Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem (2022) · cites this paper as [65]
In the citing paper
“For instance, Google researchers on the internal AI team were dismissed and blocked from publishing critiques on the large-scale language models [18,27], which would later be revealed to be critical to the company's future product roadmap [23,85,87].”
From Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance (2022) · cites this paper as [87]
In the citing paper
“Transparency requires the disclosure of the information on a system, and has long been a recognized requirement in software engineering [207, 88].”
From Trustworthy AI: From Principles to Practices (2023) · cites this paper as [88]
In the citing paper
“Wilson et al. (2021) described a candidate-screening system that incorporated fairness evaluation and external review as part of product development.”
From The Impact of Structured Validation and Audit Frameworks on the Fairness and Efficiency of AI-Driven Hiring Systems (2025) · cites this paper as Wilson et al., 2021
In the citing paper
“By assessing the capabilities and limitations of AI systems prior to deployment, auditing helps technology providers identify and mitigate risks before harm occurs (Floridi & Strait, 2020;Wilson et al., 2021).”
From Auditing of AI: Legal, Ethical and Technical Approaches (2023) · cites this paper as Wilson et al., 2021
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Cortes, C., & Vapnik, V. (1995) · Machine-mediated learning · doi:10.1023/a:1022627411411 · Cited by 48,445 · cited 1×
“Each feature distribution is re-centered around zero and scaled to unit variance. pymetrics then uses a proprietary implementation of a Support Vector Machine (SVM) algorithm [12] to train predictive models.”
MethodA Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · cited 1×
“Several tools have been developed by academics to facilitate auditing of black-box ML models, including LIME [52] and SHAP [40].”
Method"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
International Organization for Standardization
(2011) · SpringerReference · doi:10.1007/springerreference_16847 · Cited by 12,534
International Organization for Standardization
(2015) · Encyclopedia of Astrobiology · doi:10.1007/978-3-662-44185-5_100574 · Cited by 6,716
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · cited 1×
“These voices have grown louder and the concerns more acute as data-driven ML systems have seen increased adoption [5].”
BackgroundSemantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887 · cited 1×
“This includes examining broad classes of systems like search [14, 14, 25, 28, 32, 35, 36, 41, 53–55], e-commerce [10, 26], news recommendation [4, 31], online advertising [58, 60], maps [57], ridesharing [9, 30], online reviews and ratings [17, 18], natural language processing [7], and recommendation [29].”
BackgroundRaji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245 · cited 5×
“…algorithm auditing to reverse engineering by outsiders, in that the goal is to make “black-box” systems more transparent regardless of the system creator’s intent [13], this conceptualization has since been expanded to include audits carried out by ethically and morally-conscious insiders [51].”
BackgroundInfluentialCertifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827 · cited 1×
“While pre-training [19] and during-training model de-biasing methods [63] exist, they require that training data include complete demographic information, which is not always available in employment contexts.”
Background
Research works printed but not linked: 10
- Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Proc. of FAT*.
- Nicholas Diakopoulos Daniel Trielli Jennifer Stark and Sean Mussenden. 2018. I Vote For---How Search Informs Our Choice of Candidate. In Digital Dominance: The Power of Google Amazon Facebook and Apple M. Moore and D. Tambini (Eds.). 22.
- Danielle Ensign, Sorelle A. Friedler, Scott Neville, Carlos Scheidegger, and Suresh Venkatasubramanian. 2018. Runaway Feedback Loops in Predictive Policing. In Proc. of FAT*.
- Pauline T. Kim. 2017. Data-Driven Discrimination at Work. William & Mary Law Review 58 (2017).
- pymetrics, inc. 2019. [Confidential] Fairness Testing Procedures.
- pymetrics, inc. 2019. [Confidential] Games, Measures and Factors: Measurement Validity.
- pymetrics, inc. 2019. [Confidential] Job Analysis Methods & Process.
- pymetrics, inc. 2019. [Confidential] Technical Brief for pymetrics, inc.
- pymetrics, inc. 2020. [Confidential] Demographic Disclosure Study.
- Christian Sandvig, Kevin Hamilton, Karrie Karahalios, and Cedric Langbort. 2014. Auditing Algorithms: Research Methods for Detecting Discrimination on Internet Platforms. In Proc. of Data and Discrimination: Converting Critical Concerns into Productive Inquiry, a preconference at the Annual Meeting of the International Communication Association.
Cited by
Items 101–200 of 240
System Cards for AI-Based Decision-Making for Public Policy
Gursoy, F., & Kakadiaris, I. A. (2022) · arXiv · doi:10.48550/arxiv.2203.04754 · Cited by 7
“Few studies [29], [30] discuss actual applications of audits on algorithmic products.”
BackgroundHuman Resource Management and AI: Building A Collaborative Contextual Transparency Database
Simpson, E., Ermovick, R., & Sloane, M. (2026) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3846167.3848594 · Cited by 7
Measuring Validity in LLM-based Resume Screening
Castleman, J., Shen, Z., Metevier, B., Springer, M., & Korolova, A. (2026) · Proceedings of IASEAI Conference · doi:10.1609/iaseai.v2i1.43017 · Cited by 8
Fourth Workshop on Recommender Systems for Human Resources (RecSys in HR 2024)
Bogers, T., Graus, D., Kaya, M., Johnson, C., Decorte, J.-J., & Bie, T. D. (2024) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3640457.3687109 · Cited by 7
“Their applications range from assisting the talent acquisition process through matching [13], analyzing resumes or other user representations for candidate screening [25] and automated assessment [17, 19], to broader tasks such as recommendations for upskilling [24].”
Chen, S., Xie, J., Wang, G., Hai-zhou, W., Cheng, H., & Huang, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713341 · Cited by 7
To Impress an Algorithm: Minoritized Applicants’ Perceptions of Fairness in AI Hiring Systems
Girona, A. E., & Yarger, L. K. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-57860-1_4 · Cited by 7
Prost, F., Packer, B., Chen, J., Li, W., Kremp, P., Blumm, N., Wang, S., Doshi, T. S., Osadebe, T., Heldt, L., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.07755 · Cited by 7
Global Approaches to Auditing Artificial Intelligence: A Literature Review
(IPIE), I. P. O. T. I. E., Sylvester, P., Chun, W. H. K., Hannák, A., Mendoza, M., Broussard, M., Сергеевич, Е. Р., Nelson, A., Sandvig, C., Sesan, G., et al. (2024) · doi:10.61452/bwym7397 · Cited by 7
Self-citationOttun, A.-R. O., & Flores, H. (2025) · doi:10.36227/techrxiv.176118749.93102582/v1 · Cited by 6
Active Fourier Auditor for Estimating Distributional Properties of ML Models
Ajarra, A., Ghosh, B., & Basu, D. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i15.33682 · Cited by 6
Practical Perspectives on Fairness In Technology-Aided Personnel Selection
Müller, I., Dill, K., & Mihaljević, H. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757647 · Cited by 6
Fabris, A., Rus, C., Saldivar, J., Gatzioura, A., Biega, A. J., & Castillo, C. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104506 · Cited by 7
Mining the online infosphere: A survey
Adak, S., Chakraborty, S., Das, P., Das, M., Dash, A., Hazra, R., Mathew, B., Saha, P., Sarkar, S., & Mukherjee, A. (2022) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1453 · Cited by 6
González-Sendino, R., Serrano, E., & Bajo, J. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109979 · Cited by 6
Fintech, Racial Equity, and an Inclusive Financial System
Development, F. R. B. O. S. F. C. (2021) · doi:10.24148/cdir2021-02 · Cited by 6
Towards A Global AI Auditing Framework: Assessment and Recommendations
(IPIE), I. P. O. T. I. E., Faveri, B., Johnson-León, M., Sylvester, P., Chun, W. H. K., Hannák, A., Mendoza, M., Broussard, M., Сергеевич, Е. Р., Nelson, A., et al. (2025) · doi:10.61452/zwed1485 · Cited by 6
Self-citationReduced AI Acceptance After the Generative AI Boom: Evidence From a Two-Wave Survey StudyPreprint
Baumann, J., Urman, A., Leicht‐Deobald, U., Roman, Z. J., Hannák, A., & Christen, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.23578 · Cited by 6
Quantifying Query Fairness Under Unawareness
Jaenich, T., Moreo, A., Fabris, A., McDonald, G., Esuli, A., Ounis, I., & Sebastiani, F. (2026) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.17675 · Cited by 5
Second Workshop on Recommender Systems for Human Resources (RecSys in HR 2022)
Bogers, T., Graus, D., Kaya, M., Gutiérrez, F., Mesbah, S., & Johnson, C. (2022) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3523227.3547414 · Cited by 5
Economies of Virtue – The Circulation of 'Ethics' in AI
Hristova, T., & Magee, L. (2022) · ePrints Soton (University of Southampton) · doi:10.25969/mediarep/19267 · Cited by 5
Graus, D., Bogers, T., Kaya, M., Gutiérrez, F., & Verbert, K. (2021) · ACM SIGIR Forum · doi:10.1145/3527546.3527567 · Cited by 5
“…the talent acquisition process through matching [Kentha-padi et al., 2017], analyzing resumes or other user representations for candidate screening [Wilson et al., 2021] and automated assessment [Naim et al., 2018; Liem et al., 2018], to broader tasks such as recommendations for upskilling…”
BackgroundAuditing the Imputation Effect on Fairness of Predictive Analytics in Higher EducationPreprint
Anahideh, H., Haghighat, P., Nezami, N., & G`andara, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.07908 · Cited by 5
Concrete Safety for ML Problems: System Safety for ML Development and AssessmentPreprint
Jatho, E. W., Mailloux, L. O., Williams, E. D., McClure, P., & Kroll, J. A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.02972 · Cited by 5
The Societal Impacts of Algorithmic Decision-Making
Raghavan, M. (2021) · eCommons (Cornell University) · doi:10.7298/eewn-0e35 · Cited by 5
Karan, A., Balepur, N., & Sundaram, H. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594038 · Cited by 4
“Some are done in conjunction with system providers [65], which can lead to charges of conflict of interest [69].”
BackgroundPerkowski, P., & Scofield, C. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4367113 · Cited by 4
Bayesian Orchestration of Multi-LLM Agents for Cost-Aware Sequential Decision-MakingPreprint
Amin, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.01522 · Cited by 4
Evaluating an AI Tutor for Bias Across Different Foundation Models
Vinodh, A., Harvey, E., Almoubayyed, H., Yu, R., Brooks, C. A., Koenecke, A., & Kizilcec, R. F. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-98465-5_43 · Cited by 3
Inherent Tradeoffs in the Fair Determination of Risk Scores
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603206 · Cited by 3
Sakib, M. N., Rayasam, N. M., & Dey, S. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3816931 · Cited by 3
Discrimination DRIFT: A Practical Framework for Responsible AI
Wyer, S. (2026) · AI and Ethics · doi:10.1007/s43681-026-01211-2 · Cited by 3
“Case studies, such as pymetrics’ candidate screening tool, demonstrate the practical application of fairness principles and independent auditing in AI systems in the real world [156 ].”
RecSys in HR: Workshop on Recommender Systems for Human Resources
Bogers, T., Graus, D., Kaya, M., Gutiérrez, F., & Verbert, K. (2021) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3460231.3470935 · Cited by 3
“Their applications range from assisting the talent acquisition process through matching [8], analyzing resumes or other user representations for candidate screening [20] and automated assessment [12, 14], to broader tasks such as recommendations for upskilling [19].”
BackgroundAn Offer You Cannot Refuse? Trends in the Coercive Impact of Amazon Book Recommendations
Rystrøm, J. H. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-71975-2_1 · Cited by 3
Training philosopher engineers for better AI
Ball, B., & Koliousis, A. (2022) · AI & Society · doi:10.1007/s00146-022-01535-7 · Cited by 3
“Fair-ness is an active area of research in AI in general (Bakalar et al. 2021) and in hiring systems in particular (Wilson et al. 2021).”
BackgroundThird Workshop on Recommender Systems for Human Resources (RecSys in HR 2023)
Bogers, T., Graus, D., Kaya, M., Johnson, C., & Decorte, J.-J. (2023) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3604915.3608755 · Cited by 3
“Their applications range from assisting the talent acquisition process through matching [11], analyzing resumes or other user representations for candidate screening [23] and automated assessment [15, 17], to broader tasks such as recommendations for upskilling [22].”
BackgroundSystem Safety Engineering for Social and Ethical ML Risks: A Case StudyPreprint
Jatho, E. W., Mailloux, L. O., Rismani, S., Williams, E. D., & Kroll, J. A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.04602 · Cited by 3
“Practitioners and academics alike recognize the need for valid evaluation practices and welcome standardized frameworks [12, 13, 14, 15].”
BackgroundReducing Population-level Inequality Can Improve Demographic Group Fairness: a Twitter Case StudyPreprint
Ghosh, A., Lazovich, T., Lum, K., & Wilson, C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.08135 · Cited by 3
“Evidence shows these methods can effectively mitigate unfairness: pymetrics builds an ensemble model by excluding models that fail the four-fifths rule [44], while LinkedIn reduces bias by avoiding experiment versions that increase overall inequality [35].”
MethodSelf-citationChen, S., Xie, J., Wang, G., Wang, H., Cheng, H., & Huang, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.10280 · Cited by 3
Framing gender bias in the design of AI recruitment technology
Collett, C. (2024) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.5287/ora-mnrbyddp8 · Cited by 3
Fifth Workshop on Recommender Systems for Human Resources (RecSys in HR 2025)
Bogers, T., Kaya, M., Decorte, J.-J., Johnson, C., & Bied, G. (2025) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3705328.3748499 · Cited by 2
“Their applications range from assisting the talent acquisition process through matching [12], analyzing resumes or other user representations for candidate screening [24] and automated assessment [16, 18], to broader tasks such as recommendations for upskilling [23] or future career steps.”
Sandvig, C., Aidinoff, M., Nelson, A., Vaccaro, K., Karahalios, K. G., Bhandari, E., Biddle, E. R., Armstrong, L., Matias, J. N., Eslami, M., et al. (2026) · The MIT Press eBooks · doi:10.7551/mitpress/15997.001.0001 · Cited by 2
Revillod, G. (2026) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-83549-294-920261012 · Cited by 2
Resume-ing Control: (Mis)Perceptions of Agency Around GenAI Use in Recruiting Workflows
Surati, S., Bellini, R. F., & Black, E. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806498 · Cited by 2
Auditing a Robotic System for the AI Act
Lucaj, L., Bok, F., & van der Smagt, P. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812223 · Cited by 2
“…consequential contexts [Angwin et al. 2016; Buolamwini and Gebru 2018; Chouldechova et al. 2018; Chowdhury and Mulani 2018; Groves et al. 2024; Wilson et al. 2021], ensuring accountability in algorithmic decision-making [Birhane et al. 2024; Brown et al. 2021; Cen and Alur 2024; Raji et al.…”
Bias, transparency and accountability in algorithmic human resource management
Ashaye, A. L., Orisakahunsi, O. F., Lawal, O. L., Adeniyi, S. A., Ikeakaonwu, O. M., & Ishaku, P. K. (2026) · Strategic Business Research · doi:10.1016/j.sbr.2026.100308 · Cited by 2
The Impact of Features Used by Algorithms on Perceptions of Fairness
Estornell, A., Zhang, T., Das, S., Ho, C.-J., Juba, B., & Vorobeychik, Y. (2024) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2024/42 · Cited by 2
“…Buolamwini and Gebru, 2018; Mehrabi et al. , 2021; Hort et al. , 2022], auditing algorithms for bias [Washington, 2018; Buolamwini and Gebru, 2018; Wilson et al. , 2021], and designing algorithms which adhere to these definitions of fairness [Hardt et al. , 2016; Kusner et al. , 2017; Agarwal et…”
BackgroundWise Fusion : Group Fairness Enhanced Rank Fusion
Cachel, K., & Rundensteiner, E. A. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679649 · Cited by 2
“The field of fair ranking is a broad research area, including many different fairness metrics [12, 30, 40, 69, 79, 83, 93], bias-mitigation methods, empirical studies [8, 22, 38, 41, 72, 86, 90], and relevant surveys [33, 59, 71, 98, 99].”
Coded Bias in Applicant Tracking Systems
Dhanabal, P., & Hao, E. (2024) · Journal of Purdue Undergraduate Research · doi:10.7771/2158-4052.1720 · Cited by 2
Algorithmic Monoculture and Social Welfare
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603211 · Cited by 2
Fairness Criteria and Mitigation of AI Bias
Kim, H.-E. (2021) · 한국심리학회지: 일반 · doi:10.22257/kjp.2021.12.40.4.459 · Cited by 2
The Initial Screening Order Problem
Álvarez, J. M., Mastropietro, A., & Ruggieri, S. (2025) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3701551.3703497 · Cited by 2
“In this paper we explore a new problem within candidate screening: the initial screening order problem.”
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603198 · Cited by 2
Measuring the misinformation ecosystem: audiences, platforms, and storytellers
Jiang, S. (2021) · Digital Repository Service (Northeastern University) · doi:10.17760/d20412317 · Cited by 2
Self-citationThe Technology of Outrage: Bias in Artificial IntelligencePreprint
Bridewell, W. B., Bello, P., & Bringsjord, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.17336 · Cited by 2
Capital and CHI: Technological Capture and How It Structures CHI ResearchPreprint
Gilbert, É. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.14092 · Cited by 2
Xiao, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.09233 · Cited by 2
“Independent audits confirm that their AI screening tool reduces demographic bias while maintaining selection accuracy, ensuring balanced outcomes across gender and racial groups [58].”
Fairness and bias in algorithmic recruitment tools: An interdisciplinary approach
Hilliard, A. (2025) · Goldsmiths (University of London) · doi:10.25602/gold.00038521 · Cited by 2
FLARE-AI: Flaw Reporting for AIPreprint
Longpre, S., Zhu, E., Ezell, C., Ghosh, A., McGregor, S., Paeth, K., Klyman, K., Kapoor, S., Bommasani, R., Appel, R. E., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.31567 · Cited by 2
“As ML stakes grow, several approaches have emerged for evaluating AI systems, including cooperative audits (Wilson et al., 2021), participatory audits (Dennler et al., 2023), and AI red teaming (OpenAI, 2023; Ganguli et al., 2022).”
Self-citationFairness, Accountability and Transparency
Taylor, L., & MACKENZIE, J. (2025) · Wiley eBooks · doi:10.1002/9781394240821.ch10 · Cited by 1
Fair by Design? The Legal and Ethical Challenges of Algorithmic Hiring
da Silva, A. P., & da Costa, E. C. S. (2025) · Studies in Law and Justice · doi:10.56397/slj.2025.08.01 · Cited by 1
Fairness-Aware Interactive Target Variable Definition
Gala, D., Phillips-Brown, M., Goel, N., Prunkl, C., Jubete, L. A., Corcoran, M., & Eitel-Porter, R. (2025) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2025/1260 · Cited by 1
“In the real-world hiring algorithms that are based in cognitive tests, developers often define “good” employee by having an employer identify a group of current employees whom the employer deems “good” for a given role [Wilson et al. , 2021].”
Malpani, A., Panigrahi, A., Bhosale, V. P., Mahalle, A., Alsailawi, H. A., & Mudhafar, M. (2025) · IEEE International Conference on ICT in Business Industry & Government (ICTBIG) · doi:10.1109/ictbig68706.2025.11323785 · Cited by 1
“This article aims to evaluate the effectiveness of some bias mitigation approaches in AI-driven recruitment systems and to suggest a framework for ethically responsible human selection to implement AI in HR that promotes a fair, transparent, and inclusive set of recruitment practices [5].”
RetractedEthics, Bias Mitigation, and Model Risk Management
Parween, S., Dubey, N., Bharadwaj, R., Reddy, B. A., Sanghavi, H., Prasan, U. D., & Bhoopathy, V. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-6851-1.ch006 · Cited by 1
Standardization of AI Ethics: Stakeholders, Values and Profit
Corrêa, N. K., & Mönig, J. M. (2026) · Technosophy · doi:10.14361/9783839478431 · Cited by 1
Yi, H., Xian, L., Zhang, Y., Zeng, J., & Zhang, Z. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812239 · Cited by 1
Caught in the Loop: How Algorithmic Management Disempowers Frontline Supervisors
Oezdemir, F., Alizadeh, A., Benlian, A., & Wiener, M. J. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812344 · Cited by 1
Kun, A. L. (2026) · Cambridge University Press eBooks · doi:10.1017/9781009625180.005 · Cited by 1
Applications of Risk Science to AI Fairness Evaluation: Principles, Challenges, and Best Practices
Wilson, K., Kang, S., Dash, S., & Caliskan, A. (2026) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v9i4.43345 · Cited by 1
The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasons
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603204 · Cited by 1
Mitigating Bias in Algorithmic Decision-making: Evaluating Claims and Practices
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603212 · Cited by 1
Auditing Fairness and Imputation Impact in Predictive Analytics for Higher Education.Preprint
Anahideh, H., Nezami, N., & Gándara, D. (2021) · arXiv (Cornell University) · Cited by 1
Gala, D., Phillips-Brown, M., Goel, N., Prunkl, C., Jubete, L. A., Corcoran, M., & Eitel-Porter, R. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.06031 · Cited by 1
“In the real-world hiring algorithms that are based in cognitive tests, developers often define “good” employee by having an employer identify a group of current employees whom the employer deems “good” for a given role [Wilson et al. , 2021].”
Measuring the Effectiveness of Ethics-Based Auditing – A Semi-Systematic Literature ReviewPreprint
Shafik, E., Puncheva, P., Bharathy, G., & Pannai, E. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.5498160 · Cited by 1
On Worker Behavior in the AI-Enabled Service Triad
Snyder, C. (2025) · Deep Blue (University of Michigan) · doi:10.7302/25733 · Cited by 1
The Initial Screening Order Problem
Alvarez, J. M., Mastropietro, A., & Ruggieri, S. (2023) · arXiv · doi:10.48550/arxiv.2307.15398 · Cited by 1
MS Industrial and Organizational Psychology from Missouri State University Department of Psychology and SHRM- CP, U., & Ahmad, S. (2025) · International Journal of Research and Applied Innovations · doi:10.15662/ijrai.2025.0806023
Algorithmic monoculture and social welfare
(2023) · The Societal Impacts of Algorithmic Decision-Making · doi:10.1145/3603195.3603202
Sharma, G., & Molamohammadi, M. (2026) · arXiv · doi:10.48550/arxiv.2604.22679
“Independent evidence supporting vendor bias-reduction claims remains limited [5], and independent audits are rare [67].”
Leutner, F., Akhtar, R., & Chamorro‐Premuzic, T. (2022) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-83867-559-220221006
Some Adventures in Reliability Estimation
Thissen-Roe, A. (2022) · Springer proceedings in mathematics & statistics · doi:10.1007/978-3-031-04572-1_1
Candidate Prioritization Using Automated Behavioural Interview with Deep Learning
Panjwani, H. V., & Phamila, Y. A. V. (2023) · Advances in science and technology · doi:10.4028/p-y3z104
“Wilson[10] presented the case study of how pymetrics software being used for hiring, answers to assessment has been used as inputs to machine learning model to predict hiring match.”
MethodConscious AI Should Be Managed Similarly to HumansPreprint
Saurabh, A. (2023) · doi:10.58445/rars.194
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603213
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603205
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603196
How Do Classifiers Induce Agents to Behave Strategically?
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603201
How Do Classifiers Induce Agents to Behave Strategically?
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603210
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603214
Reviews in Recommender Systems: 2022
(2024) · Frontiers research topics · doi:10.3389/978-2-8325-4766-3
Generation Z Technological Frames of AI Game-Based Assessment
Trinh, N. T. T., & Elbanna, A. R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72234-9_6
AI in Pre-employment Assessments; How Unbiased, Fair and Objective Is It?
Arnaoutis, V., Hinton, C., van Zijl, L., & Lorenzen, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-52082-2_9
Liu, Y., Cao, Q., Shen, H., Zhang, K., Wu, Y., & Cheng, X. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757414
The Externalities of Exploration and How Data Diversity Helps Exploitation
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603199
Selection Problems in the Presence of Implicit Bias
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603200
Fair group-based exposure of documents in ranked search results
Jänich, T. (2025) · Enlighten: Theses (The University of Glasgow) · doi:10.5525/gla.thesis.85226
Navigating the Complexities of Algorithmic Auditing: Challenges and Considerations
Vecchione, B. (2023) · eCommons (Cornell University) · doi:10.7298/9826-gh79
Application des fonctions d'influence en traitement automatique du langage
Rancourt, F. (2024) · Archipel (Université du Québec à Montréal)
Algorithmic bias in graph-based recommender systems
Fabbri, F. (2022) · Repositori digital de la UPF (Universitat Pompeu Fabra)
Degelijke datasets en methoden voor webbeveiligingsonderzoek
Pochat, V. L. (2023) · Lirias
ALGORITHMIC APPROACHES TO AUTOMATED CANDIDATE SCREENING
Данчак, О., & VOVK, O. (2025) · Herald of Khmelnytskyi National University Technical sciences · doi:10.31891/2307-5732-2025-355-19
Metadata
- Authors
- Wilson, C., Ghosh, A., Jiang, S., Mislove, A., Baker, L., Szary, J., Trindel, K., & Polli, F.
- Affiliations
- Northeastern University (US)
- pymetrics, inc.
- Venue
- Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2021-03
- Pages
- 666-677
- OpenAlex
- W3133759846
- Cited by
- 133
- References
- 53
- Citations by year
- 2026: 19 · 2025: 17 · 2024: 17 · 2023: 45 · 2022: 27 · 2021: 8
- FWCI
- 21.93
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- Funders
- National Science Foundation (IIS-1553088)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Law › Artificial Intelligence in Law
- Keywords
- algorithmic fairness, algorithmic auditing, algorithmic accountability, socio-technical systems, algorithmic bias, transparency, social accountability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.