CoChat
Sign Up
Cited by 85Ethics and Social Impacts of AIAI and HR TechnologiesNames, Identity, and Discrimination Research

Identifying and Improving Disability Bias in GPT-Based Resume Screening

Open access
Glazko, Kate, Mohammed, Yusuf, Kosa, Ben, Potluri, Venkatesh, Mankoff, Jennifer·2024·The 2024 ACM Conference on Fairness Accountability and Transparency
DOI 10.1145/3630106.3658933
Open this paper in CoChat — free account

Abstract

As Generative AI rises in adoption, its use has expanded to include domains such as hiring and recruiting. However, without examining the potential of bias, this may negatively impact marginalized populations, including people with disabilities. To address this important concern, we present a resume audit study, in which we ask ChatGPT (specifically, GPT-4) to rank a resume against the same resume enhanced with an additional leadership award, scholarship, panel presentation, and membership that are disability-related. We find that GPT-4 exhibits prejudice towards these enhanced CVs. Further, we show that this prejudice can be quantifiably reduced by training a custom GPTs on principles of DEI and disability justice. Our study also includes a unique qualitative analysis of the types of direct and indirect ableism GPT-4 uses to justify its biased decisions and suggest directions for additional bias mitigation work. Additionally, since these justifications are presumably drawn from training data containing real-world biased statements made by humans, our analysis suggests additional avenues for understanding and addressing human bias.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3630106.3658933

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Media presentation of people with disabilities influences their livelihood through access to education and employment.

    “How people with disabilities (PwD) are presented in media plays a pivotal role in shaping public perceptions and social norms [34, 105], influencing livelihood through access to education and employment [47].”

    From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [47]

    Section 7.1 (p. 14) describes discrimination against disabled jobseekers and disparities in employment, salaries, and career advancement, supporting the employment-related part of the cited clause. The paper does not establish that media portrayals affect access to education or employment; its discussion of stereotypes in Section 4.4 (p. 12) concerns GPT-4 resume screening, not media presentation. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 67

  1. [2][2] Mason Ameri, Lisa Schur, Meera Adya, F Scott Bentley, Patrick McKay, and Douglas Kruse. 2018. The disability employment puzzle: A field experiment on employer hiring behavior. ILR Review 71, 2 (2018), 329-364. -
  2. [5][5] Patricia Berne, Aurora Levins Morales, David Langstaff, and Sins Invalid. 2018. Ten principles of disability justice. WSQ: Women's Studies Quarterly 46, 1 (2018), 227-230. -
  3. [6][6] Marianne Bertrand and Sendhil Mullainathan. 2004. Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination. American economic review 94, 4 (2004), 991-1013. -
  4. [7][7] Vegar Bjørnshagen and Elisabeth Ugreninov. 2021. Disability disadvantage: Experimental evidence of hiring discrimination against wheelchair users. European Sociological Review 37, 5 (03 2021), 818-833. -
  5. [8][8] Stephan Alexander Böhm, Christoph Breier, and Miriam Karin Baumgaertner. 2019. Getting stuck on the corporate ladder: The effect of disability on career progress. In Academy of Management Proceedings , Vol. 2019. Academy of Management Briarcliff Manor, NY 10510, 14611. -
  6. [10][10] Maarten Buyl, Christina Cociancig, Cristina Frattone, and Nele Roekens. 2022. Tackling algorithmic disability discrimination in the hiring process: An ethical, legal and technical analysis. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency . 1071-1082. -
  7. [12][12] Kathy Charmaz. 2010. Disclosing illness and disability in the workplace. Journal of International Education in Business 3, 1/2 (2010), 6-19. -
  8. [13][13] James L Cherney. 2011. The rhetoric of ableism. Disability Studies Quarterly 31, 3 (2011). -
  9. [14][14] Eva Derous and Ann Marie Ryan. 2012. Documenting the adverse impact of résumé screening: Degree of ethnic identification matters. International Journal of Selection and Assessment 20, 4 (2012), 464-474. -
  10. [15][15] Ketki V Deshpande, Shimei Pan, and James R Foulds. 2020. Mitigating demographic bias in AI-based resume filtering. In Adjunct publication of the 28th ACM conference on user modeling, adaptation and personalization . 268-275. -
  11. [18][18] Yingpeng Du, Di Luo, Rui Yan, Hongzhi Liu, Yang Song, Hengshu Zhu, and Jie Zhang. 2023. Enhancing job recommendation through LLM-based generative adversarial networks. arXiv preprint arXiv:2307.10747 (2023). - · arXiv
  12. [19][19] Heather D. Evans. 2019. 'Trial by fire': Forms of impairment disclosure and implications for disability identity. Disability & Society 34, 5 (2019), 726-746. -
  13. [20][20] S. Michael Gaddis. 2018. An introduction to audit studies in the social sciences . Springer. -
  14. [21][21] Vinitha Gadiraju, Shaun Kane, Sunipa Dev, Alex Taylor, Ding Wang, Emily Denton, and Robin Brewer. 2023. " I wouldn't say offensive but...": Disability-centered perspectives on large language models. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency . 205-216. -
  15. [23][23] Vahid Ghafouri, Vibhor Agarwal, Yong Zhang, Nishanth Sastry, Jose Such, and Guillermo Suarez-Tangil. 2023. AI in the gray: Exploring moderation policies in dialogic large language models vs. human answers in controversial topics. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management . 556-565. -
  16. [25][25] Kate S. Glazko, Momona Yamagami, Aashaka Desai, Kelly Avery Mack, Venkatesh Potluri, Xuhai Xu, and Jennifer Mankoff. 2023. An autoethnographic case study of generative artificial intelligence's utility for accessibility. In Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility . 1-8. -
  17. [26][26] W. Drew Gouvier, Sara Sytsma-Jordan, and Stephen Mayville. 2003. Patterns of discrimination in hiring job applicants with disabilities: The role of disability type, job complexity, and public contact. Rehabilitation psychology 48, 3 (2003), 175. -
  18. [27][27] Joe Graffam, Alison Shinkfield, Kaye Smith, and Udo Polzin. 2002. Factors that influence employer decisions in hiring and retaining an employee with a disability. Journal of Vocational Rehabilitation 17, 3 (2002), 175-181. -
  19. [28][28] Susan Grimes, Erica Southgate, Jill Scevak, and Rachel Buchanan. 2019. University student perspectives on institutional non-disclosure of disability and learning challenges: Reasons for staying invisible. International Journal of Inclusive Education 23, 6 (2019), 639-655. -
  20. [29][29] Morley Gunderson and Byron Y Lee. 2016. Pay discrimination against persons with disabilities: Canadian evidence from PALS. The International Journal of Human Resource Management 27, 14 (2016), 1531-1549. -
  21. [30][30] Amit Gupta and Pushpendra Priyadarshi. 2020. When affirmative action is not enough: Challenges in career development of persons with disability. Equality, Diversity and Inclusion: An International Journal 39, 6 (2020), 617-639. -
  22. [31][31] Christopher Harris. 2023. Mitigating age biases in resume screening AI models. In The International FLAIRS Conference Proceedings , Vol. 36. -
  23. [33][33] Saad Hassan, Matt Huenerfauth, and Cecilia Ovesdotter Alm. 2021. Unpacking the interdependent systems of discrimination: Ableist bias in NLP systems through an intersectional lens. arXiv preprint arXiv:2110.00521 (2021). - · arXiv
  24. [34][34] David R Hekman, Stefanie K Johnson, Maw-Der Foo, and Wei Yang. 2017. Does diversity-valuing behavior result in diminished performance ratings for non-white and female leaders? Academy of Management Journal 60, 2 (2017), 771-797. -
  25. [35][35] Christine A Henle, Brian R Dineen, and Michelle K Duffy. 2019. Assessing intentional resume deception: Development and nomological network of a resume fraud measure. Journal of Business and Psychology 34 (2019), 87-106. -
  26. [36][36] Brienna Herold, James Waller, and Raja Kushalnagar. 2022. Applying the stereotype content model to assess disability bias in popular pre-trained NLP models underlying AI-based assistive technologies. In Ninth Workshop on Speech and Language Processing for Assistive Technologies (SLPAT-2022) . 58-65. -
  27. [39][39] Gwen E Jones. 1997. Advancement opportunity issues for persons with disabilities. Human Resource Management Review 7, 1 (1997), 55-76. -
  28. [40][40] Sonia K. Kang, Katherine A. DeCelles, András Tilcsik, and Sora Jun. 2016. Whitened résumés: Race and self-presentation in the labor market. Administrative science quarterly 61, 3 (2016), 469-502. -
  29. [41][41] David M. Kaplan and James E. Fisher. 2009. A rose by any other name: Identity and impression management in résumés. Employee Responsibilities and Rights Journal 21 (2009), 319-332. -
  30. [42][42] Sara Kassir, Lewis Baker, Jackson Dolphin, and Frida Polli. 2023. AI for hiring in context: A perspective on overcoming the unique challenges of employment research to mitigate disparate impact. AI and Ethics 3, 3 (2023), 845-868. -
  31. [43][43] Judd B. Kessler, Corinne Low, and Colin D. Sullivan. 2019. Incentivized resume rating: Eliciting employer preferences without deception. American Economic Review 109, 11 (2019), 3713-3744. -
  32. [45][45] Akhil Alfons Kodiyan. 2019. An overview of ethical issues in using AI systems in hiring with a case study of Amazon's AI based hiring tool. Researchgate Preprint (2019), 1-19. -
  33. [46][46] Mukta Kulkarni. 2022. Hiding but hoping to be found: Workplace disclosure dilemmas of individuals with hidden disabilities. Equality, Diversity and Inclusion: An International Journal 41, 3 (2022), 491-507. -
  34. [47][47] Alain Lacroux and Christelle Martin-Lacroux. 2022. Should I trust the artificial intelligence to recruit? Recruiters' perceptions and behavior when faced with algorithm-based recommendation systems during resume screening. Frontiers in Psychology 13 (2022), 895997. -
  35. [48][48] Peter Leasure. 2021. A concise guide to designing and implementing an experimental correspondence audit that examines the impact of criminal history on hiring outcomes. Available at SSRN 3929613 (2021). -
  36. [52][52] Yannick l'Horty, Naomie Mahmoudi, Pascale Petit, and François-Charles Wolff. 2022. Is disability more discriminatory in hiring than ethnicity, address or gender? Evidence from a multi-criteria correspondence experiment. Social Science & Medicine 303 (2022), 114990. -
  37. [55][55] Jennifer Elizabeth Marshall, Colm Fearon, Marianne Highwood, and Katy Warden. 2020. 'What should I say to my employer...if anything?' - My disability disclosure dilemma. International Journal of Educational Management 34, 7 (2020), 1105-1117. -
  38. [57][57] Daniel McDuff, Shuang Ma, Yale Song, and Ashish Kapoor. 2019. Characterizing bias in classifiers using generative models. Advances in neural information processing systems 32 (2019). -
  39. [59][59] Katelyn Mei, Sonia Fereidooni, and Aylin Caliskan. 2023. Bias against 93 stigmatized groups in masked language models and downstream sentiment classification tasks. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency . 1699-1710. -
  40. [60][60] Emma Mishel. 2016. Discrimination against queer women in the US workforce: A résumé audit study. Socius 2 (2016), 2378023115621316. -
  41. [61][61] Haley Moss. 2020. Screened out onscreen: Disability discrimination, hiring bias, and artificial intelligence. Denv. L. Rev. 98 (2020), 775. -
  42. [62][62] Dena F Mujtaba and Nihar R Mahapatra. 2019. Ethical considerations in AI-based recruitment. In 2019 IEEE International Symposium on Technology and Society (ISTAS) . IEEE, 1-7. -
  43. [63][63] Daniel Nemirovsky, Nicolas Thiebaut, Ye Xu, and Abhishek Gupta. 2021. Providing actionable feedback in hiring marketplaces using generative adversarial networks. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining . 1089-1092. -
  44. [64][64] Selin E Nugent and Susan Scott-Parker. 2022. Recruitment AI has a disability problem: Anticipating and mitigating unfair automated hiring decisions. In Towards Trustworthy Artificial Intelligent Systems . Springer, 85-96. -
  45. [65][65] Jisoo Ock. 2022. The practical impact of bias against minority group applicants in resume screening on personnel selection outcomes. Sustainability 14, 15 (2022), 9438. -
  46. [69][69] Kaja Larsen Østerud. 2023. Disability discrimination: Employer considerations of disabled jobseekers in light of the ideal worker. Work, Employment and Society 37, 3 (2023), 740-756. -
  47. [71][71] Alejandro Pena, Ignacio Serna, Aythami Morales, and Julian Fierrez. 2020. Bias in multimodal AI: Testbed for fair automatic recruitment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops . 28-29. -
  48. [72][72] Tammy Prater and Sara Bliss Kiser. 2002. Lies, lies, and more lies. SAM Advanced Management Journal 67, 2 (2002), 9. -
  49. [73][73] Nitin Rane. 2023. Role and challenges of ChatGPT and similar generative artificial intelligence in human resource management. Available at SSRN 4603230 (2023). -
  50. [74][74] Chaminda Rathnayake and Aruni Gunawardana. 2023. The role of generative AI in enhancing human resource management recruitment, training, and performance evaluation perspectives. International Journal of Social Analytics 8, 11 (2023), 13-22. -
  51. [75][75] Jean-François Ravaud, Béatrice Madiot, and Isabelle Ville. 1992. Discrimination towards disabled people seeking employment. Social Science & Medicine 35, 8 (1992), 951-958. -
  52. [76][76] Alene Rhea, Kelsey Markey, Lauren D'Arinzo, Hilke Schellmann, Mona Sloane, Paul Squires, and Julia Stoyanovich. 2022. Resume format, LinkedIn URLs and other unexpected influences on AI personality prediction in hiring: Results of an audit. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society . 572-587. -
  53. [77][77] Emily Sheng, Josh Arnold, Zhou Yu, Kai-Wei Chang, and Nanyun Peng. 2021. Revealing persona biases in dialogue systems. arXiv preprint arXiv:2104.08728 (2021). - · arXiv
  54. [78][78] Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2019. The woman worked as a babysitter: On biases in language generation. arXiv preprint arXiv:1909.01326 (2019). - · arXiv
  55. [79][79] Eric Michael Smith and Adina Williams. 2021. Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue models. arXiv preprint arXiv:2109.03300 (2021). - · arXiv
  56. [80][80] Calvin Solomon. 2020. Autism and employment: Implications for employers and adults with ASD. Journal of Autism and Developmental Disorders 50, 11 (2020), 4209-4217. -
  57. [81][81] Alissa C. Stevens, Dianna D. Carroll, Elizabeth A. Courtney-Long, Qing C. Zhang, Michelle L. Sloan, Shannon Griffin-Blake, and Georgina Peacock. 2016. Adults with one or more functional disabilities-United States, 2011-2014. Morbidity and Mortality Weekly Report 65, 38 (2016), 1021-1025. -
  58. [82][82] Luhang Sun, Mian Wei, Yibing Sun, Yoo Ji Suh, Liwei Shen, and Sijia Yang. 2023. Smiling women pitching down: Auditing representational and presentational gender biases in image generative AI. arXiv preprint arXiv:2305.10566 (2023). - · arXiv
  59. [83][83] Siri Thanasombat and John Trasviña. 2005. Screening names instead of qualifications: Testing with emailed resumes reveals racial preferences. AAPI Nexus: Policy, Practice and Community 3, 2 (2005), 105-115. -
  60. [84][84] Nicholas Tilmes. 2022. Disability, fairness, and algorithmic bias in AI recruitment. Ethics and Information Technology 24, 2 (2022), 21. -
  61. [85][85] Thanh Tung Tran, Truong Giang Nguyen, Thai Hoa Dang, and Yuta Yoshinaga. 2023. Improving human resources' efficiency with a generative AI-based resume analysis solution. In International Conference on Future Data and Security Engineering . Springer, 352-365. -
  62. [86][86] Swanand Vaishampayan, Sahar Farzanehpour, and Chris Brown. 2023. Procedural justice and fairness in automated resume parsers for tech hiring: Insights from candidate perspectives. In 2023 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) . IEEE, 103-108. -
  63. [87][87] Elmira Van den Broek, Anastasia Sergeeva, and Marleen Huysman. 2021. When the machine meets the expert: An ethnography of developing AI for hiring. MIS quarterly 45, 3 (2021). -
  64. [88][88] Akshaj Kumar Veldanda, Fabian Grob, Shailja Thakur, Hammond Pearce, Benjamin Tan, Ramesh Karri, and Siddharth Garg. 2023. Are Emily and Greg still more employable than Lakisha and Jamal? Investigating algorithmic hiring bias in the era of ChatGPT. arXiv preprint arXiv:2310.05135 (2023). - · arXiv
  65. [89][89] Pranav Narayanan Venkit, Mukund Srinath, and Shomir Wilson. 2022. A study of implicit bias in pretrained language models against people with disabilities. In Proceedings of the 29th International Conference on Computational Linguistics . 1324-1332. -
  66. [93][93] Chantelle Wood and Megan Freeth. 2016. Students' stereotypes of autism. Journal of Educational Issues 2, 2 (2016), 131-140. -
  67. [96][96] A. Zarb. 2022. Assessing the role of gender in hiring: a field experiment on labour market discrimination. SN Business & Economics 2 (2022), 191. -

Cited by

Sorted by citations

Authors
Glazko, K., Mohammed, Y., Kosa, B., Potluri, V., & Mankoff, J.
Affiliations
University of Washington (US)
ORCID
Venue
The 2024 ACM Conference on Fairness Accountability and Transparency
Publisher
ACM
Published
2024-06-03
Pages
687-700
DOI
10.1145/3630106.3658933
OpenAlex
W4391622928
Cited by
85
References
5
Citations by year
2026: 37 · 2025: 41 · 2024: 7
FWCI
30.98
Type
conference paper
Language
en
Open access
Licence
Locations
  • ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • National Science Foundation BPE (UWEXP) (EDA 2009977)
  • MSR Azure Credits
  • NSF CSGrad4US
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › AI and HR Technologies
  • Social Sciences › Sociology and Political Science › Names, Identity, and Discrimination Research
Keywords
disability bias, resume screening, GPT-4, ChatGPT, hiring bias, ableism, disability justice, bias mitigation, resume ranking, recruitment automation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review