Social Biases in NLP Models as Barriers for Persons with Disabilities
Open accessAbstract
Building equitable and inclusive NLP technologies demands consideration of whether and how social attitudes are represented in ML models.In particular, representations encoded in models often inadvertently perpetuate undesirable social biases from the data on which they are trained.In this paper, we present evidence of such undesirable biases towards mentions of disability in two different English language models: toxicity prediction and sentiment analysis.Next, we demonstrate that the neural embeddings that are the critical first step in most NLP pipelines similarly contain undesirable biases towards mentions of disability.We end by highlighting topical biases in the discourse about disability which may contribute to the observed model biases; for instance, gun violence, homelessness, and drug addiction are over-represented in texts discussing mental illness.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The presence of biases in large language models has been well-documented, including negative sentiment and toxicity directed towards some social groups, stereotypical linguistic associations, and lack of recognition of certain language dialects.
“From negative sentiment and toxicity directed towards some social groups, to stereotypical linguistic associations, to lack of recognition of certain language dialects, the presence of biases of LLMs have been well-documented ( e.g. , Blodgett and O'Connor 2017; Hutchinson et al. 2020; Mei, Fereidooni, and Caliskan 2023; Mˇ echura 2022; Mozafari, Farahbakhsh, and Crespi 2020; Sap et al. 2019; Shen”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Hutchinson et al. 2020
Sections 3 and 4 report that disability-related phrases can receive elevated toxicity scores and negative sentiment associations, including in BERT word predictions. The cited paper does not establish a lack of recognition of language dialects; its evidence concerns disability bias in specific NLP models, rather than LLMs generally. Page numbers are unavailable in our copy. (LLM, conf=0.86)
Experimental — accuracy not yet measuredLanguage models inherit biases and dominant views from their training corpus.
“Text suggestions based on language models have been known to inherit biases and dominant 'views' from the text corpus used to train the language model [22,39,43,55].”
From Interacting with Next-Phrase Suggestions: How Suggestion Systems Aid and Influence the Cognitive Processes of Writing (2023) · cites this paper as [39]
LLMs learn from web-sourced text data, acquiring inherent biases.
“LLMs learn from web-sourced text data, acquiring inherent biases (31–33) and—in some cases—replicate excerpts from their training data (34).”
From How should the advancement of large language models affect the practice of science? (2025) · cites this paper as [33]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · see citing works
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · see citing works
Algorithms of oppression: how search engines reinforce racism
Polynczuk-Alenius, K. (2019) · Information, Communication & Society · doi:10.1080/1369118x.2019.1593485 · Cited by 2,934 · see citing works
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887 · see citing works
Algorithms of Oppression: How Search Engines Reinforce Racism
Noble, S. U. (2020) · doi:10.2307/j.ctt1pwt9w5 · Cited by 2,709 · see citing works
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · see citing works
Self-citationA Synopsis of Linguistic Theory, 1930-1955
Firth, J. R. (1957) · Medical Entomology and Zoology · Cited by 1,675 · see citing works
Fairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631 · see citing works
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615 · see citing works
Pervasiveness and correlates of implicit attitudes and stereotypes
Nosek, B. A., Smyth, F. L., Hansen, J. J., Devos, T., Lindner, N. M., Ranganath, K. A., Smith, C. T., Olson, K. R., Chugh, D., Greenwald, A. G., et al. (2007) · European Review of Social Psychology · doi:10.1080/10463280701489053 · Cited by 1,247 · see citing works
Word embeddings quantify 100 years of gender and ethnic stereotypes
Garg, N., Schiebinger, L., Jurafsky, D., & Zou, J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1720347115 · Cited by 1,065 · see citing works
The Risk of Racial Bias in Hate Speech Detection
Sap, M., Card, D., Gabriel, S., Choi, Y., & Smith, N. A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1163 · Cited by 817 · see citing works
Measuring and Mitigating Unintended Bias in Text Classification
Dixon, L., Li, J., Sorensen, J., Thain, N., & Vasserman, L. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278729 · Cited by 674 · see citing works
More than words: Social networks’ text mining for consumer brand sentiments
Mostafa, M. M. (2013) · Expert Systems with Applications · doi:10.1016/j.eswa.2013.01.019 · Cited by 624 · see citing works
Monroe, B. L., Colaresi, M. P., & Quinn, K. M. (2008) · Political Analysis · doi:10.1093/pan/mpn018 · Cited by 542 · see citing works
On Measuring Social Biases in Sentence Encoders
May, C., Wang, A., Bordia, S., Bowman, S. R., & Rudinger, R. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1063 · Cited by 461 · see citing works
Public awareness, attitudes and beliefs regarding intellectual disability: a systematic review
Scior, K. (2011) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2011.07.005 · Cited by 415 · see citing works
Racial Bias in Hate Speech and Abusive Language Detection Datasets
Davidson, T., Bhattacharya, D., & Weber, I. (2019) · Proceedings of the Third Workshop on Abusive Language Online · doi:10.18653/v1/w19-3504 · Cited by 414 · see citing works
Measuring Bias in Contextualized Word Representations
Kurita, K., Vyas, N., Pareek, A., Black, A. W., & Tsvetkov, Y. (2019) · Proceedings of the First Workshop on Gender Bias in Natural Language Processing · doi:10.18653/v1/w19-3823 · Cited by 348 · see citing works
Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
De-Arteaga, M., Romanov, A., Wallach, H., Chayes, J., Borgs, C., Chouldechova, A., Geyik, S., Kenthapadi, K., & Kalai, A. T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287572 · Cited by 337 · see citing works
Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification
Borkan, D., Dixon, L., Sorensen, J. S., Thain, N., & Vasserman, L. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308560.3317593 · Cited by 335 · see citing works
The social impact of natural language processing
Hovy, D., & Spruit, S. L. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p16-2096 · Cited by 329 · see citing works
Manzini, T., Chong, L. Y., Black, A. W., & Tsvetkov, Y. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1062 · Cited by 284 · see citing works
Delayed Impact of Fair Machine Learning
Liu, L. T., Dean, S., Rolf, E., Simchowitz, M., & Hardt, M. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/862 · Cited by 236 · see citing works
Challenges and frontiers in abusive content detection
Vidgen, B., Harris, A., Nguyen, D., Tromble, R., Hale, S. A., & Margetts, H. Z. (2019) · Workshop on Abusive Language Online (ALW) · doi:10.18653/v1/w19-3509 · Cited by 206 · see citing works
Counterfactual Fairness in Text Classification through Robustness
Garg, S., Perot, V., Limtiaco, N., Taly, A., Ed, H., & Beutel, A. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3317950 · Cited by 200 · see citing works
Runaway Feedback Loops in Predictive Policing
Ensign, D., Friedler, S. A., Neville, S., Scheidegger, C., & Venkatasubramanian, S. (2017) · arXiv · doi:10.48550/arxiv.1706.09847 · Cited by 161 · see citing works
Crossmod: A Cross-Community Learning-based System to Assist Reddit Moderators
Chandrasekharan, E., Gandhi, C., Mustelier, M. W., & Gilbert, E. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359276 · Cited by 151 · see citing works
The Social Cost of Strategic Classification
Milli, S., Miller, J. P., Dragan, A. D., & Hardt, M. (2019) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3287560.3287576 · Cited by 140 · see citing works
A Just and Comprehensive Strategy for Using NLP to Address Online Abuse
Jurgens, D., Hemphill, L., & Chandrasekharan, E. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1357 · Cited by 124 · see citing works
Perturbation Sensitivity Analysis to Detect Unintended Model Biases
Prabhakaran, V., Hutchinson, B., & Mitchell, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1578 · Cited by 115 · see citing works
Self-citationThe Social Impact Of Natural Language Processing
Hovy, D., & Spruit, S. L. (2017) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1000925 · Cited by 80 · see citing works
The Social Impact Of Natural Language Processing
Hovy, D., & Spruit, S. L. (2017) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1000924 · Cited by 80 · see citing works
(2018) · Cited by 78 · see citing works
Hanson, V. L., Cavender, A. C., & Trewin, S. M. (2015) · interactions · doi:10.1145/2828432 · Cited by 57 · see citing works
RtGender: A Corpus for Studying Differential Responses to Gender
Voigt, R., Jurgens, D., Prabhakaran, V., Jurafsky, D., & Tsvetkov, Y. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3di79eiki43x · Cited by 52 · see citing works
Self-citationModeration Techniques for Social Media Content
Veglis, A. A. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07632-4_13 · Cited by 41 · see citing works
The Linguistic Ideologies of Deep Abusive Language Classification
Castelle, M. (2018) · Workshop on Abusive Language Online (ALW) · doi:10.18653/v1/w18-5120 · Cited by 31 · see citing works
Toward Gender-Inclusive Coreference Resolution
Cao, Y. T., & Daumé, H., III (2020) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/2020.acl-main.418 · Cited by 22 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
2 of the 43 references couldn’t be matched to a known paper.
Research works printed but not linked: 4
- Su Lin Blodgett, Solon Barocas, Hal III Daume, and Hanna Wallach. 2020. Language (technology) is power: The need to be explicit about NLP harms. In Proceedings of the Annual Meeting of the Associ- ation for Computational Lingustics (ACL).
- Alexander D'Amour, Hansa Srinivasan, James At- wood, Pallavi Baljekar, D Sculley, and Yoni Halpern. 2020. Fairness is not static: Deeper understanding of long term fairness via simulation studies. In Pro- ceedings of the 2020 Conference on Fairness, Ac- countability, and Transparency, pages 525-534.
- Donald Martin Jr., Vinodkumar Prabhakaran, Jill Kuhlberg, Andrew Smart, and William Isaac. 2020. Participatory problem formulation for fairer ma- chine learning through community based system dy- namics approach. In ICLR Workshop on Machine Learning in Real Life (ML-IRL).
- Deirdre K Mulligan, Joshua A Kroll, Nitin Kohli, and Richmond Y Wong. 2019. This thing called fairness: Disciplinary confusion realizing a value in technol- ogy. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW):1-36.
Cited by
Sorted by citations
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · see citing works
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · see citing works
Self-citationLanguage (Technology) is Power: A Critical Survey of “Bias” in NLP
Blodgett, S. L., Barocas, S., Daumé III, H., & Wallach, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.485 · Cited by 1,050 · see citing works
“…et al. (2019); Sun et al. (2019); Tan and Celis (2019); Webster et al. (2019); Zmigrod et al. (2019); Gyamfi et al. (2020); Hube et al. (2020); Hutchinson et al. (2020); Kim et al. (2020); Nadeem et al. (2020); Papakyriakopoulos et al. (2020); Ravfogel et al. (2020); Rozado (2020); Sen and…”
InfluentialLund, B. D., Wang, T., Mannuru, N. R., Nie, B., Shimray, S., & Wang, Z. (2023) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24750 · Cited by 919 · see citing works
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693 · see citing works
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 665 · see citing works
Data and its (dis)contents: A survey of dataset development and use in machine learning research
Paullada, A., Raji, I. D., Bender, E. M., Denton, E., & Hanna, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100336 · Cited by 548 · see citing works
Self-citationThe Pile: An 800GB Dataset of Diverse Text for Language Modeling
Gao, L., Biderman, S., Black, S., Golding, L., Hoppe, T., Foster, C., Phang, J., He, H., Thite, A., Nabeshima, N., et al. (2021) · arXiv · doi:10.48550/arxiv.2101.00027 · Cited by 492 · see citing works
Large Language Models in Medicine: The Potentials and Pitfalls
Omiye, J. A., Gui, H., Rezaei, S. J., Zou, J., & Daneshjou, R. (2024) · Annals of Internal Medicine · doi:10.7326/m23-2772 · Cited by 394 · see citing works
Biases in Large Language Models: Origins, Inventory, and Discussion
Navigli, R., Conia, S., & Ross, B. (2023) · Journal of Data and Information Quality · doi:10.1145/3597307 · Cited by 364 · see citing works
Datasets: A Community Library for Natural Language Processing
Lhoest, Q., Villanova del Moral, A., Jernite, Y., Thakur, A., von Platen, P., Patil, S., Chaumond, J., Drame, M., Plu, J., Tunstall, L., et al. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2021.emnlp-demo.21 · Cited by 343 · see citing works
Hutchinson, B., Smart, A., Hanna, A., Denton, R., Greer, C., Kjartansson, O., Barnes, P., & Mitchell, M. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445918 · Cited by 279 · see citing works
Self-citationDealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations
Davani, A. M., Díaz, M., & Prabhakaran, V. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00449 · Cited by 237 · see citing works
Self-citationRe-imagining Algorithmic Fairness in India and Beyond
Sambasivan, N., Arnesen, E., Hutchinson, B., Doshi, T., & Prabhakaran, V. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445896 · Cited by 228 · see citing works
“While other axes of discrimination and injustices such as disability status [91], age [59], and sexual orientation [76] have gotten some attention, biases relevant to other geographies and cultures are not explored (e.”
BackgroundSelf-citationOn the genealogy of machine learning datasets: A critical history of ImageNet
Denton, E., Hanna, A., Amironesei, R., Smart, A., & Nicole, H. (2021) · Big Data & Society · doi:10.1177/20539517211035955 · Cited by 221 · see citing works
Self-citationGeDi: Generative Discriminator Guided Sequence Generation
Krause, B., Gotmare, A. D., McCann, B., Keskar, N. S., Joty, S., Socher, R., & Rajani, N. F. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.424 · Cited by 220 · see citing works
DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts
Liu, A., Sap, M., Lu, X., Swayamdipta, S., Bhagavatula, C., Smith, N. A., & Choi, Y. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.522 · Cited by 199 · see citing works
Foundation and large language models: fundamentals, challenges, opportunities, and social impacts
Myers, D., Mohawesh, R., Chellaboina, V. I., Sathvik, A. L., Venkatesh, P., Ho, Y.-H., Henshaw, H., Al-Hawawreh, M. S., Berdik, D., & Jararweh, Y. (2023) · Cluster Computing · doi:10.1007/s10586-023-04203-7 · Cited by 197 · see citing works
Perspectives on Large Language Models for Relevance Judgment
Faggioli, G., Dietz, L., Clarke, C. L. A., Demartini, G., Hagen, M., Hauff, C., Kando, N., Kanoulas, E., Potthast, M., Stein, B., et al. (2023) · Proceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval · doi:10.1145/3578337.3605136 · Cited by 184 · see citing works
Guo, W., & Caliskan, A. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462536 · Cited by 180 · see citing works
Toxicity in chatgpt: Analyzing persona-assigned language models
Deshpande, A., Murahari, V., Rajpurohit, T., Kalyan, A., & Narasimhan, K. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.88 · Cited by 178 · see citing works
Disability, fairness, and algorithmic bias in AI recruitment
Tilmes, N. (2022) · Ethics and Information Technology · doi:10.1007/s10676-022-09633-2 · Cited by 169 · see citing works
Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study
Cao, Y., Zhou, L., Lee, S., Cabello, L., Chen, M., & Hershcovich, D. (2023) · Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP) · doi:10.18653/v1/2023.c3nlp-1.7 · Cited by 163 · see citing works
Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation
Zhang, J., Bao, K., Zhang, Y., Wang, W., Feng, F., & He, X. (2023) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3604915.3608860 · Cited by 162 · see citing works
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Yoo, K. M., Park, D., Kang, J., Lee, S.-W., & Park, W. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.192 · Cited by 151 · see citing works
Characterising Bias in Compressed ModelsPreprint
Hooker, S., Moorosi, N., Clark, G., Samy Bengio, A., & Denton, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.03058 · Cited by 132 · see citing works
Self-citationGarrido-Muñoz, I., Montejo-Ráez, A., Martínez-Santiago, F., & Ureña-López, L. A. (2021) · Applied Sciences · doi:10.3390/app11073184 · Cited by 123 · see citing works
A taxonomy and review of generalization research in NLP
Hupkes, D., Giulianelli, M., Dankers, V., Artetxe, M., Elazar, Y., Pimentel, T., Christodoulopoulos, C., Lasri, K., Saphra, N., Sinclair, A., et al. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00729-y · Cited by 121 · see citing works
Auto-Debias: Debiasing Masked Language Models with Automated Biased Prompts
Guo, Y., Yang, Y., & Abbasi, A. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.72 · Cited by 120 · see citing works
Rethinking search: making domain experts out of dilettantes
Metzler, D., Tay, Y., Bahri, D., & Najork, M. (2021) · ACM SIGIR Forum · doi:10.1145/3476415.3476428 · Cited by 112 · see citing works
Generative AI meets Responsible AI: Practical Challenges and Opportunities
Kenthapadi, K., Lakkaraju, H., & Rajani, N. F. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599557 · Cited by 102 · see citing works
A survey on multilingual large language models: corpora, alignment, and bias
Xu, Y., Hu, L., Zhao, J., Qiu, Z., Xu, K., Ye, Y., & Gu, H. (2025) · Frontiers of Computer Science · doi:10.1007/s11704-024-40579-4 · Cited by 102 · see citing works
Bringing the People Back In: Contesting Benchmark Machine Learning Datasets
Denton, R., Hanna, A., Amironesei, R., Smart, A., Nicole, H., & Scheuerman, M. K. (2020) · arXiv · doi:10.48550/arxiv.2007.07399 · Cited by 100 · see citing works
Self-citationRobust Natural Language Processing: Recent Advances, Challenges, and Future Directions
Omar, M., Choi, S., Nyang, D., & Mohaisen, A. (2022) · IEEE Access · doi:10.1109/access.2022.3197769 · Cited by 96 · see citing works
A Survey of Race, Racism, and Anti-Racism in NLP
Field, A., Blodgett, S. L., Talat, Z., & Tsvetkov, Y. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.149 · Cited by 94 · see citing works
Do large language models have a legal duty to tell the truth?
Wachter, S., Mittelstadt, B., & Russell, C. (2024) · Royal Society Open Science · doi:10.1098/rsos.240197 · Cited by 85 · see citing works
An Autoethnographic Case Study of Generative Artificial Intelligence's Utility for Accessibility
Glazko, K. S., Yamagami, M., Desai, A., Mack, K. A., Potluri, V., Xu, X., & Mankoff, J. (2023) · The 25th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3597638.3614548 · Cited by 80 · see citing works
Training data influence analysis and estimation: a survey
Hammoudeh, Z., & Lowd, D. (2024) · Machine Learning · doi:10.1007/s10994-023-06495-7 · Cited by 80 · see citing works
Agrawal, A., Ali, A., & Boyd, S. (2021) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000090 · Cited by 78 · see citing works
CyBERT: Cybersecurity Claim Classification by Fine-Tuning the BERT Language Model
Ameri, K., Hempel, M., Sharif, H., Lopez, J., & Perumalla, K. S. (2021) · Journal of Cybersecurity and Privacy · doi:10.3390/jcp1040031 · Cited by 78 · see citing works
Conceptualising fairness: three pillars for medical algorithms and health equity
Sikstrom, L., Maslej, M. M., Hui, K., Findlay, Z., Buchman, D. Z., & Hill, S. L. (2022) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2021-100459 · Cited by 76 · see citing works
Nbias: A natural language processing framework for BIAS identification in text
Raza, S., Garg, M., Reji, D. J., Bashir, S. R., & Ding, C. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121542 · Cited by 76 · see citing works
How should the advancement of large language models affect the practice of science?
Binz, M., Alaniz, S., Roskies, A., Aczel, B., Bergstrom, C. T., Allen, C., Schad, D., Wulff, D., West, J. D., Zhang, Q., et al. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2401227121 · Cited by 74 · see citing works
Probing Toxic Content in Large Pre-Trained Language Models
Ousidhoum, N. D., Zhao, X., Fang, T., Song, Y., & Yeung, D. (2021) · doi:10.18653/v1/2021.acl-long.329 · Cited by 74 · see citing works
Sousa, S., & Kern, R. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10204-6 · Cited by 74 · see citing works
Nationality Bias in Text Generation
Venkit, P. N., Gautam, S., Panchanadikar, R., Huang, T.-H., & Wilson, S. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.9 · Cited by 74 · see citing works
A survey on large language models with multilingualism: recent advances and new frontiers
Huang, K., Mo, F., Zhang, X., Li, H., Li, Y., Zhang, Y., yi, W., Mao, Y., Liu, J., Xu, Y., et al. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11534-5 · Cited by 73 · see citing works
Czarnowska, P., Vyas, Y., & Shah, K. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00425 · Cited by 72 · see citing works
Beyond Discrimination: Generative AI Applications and Ethical Challenges in Forensic Psychiatry
Tortora, L. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1346059 · Cited by 72 · see citing works
GeDi: Generative Discriminator Guided Sequence GenerationPreprint
Krause, B., Gotmare, A. D., McCann, B., Keskar, N. S., Joty, S., Socher, R., & Rajani, N. F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.06367 · Cited by 72 · see citing works
What’s in the Box? An Analysis of Undesirable Content in the Common Crawl Corpus
Luccioni, A., & Viviano, J. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers) · doi:10.18653/v1/2021.acl-short.24 · Cited by 69 · see citing works
“They only care to show us the wheelchair”: disability representation in text-to-image AI models
Mack, K. A., Qadri, R., Denton, R., Kane, S. K., & Bennett, C. L. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642166 · Cited by 67 · see citing works
Self-citationImplicit bias in large language models: Experimental proof and implications for education
Warr, M., Oster, N. J., & Isaac, R. (2024) · Journal of Research on Technology in Education · doi:10.1080/15391523.2024.2395295 · Cited by 66 · see citing works
Emergent AI-assisted discourse: a case study of a second language writer authoring with ChatGPT
Jacob, S. R., Tate, T. P., & Warschauer, M. (2024) · Journal of China Computer-Assisted Language Learning · doi:10.1515/jccall-2024-0011 · Cited by 64 · see citing works
Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models
Ahia, O., Kumar, S., Gonen, H., Kasai, J., Mortensen, D. R., Smith, N. A., & Tsvetkov, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.614 · Cited by 63 · see citing works
Hate Speech Classifiers Learn Normative Social Stereotypes
Davani, A. M., Atari, M., Kennedy, B., & Dehghani, M. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00550 · Cited by 62 · see citing works
Commercial Use of Emotion Artificial Intelligence (AI): Implications for Psychiatry
Monteith, S., Glenn, T., Geddes, J. R., Whybrow, P. C., & Bauer, M. (2022) · Current Psychiatry Reports · doi:10.1007/s11920-022-01330-7 · Cited by 61 · see citing works
You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings
Talat, Z., Névéol, A., Biderman, S., Clinciu, M., Dey, M., Longpre, S., Luccioni, A. S., Masoud, M., Mitchell, M., Radev, D., et al. (2022) · doi:10.18653/v1/2022.bigscience-1.3 · Cited by 61 · see citing works
Addressing bias in artificial intelligence for public health surveillance
Flores, L., Kim, S., & Young, S. D. (2023) · Journal of Medical Ethics · doi:10.1136/jme-2022-108875 · Cited by 61 · see citing works
AI in interpreting: Ethical considerations
Horváth, I. (2022) · Across Languages and Cultures · doi:10.1556/084.2022.00108 · Cited by 56 · see citing works
Datasets: A Community Library for Natural Language ProcessingPreprint
Lhoest, Q., del Moral, A. V., Jernite, Y., Thakur, A., von Platen, P., Patil, S. S., Chaumond, J., Drame, M., Plu, J., Tunstall, L., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.02846 · Cited by 54 · see citing works
Discovering Language Model Behaviors with Model-Written Evaluations
Perez, E., Ringer, S., Lukošiūtė, K., Nguyen, K., Chen, E., Heiner, S., Pettit, C., Olsson, C., Kundu, S., Kadavath, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.09251 · Cited by 53 · see citing works
Changing the World by Changing the Data
Rogers, A. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.170 · Cited by 51 · see citing works
A Primer for Evaluating Large Language Models in Social-Science Research
Abdurahman, S., Ziabari, A. S., Moore, A. K., Bartels, D. M., & Dehghani, M. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251325174 · Cited by 51 · see citing works
Shen, T., Li, J., Bouadjenek, M. R., Mai, Z., & Sanner, S. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103139 · Cited by 49 · see citing works
Bhat, A., Agashe, S., Oberoi, P., Mohile, N., Jangir, R., & Joshi, A. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584060 · Cited by 47 · see citing works
Data Governance in the Age of Large-Scale Data-Driven Language Technology
Jernite, Y., Nguyen, H., Biderman, S., Rogers, A., Masoud, M., Danchev, V., Tan, S., Luccioni, A. S., Subramani, N., Johnson, I., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3534637 · Cited by 45 · see citing works
““Clean” text additionally has been misrepresented as being “unbiased” against any community—a notion that has been strongly contested [22, 73, 130].”
BackgroundHassan, S., Huenerfauth, M., & Alm, C. O. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.267 · Cited by 43 · see citing works
Data Representativeness in Accessibility Datasets: A Meta-Analysis
Kamikubo, R., Wang, L., Marte, C., Mahmood, A., & Kacorri, H. (2022) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3517428.3544826 · Cited by 41 · see citing works
Your fairness may vary: Pretrained language model fairness in toxic text classification
Baldini, I., Wei, D., Ramamurthy, K. N., Singh, M., & Yurochkin, M. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.176 · Cited by 39 · see citing works
Reliable and Safe Use of Machine Translation in Medical Settings
Mehandru, N., Robertson, S., & Salehi, N. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533244 · Cited by 37 · see citing works
GPT3Mix: Leveraging Large-scale Language Models for Text AugmentationPreprint
Yoo, K. M., Park, D., Kang, J., Lee, S.-W., & Park, W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08826 · Cited by 36 · see citing works
Ding, Z., Srinivasan, A., Macneil, S., & Chan, J. (2023) · Creativity and Cognition · doi:10.1145/3591196.3593516 · Cited by 35 · see citing works
Flitcroft, M. A., Sheriff, S. A., Wolfrath, N., Maddula, R., McConnell, L., Xing, Y., Haines, K., Wong, S. L., & Kothari, A. N. (2024) · Annals of Surgical Oncology · doi:10.1245/s10434-024-15549-6 · Cited by 33 · see citing works
Kroon, A. C., Welbers, K., Trilling, D., & van Atteveldt, W. (2023) · Communication Methods and Measures · doi:10.1080/19312458.2023.2261372 · Cited by 32 · see citing works
Krook, J., Winter, P., Downer, J., & Blockx, J. (2025) · AI and Ethics · doi:10.1007/s43681-025-00674-z · Cited by 32 · see citing works
Multimodal large language models for inclusive collaboration learning tasks
Lewis, A. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop · doi:10.18653/v1/2022.naacl-srw.26 · Cited by 30 · see citing works
Ameen, S., Wong, M. C., Yee, K., & Turner, P. A. (2022) · Applied Sciences · doi:10.3390/app12073341 · Cited by 30 · see citing works
Dewitte, P. (2024) · Computer law & security review · doi:10.1016/j.clsr.2024.106019 · Cited by 30 · see citing works
Towards Culturally Adaptive Large Language Models in Mental Health: Using ChatGPT as a Case Study
Aleem, M., Zahoor, I., & Naseem, M. (2024) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3678884.3681858 · Cited by 29 · see citing works
Detecting Cross-Geographic Biases in Toxicity Modeling on Social Media
Ghosh, S., Baker, D., Jurgens, D., & Prabhakaran, V. (2021) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/2021.wnut-1.35 · Cited by 29 · see citing works
Self-citationPipelines for Social Bias Testing of Large Language Models
Nozza, D., Bianchi, F., & Hovy, D. (2022) · doi:10.18653/v1/2022.bigscience-1.6 · Cited by 29 · see citing works
Sheth, S., Baker, H. P., Prescher, H., & Strelzow, J. (2024) · Journal of the American Academy of Orthopaedic Surgeons · doi:10.5435/jaaos-d-23-00787 · Cited by 28 · see citing works
Reliability Testing for Natural Language Processing Systems
Tan, S., Joty, S., Baxter, K., Taeihagh, A., Bennett, G. A., & Kan, M. (2021) · doi:10.18653/v1/2021.acl-long.321 · Cited by 27 · see citing works
Singhal, P., Gokhale, S., Shah, A., Jain, D. K., Walambe, R. A., Ekárt, A., & Kotecha, K. V. (2025) · Discover Applied Sciences · doi:10.1007/s42452-025-06659-1 · Cited by 26 · see citing works
Comparative Evaluation of Label-Agnostic Selection Bias in Multilingual Hate Speech Datasets
Ousidhoum, N. D., Song, Y., & Yeung, D. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.199 · Cited by 25 · see citing works
Demonstrations of the Potential of AI-based Political Issue Polling
Sanders, N. E., Ulinich, A., & Schneier, B. (2023) · Harvard Data Science Review · doi:10.1162/99608f92.1d3cf75d · Cited by 25 · see citing works
Venkit, P. N., Srinath, M., & Wilson, S. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.3 · Cited by 23 · see citing works
A Framework for Deprecating Datasets: Standardizing Documentation, Identification, and Communication
Luccioni, A. S., Corry, F., Sridharan, H., Ananny, M., Schultz, J., & Crawford, K. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533086 · Cited by 22 · see citing works
Palmer, Z. B., & Oswal, S. K. (2025) · Journal of Business and Technical Communication · doi:10.1177/10506519241280644 · Cited by 22 · see citing works
Janghorbani, S., & de Melo, G. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.126 · Cited by 21 · see citing works
A study on the distribution of social biases in self-supervised learning visual models
Sirotkin, K., Carballeira, P., & Escudero-Viñolo, M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01019 · Cited by 21 · see citing works
Opportunities and Risks of LLMs for Scalable Deliberation with Polis
Small, C. T., Vendrov, I., Durmus, E., Homaei, H., Barry, E., Cornebise, J., Suzman, T., Ganguli, D., & Megill, C. (2023) · arXiv · doi:10.48550/arxiv.2306.11932 · Cited by 20 · see citing works
“Despite significant efforts on the part of researchers developing LLMs, there remain issues around bias (Santurkar et al., 2023; Hutchinson et al., 2020; Kurita et al., 2019; Basta et al., 2019; Abid et al., 2021b; Sap et al., 2020), prompt injection attacks (Greshake et al.”
BackgroundAlshaigy, B., & Grande, V. (2024) · Nordic Conference on Human-Computer Interaction · doi:10.1145/3677045.3685493 · Cited by 20 · see citing works
Autoethnographic Insights from Neurodivergent GAI “Power Users”
Glazko, K. S., Cha, J., Lewis, A., Kosa, B., Wimer, B. L., Zheng, A., Zheng, Y., & Mankoff, J. C. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713670 · Cited by 20 · see citing works
The Sentiment Problem: A Critical Survey towards Deconstructing Sentiment Analysis
Venkit, P. N., Srinath, M., Gautam, S., Venkatraman, S., Gupta, V. K., Passonneau, R. J., & Wilson, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.848 · Cited by 20 · see citing works
Undesirable Biases in NLP: Addressing Challenges of Measurement
van der Wal, O., Bachmann, D., Leidinger, A., van Maanen, L., Zuidema, W. H., & Schulz, K. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15195 · Cited by 20 · see citing works
Xu, T., & Jumaat, N. F. (2024) · International Journal of Interactive Mobile Technologies (iJIM) · doi:10.3991/ijim.v18i15.49219 · Cited by 19 · see citing works
- Authors
- Hutchinson, B., Prabhakaran, V., Denton, E., Webster, K., Zhong, Y., & Denuyl, S.
- Venue
- Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
- Published
- 2020
- Pages
- 5491-5501
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W3035296331
- Cited by
- 274
- References
- 43
- Citations by year
- 2026: 21 · 2025: 41 · 2024: 44 · 2023: 66 · 2022: 44 · 2021: 48 · 2020: 10
- FWCI
- 43.56
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Social Sciences › Safety Research › Disability Rights and Representation
- Keywords
- social bias, disability, natural language processing models, toxicity prediction, sentiment analysis, neural embeddings, bias in language models, mental illness discourse, inclusive technology, disability representation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
