The DOI matches its Crossref record; 34 of the 35 references it cites were found in a registry or an index.
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models
Open accessAbstract
Warning: This paper contains explicit statements of offensive stereotypes and may be upsetting.Pretrained language models, especially masked language models (MLMs) have seen success across many NLP tasks.However, there is ample evidence that they use the cultural biases that are undoubtedly present in the corpora they are trained on, implicitly creating harm with biased representations.To measure some forms of social bias in language models against protected demographic groups in the US, we introduce the Crowdsourced Stereotype Pairs benchmark (CrowS-Pairs).CrowS-Pairs has 1508 examples that cover stereotypes dealing with nine types of bias, like race, religion, and age.In CrowS-Pairs a model is presented with two sentences: one that is more stereotyping and another that is less stereotyping.The data focuses on stereotypes about historically disadvantaged groups and contrasts them with advantaged groups.We find that all three of the widelyused MLMs we evaluate substantially favor sentences that express stereotypes in every category in CrowS-Pairs.As work on building less biased models advances, this dataset can be used as a benchmark to evaluate progress.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In the literature, many works refer to the metric as the dataset, and use these interchangeably.”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Nangia et al. 2020
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Each pair consists of one sentence demonstrating a stereotype, and the other sentence replacing the disadvantaged group with an advantaged one (”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Nangia et al. 2020
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“InstructGPT does not significantly improve over GPT-3 on the Winogender (Rudinger et al., 2018) and CrowSPairs (Nangia et al., 2020) datasets.”
From Training language models to follow instructions with human feedback (2022) · cites this paper as Nangia et al., 2020
The cited source says what the claim attributes to it.
In the citing paper
“InstructGPT does not significantly improve over GPT-3 on the Winogender (Rudinger et al., 2018) and CrowSPairs (Nangia et al., 2020) datasets.”
From Training language models to follow instructions with human feedback (2022) · cites this paper as Nangia et al., 2020
The cited source says what the claim attributes to it.
In the citing paper
“One example is the CrowS-Pairs (Nangia et al. 2020) dataset consisting of stereotype sentence pairs proposed alongside a pseudo-log-likelihood metric over this specific set of sentence pairs:”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Nangia et al. 2020
The cited source says what the claim attributes to it.
In the citing paper
“The dataset covers nine types of biases: race, gender, sexual orientation, religion, age, nationality, disability, physical appearance, and socioeconomic status.”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Nangia et al. 2020
The cited source says what the claim attributes to it.
In the citing paper
“Crowdsourced Stereotype Pairs (CrowS-Pairs) (Nangia et al. 2020) provides 1,508 sentence pairs to evaluate stereotypes of historically disadvantaged social groups.”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Nangia et al. 2020
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342
Self-citationSemantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Zhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., & Fidler, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.11 · Cited by 2,264
Know What You Don’t Know: Unanswerable Questions for SQuAD
Rajpurkar, P., Jia, R., & Liang, P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2124 · Cited by 2,240
Black sexual politics: African Americans, gender, and the new racism
(2005) · Choice Reviews Online · doi:10.5860/choice.42-3729 · Cited by 1,940
Research works printed but not linked: 6
- Emily Dinan, Angela Fan, Ledell Wu, Jason Weston, Douwe Kiela, and Adina Williams. 2020. Multi- dimensional gender bias classification. ArXiv.
- Paul Pu Liang, Irene Mengze Li, Emily Zheng, Yao Chong Lim, Ruslan Salakhutdinov, and Louis- Philippe Morency. 2020. Towards debiasing sen- tence representations. In Proceedings of the 2020 Association for Computational Linguistics. Associa- tion for Computational Linguistics.
- Alex Wang and Kyunghyun Cho. 2019. BERT has a mouth, and it must speak: BERT as a Markov random field language model. In Proceedings of the Workshop on Methods for Optimizing and Eval- uating Neural Language Generation, pages 30-36, Minneapolis, Minnesota. Association for Computa- tional Linguistics. · DOI
- Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019. SuperGLUE: A stickier benchmark for general-purpose language un- derstanding systems. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d Álché Buc, E. Fox, and R. Gar- nett, editors, Advances in Neural Information Pro- cessing Systems 32, pages 3266-3280.
- Kellie Webster, Marta Recasens, Vera Axelrod, and Ja- son Baldridge. 2018. Mind the GAP: A balanced corpus of gendered ambiguous pronouns. Transac- tions of the Association for Computational Linguis- tics, 6:605-617. · DOI · Paper page
- Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Rémi Louf, Morgan Funtow- icz, and Jamie Brew. 2019. HuggingFace's trans- formers: State-of-the-art natural language process- ing. ArXiv.
Cited by
Sorted by citations
Items 101–200 of 456
Federated deep long-tailed learning: A survey
Li, K., Li, Y., Zhang, J., Liu, X., & Ma, Z. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127906 · Cited by 21
ALGAN: Time Series Anomaly Detection with Adjusted-LSTM GAN
Bashar, M. A., & Nayak, R. (2025) · International Journal of Data Science and Analytics · doi:10.1007/s41060-025-00810-2 · Cited by 33
From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language Models
Mendelsohn, J., Bras, R. J. L., Choi, Y., & Sap, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.845 · Cited by 33
ROBBIE: Robust Bias Evaluation of Large Generative Language Models
Esiobu, D., Tan, X., Hosseini, S., Ung, M., Zhang, Y., Fernandes, J., Dwivedi-Yu, J., Presani, E., Williams, A., & Smith, E. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.230 · Cited by 49
MABEL: Attenuating Gender Bias using Textual Entailment Data
He, J., Xia, M., Fellbaum, C., & Chen, D. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.657 · Cited by 39
A Better Way to Do Masked Language Model Scoring
Kauf, C., & Ivanova, A. A. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.80 · Cited by 25
Fairness-Aware Structured Pruning in Transformers
Zayed, A., Mordido, G., Shabanian, S., Baldini, I., & Chandar, S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i20.30256 · Cited by 25
Specification overfitting in artificial intelligence
Roth, B., de Araujo, P. H. L., Xia, Y., Kaltenbrunner, S., & Korab, C. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-11040-6 · Cited by 21
Low-Complexity Probing via Finding Subnetworks
Cao, S., Sanh, V., & Rush, A. M. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.74 · Cited by 51
Socially Aware Bias Measurements for Hindi Language Representations
Malik, V., Dev, S., Nishi, A., Peng, N., & Chang, K.-W. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.76 · Cited by 21
Computational Modeling of Stereotype Content in Text
Fraser, K., Kiritchenko, S., & Nejadgholi, I. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.826207 · Cited by 23
Choose Your Lenses: Flaws in Gender Bias Evaluation
Orgad, H., & Belinkov, Y. (2022) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2022.gebnlp-1.17 · Cited by 24
In the Name of Fairness: Assessing the Bias in Clinical Record De-identification
Xiao, Y., Lim, S., Pollard, T., & Ghassemi, M. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3593982 · Cited by 23
AI Chatbots and Linguistic Injustice
Park, S. (2024) · Journal of Universal Language · doi:10.22425/jul.2024.25.1.99 · Cited by 23
KoBBQ: Korean Bias Benchmark for Question Answering
Jin, J., Kim, J., Lee, N., Yoo, H., Oh, A. H., & Lee, H. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00661 · Cited by 27
Balestri, R. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1558696 · Cited by 25
Gender Bias in Natural Language Processing and Computer Vision: A Comparative Survey
Bartl, M., Mandal, A., Leavy, S., & Little, S. (2024) · ACM Computing Surveys · doi:10.1145/3700438 · Cited by 23
Li, Y., Du, M., Wang, X., & Wang, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.797 · Cited by 18
Acceptability Judgements via Examining the Topology of Attention Maps
Cherniavskii, D., Tulchinskii, E., Mikhailov, V., Proskurina, I., Kushnareva, L., Artemova, E., Barannikov, S., Piontkovskaya, I., Piontkovski, D., & Burnaev, E. V. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.7 · Cited by 21
Venkit, P. N., Srinath, M., & Wilson, S. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.3 · Cited by 33
Towards fair decision: A novel representation method for debiasing pre-trained models
He, J., Lin, N., Bai, Q., Liang, H., Zhou, D. M., & Yang, A. (2024) · Decision Support Systems · doi:10.1016/j.dss.2024.114208 · Cited by 16
Pellert, M., Lechner, C. M., Wagner, C., Rammstedt, B., & Strohmaier, M. (2022) · doi:10.31234/osf.io/jv5dt · Cited by 16
KoSBI: A Dataset for Mitigating Social Bias Risks Towards Safer Large Language Model Applications
Lee, H., Hong, S., Park, J., Kim, T., Kim, G., & Ha, J.-W. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry Track) · doi:10.18653/v1/2023.acl-industry.21 · Cited by 15
Fewer Errors, but More Stereotypes? The Effect of Model Size on Gender Bias
Tal, Y., Magar, I., & Schwartz, R. (2022) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2022.gebnlp-1.13 · Cited by 25
SeeGULL: A Stereotype Benchmark with Broad Geo-Cultural Coverage Leveraging Generative Models
Jha, A., Davani, A. M., Reddy, C. K. A., Dave, S., Prabhakaran, V., & Dev, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.548 · Cited by 43
Social-Group-Agnostic Bias Mitigation via the Stereotype Content Model
Omrani, A., Ziabari, A. S., Yu, C., Golazizian, P., Kennedy, B., Atari, M., Ji, H., & Dehghani, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.227 · Cited by 16
FORK: A Bite-Sized Test Set for Probing Culinary Cultural Biases in Commonsense Reasoning Models
Palta, S., & Rudinger, R. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.631 · Cited by 16
A comparative study on the impact of model compression techniques on fairness in language models
Ramesh, K., Chavan, A., Pandit, S., & Sitaram, S. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.878 · Cited by 15
Detecting Unintended Social Bias in Toxic Language Datasets
Sahoo, N., Gupta, H., & Bhattacharyya, P. (2022) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2022.conll-1.10 · Cited by 25
Manifestations of xenophobia in AI systems
Tomašev, N., Maynard, J. L., & Gabriel, I. (2024) · AI & Society · doi:10.1007/s00146-024-01893-4 · Cited by 22
Li, Y., Du, M., Song, R., Wang, X., Sun, M., & Wang, Y. (2024) · Artificial Intelligence · doi:10.1016/j.artint.2024.104143 · Cited by 16
Howard, P., Madasu, A., Le, T., Lujan-Moreno, G. A., Bhiwandiwalla, A., & Lal, V. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01138 · Cited by 37
How Reliable are Model Diagnostics?
Aribandi, V., Tay, Y., & Metzler, D. (2021) · Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 · doi:10.18653/v1/2021.findings-acl.155 · Cited by 15
FairPrism: Evaluating Fairness-Related Harms in Text Generation
Fleisig, E., Amstutz, A., Atalla, C., Blodgett, S. L., Daumé III, H., Olteanu, A., Sheng, E., Vann, D., & Wallach, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.343 · Cited by 14
Choenni, R., Shutova, E., & van Rooij, R. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.111 · Cited by 20
On Measuring Social Biases in Prompt-Based Multi-Task Learning
Akyürek, A. F., Paik, S., Kocyigit, M. Y., Akbıyık, S., Runyun, Ş. L., & Wijaya, D. T. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.42 · Cited by 21
Debiasing pretrained text encoders by paying attention to paying attention
Gaci, Y., Benatallah, B., Casati, F., & Benabdeslem, K. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.651 · Cited by 14
Quantifying gender bias towards politicians in cross-lingual language models
Stańczak, K., Ray Choudhury, S., Pimentel, T., Cotterell, R., & Augenstein, I. (2023) · PLOS ONE · doi:10.1371/journal.pone.0277640 · Cited by 13
SODAPOP: Open-Ended Discovery of Social Biases in Social Commonsense Reasoning Models
An, H., Li, Z., Zhao, J., & Rudinger, R. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.116 · Cited by 24
Language Translation as a Socio-Technical System:Case-Studies of Mixed-Initiative Interactions
Santy, S., Bali, K., Choudhury, M., Dandapat, S., Ganu, T., Shukla, A., Shah, J., & Seshadri, V. (2021) · ACM SIGCAS Conference on Computing and Sustainable Societies (COMPASS) · doi:10.1145/3460112.3471954 · Cited by 15
Undesirable Biases in NLP: Addressing Challenges of Measurement
van der Wal, O., Bachmann, D., Leidinger, A., van Maanen, L., Zuidema, W. H., & Schulz, K. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15195 · Cited by 28
Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue modelsPreprint
Smith, E. M., & Williams, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.03300 · Cited by 33
CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Models
Zhao, J., Fang, M., Shi, Z., Li, Y., Chen, L., & Pechenizkiy, M. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.757 · Cited by 14
Bridging Fairness and Environmental Sustainability in Natural Language Processing
Hessenthaler, M., Strubell, E., Hovy, D., & Lauscher, A. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.533 · Cited by 16
Debiasing large language models: research opportunities*
Yogarajan, V., Dobbie, G., & Keegan, T. T. (2024) · Journal of the Royal Society of New Zealand · doi:10.1080/03036758.2024.2398567 · Cited by 18
Parity benchmark for measuring bias in LLMs
Simpson, S., Nukpezah, J., Brooks, K., & Pandya, R. (2024) · AI and Ethics · doi:10.1007/s43681-024-00613-4 · Cited by 12
Yi, F., Singh, A., & Tao, Z. (2024) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000101 · Cited by 13
Emerging trends: Unfair, biased, addictive, dangerous, deadly, and insanely profitable
Church, K., Schoene, A. M., Ortega, J. E., Chandrasekar, R., & Kordoni, V. (2022) · Natural Language Engineering · doi:10.1017/s1351324922000481 · Cited by 14
Logic Against Bias: Textual Entailment Mitigates Stereotypical Sentence Reasoning
Luo, H., & Glass, J. R. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.89 · Cited by 15
A Primer on Large Language Models and their LimitationsPreprint
Johnson, S., & Hyland-Wood, D. (2025) · Qeios · doi:10.32388/nhjyvs · Cited by 20
Limisiewicz, T., & Mareček, D. (2022) · Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2022.gebnlp-1.3 · Cited by 11
Fair and Argumentative Language Modeling for Computational Argumentation
Holtermann, C., Lauscher, A., & Ponzetto, S. P. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.541 · Cited by 16
Which Stereotypes Are Moderated and Under-Moderated in Search Engine Autocompletion?
Leidinger, A., & Rogers, R. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594062 · Cited by 12
Layered Bias: Interpreting Bias in Pretrained Large Language Models
Prakash, N., & Lee, R. K.-W. (2023) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2023.blackboxnlp-1.22 · Cited by 15
Comparing Biases and the Impact of Multilingual Training across Multiple Languages
Levy, S., John, N. A., Liu, L., Vyas, Y., Ma, J., Fujinuma, Y., Ballesteros, M., Castelli, V., & Roth, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.634 · Cited by 18
Abrar, A., Oeshy, N. T., Kabir, M., & Ananiadou, S. (2025) · AI & Society · doi:10.1007/s00146-025-02721-z · Cited by 24
Analyzing Stereotypes in Generative Text Inference Tasks
Sotnikova, A., Cao, Y., Daumé, H., & Rudinger, R. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.355 · Cited by 11
Social norm bias: residual harms of fairness-aware algorithms
Cheng, M., De‐Arteaga, M., Mackey, L. W., & Kalai, A. T. (2023) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00910-8 · Cited by 15
Learning to Initialize: Can Meta Learning Improve Cross-task Generalization in Prompt Tuning?
Qin, C., Joty, S., Li, Q., & Zhao, R. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.659 · Cited by 14
The tail wagging the dog: Dataset construction biases of social bias benchmarks
Selvam, N., Dev, S., Khashabi, D., Khot, T., & Chang, K.-W. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2023.acl-short.118 · Cited by 11
Iterative adversarial removal of gender bias in pretrained word embeddings
Gaci, Y., Benatallah, B., Casati, F., & Benabdeslem, K. (2022) · ACM Symposium on Applied Computing · doi:10.1145/3477314.3507274 · Cited by 10
This prompt is measuring <mask>: evaluating bias evaluation in language models
Goldfarb-Tarrant, S., Ungless, E. L., Balkır, E., & Blodgett, S. L. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.139 · Cited by 26
Measuring Gender Bias in Contextualized Embeddings
Katsarou, S., Rodríguez-Gálvez, B., & Shanahan, J. (2022) · doi:10.3390/cmsf2022003003 · Cited by 11
Multi-Agent Risks from Advanced AI
Hammond, L., Chan, A., Clifton, J., Hoelscher-Obermaier, J., Khan, A., McLean, E., Smith, C., Barfuss, W., Foerster, J., Gavenčiak, T., et al. (2025) · arXiv · doi:10.48550/arxiv.2502.14143 · Cited by 305
Fairness of Large Language Models in Education
Gao, R., Ni, Q., & Hu, B. (2024) · International Conference on Intelligent Education and Computer Technology (IECT) · doi:10.1145/3687311.3687404 · Cited by 9
LIBRA: Measuring Bias of Large Language Model from a Local Context
Pang, B., Qiao, T., Walker, C. G., Cunningham, C. W., & Koh, Y. S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88708-6_1 · Cited by 9
Cantini, R., Orsino, A., Ruggiero, M., & Talia, D. (2025) · Machine Learning · doi:10.1007/s10994-025-06862-6 · Cited by 34
A Survey on Bias in Machine Learning Research
Mikołajczyk, A., & Grochowski, M. (2025) · IEEE Access · doi:10.1109/access.2025.3649998 · Cited by 10
StereoKG: Data-Driven Knowledge Graph Construction For Cultural Knowledge and Stereotypes
Deshpande, A., Ruiter, D., Mosbach, M., & Klakow, D. (2022) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2022.woah-1.7 · Cited by 15
Mind Your Bias: A Critical Review of Bias Detection Methods for Contextual Language Models
Husse, S., & Spitz, A. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.311 · Cited by 14
Towards Ethics by Design in Online Abusive Content DetectionPreprint
Kiritchenko, S., & Nejadgholi, I. (2020) · arXiv (Cornell University) · doi:10.4224/40002027 · Cited by 15
A Brief Survey on Safety of Large Language Models
Gao, Z., Liu, X., Lan, Y.-S., & Yang, Z. (2024) · Journal of Computing and Information Technology · doi:10.20532/cit.2024.1005778 · Cited by 11
The Root Shapes the Fruit: On the Persistence of Gender-Exclusive Harms in Aligned Language Models
Ovalle, A., Pavasovic, K. L., Martin, L., Zettlemoyer, L., Smith, E. M., Chang, K.-W., Williams, A., & Sagun, L. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732196 · Cited by 12
Speciesist language and nonhuman animal bias in English Masked Language Models
Takeshita, M., Rzepka, R., & Araki, K. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103050 · Cited by 10
Eliciting and Understanding Cross-task Skills with Task-level Mixture-of-Experts
Ye, Q., Zha, J., & Ren, X. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.189 · Cited by 19
Bipol: A novel multi-axes bias evaluation metric with explainability for NLP
Alkhaled, L., Adewumi, T., & Sabry, S. S. (2023) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100030 · Cited by 8
SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models
Nagireddy, M., Chiazor, L., Singh, M., & Baldini, I. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i19.30142 · Cited by 8
Deciphering Stereotypes in Pre-Trained Language Models
Ma, W., Scheible, H., Wang, B., Veeramachaneni, G., Chowdhary, P., Sun, A., Koulogeorge, A., Wang, L., Yang, D., & Vosoughi, S. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.697 · Cited by 8
Intersectional Stereotypes in Large Language Models: Dataset and Analysis
Ma, W., Chiang, B., Wu, T., Wang, L., & Vosoughi, S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.575 · Cited by 11
Regional bias in monolingual English language models
Lyu, J., Dost, K., Koh, Y. S., & Wicker, J. S. (2024) · Machine Learning · doi:10.1007/s10994-024-06555-6 · Cited by 8
Listen and Speak Fairly: a Study on Semantic Gender Bias in Speech Integrated Large Language Models
Lin, Y., Lin, T.-Q., Yang, C.-K., Lu, K.-H., Chen, W.-C., Kuan, C.-Y., & Lee, H.-Y. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832317 · Cited by 9
Haghighi, N., Yu, S., Landay, J. A., & Rosner, D. K. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713633 · Cited by 12
Building Stereotype Repositories with Complementary Approaches for Scale and Depth
Dev, S., Jha, A., Goyal, J., Tewari, D., Dave, S., & Prabhakaran, V. (2023) · Workshop on Cross-Cultural Considerations in NLP (C3NLP) · doi:10.18653/v1/2023.c3nlp-1.9 · Cited by 8
Conceptor-Aided Debiasing of Large Language Models
Li, Y., Ungar, L., & Sedoc, J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.661 · Cited by 7
Socially Responsible Hate Speech Detection: Can Classifiers Reflect Social Stereotypes?
Vargas, F., Carvalho, I., Hürriyetoğlu, A., KNAW Humanities Cluster, D., Pardo, T. A. S., & Benevenuto, F. (2023) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-092-2_126 · Cited by 8
A Multi-dimensional study on Bias in Vision-Language models
Ruggeri, G., & Nozza, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.403 · Cited by 13
Understanding latent affective bias in large pre-trained neural language models
Kadan, A., Deepak, P., Bhadra, S., Gangan, M. P., & Lajish, V. L. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2024.100062 · Cited by 10
Detecting Linguistic Indicators for Stereotype Assessment with Large Language Models
Görge, R., Mock, M., & Allende‐Cid, H. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732181 · Cited by 10
Improving LLM Outputs Against Jailbreak Attacks With Expert Model Integration
Tsmindashvili, T., Kolkhidashvili, A., Kurtskhalia, D., Maghlakelidze, N., Mekvabishvili, E., Dentoshvili, G., Shamilov, O., Gachechiladze, Z., Saporta, S., & Choladze, D. D. (2025) · IEEE Access · doi:10.1109/access.2025.3592458 · Cited by 7
Yue, H., Palermo, C., Liu, R., & He, Y. (2025) · Chinese/English Journal of Educational Measurement and Evaluation · doi:10.59863/famj7696 · Cited by 9
Paul, V. (2024) · International Conference on Communication, Computing, Smart Materials and Devices (ICCCSMD) · doi:10.1109/icccsmd63546.2024.11015127 · Cited by 6
What social attitudes about gender does BERT encode? Leveraging insights from psycholinguistics
Watson, J., Beekhuizen, B., & Stevenson, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.375 · Cited by 8
Trust, Privacy and Security Aspects of Bias and Fairness in Machine Learning
Atabek, A., Eralp, E., & Gürsoy, M. E. (2023) · IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA) · doi:10.1109/tps-isa58951.2023.00023 · Cited by 8
The Bias that Lies Beneath: Qualitative Uncovering of Stereotypes in Large Language Models
Babonnaud, W., Delouche, E., & Lahlouh, M. (2024) · Linköping electronic conference proceedings · doi:10.3384/ecp208022 · Cited by 9
Reinforcement Guided Multi-Task Learning Framework for Low-Resource Stereotype Detection
Pujari, R., Oveson, E., Kulkarni, P., & Nouri, E. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.462 · Cited by 9
An Analysis of The Effects of Decoding Algorithms on Fairness in Open-Ended Language Generation
Dhamala, J., Kumar, V., Gupta, R., Chang, K.-W., & Galstyan, A. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10022722 · Cited by 10
Language model acceptability judgements are not always robust to context
Sinha, K., Gauthier, J., Mueller, A., Misra, K., Fuentes, K., Lévy, R., & Williams, A. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.333 · Cited by 12
Metadata
- Authors
- Nangia, N., Vania, C., Bhalerao, R., & Bowman, S. R.
- Affiliations
- New York University (US)
- ORCID
- Bhalerao, Rasika
- Venue
- Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Published
- 2020
- Pages
- 1953-1967
- OpenAlex
- W3089430725
- Cited by
- 446
- References
- 35
- Citations by year
- 2026: 71 · 2025: 93 · 2024: 64 · 2023: 121 · 2022: 60 · 2021: 36 · 2020: 1
- FWCI
- 56.53
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- TIB Data Manager
- Funders
- National Science Foundation (1922658)
- Nvidia
- Samsung
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- social bias, masked language models, stereotypes, benchmark dataset, race, age, crowdsourced data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
