Societal Biases in Language Generation: Progress and Challenges
Open accessAbstract
Emily Sheng, Kai-Wei Chang, Prem Natarajan, Nanyun Peng. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLMs internalize, spread, and potentially magnify harmful information and social biases existing in crawled training corpora.
“LLMs have been found to internalize, spread, and potentially magnify harmful information existing in the crawled training corpora, usually, toxic languages, like offensiveness, hate speech, and insults [53], as well as social biases like stereotypes towards people with a particular demographic identity ( e.g. , gender, race, religion, occupation, and ideology) [175].”
From A Survey on Evaluation of Large Language Models (2024) · cites this paper as [175]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1× · see citing works
“Furthermore, by filtering out all words deemed “bad”, Bender et al. (2021) warns that we remove the discourse of marginalized populations.”
BackgroundXLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687 · see citing works
VADER: a parsimonious rule-based model for sentiment analysis of social media text
Hutto, C., & Gilbert, E. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14550 · Cited by 6,087 · see citing works
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · see citing works
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · see citing works
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · see citing works
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469 · see citing works
The Curious Case of Neural Text Degeneration
Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09751 · Cited by 1,304 · see citing works
Hierarchical Neural Story Generation
Fan, A., Lewis, M., & Dauphin, Y. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1082 · Cited by 1,266 · see citing works
Hierarchical Neural Story Generation
Fan, A., Lewis, M., & Dauphin, Y. (2018) · arXiv · doi:10.48550/arxiv.1805.04833 · Cited by 1,242 · see citing works
DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation
Zhang, Y., Sun, S., Galley, M., Chen, Y.-C., Brockett, C., Gao, X., Gao, J., Liu, J., & Dolan, B. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.18653/v1/2020.acl-demos.30 · Cited by 1,076 · see citing works
Language (Technology) is Power: A Critical Survey of “Bias” in NLP
Blodgett, S. L., Barocas, S., Daumé III, H., & Wallach, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.485 · Cited by 1,050 · cited 3× · see citing works
“We survey detrimental representational9 and allocational10 impacts (Crawford, 2017; Barocas et al., 2017; Blodgett et al., 2020) used to motivate”
BackgroundBender, E. M., & Friedman, B. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00041 · Cited by 854 · see citing works
Extracting Training Data from Large Language Models
Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07805 · Cited by 727 · see citing works
Universal Adversarial Triggers for Attacking and Analyzing NLP
Wallace, E., Feng, S., Kandpal, N., Gardner, M., & Singh, S. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1221 · Cited by 642 · see citing works
Data and its (dis)contents: A survey of dataset development and use in machine learning research
Paullada, A., Raji, I. D., Bender, E. M., Denton, E., & Hanna, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100336 · Cited by 548 · cited 1× · see citing works
“Paullada et al. (2020), Bender and Fried-man (2018), and Gebru et al. (2018) provide more comprehensive surveys and frameworks that focus on aspects of data creation and management that could lead to biases, and we refer readers to their works for more discussion.”
BackgroundUnequal Representation and Gender Stereotypes in Image Search Results for Occupations
Kay, M., Matuszek, C., & Munson, S. A. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702520 · Cited by 509 · see citing works
Plug and Play Language Models: A Simple Approach to Controlled Text Generation
Dathathri, S., Madotto, A., Lan, J., Hung, J., Frank, E., Molino, P., Yosinski, J., & Liu, R. (2019) · arXiv · doi:10.48550/arxiv.1912.02164 · Cited by 508 · see citing works
Evaluating Gender Bias in Machine Translation
Stanovsky, G., Smith, N. A., & Zettlemoyer, L. (2019) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1164 · Cited by 492 · see citing works
Mitigating Gender Bias in Natural Language Processing: Literature Review
Sun, T., Gaut, A., Tang, S., Huang, Y., ElSherief, M., Zhao, J., Mirza, D., Belding, E., Chang, K.-W., & Wang, W. Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1159 · Cited by 492 · see citing works
Self-citationPersistent Anti-Muslim Bias in Large Language Models
Abid, A., Farooqi, M., & Zou, J. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462624 · Cited by 476 · see citing works
(2020) · arXiv (Cornell University) · Cited by 461 · see citing works
The Woman Worked as a Babysitter: On Biases in Language Generation
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1339 · Cited by 431 · cited 10× · see citing works
“Language models are the core components for many NLG and NLU tasks, and this task enables directly quantifying biases in large, pre-trained language models (Bordia and Bowman, 2019; Sheng et al., 2019; Solaiman et al., 2019; Brown et al., 2020).”
BackgroundMethodInfluentialSelf-citationAssessing gender bias in machine translation: a case study with Google Translate
Prates, M. O. R., Avelar, P. H., & Lamb, L. C. (2020) · Neural Computing and Applications · doi:10.1007/s00521-019-04144-6 · Cited by 410 · see citing works
Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems
Kiritchenko, S., & Mohammad, S. (2018) · Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/s18-2005 · Cited by 384 · see citing works
Learning Gender-Neutral Word Embeddings
Zhao, J., Zhou, Y., Li, Z., Wang, W., & Chang, K.-W. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1521 · Cited by 376 · see citing works
Self-citationReducing Gender Bias in Abusive Language Detection
Park, J. H., Shin, J., & Fung, P. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1302 · Cited by 323 · see citing works
Release Strategies and the Social Impacts of Language Models
Solaiman, I., Brundage, M., Clark, J., Askell, A., Herbert-Voss, A., Wu, J., Radford, A., Krueger, G., Kim, J. W., Kreps, S., et al. (2019) · arXiv · doi:10.48550/arxiv.1908.09203 · Cited by 292 · see citing works
Gender and Representation Bias in GPT-3 Generated Stories
Lucy, L., & Bamman, D. (2021) · Proceedings of the Third Workshop on Narrative Understanding · doi:10.18653/v1/2021.nuse-1.5 · Cited by 287 · see citing works
Gender Bias in Neural Natural Language Processing
Lu, K., Mardziel, P., Wu, F., Amancharla, P., & Datta, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-62077-6_14 · Cited by 284 · see citing works
Fairness Without Demographics in Repeated Loss MinimizationPreprint
Hashimoto, T., Srivastava, M., Namkoong, H., & Liang, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.08010 · Cited by 263 · see citing works
BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation
Dhamala, J., Sun, T., Kumar, V., Krishna, S., Pruksachatkun, Y., Chang, K.-W., & Gupta, R. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445924 · Cited by 244 · see citing works
Self-citationCounterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology
Zmigrod, R., Mielke, S. J., Wallach, H., & Cotterell, R. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1161 · Cited by 241 · see citing works
Predictive biases in natural language processing models: A conceptual framework and overview
Shah, D. S., Schwartz, H. A., & Hovy, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.468 · Cited by 241 · see citing works
Re-imagining Algorithmic Fairness in India and Beyond
Sambasivan, N., Arnesen, E., Hutchinson, B., Doshi, T., & Prabhakaran, V. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445896 · Cited by 228 · cited 1× · see citing works
“Bias Evaluations It is difficult to evaluate societal biases in NLG tasks because NLG can be open-domain, and there are many different notions of biases from various backgrounds and cultures (Sambasivan et al., 2021).”
BackgroundSelf-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP
Schick, T., Udupa, S., & Schütze, H. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00434 · Cited by 227 · see citing works
GeDi: Generative Discriminator Guided Sequence Generation
Krause, B., Gotmare, A. D., McCann, B., Keskar, N. S., Joty, S., Socher, R., & Rajani, N. F. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.424 · Cited by 220 · see citing works
Identifying and Reducing Gender Bias in Word-Level Language Models
Bordia, S., & Bowman, S. R. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop · doi:10.18653/v1/n19-3002 · Cited by 219 · see citing works
Gender Bias in Machine Translation
Savoldi, B., Gaido, M., Bentivogli, L., Negri, M., & Turchi, M. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00401 · Cited by 189 · see citing works
Equalizing Gender Bias in Neural Machine Translation with Word Embeddings Techniques
Font, J. E., & Costa-jussà, M. (2019) · Proceedings of the First Workshop on Gender Bias in Natural Language Processing · doi:10.18653/v1/w19-3821 · Cited by 187 · see citing works
Ethical Challenges in Data-Driven Dialogue Systems
Henderson, P., Sinha, K., Gontier, N., Ke, N. R., Fried, G., Lowe, R., & Pineau, J. (2018) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3278721.3278777 · Cited by 166 · see citing works
Getting Gender Right in Neural Machine Translation
Vanmassenhove, E., Hardmeier, C., & Way, A. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1334 · Cited by 158 · see citing works
Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation
Dinan, E., Fan, A., Williams, A., Urbanek, J., Kiela, D., & Weston, J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.656 · Cited by 157 · see citing works
Automatically Neutralizing Subjective Bias in Text
Pryzant, R., Martinez, R. D., Dass, N., Kurohashi, S., Jurafsky, D., & Yang, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5385 · Cited by 155 · see citing works
Decolonising Speech and Language Technology
Bird, S. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.313 · Cited by 151 · see citing works
(2020) · neural information processing systems · Cited by 142 · see citing works
Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models
Tamkin, A., Brundage, M., Clark, J., & Ganguli, D. (2021) · arXiv · doi:10.48550/arxiv.2102.02503 · Cited by 135 · see citing works
Distilling Knowledge Learned in BERT for Text Generation
Chen, Y.-C., Gan, Z., Yu, C., Liu, J., & Liu, J. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.705 · Cited by 131 · see citing works
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
Huang, P.-S., Zhang, H., Jiang, R., Stanforth, R., Welbl, J., Rae, J., Maini, V., Yogatama, D., & Kohli, P. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.7 · Cited by 124 · see citing works
Confronting Abusive Language Online: A Survey from the Ethical and Human Rights Perspective
Kiritchenko, S., Nejadgholi, I., & Fraser, K. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12590 · Cited by 118 · see citing works
Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem
Saunders, D., & Byrne, B. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.690 · Cited by 100 · see citing works
Groenwold, S., Ou, L.-H., Parekh, A., Honnavalli, S., Levy, S., Mirza, D., & Wang, W. Y. (2020) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2020.emnlp-main.473 · Cited by 100 · see citing works
Towards Controllable Biases in Language Generation
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.291 · Cited by 95 · see citing works
Self-citationHONEST: Measuring Hurtful Sentence Completion in Language Models
Nozza, D., Bianchi, F., & Hovy, D. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.191 · Cited by 95 · see citing works
Multi-Dimensional Gender Bias Classification
Dinan, E., Fan, A., Wu, L., Weston, J., Kiela, D., & Williams, A. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.23 · Cited by 94 · see citing works
Does Gender Matter? Towards Fairness in Dialogue Systems
Liu, H., Dacon, J., Fan, W., Liu, H., Liu, Z., & Tang, J. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.390 · Cited by 81 · see citing works
GeDi: Generative Discriminator Guided Sequence GenerationPreprint
Krause, B., Gotmare, A. D., McCann, B., Keskar, N. S., Joty, S., Socher, R., & Rajani, N. F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.06367 · Cited by 72 · see citing works
Mitigating Gender Bias in Machine Translation with Target Gender AnnotationsPreprint
Stafanovičs, A., Pinnis, M., & Bergmanis, T. (2020) · Conference on Machine Translation · doi:10.48550/arxiv.2010.06203 · Cited by 71 · see citing works
Reducing Gender Bias in Word-Level Language Models with a Gender-Equalizing Loss Function
Qian, Y., Muaz, U., Zhang, B., & Hyun, J. W. (2019) · Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (ACL SRW) · doi:10.18653/v1/p19-2031 · Cited by 63 · see citing works
PowerTransformer: Unsupervised Controllable Revision for Biased Language Correction
Ma, X., Sap, M., Rashkin, H., & Choi, Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.602 · Cited by 61 · see citing works
Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learning
Liu, H., Wang, W., Wang, Y., Liu, H., Liu, Z., & Tang, J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.64 · Cited by 57 · see citing works
On Measuring Gender Bias in Translation of Gender-neutral Pronouns
Cho, W. I., Kim, J. W., Kim, S., & Kim, N. S. (2019) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/w19-3824 · Cited by 56 · see citing works
Tomalin, M., Byrne, B., Concannon, S. J., Saunders, D., & Ullmann, S. (2021) · Ethics and Information Technology · doi:10.1007/s10676-021-09583-1 · Cited by 55 · see citing works
“You Sound Just Like Your Father” Commercial Machine Translation Systems Include Stylistic Biases
Hovy, D., Bianchi, F., & Fornaciari, T. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.154 · Cited by 47 · see citing works
Silva, A., Tambwekar, P., & Gombolay, M. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.189 · Cited by 46 · see citing works
They, Them, Theirs: Rewriting with Gender-Neutral EnglishPreprint
Sun, T., Webster, K., Shah, A., Wang, W. Y., & Johnson, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.06788 · Cited by 42 · see citing works
“Nice Try, Kiddo”: Investigating Ad Hominems in Dialogue Responses
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.60 · Cited by 42 · see citing works
Self-citation“You are grounded!”: Latent Name Artifacts in Pre-trained Language Models
Shwartz, V., Rudinger, R., & Tafjord, O. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.556 · Cited by 41 · see citing works
Neural Machine Translation Doesn't Translate Gender Coreference Right Unless You Make ItPreprint
Saunders, D., Sallis, R., & Byrne, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.05332 · Cited by 41 · see citing works
Automatic Gender Identification and Reinflection in Arabic
Habash, N. Y., Bouamor, H., & Chung, C. (2019) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/w19-3822 · Cited by 35 · see citing works
Filling Gender & Number Gaps in Neural Machine Translation with Black-box Context Injection
Moryossef, A., Aharoni, R., & Goldberg, Y. (2019) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/w19-3807 · Cited by 35 · see citing works
Mitigating Gender Bias in Machine Translation with Target Gender Annotations
Stafanovičs, A., Bergmanis, T., & Pinnis, M. (2020) · Conference on Machine Translation (WMT) · doi:10.18653/v1/2020.wmt-1.73 · Cited by 35 · see citing works
Discovering and Categorising Language Biases in Reddit
Ferrer, X., van Nuenen, T., Such, J. M., & Criado, N. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v15i1.18048 · Cited by 34 · see citing works
Gender aware spoken language translation applied to English-Arabic
ElAraby, M., Tawfik, A. Y., Khaled, M., Hassan, H., & Osama, A. (2018) · International Conference on Natural Language and Speech Processing (ICNLSP) · doi:10.1109/icnlsp.2018.8374387 · Cited by 29 · see citing works
Automatically Identifying Gender Issues in Machine Translation using Perturbations
Gonen, H., & Webster, K. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.180 · Cited by 28 · see citing works
Investigating Failures of Automatic Translation in the Case of Unambiguous GenderPreprint
Renduchintala, A., & Williams, A. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2104.07838 · Cited by 28 · see citing works
Investigating Societal Biases in a Poetry Composition SystemPreprint
Sheng, E., & Uthus, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.02686 · Cited by 26 · see citing works
Self-citationReducing Non-Normative Text Generation from Language Models
Peng, X., Li, S., Frazier, S., & Riedl, M. O. (2020) · International Conference on Natural Language Generation · doi:10.18653/v1/2020.inlg-1.43 · Cited by 25 · see citing works
Fine-tuning neural machine translation on gender-balanced datasets
Costa‐jussà, M. R., & Sánchez, A. D. J. (2020) · Cited by 24 · see citing works
Towards Cross-Lingual Generalization of Translation Gender Bias
Cho, W. I., Kim, J., Yang, J., & Kim, N. S. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445907 · Cited by 23 · see citing works
Revealing Persona Biases in Dialogue SystemsPreprint
Sheng, E., Arnold, J., Yu, Z., Chang, K.-W., & Peng, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08728 · Cited by 21 · see citing works
Self-citationDefining and Evaluating Fair Natural Language Generation
Yeo, C., & Chen, A. (2020) · doi:10.18653/v1/2020.winlp-1.27 · Cited by 20 · see citing works
Decoding and Diversity in Machine TranslationPreprint
Roberts, N. P., Liang, D., Neubig, G., & Lipton, Z. C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.13477 · Cited by 20 · see citing works
Farkas, A., & Németh, R. (2021) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2021.100239 · Cited by 20 · see citing works
Gender-Aware Reinflectionusing Linguistically Enhanced Neural Models
International Committee for Computational Linguistics 2020, Alhafni, B., Bouamor, H., & Habash, N. (2020) · doi:10.48448/6nn5-6v05 · Cited by 20 · see citing works
Basta, C., Costa‐jussà, M. R., & Fonollosa, J. A. R. (2020) · Widening Natural Language Processing Workshop (WiNLP) · doi:10.18653/v1/2020.winlp-1.25 · Cited by 19 · see citing works
Curry, A. C., Robertson, J., & Rieser, V. (2020) · Cited by 17 · see citing works
Persistent Anti-Muslim Bias in Large Language ModelsPreprint
Abid, A., Farooqi, M., & Zou, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.05783 · Cited by 17 · see citing works
First the Worst: Finding Better Gender Translations During Beam Search
Saunders, D., Sallis, R., & Byrne, B. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.301 · Cited by 16 · see citing works
Gender Coreference and Bias Evaluation at WMT 2020
Kocmi, T., Limisiewicz, T., & Stanovsky, G. (2020) · Conference on Machine Translation (WMT) · doi:10.18653/v1/2020.wmt-1.39 · Cited by 14 · see citing works
Improving Gender Translation Accuracy with Filtered Self-TrainingPreprint
Choubey, P. K., Currey, A., Mathur, P., & Dinu, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.07695 · Cited by 10 · see citing works
On-the-Fly Controlled Text Generation with Experts and Anti-Experts.Preprint
Liu, A., Sap, M., Lu, X., Swayamdipta, S., Bhagavatula, C., Smith, N. A., & Choi, Y. (2021) · arXiv (Cornell University) · Cited by 9 · see citing works
Gender Bias in Multilingual Neural Machine Translation: The Architecture MattersPreprint
Costa‐jussà, M. R., Escolano, C., Basta, C., Ferrando, J., Batlle, R., & Kharitonova, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.13176 · Cited by 7 · see citing works
Reducing Non-Normative Text Generation from Language ModelsPreprint
Peng, X., Li, S., Frazier, S., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.08764 · Cited by 7 · see citing works
Discovering and Categorising Language Biases in RedditPreprint
Ferrer, X., van Nuenen, T., Such, J. M., & Criado, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.02754 · Cited by 7 · see citing works
Gender Bias in Multilingual Neural Machine Translation: The Architecture MattersPreprint
Costa‐jussà, M. R., Escolano, C., Basta, C., Ferrando, J., Batlle, R., & Kharitonova, K. (2020) · arXiv (Cornell University) · Cited by 7 · see citing works
How True is GPT-2? An Empirical Analysis of Intersectional Occupational Biases.Preprint
Kirk, H. R., Jun, Y., Haider, I., Benussi, E., Volpin, F., Dreyer, F. A., Shtedritski, A., & Asano, Y. M. (2021) · arXiv (Cornell University) · Cited by 5 · see citing works
The Truth is Out There: Investigating Conspiracy Theories in Text Generation.Preprint
Levy, S., Saxon, M., & Wang, W. Y. (2021) · arXiv (Cornell University) · Cited by 4 · see citing works
Dialect Diversity in Text Summarization on TwitterPreprint
Celis, L. E., & Keswani, V. (2020) · arXiv (Cornell University) · Cited by 2 · see citing works
Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP
Schick, T., Udupa, S., & Schütze, H. (2021) · arXiv · doi:10.48550/arxiv.2103.00453 · Cited by 1 · see citing works
First the worst: Finding better gender translations during beam searchPreprint
Saunders, D., Sallis, R., & Byrne, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.07429
Dialect Diversity in Text Summarization on TwitterPreprint
Keswani, V., & Celis, L. E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.07860
Flagged first, then by citations
Research works printed but not linked: 9
- Saunders and Byrne (2020);
- Saunders et al. (2020); Kocmi et al. (2020); Costa-jussà and de Jorge (2020); Costa-jussà et al. (2020);
- Choubey et al. (2021); Saunders et al. (2021); Tomalin et al. (2021) Re-writing References
- Solon Barocas, Kate Crawford, Aaron Shapiro, and Hanna Wallach. 2017. The problem with bias: Al- locative versus representational harms in machine learning. In 9th Annual Conference of the Special Interest Group for Computing, Information and So- ciety.
- Kate Crawford. 2017. The trouble with bias. Keynote at NeurIPS.
- Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hi- erarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889-898.
- Anna Farkas and Renáta Németh. 2020. How to mea- sure gender bias in machine translation: Optimal translators, multiple reference points. arXiv preprint arXiv:2011.06445. · arXiv
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
- Alex Wang and Kyunghyun Cho. 2019. BERT has a mouth, and it must speak: BERT as a Markov random field language model. In Proceedings of the Workshop on Methods for Optimizing and Eval- uating Neural Language Generation, pages 30-36, Minneapolis, Minnesota. Association for Computa- tional Linguistics. · DOI
Cited by
Sorted by citations
A Survey on Evaluation of Large Language Models
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3641289 · Cited by 2,842 · see citing works
“, gender, race, religion, occupation, and ideology) [166].”
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · see citing works
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 665 · see citing works
Can Large Language Models Transform Computational Social Science?
Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., & Yang, D. (2024) · Computational Linguistics · doi:10.1162/coli_a_00502 · Cited by 585 · see citing works
Khan, W., Daud, A. Y., Khan, K., Muhammad, S., & Haq, R. (2023) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100026 · Cited by 238 · see citing works
Jo, E., Epstein, D. A., Jung, H., & Kim, Y.-H. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581503 · Cited by 174 · see citing works
"I wouldn't say offensive but...": Disability-Centered Perspectives on Large Language Models
Gadiraju, V., Kane, S., Dev, S., Taylor, A., Wang, D., Denton, R., & Brewer, R. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3593989 · Cited by 155 · see citing works
“Kelly is a Warm Person, Joseph is a Role Model”: Gender Biases in LLM-Generated Reference Letters
Wan, Y., Pu, G., Sun, J., Garimella, A., Chang, K.-W., & Peng, N. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.243 · Cited by 175 · see citing works
Self-citationHarms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies
Dev, S., Monajatipoor, M., Ovalle, A., Subramonian, A., Phillips, J. M., & Chang, K.-W. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.150 · Cited by 135 · see citing works
Self-citationZhang, R., Li, H., Meng, H., Zhan, J., Gan, H., & Lee, Y.-C. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713429 · Cited by 123 · see citing works
“These systems can cause a variety of harms, from emotional distress to privacy violations to the spread of misinformation and reinforcement of harmful stereotypes [86, 108].”
Addressing 6 challenges in generative AI for digital health: A scoping review
Templin, T., Perez, M. W., Sylvia, S., Leek, J. T., & Sinnott-Armstrong, N. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000503 · Cited by 115 · see citing works
Language Model Behavior: A Comprehensive Survey
Chang, T. A., & Bergen, B. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00492 · Cited by 105 · see citing works
History, development, and principles of large language models: an introductory survey
Wang, Z., Chu, Z., Doan, T. V., Ni, S., Yang, M., & Zhang, W. (2025) · AI and Ethics · doi:10.1007/s43681-024-00583-7 · Cited by 109 · see citing works
The Security of Using Large Language Models: A Survey with Emphasis on ChatGPT
Zhou, W., Zhu, X., Han, Q., Li, L., Chen, X., Wen, S., & Xiang, Y. (2024) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2024.124983 · Cited by 101 · see citing works
The Sociodemographic Biases in Machine Learning Algorithms: A Biomedical Informatics Perspective
Franklin, G., Stephens, R., Piracha, M., Tiosano, S., Lehouillier, F., Koppel, R., & Elkin, P. (2024) · Life · doi:10.3390/life14060652 · Cited by 76 · see citing works
“I’m sorry to hear that”: Finding new biases in language models with a holistic descriptor dataset
Smith, E. M., Hall, M., Kambadur, M., Presani, E., & Williams, A. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.625 · Cited by 97 · see citing works
Liu, Y., Li, D., Wang, K., Xiong, Z., Shi, F., Wang, J., Li, B., & Hang, B. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103809 · Cited by 81 · see citing works
Cultural Bias in Large Language Models: A Comprehensive Analysis and Mitigation Strategies
Liu, Z. (2025) · Journal of Transcultural Communication · doi:10.1515/jtc-2023-0019 · Cited by 77 · see citing works
Czarnowska, P., Vyas, Y., & Shah, K. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00425 · Cited by 72 · see citing works
Quantifying and alleviating political bias in language models
Liu, R., Jia, C., Wei, J., Xu, G., & Vosoughi, S. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2021.103654 · Cited by 68 · see citing works
You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings
Talat, Z., Névéol, A., Biderman, S., Clinciu, M., Dey, M., Longpre, S., Luccioni, A. S., Masoud, M., Mitchell, M., Radev, D., et al. (2022) · doi:10.18653/v1/2022.bigscience-1.3 · Cited by 61 · see citing works
FLUTE: Figurative Language Understanding through Textual Explanations
Chakrabarty, T., Saakyan, A., Ghosh, D., & Muresan, S. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.481 · Cited by 45 · see citing works
Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models
Ahia, O., Kumar, S., Gonen, H., Kasai, J., Mortensen, D. R., Smith, N. A., & Tsvetkov, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.614 · Cited by 63 · see citing works
Ovalle, A., Goyal, P., Dhamala, J., Jaggers, Z., Chang, K.-W., Galstyan, A., Zemel, R., & Gupta, R. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594078 · Cited by 43 · see citing works
Self-citationLanguage Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey
Kumar, S., Balachandran, V., Njoo, L., Anastasopoulos, A., & Tsvetkov, Y. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.241 · Cited by 38 · see citing works
Mirowski, P., Love, J., Mathewson, K., & Mohamed, S. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658993 · Cited by 37 · see citing works
Towards Identifying Social Bias in Dialog Systems: Framework, Dataset, and Benchmark
Zhou, J., Deng, J., Mi, F., Li, Y., Wang, Y., Huang, M., Jiang, X., Liu, Q., & Meng, H. M. L. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.262 · Cited by 31 · see citing works
Pipelines for Social Bias Testing of Large Language Models
Nozza, D., Bianchi, F., & Hovy, D. (2022) · doi:10.18653/v1/2022.bigscience-1.6 · Cited by 29 · see citing works
Han, X., Kumar, S., & Tsvetkov, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.647 · Cited by 41 · see citing works
AI-Generated Text Detector for Arabic Language Using Encoder-Based Transformer Architecture
Alshammari, H., El-Sayed, A., & Elleithy, K. M. (2024) · Big Data and Cognitive Computing · doi:10.3390/bdcc8030032 · Cited by 26 · see citing works
Evaluation of African American Language Bias in Natural Language Generation
Deas, N., Grieser, J. A., Kleiner, S., Patton, D. U., Turcan, E., & McKeown, K. R. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.421 · Cited by 29 · see citing works
Nguyen, H., Nguyen, V., López-Fierro, S., Ludovise, S., & Santagata, R. (2024) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3657604.3662033 · Cited by 23 · see citing works
Inspecting the Geographical Representativeness of Images from Text-to-Image Models
Basu, A., Babu, R. V., & Pruthi, D. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00474 · Cited by 21 · see citing works
Nguyen, H., Nguyen, V., Ludovise, S., & Santagata, R. (2024) · Learning Media and Technology · doi:10.1080/17439884.2024.2435834 · Cited by 19 · see citing works
Yu, S., Androsov, A., & Yan, H. (2025) · Computers & Education · doi:10.1016/j.compedu.2025.105307 · Cited by 20 · see citing works
Challenging Fairness: A Comprehensive Exploration of Bias in LLM-Based Recommendations
Sakib, S. K., & Das, A. B. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825082 · Cited by 17 · see citing works
Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications
Zhou, K., Blodgett, S. L., Trischler, A., Daumé, H., Suleman, K., & Olteanu, A. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.24 · Cited by 16 · see citing works
Lu, H., Lin, Y., Li, Z., Yiu, M. L., Gao, Y., & Uddin, S. (2025) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2025.103216 · Cited by 16 · see citing works
Li, Y., Du, M., Song, R., Wang, X., Sun, M., & Wang, Y. (2024) · Artificial Intelligence · doi:10.1016/j.artint.2024.104143 · Cited by 16 · see citing works
Toward Robust Arabic AI-Generated Text Detection: Tackling Diacritics Challenges
Alshammari, H., & Elleithy, K. M. (2024) · Information · doi:10.3390/info15070419 · Cited by 15 · see citing works
Phutane, M., Seelam, A., & Vashistha, A. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732128 · Cited by 14 · see citing works
Liu, H., Cao, Y., Wu, X., Qiu, C., Gu, J., Liu, M., & Hershcovich, D. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104099 · Cited by 14 · see citing works
Exploring biases related to the use of large language models in a multilingual depression corpus
Pérez-Toro, P. A., Dineley, J., Iniesta, R., Zhang, Y., Matcham, F., Siddi, S., Lamers, F., Haro, J. M., Penninx, B. W. J. H., Folarin, A., et al. (2025) · Scientific Reports · doi:10.1038/s41598-025-19980-x · Cited by 16 · see citing works
LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting
Ramos, R., Martins, B., & Elliott, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.104 · Cited by 17 · see citing works
Contrastive Conditioning for Assessing Disambiguation in MT: A Case Study of Distilled Bias
Vamvas, J., & Sennrich, R. (2021) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.803 · Cited by 27 · see citing works
CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Models
Zhao, J., Fang, M., Shi, Z., Li, Y., Chen, L., & Pechenizkiy, M. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.757 · Cited by 14 · see citing works
Detecting Gender Bias in Transformer-based Models: A Case Study on BERTPreprint
Li, B., Peng, H., Sainju, R., Yang, J., Yang, L., Liang, Y., Jiang, W., Wang, B., Liu, H., & Ding, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.15733 · Cited by 15 · see citing works
Which Stereotypes Are Moderated and Under-Moderated in Search Engine Autocompletion?
Leidinger, A., & Rogers, R. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594062 · Cited by 12 · see citing works
Exploring Social Biases of Large Language Models in a College Artificial Intelligence Course
Kolisko, S., & Anderson, C. J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i13.26879 · Cited by 11 · see citing works
Conversational User Interfaces and Agents
Biancofiore, G. M., Palma, D. D., Pomo, C., Narducci, F., & Noia, T. D. (2025) · Human-computer interaction series · doi:10.1007/978-3-031-61375-3_4 · Cited by 12 · see citing works
Budnarowski, D., Jereczek, D., Detka, K., & Wieczorek, I. (2025) · Applied Sciences · doi:10.3390/app15169067 · Cited by 20 · see citing works
Abrar, A., Oeshy, N. T., Kabir, M., & Ananiadou, S. (2025) · AI & Society · doi:10.1007/s00146-025-02721-z · Cited by 15 · see citing works
Culturally Aware Natural Language Inference
Huang, J., & Yang, D. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.509 · Cited by 10 · see citing works
Evaluating Synthetic Data Generation from User Generated Text
Chim, J., Ive, J., & Liakata, M. (2024) · Computational Linguistics · doi:10.1162/coli_a_00540 · Cited by 10 · see citing works
Exploiting Biased Models to De-bias Text: A Gender-Fair Rewriting Model
Amrhein, C., Schottmann, F., Sennrich, R., & Läubli, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.246 · Cited by 10 · see citing works
Thaichana, P., Oo, M. Z., Thorup, G. L., Chansakaow, C., Arworn, S., & Rerkasem, K. (2025) · The International Journal of Lower Extremity Wounds · doi:10.1177/15347346241312814 · Cited by 13 · see citing works
Embracing large language model (LLM) technologies in hydrology research
Ma, Z., Peng, B., Yue, Z., Zeng, H., Pan, M., Wu, X., Yang, J., Mai, L., & Guan, K. (2025) · Environmental Research Water · doi:10.1088/3033-4942/addd43 · Cited by 10 · see citing works
Xiao, Z., Zhang, S., Lai, V. W.-M., & Liao, Q. V. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.676 · Cited by 17 · see citing works
A Framework for Auditing Chatbots for Dialect-Based Quality-of-Service Harms
Harvey, E., Kizilcec, R. F., & Koenecke, A. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732137 · Cited by 8 · see citing works
Towards Robust NLG Bias Evaluation with Syntactically-diverse Prompts
Aggarwal, A., Sun, J., & Peng, N. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.445 · Cited by 8 · see citing works
Self-citationConditional Supervised Contrastive Learning for Fair Text Classification
Chi, J., Shand, W., Yu, Y., Chang, K.-W., Zhao, H., & Tian, Y. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.199 · Cited by 8 · see citing works
Self-citationSpeciesist language and nonhuman animal bias in English Masked Language Models
Takeshita, M., Rzepka, R., & Araki, K. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103050 · Cited by 10 · see citing works
The tail wagging the dog: Dataset construction biases of social bias benchmarks
Selvam, N., Dev, S., Khashabi, D., Khot, T., & Chang, K.-W. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2023.acl-short.118 · Cited by 11 · see citing works
Self-citationControlling Bias Exposure for Fair Interpretable Predictions
He, Z., Wang, Y., McAuley, J., & Majumder, B. P. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.431 · Cited by 8 · see citing works
Harrag, F., Mahdadi, C., & Ziad, A. N. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-37249-0_13 · Cited by 7 · see citing works
S-Eval: Towards Automated and Comprehensive Safety Evaluation for Large Language Models
Yuan, X., Li, J., Wang, D., Chen, Y., Mao, X., Huang, L., Chen, J., Xue, H., Liu, X., Wang, W., et al. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728971 · Cited by 27 · see citing works
Style Pooling: Automatic Text Style Obfuscation for Improved Classification Fairness
Mireshghallah, F., & Berg-Kirkpatrick, T. (2021) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.152 · Cited by 20 · see citing works
Towards Better Detection of Biased Language with Scarce, Noisy, and Biased Annotations
Li, Z., Lu, Z., & Yin, M. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534142 · Cited by 7 · see citing works
A Multi-dimensional study on Bias in Vision-Language models
Ruggeri, G., & Nozza, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.403 · Cited by 7 · see citing works
DrBERT: A Robust Pre-trained Model in French for Biomedical and Clinical domainsPreprint
Labrak, Y., Bazoge, A., Dufour, R., Rouvier, M., Morin, E., Daille, B., & Gourraud, P. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.04.03.535368 · Cited by 7 · see citing works
An Analysis of The Effects of Decoding Algorithms on Fairness in Open-Ended Language Generation
Dhamala, J., Kumar, V., Gupta, R., Chang, K.-W., & Galstyan, A. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10022722 · Cited by 7 · see citing works
Self-citationCONSISTENT: Open-Ended Question Generation From News Articles
Chakrabarty, T., Lewis, J., & Muresan, S. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.517 · Cited by 6 · see citing works
Explicit Syntactic Guidance for Neural Text Generation
Li, Y., Cui, L., Yan, J., Yin, Y., Bi, W., Shi, S., & Zhang, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.788 · Cited by 8 · see citing works
Lee, H., Hong, S., Park, J., Kim, T., Cha, M., Choi, Y., Kim, B., Kim, G., Lee, E., Lim, Y., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.370 · Cited by 5 · see citing works
Detecting intersectionality in NER models: A data-driven approach
Lassen, I. M. S., Almasi, M., Enevoldsen, K., & Kristensen‐McLachlan, R. D. (2023) · Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature (LaTeCH-CLfL) · doi:10.18653/v1/2023.latechclfl-1.13 · Cited by 5 · see citing works
Ersoy, A., Vizcarra, G., Mayeesha, T. T., & Müller, B. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.175 · Cited by 5 · see citing works
Robust Natural Language Understanding with Residual Attention Debiasing
Wang, F., Huang, J. Y., Yan, T., Zhou, W., & Chen, M. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.32 · Cited by 5 · see citing works
What do Bias Measures MeasurePreprint
Dev, S., Sheng, E., Zhao, J., Sun, J., Hou, Y., Sanseverino, M., Kim, J., Peng, N., & Chang, K.-W. (2021) · arXiv (Cornell University) · Cited by 6 · see citing works
Self-citationThe State of Profanity Obfuscation in Natural Language Processing Scientific Publications
Nozza, D., & Hovy, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.240 · Cited by 4 · see citing works
An Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language Models
Hosseini, S., Palangi, H., & Awadallah, A. H. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.11 · Cited by 7 · see citing works
Hi Guys or Hi Folks? Benchmarking Gender-Neutral Machine Translation with the GeNTE Corpus
Piergentili, A., Savoldi, B., Fucci, D., Negri, M., & Bentivogli, L. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.873 · Cited by 6 · see citing works
AUTOMATIC QUESTION GENERATION THROUGH WORD VECTOR SYNCHRONIZATION USING LAMMA
Khandait, K., Bhura, S. A., & Asole, S. S. (2022) · Indian Journal of Computer Science and Engineering · doi:10.21817/indjcse/2022/v13i4/221304046 · Cited by 4 · see citing works
Behavior Cloned Transformers are Neurosymbolic Reasoners
Wang, R., Jansen, P., Côté, M.-A., & Ammanabrolu, P. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.204 · Cited by 4 · see citing works
A Survey of Hallucination in Large Language Models
何, 金. (2026) · Artificial Intelligence and Robotics Research · doi:10.12677/airr.2026.151016 · Cited by 3 · see citing works
Societal Implications of Recommendation Systems: A Technical Perspective
Gonçalves-Sá, J., & Pinheiro, F. L. (2023) · Law, governance and technology series · doi:10.1007/978-3-031-41264-6_3 · Cited by 3 · see citing works
Bilingual generated text detection through semantic and statistical analysis
Min, C., Zhang, R., & Liu, J. (2025) · Intelligent Data Analysis · doi:10.1177/1088467x241307192 · Cited by 3 · see citing works
Controlled Language Generation for Language Learning Items
Stowe, K., Ghosh, D., & Zhao, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-industry.30 · Cited by 3 · see citing works
Saliency Guided Debiasing: Detecting and mitigating biases in LMs using feature attribution
Joshi, R. K., Chatterjee, A., & Ekbal, A. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126851 · Cited by 3 · see citing works
Towards comparable ratings: Exploring bias in German physician reviews
Kersting, J., Maoro, F., & Geierhos, M. (2023) · Data & Knowledge Engineering · doi:10.1016/j.datak.2023.102235 · Cited by 3 · see citing works
Geographical Erasure in Language Generation
Schwöbel, P., Golebiowski, J., Donini, M., Archambeau, C., & Pruthi, D. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.823 · Cited by 5 · see citing works
A multifaceted approach to detect gender biases in Natural Language Generation
Consuegra-Ayala, J. P., Martínez-Murillo, I., Lloret, E., Moreda, P., & Palomar, M. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112367 · Cited by 1 · see citing works
Are Male Candidates Better than Females? Debiasing BERT Resume Retrieval System
Zhang, S., Li, P., & Cai, Z. (2022) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc53654.2022.9945184 · Cited by 3 · see citing works
Human-Centered AI for Migrant Integration Through LLM and RAG Optimization
Castellanos–Nieves, D., & García-Forte, L. (2024) · Applied Sciences · doi:10.3390/app15010325 · Cited by 2 · see citing works
Guiding the Next Technological Revolution
Gutierrez, R. (2023) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-0368-9.ch011 · Cited by 3 · see citing works
Entity-Based Evaluation of Political Bias in Automatic Summarization
Zhou, K., & Tan, C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.696 · Cited by 5 · see citing works
Understanding and Mitigating Gender Bias in Information Retrieval Systems
Seyedsalehi, S., Bigdeli, A., Arabzadeh, N., AlMousawi, B., Marshall, Z., Zihayat, M., & Ebrahim Bagheri, A. (2025) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000103 · Cited by 3 · see citing works
Evaluation of Quality of Slovak Language Use in LLMS
Dobeš, M. (2025) · Acta Electrotechnica et Informatica · doi:10.2478/aei-2025-0004 · Cited by 2 · see citing works
Ensemble Transfer Learning for Multilingual Coreference Resolution
Lai, T., & Ji, H. (2023) · Workshop on Computational Approaches to Discourse (CODI) · doi:10.18653/v1/2023.codi-1.3 · Cited by 3 · see citing works
COFFEE: Counterfactual Fairness for Personalized Text Generation in Explainable Recommendation
Wang, N., Wang, Q., Wang, Y.-C., Sanjabi, M., Liu, J., Firooz, H., Wang, H., & Nie, S. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.819 · Cited by 2 · see citing works
Evaluating the Effects of Prompt Perturbation on Bias and Hallucination in Large Language Models
Yousefi, M., Shahi, A., Sharifi, M., Romera, A. J. J., Hoermann, S., & Piumsomboon, T. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-6588-4_25 · Cited by 1 · see citing works
- Authors
- Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N.
- Affiliations
- University of Southern California (US)
- University of California, Los Angeles (US)
- Venue
- Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
- Published
- 2021
- Pages
- 4275-4293
- OpenAlex
- W3174685870
- Cited by
- 131
- References
- 99
- Citations by year
- 2026: 18 · 2025: 28 · 2024: 18 · 2023: 43 · 2022: 16 · 2021: 8
- FWCI
- 19.25
- Type
- conference paper
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- language generation, algorithmic fairness, algorithmic bias, natural language generation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review