The DOI matches its Crossref record; 111 of the 112 references it cites were found in a registry or an index.
Societal Biases in Language Generation: Progress and Challenges
Open accessAbstract
Emily Sheng, Kai-Wei Chang, Prem Natarajan, Nanyun Peng. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“LLMs have been found to internalize, spread, and potentially magnify harmful information existing in the crawled training corpora, usually, toxic languages, like offensiveness, hate speech, and insults [53], as well as social biases like stereotypes towards people with a particular demographic identity ( e.g. , gender, race, religion, occupation, and ideology) [175].”
From A Survey on Evaluation of Large Language Models (2024) · cites this paper as [175]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1×
“Furthermore, by filtering out all words deemed “bad”, Bender et al. (2021) warns that we remove the discourse of marginalized populations.”
BackgroundXLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687
VADER: a parsimonious rule-based model for sentiment analysis of social media text
Hutto, C., & Gilbert, E. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14550 · Cited by 6,087
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469
The Curious Case of Neural Text Degeneration
Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09751 · Cited by 1,304
Hierarchical Neural Story Generation
Fan, A., Lewis, M., & Dauphin, Y. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1082 · Cited by 1,266
Hierarchical Neural Story Generation
Fan, A., Lewis, M., & Dauphin, Y. (2018) · arXiv · doi:10.48550/arxiv.1805.04833 · Cited by 1,242
DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation
Zhang, Y., Sun, S., Galley, M., Chen, Y.-C., Brockett, C., Gao, X., Gao, J., Liu, J., & Dolan, B. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.18653/v1/2020.acl-demos.30 · Cited by 1,076
Language (Technology) is Power: A Critical Survey of “Bias” in NLP
Blodgett, S. L., Barocas, S., Daumé III, H., & Wallach, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.485 · Cited by 1,050 · cited 3×
“We survey detrimental representational9 and allocational10 impacts (Crawford, 2017; Barocas et al., 2017; Blodgett et al., 2020) used to motivate”
BackgroundBender, E. M., & Friedman, B. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00041 · Cited by 854
Extracting Training Data from Large Language Models
Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07805 · Cited by 727
Universal Adversarial Triggers for Attacking and Analyzing NLP
Wallace, E., Feng, S., Kandpal, N., Gardner, M., & Singh, S. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1221 · Cited by 642
Data and its (dis)contents: A survey of dataset development and use in machine learning research
Paullada, A., Raji, I. D., Bender, E. M., Denton, E., & Hanna, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100336 · Cited by 548 · cited 1×
“Paullada et al. (2020), Bender and Fried-man (2018), and Gebru et al. (2018) provide more comprehensive surveys and frameworks that focus on aspects of data creation and management that could lead to biases, and we refer readers to their works for more discussion.”
BackgroundThe Woman Worked as a Babysitter: On Biases in Language Generation
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1339 · Cited by 537 · cited 10×
“Language models are the core components for many NLG and NLU tasks, and this task enables directly quantifying biases in large, pre-trained language models (Bordia and Bowman, 2019; Sheng et al., 2019; Solaiman et al., 2019; Brown et al., 2020).”
BackgroundMethodInfluentialSelf-citationUnequal Representation and Gender Stereotypes in Image Search Results for Occupations
Kay, M., Matuszek, C., & Munson, S. A. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702520 · Cited by 509
Plug and Play Language Models: A Simple Approach to Controlled Text Generation
Dathathri, S., Madotto, A., Lan, J., Hung, J., Frank, E., Molino, P., Yosinski, J., & Liu, R. (2019) · arXiv · doi:10.48550/arxiv.1912.02164 · Cited by 508
Evaluating Gender Bias in Machine Translation
Stanovsky, G., Smith, N. A., & Zettlemoyer, L. (2019) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1164 · Cited by 492
Mitigating Gender Bias in Natural Language Processing: Literature Review
Sun, T., Gaut, A., Tang, S., Huang, Y., ElSherief, M., Zhao, J., Mirza, D., Belding, E., Chang, K.-W., & Wang, W. Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1159 · Cited by 492
Self-citationPersistent Anti-Muslim Bias in Large Language Models
Abid, A., Farooqi, M., & Zou, J. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462624 · Cited by 476
(2020) · arXiv (Cornell University) · Cited by 461
Learning Gender-Neutral Word Embeddings
Zhao, J., Zhou, Y., Li, Z., Wang, W. W., & Chang, K.-W. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1521 · Cited by 426
Self-citationAssessing gender bias in machine translation: a case study with Google Translate
Prates, M. O. R., Avelar, P. H., & Lamb, L. C. (2020) · Neural Computing and Applications · doi:10.1007/s00521-019-04144-6 · Cited by 410
Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems
Kiritchenko, S., & Mohammad, S. (2018) · Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/s18-2005 · Cited by 384
Reducing Gender Bias in Abusive Language Detection
Park, J. H., Shin, J., & Fung, P. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1302 · Cited by 323
Release Strategies and the Social Impacts of Language Models
Solaiman, I., Brundage, M., Clark, J., Askell, A., Herbert-Voss, A., Wu, J., Radford, A., Krueger, G., Kim, J. W., Kreps, S., et al. (2019) · arXiv · doi:10.48550/arxiv.1908.09203 · Cited by 292
Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology
Zmigrod, R., Mielke, S. J., Wallach, H., & Cotterell, R. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1161 · Cited by 290
Gender and Representation Bias in GPT-3 Generated Stories
Lucy, L., & Bamman, D. (2021) · Proceedings of the Third Workshop on Narrative Understanding · doi:10.18653/v1/2021.nuse-1.5 · Cited by 287
Gender Bias in Neural Natural Language Processing
Lu, K., Mardziel, P., Wu, F., Amancharla, P., & Datta, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-62077-6_14 · Cited by 284
Fairness Without Demographics in Repeated Loss MinimizationPreprint
Hashimoto, T., Srivastava, M., Namkoong, H., & Liang, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.08010 · Cited by 278
Kirk, H. R., Jun, Y., Volpin, F., Iqbal, H., Benussi, E., Dreyer, F., Shtedritski, A., & Asano, Y. M. (2021) · Neural Information Processing Systems · doi:10.48550/arxiv.2102.04130 · Cited by 251
BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation
Dhamala, J., Sun, T., Kumar, V., Krishna, S., Pruksachatkun, Y., Chang, K.-W., & Gupta, R. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445924 · Cited by 244
Self-citationPredictive biases in natural language processing models: A conceptual framework and overview
Shah, D. S., Schwartz, H. A., & Hovy, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.468 · Cited by 241
Gender Bias in Machine Translation
Savoldi, B., Gaido, M., Bentivogli, L., Negri, M., & Turchi, M. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00401 · Cited by 229
Re-imagining Algorithmic Fairness in India and Beyond
Sambasivan, N., Arnesen, E., Hutchinson, B., Doshi, T., & Prabhakaran, V. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445896 · Cited by 228 · cited 1×
“Bias Evaluations It is difficult to evaluate societal biases in NLG tasks because NLG can be open-domain, and there are many different notions of biases from various backgrounds and cultures (Sambasivan et al., 2021).”
BackgroundSelf-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP
Schick, T., Udupa, S., & Schütze, H. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00434 · Cited by 227
GeDi: Generative Discriminator Guided Sequence Generation
Krause, B., Gotmare, A. D., McCann, B., Keskar, N. S., Joty, S., Socher, R., & Rajani, N. F. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.424 · Cited by 220
Identifying and Reducing Gender Bias in Word-Level Language Models
Bordia, S., & Bowman, S. R. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop · doi:10.18653/v1/n19-3002 · Cited by 219
Getting Gender Right in Neural Machine Translation
Vanmassenhove, E., Hardmeier, C., & Way, A. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1334 · Cited by 193
Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation
Dinan, E., Fan, A., Williams, A., Urbanek, J., Kiela, D., & Weston, J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.656 · Cited by 190
Equalizing Gender Bias in Neural Machine Translation with Word Embeddings Techniques
Font, J. E., & Costa-jussà, M. (2019) · Proceedings of the First Workshop on Gender Bias in Natural Language Processing · doi:10.18653/v1/w19-3821 · Cited by 187
Automatically Neutralizing Subjective Bias in Text
Pryzant, R., Martinez, R. D., Dass, N., Kurohashi, S., Jurafsky, D., & Yang, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5385 · Cited by 171
Ethical Challenges in Data-Driven Dialogue Systems
Henderson, P., Sinha, K., Gontier, N., Ke, N. R., Fried, G., Lowe, R., & Pineau, J. (2018) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3278721.3278777 · Cited by 166
GeDi: Generative Discriminator Guided Sequence GenerationPreprint
Krause, B., Gotmare, A. D., McCann, B., Keskar, N. S., Joty, S., Socher, R., & Rajani, N. F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.06367 · Cited by 164
Distilling Knowledge Learned in BERT for Text Generation
Chen, Y.-C., Gan, Z., Yu, C., Liu, J., & Liu, J. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.705 · Cited by 151
Decolonising Speech and Language Technology
Bird, S. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.313 · Cited by 151
(2020) · neural information processing systems · Cited by 142
Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models
Tamkin, A., Brundage, M., Clark, J., & Ganguli, D. (2021) · arXiv · doi:10.48550/arxiv.2102.02503 · Cited by 135
Confronting Abusive Language Online: A Survey from the Ethical and Human Rights Perspective
Kiritchenko, S., Nejadgholi, I., & Fraser, K. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12590 · Cited by 126
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
Huang, P.-S., Zhang, H., Jiang, R., Stanforth, R., Welbl, J., Rae, J., Maini, V., Yogatama, D., & Kohli, P. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.7 · Cited by 124
Towards Controllable Biases in Language Generation
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.291 · Cited by 123
Self-citationMulti-Dimensional Gender Bias Classification
Dinan, E., Fan, A., Wu, L., Weston, J., Kiela, D., & Williams, A. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.23 · Cited by 117
Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem
Saunders, D., & Byrne, B. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.690 · Cited by 100
HONEST: Measuring Hurtful Sentence Completion in Language Models
Nozza, D., Bianchi, F., & Hovy, D. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.191 · Cited by 95
Does Gender Matter? Towards Fairness in Dialogue Systems
Liu, H., Dacon, J., Fan, W., Liu, H., Liu, Z., & Tang, J. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.390 · Cited by 81
Mitigating Gender Bias in Machine Translation with Target Gender AnnotationsPreprint
Stafanovičs, A., Pinnis, M., & Bergmanis, T. (2020) · Conference on Machine Translation · doi:10.48550/arxiv.2010.06203 · Cited by 71
PowerTransformer: Unsupervised Controllable Revision for Biased Language Correction
Ma, X., Sap, M., Rashkin, H., & Choi, Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.602 · Cited by 70
They, Them, Theirs: Rewriting with Gender-Neutral EnglishPreprint
Sun, T., Webster, K., Shah, A., Wang, W. Y., & Johnson, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.06788 · Cited by 65
Reducing Gender Bias in Word-Level Language Models with a Gender-Equalizing Loss Function
Qian, Y., Muaz, U., Zhang, B., & Hyun, J. W. (2019) · Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (ACL SRW) · doi:10.18653/v1/p19-2031 · Cited by 63
Investigating African-American Vernacular English in Transformer-Based Text Generation
Groenwold, S., Ou, L., Parekh, A., Honnavalli, S., Levy, S., Mirza, D., & Wang, W. Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.473 · Cited by 59
Tomalin, M., Byrne, B., Concannon, S. J., Saunders, D., & Ullmann, S. (2021) · Ethics and Information Technology · doi:10.1007/s10676-021-09583-1 · Cited by 58
Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learning
Liu, H., Wang, W., Wang, Y., Liu, H., Liu, Z., & Tang, J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.64 · Cited by 57
On Measuring Gender Bias in Translation of Gender-neutral Pronouns
Cho, W. I., Kim, J. W., Kim, S., & Kim, N. S. (2019) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/w19-3824 · Cited by 56
“You are grounded!”: Latent Name Artifacts in Pre-trained Language Models
Shwartz, V., Rudinger, R., & Tafjord, O. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.556 · Cited by 52
“You Sound Just Like Your Father” Commercial Machine Translation Systems Include Stylistic Biases
Hovy, D., Bianchi, F., & Fornaciari, T. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.154 · Cited by 52
“Nice Try, Kiddo”: Investigating Ad Hominems in Dialogue Responses
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.60 · Cited by 48
Self-citationSilva, A., Tambwekar, P., & Gombolay, M. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.189 · Cited by 46
Filling Gender & Number Gaps in Neural Machine Translation with Black-box Context Injection
Moryossef, A., Aharoni, R., & Goldberg, Y. (2019) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/w19-3807 · Cited by 43
Neural Machine Translation Doesn't Translate Gender Coreference Right Unless You Make ItPreprint
Saunders, D., Sallis, R., & Byrne, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.05332 · Cited by 41
Discovering and Categorising Language Biases in Reddit
Ferrer, X., van Nuenen, T., Such, J. M., & Criado, N. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v15i1.18048 · Cited by 36
Automatic Gender Identification and Reinflection in Arabic
Habash, N. Y., Bouamor, H., & Chung, C. (2019) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/w19-3822 · Cited by 35
Mitigating Gender Bias in Machine Translation with Target Gender Annotations
Stafanovičs, A., Bergmanis, T., & Pinnis, M. (2020) · Conference on Machine Translation (WMT) · doi:10.18653/v1/2020.wmt-1.73 · Cited by 35
Gender aware spoken language translation applied to English-Arabic
ElAraby, M., Tawfik, A. Y., Khaled, M., Hassan, H., & Osama, A. (2018) · International Conference on Natural Language and Speech Processing (ICNLSP) · doi:10.1109/icnlsp.2018.8374387 · Cited by 30
Automatically Identifying Gender Issues in Machine Translation using Perturbations
Gonen, H., & Webster, K. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.180 · Cited by 28
Reducing Non-Normative Text Generation from Language Models
Peng, X., Li, S., Frazier, S., & Riedl, M. O. (2020) · International Conference on Natural Language Generation · doi:10.18653/v1/2020.inlg-1.43 · Cited by 28
Investigating Failures of Automatic Translation in the Case of Unambiguous GenderPreprint
Renduchintala, A., & Williams, A. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2104.07838 · Cited by 28
Investigating Societal Biases in a Poetry Composition SystemPreprint
Sheng, E., & Uthus, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.02686 · Cited by 26
Self-citationDefining and Evaluating Fair Natural Language Generation
Yeo, C., & Chen, A. (2020) · Widening Natural Language Processing Workshop (WiNLP) · doi:10.18653/v1/2020.winlp-1.27 · Cited by 24
Fine-tuning neural machine translation on gender-balanced datasets
Costa‐jussà, M. R., & Sánchez, A. D. J. (2020) · Cited by 24
Towards Cross-Lingual Generalization of Translation Gender Bias
Cho, W. I., Kim, J., Yang, J., & Kim, N. S. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445907 · Cited by 24
Revealing Persona Biases in Dialogue SystemsPreprint
Sheng, E., Arnold, J., Yu, Z., Chang, K.-W., & Peng, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08728 · Cited by 21
Self-citationDecoding and Diversity in Machine TranslationPreprint
Roberts, N. P., Liang, D., Neubig, G., & Lipton, Z. C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.13477 · Cited by 20
Farkas, A., & Németh, R. (2021) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2021.100239 · Cited by 20
Gender-Aware Reinflectionusing Linguistically Enhanced Neural Models
International Committee for Computational Linguistics 2020, Alhafni, B., Bouamor, H., & Habash, N. (2020) · doi:10.48448/6nn5-6v05 · Cited by 20
Basta, C., Costa‐jussà, M. R., & Fonollosa, J. A. R. (2020) · Widening Natural Language Processing Workshop (WiNLP) · doi:10.18653/v1/2020.winlp-1.25 · Cited by 19
First the Worst: Finding Better Gender Translations During Beam Search
Saunders, D., Sallis, R., & Byrne, B. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.301 · Cited by 18
Curry, A. C., Robertson, J., & Rieser, V. (2020) · Cited by 17
Persistent Anti-Muslim Bias in Large Language ModelsPreprint
Abid, A., Farooqi, M., & Zou, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.05783 · Cited by 17
Discovering and Categorising Language Biases in RedditPreprint
Ferrer, X., van Nuenen, T., Such, J. M., & Criado, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.02754 · Cited by 15
Gender Coreference and Bias Evaluation at WMT 2020
Kocmi, T., Limisiewicz, T., & Stanovsky, G. (2020) · Conference on Machine Translation (WMT) · doi:10.18653/v1/2020.wmt-1.39 · Cited by 14
Improving Gender Translation Accuracy with Filtered Self-TrainingPreprint
Choubey, P. K., Currey, A., Mathur, P., & Dinu, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.07695 · Cited by 10
On-the-Fly Controlled Text Generation with Experts and Anti-Experts.Preprint
Liu, A., Sap, M., Lu, X., Swayamdipta, S., Bhagavatula, C., Smith, N. A., & Choi, Y. (2021) · arXiv (Cornell University) · Cited by 9
Gender Bias in Multilingual Neural Machine Translation: The Architecture MattersPreprint
Costa‐jussà, M. R., Escolano, C., Basta, C., Ferrando, J., Batlle, R., & Kharitonova, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.13176 · Cited by 7
Reducing Non-Normative Text Generation from Language ModelsPreprint
Peng, X., Li, S., Frazier, S., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.08764 · Cited by 7
Gender Bias in Multilingual Neural Machine Translation: The Architecture MattersPreprint
Costa‐jussà, M. R., Escolano, C., Basta, C., Ferrando, J., Batlle, R., & Kharitonova, K. (2020) · arXiv (Cornell University) · Cited by 7
How True is GPT-2? An Empirical Analysis of Intersectional Occupational Biases.Preprint
Kirk, H. R., Jun, Y., Haider, I., Benussi, E., Volpin, F., Dreyer, F. A., Shtedritski, A., & Asano, Y. M. (2021) · arXiv (Cornell University) · Cited by 5
The Truth is Out There: Investigating Conspiracy Theories in Text Generation.Preprint
Levy, S., Saxon, M., & Wang, W. Y. (2021) · arXiv (Cornell University) · Cited by 4
Dialect Diversity in Text Summarization on TwitterPreprint
Celis, L. E., & Keswani, V. (2020) · arXiv (Cornell University) · Cited by 2
Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP
Schick, T., Udupa, S., & Schütze, H. (2021) · arXiv · doi:10.48550/arxiv.2103.00453 · Cited by 1
First the worst: Finding better gender translations during beam searchPreprint
Saunders, D., Sallis, R., & Byrne, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.07429
Dialect Diversity in Text Summarization on TwitterPreprint
Keswani, V., & Celis, L. E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.07860
Data and its (dis)contents: A survey of dataset development and use in machine learning research
Paullada, A., Raji, I. D., Bender, E. M., Denton, E., & Hanna, A. (2020) · arXiv · doi:10.48550/arxiv.2012.05345
How to Measure Gender Bias in Machine Translation: Optimal Translators, Multiple Reference Points
Farkas, A., & Németh, R. (2020) · arXiv · doi:10.48550/arxiv.2011.06445
Confronting Abusive Language Online: A Survey from the Ethical and Human Rights Perspective
Kiritchenko, S., Nejadgholi, I., & Fraser, K. C. (2020) · arXiv · doi:10.48550/arxiv.2012.12305
Research works printed but not linked: 8
- Saunders and Byrne (2020);
- Saunders et al. (2020); Kocmi et al. (2020); Costa-jussà and de Jorge (2020); Costa-jussà et al. (2020);
- Choubey et al. (2021); Saunders et al. (2021); Tomalin et al. (2021) Re-writing References
- Solon Barocas, Kate Crawford, Aaron Shapiro, and Hanna Wallach. 2017. The problem with bias: Al- locative versus representational harms in machine learning. In 9th Annual Conference of the Special Interest Group for Computing, Information and So- ciety.
- Kate Crawford. 2017. The trouble with bias. Keynote at NeurIPS.
- Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hi- erarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889-898.
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
- Alex Wang and Kyunghyun Cho. 2019. BERT has a mouth, and it must speak: BERT as a Markov random field language model. In Proceedings of the Workshop on Methods for Optimizing and Eval- uating Neural Language Generation, pages 30-36, Minneapolis, Minnesota. Association for Computa- tional Linguistics. · DOI
Cited by
Sorted by citations
A Survey on Evaluation of Large Language Models
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3641289 · Cited by 4,254
“, gender, race, religion, occupation, and ideology) [166].”
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245
Can Large Language Models Transform Computational Social Science?
Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., & Yang, D. (2024) · Computational Linguistics · doi:10.1162/coli_a_00502 · Cited by 585
Khan, W., Daud, A. Y., Khan, K., Muhammad, S., & Haq, R. (2023) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100026 · Cited by 260
Metadata
- Authors
- Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N.
- Affiliations
- University of Southern California (US)
- University of California, Los Angeles (US)
- Venue
- Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
- Published
- 2021
- Pages
- 4275-4293
- OpenAlex
- W3174685870
- Cited by
- 131
- References
- 99
- Citations by year
- 2026: 18 · 2025: 28 · 2024: 18 · 2023: 43 · 2022: 16 · 2021: 8
- FWCI
- 19.25
- Type
- conference paper
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- language generation, algorithmic fairness, algorithmic bias, natural language generation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
