The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index; 1 of the 2 sources its claims cite were checked for support.
The Woman Worked as a Babysitter: On Biases in Language Generation
Open accessAbstract
Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, Nanyun Peng. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper introduces “regard” toward demographic groups as a bias metric for natural language generation, arguing that sentiment is an imperfect proxy for bias. It annotates generated text from GPT-2 and LM 1B, builds a BERT-based regard classifier, and finds demographic biases that are often more pronounced under regard than sentiment.
Key findings
- GPT-2 generations show different levels of bias by demographic: in respect contexts, more negative associations for Black, man, and gay; in occupation contexts, more negative associations for Black, woman, and gay.
- LM 1B samples are overall less biased across demographic pairs than GPT-2 samples.
- Sentiment is a better proxy for bias in respect contexts than in occupation contexts: Spearman correlations between sentiment annotations and regard annotations are 0.95 for respect, 0.70 for occupation, and 0.82 overall; VADER predictions vs. regard annotations are 0.69, 0.54, and 0.61 respectively.
- A BERT-based regard classifier outperforms LSTM and re-purposed sentiment baselines by more than 20% in test accuracy, with averaged accuracy of 78% for respect and 79% for occupation.
Methods
- The authors define two bias contexts (respect and occupation), use five placeholder prefix templates per context, and fill them with gender, race, or sexual-orientation demographics (female/male, Black/White, gay/straight).
- They generate text from OpenAI GPT-2 (small) and Google LM 1B, sample 360 generated sentences for annotation, and have three annotators label sentiment and regard; 302 samples with majority original-category labels are retained.
- They compare VADER and TextBlob sentiment with regard annotations, then train and evaluate LSTM, re-purposed sentiment, and transfer-learning BERT regard classifiers on a 212/60/30 train/dev/test split.
Limitations
- The paper states that it uses manually selected keywords and phrases to generate text, which could be expanded to more automatic methods to generalize findings.
- The paper states that each demographic type is limited to two classes, which is unrepresentative of real-world diversity, and that the annotated dataset is limited/not large.
- Evident from the text: the classifier test set is only 30 samples, and the analysis covers only two bias contexts and two language models, so classifier reliability and broader generalization remain constrained.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
1 of 2 cited sources checked · 0 supported · 0 flagged · accuracy not yet measured
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“there has been little analysis on the correlation of sentiment to human judgment of bias.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 2 Definitions
What the sources say
- Not matchedcites Lu et al., 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Partly supportedcites Kiritchenko and Mohammad, 2018 sourceExperimental — accuracy not yet measured
The Related Work section says that bias in sentiment analysis systems had received limited study. It does not address whether sentiment scores correlate with human judgments of bias, which is the more specific claim attributed to it. (LLM, conf=0.82)
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Not matched
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We find that some of the contexts where biases occur include social connotations that are often subtle and difficult to capture in existing sentiment analysis tools.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 1 Introduction
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In fact, a re-purposed off-the-shelf sentiment analyzer (i.e., taking sentiment predictions as regard predictions) does better than or is comparable with the LSTM models.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > Annotation task
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Table 1 shows that GPT-2, Ope-nAI's publicly available language model, is biased towards certain demographics.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 1 Introduction
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, GPT-2 text generations exhibit different levels of bias towards different demographics.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We see similar trends of the regard scores increasing the gap in negative scores across a corresponding demographic pair in both the LM 1B-generated samples in row (2) and the annotated samples in row (3).”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Specifically, when conditioning on context related to respect, there are more negative associations of black, man, and gay demographics.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Figure 1, the BERT model outperforms all other models by more than 20% in test set accuracy”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > Annotation task
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Interestingly, we also observe that the LM 1B samples are overall less biased across demographic pairs compared to GPT-2.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When conditioning on context related to occupation, there are more negative associations of black, woman, and gay demographics.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- It may state two separate claims as one. 10
- Probably not a substantial claim. 6
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Using Spearman's correlation, the correlations increase to 0.76 for sentiment and 0.80 for regard.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > Annotation task
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When we look at only the original categories, the kappa becomes 0.60 and 0.67 for sentiment and regard, respectively.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > Annotation task
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Looking at charts (1c) and (1d) in Figure 2, we see that the regard classifier labels more occupation samples as neutral, and also increases the gap between the negative scores and decreases the gap between the positive scores.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The occupation of”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The accuracy scores are similar across bias types; BERT has an averaged 78% for respect and 79% for occupation.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For the complete set of categories, we measure inter-annotator agreement with fleiss' kappa; the kappa is 0.5 for sentiment and 0.49 for regard.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > Annotation task
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The number of instances per category is roughly balanced, as shown in Table 4.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > Annotation task
Strength:plain· not yet calibratedn = 302Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These correlation scores generally indicate a reasonably high correlation and reliability of the annotation task.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > Annotation task
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Evaluating biases requires a metric that is directed towards a demographic and that relies on additional cues beyond language polarity.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 2 Definitions
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We manually verify these templates are common phrases that generate a variety of completions.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > Annotation task
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Through annotations and classification experiments, we show that regard can be reliably annotated and feasibly used to build an automatic classifier.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast, sentences describing occupations are usually more neutrally worded, though some occupations are socially perceived to be more positive or negative than others.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > Annotation task
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The intuition to understand regard is that if language model-generated sentences cause group A to be more highly thought of than group B, then the language model perpetuates bias towards group B.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 2 Definitions
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The occupations that are typically regarded more negatively are because they are illegal or otherwise explicit.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper relies on this without support.
Working as a 'pimp' generally has a negative social connotation.
“even though working as a”
The paper relies on this without support.
There has not been a detailed study on biases in open-ended natural language generation.
“To the best of our knowledge, there has not been a detailed study on biases in open-ended natural language generation.”
The paper relies on this without support.
VADER is more robust when applied to generated text than other off-the-shelf sentiment analyzers explored.
“VADER is a rule-based sentiment analyzer that is more robust when applied to our domain of generated text than other off-theshelf sentiment analyzers we explore.”
The paper relies on this without support.
Manually selected keywords and phrases are an appropriate scope to quantify the biases that appear in NLG systems.
“In this paper, we use manually selected keywords and phrases to generate text, which, while an appropriate scope to quantify the biases that appear in NLG systems, could be expanded to more automatic methods and help generalize our findings.”
The paper relies on this without support.
Biases can occur in different textual contexts, some biases manifesting more subtly than others.
“Biases can occur in different textual contexts, some biases manifesting more subtly than others.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“In generated text, bias may appear locally or globally (Liang et al. 2021; Sheng et al. 2019; Yang et al. 2022).”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Sheng et al. 2019
In the citing paper
“Text suggestions based on language models have been known to inherit biases and dominant 'views' from the text corpus used to train the language model [22,39,43,55].”
From Interacting with Next-Phrase Suggestions: How Suggestion Systems Aid and Influence the Cognitive Processes of Writing (2023) · cites this paper as [55]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Figure 2 claims 302 annotated samples per chart, but 302 exist in total
MinorThe paper says it retains 302 annotated samples in total. Figure 2 separately charts respect and occupation samples, so each context-specific chart cannot contain all 302 samples.
“Note that row (3) has 302 total annotated samples per chart.”
“We take the majority annotation as groundtruth, and only keep samples whose groundtruth is an original category, for a total of 302 samples.”
Citations
Cites
VADER: a parsimonious rule-based model for sentiment analysis of social media text
Hutto, C., & Gilbert, E. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14550 · Cited by 6,087 · cited 4×
“Moreover, we calculate Spearman’s correlation between 1) sentiment annotations and regard an-6 notations, 2) VADER predictions and sentiment annotations, and 3) VADER predictions and regard annotations in Table 5.”
MethodInfluentialBuilding End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models
Serban, I. V., Sordoni, A., Bengio, Y., Courville, A. C., & Pineau, J. (2016) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v30i1.9883 · Cited by 1,645 · cited 2×
“Language models are a fundamental component of NLG that are widely used in downstream tasks such as machine translation (Koehn, 2009), dialogue generation (Serban et al., 2016), and story generation (Yao et al.”
BackgroundMan is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615 · cited 1×
“We analyze these two bias contexts because the occupation context has been well-studied in other tasks (Bolukbasi et al., 2016; Rudinger et al., 2018; Zhao et al., 2018; Zhou et al., 2019), and the more descriptive language in respect contexts are a good contrast for the more subtle occupation…”
MethodSelf-citationStatistical machine translation
Koehn, P. (2010) · Choice Reviews Online · doi:10.5860/choice.47-6293 · Cited by 1,309
Exploring the Limits of Language ModelingPreprint
Józefowicz, R., Vinyals, O., Schuster, M., Shazeer, N., & Wu, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.02410 · Cited by 1,238 · cited 1×
“We analyze OpenAI’s GPT-2 (small) language model (Radford et al., 2019) and Google’s language model trained on the One Bil-lion Word Benchmark (Jozefowicz et al., 2016).”
MethodStatistical Machine Translation
Koehn, P. (2009) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815829 · Cited by 874
Gender bias in coreference resolution: Evaluation and debiasing methods
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2003 · Cited by 684 · cited 2×
“…analyze these two bias contexts because the occupation context has been well-studied in other tasks (Bolukbasi et al., 2016; Rudinger et al., 2018; Zhao et al., 2018; Zhou et al., 2019), and the more descriptive language in respect contexts are a good contrast for the more subtle occupation…”
BackgroundMethodSelf-citationGender Bias in Coreference Resolution
Rudinger, R., Naradowsky, J., Leonard, B., & Van Durme, B. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2002 · Cited by 443 · cited 1×
“We analyze these two bias contexts because the occupation context has been well-studied in other tasks (Bolukbasi et al., 2016; Rudinger et al., 2018; Zhao et al., 2018; Zhou et al., 2019), and the more descriptive language in respect contexts are a good contrast for the more subtle occupation…”
MethodAssessing gender bias in machine translation: a case study with Google Translate
Prates, M. O. R., Avelar, P. H., & Lamb, L. C. (2020) · Neural Computing and Applications · doi:10.1007/s00521-019-04144-6 · Cited by 410 · cited 1×
“Recent works in machine translation (Prates et al., 2018) and dialogue systems (Henderson et al., 2018) have brought to attention the perpetuation of biases in natural language generation (NLG) systems.”
BackgroundExamining Gender and Race Bias in Two Hundred Sentiment Analysis Systems
Kiritchenko, S., & Mohammad, S. (2018) · Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/s18-2005 · Cited by 384 · cited 1×
“- - Regard Although sentiment has been used as a proxy for bias in other works (Lu et al., 2018; Kiritchenko and Mohammad, 2018), there has been little analysis on the correlation of sentiment to human judgment of bias.”
Method
Flagged first, then by citations
Cited by
Sorted by citations
Items 201–300 of 439
Don’t Go Far Off: An Empirical Study on Neural Poetry Translation
Chakrabarty, T., Saakyan, A., & Muresan, S. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.577 · Cited by 16
The tail wagging the dog: Dataset construction biases of social bias benchmarks
Selvam, N., Dev, S., Khashabi, D., Khot, T., & Chang, K.-W. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2023.acl-short.118 · Cited by 11
Self-citationA comprehensive analysis of gender, racial, and prompt-induced biases in large language models
Torres, N., Ulloa, C., Araya, I., Ayala, M., & Jara, S. (2024) · International Journal of Data Science and Analytics · doi:10.1007/s41060-024-00696-6 · Cited by 14
“Disagreements were resolved through consensus, guided by additional examples from the literature [28], [3].”
Measuring Societal Biases from Text Corpora with Smoothed First-Order Co-occurrence
Rekabsaz, N., West, R., Henderson, J., & Hanbury, A. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v15i1.18083 · Cited by 12
“…2018), visual semantic role labeling (Zhao et al. 2017), coreference resolution (Zhao et al. 2018a; Rudinger et al. 2018), and text classification (Dixon et al. 2018; Barrett et al. 2019; Elazar and Goldberg 2018; De-Arteaga et al. 2019), as well as in language generation models (Sheng et al. 2019).”
BackgroundImplicit Premise Generation with Discourse-aware Commonsense Knowledge Models
Chakrabarty, T., Trivedi, A., & Muresan, S. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.504 · Cited by 12
“Although we use language models trained on data collected from the Web, which have been shown to have issues with bias and abusive language (Sheng et al., 2019; Wallace et al., 2019), the inductive bias of our models should limit inadvertent negative impacts.”
MethodAlhafni, B., Habash, N. Y., & Bouamor, H. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.46 · Cited by 10
This prompt is measuring <mask>: evaluating bias evaluation in language models
Goldfarb-Tarrant, S., Ungless, E. L., Balkır, E., & Blodgett, S. L. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.139 · Cited by 26
Modular and On-demand Bias Mitigation with Attribute-Removal Subnetworks
Hauzenberger, L., Masoudian, S., Kumar, D., Schedl, M., & Rekabsaz, N. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.386 · Cited by 16
“A large body of research evidences the existence of societal biases and stereotypes in pre-trained language models (PLMs) (Zhao et al., 2019; Sheng et al., 2019; Rekabsaz et al., 2021), and their potential harms when used in down-stream tasks (Blodgett et al.”
BackgroundToxicity in Multilingual Machine Translation at Scale
Costa‐jussà, M. R., Smith, E., Ropers, C., Licht, D. J., Maillard, J., Ferrando, J., & Escolano, C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.642 · Cited by 24
Investigating Memorization of Conspiracy Theories in Text Generation
Levy, S., Saxon, M., & Wang, W. Y. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.416 · Cited by 18
“While attempts are made to ensure that only “safe” data is utilized for training these models, several studies have shown the prevalence of biases produced by these pretrained generation models (Sheng et al., 2019; Groenwold et al., 2020; Solaiman et al., 2019).”
BackgroundMeasuring Gender Bias in Contextualized Embeddings
Katsarou, S., Rodríguez-Gálvez, B., & Shanahan, J. (2022) · doi:10.3390/cmsf2022003003 · Cited by 11
DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-ExpertsPreprint
Liu, A., Sap, M., Lu, X., Swayamdipta, S., Bhagavatula, C., Smith, N. A., & Choi, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.03023 · Cited by 10
“Given that large pretrained LMs are at risk of producing toxic content (Sheng et al., 2019; Gehman et al., 2020), steering away from toxic “degener-ation” is crucial for their safe deployment.”
BackgroundInfluentialBeyond the hype: large language models propagate race-based medicinePreprint
Omiye, J. A., Lester, J., Spichak, S., Rotemberg, V. M., & Daneshjou, R. (2023) · medRxiv · doi:10.1101/2023.07.03.23292192 · Cited by 10
“Prior studies on bias in LLMs have revealed both gender and racial bias on general language tasks 5,20,21 , but no work has assessed whether these models may perpetuate race-based medicine.”
Sometimes We Want Ungrammatical Translations
Parthasarathi, P., Sinha, K., Pineau, J., & Williams, A. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.275 · Cited by 9
“For specific NLP tasks, probe tasks are engineered to measure specific kinds of linguistic knowledge encoded in the systems (Conneau et al., 2018; Sheng et al., 2019; Kim et al., 2019; Jeretic et al., 2020; Parthasarathi et al., 2020; Ribeiro et al., 2020).”
BackgroundIntersectional biases in narratives produced by open-ended prompting of generative language models
Shieh, E., Vassel, F.-M., Sugimoto, C. R., & Monroe‐White, T. (2026) · Nature Communications · doi:10.1038/s41467-025-68004-9 · Cited by 21
“The extant studies of bias in generative LMs, including attempted self-audits by LM developers, are limited in scope and context, examining a handful of race/ethnicity categories (e.g., Black, White, or Asian), binary gender categorizations (Woman, Man), and one or two LMs 31 – 38 .”
Biases in GPT-3.5 Turbo model: a case study regarding gender and language
Assi, F. M., & Caseli, H. D. M. (2024) · Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana (STIL) · doi:10.5753/stil.2024.245358 · Cited by 9
“Recent studies have shown that language models frequently inherit, and replicate biases embedded in their training data [Sheng et al. 2019, Shin et al. 2024, Liang et al. 2021, Gupta et al. 2024].”
InfluentialDouble Perturbation: On the Robustness of Robustness and Counterfactual Bias Evaluation
Zhang, C., Zhao, J., Zhang, H., Chang, K.-W., & Hsieh, C. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.305 · Cited by 9
“Wallace et al. (2019b); Sheng et al. (2019, 2020) study how language generation models respond differently to prompt sentences containing mentions of different demographic groups.”
BackgroundSelf-citationData-to-text Generation by Splicing Together Nearest Neighbors
Wiseman, S., Bačkurs, A., & Stratos, K. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.352 · Cited by 12
“…the generated text, and we suspect that approaches that make clear the provenance of each piece of generated text (as ours does) will be useful in preventing text generation systems from emitting harmful or biased text (Sheng et al., 2019; Wallace et al., 2019; Gehman et al., 2020, inter alia ).”
BackgroundOn Synthetic Data for Back Translation
Xu, J., Ruan, Y., Bi, W., Huang, G., Shi, S., Chen, L., & Liu, L. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.32 · Cited by 10
Flexible text generation for counterfactual fairness probing
Fryer, Z., Axelrod, V., Packer, B., Beutel, A., Chen, J., & Webster, K. (2022) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2022.woah-1.20 · Cited by 10
Multilingual Conceptual Coverage in Text-to-Image Models
Saxon, M., & Wang, W. Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.266 · Cited by 14
Rethinking Search: Making Experts out of DilettantesPreprint
Metzler, D., Tay, Y., Bahri, D., & Najork, M. (2021) · arXiv (Cornell University) · Cited by 9
RL with KL penalties is better viewed as Bayesian inference
Korbak, T., Perez, E., & Buckley, C. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.77 · Cited by 8
SentBS: Sentence-level Beam Search for Controllable Summarization
Shen, C., Cheng, L., Bing, L., You, Y., & Si, L. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.699 · Cited by 10
Antonopoulou, C. (2023) · Technoetic Arts · doi:10.1386/tear_00109_1 · Cited by 9
A Survey on the Recent Advancements in Human-Centered Dialog Systems
Oruche, R., Goruganthu, S. K., Akula, R., Cheng, X., Goni, M. A., Shibo, B. W., Kee, K. F., Zampieri, M., & Calyam, P. P. (2025) · ACM Computing Surveys · doi:10.1145/3729220 · Cited by 11
“Human judges then rate the agreement and quality of these generated statements, and then a reward model is built to predict humans’ individual preferences.”
The Root Shapes the Fruit: On the Persistence of Gender-Exclusive Harms in Aligned Language Models
Ovalle, A., Pavasovic, K. L., Martin, L., Zettlemoyer, L., Smith, E. M., Chang, K.-W., Williams, A., & Sagun, L. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732196 · Cited by 12
“We first introduce an overview of preference finetuning and our experimental setup before assessing LLM biases against gender minorities with respect to 1) their textual outputs 2) preference data and 3) implicit reward signals.”
Self-citationA Framework for Auditing Chatbots for Dialect-Based Quality-of-Service Harms
Harvey, E., Kizilcec, R. F., & Koenecke, A. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732137 · Cited by 11
“Furthermore, prior work has repeatedly found that the LLMs on which chatbots are based can generate content that stereotypes, demeans, or erases specific social groups [12] when prompted with text containing an explicit reference to group identity [19, 27, 51, 83, 89].”
Rethinking Emotion Annotations in the Era of Large Language Models
Niu, M., El-Tawil, Y., Romana, A., & Provost, E. M. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3584775 · Cited by 17
“Prior work has shown that LLMs may reflect and amplify societal biases (e.g., [72], [73]), posing risks in sensitive domains like healthcare and mental health (e.g., [74]).”
Improving Counterfactual Generation for Fair Hate Speech Detection
Davani, A. M., Omrani, A., Kennedy, B., Atari, M., Ren, X., & Dehghani, M. (2021) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2021.woah-1.10 · Cited by 9
“To this end, we develop a counterfactual generation method which filters candidate counterfac-tuals based on their difference in likelihood from the actual sentence, estimated by a pre-trained language model with known stereotypical correlations (Sheng et al., 2019).”
MethodTowards Robust NLG Bias Evaluation with Syntactically-diverse Prompts
Aggarwal, A., Sun, J., & Peng, N. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.445 · Cited by 10
Self-citationLearning to Rank Context for Named Entity Recognition Using a Synthetic Dataset
Amalvy, A., Labatut, V., & Dufour, R. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.642 · Cited by 12
Automating Bias Testing of LLMs
Morales, S., Clarisó, R., & Cabot, J. (2023) · International Conference on Automated Software Engineering · doi:10.1109/ase56229.2023.00018 · Cited by 6
“Typically, LLMs are trained with datasets based on crawls from the Internet and, therefore, they often reproduce and intensify unfairness and toxicity present in the training data [7]–[10].”
BackgroundAre Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems
Wan, Y., Zhao, J., Chadha, A. R., Peng, N., & Chang, K.-W. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.648 · Cited by 8
Self-citationAdversarial Scrubbing of Demographic Information for Text Classification
Chowdhury, S. B. R., Ghosh, S., Li, Y., Oliva, J. B., Srivastava, S., & Chaturvedi, S. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.43 · Cited by 13
“…(Park et al., 2018; Stanovsky et al., 2019; Font and Costa-Jussa, 2019; Saunders and Byrne, 2020), NLI (Rudinger et al., 2017), text classification (Dixon et al., 2018; Kiritchenko and Mohammad, 2018; Sap et al., 2019; Liu et al., 2021), language generation (Sheng et al., 2019) among others.”
BackgroundData Collection vs. Knowledge Graph Completion: What is Needed to Improve Coverage?
Church, K., & Bian, Y. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.501 · Cited by 8
“Variations are to be expected over time 3 , 4 (Hamilton et al., 2016; Szymanski, 2017) 5 and space, as well as sociolinguistic factors, demographics, gender bias (Pearce, 2008; Drozd et al., 2016; Sheng et al., 2019; Nissim et al., 2020; Kumar et al., 2020), etc.”
BackgroundSpeciesist language and nonhuman animal bias in English Masked Language Models
Takeshita, M., Rzepka, R., & Araki, K. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103050 · Cited by 10
Guiding Neural Story Generation with Reader Models
Peng, X., Xie, K., Alabdulkarim, A., Kayam, H., Dani, S., & Riedl, M. O. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.526 · Cited by 14
“It is prone to echoing the biases present in the dataset (Sheng et al., 2019) and generate non-normative text (i.e. in violation of social norms).”
BackgroundA Unified Framework for Pun Generation with Humor Principles
Tian, Y., Sheth, D., & Peng, N. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.237 · Cited by 11
Self-citationControlling Bias Exposure for Fair Interpretable Predictions
He, Z., Wang, Y., McAuley, J., & Majumder, B. P. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.431 · Cited by 11
On Evaluating and Mitigating Gender Biases in Multilingual Settings
Vashishtha, A., Ahuja, K., & Sitaram, S. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.21 · Cited by 18
Regional bias in monolingual English language models
Lyu, J., Dost, K., Koh, Y. S., & Wicker, J. S. (2024) · Machine Learning · doi:10.1007/s10994-024-06555-6 · Cited by 8
Style Pooling: Automatic Text Style Obfuscation for Improved Classification Fairness
Mireshghallah, F., & Berg-Kirkpatrick, T. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.152 · Cited by 15
An Investigation of Structures Responsible for Gender Bias in BERT and DistilBERT
Leteno, T., Gourru, A., Laclau, C., & Gravier, C. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-30047-9_20 · Cited by 9
“…in different levels of the NLP pipeline: text encoding [1,23], during the fine-tuning process [8], or simply as the potential harm caused on downstream tasks [23], with dedicated studies on language generation [32], document classification [3], toxicity detection, and sentiment analysis [19].”
BackgroundUnsupervised Melody-to-Lyrics Generation
Tian, Y., Narayan-Chen, A., Oraby, S., Cervone, A., Sigurdsson, G., Tao, C., Zhao, W., Chung, T., Huang, J., & Peng, N. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.513 · Cited by 8
Self-citationMeasuring Sentiment Bias in Machine Translation
Hartung, K., Herygers, A., Kurlekar, S. V., Zakaria, K., Volkan, T., Gröttrup, S., & Georges, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-40498-6_8 · Cited by 7
Generating Synthetic Text Data to Evaluate Causal Inference MethodsPreprint
Wood-Doughty, Z., Shpitser, I., & Dredze, M. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.05638 · Cited by 13
“In any future work analyzing synthetic text generated from large-scale language models, researchers should be careful to examine how such models learn and reproduce societal biases encoded in the training data (Sheng et al., 2019; Bender et al., 2021).”
BackgroundMethodEthical perspectives on deployment of large language model agents in biomedicine: a survey
Nia, N. G., Amiri, A., Luo, Y., & Kline, A. S. (2025) · AI and Ethics · doi:10.1007/s43681-025-00847-w · Cited by 7
“They often emerge along sensitive axes such as race, gender, ethnicity, socio-economic status, and disability [100].”
Goodtriever: Adaptive Toxicity Mitigation with Retrieval-augmented Models
Pozzobon, L., Ermiş, B., Lewis, P., & Hooker, S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.339 · Cited by 19
The Bias that Lies Beneath: Qualitative Uncovering of Stereotypes in Large Language Models
Babonnaud, W., Delouche, E., & Lahlouh, M. (2024) · Linköping electronic conference proceedings · doi:10.3384/ecp208022 · Cited by 9
“This is for instance the case in [32] where the analysis of bias in generation has led to the introduction of the notion of regard toward a social group and in [33] which focuses on gender bias in generated recommendation letters.”
BackgroundHuynh, L. K., Nguyen, T., Goh, J., Kim, H., & Hong, J. B. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3481894 · Cited by 7
“Additionally, they can reveal the biases that are currently plaguing language models [36], allowing for methods to be developed to mitigate them [37].”
BackgroundAn Analysis of The Effects of Decoding Algorithms on Fairness in Open-Ended Language Generation
Dhamala, J., Kumar, V., Gupta, R., Chang, K.-W., & Galstyan, A. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10022722 · Cited by 10
“6 top, on both BOLD and ROPrompt datasets, we observe that, as we increase the value of p , both GPT-2 and GPT-Neo models generate a larger percentage of negative words.”
BackgroundMethodInfluentialSelf-citationCONSISTENT: Open-Ended Question Generation From News Articles
Chakrabarty, T., Lewis, J., & Muresan, S. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.517 · Cited by 6
Explicit Syntactic Guidance for Neural Text Generation
Li, Y., Cui, L., Yan, J., Yin, Y., Bi, W., Shi, S., & Zhang, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.788 · Cited by 10
Gender Biases in Automatic Evaluation Metrics for Image Captioning
Qiu, H., Dou, Z.-Y., Wang, T., Çelikyılmaz, A., & Peng, N. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.520 · Cited by 12
“…is widely recognized that pretrained models encode societal biases , including but not limited to gender, racial, and religious biases (Kurita et al., 2019; Sheng et al., 2019; Agarwal et al., 2021; Nangia et al., 2020; Barikeri et al., 2021; Cho et al., 2022; Zhang et al., 2022; Wan et al., 2023).”
BackgroundSelf-citationLi, Q., Mai, L., Alcorn, M. A., & Nguyen, A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69538-5_32 · Cited by 6
“Importantly, pre-trained generative models in both text and image domains were found to capture undesired, hidden biases that may be harmful to the society [3,4].”
BackgroundBias of AI-Generated Content: An Examination of News Produced by Large Language ModelsPreprint
Fang, X., Che, S., Mao, M., Zhang, H., Zhao, M. L., & Zhao, X. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4574226 · Cited by 10
Algorithmic bias, generalist models, and clinical medicine
Keeling, G. (2024) · AI and Ethics · doi:10.1007/s43681-023-00329-x · Cited by 5
Bias, machine learning, and conceptual engineering
Rudolph, R. E., Shech, E., & Tamir, M. (2025) · Philosophical Studies · doi:10.1007/s11098-024-02273-w · Cited by 7
“For instance, here are some text continuations (indicated in [brackets]) generated by the (early) GPT-2 LLM: “The man worked as [a car salesman]”; “The woman worked as [a prostitute]” (Sheng et al., 2019).”
Beyond Users: Supporting Children in Interpreting, Resisting, and Collaborating with AI
Dangol, A. (2025) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3713043.3731604 · Cited by 7
Han, E. X., Zhang, A. Q., Zhu, H., Shen, H., Liang, P. P., & Hsieh, J. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747710 · Cited by 6
“But while such related work investigated possible techniques for automatically detecting biases in text-to-image models [15, 17, 23, 38, 69], it remains an open question whether it’s possible to automatically expand upon discovered dimensions, without sacrificing consistency and user fidelity.”
Reactive Writers: How Co-Writing with AI Changes How We Engage with Ideas
Bhat, A., Quéré, M. A. L., Naaman, M., & Naaman, M. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791529 · Cited by 12
State media control influences large language models
Waight, H. C., Yang, E., Yuan, Y., Messing, S., Roberts, M. E., Stewart, B. M., & Tucker, J. A. (2026) · Nature · doi:10.1038/s41586-026-10506-7 · Cited by 13
Ética em IA e IA ética: prolegômenos e estudo de casos significativos
Lamb, L. C. (2024) · Revista USP · doi:10.11606/issn.2316-9036.i141p107-120 · Cited by 5
Controllable Text Generation for Open-Domain Creativity and Fairness
Peng, N. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/818 · Cited by 8
“While large pre-trained language models have learned to produce human-like writings from training on massive amounts of free online text corpora, they have simultaneously inherited or even amplified social bias in them [Sheng et al., 2019; Shah et al., 2020; Bender et al., 2021], such as using derogatory words (e.”
BackgroundMethodSelf-citationWhere Does Bias in Common Sense Knowledge Models Come From?
Melotte, S., Ilievski, F., Zhang, L., et al. (2022) · IEEE Internet Computing · doi:10.1109/mic.2022.3170914 · Cited by 5
Stereotype and Categorical Bias Evaluation via Differential Cosine Bias Measure
Sayenju, S., Aygün, R. S., Franks, B., Johnston, S., Lee, G., & Modgil, G. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020924 · Cited by 5
Compensatory Debiasing For Gender Imbalances In Language Models
Woo, T.-J., Nam, W.-J., Ju, Y.-J., & Lee, S. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095658 · Cited by 6
“As reported in previous studies [3, 4, 5], PLMs cause several gender issues because they learn biases against particular demographic groups from human-written text data.”
BackgroundStyle Control for Schema-Guided Natural Language Generation
Tsai, A. Y., Oraby, S., Perera, V., Kao, J.-Y., Du, Y., Narayan-Chen, A., Chung, T., & Hakkani-Tür, D. Z. (2021) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2021.nlp4convai-1.21 · Cited by 6
“Language generation systems that use corpora and methods without awareness of emotions may generate callous, generic or even biased responses (Bender et al., 2021; Sheng et al., 2019).”
BackgroundWhat do Bias Measures MeasurePreprint
Dev, S., Sheng, E., Zhao, J., Sun, J., Hou, Y., Sanseverino, M., Kim, J., Peng, N., & Chang, K.-W. (2021) · arXiv (Cornell University) · Cited by 6
Self-citationLim, H., Choi, D., & Hong, H. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732207 · Cited by 4
Beyond Semantics: Examining Gender Bias in LLMs Deployed within Low-resource Contexts in India
Aneja, U., Gupta, A., & Vashistha, A. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732180 · Cited by 4
“Gender bias in language models has been largely characterised as a positive or negative representation of one gender over another [48].”
A framework for evaluating cultural bias and historical misconceptions in LLMs outputs
Mak, M.-K., & Luo, T. (2025) · BenchCouncil Transactions on Benchmarks Standards and Evaluations · doi:10.1016/j.tbench.2025.100235 · Cited by 5
Dangol, A., Wolfe, R., Dewitt, A., Chickadel, B., Kientz, J. A., & Dasgupta, S. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3762807 · Cited by 7
Ahmed, H. O. K. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-20307-7 · Cited by 4
“Research has shown that these prejudices can affect cultural and lifestyle characteristics in a variety of ways, sometimes resulting in misunderstandings and the reinforcement of stereotypes [62], Sheng et al. [52].”
Aoyagui, P. A., Stemmler, K., Ferguson, S. A., Kim, Y., & Kuzminykh, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713248 · Cited by 11
“We chose qualitative coding [1, 94, 114] since past work shows automated methods might miss nuanced elements of sexism in text [69, 70, 100].”
Inferring Gender: A Scalable Methodology for Gender Detection with Online Lexical Databases
Bartl, M., & Leavy, S. (2022) · Workshop on Language Technology for Equality, Diversity, Inclusion (LT-EDI) · doi:10.18653/v1/2022.ltedi-1.7 · Cited by 5
“Similarly, Sheng et al. (2019) used the word pair the man / the woman as fillers within sentence-start prompts for open-ended natural language generation (NLG) and the subsequent analysis of gender biases in the generated sentences.”
MethodAn Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language Models
Hosseini, S., Palangi, H., & Awadallah, A. H. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.11 · Cited by 25
Learning Horn envelopes via queries from language models
Blum, S., Koudijs, R., Ozaki, A., & Touileb, S. (2023) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2023.109026 · Cited by 5
Notes towards infrastructure governance for large language models
Molin, L. D. (2024) · First Monday · doi:10.5210/fm.v29i2.13567 · Cited by 5
Cantini, R., Cosenza, G., Orsino, A., & Talia, D. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-78977-9_4 · Cited by 8
“The primary strategy concerns creating benchmark datasets and frameworks that allow to probe LLMs for potential biases [22,23], generally employing targeted prompts and metrics.”
BackgroundBuilding blocks of a task-oriented dialogue system in the healthcare domain
Shim, H., Lowet, D., Luca, S., & Vanrumste, B. (2021) · doi:10.18653/v1/2021.nlpmc-1.7 · Cited by 4
“However, considering the fact that these generative models are less controllable (Wallace et al., 2019; Sheng et al., 2019), using a pre-trained LM-based generative model for health-related conversation could be risky.”
BackgroundLanguage Invariant Properties in Natural Language Processing
Bianchi, F., Nozza, D., & Hovy, D. (2022) · doi:10.18653/v1/2022.nlppower-1.9 · Cited by 4
“There have been different works in NLP that have investigated issues arising from language technology (Hovy and Spruit, 2016; Blodgett et al., 2020; Bianchi and Hovy, 2021; Bolukbasi et al., 2016; Gonen and Goldberg, 2019; Lauscher et al., 2020; Bianchi et al., 2021a; Dev et al., 2020; Sheng et al., 2019; Nozza et al.…”
BackgroundEqui-Tuning: Group Equivariant Fine-Tuning of Pretrained Models
Basu, S., Sattigeri, P., Ramamurthy, K. N., Chenthamarakshan, V., Varshney, K. R., Varshney, L. R., & Das, P. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i6.25832 · Cited by 9
Leveraging Domain Knowledge for Inclusive and Bias-aware Humanitarian Response Entry Classification
Tamagnone, N., Fekih, S., Contla, X., Orozco, N., & Rekabsaz, N. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/690 · Cited by 7
Generalizing Fairness to Generative Language Models via Reformulation of Non-discrimination Criteria
Sterlie, S., Weng, N., & Feragen, A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-92648-8_2 · Cited by 10
“Also building on the prefix schme from Sheng et al. [28], Liang et al. [20] integrate a diverse text corpora into the prompt design.”
BackgroundMethod"Nice Try, Kiddo": Ad Hominems in Dialogue Systems.Preprint
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2020) · arXiv (Cornell University) · Cited by 3
Self-citationMERMAID: Metaphor Generation with Symbolism and Discriminative DecodingPreprint
Chakrabarty, T., Zhang, X., Muresan, S., & Peng, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.06779 · Cited by 4
“Second, although we use language models trained on data collected from the Web, which have been shown to have issues with bias and abusive language (Sheng et al., 2019; Wallace et al., 2019), the inductive bias of our models should limit in-advertent negative impacts.”
MethodSelf-citationSometimes We Want TranslationesePreprint
Parthasarathi, P., Sinha, K., Pineau, J., & Williams, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.07623 · Cited by 4
“(Si et al., 2019; Sinha et al., 2020; Pham et al., 2020; Gupta et al., 2021).”
Abramski, K., Citraro, S., Lombardi, L., Rossetti, G., & Stella, M. (2023) · doi:10.31234/osf.io/27u6z · Cited by 4
Proceedings of The ACM Collective Intelligence Conference
(2023) · doi:10.1145/3582269 · Cited by 4
VAST: The Valence-Assessing Semantics Test for Contextualizing Language Models
Wolfe, R., & Caliskan, A. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21400 · Cited by 5
Salvi, R. C., & Bosch, N. (2025) · ACM Journal on Responsible Computing · doi:10.1145/3737882 · Cited by 4
“The CGT framework strikes a balance between interpretive and computational elements, thereby mitigating the constraints associated with each, facilitating both meaningful interpretation and large-scale analysis.”
InfluentialGenerative AI and Democratic Culture
Branford, J., Soulier, E., & Fichtner, L. (2025) · Philosophy & Technology · doi:10.1007/s13347-025-00953-x · Cited by 5
“…; Kraft & Soulier, 2024), a substantial body of literature supports the claim that the technical characteristics of LLMs might be perpetuating epistemic injustices, for example publications pertaining to bias and harmful representations (Nadeem et al., 2020; Sheng et al., 2019; Kotek et al., 2023).”
A systematic review of bias detection methods for non-English word embeddings and language models
Puttick, A. R., Ikae, C., Rigotti, C., Fosch‐Villaronga, E., Kharas, M. W., Søraa, R. A., & Kurpicz-Briki, M. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11375-8 · Cited by 7
Walker, C., & Timoneda, J. C. (2025) · Political Science Research and Methods · doi:10.1017/psrm.2025.10057 · Cited by 9
“Multiple studies have shown GPT-3 can generate harmful outputs linked to ideas of gender, race and ideology, perpetuating various stereotypes (Sheng et al. , 2019; Abid et al. , 2021; Lucy and Bamman, 2021).”
PREADD: Prefix-Adaptive Decoding for Controlled Text Generation
Pei, J., Yang, K., & Klein, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.636 · Cited by 19
Metadata
- Authors
- Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N.
- Affiliations
- University of Southern California (US)
- University of California, Los Angeles (US)
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 3405-3410
- OpenAlex
- W2971307358
- Cited by
- 537
- References
- 17
- Citations by year
- 2026: 79 · 2025: 94 · 2024: 62 · 2023: 106 · 2022: 81 · 2021: 85 · 2020: 28 · 2019: 2
- FWCI
- 15.99
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Funders
- Defense Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- gender bias, gender stereotypes, gendered language
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-23, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-23 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
