The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index; 1 of the 2 sources its claims cite were checked for support.
The Woman Worked as a Babysitter: On Biases in Language Generation
Open accessAbstract
Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, Nanyun Peng. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper introduces “regard” toward demographic groups as a bias metric for natural language generation, arguing that sentiment is an imperfect proxy for bias. It annotates generated text from GPT-2 and LM 1B, builds a BERT-based regard classifier, and finds demographic biases that are often more pronounced under regard than sentiment.
Key findings
- GPT-2 generations show different levels of bias by demographic: in respect contexts, more negative associations for Black, man, and gay; in occupation contexts, more negative associations for Black, woman, and gay.
- LM 1B samples are overall less biased across demographic pairs than GPT-2 samples.
- Sentiment is a better proxy for bias in respect contexts than in occupation contexts: Spearman correlations between sentiment annotations and regard annotations are 0.95 for respect, 0.70 for occupation, and 0.82 overall; VADER predictions vs. regard annotations are 0.69, 0.54, and 0.61 respectively.
- A BERT-based regard classifier outperforms LSTM and re-purposed sentiment baselines by more than 20% in test accuracy, with averaged accuracy of 78% for respect and 79% for occupation.
Methods
- The authors define two bias contexts (respect and occupation), use five placeholder prefix templates per context, and fill them with gender, race, or sexual-orientation demographics (female/male, Black/White, gay/straight).
- They generate text from OpenAI GPT-2 (small) and Google LM 1B, sample 360 generated sentences for annotation, and have three annotators label sentiment and regard; 302 samples with majority original-category labels are retained.
- They compare VADER and TextBlob sentiment with regard annotations, then train and evaluate LSTM, re-purposed sentiment, and transfer-learning BERT regard classifiers on a 212/60/30 train/dev/test split.
Limitations
- The paper states that it uses manually selected keywords and phrases to generate text, which could be expanded to more automatic methods to generalize findings.
- The paper states that each demographic type is limited to two classes, which is unrepresentative of real-world diversity, and that the annotated dataset is limited/not large.
- Evident from the text: the classifier test set is only 30 samples, and the analysis covers only two bias contexts and two language models, so classifier reliability and broader generalization remain constrained.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
17 Own result · 11 Own interpretation · 4 No basis given · 1 Partly supported · 1 Own method · accuracy not yet measured
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“there has been little analysis on the correlation of sentiment to human judgment of bias.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 2 Definitions
What the sources say
- Not matchedcites Lu et al., 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Partly supportedcites Kiritchenko and Mohammad, 2018 sourceExperimental — accuracy not yet measured
The Related Work section says that bias in sentiment analysis systems had received limited study. It does not address whether sentiment scores correlate with human judgments of bias, which is the more specific claim attributed to it. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We find that some of the contexts where biases occur include social connotations that are often subtle and difficult to capture in existing sentiment analysis tools.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 1 Introduction
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In fact, a re-purposed off-the-shelf sentiment analyzer (i.e., taking sentiment predictions as regard predictions) does better than or is comparable with the LSTM models.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > Annotation task
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Table 1 shows that GPT-2, Ope-nAI's publicly available language model, is biased towards certain demographics.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 1 Introduction
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, GPT-2 text generations exhibit different levels of bias towards different demographics.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
plaincorrelationaln = 500- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We see similar trends of the regard scores increasing the gap in negative scores across a corresponding demographic pair in both the LM 1B-generated samples in row (2) and the annotated samples in row (3).”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Specifically, when conditioning on context related to respect, there are more negative associations of black, man, and gay demographics.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
plaincorrelationaln = 500- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Figure 1, the BERT model outperforms all other models by more than 20% in test set accuracy”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > Annotation task
plainn = 30- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Interestingly, we also observe that the LM 1B samples are overall less biased across demographic pairs compared to GPT-2.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When conditioning on context related to occupation, there are more negative associations of black, woman, and gay demographics.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
plaincorrelationaln = 500- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 10
- Probably not a substantial claim. 6
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Using Spearman's correlation, the correlations increase to 0.76 for sentiment and 0.80 for regard.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > Annotation task
plaindirectn = 302- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When we look at only the original categories, the kappa becomes 0.60 and 0.67 for sentiment and regard, respectively.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > Annotation task
plainn = 302- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Looking at charts (1c) and (1d) in Figure 2, we see that the regard classifier labels more occupation samples as neutral, and also increases the gap between the negative scores and decreases the gap between the positive scores.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
plaindirectn = 500- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The occupation of”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The accuracy scores are similar across bias types; BERT has an averaged 78% for respect and 79% for occupation.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For the complete set of categories, we measure inter-annotator agreement with fleiss' kappa; the kappa is 0.5 for sentiment and 0.49 for regard.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > Annotation task
plainn = 360- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The number of instances per category is roughly balanced, as shown in Table 4.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > Annotation task
plainn = 302- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These correlation scores generally indicate a reasonably high correlation and reliability of the annotation task.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > Annotation task
hedgedcorrelationaln = 360- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Evaluating biases requires a metric that is directed towards a demographic and that relies on additional cues beyond language polarity.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 2 Definitions
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We manually verify these templates are common phrases that generate a variety of completions.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > Annotation task
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Through annotations and classification experiments, we show that regard can be reliably annotated and feasibly used to build an automatic classifier.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
boostedn = 302- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast, sentences describing occupations are usually more neutrally worded, though some occupations are socially perceived to be more positive or negative than others.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > Annotation task
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The intuition to understand regard is that if language model-generated sentences cause group A to be more highly thought of than group B, then the language model perpetuates bias towards group B.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 2 Definitions
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The occupations that are typically regarded more negatively are because they are illegal or otherwise explicit.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper relies on this without support.
Working as a 'pimp' generally has a negative social connotation.
“even though working as a”
The paper relies on this without support.
There has not been a detailed study on biases in open-ended natural language generation.
“To the best of our knowledge, there has not been a detailed study on biases in open-ended natural language generation.”
The paper relies on this without support.
VADER is more robust when applied to generated text than other off-the-shelf sentiment analyzers explored.
“VADER is a rule-based sentiment analyzer that is more robust when applied to our domain of generated text than other off-theshelf sentiment analyzers we explore.”
The paper relies on this without support.
Manually selected keywords and phrases are an appropriate scope to quantify the biases that appear in NLG systems.
“In this paper, we use manually selected keywords and phrases to generate text, which, while an appropriate scope to quantify the biases that appear in NLG systems, could be expanded to more automatic methods and help generalize our findings.”
The paper relies on this without support.
Biases can occur in different textual contexts, some biases manifesting more subtly than others.
“Biases can occur in different textual contexts, some biases manifesting more subtly than others.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“In generated text, bias may appear locally or globally (Liang et al. 2021; Sheng et al. 2019; Yang et al. 2022).”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Sheng et al. 2019
In the citing paper
“Text suggestions based on language models have been known to inherit biases and dominant 'views' from the text corpus used to train the language model [22,39,43,55].”
From Interacting with Next-Phrase Suggestions: How Suggestion Systems Aid and Influence the Cognitive Processes of Writing (2023) · cites this paper as [55]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ The Woman Worked as a Babysitter: On Biases in Language Generation > 5 Biases in language generation systems
“Note that row (3) has 302 total annotated samples per chart.”
§ The Woman Worked as a Babysitter: On Biases in Language Generation > Annotation task
“We take the majority annotation as groundtruth, and only keep samples whose groundtruth is an original category, for a total of 302 samples.”
Citations
Cites
VADER: a parsimonious rule-based model for sentiment analysis of social media text
Hutto, C., & Gilbert, E. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14550 · Cited by 6,087 · cited 4×
“Moreover, we calculate Spearman’s correlation between 1) sentiment annotations and regard an-6 notations, 2) VADER predictions and sentiment annotations, and 3) VADER predictions and regard annotations in Table 5.”
MethodInfluentialBuilding End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models
Serban, I. V., Sordoni, A., Bengio, Y., Courville, A. C., & Pineau, J. (2016) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v30i1.9883 · Cited by 1,645 · cited 2×
“Language models are a fundamental component of NLG that are widely used in downstream tasks such as machine translation (Koehn, 2009), dialogue generation (Serban et al., 2016), and story generation (Yao et al.”
BackgroundMan is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615 · cited 1×
“We analyze these two bias contexts because the occupation context has been well-studied in other tasks (Bolukbasi et al., 2016; Rudinger et al., 2018; Zhao et al., 2018; Zhou et al., 2019), and the more descriptive language in respect contexts are a good contrast for the more subtle occupation…”
MethodSelf-citationStatistical machine translation
Koehn, P. (2010) · Choice Reviews Online · doi:10.5860/choice.47-6293 · Cited by 1,309
Exploring the Limits of Language ModelingPreprint
Józefowicz, R., Vinyals, O., Schuster, M., Shazeer, N., & Wu, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.02410 · Cited by 1,238 · cited 1×
“We analyze OpenAI’s GPT-2 (small) language model (Radford et al., 2019) and Google’s language model trained on the One Bil-lion Word Benchmark (Jozefowicz et al., 2016).”
MethodStatistical Machine Translation
Koehn, P. (2009) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815829 · Cited by 874
Gender bias in coreference resolution: Evaluation and debiasing methods
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2003 · Cited by 684 · cited 2×
“…analyze these two bias contexts because the occupation context has been well-studied in other tasks (Bolukbasi et al., 2016; Rudinger et al., 2018; Zhao et al., 2018; Zhou et al., 2019), and the more descriptive language in respect contexts are a good contrast for the more subtle occupation…”
BackgroundMethodSelf-citationGender Bias in Coreference Resolution
Rudinger, R., Naradowsky, J., Leonard, B., & Van Durme, B. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2002 · Cited by 443 · cited 1×
“We analyze these two bias contexts because the occupation context has been well-studied in other tasks (Bolukbasi et al., 2016; Rudinger et al., 2018; Zhao et al., 2018; Zhou et al., 2019), and the more descriptive language in respect contexts are a good contrast for the more subtle occupation…”
MethodAssessing gender bias in machine translation: a case study with Google Translate
Prates, M. O. R., Avelar, P. H., & Lamb, L. C. (2020) · Neural Computing and Applications · doi:10.1007/s00521-019-04144-6 · Cited by 410 · cited 1×
“Recent works in machine translation (Prates et al., 2018) and dialogue systems (Henderson et al., 2018) have brought to attention the perpetuation of biases in natural language generation (NLG) systems.”
BackgroundExamining Gender and Race Bias in Two Hundred Sentiment Analysis Systems
Kiritchenko, S., & Mohammad, S. (2018) · Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/s18-2005 · Cited by 384 · cited 1×
“- - Regard Although sentiment has been used as a proxy for bias in other works (Lu et al., 2018; Kiritchenko and Mohammad, 2018), there has been little analysis on the correlation of sentiment to human judgment of bias.”
Method
Cited by
Sorted by citations
Items 301–400 of 439
Ducel, F., Névéol, A., & Fort, K. (2024) · Language Resources and Evaluation · doi:10.1007/s10579-024-09780-6 · Cited by 5
“Given a pretrained language model capable of generating text in inflected languages, prompt templates are created to trigger the generation of a cover letter.”
Use of AI in family medicine publications: a joint editorial from journal editors
Schrager, S., Seehusen, D. A., Sexton, S. M., Richardson, C. R., Neher, J. O., Pimlott, N., Bowman, M. A., Rodríguez, J. E., Morley, C. P., Li, L., et al. (2025) · Family Medicine and Community Health · doi:10.1136/fmch-2024-003238 · Cited by 4
Detecting Bias in Large Language Models: Fine-Tuned KcBERT
Lee, J., & Chung, T. (2025) · Lecture notes in computer science · doi:10.1007/978-981-97-8705-0_6 · Cited by 5
Axolotl: Fairness through Assisted Prompt Rewriting of Large Language Model Outputs
Ebrahimi, S., Chen, K., Asudeh, A., Das, G., & Koudas, N. (2024) · IEEE International Conference on Knowledge Graph (ICKG) · doi:10.1109/ickg63256.2024.00017 · Cited by 4
Salutari, F., Ramos, J., Rahmani, H. A., Linguaglossa, L., & Lipani, A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-33374-3_42 · Cited by 3
“Significant research efforts have been made to identify race and gender bias embedded in large models [1,20,5,33,26].”
Uncovering and Categorizing Social Biases in Text-to-SQL
Liu, Y., Yan, G., Zhe, S., Chen, X., Ash, E., & Lou, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.759 · Cited by 3
Language Detoxification with Attribute-Discriminative Latent Space
Kwak, J. M., Kim, M., & Hwang, S. J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.565 · Cited by 9
Selitskiy, S., Inoue, C., Schetinin, V., & Jakaite, L. (2023) · International Conference on Social Networks Analysis, Management and Security (SNAMS) · doi:10.1109/snams60348.2023.10375471 · Cited by 4
A multifaceted approach to detect gender biases in Natural Language Generation
Consuegra‐Ayala, J. P., Martínez-Murillo, I., Lloret, E., Moreda, P., & Palomar, M. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112367 · Cited by 3
Constructing Highly Inductive Contexts for Dialogue Safety through Controllable Reverse Generation
Zhang, Z., Cheng, J., Sun, H., Deng, J., Mi, F., Wang, Y., Shang, L., & Huang, M. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.270 · Cited by 9
How people talk about each other: Modeling Generalized Intergroup Bias and Emotion
Govindarajan, V. S., Atwell, K., Sinno, B., Alikhani, M., Beaver, D. I., & Li, J. J. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.183 · Cited by 4
“Currently, most work studying bias in NLP situates bias as negative or pejorative language use towards an individual or group based on traits like race, gender, etc (Kaneko and Bollegala, 2019; Sheng et al., 2019; Sap et al., 2020; Webson et al., 2020; Pryzant et al., 2020; Sheng et al., 2020).”
Nano: Nested Human-in-the-Loop Reward Learning for Few-shot Language Model Control
Fan, X., Lyu, Y., Liang, P. P., Salakhutdinov, R., & Morency, L.-P. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.758 · Cited by 3
“Unconditional language models have been shown to perpetuate undesirable stereotypes during generation which disproportionately harm underrepresented social groups (Liang et al., 2020; Ravfogel et al., 2020; Sheng et al., 2020, 2019).”
BackgroundDon't Go Far Off: An Empirical Study on Neural Poetry TranslationPreprint
Chakrabarty, T., Saakyan, A., & Muresan, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.02972 · Cited by 12
Stars, Stripes, and Silicon: Unravelling the ChatGPT’s All-American, Monochrome, Cis-centric Bias
Torrielli, F. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-74630-7_19 · Cited by 3
“As of 2023, cognitive science has identified innumerable cognitive biases [7,8,32,47,55] which can lead to flawed reasoning, irrationality, and potentially harmful consequences [49].”
Improving ethical sensitivity for ethical decision-making in conversational artificial intelligence
Kyungsun, Y., & Ahn, S. (2025) · Discover Computing · doi:10.1007/s10791-025-09520-6 · Cited by 2
“NLP models sometimes generate content that is prejudiced against specific religious beliefs and practices [7].”
ImageBiTe: A Framework for Evaluating Representational Harms in Text-to-Image Models
Morales, S., Clarisó, R., & Cabot, J. (2025) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain66642.2025.00019 · Cited by 2
Gender Trouble in Language Models: An Empirical Audit Guided by Gender Performativity Theory
Hafner, F. S., Valdivia, A., & Rocher, L. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732112 · Cited by 7
“For example, researchers have used large dictionaries of ‘bias probes’ and ‘prefix templates’ to elicit biased sentence completions [40, 41].”
Gender bias in machine learning: insights from official labour statistics and textual analysis
Mastromichalakis, O. M., Filandrianos, G., Symeonaki, M., Stamatopoulou, G., Parsanoglou, D., & Stamou, G. B. (2025) · Quality & Quantity · doi:10.1007/s11135-025-02261-0 · Cited by 2
“Similarly, Sheng et al. (2019) examined the generation of biassed outputs in NLP systems, emphasising the role of stereotypes in shaping the portrayal of gendered professions.”
Cascading Adversarial Bias from Injection to Distillation in Language Models
Chaudhari, H., Hayes, J., Jagielski, M., Shumailov, I., Nasr, M., & Oprea, A. (2025) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3719027.3765122 · Cited by 2
AI-Driven Fairness Testing of Large Language Models: A Preliminary Study
Romero-Arjona, M., Parejo, J. A., Alonso, J. C. G., Sánchez, A. B., Arrieta, A., & Segura, S. (2025) · IEEE International Conference on Software Analysis, Evolution and Reengineering - Companion (SANER-C) · doi:10.1109/saner-c66551.2025.00011 · Cited by 2
“Template-based methods use pre-designed templates to evaluate the model capabilities through controlled input variations [9], [22].”
Discrimination DRIFT: A Practical Framework for Responsible AI
Wyer, S. (2026) · AI and Ethics · doi:10.1007/s43681-026-01211-2 · Cited by 2
“The issue extends beyond facial recognition to other domains, including criminal justice and natural language processing [57, 133, 146].”
Ahmadzadeh, S., Karthick, G., & Alsafi, T. (2026) · Future Internet · doi:10.3390/fi18080393 · Cited by 6
Generating Explanations for Recommendation Systems via Injective VAE
Cai, Z. (2021) · Proceedings · doi:10.1109/icdm51629.2021.00115 · Cited by 2
Unmasking the Stereotypes: Evaluating Social Biases in Chinese BERT
Zhang, S., & Li, P. (2022) · 2022 4th International Conference on Natural Language Processing (ICNLP) · doi:10.1109/icnlp55136.2022.00059 · Cited by 2
“In addition, bias has been found in machine translation [10][11][12], emotion analysis [13], speech recognition [14], recommendation system [15], language generation [16] and other fields.”
BackgroundEntity-Based Evaluation of Political Bias in Automatic Summarization
Zhou, K., & Tan, C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.696 · Cited by 5
“Many studies have covered representation bias and social bias (e.g., gender and race) (Steen and Markert, 2023; Sheng et al., 2019; Liang et al., 2021; Yeo and Chen, 2020; Garimella et al., 2021; Dhamala et al., 2021).”
BackgroundPrien, T., & Goldhammer, K. (2024) · Springer eBooks · doi:10.1007/978-3-658-34048-3_89-1 · Cited by 5
Mehrotra, S., Raghavendra, P., Prajesh, B., Kambale, H., & Majumdar, P. (2025) · ACM International Conference on AI in Finance (ICAIF) · doi:10.1145/3768292.3770355 · Cited by 4
“Sheng et al. [27] introduced the ‘Regard’ metric by creating a bias evaluation dataset with manually annotated sentiment and regard scores, and contributed a Regard Classifier for evaluation of bias in unseen text.”
Machlovi, N., Saleki, M., Ababio, I. B., & Amin, R. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-13184-3_24 · Cited by 2
“Therefore, it has become evident that moderation techniques are essential for regulating interactions between humans and LLMs [41,33].”
Costa‐jussà, M. R., Andrews, P., Smith, E., Hansanti, P., Ropers, C., Kalbassi, E., Gao, C., Licht, D. J., & Wood, C. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.874 · Cited by 18
"Nice Try, Kiddo": Investigating Ad Hominems in Dialogue ResponsesPreprint
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.12820 · Cited by 2
“Sheng et al. (2019) propose metrics to evaluate societal biases in language generation systems and Curry and Rieser (2018) study how conversational systems respond to sexual harassment.”
Self-citationPrien, T., & Goldhammer, K. (2024) · Springer eBooks · doi:10.1007/978-3-658-39909-2_89 · Cited by 2
Master of Deceit: Comparative Analysis of Human and Machine-Generated Deceptive Text
Trinh, Q. M., Zarin, S., & Rezapour, R. (2025) · ACM Web Science Conference (WebSci) · doi:10.1145/3717867.3717914 · Cited by 1
“This aligns with prior research suggesting that when given neutral or professional prompts, LLMs tend to generate more balanced gender representation [16, 48].”
Towards Fair Rankings: Leveraging LLMs for Gender Bias Detection and Measurement
Mousavian, M., Abbasiantaeb, Z., Aliannejadi, M., & Crestani, F. (2025) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3731120.3744620 · Cited by 1
“In NLP, gender bias has been widely studied, with research focusing on its presence in embeddings [10, 19, 30], LLM [28], and downstream applications such as machine translation [31], question answering [27, 37], text generation [44], and Natural Language Inference (NLI) [9].”
Evaluating an AI Tutor for Bias Across Different Foundation Models
Vinodh, A., Harvey, E., Almoubayyed, H., Yu, R., Brooks, C. A., Koenecke, A., & Kizilcec, R. F. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-98465-5_43 · Cited by 1
Query-efficient and dataset-independent red teaming for LLMs content safety evaluation
Liu, S., Cheng, X., & Su, S. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114404 · Cited by 2
SafeText: A Unified Approach for Detecting and Mitigating Toxicity and Bias in Textual Data
Agarwal, A., Bera, A., & De, T. (2025) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-032-05832-4_5 · Cited by 2
Rivera, J., Zapata, S., Pizarro, R., & Keith, B. (2025) · Knowledge · doi:10.3390/knowledge5040025 · Cited by 1
Homophily-induced emergence of biased structures in LLM-based multi-agent AI systems
Mehdizadeh, A., & Hilbert, M. (2025) · Social Network Analysis and Mining · doi:10.1007/s13278-025-01535-7 · Cited by 4
“Recent studies suggest that models that are trained on large-scale web data often encode and reproduce gender stereotypes, which may shape how they represent social roles, occupations, and interpersonal dynamics [70, 71, 72, 73, 74].”
Religion and Racial Bias in Artificial Intelligence Large Language Models
Tom, J. C., Ferguson, T. W., & Martinez, B. C. (2025) · Socius Sociological Research for a Dynamic World · doi:10.1177/23780231251377210 · Cited by 1
“LLMs have been shown to produce stereotyped content on the basis of race, gender, and sexuality, even when given ordinary user prompts (Acerbi and Stubbersfield 2023; Fang et al. 2024; Sheng et al. 2019).”
Towards a Fuzzy MCDM-Based Framework for Multi-Metric Bias Auditing in Generative AI
Ravikumar, R., Rajan, A. V., Narayan, A., & Maheshwari, P. (2025) · International Conference on Information Technology Trends (ITT) · doi:10.1109/itt69610.2025.11352932 · Cited by 1
“However, the open-ended and context-dependent Gen-AI outputs are more complex to evaluate for fairness and are dependent on multi-metric auditing frameworks and benchmark datasets such as StereoSet, CrowS-Pairs, and WinoBias [11], [12], [13].”
Inference Time Debiasing Concepts in Diffusion Models
Kupssinskü, L. S., Bochernitsan, M. N., Kopper, J., Parraga, O., & Barros, R. C. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-15984-7_6 · Cited by 1
When Algorithms Favor the Underrepresented – Race and Gender Biases in LLM Résumé Evaluations
Kim, S. C., Hernandez, I., & Li, J. (2026) · Journal of Personnel Psychology · doi:10.1027/1866-5888/a000388 · Cited by 2
Measuring Gender Bias in Word Embeddings for Russian Language
Pestova, A. S. (2021) · Computational Linguistics and Intellectual Technologies · doi:10.28995/2075-7182-2021-20-1151-1160 · Cited by 1
“Different types of models, regardless of the task and training methods, demonstrate social biases in terms of gender, race, and religion [24, 36].”
BackgroundMorabito, R., Kabbara, J., & Emami, A. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.280 · Cited by 1
Analyzing the Limits of Self-Supervision in Handling Bias in Language
Bauer, L., Gopalakrishnan, K., Gella, S., Liu, Y., Bansal, M., & Hakkani-Tür, D. Z. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.545 · Cited by 2
“There has been a large body of work focused on defining and measuring social bias during natural language generation (Sheng et al., 2019; Nadeem et al., 2021; Dev et al., 2021), neural toxic degener-”
BackgroundInfluentialShaier, S., Bennett, K., Hunter, L., & Kann, K. (2023) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2023.ijcnlp-main.36 · Cited by 4
All Should Be Equal in the Eyes of LMs: Counterfactually Aware Fair Text Generation
Banerjee, P., Java, A., Jandial, S., Shahid, S., Furniturewala, S., Krishnamurthy, B., & Bhatia, S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29719 · Cited by 4
“For each model-pair, we randomly sample 12 prompts from NLG-bias dataset (Sheng et al. 2019) and ask three questions per prompt: which of the two sentences (i) is more biased (ii) has more negative sentiment , and, (iii) is more fluent , i.e 36 questions = 12 prompts × 3 .”
MethodPsychological AI: A critical analysis of capabilities, limitations, and ramifications
Wang, B. (2024) · Applied and Computational Engineering · doi:10.54254/2755-2721/53/20241373 · Cited by 1
EnronSR: A Benchmark for Evaluating AI-Generated Email Replies
Shay, M., Davidson, R., & Grinberg, N. (2024) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v18i1.31448 · Cited by 3
“It has been shown that language models generate stereotypical occupations and varying levels of respect for different genders and races (Sheng et al. 2019; Kirk et al. 2021; Lucy and Bam-man 2021).”
BackgroundMeasuring Bias in Generated Text Using Language Models—GPT-2 and BERT
Masuma, F., Chakraborty, P., Islam, A.-A.-U., Talukder, P. C., Roy, P., & Yousuf, M. A. (2024) · Lecture notes in networks and systems · doi:10.1007/978-981-97-0892-5_39 · Cited by 1
Clustering Discourses: Racial Biases in Short Stories about Women Generated by Large Language Models
Bonil, G., Gondim, J., dos Santos, M., Hashiguti, S. T., Maia, H., da Silva, N. F. F., Pedrini, H., & Avila, S. (2025) · Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana (STIL) · doi:10.5753/stil.2025.37814 · Cited by 2
“Our work also contributes to the still-scarce literature on Portuguese-language LLM outputs, an aspect rarely explored outside [Assi and Caseli 2024].”
Prompt Fairness: Sub-group Disparities in LLMs
Zhong, M., Teku, N., & Tandon, R. (2025) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/ieeeconf67917.2025.11443738 · Cited by 1
“More recently, fairness concerns have extended to large generative and language models, including bias measurement [17] and mitigation techniques spanning data interventions, model regularization, and prompt-based strategies [18]–[20].”
A Human-machine Interface for Few-shot Rule Synthesis for Information Extraction
Vacareanu, R., Barbosa, G. C. G., Noriega-Atala, E., Hahn-Powell, G., Sharp, R., Valenzuela-Escárcega, M. A., & Surdeanu, M. (2022) · doi:10.18653/v1/2022.naacl-demo.8 · Cited by 1
“In other words, previous work is much more opaque and thus more difficult to interpret, debug, adjust, maintain, and protect from hidden biases present in the training data (e.g., Kurita et al., 2019; Sheng et al., 2019).”
BackgroundGestión de la identidad en la prevención de medios de vigilancia social
GALLEGOS, D. (2023) · Revista de Investigación Académica Sin Frontera División de Ciencias Económicas y Sociales · doi:10.46589/rdiasf.vi39.550 · Cited by 1
A user-centred approach to information retrieval
Aloteibi, S. (2020) · Apollo (University of Cambridge) · doi:10.17863/cam.77137 · Cited by 1
A Primer on ChatGPT: Coherence Does Not Imply CorrectnessPreprint
Ubani, S. (2023) · EdArXiv (OSF Preprints) · doi:10.35542/osf.io/8fmrn · Cited by 1
Dewitte, P. (2024) · doi:10.31219/osf.io/gdt3z · Cited by 1
Use of AI in Family Medicine Publications: A Joint Editorial From Journal Editors
Schrager, S., Seehusen, D. A., Sexton, S. M., Richardson, C. R., Neher, J. O., Pimlott, N., Bowman, M. A., Rodríguez, J. E., Morley, C. P., Li, L., et al. (2025) · PRiMER · doi:10.22454/primer.2025.889328 · Cited by 1
Enhancing Gender-Neutral Language with LoRA Fine-Tuning of DeepSeek-R1 for Offline AI ApplicationsPreprint
Chintalapudi, R., Perugu, D., Andra, R., Gorrepati, N., Kumar, A., & Sakhamuri, S. B. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5225651 · Cited by 1
Writing with AI Can Reduce Gender Bias in Hiring Evaluations
Liu, A. T. H., Lee, M., & Bai, X. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791136
“Existing Biases in LLMs. AI writing tools may inadvertently worsen gender biases as AI systems often reflect and spread the biases found in their training data [175, 200].”
Understanding Natural Language with Commonsense Knowledge Representation, Reasoning, and Simulation
Bosselut, A. (2020) · ResearchWorks at the University of Washington (University of Washington)
Retos y desafíos de las universidades 2020-2030
Arbeláez-Campillo, D. F., & Rojas-Bahamón, M. J. (2021) · Revista Amazonia Investiga · doi:10.34069/ai/2021.38.02.0
Towards controllable neural generation of arguments
Hua, X. (2021) · Digital Repository Service (Northeastern University) · doi:10.17760/d20412319
On factuality in neural language models
Nadeem, M. (2021)
Defences and threats in safe deep learning
Chan, A. (2021) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/152976
Adversarial Scrubbing of Demographic Information for Text ClassificationPreprint
Chowdhury, S. B. R., Ghosh, S., Li, Y., Oliva, J. B., Srivastava, S., & Chaturvedi, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.08613 · Cited by 4
“…(Park et al., 2018; Stanovsky et al., 2019; Font and Costa-Jussa, 2019; Saunders and Byrne, 2020), NLI (Rudinger et al., 2017), text classification (Dixon et al., 2018; Kiritchenko and Mohammad, 2018; Sap et al., 2019; Liu et al., 2021), language generation (Sheng et al., 2019) among others.”
BackgroundStyle Control for Schema-Guided Natural Language GenerationPreprint
Tsai, A. Y., Oraby, S., Perera, V., Kao, J.-Y., Du, Y., Narayan-Chen, A., Chung, T., & Hakkani-Tür, D. Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.12211 · Cited by 4
“Language generation systems that use corpora and methods without awareness of emotions may generate callous, generic or even biased responses (Bender et al., 2021; Sheng et al., 2019).”
BackgroundHandouts don’t exist. Hustle or you don’t eat.
McGarrigle, C. (2021) · Media-N · doi:10.21900/j.median.v17i2.764
CFL: Causally Fair Language Models Through Token-level Attribute Controlled Generation
Madhavan, R., Garg, R., Wadhawan, K., & Mehta, S. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.720 · Cited by 4
“The twin issues of toxicity and bias in text generation are important challenges to such deployment (Holtzman et al., 2019; Bender et al., 2021; McGuffie and Newhouse, 2020; Sheng et al., 2019; Fiske, 1993).”
BackgroundCounterfactual Probing for the Influence of Affect and Specificity on Intergroup Bias
Govindarajan, V. S., Beaver, D. I., Mahowald, K., & Li, J. J. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.813 · Cited by 1
“…features: specificity (Li, 2017) is a pragmatic feature of text that measures the level of detail (similar to abstract–concrete axis), while affect is a feature that measures the attitude of a speaker towards their target (Sheng et al., 2019) in an utterance (analogous to social desirability).”
BackgroundMethodInfluentialNaturalAdversaries: Can Naturalistic Adversaries Be as Effective as Artificial Adversaries?
Gabriel, S., Palangi, H., & Choi, Y. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.413
Measuring bias in Instruction-Following models with P-AT
Onorati, D., Ruzzetti, E. S., Venditti, D., Ranaldi, L., & Zanzotto, F. M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.539
“…of models, the underlying LLMs generate toxic or offensive content (Gehman et al., 2020), and reproduce biases that exist in the training data (Sheng et al., 2019; Kurita et al., 2019; Ranaldi et al., 2023): it is not clear if and to what extent Instruction-Following models can exhibit the…”
BackgroundHarnessing Black-Box Control to Boost Commonsense in LM’s Generation
Tian, Y., Zhang, F., & Peng, N. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.329 · Cited by 1
Self-citationKako „razmišljaju“ veliki jezični modeli i možemo li im vjerovati
Dobša, J. (2023) · Politehnika · doi:10.36978/cte.7.2.2
LangBiTe: An open-source platform to automate bias testing of large language models
Morales, S., Clarisó, R., & Cabot, J. (2025) · SoftwareX · doi:10.1016/j.softx.2025.102248
“However, since they are generally trained on large datasets grounded on web crawls, they often overlook ethical aspects and tend to reflect biases that are present on the internet [1,2,3,4,5,6].”
BackgroundMore Distinctively Black and Feminine Faces Lead to Increased Stereotyping in Vision-Language ModelsPreprint
Lee, M. H. J., Montgomery, J., & Lai, C. K. (2024) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/wvxy2 · Cited by 2
Generalization and Fairness Optimization in Pretrained Language Models
Ghanbarzadeh, S. (2024) · doi:10.12794/metadc2332571
Assessing Bias in Self-supervised and Supervised Models for Temporal Data
Liatsikou, M., Yfantidou, S., & Vakali, A. I. (2024) · Hellenic Conference on Artificial Intelligence · doi:10.1145/3688671.3688753 · Cited by 1
“However, bias evaluations in SSL have been more prevalent in CV [28] and NLP, including recent advances in generative models [33].”
Use of AI in Family Medicine Publications: A Joint Editorial From Journal Editors
Schrager, S., Seehusen, D. A., Sexton, S. M., Richardson, C. R., Neher, J. O., Pimlott, N., Bowman, M. A., Rodríguez, J. E., Morley, C. P., Li, L., et al. (2025) · The Annals of Family Medicine · doi:10.1370/afm.240575
Debiasing Large Language Models with Structured Knowledge
Ma, C., Tianyu, Z. H. A. O., & Okumura, M. (2025) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2024edp7326
Berlincioni, L., Cultrera, L., Becattini, F., Bertini, M., & Bimbo, A. D. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-92648-8_7 · Cited by 1
An Adversarial Toxicity Prompt Generator Exploiting Multilingual Code-Switching in LLMs
Wadgaonkar, V. S., & Ratha, N. (2025) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai64502.2025.00156
“Static toxicity datasets become obsolete rapidly due to evolving slang, adversarial tactics, and the emergence of new toxic expressions [1].”
Advanced NLP Techniques for Enhancing ChatGPT Performance
Roy, B., Roy, P., Ghosh, S., Paul, T., & Paul, S. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-9841-8.ch001
Evaluating Generative AI in Education: LLM as English Tutor
Vargas, H. J., Pereira, R. L. C. S., de Moraes, A. F., & Silva, L. A. (2025) · Advances in intelligent systems and computing · doi:10.1007/978-3-031-99474-6_40 · Cited by 1
Ogbuokiri, B., Obaido, G., Kamalu, C., Aruleba, K., Achilonu, O., Mienye, I. D., Echezona, S. C., Ujah-Ogbuagu, B. C., & Seyyed-Kalantari, L. (2025) · Machine Learning with Applications · doi:10.1016/j.mlwa.2025.100717 · Cited by 2
de Sena, R. L. R., Souza, M., Santana, A. S., & de Melo, J. L. L. (2025) · Anais do Workshop sobre as Implicações da Computação na Sociedade (WICS) · doi:10.5753/wics.2025.9056
Yeganegi, M. R., Hassani, H., & Komendantova, N. (2025) · Information · doi:10.3390/info16080679
Enhancing implicit hate speech detection via LLM-generated adversarial samples
Zhang, L., Deng, H., Gao, W., Yu, Y., & Xu, R. (2025) · International Journal of Systems Assurance Engineering and Management · doi:10.1007/s13198-025-02909-y
Towards Sustainable Image Synthesis: A Comprehensive Review of Text-to-Image Generation Models
Bharne, S., Sapkale, P., Sarda, E., Padiya, P., & Salunkhe, S. (2025) · International Research Journal of Multidisciplinary Technovation · doi:10.54392/irjmt2557
Tiwari, A., & Kumar, P. (2025) · IGI Global eBooks · doi:10.4018/979-8-3373-0877-7.ch008
Preliminary Ranking-Based Selection for Optimized Retriever Configuration in RAG Systems
Paranhos, S. L., Tomazini, J. N., de Oliveira, S. T., Camilo-Junior, C. G., & de Oliveira, S. S. T. (2025) · Simpósio Brasileiro de Bancos de Dados (SBBD) · doi:10.5753/sbbd.2025.247492
“This not only creates barriers to adoption for non-expert organizations but also risks propagating misinformation, reinforcing biases, and failing to retrieve critical information in high-stakes domains [Bender et al., 2021, Sheng et al., 2019].”
Towards Trustworthy AI: Enhancing Factuality, Bias, and Compliance in LLMs
Amatya, P. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761664
“Sheng et al. [17] highlight gender and racial stereotypes in generated content.”
BharatBBQ: A Multilingual Bias Benchmark for Question Answering in the Indian Context
Tomar, A., Sahoo, N. R., & Bhattacharyya, P. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.55
Mitigating Bias in Large Language Models Associated with Occupational Stereotypes
Chakraborty, C., & Howard, A. (2025) · IEEE International Symposium on Technology and Society (ISTAS) · doi:10.1109/istas65609.2025.11269614
“Research should analyze these effects and promote best practices for bias-aware prompting [12].”
Acharya: A Multimodal BERT & CNN and LLM Agent Architecture in Job Matching for Ethical Candidate
Sekar, R. R., Rao, P. V., Ruthik, P., Sumanth, P., & Reddy, B. M. K. (2025) · International Conference on Innovative Data Communication Technologies and Application (ICIDCA) · doi:10.1109/icidca66325.2025.11280422
“According to their results, widely used models such as SpanBERT and CorefQA scored 10–15% lower on gender-neutral pronouns than on gendered pronouns, indicating a huge bias against inclusive language, Sheng et al. [15] introduced CrowS-Pairs to quantify bias in LMs and demonstrated that GPT-2 (67”
Exacerbating Differences in Polarity: Bias Adversarial Attack on Generative Large Language Models
Gu, T., Qing, G., Hao, F., & Chang, L. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3645995
“Metrics: We utilize three metrics for evaluating bias: (1) Regard score [30], which measures the semantic polarity of the generated text with respect to the group through the regard classifier.”
InfluentialChen, C., Wu, X., Zhuang, W., Wang, J., Bao, J., & Li, X. (2026) · Journal of Engineering Design · doi:10.1080/09544828.2025.2609041
Metadata
- Authors
- Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N.
- Affiliations
- University of Southern California (US)
- University of California, Los Angeles (US)
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 3405-3410
- OpenAlex
- W2971307358
- Cited by
- 537
- References
- 17
- Citations by year
- 2026: 79 · 2025: 94 · 2024: 62 · 2023: 106 · 2022: 81 · 2021: 85 · 2020: 28 · 2019: 2
- FWCI
- 15.99
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Funders
- Defense Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- gender bias, gender stereotypes, gendered language
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-23, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-23 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
