The DOI matches its Crossref record; 26 of the 35 references it cites were found in a registry or an index; 9 of the 17 sources its claims cite were checked for support.
StereoSet: Measuring stereotypical bias in pretrained language models
Open accessAbstract
Moin Nadeem, Anna Bethke, Siva Reddy. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
StereoSet introduces a large-scale natural English benchmark of 16,995 Context Association Tests across gender, profession, race, and religion to measure stereotypical bias alongside language modeling ability in pretrained language models. It finds that popular models such as BERT, GPT2, ROBERTA, and XLNET show strong stereotypical biases, and that stronger language modeling correlates with greater bias (Spearman ρ = 0.87).
Key findings
- StereoSet contains 321 target terms and 16,995 test instances (triplets): 8,498 intrasentence CATs and 8,497 intersentence CATs across four domains.
- All pretrained models evaluated have higher language modeling scores (lms) than RANDOMLM but also exhibit more stereotypical behavior than RANDOMLM (ss = 50). In Table 4, GPT2-large reaches lms = 89.6 and ss = 62.7; ROBERTA-base has ss = 58.5.
- Language modeling ability and stereotype score are strongly correlated: Spearman rank correlation ρ = 0.87. The ENSEMBLE model is reported as the most biased model (ss = 62.5 in the text; 62.2 in Table 4).
- SENTIMENTLM achieves a high stereotype score (ss = 60.8) but a lower language modeling score (lms = 65.1), indicating sentiment can help detect stereotypical associations but is insufficient for distinguishing meaningful from meaningless sentences.
Methods
- The paper designs the Context Association Test (CAT) with two formats: intrasentence fill-in-the-blank and intersentence discourse-level association. Each instance includes a target context and three associations: stereotype, anti-stereotype, and meaningless.
- Data were crowdsourced via Amazon Mechanical Turk from workers in the United States. Target terms were curated from Wikidata relations for profession, race, and religion, plus gender terms. Each CAT was written by one worker and validated by five others; CATs were retained when at least three validators agreed on labels, yielding 83% of collected CATs.
- Models evaluated include BERT, ROBERTA, XLNET, and GPT2. Evaluation uses likelihood-based and pseudo-likelihood-based scoring, with metrics: language modeling score (lms), stereotype score (ss), and idealized CAT score (icat). For intersentence tasks, Next Sentence Prediction heads were trained for models lacking them.
Limitations
- The paper states that evaluating full language modeling ability is beyond its scope, that StereoSet has no training set by design, and that neutral associations were discarded because crowdworkers had low inter-annotator agreement and often produced implicitly stereotypical sentences.
- The paper states that achieving ideal StereoSet performance does not guarantee a model is unbiased, that icat is not intended as the sole criterion for model selection, and that further research is needed; it also notes lack of computing resources to train the same model on increasing amounts of corpora.
- Evident from the text: the dataset is US English and collected only from US crowdworkers, so the stereotypes are tailored to the United States; it covers only four domains; validation retains CATs with as few as three of five agreeing validators; and some reported numbers differ between the narrative and tables (e.g., GPT2-large lms/ss and ENSEMBLE ss), which may complicate interpretation.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
34 Own result · 8 Own interpretation · 6 Partly supported · 6 Own method · 5 No basis given · 1 Supported · 1 Supported so far · 1 Sources not found · 1 No full text of the sources · accuracy not yet measured
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“they mainly evaluate bias of pretrained language models on a small set of artificial sentences, even though these models are trained on natural data”
§ StereoSet: Measuring stereotypical bias in pretrained language models > Abstract
What the sources say
- Not checked — no full text of the sourcecites May et al., 2019 source
- Partly supportedcites Kurita et al., 2019 sourceExperimental — accuracy not yet measured
The Introduction and Table 1 describe evaluating BERT bias with a small set of simple template sentences, and the paper discusses an earlier sentence-embedding bias test. But it does not establish that existing bias evaluations mainly use artificial sentences; it also examines bias on a Wikipedia-derived pronoun-resolution dataset. Page numbers are unavailable in our copy. (LLM, conf=0.78)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Third, they focus on single word terms and ignore multiword terms like construction worker.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 1 Introduction
What the sources say
- Not checked — no full text of the sourcecites May et al., 2019 source
- Partly supportedcites Kurita et al., 2019 sourceExperimental — accuracy not yet measured
The Introduction (page unavailable) describes this paper’s bias measure as querying BERT for a particular token and illustrates it with single-word targets and attributes. It does not establish that contextual-embedding bias studies generally ignore multiword terms, or specifically discuss “construction worker.” (LLM, conf=0.78)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“current evaluations focus on measuring bias without considering the language modeling ability of a model, which could lead to misleading trust on a model even if it is a poor language model.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > Abstract
What the sources say
- Not checked — no full text of the sourcecites May et al., 2019 source
- Partly supportedcites Kurita et al., 2019 sourceExperimental — accuracy not yet measured
The Introduction and §5 Real World Implications discuss existing bias-evaluation methods and say they can give inconsistent or unclear results for contextualized embeddings. But the paper does not say those evaluations neglect a model’s language-modeling ability, or that they could inspire misplaced trust in a poor language model. Its proposed method uses masked-token probabilities to measure bias, not to assess overall language-modeling quality. (LLM, conf=0.78)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Lastly, they study bias of a model independent of its language modeling ability which could lead to undeserved trust in a model if it is a poor language model.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 1 Introduction
What the sources say
- Not checked — no full text of the sourcecites May et al., 2019 source
- Partly supportedcites Kurita et al., 2019 sourceExperimental — accuracy not yet measured
The Introduction describes measuring BERT’s bias using masked-token predictions and normalizing those predictions against a prior probability. It does not establish that the bias measure is independent of language-modeling ability or discuss undeserved trust in a poor language model. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“First, the context does not reflect the natural usage of a word.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 1 Introduction
What the sources say
- Not checked — no full text of the sourcecites May et al., 2019 source
- Partly supportedcites Kurita et al., 2019 sourceExperimental — accuracy not yet measured
The Introduction (page unavailable) says the authors create simple template sentences to measure bias, supporting the claim that the study uses constructed contexts. It does not say that those contexts fail to reflect a word’s natural usage; that criticism is not established by this source. (LLM, conf=0.79)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Second, they require stereotypical attribute terms to be predefined (e.g., pleasant and unpleasant terms).”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 1 Introduction
What the sources say
- Not checked — no full text of the sourcecites May et al., 2019 source
- Supportedcites Kurita et al., 2019 source
Section 3.2 says the authors test BERT using specified sets of target and attribute words, including pleasant and unpleasant attributes. The Introduction also says their method requires constructing template sentences containing attribute words of interest. These passages support the claim that the bias tests described in this paper use predefined attribute terms. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While GPT2-large is the most stereotypical model of all pretrained models (60.1),ROBERTA-base is the least stereotypical model (50.5).”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 7 Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“SENTIMENTLM achieves the highest stereotypical score compared to all pretrained models, indicating that sentiment can indeed be exploited to detect stereotypical associations.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 7 Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These keywords indicate that target terms in gender and race are associated with physical attributes such as beautiful, feminine, masculine, etc.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 4.4 Dataset analysis
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As evident in Table 2, people do not always associate stereotypes with negative associations (e.g., Asians are good at math has positive sentiment).”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 4.4 Dataset analysis
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 17
- It may state two separate claims as one. 14
Every source the paper cites for this claim was checked and supports it.
In the paper
“The seminal works of Bolukbasi et al. (2016) and Caliskan et al. (2017) show that word embeddings such as word2vec (Mikolov et al., 2013) and GloVe (Pennington et al., 2014) contain stereotypical biases using diagnostic methods like word analogies and association tests.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 1 Introduction
What the sources say
- Supportedcites Bolukbasi et al. (2016) source
Section 4 (pp. 6–7) compares word2vec and GloVe embeddings, finds gender bias in occupation-word associations across both, and describes using analogies to identify stereotypes. This supports the claim attributed to Bolukbasi et al.; the sentence also cites another work, so it need not establish that Bolukbasi et al. used every diagnostic method listed. (LLM, conf=0.94)
- Supportedcites Caliskan et al. (2017) source
The Results (p. 3) report human-like biases in GloVe using association-based tests. Data and training (p. 9) says the authors repeated the tests with word2vec and found statistically significant effects in every experiment. (LLM, conf=0.94)
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Moreover, Aribandi et al. (2021) has shown that psuedo-likelihood has higher variance than likelihood scoring.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > Impact of scoring mechanism
What the sources say
- Partly supportedcites Aribandi et al. (2021) sourceExperimental — accuracy not yet measured
Section 3.1 reports that CrowS-Pairs has far higher variance than StereoSet. Section 2.2 identifies CrowS-Pairs’ pseudo-likelihood as one possible reason, but also cites its smaller dataset and greater number of categories. The paper does not establish that pseudo-likelihood generally has higher variance than likelihood scoring. (LLM, conf=0.86)
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This filtering results in selecting 83% of the CATs, indicating that there is regularity in stereotypical views among the workers.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 4.3 CATs validation and human agreement
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Among models, GPT2-large is the best performing language model (88.3) followed by GPT2-medium (85.9).”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 7 Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This aligns with expectations and indicates that multiple annotators use similar attributes.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 4.4 Dataset analysis
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We crowdsource Stere-oSet, a dataset for associative contexts in English containing 4 target domains, 321 target terms and 16,995 test instances (triplets).”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 1 Introduction
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As the model size increases, we see that its language modeling ability (lms) increases, and correspondingly its stereotypical score.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 7 Results and discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our NSP classifier achieves an 94.6% accuracy with ROBERTA-base, a 97.1% accuracy with ROBERTA-large, a 93.4% accuracy with XLNET-base and 94.1% accuracy with XL-NET-large.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > Additional information
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our NSP classifier obtains a 92.5% accuracy on GPT2-small, 94.2% on GPT2-medium, and 96.1% on GPT2-large.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > Additional information
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our fine-tuned model achieves an 92% test accuracy on the Large Movie Review dataset.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > A.2 Fine-Tuning BERT for Sentiment Analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Our results are comparable to Devlin et al. (2019), with each model obtaining 93-98% accuracy against the test set of 3.5 million examples.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > Additional information
What the sources say
- Not checked — no full text of the sourcecites Devlin et al. (2019) source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In practice, we found that collecting neutral contexts are prone to implicit biases and has low inter-annotator agreement (Section 4).”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 2.2 Formulation
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The model is relatively less biased on race than on others (ss = 61.8).”
§ StereoSet: Measuring stereotypical bias in pretrained language models > Domain-wise bias
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The stereotypes elicited from US crowdworkers are representative of stereotypes in the United States.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Surprisingly, the corpora size does not correlate with either lms or ss.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > Impact of scoring mechanism
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The results exhibit similar trends on the development and test sets.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 7 Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It is important to note that the classifier also classifies neutral sentiment such as My housekeeper is a Mexican as positive.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > A.2 Fine-Tuning BERT for Sentiment Analysis
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The outlier is Muslim, although it has established stereotypes indicated by the high human agreement (see Table 10).”
§ StereoSet: Measuring stereotypical bias in pretrained language models > Domain-wise bias
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The stereotypes elicited from US crowdworkers are representative of stereotypes in the United States.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We restrict ourselves to crowdworkers in USA since stereotypes could change based on the country.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 4 Dataset Creation
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Rests on an assumption: Stereotypes could change based on the country.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“For each context and its associations, we ask five validators to classify each association into a stereotype, an anti-stereotype or a meaningless association.We only retain CATs where at least three validators agree on the labels.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 4.3 CATs validation and human agreement
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
In the paper
“Caliskan et al. show that male names are more likely to be associated with career terms than female names where the association is measured using embedding similarity.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 1 Introduction
What the sources say
- Not matchedcites Caliskan et al.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, its language model performance is worse, which is expected, since sentiment alone isn't sufficient to distinguish meaningful and meaningless sentences.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 7 Results and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We split StereoSet based on the target terms: 25% of the target terms and their instances for the development set and 75% for the test set.We ensure terms in the development set and test set are disjoint.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 5.1 Development and test sets
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, we hold that some of these neutral sentences aren't truly neutral; the chess player was bearded may inadvertently conceal stereotypes, since both chess players and bearded men are commonly seen as wise.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > A.4 Collecting Neutral Associations
hedgedconditionalSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We required all crowdworkers to be in the United States and have a HIT acceptance rate greater than 97%.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 9.1 Curation Rationale
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We collected this data via Amazon Mechanical Turk (AMT), where each example was written by one crowdworker and validated by four other crowdworkers.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 9.1 Curation Rationale
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“For all models, we train identical Next Sentence Prediction heads on identical datasets (details given in Appendix A.5), and compute the log likelihood that any given target sentence follows the context.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 6.1 Masked Language Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper relies on this without support.
Stereotypes could change based on the country.
“We restrict ourselves to crowdworkers in USA since stereotypes could change based on the country.”
The authors state this as an assumption.
Language modeling ability and stereotypical bias are of equal importance for model comparison.
“we introduce a simple point-estimate called the idealized CAT (icat) score for model comparison assuming equal importance to language modeling ability and stereotypical bias.”
The paper relies on this without support.
The real world distribution of corpora likely reflects stereotypes.
“The correlation between lms and ss is unfortunate and perhaps un- avoidable as long as we rely on the real world distribution of corpora to train language models since these corpora are likely to reflect stereotypes.”
The paper relies on this without support.
BERT has been trained with a Next Sentence Prediction classification head to provide p(s|c), while the other models have not.
“While BERT has been trained with a Next Sentence Prediction classification head to provide p(s|c), the other models have not.”
The paper relies on this without support.
The stereotypes elicited from US crowdworkers are representative of stereotypes in the United States.
“StereoSet is in English and is tailored for the stereotypes that exist in the United States.”
The paper relies on this without support.
The NSP classifier's accuracy on a Wikipedia-derived corpus means it can measure language modeling ability on StereoSet.
“Our results are comparable to Devlin et al. (2019), with each model obtaining 93-98% accuracy against the test set of 3.5 million examples.”
The paper relies on this without support.
The sentiment classifier's accuracy on movie reviews means it can classify sentiment in StereoSet.
“Our fine-tuned model achieves an 92% test accuracy on the Large Movie Review dataset.”
The paper relies on this without support.
Human agreement scores in Table 10 are a proxy for most and least biased terms.
“see Table 10 for humanagreement scores, a proxy for most and least biased terms”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“For each type of bias, the dataset presents a context sentence with three options: one with a stereotype, one with a neutral or positive connotation (”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Nadeem, Bethke, and Reddy 2021
The cited source says what the claim attributes to it.
In the citing paper
“StereoSet presents 16,995 crowdsourced instances measuring race, gender, religion, and profession stereotypes.”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Nadeem, Bethke, and Reddy 2021
The cited source says what the claim attributes to it.
In the citing paper
“StereoSet evaluates intrasentence bias within a sentence with fill-in-the-blank sentences, where the options describe a social group in the sentence context”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Nadeem, Bethke, and Reddy 2021
The cited source says what the claim attributes to it.
In the citing paper
“It measures intersentence bias between sentences in a discourse with three continuation options, where the first sentence mentions a social group.”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Nadeem, Bethke, and Reddy 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“Intersentence Gender 40 996 15.55 Profession 120 3,269 16.05 Race 149 3,989 14.98 Religion 12 604 14.99 Total 321 8,497 15.39”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 4.3 CATs validation and human agreement
Reported 8497 · recomputed 8858
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“Intrasentence Gender 40 1,026 7.98 Profession 120 3,208 8.30 Race 149 3,996 7.63 Religion 12 623 8.18 Total 321 8,498 8.02”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 4.3 CATs validation and human agreement
Reported 8498 · recomputed 8853
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ StereoSet: Measuring stereotypical bias in pretrained language models > 4.3 CATs validation and human agreement
“Religion 12 623 8.18”
§ StereoSet: Measuring stereotypical bias in pretrained language models > A.5 General Methods for Training a Next Sentence Prediction Head
“Religion Jihad (0.86) Muslim (0.84) Holy Trinity (0.81) Quran (0.8) Trinity (0.8) Sanskrit (0.8) Mecca (0.8) Islam (0.79) baptize (0.79) Mohammed (0.79) Sharia (0.78) church (0.77) Jesus (0.77) Christ (0.77) Messiah (0.76) Vishnu (0.76) Hajj (0.76) Bible (0.75) Christian (0.74) Reincarnation (0.74) Hindu (0.74) Brahmin (0.74)Ten Commandments (0.72) Shiva (0.72)”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ StereoSet: Measuring stereotypical bias in pretrained language models > 4.3 CATs validation and human agreement
“For each context and its associations, we ask five validators to classify each association into a stereotype, an anti-stereotype or a meaningless association.”
§ StereoSet: Measuring stereotypical bias in pretrained language models > 9.1 Curation Rationale
“each example was written by one crowdworker and validated by four other crowdworkers.”
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687
Implicit social cognition: Attitudes, self-esteem, and stereotypes
Greenwald, A. G., & Banaji, M. R. (1995) · Psychological Review · doi:10.1037/0033-295x.102.1.4 · Cited by 6,682
Universal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100
Research works printed but not linked: 1
- Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the Association for Computational Linguistics, pages 142-150, Portland, Oregon, USA. Association for Computational Linguistics.
Cited by
Sorted by citations
Items 101–200 of 647
Applications of Large Language Models in the Field of Suicide Prevention: Scoping Review
Holmes, G., Tang, B., Gupta, S., Venkatesh, S., Christensen, H. M., & Whitton, A. E. (2024) · Journal of Medical Internet Research · doi:10.2196/63126 · Cited by 62
Li, H., Fu, J., & Python, A. (2025) · Journal of Medical Internet Research · doi:10.2196/71916 · Cited by 49
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey
Kumar, S., Balachandran, V., Njoo, L., Anastasopoulos, A., & Tsvetkov, Y. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.241 · Cited by 38
Gray, M., Samala, R. K., Liu, Q., Skiles, D., Xu, J., Tong, W., & Wu, L. (2023) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.3117 · Cited by 47
Understanding and Countering Stereotypes: A Computational Approach to the Stereotype Content Model
Fraser, K., Nejadgholi, I., & Kiritchenko, S. (2021) · doi:10.18653/v1/2021.acl-long.50 · Cited by 47
Bias Testing and Mitigation in LLM-based Code Generation
Huang, D., Zhang, J. M., Bu, Q., Xie, X., Chen, J., & Cui, H. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3724117 · Cited by 128
Fairness in Language Models Beyond English: Gaps and Challenges
Ramesh, K., Sitaram, S., & Choudhury, M. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.157 · Cited by 60
The MultiBERTs: BERT Reproductions for Robustness Analysis
Sellam, T., Yadlowsky, S., Wei, J., Saphra, N., D'Amour, A., Linzen, T., Bastings, J., Turc, I., Eisenstein, J., Das, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2106.16163 · Cited by 37
Empathy and Equity: Key Considerations for Large Language Model Adoption in Health Care
Koranteng, E., Rao, A. S., Flores, E. J., Lev, M. H., Landman, A., Dreyer, K. J., & Succi, M. D. (2023) · JMIR Medical Education · doi:10.2196/51199 · Cited by 39
Navigating the Ethical Terrain of AI-Generated Text Tools: A Review
Mohamed, Y. A., Mohamed, A. H. H. M., Khanan, A., Bashir, M., Adiel, M. A. E., & Elsadig, M. A. (2024) · IEEE Access · doi:10.1109/access.2024.3521945 · Cited by 44
Cyberbullying Detection With Fairness Constraints
Gencoglu, O. (2021) · IEEE Internet Computing · doi:10.1109/mic.2020.3032461 · Cited by 53
Sánchez-Junquera, J., Chulvi, B., Rosso, P., & Ponzetto, S. P. (2021) · Applied Sciences · doi:10.3390/app11083610 · Cited by 39
Assessing and alleviating state anxiety in large language models
Ben‐Zion, Z., Witte, K., Jagadish, A. K., Duek, O. A., Harpaz‐Rotem, I., Khorsandian, M.-C., Burrer, A., Seifritz, E., Homan, P., Schulz, E., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01512-6 · Cited by 59
Lawyers are Dishonest? Quantifying Representational Harms in Commonsense Knowledge Resources
Mehrabi, N., Zhou, P., Morstatter, F., Pujara, J., Ren, X., & Galstyan, A. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.410 · Cited by 35
Examining the responsible use of zero-shot AI approaches to scoring essays
Johnson, M., & Zhang, M. (2024) · Scientific Reports · doi:10.1038/s41598-024-79208-2 · Cited by 48
Federated and transfer learning for cancer detection based on image analysis
Bechar, A., Medjoudj, R., Elmir, Y., Himeur, Y., & Amira, A. (2025) · Neural Computing and Applications · doi:10.1007/s00521-024-10956-y · Cited by 38
Debiasing Pre-Trained Language Models via Efficient Fine-Tuning
Gira, M., Zhang, R., & Lee, K. (2022) · Workshop on Language Technology for Equality, Diversity, Inclusion (LT-EDI) · doi:10.18653/v1/2022.ltedi-1.8 · Cited by 38
Towards Identifying Social Bias in Dialog Systems: Framework, Dataset, and Benchmark
Zhou, J., Deng, J., Mi, F., Li, Y., Wang, Y., Huang, M., Jiang, X., Liu, Q., & Meng, H. M. L. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.262 · Cited by 34
Digital forgetting in large language models: a survey of unlearning methods
Blanco-Justicia, A., Jebreel, N., Manzanares-Salor, B., Sánchez, D., Domingo-Ferrer, J., Collell, G., & Eeik Tan, K. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-024-11078-6 · Cited by 31
Embedding Democratic Values into Social Media AIs via Societal Objective Functions
Jia, C., Lam, M. S., Mai, M. C., Hancock, J. T., & Bernstein, M. S. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641002 · Cited by 31
Pipelines for Social Bias Testing of Large Language Models
Nozza, D., Bianchi, F., & Hovy, D. (2022) · doi:10.18653/v1/2022.bigscience-1.6 · Cited by 33
BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text Generation
Sun, T., He, J., Qiu, X., & Huang, X. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.245 · Cited by 44
Gender Bias in Masked Language Models for Multiple Languages
Kaneko, M., Imankulova, A., Bollegala, D., & Okazaki, N. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.197 · Cited by 35
Monolingual and Multilingual Reduction of Gender Bias in Contextualized Representations
liang, S. M., Dufter, P., & Schütze, H. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.446 · Cited by 33
ADEPT: A DEbiasing PrompT Framework
Yang, K., Yu, C., Fung, Y. R., Li, M., & Ji, H. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i9.26279 · Cited by 34
Gendered Mental Health Stigma in Masked Language Models
Lin, I., Njoo, L., Field, A., Sharma, A., Reinecke, K., Althoff, T., & Tsvetkov, Y. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.139 · Cited by 33
Mitigating Gender Bias in Distilled Language Models via Counterfactual Role Reversal
Gupta, U., Dhamala, J., Kumar, V., Verma, A., Pruksachatkun, Y., Krishna, S., Gupta, R., Chang, K.-W., Steeg, G. V., & Galstyan, A. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.55 · Cited by 38
They, Them, Theirs: Rewriting with Gender-Neutral EnglishPreprint
Sun, T., Webster, K., Shah, A., Wang, W. Y., & Johnson, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.06788 · Cited by 65
Coding Inequity: Assessing GPT-4’s Potential for Perpetuating Racial and Gender Biases in HealthcarePreprint
Zack, T. I., Lehman, E., Süzgün, M., Rodriguez, J. A., Celi, L. A., Gichoya, J. W., Jurafsky, D., Peter Szolovits, B., Bates, D. W., Abdulnour, R.-E. E., et al. (2023) · medRxiv · doi:10.1101/2023.07.13.23292577 · Cited by 28
Detecting Cross-Geographic Biases in Toxicity Modeling on Social Media
Ghosh, S., Baker, D., Jurgens, D., & Prabhakaran, V. (2021) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/2021.wnut-1.35 · Cited by 35
Estimating the Personality of White-Box Language ModelsPreprint
Karra, S. R., Nguyen, S. T., & Tulabandhula, T. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4598766 · Cited by 77
Towards Culturally Adaptive Large Language Models in Mental Health: Using ChatGPT as a Case Study
Aleem, M., Zahoor, I., & Naseem, M. (2024) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3678884.3681858 · Cited by 36
Large scale analysis of gender bias and sexism in song lyrics
Betti, L., Abrate, C., & Kaltenbrunner, A. (2023) · EPJ Data Science · doi:10.1140/epjds/s13688-023-00384-8 · Cited by 45
Searching for Computer Vision North Stars
Fei-Fei, L., & Krishna, R. (2022) · Daedalus · doi:10.1162/daed_a_01902 · Cited by 28
T2IAT: Measuring Valence and Stereotypical Biases in Text-to-Image Generation
Wang, J., Liu, X., Di, Z., Liu, Y., & Wang, X. E. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.160 · Cited by 50
Singhal, P., Gokhale, S., Shah, A., Jain, D. K., Walambe, R. A., Ekárt, A., & Kotecha, K. V. (2025) · Discover Applied Sciences · doi:10.1007/s42452-025-06659-1 · Cited by 26
LLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actions
Hundt, A., Azeem, R., Mansouri, M. I., & Brandão, M. (2025) · International Journal of Social Robotics · doi:10.1007/s12369-025-01301-x · Cited by 56
Theory-Grounded Measurement of U.S. Social Stereotypes in English Language Models
Cao, Y., Sotnikova, A., Daumé, H., Rudinger, R., & Zou, L. X. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.92 · Cited by 30
Language Models Get a Gender Makeover: Mitigating Gender Bias with Few-Shot Data Interventions
Thakur, H., Jain, A., Vaddamanu, P., Liang, P. P., & Morency, L. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.30 · Cited by 39
An Empirical Analysis of Parameter-Efficient Methods for Debiasing Pre-Trained Language Models
Xie, Z., & Lukasiewicz, T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.876 · Cited by 41
Generative AI in Computer Science Education: Challenges and Opportunities
Franklin, D., Denny, P., Gonzalez-Maldonado, D. A., & Tran, M. (2025) · doi:10.1017/9781009581738 · Cited by 20
Unlocking Bias Detection: Leveraging Transformer-Based Models for Content Analysis
Raza, S., Bamgbose, O., Chatrath, V., Ghuge, S., Sidyakin, Y., & Muaad, A. Y. M. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3392469 · Cited by 28
Jojoa, M., Eftekhar, P., Nowrouzi‐Kia, B., & García-Zapirain, B. (2022) · AI & Society · doi:10.1007/s00146-022-01594-w · Cited by 28
Measuring and mitigating language model biases in abusive language detection
Song, R., Giunchiglia, F., Li, Y., Shi, L., & Xu, H. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103277 · Cited by 23
FAIR Enough: Develop and Assess a FAIR-Compliant Dataset for Large Language Model Training?
Raza, S., Ghuge, S., Ding, C., Dolatabadi, E., & Pandya, D. A. (2024) · Data Intelligence · doi:10.1162/dint_a_00255 · Cited by 39
Shaping the Emerging Norms of Using Large Language Models in Social Computing Research
Shen, H., Li, T., Li, T. J.-J., Park, J. S., & Yang, D. (2023) · Computer Supported Cooperative Work and Social Computing · doi:10.1145/3584931.3606955 · Cited by 21
Causal keyword driven reliable text classification with large language model feedback
Song, R. F., Li, Y., Tian, M., Wang, H., Giunchiglia, F., & Xu, H. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103964 · Cited by 25
Red teaming large language models: A comprehensive review and critical analysis
Jabbar, M. S., Al-Azani, S., Alotaibi, A., & Ahmed, M. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104239 · Cited by 31
From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language Models
Mendelsohn, J., Bras, R. J. L., Choi, Y., & Sap, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.845 · Cited by 33
ROBBIE: Robust Bias Evaluation of Large Generative Language Models
Esiobu, D., Tan, X., Hosseini, S., Ung, M., Zhang, Y., Fernandes, J., Dwivedi-Yu, J., Presani, E., Williams, A., & Smith, E. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.230 · Cited by 49
Evaluating Biased Attitude Associations of Language Models in an Intersectional Context
Sabbaghi, S. O., Wolfe, R., & Caliskan, A. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604666 · Cited by 25
Mihaljević, H., Müller, I., Dill, K., Yollu-Tok, A., & von Grafenstein, M. (2023) · AI & Society · doi:10.1007/s00146-023-01726-w · Cited by 27
MABEL: Attenuating Gender Bias using Textual Entailment Data
He, J., Xia, M., Fellbaum, C., & Chen, D. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.657 · Cited by 39
Fairness-Aware Structured Pruning in Transformers
Zayed, A., Mordido, G., Shabanian, S., Baldini, I., & Chandar, S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i20.30256 · Cited by 25
Reducing Non-Normative Text Generation from Language Models
Peng, X., Li, S., Frazier, S., & Riedl, M. O. (2020) · International Conference on Natural Language Generation · doi:10.18653/v1/2020.inlg-1.43 · Cited by 28
Natural Language Processing for Radiation Oncology: Personalizing Treatment Pathways
Lin, H., Ni, L., Phuong, C., & Hong, J. C. (2024) · Pharmacogenomics and Personalized Medicine · doi:10.2147/pgpm.s396971 · Cited by 22
Low-Complexity Probing via Finding Subnetworks
Cao, S., Sanh, V., & Rush, A. M. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.74 · Cited by 51
Computational Modeling of Stereotype Content in Text
Fraser, K., Kiritchenko, S., & Nejadgholi, I. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.826207 · Cited by 23
Choose Your Lenses: Flaws in Gender Bias Evaluation
Orgad, H., & Belinkov, Y. (2022) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2022.gebnlp-1.17 · Cited by 24
ToKen: Task Decomposition and Knowledge Infusion for Few-Shot Hate Speech Detection
AlKhamissi, B., Ladhak, F., Iyer, S. V., Stoyanov, V., Kozareva, Z., Li, X., Fung, P., Mathias, L., Çelikyılmaz, A., & Diab, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.136 · Cited by 24
Lonergan, R. M., Curry, J., Dhas, K., & Simmons, B. I. (2023) · Cureus · doi:10.7759/cureus.48788 · Cited by 19
Fairness and social bias quantification in Large Language Models for sentiment analysis
Radaideh, M. I., Kwon, O. H., & Radaideh, M. I. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113569 · Cited by 24
Large Language Models for Psychological Assessment: A Comprehensive Overview
Brickman, J., Gupta, M., & Oltmanns, J. R. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251343582 · Cited by 24
Towards an open-domain chatbot for language practice
Tyen, G., Brenchley, M., Caines, A., & Buttery, P. J. (2022) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/2022.bea-1.28 · Cited by 23
BotPercent: Estimating Bot Populations in Twitter Communities
Tan, Z., Feng, S., Sclar, M., Wan, H., Luo, M., Choi, Y., & Tsvetkov, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.954 · Cited by 23
AI Chatbots and Linguistic Injustice
Park, S. (2024) · Journal of Universal Language · doi:10.22425/jul.2024.25.1.99 · Cited by 23
KoBBQ: Korean Bias Benchmark for Question Answering
Jin, J., Kim, J., Lee, N., Yoo, H., Oh, A. H., & Lee, H. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00661 · Cited by 27
Balestri, R. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1558696 · Cited by 25
Deep Lexical Hypothesis: Identifying personality structure in natural languagePreprint
Cutler, A., & Condon, D. M. (2022) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/gdm5v · Cited by 19
An Overview on Language Models: Recent Developments and OutlookPreprint
Wei, C., Wang, Y.-C., Wang, B., & Kuo, C. ‐. J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.05759 · Cited by 15
Transformers and the Representation of Biomedical Background Knowledge
Wysocki, O., Zhou, Z., O’Regan, P., Ferreira, D. M., Wysocka, M., Landers, D., & Freitas, A. (2022) · Computational Linguistics · doi:10.1162/coli_a_00462 · Cited by 17
In the Name of Fairness: Assessing the Bias in Clinical Record De-identification
Xiao, Y., Lim, S., Pollard, T., & Ghassemi, M. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3593982 · Cited by 23
Janghorbani, S., & de Melo, G. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.126 · Cited by 32
Lahoti, P., Blumm, N., Ma, X., Kotikalapudi, R., Potluri, S., Tan, Q., Srinivasan, H., Packer, B., Beirami, A., Beutel, A., et al. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.643 · Cited by 45
A DSL for Testing LLMs for Fairness and Bias
Morales, S., Clarisó, R., & Cabot, J. (2024) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MODELS) · doi:10.1145/3640310.3674093 · Cited by 22
A survey on large language models with multilingualism: recent advances and new frontiers
Huang, K., Mo, F., Zhang, X., Li, H., Li, Y., Zhang, Y., yi, W., Mao, Y., Liu, J., Xu, Y., et al. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11534-5 · Cited by 131
What's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpus
Luccioni, A. S., & Viviano, J. D. (2021) · arXiv · doi:10.48550/arxiv.2105.02732 · Cited by 17
Gender Bias in Natural Language Processing and Computer Vision: A Comparative Survey
Bartl, M., Mandal, A., Leavy, S., & Little, S. (2024) · ACM Computing Surveys · doi:10.1145/3700438 · Cited by 23
Winder, P. D., Hildebrand, C., & Hartmann, J. (2025) · PLoS ONE · doi:10.1371/journal.pone.0325459 · Cited by 25
Li, Y., Du, M., Wang, X., & Wang, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.797 · Cited by 18
Venkit, P. N., Srinath, M., & Wilson, S. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.3 · Cited by 33
Demonstrations of the Potential of AI-based Political Issue Polling
Sanders, N. E., Ulinich, A., & Schneier, B. (2023) · Harvard Data Science Review · doi:10.1162/99608f92.1d3cf75d · Cited by 43
Towards fair decision: A novel representation method for debiasing pre-trained models
He, J., Lin, N., Bai, Q., Liang, H., Zhou, D. M., & Yang, A. (2024) · Decision Support Systems · doi:10.1016/j.dss.2024.114208 · Cited by 16
CoPrompter: User-Centric Evaluation of LLM Instruction Alignment for Improved Prompt Engineering
Joshi, I., Shahid, S., Venneti, S., Vasu, M., Zheng, Y., Li, Y., Krishnamurthy, B., & Chan, G. Y.-Y. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712102 · Cited by 24
DQI: Measuring Data Quality in NLPPreprint
Mishra, S. R., Arunkumar, A., Sachdeva, B., Bryan, C., & Baral, C. R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.00816 · Cited by 38
KoSBI: A Dataset for Mitigating Social Bias Risks Towards Safer Large Language Model Applications
Lee, H., Hong, S., Park, J., Kim, T., Kim, G., & Ha, J.-W. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry Track) · doi:10.18653/v1/2023.acl-industry.21 · Cited by 15
DS-TOD: Efficient Domain Specialization for Task-Oriented Dialog
Hung, C.-C., Lauscher, A., Ponzetto, S. P., & Glavaš, G. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.72 · Cited by 16
Fewer Errors, but More Stereotypes? The Effect of Model Size on Gender Bias
Tal, Y., Magar, I., & Schwartz, R. (2022) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2022.gebnlp-1.13 · Cited by 25
Shihadeh, J., Ackerman, M., Troske, A., Lawson, N. M., & Gonzalez, E. (2022) · IEEE Global Humanitarian Technology Conference (GHTC) · doi:10.1109/ghtc55712.2022.9910995 · Cited by 18
Anoop, K. R., Gangan, M. P., Deepak, P., & Lajish, V. L. (2022) · Lecture notes in electrical engineering · doi:10.1007/978-981-19-4453-6_2 · Cited by 17
The Ethics of Computational Social Science
Leslie, D. (2023) · Springer eBooks · doi:10.1007/978-3-031-16624-2_4 · Cited by 18
SeeGULL: A Stereotype Benchmark with Broad Geo-Cultural Coverage Leveraging Generative Models
Jha, A., Davani, A. M., Reddy, C. K. A., Dave, S., Prabhakaran, V., & Dev, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.548 · Cited by 43
Social-Group-Agnostic Bias Mitigation via the Stereotype Content Model
Omrani, A., Ziabari, A. S., Yu, C., Golazizian, P., Kennedy, B., Atari, M., Ji, H., & Dehghani, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.227 · Cited by 16
Detecting Unintended Social Bias in Toxic Language Datasets
Sahoo, N., Gupta, H., & Bhattacharyya, P. (2022) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2022.conll-1.10 · Cited by 25
Das, D., Guha, S., & Semaan, B. (2023) · Workshop on Cross-Cultural Considerations in NLP (C3NLP) · doi:10.18653/v1/2023.c3nlp-1.8 · Cited by 16
Manifestations of xenophobia in AI systems
Tomašev, N., Maynard, J. L., & Gabriel, I. (2024) · AI & Society · doi:10.1007/s00146-024-01893-4 · Cited by 22
Li, Y., Du, M., Song, R., Wang, X., Sun, M., & Wang, Y. (2024) · Artificial Intelligence · doi:10.1016/j.artint.2024.104143 · Cited by 16
How Reliable are Model Diagnostics?
Aribandi, V., Tay, Y., & Metzler, D. (2021) · Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 · doi:10.18653/v1/2021.findings-acl.155 · Cited by 15
A comparative study on the impact of model compression techniques on fairness in language models
Ramesh, K., Chavan, A., Pandit, S., & Sitaram, S. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.878 · Cited by 15
Metadata
- Authors
- Nadeem, M., Bethke, A., & Reddy, S.
- Affiliations
- Massachusetts Institute of Technology (US)
- University of California, Santa Barbara (US)
- University of Cambridge (GB)
- ORCID
- Reddy, Siva
- Venue
- Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
- Published
- 2021
- Pages
- 5356-5371
- OpenAlex
- W3019416653
- Semantic Scholar
- babeda48b10a4d638252118f2238d05a06f4ec55
- Cited by
- 643
- References
- 15
- Citations by year
- 2026: 70 · 2025: 131 · 2024: 98 · 2023: 191 · 2022: 85 · 2021: 49
- FWCI
- 95.35
- Type
- conference paper
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- stereotypical bias, pre-trained language models, gender stereotypes, language model evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-23, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-23 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
