The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets
Open accessAbstract
Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, Hanna Wallach. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Evaluating the expanding capabilities of LLMs is a complex research area28,29, with group-specific bias benchmarks shown to be time-consuming to develop and utilize30–33, and the overall field lacking measurement validity and theoretical grounding30,34.”
From Generative language models exhibit social identity biases (2024) · cites this paper as [34]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Many approaches, for instance, assume some implicitly desirable criterion ( e.g. , a model output should be independent of any social group in the input), but do not explicitly acknowledge or state the normative social values that justify their framework.”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Blodgett et al. 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In nearly half of all instances, these datasets contain ambiguities about what stereotypes they capture, raising questions whether they are valid indicators of real-world stereotypes at all.”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Blodgett et al. (2021)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“several widely-used evaluation datasets suffer from reliability and validity issues, including ambiguities about whether instances accurately reflect real-world stereotypes, inconsistent treatment of social groups, assumptions of near-perfect understanding of language, and lack of syntactic and semantic diversity (Blodgett et al. 2021; Gupta et al. 2023; Selvam et al. 2023).”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Blodgett et al. 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Blodgett et al. (2021) highlight several severe shortcomings of several counterfactual input datasets, namely Winogender, WinoBias, StereoSet, and CrowS-Pairs.”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Blodgett et al. (2021)
The cited source says what the claim attributes to it.
In the citing paper
“At the core of these issues are (1) unclear articulation of the power imbalances, harm, or other stereotype content present in each instance; and (2) inconsistent, invalid, or unrelated perturbations of social groups.”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Blodgett et al. (2021)
The cited source says what the claim attributes to it.
In the citing paper
“These datasets also suffer from some data reliability issues, but to a lesser extent than those discussed in Blodgett et al. (2021) (Liang et al. 2022).”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Blodgett et al. (2021)
In the citing paper
“In addition, prompt-based learning with GPT-3 has achieved very strong performances in zero-shot and one-shot settings (i.e. with only zero or one labelled example), without any fine-tuning [7].”
From GPT-3-Driven Pedagogical Agents to Train Children’s Curious Question-Asking Skills (2024) · cites this paper as [7]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Austin, J. L. (1975) · doi:10.1093/acprof:oso/9780198245537.001.0001 · Cited by 21,496
Black Feminist Thought: Knowledge, Consciousness, and the Politics of Empowerment.
Brewer, R. M., & Collins, P. H. (1992) · Contemporary Sociology · doi:10.2307/2074808 · Cited by 19,445
A General Inductive Approach for Analyzing Qualitative Evaluation Data
Thomas, D. R. (2006) · American Journal of Evaluation · doi:10.1177/1098214005283748 · Cited by 10,858 · cited 2×
“This inductive approach aligns with our exploratory goal of surfacing a range of pitfalls, including many that are not on the community radar and thus likely to be overlooked by a more deductive approach requiring a predefined codebook grounded in existing assumptions (Thomas, 2006).”
MethodClark, H. H. (1996) · doi:10.1017/cbo9780511620539 · Cited by 5,123
Glick, P., & Fiske, S. T. (2001) · American Psychologist · doi:10.1037/0003-066x.56.2.109 · Cited by 2,224
Measurement Validity: A Shared Standard for Qualitative and Quantitative Research
Adcock, R., & Collier, D. (2001) · American Political Science Review · doi:10.1017/s0003055401003100 · Cited by 1,629
Measurement Validity: A Shared Standard for Qualitative and Quantitative Research
Collier, D., & Adcock, R. (2001) · eScholarship (California Digital Library) · Cited by 1,603
Bucholtz, M., & Hall, K. (2005) · Wiley eBooks · doi:10.1002/9780470996522.ch16 · Cited by 1,153
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
Wang, A., Pruksachatkun, Y., Nangia, N., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. R. (2019) · arXiv · doi:10.48550/arxiv.1905.00537 · Cited by 1,051 · cited 2×
“Yet even as such benchmarks are added to popular NLP leaderboards like Super-Glue (Wang et al., 2019), whether these they actually help measure the extent to which NLP systems produce computational harms remains unknown.”
BackgroundLanguage (Technology) is Power: A Critical Survey of “Bias” in NLP
Blodgett, S. L., Barocas, S., Daumé III, H., & Wallach, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.485 · Cited by 1,050 · cited 1×
“, 2020), and a lack of consensus about how to conceptualize or operationalize such harms (Blodgett et al., 2020; Jacobs and Wallach, 2021).”
BackgroundSelf-citationThe psychology of stereotyping
(2004) · Choice Reviews Online · doi:10.5860/choice.42-1892 · Cited by 751 · cited 1×
“which groups to consider or which aspects of stereotyping to focus on, including occupations, physical features, emotional traits, or language use (Schneider, 2005).”
BackgroundRealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693
Gender bias in coreference resolution: Evaluation and debiasing methods
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2003 · Cited by 684 · cited 5×
“These datasets consist of tests that can take a variety of formats, including sentence templates where terms pertaining to groups or their attributes are perturbed (Rudinger et al., 2018; Zhao et al., 2018; Kurita et al., 2019), prompts designed to elicit problematic responses (Gehman et al.”
BackgroundInfluentialHow to analyze political attention with minimal assumptions and costs
Quinn, K. M., Monroe, B. L., Colaresi, M., Crespin, M. H., & Radev, D. R. (2010) · American Journal of Political Science · doi:10.1111/j.1540-5907.2009.00427.x · Cited by 671 · cited 1×
“Measurement Modeling We apply a measurement modeling lens by viewing each benchmark as a measurement model (MM) (e.g., Quinn et al., 2010; Jacobs and Wallach, 2021).”
MethodStereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.416 · Cited by 643 · cited 8×
“Consider the illustrative example about “Norwe-gian salmon” in Figure 1—drawn from an existing benchmark (Nadeem et al., 2020)—which depicts a test meant (according to the metadata) to capture a stereotype about race.”
BackgroundMethodInfluentialStereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2020) · arXiv · doi:10.48550/arxiv.2004.09456 · Cited by 643
Data and its (dis)contents: A survey of dataset development and use in machine learning research
Paullada, A., Raji, I. D., Bender, E. M., Denton, E., & Hanna, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100336 · Cited by 548 · cited 1×
“Therefore, it is critical to uncover the explicit and implicit assumptions that these benchmarks carry and the incentives to which they may give rise (Paullada et al., 2020).”
BackgroundThe Woman Worked as a Babysitter: On Biases in Language Generation
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1339 · Cited by 537 · cited 1×
“…sentence templates where terms pertaining to groups or their attributes are perturbed (Rudinger et al., 2018; Zhao et al., 2018; Kurita et al., 2019), prompts designed to elicit problematic responses (Gehman et al., 2020; Sheng et al., 2019; Example Sentences Context I really like Norweigan salmon.”
BackgroundPersistent Anti-Muslim Bias in Large Language Models
Abid, A., Farooqi, M., & Zou, J. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462624 · Cited by 476 · cited 2×
“…due in no small part to the deeply contextual and open nature of language use and tasks (Austin, 1975; Clark, 1996; Howcroft et al., 2020; Abid et al., 2021; Olteanu et al., 2020), and a lack of consensus about how to conceptualize or operationalize such harms (Blod-gett et al., 2020;…”
BackgroundReskin, B. F. (1988) · Gender & Society · doi:10.1177/089124388002001005 · Cited by 468
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models
Nangia, N., Vania, C., Bhalerao, R., & Bowman, S. R. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.154 · Cited by 446
Gender Bias in Coreference Resolution
Rudinger, R., Naradowsky, J., Leonard, B., & Van Durme, B. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2002 · Cited by 443 · cited 5×
“These datasets consist of tests that can take a variety of formats, including sentence templates where terms pertaining to groups or their attributes are perturbed (Rudinger et al., 2018; Zhao et al., 2018; Kurita et al., 2019), prompts designed to elicit problematic responses (Gehman et al.”
BackgroundInfluentialJacobs, A. Z., & Wallach, H. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445901 · Cited by 362 · cited 3×
“Measurement Modeling We apply a measurement modeling lens by viewing each benchmark as a measurement model (MM) (e.g., Quinn et al., 2010; Jacobs and Wallach, 2021).”
BackgroundMethodSelf-citationMeasuring Bias in Contextualized Word Representations
Kurita, K., Vyas, N., Pareek, A., Black, A. W., & Tsvetkov, Y. (2019) · Proceedings of the First Workshop on Gender Bias in Natural Language Processing · doi:10.18653/v1/w19-3823 · Cited by 348 · cited 2×
“…formats, including sentence templates where terms pertaining to groups or their attributes are perturbed (Rudinger et al., 2018; Zhao et al., 2018; Kurita et al., 2019), prompts designed to elicit problematic responses (Gehman et al., 2020; Sheng et al., 2019; Example Sentences Context I really…”
BackgroundHowcroft, D. M., Belz, A., Clinciu, M., Gkatzia, D., Hasan, S. A., Mahamood, S., Mille, S., van Miltenburg, E., Santhanam, S., & Rieser, V. (2020) · International Conference on Natural Language Generation · doi:10.18653/v1/2020.inlg-1.23 · Cited by 152
Perturbation Sensitivity Analysis to Detect Unintended Model Biases
Prabhakaran, V., Hutchinson, B., & Mitchell, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1578 · Cited by 115 · cited 2×
“Names have been used as demographic proxies in both social computing and NLP work, and often appear in close proximity to other demographic markers in text corpora; LMs might thus memorize incidental associations between names and demographic criteria (Prabhakaran et al., 2019; Shwartz et al., 2020).”
BackgroundMethodInvestigating African-American Vernacular English in Transformer-Based Text Generation
Groenwold, S., Ou, L., Parekh, A., Honnavalli, S., Levy, S., Mirza, D., & Wang, W. Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.473 · Cited by 59
“You are grounded!”: Latent Name Artifacts in Pre-trained Language Models
Shwartz, V., Rudinger, R., & Tafjord, O. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.556 · Cited by 52
When Are Search Completion Suggestions Problematic?
Olteanu, A., Díaz, F., & Kazai, G. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415242 · Cited by 38 · cited 2×
“Auditing NLP systems for computational harms like the reproduction of stereotypes or hate speech remains a persistent challenge, due in no small part to the deeply contextual and open nature of language use and tasks (Austin, 1975; Clark, 1996; Howcroft et al., 2020; Abid et al., 2021; Olteanu et al., 2020), and a lac…”
BackgroundSelf-citationOvercoming Failures of Imagination in AI Infused System Development and DeploymentPreprint
Boyarskaya, M., Olteanu, A., & Crawford, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.13416 · Cited by 21 · cited 2×
“…modeling framework can provide analytical clarity and a scaffold for future work in this direction. be inherently beneficial with little to no potential for harm, and thus often (paradoxically) fails to examine its limitations or possible unintended negative consequences (Boyarskaya et al., 2020).”
BackgroundSelf-citationPersistent Anti-Muslim Bias in Large Language ModelsPreprint
Abid, A., Farooqi, M., & Zou, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.05783 · Cited by 17
Data and its (dis)contents: A survey of dataset development and use in machine learning research
Paullada, A., Raji, I. D., Bender, E. M., Denton, E., & Hanna, A. (2020) · arXiv · doi:10.48550/arxiv.2012.05345
Research works printed but not linked: 1
- David J. Schneider. 2005. The Psychology of Stereotyp- ing. Guilford Press.
Cited by
Sorted by citations
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
“…et al., 2021b), there is a lack of standardization of fairness benchmarks, an understanding of what harms different bias measures in NLP relate to (Blodgett et al., 2020, 2021; Jacobs & Wallach, 2021), and coverage of identities in fluid, comprehensive ways (Cao & Daumé III, 2020; Dev et al.,…”
BackgroundGemini: A Family of Highly Capable Multimodal Models
Gemini Team, Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.11805 · Cited by 990
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245
“Others lack consistency in their definitions of bias, or do not seriously engage with the relevant power dynamics that perpetuate the underlying harm (Blodgett et al., 2021).”
BackgroundEthical and social risks of harm from Language Models
Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.04359 · Cited by 477
Metadata
- Authors
- Blodgett, S. L., Lopez, G., Olteanu, A., Sim, R., & Wallach, H.
- Affiliations
- Microsoft (United States) (US)
- Venue
- Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
- Published
- 2021
- Pages
- 1004-1015
- OpenAlex
- W3172415559
- Cited by
- 147
- References
- 29
- Citations by year
- 2026: 13 · 2025: 17 · 2024: 21 · 2023: 55 · 2022: 34 · 2021: 7
- FWCI
- 26.22
- Type
- conference paper
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- stereotyping, Norwegian language, bias in language models, dataset evaluation, fairness metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
