CoChat
Sign Up
Cited by 196Ethics and Social Impacts of AITopic ModelingHate Speech and Cyberbullying Detection

BBQ: A hand-built bias benchmark for question answering

Open access
Parrish, Alicia, Chen, Angelica, Nangia, Nikita, Padmakumar, Vishakh, Phang, Jason, Thompson, Jana, Htut, Phu Mon, Bowman, Samuel·2022·Findings of the Association for Computational Linguistics: ACL 2022
DOI 10.18653/v1/2022.findings-acl.165
Open this paper in CoChat — free accountOpen PDF

Abstract

It is well documented that NLP models learn social biases, but little work has been done on how these biases manifest in model outputs for applied tasks like question answering (QA).We introduce the Bias Benchmark for QA (BBQ), a dataset of question sets constructed by the authors that highlight attested social biases against people belonging to protected classes along nine social dimensions relevant for U.S. English-speaking contexts.Our task evaluates model responses at two levels: (i) given an under-informative context, we test how strongly responses refect social biases, and (ii) given an adequately informative context, we test whether the model's biases override a correct answer choice.We fnd that models often rely on stereotypes when the context is under-informative, meaning the model's outputs consistently reproduce harmful biases in this setting.Though models are more accurate when the context provides an informative answer, they still rely on stereotypes and average up to 3.4 percentage points higher accuracy when the correct answer aligns with a social bias than when it conficts, with this difference widening to over 5 points on examples targeting gender for most models tested.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2022.findings-acl.165

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • LLMs may exhibit social biases and toxicity during generation, resulting in biased outputs.

    “LLMs may exhibit social biases and toxicity [37, 53, 153] during the generation process, resulting in the production of biased outputs.”

    From A Survey on Evaluation of Large Language Models (2024) · cites this paper as [153]

    The Introduction says language models can reproduce social biases in language generation, and the Conclusion reports that QA model outputs can reinforce stereotypes. These passages support the biased-output claim, but the paper does not establish the separate claim about toxicity during generation. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • Group-specific bias benchmarks are time-consuming to develop and utilize, and the overall field lacks measurement validity and theoretical grounding.

    “Evaluating the expanding capabilities of LLMs is a complex research area28,29, with group-specific bias benchmarks shown to be time-consuming to develop and utilize30–33, and the overall field lacking measurement validity and theoretical grounding30,34.”

    From Generative language models exhibit social identity biases (2024) · cites this paper as [31]

    The Dataset (§3) and Validation (§4) describe hand-writing templates, annotating them, and repeatedly validating and revising them, supporting the point that developing BBQ takes substantial effort. Related Work notes varying definitions of bias, and Ethical Considerations (§9) warns about limits on valid use of BBQ. The paper does not establish that group-specific benchmarks are time-consuming to use or that the field as a whole lacks theoretical grounding and measurement validity. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • BBQ questions ask for a social group that meets either a negative or positive stereotype, with an ambiguous context that does not contain enough information to answer the question.

    “Questions ask for a social group that meets either a negative or positive stereotype, with an ambiguous context that does not contain enough information to answer the question”

    From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Parrish et al. 2022

    The Ambiguous Contexts section says those contexts identify two social groups but provide insufficient evidence to answer, making “Unknown” the correct choice. Section 3.2 confirms negative and non-negative question variants, but explicitly says that non-negative does not mean harmless; calling them questions about a “positive stereotype” overstates that distinction. Page numbers are unavailable in our copy. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • BBQ is a question-answering dataset of 58,492 examples to assess bias across nine social groups: age, disability status, gender, nationality, physical appearance, race/ethnicity, religion, and socioeconomic status.

    “Bias Benchmark for QA (BBQ) (Parrish et al. 2022) is a questionanswering dataset of 58,492 examples to assess bias across nine social groups: age, disability status, gender, nationality, physical appearance, race/ethnicity, religion, and socioeconomic status.”

    From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Parrish et al. 2022

    Section 3, “The Dataset,” states that BBQ contains 58,492 unique examples, and Section 3.1, “Coverage,” says it covers nine broad categories of social bias. The citing sentence names only eight: it omits sexual orientation, which Section 3.3, “Vocabulary,” identifies as a category alongside those listed. (LLM, conf=0.90)

    Experimental — accuracy not yet measured

  • Provider documentation indicates that SES appears in some evaluation suites; for example, BBQ includes SES dimensions and Google's Gemini evaluations incorporate Dollar Street for socioeconomic contexts.

    “Provider documentation indicates that SES does appear in some evaluation suites. For example, BBQ (Bias Benchmark for QA), a standardized test measuring social biases in question-answering, includes SES dimensions [72]. Google's Gemini evaluations separately incorporate Dollar Street, a dataset of household photographs across income levels, for socioeconomic contexts [34, 84].”

    From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [72]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. Kate Crawford. 2017. The trouble with bias. http: //youtube.com/watch?v=fMym_BKWQzk. Talk given at NeurIPS December 2017.
  2. Daniel Khashabi, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, and Hannaneh Ha- jishirzi. 2020. UnifedQA: Crossing format bound- aries with a single QA system. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pages 1896-1907.

Cited by

Sorted by citations

Related

Authors
Parrish, A., Chen, A., Nangia, N., Padmakumar, V., Phang, J., Thompson, J., Htut, P. M., & Bowman, S.
Affiliations
New York University (US)
ORCID
Venue
Findings of the Association for Computational Linguistics: ACL 2022
Publisher
Association for Computational Linguistics
Published
2022
Pages
2086-2105
DOI
10.18653/v1/2022.findings-acl.165
OpenAlex
W3207316473
Cited by
186
References
34
Citations by year
2026: 67 · 2025: 62 · 2024: 29 · 2023: 23 · 2022: 5
FWCI
29.56
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • Findings of the Association for Computational Linguistics: ACL (Findings of ACL) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
  • National Science Foundation (2046556, 1922658)
  • Samsung
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
Keywords
social bias, question answering, bias benchmark, stereotypes, protected classes, gender bias, model bias evaluation, large language models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review