CoChat

Bias and Fairness in Large Language Models: A Survey

Open access

The DOI matches its Crossref record; 313 of the 313 references it cites were found in a registry or an index; 151 of the 183 sources its claims cite were checked for support.

Abstract

Rapid advancements of large language models (LLMs) have enabled the processing, understanding, and generation of human-like text, with increasing integration into systems that touch our social sphere. Despite this success, these models can learn, perpetuate, and amplify harmful social biases. In this article, we present a comprehensive survey of bias evaluation and mitigation techniques for LLMs. We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing, defining distinct facets of harm and introducing several desiderata to operationalize fairness for LLMs. We then unify the literature by proposing three intuitive taxonomies, two for bias evaluation, namely, metrics and datasets, and one for mitigation. Our first taxonomy of metrics for bias evaluation disambiguates the relationship between metrics and evaluation datasets, and organizes metrics by the different levels at which they operate in a model: embeddings, probabilities, and generated text. Our second taxonomy of datasets for bias evaluation categorizes datasets by their structure as counterfactual inputs or prompts, and identifies the targeted harms and social groups; we also release a consolidation of publicly available datasets for improved access. Our third taxonomy of techniques for bias mitigation classifies methods by their intervention during pre-processing, in-training, intra-processing, and post-processing, with granular subcategories that elucidate research trends. Finally, we identify open problems and challenges for future work. Synthesizing a wide range of recent research, we aim to provide a clear guide of the existing literature that empowers researchers and practitioners to better understand and prevent the propagation of bias in LLMs.

Overview

CoChat Summary

TL;DR

This survey consolidates, formalizes, and taxonomizes bias evaluation and mitigation for large language models, covering metrics, datasets, and mitigation techniques. It proposes three taxonomies and identifies open problems around power imbalances, fairness conceptualization, evaluation standards, mitigation scalability, and theoretical guarantees.

Key findings

  • The survey proposes three taxonomies: bias evaluation metrics organized by what they use from a model (embeddings, probabilities, generated text); bias evaluation datasets organized by structure (counterfactual inputs vs. prompts); and bias mitigation techniques organized by intervention stage (pre-processing, in-training, intra-processing, post-processing).
  • It formalizes social bias for LLMs as representational harms (e.g., derogatory language, disparate system performance, erasure, exclusionary norms, misrepresentation, stereotyping, toxicity) and allocational harms (direct and indirect discrimination), and introduces fairness desiderata including fairness through unawareness, invariance, equal social group associations, equal neutral associations, and replicated distributions.
  • It reports limitations across metric classes: embedding-based and probability-based metrics may be only weakly or inconsistently correlated with downstream task bias, and generated-text metrics can be sensitive to decoding parameters; it also highlights reliability and validity issues in widely used datasets.
  • It consolidates publicly available datasets, with reported sizes including CrowS-Pairs (1,508), StereoSet (16,995), BBQ (58,492), RealToxicityPrompts (100,000), BOLD (23,679), HolisticBias (460,000), and Bias NLI (5,712,066).

Methods

  • Design: a comprehensive survey and taxonomy-building effort rather than an empirical study, synthesizing prior work on bias evaluation and mitigation for LLMs.
  • Scope: focuses on bias issues in LLMs for English, with additional languages for machine translation and multilingual models, and restricts inclusion to works that propose novel closed-form metrics, datasets, or mitigation techniques and specify applicability to social bias or fairness.
  • Analysis: organizes metrics, datasets, and mitigation techniques into taxonomies, formalizes several metrics and mitigation objectives with unified notation, discusses limitations of each class, and outlines open problems and recommendations.

Limitations

  • The paper states that it aligns with a technical solutionist perspective rather than a critical theoretical one, and that its taxonomies are organized by technical implementation details instead of downstream usage contexts or harms.
  • The paper states that it focuses narrowly on model training and evaluation, rather than a more holistic pipeline including problem formulation, data collection, and deployment/integration.
  • The paper states that it is limited to English language papers; evident from the text, it also does not report a systematic search protocol, search dates, or screening counts.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1162/coli_a_00524

Cites

313 of 313 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Image 1
CC-BY-NC-ND · From the PDF
Figure 2 Evaluation Taxonomy. For an arbitrary dataset selected for a given task, there is a subset of appropriate evaluation metrics that may measure model performance or bias.
Figure 2 Evaluation Taxonomy. For an arbitrary dataset selected for a given task, there is a subset of appropriate evaluation metrics that may measure model performance or bias.CC-BY-NC-ND · From the PDF
Figure 3 Example Embedding-Based Metrics (§ 3.3). Sentence-level encoders produce sentence embeddings that can be assessed for bias. Embedding-based metrics use cosine similarity to compare words like "doctor" to social group terms like "man." Unbiased embeddings should have similar cosine similarity to opposing social group terms.
Figure 3 Example Embedding-Based Metrics (§ 3.3). Sentence-level encoders produce sentence embeddings that can be assessed for bias. Embedding-based metrics use cosine similarity to compare words like "doctor" to social group terms like "man." Unbiased embeddings should have similar cosine similarity to opposing social group terms.CC-BY-NC-ND · From the PDF
Figure 4 Example Probability-Based Metrics (§ 3.4). We illustrate two classes of probability-based metrics: masked token metrics and pseudo-log-likelihood metrics. Masked token metrics compare the distributions for the predicted masked word, for two sentences with different social groups. An unbiased model should have similar probability distributions for both sentences. Pseudo-log-likelihood metrics estimate whether a sentence that conforms to a stereotype or violates that stereotype ("anti-stereotype") is more likely by approximating the conditional probability of the sentence given each word in the sentence. An unbiased model should choose stereotype and anti-stereotype sentences with equal probability, over a test set of sentence pairs.
Figure 4 Example Probability-Based Metrics (§ 3.4). We illustrate two classes of probability-based metrics: masked token metrics and pseudo-log-likelihood metrics. Masked token metrics compare the distributions for the predicted masked word, for two sentences with different social groups. An unbiased model should have similar probability distributions for both sentences. Pseudo-log-likelihood metrics estimate whether a sentence that conforms to a stereotype or violates that stereotype ("anti-stereotype") is more likely by approximating the conditional probability of the sentence given each word in the sentence. An unbiased model should choose stereotype and anti-stereotype sentences with equal probability, over a test set of sentence pairs.CC-BY-NC-ND · From the PDF
Image 5
CC-BY-NC-ND · From the PDF
Image 6
CC-BY-NC-ND · From the PDF
Figure 7
Figure 7 CC-BY-NC-ND · From the PDF
Figure 8 Example In-Training Mitigation Techniques (§ 5.2). We illustrate four classes of methods that modify model parameters during training. Architecture modifications change the configuration of the model, such as adding new trainable parameters with adapter modules as done in this example (Lauscher, Lueken, and Glavaš 2021). Loss function modifications introduce a new optimization objective, such as equalizing the embeddings or predicted probabilities of counterfactual tokens or sentences. Selective parameter updates freeze the majority of the weights and only tune a select few during fine-tuning to minimize forgetting of pre-trained language understanding. Filtering model parameters, in contrast, freezes all pre-trained weights and selectively prunes some based on a debiasing objective.
Figure 8 Example In-Training Mitigation Techniques (§ 5.2). We illustrate four classes of methods that modify model parameters during training. Architecture modifications change the configuration of the model, such as adding new trainable parameters with adapter modules as done in this example (Lauscher, Lueken, and Glavaš 2021). Loss function modifications introduce a new optimization objective, such as equalizing the embeddings or predicted probabilities of counterfactual tokens or sentences. Selective parameter updates freeze the majority of the weights and only tune a select few during fine-tuning to minimize forgetting of pre-trained language understanding. Filtering model parameters, in contrast, freezes all pre-trained weights and selectively prunes some based on a debiasing objective.CC-BY-NC-ND · From the PDF
Figure 9 Example Intra-Processing Mitigation Techniques (§ 5.3). We show several methods that modify a model's behavior without training or fine-tuning. Constrained next-token search may prohibit certain outputs during beam search ( e.g. , a derogatory term "@&!," in this example), or generate and rerank alternative outputs ( e.g. , "he" replaced with "she"). Modified token distribution redistributes next-word probabilities to produce more diverse outputs and avoid biased tokens. Weight distribution, in this example, illustrates how post hoc modifications to attention matrices may narrow focus to less stereotypical tokens (Zayed et al. 2023b). Modular debiasing networks fuse the main LLM with stand-alone networks that can remove specific dimensions of bias, such as gender or racial bias.
Figure 9 Example Intra-Processing Mitigation Techniques (§ 5.3). We show several methods that modify a model's behavior without training or fine-tuning. Constrained next-token search may prohibit certain outputs during beam search ( e.g. , a derogatory term "@&!," in this example), or generate and rerank alternative outputs ( e.g. , "he" replaced with "she"). Modified token distribution redistributes next-word probabilities to produce more diverse outputs and avoid biased tokens. Weight distribution, in this example, illustrates how post hoc modifications to attention matrices may narrow focus to less stereotypical tokens (Zayed et al. 2023b). Modular debiasing networks fuse the main LLM with stand-alone networks that can remove specific dimensions of bias, such as gender or racial bias.CC-BY-NC-ND · From the PDF
Image 10
CC-BY-NC-ND · From the PDF

Verification

Claims the paper makes

121 Valid · 12 Not valid · 18 Not checked · 106 Own results and interpretations

71 more claims · 34 with their cited sources checked

Create a free account to see all 257 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 151
  • It may state two separate claims as one. 70

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

1 more claim about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 313 references

Research works printed but not linked: 5

  1. Barocas, 2019, Fairness and Machine Learning: Limitations and Opportunities
  2. Blodgett, Su Lin . 2021. Sociolinguistically Driven Approaches for Just Natural Language Processing. Ph.D. thesis. University of Massachusetts Amherst.
  3. Green, 2019, ”Good” isn’t good enough, 1
  4. Radford, 2019, Language models are unsupervised multitask learners, OpenAI Blog, 1, 9
  5. Wang, 2019, BERT has a mouth, and it must speak: BERT as a Markov random field language model, 30, 10.18653/v1/w19-2304 · doi:10.18653/v1/w19-2304

Cited by

Items 301–400 of 763

Metadata

Authors
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K.
Affiliations
  • Stanford University (US)
  • Adobe Research. ryrossi@adobe.com
  • Pattern Data. joe.barrow@patterndataworks.com
  • Adobe Research. tanjim@adobe.com
  • Adobe Research. sukim@adobe.com
  • Adobe Research. dernonco@adobe.com
  • Adobe Research. tyu@adobe.com
  • Adobe Research. ruizhang@adobe.com
  • Intel (United States) (US)
Venue
Computational Linguistics
Publisher
MIT Press
Published
2024
Volume
50
Issue
3
Pages
1097-1179
ISSN
  • 0891-2017 (ISSN-L)
  • 1530-9312
OpenAlex
W4399528455
Cited by
1,245
References
280
Citations by year
2026: 596 · 2025: 451 · 2024: 188
FWCI
132.76
Type
journal article
Language
en
Open access
Locations
  • Computational Linguistics (publishedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
large language models, social bias, fairness, bias assessment, bias mitigation, bias metrics, data preprocessing, post-processing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-11-02, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-02 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.