Bias and Fairness in Large Language Models: A Survey
Open accessThe DOI matches its Crossref record; 313 of the 313 references it cites were found in a registry or an index; 151 of the 183 sources its claims cite were checked for support.
Abstract
Rapid advancements of large language models (LLMs) have enabled the processing, understanding, and generation of human-like text, with increasing integration into systems that touch our social sphere. Despite this success, these models can learn, perpetuate, and amplify harmful social biases. In this article, we present a comprehensive survey of bias evaluation and mitigation techniques for LLMs. We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing, defining distinct facets of harm and introducing several desiderata to operationalize fairness for LLMs. We then unify the literature by proposing three intuitive taxonomies, two for bias evaluation, namely, metrics and datasets, and one for mitigation. Our first taxonomy of metrics for bias evaluation disambiguates the relationship between metrics and evaluation datasets, and organizes metrics by the different levels at which they operate in a model: embeddings, probabilities, and generated text. Our second taxonomy of datasets for bias evaluation categorizes datasets by their structure as counterfactual inputs or prompts, and identifies the targeted harms and social groups; we also release a consolidation of publicly available datasets for improved access. Our third taxonomy of techniques for bias mitigation classifies methods by their intervention during pre-processing, in-training, intra-processing, and post-processing, with granular subcategories that elucidate research trends. Finally, we identify open problems and challenges for future work. Synthesizing a wide range of recent research, we aim to provide a clear guide of the existing literature that empowers researchers and practitioners to better understand and prevent the propagation of bias in LLMs.
Overview
CoChat Summary
TL;DR
This survey consolidates, formalizes, and taxonomizes bias evaluation and mitigation for large language models, covering metrics, datasets, and mitigation techniques. It proposes three taxonomies and identifies open problems around power imbalances, fairness conceptualization, evaluation standards, mitigation scalability, and theoretical guarantees.
Key findings
- The survey proposes three taxonomies: bias evaluation metrics organized by what they use from a model (embeddings, probabilities, generated text); bias evaluation datasets organized by structure (counterfactual inputs vs. prompts); and bias mitigation techniques organized by intervention stage (pre-processing, in-training, intra-processing, post-processing).
- It formalizes social bias for LLMs as representational harms (e.g., derogatory language, disparate system performance, erasure, exclusionary norms, misrepresentation, stereotyping, toxicity) and allocational harms (direct and indirect discrimination), and introduces fairness desiderata including fairness through unawareness, invariance, equal social group associations, equal neutral associations, and replicated distributions.
- It reports limitations across metric classes: embedding-based and probability-based metrics may be only weakly or inconsistently correlated with downstream task bias, and generated-text metrics can be sensitive to decoding parameters; it also highlights reliability and validity issues in widely used datasets.
- It consolidates publicly available datasets, with reported sizes including CrowS-Pairs (1,508), StereoSet (16,995), BBQ (58,492), RealToxicityPrompts (100,000), BOLD (23,679), HolisticBias (460,000), and Bias NLI (5,712,066).
Methods
- Design: a comprehensive survey and taxonomy-building effort rather than an empirical study, synthesizing prior work on bias evaluation and mitigation for LLMs.
- Scope: focuses on bias issues in LLMs for English, with additional languages for machine translation and multilingual models, and restricts inclusion to works that propose novel closed-form metrics, datasets, or mitigation techniques and specify applicability to social bias or fairness.
- Analysis: organizes metrics, datasets, and mitigation techniques into taxonomies, formalizes several metrics and mitigation objectives with unified notation, discusses limitations of each class, and outlines open problems and recommendations.
Limitations
- The paper states that it aligns with a technical solutionist perspective rather than a critical theoretical one, and that its taxonomies are organized by technical implementation details instead of downstream usage contexts or harms.
- The paper states that it focuses narrowly on model training and evaluation, rather than a more holistic pipeline including problem formulation, data collection, and deployment/integration.
- The paper states that it is limited to English language papers; evident from the text, it also does not report a systematic search protocol, search dates, or screening counts.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures










Verification
Claims the paper makes
121 Valid · 12 Not valid · 18 Not checked · 106 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Despite the growing emphasis on addressing these issues, bias and fairness research in LLMs often fails to precisely describe the harms of model behaviors: who is harmed, why the behavior is harmful, and how the harm reflects and reinforces social principles or hierarchies (Blodgett et al. 2020).”
What the sources say
- Partly supportedcites Blodgett et al. 2020 matched to “Language (Technology) is Power: A Critical Survey of “Bias” in NLP” (2020) sourceExperimental — accuracy not yet measured
Section 3.1 (Motivations; page not provided) finds that papers on bias in NLP often fail to explain which model behaviors are harmful, how, to whom, and why. Sections 4.1–4.2 call for attention to social hierarchies and explicit normative reasoning. The survey concerns NLP research broadly, however, and does not establish that this failure is common specifically in LLM research. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Similar to the shortcomings of embedding-based metrics, Delobelle et al. (2022) and Kaneko, Bollegala, and Okazaki (2022) point out that probability-based metrics may be only weakly correlated with biases that appear in downstream tasks”
- Also stated in: § 3.6 Recommendations · p. 27 · plain
“Probability-based metrics also show weak correlations with downstream biases.”
What the sources say
- Supportedcites Delobelle et al. 2022 matched to “Measuring Fairness with Biased Rulers: A Comparative Study on Bias Metrics for Pre-trained Language Models” (2022) source
Section 3 describes CrowS-Pairs as a pseudo-loglikelihood-based metric. Section 4.4 reports that CrowS-Pairs correlates less with two measures of bias in downstream tasks, supporting the hedged claim that probability-based metrics may correlate weakly with downstream bias. The finding does not establish that all probability-based metrics do so. (LLM, conf=0.90)
- Supportedcites Kaneko, Bollegala, and Okazaki 2022 matched to “Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Tasks” (2022) source
The Introduction (p. 1) identifies intrinsic measures as using likelihood scores assigned by MLMs and reports only weak correlations with downstream, task-specific bias measures. Section 5.2 (p. 8) likewise finds weak and inconsistent correlations between intrinsic and extrinsic bias scores. (LLM, conf=0.93)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 3.6 Recommendations · p. 27 · plain
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Many approaches, for instance, assume some implicitly desirable criterion ( e.g. , a model output should be independent of any social group in the input), but do not explicitly acknowledge or state the normative social values that justify their framework.”
What the sources say
- Partly supportedcites Blodgett et al. 2021 matched to “Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets” (2021) sourceExperimental — accuracy not yet measured
Section 2.3 describes assumptions about ideal model behavior, including equal probabilities for sentences about different target groups. Section 6 says the benchmarks make implicit decisions about stereotyping that are not explicitly discussed or justified. The paper examines four stereotyping benchmarks, however, rather than establishing this claim about many bias and fairness approaches generally or specifically about their normative social values. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Cabello, Jørgensen, and Søgaard (2023) illustrate that associations between the representations of protected attribute and other words can be independent of downstream performance disparities, if certain assumptions of social groups' language use are violated.”
What the sources say
- Supportedcites Cabello, Jørgensen, and Søgaard 2023 matched to “On the independence of association bias and empirical fairness in language models” (2023) source
The Bias section defines association bias as differences in how demographic-group terms are encoded and empirical fairness as performance differences across social groups. Section 5 says the two can be independent and explains that linking them relies on the In-Group Affinity Assumption about how groups use those terms; it calls that assumption problematic. No page numbers are provided in our copy. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Each stakeholder brings perspectives that may specify different fairness constraints for the same application and setting.”
What the sources say
- Partly supportedcites Jernite et al. (2022) matched to “Data Governance in the Age of Large-Scale Data-Driven Language Technology” (2022) sourceExperimental — accuracy not yet measured
Section 2.2 (p. 4) says participants bring different perspectives and warns that relying only on their values could exclude views more relevant to other language users. The paper discusses nondiscrimination and local expressions of values, but does not say that stakeholders specify different fairness constraints for the same application and setting. (LLM, conf=0.85)
- Not matchedcites Bender (2019)
An author–year citation the paper’s reference list doesn’t settle. We don’t search these by name.
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
71 more claims · 34 with their cited sources checked
Create a free account to see all 257 claims and every source verdict.
- Probably not a substantial claim. 151
- It may state two separate claims as one. 70
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While this process mitigates some risks inherent to AI-generated content, it does not eliminate potential biases associated with proprietary large language models, which lack transparency regarding training data and internal parameters [10].”
From AI-Generated Multiple Mini Interview (MMI) Stations for Medical School Admissions: Psychometric Evaluation (2026) · cites this paper as [10]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Notably, there is growing attention on assessing LLM bias [63-64] and robustness [323-324].”
From A Survey of Large Language Models (2026) · cites this paper as [63]
The cited source says what the claim attributes to it.
In the citing paper
“these models pose serious risks, as the vast, unfiltered internet data used to train them can reinforce and amplify harmful biases, disproportionately harming marginalized communities (73).”
From Emergent social conventions and collective bias in LLM populations (2025) · cites this paper as [73]
The cited source says what the claim attributes to it.
In the citing paper
“Despite the use of various mitigation techniques, such as debiasing algorithms, curating balanced datasets, or incorporating fairness-focused training objectives, eliminating bias entirely is a persistent challenge141–143.”
From Current applications and challenges in large language models for patient care: a systematic review (2025) · cites this paper as [142]
The cited source says what the claim attributes to it.
In the citing paper
“Another major concern with LLMs is their propensity to amplify historical biases.”
From Evaluation of Large Language Models for Peer Review in Transplantation Research: Algorithm Validation Study (2026) · cites this paper as [30]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.1 Counterfactual Inputs · p. 28
“| WinoBias+ | 1,367”
§ 4.1 Counterfactual Inputs · p. 29
“WinoBias+ (Vanmassenhove, Emmery, and Shterionov 2021) augments WinoBias with gender-neutral alternatives, similar to Winogender's neutral option, with 3,167 total instances.”
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 6×
“, 2020), and GPT-4 (OpenAI, 2023); autoencoding models include BERT (Devlin et al., 2019), RoBERTa (Liu et al.”
BackgroundInfluential(2014) · Cited by 33,419
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 2×
“, 2019), GPT-3 (Brown et al., 2020), and GPT-4 (OpenAI, 2023); autoencoding models include BERT (Devlin et al.”
Background"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,690
Gauriat, C.-M., Pencolé, Y., Ribot, P., & Brouillet, G. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.27 · Cited by 12,586
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“The rise and rapid advancement of large language models (LLMs) has fundamentally changed language technologies (e.g., Brown et al., 2020; Conneau et al., 2020; Devlin et al., 2019; Lewis et al., 2020; Liu et al., 2019; OpenAI, 2023; Radford et al., 2018; 2019; Raffel et al., 2020).”
BackgroundMeasuring individual differences in implicit cognition: the implicit association test
Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. K. (1998) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.74.6.1464 · Cited by 9,210 · cited 2×
“, family and occupation words), emulating the Implicit Association Test (Greenwald et al., 1998).”
Method
Research works printed but not linked: 5
- Barocas, 2019, Fairness and Machine Learning: Limitations and Opportunities
- Blodgett, Su Lin . 2021. Sociolinguistically Driven Approaches for Just Natural Language Processing. Ph.D. thesis. University of Massachusetts Amherst.
- Green, 2019, ”Good” isn’t good enough, 1
- Radford, 2019, Language models are unsupervised multitask learners, OpenAI Blog, 1, 9
- Wang, 2019, BERT has a mouth, and it must speak: BERT as a Markov random field language model, 30, 10.18653/v1/w19-2304 · doi:10.18653/v1/w19-2304
Cited by
Items 601–700 of 763
Sociodemographic Biases in Educational Counselling by Large Language Models
Adamczyk, T., Mieleszczenko-Kowszewicz, W., Bajcar, B., Chodak, G., Szczęsny, A., Markiewicz, M., Ostrowska, K., Sawczuk, A., & Kazienko, P. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29773-0_40
Going PLACES: Participatory Localized Red Teaming for Text-to-Image Safety in the Global South
Rastogi, C., Bhutani, M., Kahng, M., Muhammad, S. H., Razumovskaia, E., Suresh, P., Ahmad, I. S., Kalia, C., Mahomed, Y., Maji, M., et al. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812296 · Cited by 4
Generative AI-Supported Assessment of Engineering Capstone Proposals: A South African Case Study
Klopper, B., Blackie, H., Dyk, L. V., & Marais, H. (2026) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon67543.2026.11574460
“The degree of systematic bias in LLM-generated proposal evaluations, if present in the model, may vary with project domain, choice of solution, or the general writing style evident in a given proposal [21], [22].”
Fairness-Aware LLM-Based Automated Essay Scoring with Explainable Feedback
Upadhyay, S. (2026) · International Conference on Information Technology Based Higher Education and Training (ITHET) · doi:10.1109/ithet69978.2026.11585121
Designing Bias-Aware AI Feedback in Interactive Dialogues: Insights from a Pilot Study
Marsden, N., Herling, C., & Reichert, T. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30846-7_32
Langer, Y., Nguyen, C., & Fischer, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29596-5_9
Grenzebach, J., & Radüntz, T. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29596-5_7
Analyzing and Mitigating Bias in LLMs through Cross-Model Consensus
Xiao, K., Yang, L. T., Zhang, D. D., Tulajiang, P., & Lin, H. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11582018
“The discussion of biases in Large Language Models (LLMs) began alongside the early surge in their widespread adoption [12].”
Exploring the Potential of Large Language Models for Counter Argument GenerationPreprint
Mumtaz, F., Rauf, S. A., Nauman, S. I., Malik, M. G. A., & Iqbal, M. (2026) · Research Square · doi:10.21203/rs.3.rs-10041296/v1
Hong, C.-H., Liu, Y.-Z., & Chen, C. (2026) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2026.100350
Lv, S., Li, Y., Chen, Y., Lin, F., & Cheng, G. (2026) · International Conference on Signal Image Processing and Communication (ICSIPC) · doi:10.1109/icsipc69751.2026.11583935
“These studies indicate that to enhance the credibility of evaluation, the evaluators themselves must be placed within the evaluated environment [8].”
Exploring motivations, response quality, and fit in ChatGPT-assisted information tasks
Li, Y., & Marcum, J. H. (2026) · Journal of Librarianship and Information Science · doi:10.1177/09610006261461308
Evaluating the Usage of African-American Vernacular English in Large Language Models
Davis, D., & McCoy, R. T. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29903-1_15 · Cited by 1
StackingNet: Collective Inference Across Independent AI Foundation Models
Li, S., Liu, C., Wu, D., Zeng, Z., & Ding, L. (2026) · Advanced Science · doi:10.1002/advs.76488
Bias Beneath the Tone: Empirical Characterisation of Tone Bias in LLM-Driven UX Systems
Bodara, H., Mushfiq, M. M., & Siddiqui, I. F. (2026) · doi:10.1145/3786169.3788401
“Finally, Gallegos et al. [23] survey bias and fairness in LLMs, noting that stylistic and tonal patterns form a distinct, under-studied dimension of bias.”
Guey, W., Zhang, W., Rau, P. P., Bougault, P., Moura, V., Ucar, B., & Gomes, J. O. (2026) · Work · doi:10.1177/10519815261469150
“In addition, bias is often measured using heterogeneous output formats, open-ended generations, or task-specific metrics that hinder direct comparison across models and settings[23, 24].”
Enhancing bias detection in text classification models through high-quality diverse test cases
Djennane, L., Kardkovács, Z. T., Benatallah, B., Gaci, Y., Gaaloul, W., & Farah, Z. (2026) · Information and Software Technology · doi:10.1016/j.infsof.2026.108253
CHORD: Context Homogeneity Analysis for Robust Defense Against RAG Poisoning Attacks
Yao, R., Gao, N., & Tu, C. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-3541-4_6
Ali, N., & Kim, S. (2026) · doi:10.2196/preprints.106945
Self-citationAssisting or resisting patriarchy? a critical discourse analysis of chatgpt’s responses on feminism
Biswas, A., Sulzer, M., & Hasan, M. U. (2026) · AI and Ethics · doi:10.1007/s43681-026-01220-1
Equal Access, Unequal Interaction: A Counterfactual Audit of Large Language Models Fairness
Amiri-Margavi, A., Gharagozlou, A., Davodi, A. G., Davoudi, S. P. M., & Balyani, H. H. (2026) · IFIP advances in information and communication technology · doi:10.1007/978-3-032-30805-4_15 · Cited by 5
Schrottenbacher, P., Baumartz, D., Abrami, G., Mehler, A., & Gombert, S. (2026) · SoftwareX · doi:10.1016/j.softx.2026.103188
Hassan, I. A., & Graham, Y. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809902
A Survey of Methods for Enhancing the Robustness of Large Language Models
Li, D., & Li, D. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3435-6_29
David, B., Isingoma, B., Nakatumba-Nabende, J., & Jjingo, D. (2026) · Cureus Journal of Computer Science. · doi:10.7759/s44389-026-00167-y
NavDreamer: Video Models as Zero-Shot 3D Navigators
Huang, X., Gai, W., Wu, T., Wang, C., Zheng, Q., Liu, Z., Zhou, X., Wu, Y., & Gao, F. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3713727 · Cited by 16
Mitigating Bias in LLM-Augmented Knowledge Graphs: A Holistic Overview
Ibrahim, A., Kashef, R., & Naseribonari, S. (2026) · IEEE International Conference on Smart Sustainable Systems for Computer and Engineering Applications (3SCEA) · doi:10.1109/3scea68071.2026.11603048
“These effects can resurface following instruction tweaks and are consistent across decoding strategies and prompts, as well as amplification or reinforcement through user modelling [10].”
Fairness in Multi-Agent Systems for Software Engineering: An SDLC-Oriented Rapid Review
Yang-Smith, C., Santos, R. D. S., & Abdellatif, A. (2026) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3803437.3806715
BiasAgent: Exploiting Agent Bias for Preference Manipulation Attacks on Model Context Protocol
Guo, J. H., Wang, Z., Jiang, W., Zhang, R. Y., Xiong, J., Song, Q., Wu, H. B., & Liu, Y.-J. (2026) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2026.3714063
“With the increasing deployment of LLMs across various domains, recent studies have shown that these models may exhibit systematic biases in their responses [13], [22].”
Self-citationSchlicht, I. B., Korenčić, D., Chulvi, B., Flek, L., & Rosso, P. (2026) · Expert Systems · doi:10.1111/exsy.70360
“LLMs are known to be prone to different kinds of biases (Gallegos et al. 2024).”
Fabricated Foundations: Synthetic Data and Bias in Machine Learning
Gupta, D., Khamidova, M., Norov, J., Rakhmonkulova, F., Gaffarovich, N. M., Ashirova, A. I., & Bayramovich, S. B. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-0368-8.ch001
Alam, A. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-7749-0.ch003
Rajendran, V., & Lin, S.-P. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-33381-0_22
Style-Based Bias Backdoor Attacks on Medical LLMs Under In-Context Learning
Jiang, Y., Xiao, S., Lin, J., Hao, F., & Gu, T. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3426-4_7
Can LLMs Hire Fairly? Racial Bias in Resume ScreeningPreprint
Gao, Z., Jiang, W., & Yan, Y. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7009038
A Road Map for Tamed Interactions with Large Language Models
Scotti, V., Keim, J., Hey, T., Metzger, A., Koziolek, A., & Mirandola, R. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3832776 · Cited by 2
“Additionally, content generated by LLMs exposes to risks related to bias and toxicity [44], besides misinformation and hallucination .”
Vidyasagar, G., Chandana, B., Sravani, G., Jhansi, A., Reddy, L. S., & Venu, K. (2026) · IEEE International Conference on Intelligent Technologies for a Sustainable and Inclusive Future (ICITSIF) · doi:10.1109/icitsif69060.2026.11608763
“More recent research has demonstrated that language models can be used as zero-shot rankers, and therefore can be used to perform ranking and recommendation tasks without re-training to a large extent [8].”
Crescentini, N., Gambardella, D., Lumino, R., & Padricelli, G. M. (2026) · WELFARE E ERGONOMIA · doi:10.3280/we2026-001006
Molle, A., Saenen, N. D., Smeets, K., Bijnens, K., Aerts, M., Kremer, C., Solazzo, E., & Abrahantes, J. C. (2026) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2026.106182
Navigating Boundaries of AI Use in Physical Education Journal Research Publications
Li, Y., Wu, C., Su, Y., Lu, B., Hamilton, X. D., Lynch, T., & Hamilton, M. F. (2026) · Quest · doi:10.1080/00336297.2026.2707164
AlDahoul, N., Karim, H. B. A., & Tan, M. J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7008282
Reina, L. A. V., Ruiz-Fernández, V., & Falcão, J. (2026) · Compàs d amalgama · doi:10.1344/compas2026.13.8
Exploring Bias in Medical Applications of Large Language Models: Protocol for a Systematic Review
Rao, S. G., Boutrous, C., Kormilitzin, A., Tai, X. Y., Hotchkiss, L., Cong, C., Meinert, E., Liang, H., Dong, H., & Harrison, J. R. (2026) · JMIR Research Protocols · doi:10.2196/91348
Shen, Y., Yang, W., & Shen, Y. (2026) · IEEE Access · doi:10.1109/access.2026.3717981
“Cognitive-bias work in LLMs [2], [15], [16], [17] probes biases inside models on medical or general benchmarks, not bias expressed in corporate disclosures, and stops at detection without mitigation.”
Li, C., & Zhang, M. (2026) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.70360
“With the widespread adoption of large language models (LLMs) across diverse natural language processing (NLP) tasks, ensuring the fairness of their generated outputs has become a critical concern [ 3, 4 ].”
Yan, P. S., Li, H., Chen, N., Li, L., & Chen, P. (2026) · Theory and Society · doi:10.1007/s11186-026-09735-6 · Cited by 1
“Nevertheless, critics caution against underlying bias and a lack of representation in LLMs, which may lead to a misrepresentation of human society or flat - tening of group variances (Gallegos et al., 2024; Wang et al., 2025).”
Tracing Stereotypes in Pre-trained Transformers: From Biased Neurons to Fairer Models
Voria, G., Openja, M., Khomh, F., Catolino, G., & Palomba, F. (2026) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3793302.3793356 · Cited by 3
Can large language models help in the assessment of people with pain?
Amidei, J., Kaltenbrunner, A., Sá, J. G. F. D., & Nieto, R. (2026) · Pain Management · doi:10.1080/17581869.2026.2710351
Kavaklı, N., & Kikutadze, V. (2026) · European Scientific Journal ESJ · doi:10.19044/esj.2026.v22n20p98
Iarovoi, D., & Teh, P. L. (2026) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-032-34792-3_11
Assessing Linguistic and Content Characteristics of First-Grade Students’ Texts Using Generative AI
Then, D., Theurer, C., Cropley, D., & Pohlmann-Rother, S. (2026) · Education Sciences · doi:10.3390/educsci16081227
“However, this can be viewed critically as human raters are shown to be affected by biases—some of which are even reproduced by LLMs (Gallegos et al., 2024).”
Zhu, Y., Wang, C., & Chu, J. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2709241 · Cited by 1
Aghabayli, V. (2026) · Dialogica · doi:10.5709/dialogica-02.s1.2026-09
MacKay, M., Gosselin, S., Sonier, K., Wong, J., Zamani, S., Zarrinkalam, F., & McWhirter, J. E. (2026) · Journal of Health Communication · doi:10.1080/10810730.2026.2709807
Quantifying Social Biases in Language Model Classifiers is Domain-Dependent
Quiroga, T., Bravo-Márquez, F., & Barrière, V. (2026) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3834859
A systematic review of standards on bias and data quality from horizontal and vertical perspectives
Schmitz, A., & Poretschkin, M. (2026) · Journal of Data and Information Quality · doi:10.1145/3832313 · Cited by 1
Implicit Bias in Large Language Model Diagnosis of Eating Disorders: Experimental Vignette Study
Mccalla, D., Li, B., Jaeger, S., Jacques, J. G., Gonzalez, L., Silber, C. L., & Dykeman, C. (2026) · JMIR AI · doi:10.2196/93498 · Cited by 1
“Alternatively, model outputs may reflect cultural stereotypes present in training text [33].”
Generating and analyzing personality questionnaires using large language models
Monsa, R., Zohar, A., & Arzy, S. (2026) · iScience · doi:10.1016/j.isci.2026.116909
“64,65 Nonetheless, in the context of personality questionnaire development, the model’s broad knowledge base, drawn from diverse internet sources, may help mitigate some of the limitations associated with individual human biases by offering a wider and more integrative perspective.”
Bias and Fairness in LLM-Based Recruitment: A Systematic Review
Moutafail, A. E., & Belkhoutout, K. (2026) · EPJ Web of Conferences · doi:10.1051/epjconf/202638100015 · Cited by 2
The fallacy of AI-generated podcasts: the case of Google NotebookLM
Musi, E., Harris, R., & Yates, S. J. (2026) · Discourse Context & Media · doi:10.1016/j.dcm.2026.101049
Reinforcement Learning in the Era of Large Language Models: Challenges and Opportunities
Hao, Q., Chen, L., Qi, X.-K., Yuan, Y., Zong, Z., Chen, H. S., Zhao, K., Wang, S., Zhang, Y., Yuan, J., et al. (2026) · ACM Computing Surveys · doi:10.1145/3837057
BALM: Bias-Aware Generation for Large Language Models
Dorleon, G., & Shujaa, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-34896-8_2
Just, S. A., & Elvevåg, B. (2026) · Digital Health · doi:10.1177/20552076261478845
“29,40 Yet these safeguards solve problems that the new solutions themselves create and do not address the discussed redefinition of care.”
Yi, R., Li, Z., & Zhang, W. (2026) · SpringerBriefs in computer science · doi:10.1007/978-3-032-28658-1_5
Implications for Fairness-Aware Institutional NLP
Yi, R., Li, Z., & Zhang, W. (2026) · SpringerBriefs in computer science · doi:10.1007/978-3-032-28658-1_6
Improving LLM-based event extraction with annotation guidelines
Geromel, M., & Cimiano, P. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1797435
Theoretical Foundations: From Sociolinguistics to Computational Fairness
Yi, R., Li, Z., & Zhang, W. (2026) · SpringerBriefs in computer science · doi:10.1007/978-3-032-28658-1_2
Conclusion: Toward Fairness-Aware NLP in Professional Domains
Yi, R., Li, Z., & Zhang, W. (2026) · SpringerBriefs in computer science · doi:10.1007/978-3-032-28658-1_7
Rouabhia, R. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-9028-4.ch008
CEO-Reputation in Large Language Models
Heilmann, I. (2026) · Springer eBooks · doi:10.1007/978-3-658-52660-3_6
Indigenous Knowledge Systems and AI
Gupta, D., Gayratovich, E. N., Akishova, S., Rasulova, I., Sottarov, A., & Nazirjon, R. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-7837-4.ch008
Deb, K. S., Mukherjee, I., & Sanyal, D. K. (2026) · Innovations in Systems and Software Engineering · doi:10.1007/s11334-026-00651-7
Iancu, C. L., Oprea, S., Bârã, A., & Iordache, D.-M. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-01108-4
“(Gallegos et al. 2024) provide a comprehensive survey of bias and fairness in LLMs, systematically cataloguing evaluation metrics, adversarial debiasing strategies and mitigation pipelines tailored to generative models, demonstrating that surface-level safeguards are frequently insufficient and…”
LLM-Assisted Reviewer Assignment via Auditable Expertise Matching
Bagheri, F., Buscaldi, D., & Recupero, D. R. (2026) · IEEE Access · doi:10.1109/access.2026.3724622
“LLMs may produce fluent but incorrect content, exhibit hallucination behavior, encode societal biases, and be susceptible to prompt injection or adversarial manipulation [18], [19], [20], [21].”
The Risks of Sycophantic GenAI in Community Sport and Leisure
Yeomans, C., Nguyen, J., Bowell, P., & Demsar, V. (2026) · Leisure Sciences · doi:10.1080/01490400.2026.2717277
“A growing literature examines bias in large language models (LLMs) (Gallegos et al., 2024; Nguyen, 2024; Zewail et al., 2026), with implications for sport, leisure, and society.”
Stratified regulation: unpacking value alignment of Chinese LLMs
Liu, M., Su, Y., & Meng, T. (2026) · Chinese Sociological Review · doi:10.1080/21620555.2026.2714528
StarDTox : Is Fairness in Language Models Just a Few Prompts Away?
Tahmasebi, S., Nikzad-Khasmakhi, N., Payberah, A. H., Asgari-Chenaghlu, M., & Matskin, M. (2026) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac69091.2026.00438
“While each of these strategies has merit, they also come with notable limitations [4], [8].”
InfluentialCohen, E., Adler, Y., & Gannot, R. N. (2026) · Healthcare · doi:10.3390/healthcare14162642
“Recent research emphasizes the necessity of robust taxonomies for identifying bias in large-scale models [8] and proposes innovative representation methods for debiasing pre-trained systems [9].”
GradeDrift-LLM: Measuring Student-History-Induced Score Drift in LLM-Based Automated Grading
Anghel, C., Anghel, A. A., Crăciun, M., Cocu, A., Moldovanu, S., Sandu, C., Dragosloveanu, C. D. M., Andrei, C. A., Vulpe, D. E., Maier, C., et al. (2026) · Machine Learning and Knowledge Extraction · doi:10.3390/make8080252
Adaptation machines/machine adaptation: adaptation studies and generative AI
Winckler, R. (2026) · Adaptation · doi:10.1093/adaptation/apag025 · Cited by 1
Chameleon: A Deception Defense Strategy Against LLM-Assisted Attacker in New Power Systems
Yao, Y., Lin, Y., Yang, Q., Li, P., Liu, Y., & Tian, Z. (2026) · IEEE International Conference on Joint Cloud Computing (JCC) · doi:10.1109/jcc72984.2026.00062
“…defense services can achieve enhanced trap efficacy by deliberately exploiting the preference patterns and reasoning vulnerabilities inherent to LLMs. LLMs tend to prioritize targets that exhibit typical ex-ploitable characteristics, which is closely related to the training method of LLMs [11].”
The Migration of Bias in Law: From Human Cognition to Predictive AI System and Large Language Models
Zhong, H. (2026) · Springer eBooks · doi:10.1007/978-3-031-87993-7_185-1 · Cited by 1
Large Language Models for Optimizing Patient Recruitment Decisions in Voice Data Generation Projects
Anibal, J., Nama, G. K. C., Ganesh, S., Nafii, Y., Cruz, S. S., Daoud, V., Zanin, M., Le, T., Khorrami, P., Wood, B. J., et al. (2026) · Laryngoscope Investigative Otolaryngology · doi:10.1002/lio2.70519
Belarache, O., Abbour, F. Z., Hannouni, S., Ounacer, S., & Azzouazi, M. (2026) · International Conference on Circuit, Systems and Communication (ICCSC) · doi:10.1109/iccsc71566.2026.11650176
“Fairness assessments based solely on sentiment polarity are unlikely to detect meaningful differences in aligned LLM outputs [3].”
InfluentialCicatiello, C., & Fusco, P. (2026) · Encyclopedia · doi:10.3390/encyclopedia6090182 · Cited by 1
“Technical mitigation can intervene before training, during training, at inference, or after generation [30].”
InfluentialVision Language Models Fail to Reliably Detect Acute Myeloid Leukemia in Bone Marrow SmearsPreprint
Schulze, F., Loeffler, C. M. L., Radoynova, M., Winter, S., Röllig, C., Sockel, K., Kroschinsky, F., Bornhäuser, M., Middeke, J. M., Kather, J. N., et al. (2026) · medRxiv · doi:10.64898/2026.08.19.26359329
Data security in large language models: risks, defense, and directions
Chen, K., Zhou, X., Yu, Y., Lin, Y., Chen, H., Cai, C., & Shen, L. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-01200-9 · Cited by 2
“These harms are called ’social bias,’ a subjective and normative term widely used to refer to the differential treatment or outcomes resulting from historical and structural power asymmetry between social groups (Gallegos et al. 2024).”
InfluentialKatikala, J., Sharma, K., Kumar, T. S. R., & Sharma, S. (2026) · IEEE International Conference on Artificial Intelligence Testing (AITest) · doi:10.1109/aitest70988.2026.00040
“This is based on previous work in formal specification, NLP reliability, and holistic language-model evaluation, which demonstrates that the notions of correctness, robustness, coherence, fairness, and safety cannot be presumed in interactive AI systems [8], [9]– [11].”
Cetin, N., & Atılan, A. U. (2026) · European Journal of Therapeutics · doi:10.58600/eurjther3197
“Such gaps reflect a persistent linguistic bias in how these models are trained, and make a strong case for testing reliability, cultural fit, and guideline adherence specifically in non-English settings [14].”
Energy Efficiency and Fairness of Local Large Language Models in Data Stream Processing
Tjeerdsma, H., Nasiri, R., Owotogbe, J., Kumara, I., Tamburri, D. A., & van den Heuvel, W. (2026) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud72782.2026.00040
“However, local deployment often requires hardware accelerators, consumes substantial energy, and can introduce bias in decision-making (i.e., discrimination based on attributes such as gender, race, or age) [8], [9].”
Visuri, I., & Svensson, J. (2026) · Religion & Education · doi:10.1080/15507394.2026.2716456
“If that data is biased, that bias may be reproduced in the text generation (see e.g. Gallegos et al., 2024).”
Gender Bias in Greek Pronoun Resolution: Evaluating Multilingual and Language-Specific LLMs
Diamantopoulos, K., & Chatzikyriakidis, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37249-9_15
Trustworthy Intelligent Vehicular Networks: A Survey
Zhang, X., Madhankumar, N., Acikbas, D., Raval, R., Wang, J., Song, Z., Xia, X., Lei, X., & Zhang, D. (2026) · ACM Computing Surveys · doi:10.1145/3844946
Mathuloe, T. J., Byrne, J. R., & Bresnihan, N. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7396760
Digital linguistic imperialism: can we decolonize English when the AI writing tools are colonial?
Hosseinpur, R. M. (2026) · Applied Linguistics · doi:10.1093/applin/amag073 · Cited by 1
Flattery from robots, praise from humans: building consumer trust and comfort
Liu, R., Ji, C., & Prentice, C. (2026) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-02-2026-0204
Yalçın, Ö. N. (2026) · Cambridge University Press eBooks · doi:10.1017/9781009639057.026
Perry, A., & Cameron, C. D. (2026) · Cambridge University Press eBooks · doi:10.1017/9781009639057.024
Technology in Diverse Populations: Issues of Inclusion, Accessibility, and Bias
Thames, A. D., & Cavanagh, L. (2026) · Oxford University Press eBooks · doi:10.1093/9780197814116.003.0011
Metadata
- Authors
- Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K.
- Affiliations
- Stanford University (US)
- Adobe Research. ryrossi@adobe.com
- Pattern Data. joe.barrow@patterndataworks.com
- Adobe Research. tanjim@adobe.com
- Adobe Research. sukim@adobe.com
- Adobe Research. dernonco@adobe.com
- Adobe Research. tyu@adobe.com
- Adobe Research. ruizhang@adobe.com
- Intel (United States) (US)
- ORCID
- Venue
- Computational Linguistics
- Publisher
- MIT Press
- Published
- 2024
- Volume
- 50
- Issue
- 3
- Pages
- 1097-1179
- ISSN
- 0891-2017 (ISSN-L)
- 1530-9312
- OpenAlex
- W4399528455
- Semantic Scholar
- bcfa73aedf1b2d1ee4f168e21298a37ac55a37f7
- Cited by
- 1,245
- References
- 280
- Citations by year
- 2026: 596 · 2025: 451 · 2024: 188
- FWCI
- 132.76
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Computational Linguistics (publishedVersion, cc-by-nc-nd)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- large language models, social bias, fairness, bias assessment, bias mitigation, bias metrics, data preprocessing, post-processing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-11-02, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-02 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.