Bias and Fairness in Large Language Models: A Survey
Open accessThe DOI matches its Crossref record; 313 of the 313 references it cites were found in a registry or an index; 151 of the 183 sources its claims cite were checked for support.
Abstract
Rapid advancements of large language models (LLMs) have enabled the processing, understanding, and generation of human-like text, with increasing integration into systems that touch our social sphere. Despite this success, these models can learn, perpetuate, and amplify harmful social biases. In this article, we present a comprehensive survey of bias evaluation and mitigation techniques for LLMs. We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing, defining distinct facets of harm and introducing several desiderata to operationalize fairness for LLMs. We then unify the literature by proposing three intuitive taxonomies, two for bias evaluation, namely, metrics and datasets, and one for mitigation. Our first taxonomy of metrics for bias evaluation disambiguates the relationship between metrics and evaluation datasets, and organizes metrics by the different levels at which they operate in a model: embeddings, probabilities, and generated text. Our second taxonomy of datasets for bias evaluation categorizes datasets by their structure as counterfactual inputs or prompts, and identifies the targeted harms and social groups; we also release a consolidation of publicly available datasets for improved access. Our third taxonomy of techniques for bias mitigation classifies methods by their intervention during pre-processing, in-training, intra-processing, and post-processing, with granular subcategories that elucidate research trends. Finally, we identify open problems and challenges for future work. Synthesizing a wide range of recent research, we aim to provide a clear guide of the existing literature that empowers researchers and practitioners to better understand and prevent the propagation of bias in LLMs.
Overview
CoChat Summary
TL;DR
This survey consolidates, formalizes, and taxonomizes bias evaluation and mitigation for large language models, covering metrics, datasets, and mitigation techniques. It proposes three taxonomies and identifies open problems around power imbalances, fairness conceptualization, evaluation standards, mitigation scalability, and theoretical guarantees.
Key findings
- The survey proposes three taxonomies: bias evaluation metrics organized by what they use from a model (embeddings, probabilities, generated text); bias evaluation datasets organized by structure (counterfactual inputs vs. prompts); and bias mitigation techniques organized by intervention stage (pre-processing, in-training, intra-processing, post-processing).
- It formalizes social bias for LLMs as representational harms (e.g., derogatory language, disparate system performance, erasure, exclusionary norms, misrepresentation, stereotyping, toxicity) and allocational harms (direct and indirect discrimination), and introduces fairness desiderata including fairness through unawareness, invariance, equal social group associations, equal neutral associations, and replicated distributions.
- It reports limitations across metric classes: embedding-based and probability-based metrics may be only weakly or inconsistently correlated with downstream task bias, and generated-text metrics can be sensitive to decoding parameters; it also highlights reliability and validity issues in widely used datasets.
- It consolidates publicly available datasets, with reported sizes including CrowS-Pairs (1,508), StereoSet (16,995), BBQ (58,492), RealToxicityPrompts (100,000), BOLD (23,679), HolisticBias (460,000), and Bias NLI (5,712,066).
Methods
- Design: a comprehensive survey and taxonomy-building effort rather than an empirical study, synthesizing prior work on bias evaluation and mitigation for LLMs.
- Scope: focuses on bias issues in LLMs for English, with additional languages for machine translation and multilingual models, and restricts inclusion to works that propose novel closed-form metrics, datasets, or mitigation techniques and specify applicability to social bias or fairness.
- Analysis: organizes metrics, datasets, and mitigation techniques into taxonomies, formalizes several metrics and mitigation objectives with unified notation, discusses limitations of each class, and outlines open problems and recommendations.
Limitations
- The paper states that it aligns with a technical solutionist perspective rather than a critical theoretical one, and that its taxonomies are organized by technical implementation details instead of downstream usage contexts or harms.
- The paper states that it focuses narrowly on model training and evaluation, rather than a more holistic pipeline including problem formulation, data collection, and deployment/integration.
- The paper states that it is limited to English language papers; evident from the text, it also does not report a systematic search protocol, search dates, or screening counts.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures










Verification
Claims the paper makes
121 Valid · 12 Not valid · 18 Not checked · 106 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Despite the growing emphasis on addressing these issues, bias and fairness research in LLMs often fails to precisely describe the harms of model behaviors: who is harmed, why the behavior is harmful, and how the harm reflects and reinforces social principles or hierarchies (Blodgett et al. 2020).”
What the sources say
- Partly supportedcites Blodgett et al. 2020 matched to “Language (Technology) is Power: A Critical Survey of “Bias” in NLP” (2020) sourceExperimental — accuracy not yet measured
Section 3.1 (Motivations; page not provided) finds that papers on bias in NLP often fail to explain which model behaviors are harmful, how, to whom, and why. Sections 4.1–4.2 call for attention to social hierarchies and explicit normative reasoning. The survey concerns NLP research broadly, however, and does not establish that this failure is common specifically in LLM research. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Similar to the shortcomings of embedding-based metrics, Delobelle et al. (2022) and Kaneko, Bollegala, and Okazaki (2022) point out that probability-based metrics may be only weakly correlated with biases that appear in downstream tasks”
- Also stated in: § 3.6 Recommendations · p. 27 · plain
“Probability-based metrics also show weak correlations with downstream biases.”
What the sources say
- Supportedcites Delobelle et al. 2022 matched to “Measuring Fairness with Biased Rulers: A Comparative Study on Bias Metrics for Pre-trained Language Models” (2022) source
Section 3 describes CrowS-Pairs as a pseudo-loglikelihood-based metric. Section 4.4 reports that CrowS-Pairs correlates less with two measures of bias in downstream tasks, supporting the hedged claim that probability-based metrics may correlate weakly with downstream bias. The finding does not establish that all probability-based metrics do so. (LLM, conf=0.90)
- Supportedcites Kaneko, Bollegala, and Okazaki 2022 matched to “Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Tasks” (2022) source
The Introduction (p. 1) identifies intrinsic measures as using likelihood scores assigned by MLMs and reports only weak correlations with downstream, task-specific bias measures. Section 5.2 (p. 8) likewise finds weak and inconsistent correlations between intrinsic and extrinsic bias scores. (LLM, conf=0.93)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 3.6 Recommendations · p. 27 · plain
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Many approaches, for instance, assume some implicitly desirable criterion ( e.g. , a model output should be independent of any social group in the input), but do not explicitly acknowledge or state the normative social values that justify their framework.”
What the sources say
- Partly supportedcites Blodgett et al. 2021 matched to “Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets” (2021) sourceExperimental — accuracy not yet measured
Section 2.3 describes assumptions about ideal model behavior, including equal probabilities for sentences about different target groups. Section 6 says the benchmarks make implicit decisions about stereotyping that are not explicitly discussed or justified. The paper examines four stereotyping benchmarks, however, rather than establishing this claim about many bias and fairness approaches generally or specifically about their normative social values. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Cabello, Jørgensen, and Søgaard (2023) illustrate that associations between the representations of protected attribute and other words can be independent of downstream performance disparities, if certain assumptions of social groups' language use are violated.”
What the sources say
- Supportedcites Cabello, Jørgensen, and Søgaard 2023 matched to “On the independence of association bias and empirical fairness in language models” (2023) source
The Bias section defines association bias as differences in how demographic-group terms are encoded and empirical fairness as performance differences across social groups. Section 5 says the two can be independent and explains that linking them relies on the In-Group Affinity Assumption about how groups use those terms; it calls that assumption problematic. No page numbers are provided in our copy. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Each stakeholder brings perspectives that may specify different fairness constraints for the same application and setting.”
What the sources say
- Partly supportedcites Jernite et al. (2022) matched to “Data Governance in the Age of Large-Scale Data-Driven Language Technology” (2022) sourceExperimental — accuracy not yet measured
Section 2.2 (p. 4) says participants bring different perspectives and warns that relying only on their values could exclude views more relevant to other language users. The paper discusses nondiscrimination and local expressions of values, but does not say that stakeholders specify different fairness constraints for the same application and setting. (LLM, conf=0.85)
- Not matchedcites Bender (2019)
An author–year citation the paper’s reference list doesn’t settle. We don’t search these by name.
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
71 more claims · 34 with their cited sources checked
Create a free account to see all 257 claims and every source verdict.
- Probably not a substantial claim. 151
- It may state two separate claims as one. 70
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While this process mitigates some risks inherent to AI-generated content, it does not eliminate potential biases associated with proprietary large language models, which lack transparency regarding training data and internal parameters [10].”
From AI-Generated Multiple Mini Interview (MMI) Stations for Medical School Admissions: Psychometric Evaluation (2026) · cites this paper as [10]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Notably, there is growing attention on assessing LLM bias [63-64] and robustness [323-324].”
From A Survey of Large Language Models (2026) · cites this paper as [63]
The cited source says what the claim attributes to it.
In the citing paper
“these models pose serious risks, as the vast, unfiltered internet data used to train them can reinforce and amplify harmful biases, disproportionately harming marginalized communities (73).”
From Emergent social conventions and collective bias in LLM populations (2025) · cites this paper as [73]
The cited source says what the claim attributes to it.
In the citing paper
“Despite the use of various mitigation techniques, such as debiasing algorithms, curating balanced datasets, or incorporating fairness-focused training objectives, eliminating bias entirely is a persistent challenge141–143.”
From Current applications and challenges in large language models for patient care: a systematic review (2025) · cites this paper as [142]
The cited source says what the claim attributes to it.
In the citing paper
“Another major concern with LLMs is their propensity to amplify historical biases.”
From Evaluation of Large Language Models for Peer Review in Transplantation Research: Algorithm Validation Study (2026) · cites this paper as [30]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.1 Counterfactual Inputs · p. 28
“| WinoBias+ | 1,367”
§ 4.1 Counterfactual Inputs · p. 29
“WinoBias+ (Vanmassenhove, Emmery, and Shterionov 2021) augments WinoBias with gender-neutral alternatives, similar to Winogender's neutral option, with 3,167 total instances.”
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 6×
“, 2020), and GPT-4 (OpenAI, 2023); autoencoding models include BERT (Devlin et al., 2019), RoBERTa (Liu et al.”
BackgroundInfluential(2014) · Cited by 33,419
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 2×
“, 2019), GPT-3 (Brown et al., 2020), and GPT-4 (OpenAI, 2023); autoencoding models include BERT (Devlin et al.”
Background"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,690
Gauriat, C.-M., Pencolé, Y., Ribot, P., & Brouillet, G. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.27 · Cited by 12,586
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“The rise and rapid advancement of large language models (LLMs) has fundamentally changed language technologies (e.g., Brown et al., 2020; Conneau et al., 2020; Devlin et al., 2019; Lewis et al., 2020; Liu et al., 2019; OpenAI, 2023; Radford et al., 2018; 2019; Raffel et al., 2020).”
BackgroundMeasuring individual differences in implicit cognition: the implicit association test
Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. K. (1998) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.74.6.1464 · Cited by 9,210 · cited 2×
“, family and occupation words), emulating the Implicit Association Test (Greenwald et al., 1998).”
Method
Research works printed but not linked: 5
- Barocas, 2019, Fairness and Machine Learning: Limitations and Opportunities
- Blodgett, Su Lin . 2021. Sociolinguistically Driven Approaches for Just Natural Language Processing. Ph.D. thesis. University of Massachusetts Amherst.
- Green, 2019, ”Good” isn’t good enough, 1
- Radford, 2019, Language models are unsupervised multitask learners, OpenAI Blog, 1, 9
- Wang, 2019, BERT has a mouth, and it must speak: BERT as a Markov random field language model, 30, 10.18653/v1/w19-2304 · doi:10.18653/v1/w19-2304
Cited by
Items 501–600 of 763
Investigating spatial-temporal bias of LLMs
Li, Z. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.131542 · Cited by 2
MAKE: Memory-Associated Knowledge Editing
Park, S., Park, S., Kim, J., & Kim, H. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.26 · Cited by 6
Cross-Model Adjudication for Bias Mitigation in Large Language Models
Li, X., Li, C., Liu, W., Long, Y., Huang, Y., & Liu, Y. (2026) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2026.3662478 · Cited by 2
An, W., & Tian, X. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6194781
Lange, M., Koutsouleris, N., Carter, B., & Twumasi, R. (2026) · Healthcare Technology Letters · doi:10.1049/htl2.70058
Encounters between Artificial and Human Intelligences in Early‐Stage Innovation Work
Notini, W., Kochoska, A., & Perkel, D. (2025) · Ethnographic Praxis in Industry Conference Proceedings · doi:10.1111/epic.70015
Privacy Risk Prediction from Social Media Metadata using Feature Selection Approaches
Pranav, R., Rajoriya, M. M., & Rohini, V. (2025) · International Conference on NexGen Networks and Cybernetics (IC2NC) · doi:10.1109/ic2nc67409.2025.11376087
“Bias remains a persistent issue, as Sheng et al. [12] demonstrated in generative models and Argyle et al. [13] confirmed through their moral reasoning experiments.”
Detecting and Evaluating Bias in Large Language Models: Concepts, Methods, and Challenges
Gao, Z., Tong, L., & Zhang, Z. J. (2026) · Journal of Behavioral Data Science · doi:10.35566/jbds/gao · Cited by 4
Analyzing and Mitigating Linguistic Bias in Bangla Natural Language Processing
Akter, A., Tanvir, M., Islam, T., Chowduri, J. A., Fahadh, M. B., Islam, M. A. U., & Alam, R. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-15346-3_3
PRISM: Principled Reasoning for Identifying and Suppressing Model Biases at Scale
Zhu, X., Li, W., & Lin, J. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-7081-2_3
Confirmation, Framing, and Position Biases in LLM Responses
Магдановна, Е. Л., Firsov, A., & Kamps, J. (2026) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3786304.3787879 · Cited by 4
Evolution im Dialog: Vom Chatbot zur Conversational AI
Lehmann, J. (2026) · Springer eBooks · doi:10.1007/978-3-658-46399-1_75-1
Vulnerability of LLM Outputs to Heuristics-Inducing Prompt Structures
Kuramoto, T., Kamei, R., & Suzuki, J. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789108
Developing a Testing Framework for Applications of LLMs within Islamic Finance
Al-Syed, Q., & Mahomed, Z. (2025) · World Scientific Annual Review of Islamic Finance · doi:10.1142/s2811023425500029
Jasem, A. N. A. A. (2026) · International Journal of Research · doi:10.26643/ijr/22
Multi-Agent Debate for Content Moderation with Dynamic Group Arbitration
Jiang, Y., Wang, L., Lou, Y., Tao, X., & Hu, H. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11402348 · Cited by 1
“However, our preliminary analysis reveals that current LLM-based moderation systems exhibit systematic failures [7], [8] when encountering two specific categories of challenging content: First, contextual ambiguity poses signif-Fig.”
Persona Non Graphica: Visual Representation Biases in Human-Robot Interaction Research
Seaborn, K., & Yalçın, Ö. N. (2026) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3757279.3785541
Mapping Research on Social Cognitive Bias in Large Language Models
Zhang, S., Li, J., Gao, Y., Li, Y., Zhang, X., & Li, X. (2025) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw69685.2025.00159
“This reduced sensitivity enables biased patterns to escape human scrutiny and exacerbates social inequality through the propagation of prejudiced information [24], [25].”
Miklian, J., Hoelscher, K., & Katsos, J. E. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6210099 · Cited by 1
Emergent Biases in Large Language Models: A Critical Review of Taxonomy and Evaluations
Ismail, S., & Aygün, R. S. (2025) · IEEE International Conference on Cognitive Machine Intelligence (CogMI) · doi:10.1109/cogmi67134.2025.00036
“Unlike previous studies on LLM bias and fairness [4], [18], our contribution is a taxonomy explicitly dedicated to emergent biases, paired with a systematic critique of evaluation methodologies.”
InfluentialGarcía-Barceló, C., Tomás, D., & Mazón, J.-N. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.132062 · Cited by 3
Research Problems, Questions, and Conceptual Frameworks in a Generative Era
(2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-9340-7.ch002
Ethics, Authorship, and Transparency: Responsible AI Use in Research and Writing
(2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-9340-7.ch004
Literature Discovery and Synthesis With AI: Narrative, Scoping, and Systematic Reviews
(2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-9340-7.ch003
Detecting and Mitigating Social Bias in Transformer Language Models
Dahatonde, R. T., Rashmi, R., Motghare, M., Shelke, N. M., Ashtankar, N., & Guda, S. (2025) · IEEE Global Conference on Wireless Computing and Networking (GCWCN) · doi:10.1109/gcwcn66157.2025.11448494
Alignment Reduces Expressed but Not Encoded Gender Bias: A Unified Framework and Study
Bouchouchi, N., Lesot, M., Laugel, T., Renard, X., Marsala, C., & Detyniecki, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37654-1_21 · Cited by 3
Puyt, R. W., & Antoniou, P. H. (2026) · Strategic Change · doi:10.1002/jsc.70055 · Cited by 1
Bias Mitigation in Large Language Models for Tabular Data Classification
Lu, H., Shao, Z., Gao, J., & Uddin, S. (2026) · Machine Learning · doi:10.1007/s10994-026-07030-0 · Cited by 2
“We hypothesise that explicitly instructing the LLMs to consider fairness can enhance its ability to make equitable predictions (Gallegos et al., 2024a).”
InfluentialKu, W. L., Liu, Y., Zhang, W., & An, P. (2026) · Design for Augmented Humanity · doi:10.1177/29776481261438705
“Although mitigation approaches such as reinforcement learning from human feedback (RLHF) offer partial safeguards (Pagano et al., 2023), these mechanisms remain constrained by prede fi ned normative assumptions (Gallegos et al., 2024).”
Die Illusion einer neutralen KI: Werte statt Wahrheit
Lückenga, K. (2026) · Wirtschaftsinformatik & Management · doi:10.1365/s35764-026-00595-4
Speak2Scene: Voice-based Storyboarding
Kashyap, A., & Alves-Oliveira, P. (2026) · SoftwareX · doi:10.1016/j.softx.2026.102628
Aligned Probing: Relating Toxic Behavior and Model Internals
Waldis, A., Gautam, V., Lauscher, A., Klakow, D., & Gurevych, I. (2026) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.613 · Cited by 5
Nazaretyan, L., & Kircher, M. (2026) · Trends in Genetics · doi:10.1016/j.tig.2026.02.007 · Cited by 3
Scientific autonomy in the structural bubble: from institutional bias to AI-mediated consensus
Rubenstein, E. (2026) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2026.1766504 · Cited by 2
Park, K., Min, H., Lee, C. B., & Shin, K.-Y. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798720
Cultural Bias Detection in LLMs
Pant, M., Pandey, D., Jain, D., Sood, P., & Agrawal, P. (2026) · International Conference on Innovative Practices in Technology and Management (ICIPTM) · doi:10.1109/iciptm69057.2026.11465565
Aldaweesh, S., Alelsheikh, G., Alhamed, F., Kleek, M. G. V., & Shadbolt, N. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791060 · Cited by 4
Exploring attribution bias in LLMs: Social influences and prompt-based mitigation
Jiang, L., Cao, J., Zhu, G., & Wang, Y. (2026) · Technology in Society · doi:10.1016/j.techsoc.2026.103361 · Cited by 1
Managing the ethical risks of AI in intelligence: a multi-layered framework
Bellaby, R. W. (2026) · AI & Society · doi:10.1007/s00146-026-03045-2 · Cited by 3
“Such categorization can further embed structural feedback loops, reinforcing pre-existing inequalities and justifying further surveillance of marginalized groups (Benjamin 2019; Crawford 2021; Gallegos et al. 2024; Hanna et al. 2025; Nouis et al. 2025).”
Enforcing Privacy-By-Design: A Policy-Guided Synthetic Data Generator for GenAI Workflows
Nadella, D., Raju, S., & Chinthala, S. (2025) · International Conference on Cloud and Big Data Computing (ICCBDC) · doi:10.1109/iccbdc67784.2025.00009
“Synthetic data allows organizations to share and use data without fear of revealing sensitive information, as done by [3].”
Văduva, A.-G., ANDREESCU, A. I., & Oprea, S. (2026) · Smart innovation, systems and technologies · doi:10.1007/978-981-95-6136-0_36
Bae, I., Bazarova, N. N., Zhao, P., Hui, W., Kizilcec, R. F., & Margolin, D. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6613903
Mitigating Hallucinations in Large Language Models via Decoder Layer Skipping
Li, H., You, J., Guo, Y., Tang, K., Xie, S., & Huang, X. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464714
“Several studies have investigated hallucinations in the decoding process of LLM, identifying suspicious causes of the issue, such as errors in training data [5] and the deviation in model parameter-fitting [6].”
The Evolving Role of Large Language Models in Health Care Research and Scientific Communication
Zand, J., Šuvakov, M., Overgaard, S. M., Silvano, C. J., Meiners, L. M., Fan, J. W., Carter, R. E., Wieland, M. L., Patten, C. A., Dugani, S. B., et al. (2026) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2026.04.007
AI for IT Operations (AIOps): Applications, Challenges, and Benefits
N., S. A. D. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/407628
Testing Bias Tolerance of Large Language Models with Mirrored Paradoxical Ambiguity Probing
Amin, M. N., Nemeth, P., & Jesser, A. (2026) · IEEE International Conference on Semantic Computing (ICSC) · doi:10.1109/icsc67292.2026.00034
“This tension—context sensitivity without guaranteed stereo-type suppression—mirrors broader observations in bias evaluation work that surface-form improvements can coexist with persistent latent associations [9], [10], [16].”
Miklian, J., Hoelscher, K., & Katsos, J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6644227
Adaptive multilingual cyberbullying detection via grammatical evolution
Ismail, W. S., Ullah, H., Adnan, M., & Ullah, F. (2026) · PeerJ Computer Science · doi:10.7717/peerj-cs.3740
Chinthala, L. K., Shaban‐Nejad, A., Xu, H., Lemon, C., Farage, G., Davis, R., & Madlock‐Brown, C. (2026) · JMIR Mental Health · doi:10.2196/preprints.99394
Safe Vision-Language Models via Unsafe Weights Manipulation
Moreno D’Incà, Peruzzo, E., Xu, X., Shi, H., Sebe, N., & Mancini, M. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00455 · Cited by 3
“In the context of Large Language Models (LLMs), extensive research has focused on identifying risks through red teaming [35, 39, 48, 57, 73] - systematic stress testing to uncover harmful behaviors - and developing mitigation strategies [16, 24, 68, 76].”
Sunitha, K., & Sharma, S. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-18346-0_23
Calibrating Stereotype Preference with Multi-Agent Graph Reasoning in Large Language Models
Amin, M. N., & Jesser, A. (2025) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit68739.2025.11491743
“Scaling up models tends to increase LMS [43], but neutrality is not guaranteed; bias can persist or resurface depending on data and objectives [45].”
InfluentialAdanyin, A. C., Fagbohun, O., Akintola, A. S., Oyeladun, O., Akwaowo, P. U., Okere, C., Anyogo, F. E., & Olajuwon, O. O. (2026) · International Conference on Advances in Artificial Intelligence and Machine Learning (AAIML) · doi:10.1109/aaiml67890.2026.11498196
“Each agent is instrumented with logging of input metadata and counterfactual testing routines [11].”
A trajectory-based framework for diagnosing and calibrating social order in large language models
Wang, S., Zhao, Y. F., Han, P., & Zhai, G. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.132734
Evolution im Dialog: Vom Chatbot zur Conversational AI
Lehmann, J. (2026) · Springer eBooks · doi:10.1007/978-3-658-49614-2_75
Lightweight Fairness for LLM-Based Recommendations via Kernelized Projection and Gated Adapters
Cui, N., Wang, W. H., & Ning, Y. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-19096-3_18 · Cited by 1
Identifying Algorithmic and Domain-Specific Bias in Parliamentary Debate Summarisation
Cunningham, E., Cross, J., & Greene, D. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-19108-3_28
No LLM is Free From Bias: A Comprehensive Study of Bias Evaluation in Large Language Models
Charaka, V. K., Urlana, A., Kanumolu, G., Garlapati, B. M., & Mishra, P. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-19096-3_14 · Cited by 31
A Representation-Level Assessment of Bias Mitigation in Foundation Models
Nizhnichenkov, S., Nair, R., Daly, E. M., & Namee, B. M. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-19096-3_13 · Cited by 3
Wang, K., Wang, T., Wang, B., Li, Q., Liu, Y., Han, Q., Zhang, C. L., & Ren, Y. (2026) · Reviews in Fish Biology and Fisheries · doi:10.1007/s11160-026-10058-z · Cited by 2
“Models can reflect and amplify social biases from training data, leading to outputs that may be unfair or discriminatory (Gallegos et al. 2024), such as reinforcing stereotypes about fishing communities or regions in policy development.”
Kennedy, E., Vadlamani, S., Amuan, M. E., Stewart, I. J., Miles, S. R., Martindale, S. L., Brenner, L. A., & Pugh, M. J. (2026) · Military Medicine · doi:10.1093/milmed/usag217
Fede, G. D., Alrabie, L., & Andolina, S. (2026) · Springer eBooks · doi:10.1007/978-981-95-3658-0_49 · Cited by 1
Pérez, J. E. M., Aguilar-Cruz, P. J., & Cárdenas, J. A. H. (2026) · Cuadernos de lingüística hispánica · doi:10.19053/uptc.0121053x.n48.2026.19867
Gumelar, A. B., Paramita, A. S., Tahalele, P. L., Thohari, K., Kurniawan, A., & Salmanarrizqie, A. (2026) · International Conference on Current Research in Artificial Intelligence and Data Science (ICCRAIDS) · doi:10.1109/iccraids67816.2026.11519693
“Many works have uncovered and analyzed the statistics of gender, racial, ethnic, and nationalist-related stereotypes in the model output, mostly in intra-language setups [7], [9].”
Cross-sectional analysis of large language models in current natural language generation challenges
Maestre, M. M., Martínez-Murillo, I., Martínez, A. M., & Lloret, E. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.104922 · Cited by 1
Mitigating Security Risks in Large Language Models: A Full Lifecycle Perspective
Wang, Y. K., Bai, Z., Huo, J., Wang, B., Liu, J., Cao, H., Meng, F., Yang, X., & Gao, Y. (2026) · Machine Learning · doi:10.1007/s10994-025-06930-x
“In con - trast, LLM-generated prompts (Perez et al., 2022; Yu et al., 2023 ) improved efficiency but often resulted in low-quality cases due to inherent model biases (Abid et al., 2021; Gallegos et al., 2024).”
DetBias: A Framework for Detecting Bias in NLP Classifier Models
Adil, M. F., Aswani, S., & Shetty, S. D. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-25617-1_32
A Multi-Level Debiasing Framework for Mitigating Cultural Bias in Reviews Sentiment Analysis
Bhavya Sree, S., Viju, L. M., & Hemanth, K. S. (2026) · International Conference on Knowledge Engineering and Communication Systems (ICKECS) · doi:10.1109/ickecs70176.2026.11527641
BODNAR, D., & Koldovskyy, V. (2026) · Advances in Cyber-Physical Systems · doi:10.23939/acps2026.01.012
LLM for social good: A value-driven LLM framework to embed social good values
Li, V. O. K., Lam, J. C. K., Han, Y., Crowcroft, J., & Cheung, L. Y. L. (2026) · Data & Policy · doi:10.1017/dap.2026.10061 · Cited by 1
“As a result, these models can inadvertently perpetuate stereotypes, reinforce inequities, and produce outputs that conflict with societal norms or ethical principles (Gallegos et al., 2024).”
Baumberger, J. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-8800-7.ch001
Cross-Dataset Fairness Evaluation of Transformer Models in Sentiment Analysis
Zuiran, S., & Bhattacharyya, S. (2026) · IEEE International Conference on Computing and Machine Intelligence (ICMI) · doi:10.1109/icmi68585.2026.11539779
“Prior work showed that evaluating models across multiple datasets can reveal disparities that single-dataset testing may overlook [14], [15].”
Sarmah, N., & Bhattacharyya, R. (2026) · International Journal of Data Science and Analytics · doi:10.1007/s41060-026-01162-1 · Cited by 2
“We focused our search on publications mostly from the fields of computer vision, pattern recognition, cognitive science, engineering, ethical AI, and natural language processing [9, 18].”
Aljedaani, W., Alsulaymi, R., Alqahtani, G., & Feng, Y. (2026) · International Web for All Conference (W4A) · doi:10.1145/3800424.3800454
Jaiswal, U. K., Bhatnagar, A., Jain, J., Saini, H., Vikas, Sahu, S., & Mishra, K. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-2173-6.ch003
Between Efficiency and Fairness: Algorithmic Discrimination in AI-Based Recruitment Systems
Ertürk, M., & Ayhan, E. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-2303-7.ch005
Ruidias, R. R. S., Nunes, B. P., & da Silva, T. N. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-1468-6_27
Proxy Analysis in Bias Testing
Djennane, L., Kardkovács, Z. T., Benatallah, B., Gaci, Y., & Farah, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-28117-3_15
SoK: A taxonomy of LLM threats
Stylianou, I., Bountakas, P., Zarras, A., Farao, A., Bolgouras, V., & Xenakis, C. (2026) · Computer Science Review · doi:10.1016/j.cosrev.2026.101013 · Cited by 2
LLM‐Integrated Human–Robot Interaction System for Microrobots
Zhu, B., Salehi, A., & Yu, T. (2026) · Advanced Robotics Research · doi:10.1002/adrr.202500179
Self-citationPeters, D., Espinoza, F., Re, M. D., Ivetta, G., Benotti, L., & Calvo, R. A. (2026) · AI and Ethics · doi:10.1007/s43681-026-01196-y · Cited by 8
“These issues are also increasingly addressed at Natural Language Processing conferences [e.g. 5] which gather interdisciplinary teams to identify and tackle the technical challenges presented by cultural misalignment.”
Counterfactual Consistency LoRA for Bias Mitigation on BBQ
Feng, D., Chen, H., & Zhou, Z. (2026) · International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT) · doi:10.1109/ainit70033.2026.11557867
“Recent surveys and Chinese-language benchmarks such as CBBQ further argue that fairness evaluation should consider both benchmark de-sign and cultural context [7], [8].”
Context-Aware Counterfactual Data Augmentation for Gender Bias Mitigation in Language Models
Parihar, S., Liu, G., Parde, N., & Cheng, L. (2026) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3820356.3820358 · Cited by 3
Open-Ended Bias Discovery in Large Language ModelsPreprint
Nguyen, T. H., & Luong, N. H. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6958779
I Want an Assistant: Generative AI, Women’s Leadership, and the Redistribution of Invisible Labor
Kostihová, M., & Makarevitch, I. (2026) · Merits · doi:10.3390/merits6020017
“This is particularly important given that even recent large language models remain vulnerable to hallucination, bias, and failures of contextual judgment [57,58].”
Calibrating Social Bias in LLM and Human Responses in Postsecondary Online Discussion Forums
March, D., Xu, Z., Yao, C., & Yu, R. (2026) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3774398.3811591
CoPAMoE: Cognitive Process-Aware Mixture of Experts for Bias Understanding and MitigationPreprint
Liu, X., Wang, F., Wang, F., Jia, Z., Wang, H., & Ran, C. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6987086
AlDahoul, N., Karim, H. B. A., & Tan, M. J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6900538
Greek Mythology as a Lens for Understanding the Risks, Biases, and Ethics of Artificial Intelligence
Muhlenbach, F. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812367
Warren, G., Sun, J., Shklovski, I., & Augenstein, I. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812358 · Cited by 4
A pipeline for enabling path-specific causal fairness in observational health data
Kashyap, A., Matijevic, S., Elhadad, N., Kushner, S. A., & Joshi, S. D. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806507 · Cited by 1
“We demonstrate the utility of dimensionality reduction methods to reduce the estimation error of doubly-robust path-specific causal effects, required to detangle direct, indirect, and spurious sources of bias.”
Wasi, A. T., Heidari, O. R., Anam, M. U., Rahman, A., Agarwal, A., Patel, H. L., Kumar, B., Rafi, T. H., & Chae, D. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-1947-6_48 · Cited by 2
Identifying Patterns of Stigmatizing Language Use in the Safety Net
Gogia, K., Li, Z., Simpson, K., Davis, N. J., & Newton-Dame, R. (2026) · Journal of General Internal Medicine · doi:10.1007/s11606-026-10592-3
Lee, T., Lee, S., & Kim, S. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-29791-4_30
Self-citationMukherjee, S., Le, J. D., Nguyen, C., & Vu, T. T. (2026) · Knowledge · doi:10.3390/knowledge6030013
“One of the most extensively studied issues in LLMs is their tendency to inherit and amplify societal biases from their training data [21].”
Teammates for Whom? Linguistic Gatekeeping in LLM-Based Educational Assessment and Feedback
Zheng, C., & Chen, G. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-29791-4_9
CoPAMoE: Cognitive Process-Aware Mixture of Experts for Bias Understanding and MitigationPreprint
Liu, X., Wang, F., Wang, F., Jia, Z., Wang, H., & Ran, C. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6997071
The case for humanities-based studies of AI production and effects
Asscher, O. (2026) · AI & Society · doi:10.1007/s00146-026-03170-y
“…and cultural biases in textual and semiotic LLM production have been abundantly explored in recent years, but the predominant methodologies employed are NLP-based computational techniques and the central venues of publication are computer science conference proceedings (Gallegos et al. 2024).”
Bhat, S., & Varma, V. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29763-1_18
Metadata
- Authors
- Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K.
- Affiliations
- Stanford University (US)
- Adobe Research. ryrossi@adobe.com
- Pattern Data. joe.barrow@patterndataworks.com
- Adobe Research. tanjim@adobe.com
- Adobe Research. sukim@adobe.com
- Adobe Research. dernonco@adobe.com
- Adobe Research. tyu@adobe.com
- Adobe Research. ruizhang@adobe.com
- Intel (United States) (US)
- ORCID
- Venue
- Computational Linguistics
- Publisher
- MIT Press
- Published
- 2024
- Volume
- 50
- Issue
- 3
- Pages
- 1097-1179
- ISSN
- 0891-2017 (ISSN-L)
- 1530-9312
- OpenAlex
- W4399528455
- Semantic Scholar
- bcfa73aedf1b2d1ee4f168e21298a37ac55a37f7
- Cited by
- 1,245
- References
- 280
- Citations by year
- 2026: 596 · 2025: 451 · 2024: 188
- FWCI
- 132.76
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Computational Linguistics (publishedVersion, cc-by-nc-nd)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- large language models, social bias, fairness, bias assessment, bias mitigation, bias metrics, data preprocessing, post-processing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-11-02, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-02 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.