A Comprehensive Survey of Bias in LLMs: Current Landscape and Future Directions
Open accessAbstract
Large Language Models(LLMs) have revolutionized various applications in natural language processing (NLP) by providing unprecedented text generation, translation, and comprehension capabilities. However, their widespread deployment has brought to light significant concerns regarding biases embedded within these models. This paper presents a comprehensive survey of biases in LLMs, aiming to provide an extensive review of the types, sources, impacts, and mitigation strategies related to these biases. We systematically categorize biases into several dimensions. Our survey synthesizes current research findings and discusses the implications of biases in real-world applications. Additionally, we critically assess existing bias mitigation techniques and propose future research directions to enhance fairness and equity in LLMs. This survey serves as a foundational resource for researchers, practitioners, and policymakers concerned with addressing and understanding biases in LLMs.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Models trained on data reflecting societal biases and disparities may perpetuate or amplify these inequities when deployed in clinical practice.
“Models trained on data reflecting societal biases and disparities may perpetuate or amplify these inequities when deployed in clinical practice.”
From Social determinants of health extraction from clinical notes across institutions using large language models (2025) · cites this paper as [57]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 33
- Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., Avila, R., Babuschkin, I., Balaji, S., Balcom, V., Baltescu, P., Bao, H., Bavarian, M., Belgum, J., Bello, I., . . . Zoph, B. (2023). GPT-4 Technical Report. ArXiv. /abs/2303.08774
- Agiza, A., Mostagir, M., & Reda, S. (2024). PoliTune: Analyzing the Impact of Data Selection and Fine-Tuning on Economic and Political Biases in Large Language Models. ArXiv. /abs/2404.08699
- An, J., Huang, D., Lin, C., & Tai, M. (2024). Measuring Gender and Racial Biases in Large Language Models. ArXiv. /abs/2403.15281
- Bai, X., Wang, A., Sucholutsky, I., & Griffiths, T. L. (2024). Measuring Implicit Bias in Explicitly Unbiased Large Language Models. ArXiv. /abs/2402.04105
- Barocas, S., & Selbst, A. D. (2016). Big Data's Disparate Impact. California Law Review, 104(3), 671-732.
- Binns, R. (2017). Fairness in Machine Learning: Lessons from Political Philosophy. ArXiv. /abs/1712.03586
- Bolukbasi, T., Chang, K., Zou, J., Saligrama, V., & Kalai, A. (2016). Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. ArXiv. /abs/1607.06520
- Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., . . . Amodei, D. (2020). Language Models are Few-Shot Learners. ArXiv. /abs/2005.14165
- Caliskan, A., Bryson, J. J., & Narayanan, A. (2016). Semantics derived automatically from language corpora contain human-like biases. ArXiv. https://doi.org/10.1126/science.aal4230 · DOI · Paper page
- Cheng, R., Ma, H., & Cao, S. (2024). Deceiving to Enlighten: Coaxing LLMs to Self-Reflection for Enhanced Bias Detection and Mitigation. ArXiv. /abs/2404.10160
- Devlin, J., Chang, M., Lee, K., & Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. ArXiv. /abs/1810.04805
- Fan, Z., Chen, R., Xu, R., & Liu, Z. (2024). BiasAlert: A Plug-and-play Tool for Social Bias Detection in LLMs. ArXiv. /abs/2407.10241
- Ferrara, E. (2023). Fairness And Bias in Artificial Intelligence: A Brief Survey of Sources, Impacts, And Mitigation Strategies. ArXiv. https://doi.org/10.3390/sci6010003 · DOI · Paper page
- Gautam, S., & Srinath, M. (2024). Blind Spots and Biases: Exploring the Role of Annotator Cognitive Biases in NLP. ArXiv. /abs/2404.19071
- Gupta, S., & Ranjan, R. (2024). Evaluation of LLMs Biases Towards Elite Universities: A Persona-Based Exploration. arXiv preprint arXiv:2407.12801. · arXiv
- Gupta, S., Ranjan, R., & Singh, S. N. (2024). Comprehensive Study on Sentiment Analysis: From Rule-based to modern LLM based system. ArXiv. /abs/2409.09989
- Hochreiter, S., & Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9(8), 1735-1780.
- Huang, D., Bu, Q., Zhang, J., Xie, X., Chen, J., & Cui, H. (2023). Bias Testing and Mitigation in LLM-based Code Generation. ArXiv. /abs/2309.14345
- Kaneko, M., Bollegala, D., Okazaki, N., & Baldwin, T. (2024). Evaluating Gender Bias in Large Language Models via Chain-of-Thought Prompting. ArXiv. /abs/2401.15585
- Lin, L., Wang, L., Guo, J., & Wong, K. (2024). Investigating Bias in LLM-Based Bias Detection: Disparities between LLMs and Human Perception. ArXiv. /abs/2403.14896
- Liu, S., Maturi, T., Yi, B., Shen, S., & Mihalcea, R. (2024). The Generation Gap:Exploring Age Bias in the Underlying Value Systems of Large Language Models. ArXiv. /abs/2404.08760
- Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2019). A Survey on Bias and Fairness in Machine Learning. ArXiv. /abs/1908.09635
- Morales, S., Clarisó, R., & Cabot, J. (2024). LangBiTe: A Platform for Testing Bias in Large Language Models. ArXiv. /abs/2404.18558
- Musolesi, M. (2024). (Ir)rationality and Cognitive Biases in Large Language Models. ArXiv. /abs/2402.09193
- Oketunji, A. F., Anas, M., & Saina, D. (2023). Large Language Model (LLM) Bias Index -- LLMBI. ArXiv. https://doi.org/10.5281/zenodo.10441700, 10.13140/RG.2.2.13670.80966 · DOI
- Poulain, R., Fayyaz, H., & Beheshti, R. (2024). Bias patterns in the application of LLMs for clinical decision support: A comprehensive study. ArXiv. /abs/2404.15149
- Raj, C., Mukherjee, A., Caliskan, A., Anastasopoulos, A., & Zhu, Z. (2024). Breaking Bias, Building Bridges: Evaluation and Mitigation of Social Biases in LLMs via Contact Hypothesis. ArXiv. /abs/2407.02030
- Singh, S., Keshari, S., Jain, V., & Chadha, A. (2024). Born With a Silver Spoon? Investigating Socioeconomic Bias in Large Language Models. ArXiv. /abs/2403.14633
- Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention Is All You Need. ArXiv. /abs/1706.03762
- Xu, Z., Peng, K., Ding, L., Tao, D., & Lu, X. (2024). Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extraction. ArXiv. /abs/2403.09963
- Zhang, B. H., Lemoine, B., & Mitchell, M. (2018). Mitigating Unwanted Biases with Adversarial Learning. ArXiv. /abs/1801.07593
- Zhao, J., Wang, T., & Yatskar, M. (2019). Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing.
- Zhou, Y., Ni, Y., Gan, Y., Yin, Z., Liu, X., Zhang, J., Liu, S., Qiu, X., Ye, G., & Chai, H. (2024). Are LLMs Rational Investors? A Study on Detecting and Reducing the Financial Bias in LLMs. ArXiv. /abs/2402.12713
Cited by
Sorted by citations
Galli, C., Gavrilova, A. V., & Calciolari, E. (2025) · Information · doi:10.3390/info16050378 · Cited by 36 · see citing works
Keloth, V. K., Selek, S., Chen, Q., Gilman, C., Fu, S., Dang, Y., Chen, X., Hu, X., Zhou, Y., He, H., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01645-8 · Cited by 24 · see citing works
Large Language Model Psychometrics: A Systematic Review of Evaluation, Validation, and Enhancement
Ye, H., Jin, J., Xie, Y., Zhang, X., & Song, G. (2025) · arXiv · doi:10.48550/arxiv.2505.08245 · Cited by 8 · see citing works
- Authors
- Ranjan, R., Gupta, S., & Singh, S. N.
- Affiliations
- Carnegie Mellon University (US)
- ORCID
- Ranjan, Rajesh
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2409.16430
- OpenAlex
- W4403789150
- Cited by
- 42
- References
- 0
- Citations by year
- 2026: 17 · 2025: 23 · 2024: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- large language models, algorithmic bias, bias mitigation, fairness, equity, bias sources, natural language processing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
