Should ChatGPT be biased? Challenges and risks of bias in large language models
Open accessAbstract
As generative language models, exemplified by ChatGPT, continue to advance in their capabilities, the spotlight on biases inherent in these models intensifies. This paper delves into the distinctive challenges and risks associated with biases specifically in large-scale language models. We explore the origins of biases, stemming from factors such as training data, model specifications, algorithmic constraints, product design, and policy decisions. Our examination extends to the ethical implications arising from the unintended consequences of biased model outputs. In addition, we analyze the intricacies of mitigating biases, acknowledging the inevitable persistence of some biases, and consider the consequences of deploying these models across diverse applications, including virtual assistants, content generation, and chatbots. Finally, we provide an overview of current approaches for identifying, quantifying, and mitigating biases in language models, underscoring the need for a collaborative, multidisciplinary effort to craft AI systems that embody equity, transparency, and responsibility. This article aims to catalyze a thoughtful discourse within the AI community, prompting researchers and developers to consider the unique role of biases in the domain of generative language models and the ongoing quest for ethical AI.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Most research has focused on investigating and addressing bias in one-to-one interactions between humans and LLMs.
“While most research has focused on investigating and addressing bias in one-to-one interactions between humans and LLMs (18–20), less attention has been given to how these biases evolve through repeated communications in populations of LLM agents and, ultimately, in mixed human-LLM ecosystems (15), even though the safety of a single LLM does not necessarily imply the safety of a multi-agent system”
From Emergent social conventions and collective bias in LLM populations (2025) · cites this paper as [19]
Section 3.3 discusses identifying and mitigating bias through human review, feedback, and user control of LLM outputs. But the paper does not establish that most research focuses on one-to-one human–LLM interactions; that comparative claim goes beyond what our copy says. (LLM, conf=0.82)
Experimental — accuracy not yet measuredIn generative AI, addressing bias requires a holistic strategy.
“In the realm of generative AI, addressing bias is even more challenging as it requires a holistic strategy [14,21,22].”
From Fairness and Bias in Artificial Intelligence: A Brief Survey of Sources, Impacts, and Mitigation Strategies (2023) · cites this paper as [14]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 49 · Cited by 282
Part of this paper’s reference list could not be split into single entries.
Research works printed but not linked: 15
- S. Barocas, M. Hardt, and A. Narayanan, 2023. Fairness and machine learning: Limitations and opportunities. Cambridge, Mass.: MIT Press, and at https://fairmlbook.org, accessed 27 October 2023.
- R.K. Bellamy, K., Dey, M. Hind, S.C. Hoffman, S. Houde, K. Kannan, P. Lohia, J. Martino, S. Mehta, and A. Mojsilović, 2019. "AI fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias," IBM Journal of Research and Development, volume 63, numbers 4-5, pp. 4:1-4:15. doi: https://doi.org/10.1145/10.1147/JRD.2019.2942287, accessed 27 October 2023. · DOI
- D.K. Citron and F. Pasquale, 2014. "The scored society: Due process for automated predictions," Washington Law Review, volume 89, number 1, pp. 1-33, and at https://digitalcommons.law.uw.edu/wlr/vol89/iss1/2/, accessed 27 October 2023.
- S. Costanza-Chock, 2020. Design justice: Community-led practices to build the worlds we need. Cambridge, Mass.: MIT Press. doi: https://doi.org/10.7551/mitpress/12255.001.0001, accessed 27 October 2023. · DOI
- S. Dev and J.M. Phillips, 2019. "Attenuating bias in word vectors," Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, pp. 879-887, and at https://proceedings.mlr.press/v89/dev19a.html, accessed 27 October 2023.
- J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, 2019. "BERT: Pre-training of deep bidirectional transformers for language understanding," Proceedings of NAACL-HLT 2019, pp. 4,171-4,186, and at https://aclanthology.org/N19-1423.pdf, accessed 27 October 2023.
- H.C. Triandis, 1995. Individualism and collectivism. New York: Routledge. doi: https://doi.org/10.4324/9780429499845, accessed 27 October 2023. · DOI
- A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, L. Kaiser, and I. Polosukhin, 2017. "Attention is all you need," Advances in Neural Information Processing Systems, volume 30, at https://papers.nips.cc/paper_files/paper/2017, accessed 27 October 2023.
- S. Wachter, B. Mittelstadt, and L. Floridi, 2017. "Transparent, explainable, and accountable AI for robotics," Science Robotics, volume 2, number 6 (31 May). doi: https://doi.org/10.1126/scirobotics.aan60, accessed 27 October 2023. · DOI
- Q. Wang, B. Li, T. Xiao, J. Zhu, C. Li, D.F. Wong, and L.S. Chao, 2019. "Learning deep transformer models for machine translation," Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1,810-1,822, and at https://aclanthology.org/P19-1176.pdf, accessed 27 October 2023.
- T.-H. Wen, D. Vandyke, N. Mrkšić, M. Gašić, L.M. Rojas-Barahona, P.-H. Su, S. Ultes, and S. Young, 2017. "A network-based end-to-end trainable task-oriented dialogue system," Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, volume 1, Long papers, pp. 438-449, and at https://aclanthology.org/E17-1042/, accessed 27 October 2023.
- B.L. Whorf, 1964. Language, thought, and reality: Selected Writings of Benjamin Lee Whorf. Edited by John B. Carroll. Cambridge, Mass.: MIT Press.
- M.B. Zafar, I. Valera, M. Gomez Rodriguez, and K.P. Gummadi, 2017. "Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment," WWW '17: Proceedings of the 26th International Conference on World Wide Web, pp. 1,171-1,180. doi: https://doi.org/10.1145/3038912.3052660, accessed 27 October 2023. · DOI · Paper page
- R. Zellers, A. Holtzman, H. Rashkin, Y. Bisk, A. Farhadi, F. Roesner, and Y. Choi, 2019. "Defending against neural fake news," ANIPS'19: Proceedings of the 33rd International Conference on Neural Information Processing Systems, article number 812, pp. 9,054-9,065.
- S. Zhang, E. Dinan, J. Urbanek, A. Szlam, D. Kiela, and J. Weston, 2018. "Personalizing dialogue agents: I have a dog, do you have pets too?" Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, volume 1: Long papers, pp. 2,204-2,213, and at https://aclanthology.org/P18- 1205/, accessed 27 October 2023.
- Authors
- Ferrara, E.
- Affiliations
- University of Southern California (US)
- Corresponding author
- Ferrara, Emilio
- ORCID
- Ferrara, Emilio
- Venue
- First Monday
- Publisher
- University of Illinois Libraries
- Published
- 2023-11-07
- ISSN
- 1396-0458 (ISSN-L)
- 1396-0466
- OpenAlex
- W4388488349
- Cited by
- 282
- References
- 49
- Citations by year
- 2026: 55 · 2025: 77 · 2024: 95 · 2023: 55
- FWCI
- 70.56
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- First Monday (publishedVersion, cc-by-nc-sa)
- SSRN Electronic Journal (acceptedVersion)
- First Monday (acceptedVersion, cc-by-nc-sa)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Viterbi School of Engineering, University of Southern California
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- ChatGPT, large language models, algorithmic bias, bias mitigation, training data bias, ethical AI, generative AI risks, chatbots, virtual assistants, content creation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
