Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models
Open accessAbstract
As generative language models, exemplified by ChatGPT, continue to advance in their capabilities, the spotlight on biases inherent in these models intensifies. This article delves into the distinctive challenges and risks associated with biases specifically in large-scale language models. We explore the origins of biases, stemming from factors such as training data, model specifications, algorithmic constraints, product design, and policy decisions. Our examination extends to the ethical implications arising from the unintended consequences of biased model outputs. In addition, we analyze the intricacies of mitigating biases, acknowledging the inevitable persistence of some biases, and consider the consequences of deploying these models across diverse applications, including virtual assistants, content generation, and chatbots. Finally, we provide an overview of current approaches for identifying, quantifying, and mitigating biases in language models, underscoring the need for a collaborative, multidisciplinary effort to craft AI systems that embody equity, transparency, and responsibility. This article aims to catalyze a thoughtful discourse within the AI community, prompting researchers and developers to consider the unique role of biases in the domain of generative language models and the ongoing quest for ethical AI.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
If a mobile health app is primarily used by individuals who are already health-conscious and actively managing their wellbeing, the data collected may not accurately represent the broader population's health behaviors and needs.
“For example, if a mobile health app is primarily used by individuals who are already health-conscious and actively managing their wellbeing, the data collected may not accurately represent the broader population's health behaviors and needs [86], [90].”
From Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond (2024) · cites this paper as [90]
Cited by
Sorted by citations
Survey on large language model-enhanced reinforcement learning: Concept, taxonomy, and methods
Cao, Y., Zhao, H., Cheng, Y., Shu, T., Chen, Y., Liu, G., Liang, G., Zhao, J., Yan, J., & Li, Y. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3497992 · Cited by 174 · see citing works
Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond
Nasir, S., Khan, R. A., & Bai, S. (2024) · IEEE Access · doi:10.1109/access.2024.3369912 · Cited by 152 · see citing works
The Butterfly Effect in artificial intelligence systems: Implications for AI bias and fairness
Ferrara, E. (2024) · Machine Learning with Applications · doi:10.1016/j.mlwa.2024.100525 · Cited by 160 · see citing works
Self-citationEmergent social conventions and collective bias in LLM populations
Ashery, A. F., Aiello, L. M., & Baronchelli, A. (2025) · Science Advances · doi:10.1126/sciadv.adu9368 · Cited by 81 · see citing works
Large language models for life cycle assessments: Opportunities, challenges, and risks
Preuss, N., Alshehri, A. S., & You, F. (2024) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2024.142824 · Cited by 71 · see citing works
Xiao, Y., Zhang, T., & He, J. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e37238 · Cited by 52 · see citing works
RetractedEthical Aspects of ChatGPT in Software Engineering Research
Akbar, M. A., Khan, A. A., & Liang, P. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3318183 · Cited by 52 · see citing works
Giabbanelli, P. J. (2023) · Winter Simulation Conference · doi:10.1109/wsc60868.2023.10408017 · Cited by 30 · see citing works
Aloudat, M. Z., Aboumadi, A., Soliman, A., Al-Mohammed, H. A., Al-Ali, M. S., Mahgoub, A., Barhamgi, M., & Yaacoub, E. E. (2025) · IEEE Access · doi:10.1109/access.2025.3555753 · Cited by 27 · see citing works
Rahman, M. A., Alqahtani, L., Albooq, A., & Ainousah, A. (2024) · Learning and Technology Conference (L&T) · doi:10.1109/lt60077.2024.10469434 · Cited by 32 · see citing works
A Glimpse of chatGPT: An Introduction of Features, Challenges, and Threads in Higher Education
Arista, A., Shuib, L. M., & Ismail, M. A. (2023) · International Conference on Informatics, Multimedia, Cyber and Information System (ICIMCIS) · doi:10.1109/icimcis60089.2023.10349057 · Cited by 20 · see citing works
Yip, D. W., Esmradi, A., & Chan, C. F. (2023) · IEEE Asia-Pacific Conference on Computer Science and Data Engineering (CSDE) · doi:10.1109/csde59766.2023.10487667 · Cited by 19 · see citing works
Koh, E., Zhang, L., Lee, A. V. Y., & Wang, H. (2024) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2024.3385009 · Cited by 12 · see citing works
Large Language Models in Wargaming: Methodology, Application, and Robustness
Chen, Y., & Chu, S. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00295 · Cited by 10 · see citing works
Palanisamy, B., Chakrabarti, A., Singh, A., Hassija, V., Chalapathi, G., & Singh, A. R. (2024) · IEEE Access · doi:10.1109/access.2024.3521596 · Cited by 10 · see citing works
Integration of Large Language Models into Higher Education: A Perspective from Learners
Zdravkova, K., Dalipi, F., & Ahlgren, F. (2023) · International Symposium on Computers in Education · doi:10.1109/siie59826.2023.10423681 · Cited by 8 · see citing works
Quantifying Bias in Text-to-Image Generative Models
Vice, J., Akhtar, N., Hartley, R. I., & Mian, A. S. (2025) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2025.3572115 · Cited by 13 · see citing works
Detecting the presence of social bias in GPT-3.5 using association tests
Mhatre, A. (2023) · International Conference on Advanced Computing Technologies and Applications (ICACTA) · doi:10.1109/icacta58201.2023.10392776 · Cited by 6 · see citing works
Semantics Preserving Emoji Recommendation with Large Language Models
Qiu, Z., Qiu, K., Lyu, H., Xiong, W., & Luo, J. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825696 · Cited by 6 · see citing works
Rayavaram, P., Narain, S., & Martin, F. (2024) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie61694.2024.10892958 · Cited by 4 · see citing works
Perception of Bias in ChatGPT: Analysis of Social Media Data
Wahbeh, A., Al-Ramahi, M. A., El-Gayar, O., Elnoshokaty, A., & Nasralah, T. (2023) · IEEE Global Conference on Artificial Intelligence and Internet of Things (GCAIoT) · doi:10.1109/gcaiot61060.2023.10385099 · Cited by 4 · see citing works
Would Business Applications Such as the FinTech Benefit by ChatGPT? If So, Where and How?
Gill, M. A., Zhou, X., Nabi, F., Genrich, R., & Gururajan, R. (2023) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat59888.2023.00090 · Cited by 2 · see citing works
Large Language Model Jailbreak Based on Stylistic Rewriting
Hou, W., Hu, N., Hu, Y., & Luo, W. (2025) · IEEE International Conference on Data Science in Cyberspace · doi:10.1109/dsc67331.2025.00068 · Cited by 1 · see citing works
Are Horses Always Strong and Donkeys Dumb? Animal Bias in Vision Language Models
Anas, M., Nadeem, M., Sohail, S. S., Cambria, E., & Hussain, A. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228584 · Cited by 1 · see citing works
Group size effects and collective misalignment in LLM multi-agent systems
Flint, A., Aiello, L. M., Pastor‐Satorras, R., & Baronchelli, A. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2531697123 · Cited by 1 · see citing works
Generative AI in Academic Publishing
Kumar, A., McKeown, D., Syed, H., Craig, C. J., Rupley, W. H., Ahmed, A., Kashwani, S., & Siddiqui, K. A. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/407619
Denny, P. C., Leinonen, J., Hellas, A., Sarsa, S., Liut, M., & Khosravi, H. (2026) · Western Canadian Conference on Computing Education · doi:10.1145/3828792.3828801
Alperen, A. V. C. I., & Demir, E. (2026) · KADEM Kadın Araştırmaları Dergisi · doi:10.21798/kadem.2026.220
Fehring, L., Frings, J., Rust, P., Kempny, C., Thürmann, P., Tamerler, E., & Meister, S. (2026) · medRxiv · doi:10.64898/2026.10.01.26364479
- Authors
- Ferrara, E.
- Affiliations
- University of Southern California (US)
- Corresponding author
- Ferrara, Emilio
- ORCID
- Ferrara, Emilio
- Venue
- First Monday
- Publisher
- Elsevier BV
- Published
- 2023
- ISSN
- 1396-0458 (ISSN-L)
- OpenAlex
- W4388488349
- Cited by
- 282
- References
- 0
- Citations by year
- 2026: 55 · 2025: 77 · 2024: 95 · 2023: 55
- FWCI
- 70.56
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by-nc-sa
- Locations
- First Monday (publishedVersion, cc-by-nc-sa)
- SSRN Electronic Journal (acceptedVersion)
- First Monday (acceptedVersion, cc-by-nc-sa)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Viterbi School of Engineering, University of Southern California
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- ChatGPT, large language models, algorithmic bias, bias mitigation, training data bias, ethical AI, generative AI risks, chatbots, virtual assistants, content creation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
