Bias of AI-generated content: an examination of news produced by large language models
Open accessPublished version of a preprint
The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
Large language models (LLMs) have the potential to transform our lives and work through the content they generate, known as AI-Generated Content (AIGC). To harness this transformation, we need to understand the limitations of LLMs. Here, we investigate the bias of AIGC produced by seven representative LLMs, including ChatGPT and LLaMA. We collect news articles from The New York Times and Reuters, both known for their dedication to provide unbiased news. We then apply each examined LLM to generate news content with headlines of these news articles as prompts, and evaluate the gender and racial biases of the AIGC produced by the LLM by comparing the AIGC and the original news articles. We further analyze the gender bias of each LLM under biased prompts by adding gender-biased messages to prompts constructed from these news headlines. Our study reveals that the AIGC produced by each examined LLM demonstrates substantial gender and racial biases. Moreover, the AIGC generated by each LLM exhibits notable discrimination against females and individuals of the Black race. Among the LLMs, the AIGC generated by ChatGPT demonstrates the lowest level of bias, and ChatGPT is the sole model capable of declining content generation when provided with biased prompts.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures










Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In LLM research, bias direction varies by model and task, with some studies finding pro-Black and others anti-Black patterns depending on the context [3, 29, 46].”
From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [29]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Findings on direction are mixed: An et al. [3] found consistent pro-female bias, while others find anti-female patterns depending on task [29].”
From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [29]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Neither race nor gender showed consistent effects, a result that diverges from the existing literature's primary focus on these two dimensions [3, 29, 94].”
From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [29]
The cited source says what the claim attributes to it.
In the citing paper
“Additionally, LLM-generated content often exhibits inherent biases [20] and can be misused for malicious purposes [37,53], further complicating efforts to maintain information integrity.”
From Beyond Binary: Towards Fine-Grained LLM-Generated Text Detection via Role Recognition and Involvement Measurement (2025) · cites this paper as [20]
In the citing paper
“AI algorithms that have been trained on biased data may disseminate false information or reinforce prejudices (Fang et al., 2024).”
From AI literacy in media education: A comparative study of mass communication students in the UAE and Pakistan (2025) · cites this paper as Fang et al., 2024
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
An Introduction to Categorical Data Analysis
Agresti, A. (2007) · Wiley Series in Probability and Statistics · doi:10.1002/0470114754 · Cited by 6,689
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
The Earth Mover's Distance as a Metric for Image Retrieval
Rubner, Y., Tomasi, C., & Guibas, L. J. (2000) · International Journal of Computer Vision · doi:10.1023/a:1026543900054 · Cited by 4,568
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Pennycook, G., & Rand, D. G. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2021.02.007 · Cited by 1,360 · cited 1×
“In future research, considering that people tend to trust news based on the news source and names of the journalist 22 , it would be intriguing to explore whether an LLM generates more or less biased content when the prompt includes both the news title and journalist name.”
BackgroundFriedman, B., & Nissenbaum, H. (1996) · ACM Transactions on Information Systems · doi:10.1145/230538.230561 · Cited by 1,203 · cited 1×
“At the sentence level, the bias of AIGC is evaluated as the difference between sentences related to each investigated population group in AIGC and their counterparts in the news articles produced by the selected news agencies, in terms of their expressed sentiments and toxicities.”
BackgroundStereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.416 · Cited by 643
Research works printed but not linked: 5
- Baker, R. S. & Hawn, A. Algorithmic bias in education. Int. J. Artif. Intell. Educ. 32, 1–41 (2021).
- Liang, P. P., Wu, C., Morency, L. -P. & Salakhutdinov, R. Towards understanding and mitigating social biases in language models, in International Conference on Machine Learning, 6565–6576 (PMLR, 2021).
- Nadeem, A., Abedin, B. & Marjanovic, O. Gender bias in AI: A review of contributing factors and mitigating strategies. ACIS 2020 Proc. (2020).
- Řehůřek, R., & Sojka, P. Software framework for topic modelling with large corpora, in Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, 45–50 (ELRA, Valletta, Malta, 2010).
- Sharpe, D. Chi-square test is statistically significant: Now what?. Pract. Assessment Res. Eval. 20, 8 (2019).
Cited by
A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly
Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., & Zhang, Y. (2024) · High-Confidence Computing · doi:10.1016/j.hcc.2024.100211 · Cited by 1,083
“In this section, we examine the range of existing defense methods against various attacks and vulnerabilities associated with LLMs 1 .”
BackgroundGe, J., Sun, S., Owens, J., Galvez, V., Gologorskaya, O., Lai, J. C., Pletcher, M. J., & Lai, K. (2024) · Hepatology · doi:10.1097/hep.0000000000000834 · Cited by 178
Swisher, A. R., Wu, A. W., Liu, G. C., Lee, M., Carle, T. R., & Tang, D. M. (2024) · Otolaryngology · doi:10.1002/ohn.927 · Cited by 91
Mei, P., Brewis, D. N., Nwaiwu, F., Sumanathilaka, D., Alva-Manchego, F., & Demaree-Cotton, J. (2025) · Computers in Human Behavior: Artificial Humans · doi:10.1016/j.chbah.2025.100140 · Cited by 50
“More widely it has been noted that there is a bias aGenAInst AI use, which is often manifested in hesitancy among individuals to fully accept AI-generated intellectual, artistic and creative works due to the perception of it being "superhuman" (Fang et al., 2024; Ragot et al., 2020).”
Jang, M. (2024) · Informatics · doi:10.3390/informatics11030054 · Cited by 45
Metadata
- Authors
- Fang, X., Che, S., Mao, M., Zhang, H., Zhao, M., & Zhao, X.
- Affiliations
- University of Delaware (US)
- Tsinghua University (CN)
- Chinese University of Hong Kong, Shenzhen (CN)
- Shanghai University of Finance and Economics (CN)
- Corresponding author
- Fang, Xiao
- Che, Shangkun
- Venue
- Scientific Reports
- Published
- 2024-03-04
- Volume
- 14
- Issue
- 1
- Pages
- 5224-5224
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC10909834
- OpenAlex
- W4392362609
- Cited by
- 194
- References
- 28
- Citations by year
- 2026: 65 · 2025: 93 · 2024: 34
- FWCI
- 79.58
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Scientific Reports (publishedVersion, cc-by)
- Research Square (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Library, Museums and Press - UDSpace (University of Delaware) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- AI-generated content, large language models, algorithmic bias, gender bias, racial bias, ChatGPT, llamas, discrimination against women, Black people, New York Times, Reuters
- MeSH terms
- Animals, Aortic Valve Insufficiency, Camelids, New World, Female, Humans, Language, Male, Bias, Sexism
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Bias of AI-generated content: an examination of news produced by large language models. https://app.cochat.ai/papers/10.1038/s41598-024-55686-2
- The paper
- Fang, X., Che, S., Mao, M., Zhang, H., Zhao, M., & Zhao, X. (2024). Bias of AI-generated content: an examination of news produced by large language models. Scientific Reports. https://doi.org/10.1038/s41598-024-55686-2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.