Bias of AI-generated content: an examination of news produced by large language models
Open accessPublished version of a preprint
The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
Large language models (LLMs) have the potential to transform our lives and work through the content they generate, known as AI-Generated Content (AIGC). To harness this transformation, we need to understand the limitations of LLMs. Here, we investigate the bias of AIGC produced by seven representative LLMs, including ChatGPT and LLaMA. We collect news articles from The New York Times and Reuters, both known for their dedication to provide unbiased news. We then apply each examined LLM to generate news content with headlines of these news articles as prompts, and evaluate the gender and racial biases of the AIGC produced by the LLM by comparing the AIGC and the original news articles. We further analyze the gender bias of each LLM under biased prompts by adding gender-biased messages to prompts constructed from these news headlines. Our study reveals that the AIGC produced by each examined LLM demonstrates substantial gender and racial biases. Moreover, the AIGC generated by each LLM exhibits notable discrimination against females and individuals of the Black race. Among the LLMs, the AIGC generated by ChatGPT demonstrates the lowest level of bias, and ChatGPT is the sole model capable of declining content generation when provided with biased prompts.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures










Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In LLM research, bias direction varies by model and task, with some studies finding pro-Black and others anti-Black patterns depending on the context [3, 29, 46].”
From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [29]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Findings on direction are mixed: An et al. [3] found consistent pro-female bias, while others find anti-female patterns depending on task [29].”
From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [29]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Neither race nor gender showed consistent effects, a result that diverges from the existing literature's primary focus on these two dimensions [3, 29, 94].”
From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [29]
The cited source says what the claim attributes to it.
In the citing paper
“Additionally, LLM-generated content often exhibits inherent biases [20] and can be misused for malicious purposes [37,53], further complicating efforts to maintain information integrity.”
From Beyond Binary: Towards Fine-Grained LLM-Generated Text Detection via Role Recognition and Involvement Measurement (2025) · cites this paper as [20]
In the citing paper
“AI algorithms that have been trained on biased data may disseminate false information or reinforce prejudices (Fang et al., 2024).”
From AI literacy in media education: A comparative study of mass communication students in the UAE and Pakistan (2025) · cites this paper as Fang et al., 2024
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
An Introduction to Categorical Data Analysis
Agresti, A. (2007) · Wiley Series in Probability and Statistics · doi:10.1002/0470114754 · Cited by 6,689
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
The Earth Mover's Distance as a Metric for Image Retrieval
Rubner, Y., Tomasi, C., & Guibas, L. J. (2000) · International Journal of Computer Vision · doi:10.1023/a:1026543900054 · Cited by 4,568
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Pennycook, G., & Rand, D. G. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2021.02.007 · Cited by 1,360 · cited 1×
“In future research, considering that people tend to trust news based on the news source and names of the journalist 22 , it would be intriguing to explore whether an LLM generates more or less biased content when the prompt includes both the news title and journalist name.”
BackgroundFriedman, B., & Nissenbaum, H. (1996) · ACM Transactions on Information Systems · doi:10.1145/230538.230561 · Cited by 1,203 · cited 1×
“At the sentence level, the bias of AIGC is evaluated as the difference between sentences related to each investigated population group in AIGC and their counterparts in the news articles produced by the selected news agencies, in terms of their expressed sentiments and toxicities.”
BackgroundStereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.416 · Cited by 643
StereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2020) · arXiv · doi:10.48550/arxiv.2004.09456 · Cited by 643
Defending Against Neural Fake News
Zellers, R., Holtzman, A., Rashkin, H., Bisk, Y., Farhadi, A., Roesner, F., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1905.12616 · Cited by 569
The Earth Mover's distance is the Mallows distance: some insights from statistics
Levina, E., & Bickel, P. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937632 · Cited by 562
The Woman Worked as a Babysitter: On Biases in Language Generation
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1339 · Cited by 537
Mitigating Gender Bias in Natural Language Processing: Literature Review
Sun, T., Gaut, A., Tang, S., Huang, Y., ElSherief, M., Zhao, J., Mirza, D., Belding, E., Chang, K.-W., & Wang, W. Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1159 · Cited by 492
Gonen, H., & Goldberg, Y. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1061 · Cited by 454
The effect of publishing peer review reports on referee behavior in five scholarly journals
Bravo, G., Grimaldo, F., López-Iñesta, E., Mehmani, B., & Squazzoni, F. (2019) · Nature Communications · doi:10.1038/s41467-018-08250-2 · Cited by 273
Holistic Evaluation of Language Models
Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., Zhang, Y., Narayanan, D., Wu, Y., Kumar, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.09110 · Cited by 255
The Evolution of Topic Modeling
Churchill, R., & Singh, L. (2022) · ACM Computing Surveys · doi:10.1145/3507900 · Cited by 244 · cited 1×
“To this end, we leveraged the technique of topic modeling because it is widely used to uncover prevalent semantics from a collection of documents 37 .”
MethodGonen, H., & Goldberg, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.03862 · Cited by 225
Automated identification of media bias in news articles: an interdisciplinary literature review
Hamborg, F., Donnay, K., & Gipp, B. (2019) · International Journal on Digital Libraries · doi:10.1007/s00799-018-0261-y · Cited by 216
The Woman Worked as a Babysitter: On Biases in Language GenerationPreprint
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01326 · Cited by 205
Beukeboom, C. J., & Burgers, C. (2019) · Review of Communication Research · doi:10.12840/issn.2255-4165.017 · Cited by 126
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
Huang, P.-S., Zhang, H., Jiang, R., Stanforth, R., Welbl, J., Rae, J., Maini, V., Yogatama, D., & Kohli, P. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.7 · Cited by 124
Self-citationGalos, D. R., & Coppock, A. (2023) · Science Advances · doi:10.1126/sciadv.ade7979 · Cited by 112
Contrasting Linguistic Patterns in Human and LLM-Generated News Text
Muñoz-Ortiz, A., Gómez-Rodríguez, C., & Vilares, D. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10903-2 · Cited by 94
Mitigating Gender Bias in Machine Learning Data Sets
Leavy, S., Meaney, G., Wade, K., & Greene, D. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-52485-2_2 · Cited by 70
A Large-Scale Test of Gender Bias in the Media
Shor, E., van de Rijt, A., & Fotouhi, B. (2019) · Sociological Science · doi:10.15195/v6.a20 · Cited by 66 · cited 1×
“For example, we observe that the percentage of female words is significantly less than the percentage of male words in a news article on average, which aligns with the well-established fact that women tend to receive less media coverage than men 24 .”
BackgroundCentola, D., Guilbeault, D., Sarkar, U., Khoong, E., & Zhang, J. (2021) · Nature Communications · doi:10.1038/s41467-021-26905-5 · Cited by 43
Automated Journalism as a Source of and a Diagnostic Device for Bias in Reporting
Leppänen, L., Tuulonen, H., & Sirén‐Heikel, S. (2020) · Media and Communication · doi:10.17645/mac.v8i3.3022 · Cited by 32 · cited 2×
“Further investigation is needed to explore other types of biases, such as position bias 14 and word-specific biases 15 .”
BackgroundKirk, H. R., Jun, Y., Iqbal, H., Benussi, E., Volpin, F., Dreyer, F. A., Shtedritski, A., & Asano, Y. M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.04130 · Cited by 25 · cited 1×
“Our study, on the other hand, evaluates the bias embodied in documents (i.e., news articles) produced by an LLM, in terms of its word choices as well as expressed sentiments, toxicities, and semantics in these documents.”
BackgroundSentiments analysis of fMRI using automatically generated stimuli labels under naturalistic paradigm
mahrukh, R., Shakil, S., & Malik, A. S. (2023) · Scientific Reports · doi:10.1038/s41598-023-33734-7 · Cited by 25
Gender bias goes away when grant reviewers focus on the science
Guglielmi, G. (2018) · Nature · doi:10.1038/d41586-018-01212-0 · Cited by 24 · cited 1×
“And the document-level bias of AIGC is assessed as the difference between documents in AIGC and their counterparts produced by the selected news agencies, in terms of their expressed semantics regarding each investigated population group.”
MethodReducing Sentiment Bias in Language Models via Counterfactual Evaluation
Huang, P.-S., Zhang, H., Jiang, R., Stanforth, R., Welbl, J., Rae, J. W., Maini, V., Yogatama, D., & Kohli, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.03064 · Cited by 22 · cited 1×
“Our evaluation differs from existing studies that primarily focus on evaluating the bias manifested in short phrases (e.g., a word or a sentence) generated by language models 18–21 .”
MethodSelf-citationHannabuss, S. (1995) · Library Management · doi:10.1108/01435129510076204 · Cited by 7 · cited 2×
“Specifically, a topic model is trained to discover prevailing topics from a corpus of documents and each topic is represented as a distribution of words, measuring how likely each word is used when expressing the semantics regarding that topic.”
BackgroundMethodAn Efficient Technique of Predicting Toxicity on Music Lyrics Machine Learning
Noor, N. B., & Ahmed, I. (2023) · International Conference on Electrical, Computer and Communication Engineering · doi:10.1109/ecce57851.2023.10101658 · Cited by 7
Research works printed but not linked: 5
- Baker RS Hawn A Algorithmic bias in education Int. J. Artif. Intell. Educ. 2021 32 1 41
- Liang, P. P., Wu, C., Morency, L. -P. & Salakhutdinov, R. Towards understanding and mitigating social biases in language models, in International Conference on Machine Learning, 6565–6576 (PMLR, 2021).
- Nadeem, A., Abedin, B. & Marjanovic, O. Gender bias in AI: A review of contributing factors and mitigating strategies. ACIS 2020 Proc. (2020).
- Řehůřek, R., & Sojka, P. Software framework for topic modelling with large corpora, in Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, 45–50 (ELRA, Valletta, Malta, 2010).
- Sharpe D Chi-square test is statistically significant: Now what? Pract. Assessment Res. Eval. 2019 20 8
Cited by
A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly
Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., & Zhang, Y. (2024) · High-Confidence Computing · doi:10.1016/j.hcc.2024.100211 · Cited by 1,083
“In this section, we examine the range of existing defense methods against various attacks and vulnerabilities associated with LLMs 1 .”
BackgroundGe, J., Sun, S., Owens, J., Galvez, V., Gologorskaya, O., Lai, J. C., Pletcher, M. J., & Lai, K. (2024) · Hepatology · doi:10.1097/hep.0000000000000834 · Cited by 178
Swisher, A. R., Wu, A. W., Liu, G. C., Lee, M., Carle, T. R., & Tang, D. M. (2024) · Otolaryngology · doi:10.1002/ohn.927 · Cited by 91
Mei, P., Brewis, D. N., Nwaiwu, F., Sumanathilaka, D., Alva-Manchego, F., & Demaree-Cotton, J. (2025) · Computers in Human Behavior: Artificial Humans · doi:10.1016/j.chbah.2025.100140 · Cited by 50
“More widely it has been noted that there is a bias aGenAInst AI use, which is often manifested in hesitancy among individuals to fully accept AI-generated intellectual, artistic and creative works due to the perception of it being "superhuman" (Fang et al., 2024; Ragot et al., 2020).”
Jang, M. (2024) · Informatics · doi:10.3390/informatics11030054 · Cited by 45
Metadata
- Authors
- Fang, X., Che, S., Mao, M., Zhang, H., Zhao, M., & Zhao, X.
- Affiliations
- University of Delaware (US)
- Tsinghua University (CN)
- Chinese University of Hong Kong, Shenzhen (CN)
- Shanghai University of Finance and Economics (CN)
- Corresponding author
- Fang, Xiao
- Che, Shangkun
- Venue
- Scientific Reports
- Published
- 2024-03-04
- Volume
- 14
- Issue
- 1
- Pages
- 5224-5224
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC10909834
- OpenAlex
- W4392362609
- Cited by
- 194
- References
- 28
- Citations by year
- 2026: 65 · 2025: 93 · 2024: 34
- FWCI
- 79.58
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Scientific Reports (publishedVersion, cc-by)
- Research Square (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Library, Museums and Press - UDSpace (University of Delaware) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- AI-generated content, large language models, algorithmic bias, gender bias, racial bias, ChatGPT, llamas, discrimination against women, Black people, New York Times, Reuters
- MeSH terms
- Animals, Aortic Valve Insufficiency, Camelids, New World, Female, Humans, Language, Male, Bias, Sexism
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.