CoChat

Bias of AI-generated content: an examination of news produced by large language models

Open access

Published version of a preprint

The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.

Abstract

Large language models (LLMs) have the potential to transform our lives and work through the content they generate, known as AI-Generated Content (AIGC). To harness this transformation, we need to understand the limitations of LLMs. Here, we investigate the bias of AIGC produced by seven representative LLMs, including ChatGPT and LLaMA. We collect news articles from The New York Times and Reuters, both known for their dedication to provide unbiased news. We then apply each examined LLM to generate news content with headlines of these news articles as prompts, and evaluate the gender and racial biases of the AIGC produced by the LLM by comparing the AIGC and the original news articles. We further analyze the gender bias of each LLM under biased prompts by adding gender-biased messages to prompts constructed from these news headlines. Our study reveals that the AIGC produced by each examined LLM demonstrates substantial gender and racial biases. Moreover, the AIGC generated by each LLM exhibits notable discrimination against females and individuals of the Black race. Among the LLMs, the AIGC generated by ChatGPT demonstrates the lowest level of bias, and ChatGPT is the sole model capable of declining content generation when provided with biased prompts.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41598-024-55686-2

Cites

36 of 36 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Framework for Evaluating Bias of AIGC. (a) We proxy unbiased content with the news articles collected from The New York Times and Reuters. Please see the Section of “Data” for the justification of choosing these news agencies. We then apply an LLM to produce AIGC with headlines of these news articles as prompts and evaluate the gender and racial biases of AIGC by comparing it with the original news articles at the word, sentence, and document levels. (b) Examine the gender bias of AIGC under biased prompts.
Figure 1 Framework for Evaluating Bias of AIGC. (a) We proxy unbiased content with the news articles collected from The New York Times and Reuters. Please see the Section of “Data” for the justification of choosing these news agencies. We then apply an LLM to produce AIGC with headlines of these news articles as prompts and evaluate the gender and racial biases of AIGC by comparing it with the original news articles at the word, sentence, and document levels. (b) Examine the gender bias of AIGC under biased prompts.CC BY · PubMed Central
Figure 2 Gender Bias at Word Level. (a) Word level gender bias of an LLM and its 95% confidence interval (error bar), measured using the average Wasserstein distance defined by Eq. (4). For example, the gender bias score of 0.2407 by Grover indicates that, on average, the absolute difference between the percentage of male (or female) specific words out of all gender related words in a news article generated by Grover and that percentage in its counterpart collected from The New York Times or Reuters is 24.07%. (b) Percentage of female prejudice news articles generated by an LLM. We define a news article generated by an LLM as exhibiting female prejudice if the percentage of female specific words in it is lower than that percentage in its counterpart collected from The New York Times or Reuters. (c) Decrease of female specific words in female prejudice news articles generated by an LLM and its 95% confidence interval (error bar). For example, the score of -39.64% by Grover, reveals that, averaged across all female prejudice news articles generated by Grover, the percentage of female specific words is reduced from x% in their counterparts collected from The New York Times and Reuters to (x − 39.64)% in those generated by Grover.
Figure 2 Gender Bias at Word Level. (a) Word level gender bias of an LLM and its 95% confidence interval (error bar), measured using the average Wasserstein distance defined by Eq. (4). For example, the gender bias score of 0.2407 by Grover indicates that, on average, the absolute difference between the percentage of male (or female) specific words out of all gender related words in a news article generated by Grover and that percentage in its counterpart collected from The New York Times or Reuters is 24.07%. (b) Percentage of female prejudice news articles generated by an LLM. We define a news article generated by an LLM as exhibiting female prejudice if the percentage of female specific words in it is lower than that percentage in its counterpart collected from The New York Times or Reuters. (c) Decrease of female specific words in female prejudice news articles generated by an LLM and its 95% confidence interval (error bar). For example, the score of -39.64% by Grover, reveals that, averaged across all female prejudice news articles generated by Grover, the percentage of female specific words is reduced from x% in their counterparts collected from The New York Times and Reuters to (x − 39.64)% in those generated by Grover.CC BY · PubMed Central
Figure 3 Racial Bias at Word Level. (a) Word level racial bias of an LLM and its 95% confidence interval (error bar), measured using the average Wasserstein distance defined by Eq. (4). For example, the racial bias score of 0.3740 by Grover indicates that, on average, the absolute difference between the percentage of words related to an investigated race (White, Black, or Asian) out of all race-related words in a news article generated by Grover and that percentage in its counterpart collected from The New York Times or Reuters is as high as 37.40%. (b) Average difference between the percentage of White (or Black or Asian)-race specific words in a news article generated by an LLM and that percentage in its counterpart collected from The New York Times or Reuters. Error bar reflects 95% confidence interval. (c) Percentage of Black prejudice news articles generated by an LLM. We define a news article generated by an LLM as showing Black prejudice if the percentage of Black-race specific words in it is lower than that percentage in its counterpart collected from The New York Times or Reuters. (d) Decrease of Black-race specific words in Black prejudice news articles generated by an LLM and its 95% confidence interval (error bar). For example, the score of − 48.64% by Grover, reveals that, averaged across all Black prejudice news articles generated by Grover, the percentage of Black-race specific words is reduced from x% in their counterparts collected from The New York Times and Reuters to (x − 48.64)% in those generated by Grover.
Figure 3 Racial Bias at Word Level. (a) Word level racial bias of an LLM and its 95% confidence interval (error bar), measured using the average Wasserstein distance defined by Eq. (4). For example, the racial bias score of 0.3740 by Grover indicates that, on average, the absolute difference between the percentage of words related to an investigated race (White, Black, or Asian) out of all race-related words in a news article generated by Grover and that percentage in its counterpart collected from The New York Times or Reuters is as high as 37.40%. (b) Average difference between the percentage of White (or Black or Asian)-race specific words in a news article generated by an LLM and that percentage in its counterpart collected from The New York Times or Reuters. Error bar reflects 95% confidence interval. (c) Percentage of Black prejudice news articles generated by an LLM. We define a news article generated by an LLM as showing Black prejudice if the percentage of Black-race specific words in it is lower than that percentage in its counterpart collected from The New York Times or Reuters. (d) Decrease of Black-race specific words in Black prejudice news articles generated by an LLM and its 95% confidence interval (error bar). For example, the score of − 48.64% by Grover, reveals that, averaged across all Black prejudice news articles generated by Grover, the percentage of Black-race specific words is reduced from x% in their counterparts collected from The New York Times and Reuters to (x − 48.64)% in those generated by Grover.CC BY · PubMed Central
Figure 4 Gender Bias on Sentiment at Sentence Level. (a) An LLM’s gender bias on sentiment and its 95% confidence interval (error bar), measured using Eq. (5). For example, Grover attains 0.1483, which indicates that, on average, the maximal absolute difference between the average sentiment score of sentences pertaining to a population group (i.e., male or female) in a news article generated by Cohere and that score in its counterpart collected from The New York Times or Reuters is 0.1483. (b) Percentage of female prejudice news articles with respect to sentiment generated by an LLM. We define a news article generated by an LLM as exhibiting female prejudice with respect to sentiment if the average sentiment score of sentences related to females in the article is lower than the average sentiment score of sentences associated with females in its counterpart obtained from The New York Times or Reuters. (c) Sentiment score reduction in female prejudice news articles generated by an LLM and its 95% confidence interval (error bar). For example, the measurement score of -0.1441 by Grover, means that, on average, the average sentiment score of sentences related to females in a female prejudice news article generated by Grover is reduced by 0.1441, compared to its counterpart collected from The New York Times and Reuters.
Figure 4 Gender Bias on Sentiment at Sentence Level. (a) An LLM’s gender bias on sentiment and its 95% confidence interval (error bar), measured using Eq. (5). For example, Grover attains 0.1483, which indicates that, on average, the maximal absolute difference between the average sentiment score of sentences pertaining to a population group (i.e., male or female) in a news article generated by Cohere and that score in its counterpart collected from The New York Times or Reuters is 0.1483. (b) Percentage of female prejudice news articles with respect to sentiment generated by an LLM. We define a news article generated by an LLM as exhibiting female prejudice with respect to sentiment if the average sentiment score of sentences related to females in the article is lower than the average sentiment score of sentences associated with females in its counterpart obtained from The New York Times or Reuters. (c) Sentiment score reduction in female prejudice news articles generated by an LLM and its 95% confidence interval (error bar). For example, the measurement score of -0.1441 by Grover, means that, on average, the average sentiment score of sentences related to females in a female prejudice news article generated by Grover is reduced by 0.1441, compared to its counterpart collected from The New York Times and Reuters.CC BY · PubMed Central
Figure 5 Racial Bias on Sentiment at Sentence Level. (a) An LLM’s racial bias on sentiment and its 95% confidence interval (error bar), measured using Eq. (5). The racial bias score of 0.1480 by Grover indicates that, on average, the maximal absolute difference between the average sentiment score of sentences pertaining to a population group (i.e., White, Black, or Asian) in a news article generated by Grover and that score in its counterpart collected from The New York Times or Reuters is 0.1480. (b) Percentage of Black prejudice news articles with respect to sentiment generated by an LLM. We define a news article generated by an LLM as exhibiting Black prejudice with respect to sentiment if the average sentiment score of sentences related to the Black race in that article is lower than the average sentiment score of sentences associated with the Black race in its counterpart obtained from The New York Times or Reuters. (c) Decrease of sentiment score in Black prejudice news articles generated by an LLM and its 95% confidence interval (error bar). Taking Grover as an example, on average, the average sentiment score of sentences related to the Black race in a Black prejudice news article generated by Grover is decreased by 0.1443, compared to its counterpart collected from The New York Times or Reuters.
Figure 5 Racial Bias on Sentiment at Sentence Level. (a) An LLM’s racial bias on sentiment and its 95% confidence interval (error bar), measured using Eq. (5). The racial bias score of 0.1480 by Grover indicates that, on average, the maximal absolute difference between the average sentiment score of sentences pertaining to a population group (i.e., White, Black, or Asian) in a news article generated by Grover and that score in its counterpart collected from The New York Times or Reuters is 0.1480. (b) Percentage of Black prejudice news articles with respect to sentiment generated by an LLM. We define a news article generated by an LLM as exhibiting Black prejudice with respect to sentiment if the average sentiment score of sentences related to the Black race in that article is lower than the average sentiment score of sentences associated with the Black race in its counterpart obtained from The New York Times or Reuters. (c) Decrease of sentiment score in Black prejudice news articles generated by an LLM and its 95% confidence interval (error bar). Taking Grover as an example, on average, the average sentiment score of sentences related to the Black race in a Black prejudice news article generated by Grover is decreased by 0.1443, compared to its counterpart collected from The New York Times or Reuters.CC BY · PubMed Central
Figure 6 Gender Bias at Document Level. (a) Document level gender bias of an LLM and its 95% confidence interval (error bar), measured using Eq. (9). For example, the gender bias score of 0.2377 by Grover indicates that, on average, the absolute difference between the percentage of male (or female) pertinent topic out of all gender pertinent topics in a news article generated by Grover and that percentage in its counterpart collected from The New York Times or Reuters is 23.77%. (b) Percentage of document level female prejudice news articles generated by an LLM. We define a news article generated by an LLM as exhibiting female prejudice at the document level if the percentage of female pertinent topics in it is lower than that percentage in its counterpart collected from The New York Times or Reuters. (c) Decrease of female pertinent topics in document level female prejudice news articles generated by an LLM and its 95% confidence interval (error bar). For example, the score of − 37.60% by Grover, reveals that, averaged across all document level female prejudice news articles generated by Grover, the percentage of female pertinent topics is reduced from x% in their counterparts collected from The New York Times and Reuters to (x − 37.60)% in those generated by Grover.
Figure 6 Gender Bias at Document Level. (a) Document level gender bias of an LLM and its 95% confidence interval (error bar), measured using Eq. (9). For example, the gender bias score of 0.2377 by Grover indicates that, on average, the absolute difference between the percentage of male (or female) pertinent topic out of all gender pertinent topics in a news article generated by Grover and that percentage in its counterpart collected from The New York Times or Reuters is 23.77%. (b) Percentage of document level female prejudice news articles generated by an LLM. We define a news article generated by an LLM as exhibiting female prejudice at the document level if the percentage of female pertinent topics in it is lower than that percentage in its counterpart collected from The New York Times or Reuters. (c) Decrease of female pertinent topics in document level female prejudice news articles generated by an LLM and its 95% confidence interval (error bar). For example, the score of − 37.60% by Grover, reveals that, averaged across all document level female prejudice news articles generated by Grover, the percentage of female pertinent topics is reduced from x% in their counterparts collected from The New York Times and Reuters to (x − 37.60)% in those generated by Grover.CC BY · PubMed Central
Figure 7 Racial Bias at Document Level. (a) Document level racial bias of an LLM and its 95% confidence interval (error bar), measured using Eq. (9). For example, the racial bias score of 0.4853 by Grover indicates that, on average, the absolute difference between the percentage of topics pertaining to an investigated race (White, Black, or Asian) out of all race pertinent topics in a news article generated by Grover and that percentage in its counterpart collected from The New York Times or Reuters is as high as 48.53%. (b) Percentage of document level Black prejudice news articles generated by an LLM. We define a news article generated by an LLM as showing Black prejudice at the document level if the percentage of Black-race pertinent topics in it is lower than that percentage in its counterpart collected from The New York Times or Reuters. (c) Decrease of Black-race pertinent topics in document level Black prejudice news articles generated by an LLM and its 95% confidence interval (error bar). For example, the score of − 35.69% by Grover, reveals that, averaged across all document level Black prejudice news articles generated by Grover, the percentage of Black-race pertinent topics is reduced from x% in their counterparts collected from The New York Times and Reuters to (x − 35.69)% in those generated by Grover.
Figure 7 Racial Bias at Document Level. (a) Document level racial bias of an LLM and its 95% confidence interval (error bar), measured using Eq. (9). For example, the racial bias score of 0.4853 by Grover indicates that, on average, the absolute difference between the percentage of topics pertaining to an investigated race (White, Black, or Asian) out of all race pertinent topics in a news article generated by Grover and that percentage in its counterpart collected from The New York Times or Reuters is as high as 48.53%. (b) Percentage of document level Black prejudice news articles generated by an LLM. We define a news article generated by an LLM as showing Black prejudice at the document level if the percentage of Black-race pertinent topics in it is lower than that percentage in its counterpart collected from The New York Times or Reuters. (c) Decrease of Black-race pertinent topics in document level Black prejudice news articles generated by an LLM and its 95% confidence interval (error bar). For example, the score of − 35.69% by Grover, reveals that, averaged across all document level Black prejudice news articles generated by Grover, the percentage of Black-race pertinent topics is reduced from x% in their counterparts collected from The New York Times and Reuters to (x − 35.69)% in those generated by Grover.CC BY · PubMed Central
Figure 8 Gender bias comparison at word level: unbiased prompt versus biased prompt (a) Comparison between the proportion of female prejudice news articles generated by an LLM under unbiased prompts and the proportion of female prejudice news articles produced by the LLM using biased prompts. (b) Comparison between the decrease of female specific words in female prejudice news articles generated by an LLM under unbiased prompts and the decrease of female specific words in female prejudice news articles generated by the LLM under biased prompts. Error bar indicates 95% confidence interval.
Figure 8 Gender bias comparison at word level: unbiased prompt versus biased prompt (a) Comparison between the proportion of female prejudice news articles generated by an LLM under unbiased prompts and the proportion of female prejudice news articles produced by the LLM using biased prompts. (b) Comparison between the decrease of female specific words in female prejudice news articles generated by an LLM under unbiased prompts and the decrease of female specific words in female prejudice news articles generated by the LLM under biased prompts. Error bar indicates 95% confidence interval.CC BY · PubMed Central
Figure 9 Gender bias comparison at sentence level: unbiased prompt versus biased prompt (a) Comparison between the percentage of female prejudice news articles with respect to sentiment generated by an LLM under unbiased prompts and the percentage of female prejudice news articles with respect to sentiment produced by the LLM using biased prompts. (b) Comparison between sentiment score reduction in female prejudice news articles generated by an LLM under unbiased prompts and sentiment score reduction in female prejudice news articles generated by the LLM under biased prompts. Error bar indicates 95% confidence interval.
Figure 9 Gender bias comparison at sentence level: unbiased prompt versus biased prompt (a) Comparison between the percentage of female prejudice news articles with respect to sentiment generated by an LLM under unbiased prompts and the percentage of female prejudice news articles with respect to sentiment produced by the LLM using biased prompts. (b) Comparison between sentiment score reduction in female prejudice news articles generated by an LLM under unbiased prompts and sentiment score reduction in female prejudice news articles generated by the LLM under biased prompts. Error bar indicates 95% confidence interval.CC BY · PubMed Central
Figure 10 Gender Bias Comparison at the Document Level: Unbiased Prompt versus Biased Prompt. (a) Comparison between the proportion of document level female prejudice news articles generated by an LLM under unbiased prompts and the proportion of document level female prejudice news articles produced by the LLM using biased prompts. (b) Comparison between the decrease of female pertinent topics in female prejudice news articles generated by an LLM under unbiased prompts and the decrease of female pertinent topics in female prejudice news articles generated by the LLM under biased prompts. Error bar indicates 95% confidence interval.
Figure 10 Gender Bias Comparison at the Document Level: Unbiased Prompt versus Biased Prompt. (a) Comparison between the proportion of document level female prejudice news articles generated by an LLM under unbiased prompts and the proportion of document level female prejudice news articles produced by the LLM using biased prompts. (b) Comparison between the decrease of female pertinent topics in female prejudice news articles generated by an LLM under unbiased prompts and the decrease of female pertinent topics in female prejudice news articles generated by the LLM under biased prompts. Error bar indicates 95% confidence interval.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 36 references

Research works printed but not linked: 5

  1. Baker, R. S. & Hawn, A. Algorithmic bias in education. Int. J. Artif. Intell. Educ. 32, 1–41 (2021).
  2. Liang, P. P., Wu, C., Morency, L. -P. & Salakhutdinov, R. Towards understanding and mitigating social biases in language models, in International Conference on Machine Learning, 6565–6576 (PMLR, 2021).
  3. Nadeem, A., Abedin, B. & Marjanovic, O. Gender bias in AI: A review of contributing factors and mitigating strategies. ACIS 2020 Proc. (2020).
  4. Řehůřek, R., & Sojka, P. Software framework for topic modelling with large corpora, in Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, 45–50 (ELRA, Valletta, Malta, 2010).
  5. Sharpe, D. Chi-square test is statistically significant: Now what?. Pract. Assessment Res. Eval. 20, 8 (2019).

Cited by

See all 194

Metadata

Authors
Fang, X., Che, S., Mao, M., Zhang, H., Zhao, M., & Zhao, X.
Affiliations
  • University of Delaware (US)
  • Tsinghua University (CN)
  • Chinese University of Hong Kong, Shenzhen (CN)
  • Shanghai University of Finance and Economics (CN)
Corresponding author
  • Fang, Xiao
  • Che, Shangkun
Venue
Scientific Reports
Published
2024-03-04
Volume
14
Issue
1
Pages
5224-5224
ISSN
2045-2322 (ISSN-L)
OpenAlex
W4392362609
Cited by
194
References
28
Citations by year
2026: 65 · 2025: 93 · 2024: 34
FWCI
79.58
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by)
  • Research Square (acceptedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Library, Museums and Press - UDSpace (University of Delaware) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
Keywords
AI-generated content, large language models, algorithmic bias, gender bias, racial bias, ChatGPT, llamas, discrimination against women, Black people, New York Times, Reuters
MeSH terms
Animals, Aortic Valve Insufficiency, Camelids, New World, Female, Humans, Language, Male, Bias, Sexism

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Bias of AI-generated content: an examination of news produced by large language models. https://app.cochat.ai/papers/10.1038/s41598-024-55686-2
The paper
Fang, X., Che, S., Mao, M., Zhang, H., Zhao, M., & Zhao, X. (2024). Bias of AI-generated content: an examination of news produced by large language models. Scientific Reports. https://doi.org/10.1038/s41598-024-55686-2

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.