The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index; 1 of the 3 sources its claims cite were checked for support.
How persuasive is AI-generated propaganda?
Open accessAbstract
Can large language models, a form of artificial intelligence (AI), generate persuasive propaganda? We conducted a preregistered survey experiment of US respondents to investigate the persuasiveness of news articles written by foreign propagandists compared to content generated by GPT-3 davinci (a large language model). We found that GPT-3 can create highly persuasive text as measured by participants’ agreement with propaganda theses. We further investigated whether a person fluent in English could improve propaganda persuasiveness. Editing the prompt fed to GPT-3 and/or curating GPT-3’s output made GPT-3 even more persuasive, and, under certain conditions, as persuasive as the original propaganda. Our findings suggest that propagandists could use AI to create convincing content with limited effort.
Overview
CoChat Summary
TL;DR
A preregistered survey experiment with 8,221 US respondents found that GPT-3 davinci could generate propaganda articles that were highly persuasive, though slightly less persuasive than real-world foreign covert propaganda on average. Human editing of prompts and curation of outputs made GPT-3-generated propaganda as persuasive as, or more persuasive than, the original propaganda.
Key findings
- Original propaganda was highly persuasive: agreement with thesis statements rose from 24.4% in the control to 47.4% among those who read original propaganda, a 23 percentage point increase.
- GPT-3-generated propaganda was also highly persuasive: agreement was 43.5% versus 24.4% in the control, a 19.1 percentage point increase, but 3.9 percentage points lower than original propaganda on average.
- Human curation helped: after removing two of 18 GPT-3 outputs that did not advance the intended thesis, agreement rose to 45.6%, and the difference from original propaganda was no longer statistically significant.
- Editing prompts and selecting the best of three outputs produced even stronger effects: agreement was 52.7% compared with 47.4% for original propaganda; edited-prompt-only articles reached 46.4%, not statistically different from 47.4%.
Methods
- Preregistered survey experiment in December 2021 with US adults recruited via Lucid using quota sampling; final sample was 8,221 after attention-check failures and completion times under 3 minutes were excluded.
- Six English-language foreign covert propaganda articles (151–308 words) from real-world campaigns originating from Iran or Russia were compared with GPT-3 davinci outputs on the same topics; GPT-3 generated three articles per topic, with outputs under 686 or over 1,936 characters discarded.
- Respondents rated agreement with thesis statements for four randomly selected topics without reading an article (control), then read articles on the remaining two topics; analyses regressed percent agreement and scaled agreement on issue and article indicators.
Limitations
- The paper states its estimates may represent a lower bound because large language models are rapidly improving and because the experiment tested exposure to a single article, whereas propagandists could expose citizens to many articles.
- The paper states future research could test AI-generated propaganda across a wider range of issues and with information from multiple sources.
- Evident from the text: the study uses US respondents, six topics, and GPT-3 davinci in December 2021, so findings may not generalize to other populations, topics, models, or later AI systems.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
1 Valid · 1 Not valid · 2 Not checked · 29 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Two of the GPT-3 propaganda articles (out of 18 total) did not advance the intended claim.”
§ How persuasive is AI-generated propaganda? > Results > Human–machine teaming
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If a propagandist edited the input and selected the best of the three outputs on each topic, the GPT-3-generated propaganda would be even more persuasive than the original propaganda (52.7% compared to 47.4%).”
§ How persuasive is AI-generated propaganda? > Results > Human–machine teaming
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For most issues, each GPT-3-generated article was about as persuasive as the original propaganda.”
§ How persuasive is AI-generated propaganda? > Results > Persuasiveness of GPT-3-generated propaganda
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Figures S7 and S8 show that the persuasive effects of the original propaganda and GPT-3 propaganda were fairly consistent across social groups. We did not find substantial heterogeneity in treatment effects when we split the sample by demographic variables, partisanship/ideology, news consumption, time spent on social media, and more.”
§ How persuasive is AI-generated propaganda? > Results > Persuasiveness of GPT-3-generated propaganda
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The poor performance of these articles and a few others caused GPT-3 to perform slightly less well than the original propaganda on average, when, in Fig. 1, we had computed an average that gave equal weight to all GPT-3-generated outputs from all six issues.”
§ How persuasive is AI-generated propaganda? > Results > Persuasiveness of GPT-3-generated propaganda
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
11 more claims
Create a free account to see all 33 claims and every source verdict.
- It may state two separate claims as one. 11
- Probably not a substantial claim. 12
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“But users increasingly cannot tell the difference between human writing and LLM writing when presented with text (30–32), and between a human interlocutor or a contemporary chatbot when in interaction (17, 26).”
From The benefits and dangers of anthropomorphic conversational agents (2025) · cites this paper as [32]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“as well as persuasive propaganda (20).”
From The impact of generative artificial intelligence on socioeconomic inequalities and policy making (2024) · cites this paper as [20]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“These mechanisms illustrate how affect-driven content can override cognitive vigilance, aligning with recent work on emotional misinformation (Goldstein et al., 2024; Perach et al., 2023).”
From The illusion of reality: How AI-generated images (AIGIs) are fooling social media users (2026) · cites this paper as Goldstein et al., 2024
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“At the very least, they support dissenting voices on the persuasive impact of either customization or elaboration (11, 28, 74, 76, 77).”
From Testing theories of political persuasion using AI (2025) · cites this paper as [74]
Experimental — accuracy not yet measured
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Self-citationPrior exposure increases perceived accuracy of fake news
Pennycook, G., Cannon, T. D., & Rand, D. G. (2018) · Journal of Experimental Psychology General · doi:10.1037/xge0000465 · Cited by 1,268 · cited 1×
“SEs and 95% CIs are shown. exposed multiple times (18), that repeated exposure to content on a social media platform leads to a higher rate of sharing that content (19, 20), and that belief in false information continues to grow loga-rithmically with additional exposures (21).”
BackgroundFrequency and the conference of referential validity
Hasher, L., Goldstein, D., & Toppino, T. (1977) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(77)80012-1 · Cited by 920
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764 · cited 1×
“Recently, many have voiced concern that new artificial intelligence (AI) tools could supercharge covert propaganda campaigns by allowing propagandists to mass produce text at low cost ( 8 – 10).”
BackgroundHolistic Evaluation of Language Models
Bommasani, R., Liang, P., & Lee, T. (2023) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.15007 · Cited by 480
Köbis, N., & Mossink, L. D. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2020.106553 · Cited by 406
All the News That’s Fit to Fabricate: AI-Generated Text as a Tool of Media Misinformation
Kreps, S., McCain, R. M., & Brundage, M. (2022) · Journal of Experimental Political Science · doi:10.1017/xps.2020.37 · Cited by 360 · cited 1×
“Scholars have examined whether people rate AI-generated news articles as credible (12) and recognize when AI-generated content is false (13), and whether elected officials reply to AI-generated constituent letters (14).”
BackgroundMisinformation During a Pandemic
Bursztyn, L., Rao, A., Roth, C., & Yanagizawa-Drott, D. (2020) · National Bureau of Economic Research · doi:10.3386/w27417 · Cited by 265
Holistic Evaluation of Language Models
Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., Zhang, Y., Narayanan, D., Wu, Y., Kumar, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.09110 · Cited by 255
AI model GPT-3 (dis)informs us better than humans
Spitale, G., Biller-Andorno, N., & Germani, F. (2023) · Science Advances · doi:10.1126/sciadv.adh1850 · Cited by 241
The effects of repetition frequency on the illusory truth effect
Hassan, A., & Barber, S. J. (2021) · Cognitive Research: Principles and Implications · doi:10.1186/s41235-021-00301-5 · Cited by 222 · cited 1×
“SEs and 95% CIs are shown. exposed multiple times (18), that repeated exposure to content on a social media platform leads to a higher rate of sharing that content (19, 20), and that belief in false information continues to grow loga-rithmically with additional exposures (21).”
BackgroundInformation utility and the multiple source effect
Harkins, S. G., & Petty, R. E. (1987) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.52.2.260 · Cited by 210 · cited 1×
“Similarly, research in psychology has shown that people are more likely to believe misinformation when it comes from multiple sources, rather than one (the “mul-tiple source effect”) (23) and more likely to adopt a position when it is supported by a greater number of arguments (24).”
BackgroundZeng, W., Ren, X., Su, T., Wang, H., Liao, Y., Wang, Z., Jiang, X., Yang, Z., Wang, K., Zhang, X., et al. (2021) · arXiv · doi:10.48550/arxiv.2104.12369 · Cited by 172
Goldstein, J. A., Sastry, G., Musser, M., DiResta, R., Gentzel, M., & Sedova, K. (2023) · arXiv · doi:10.48550/arxiv.2301.04246 · Cited by 152
Self-citationThe relation of cognitive and memorial processes to persuasion in a simulated jury trial
Calder, B. J., Insko, C. A., & Yandell, B. (1974) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.1974.tb02808.x · Cited by 139 · cited 1×
“Similarly, research in psychology has shown that people are more likely to believe misinformation when it comes from multiple sources, rather than one (the “mul-tiple source effect”) (23) and more likely to adopt a position when it is supported by a greater number of arguments (24).”
BackgroundThe illusory truth effect leads to the spread of misinformation
Vellani, V., Zheng, S., Ercelik, D., & Sharot, T. (2023) · Cognition · doi:10.1016/j.cognition.2023.105421 · Cited by 83
Truth, Lies, and Automation: How Language Models Could Change Disinformation
Center for Security and Emerging Technology, Buchanan, B., Lohn, A., Musser, M., & Sedova, K. (2021) · doi:10.51593/2021ca003 · Cited by 74
Cyberwar: How Russian Hackers and Trolls Helped Elect a President: What We Don't, Can't, and Do Know
Jamieson, K. H. (2020) · doi:10.1093/oso/9780190058838.001.0001 · Cited by 67
All the News that’s Fit to Fabricate: AI-Generated Text as a Tool of Media MisinformationPreprint
Kreps, S. E., McCain, R. M., & Brundage, M. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3525002 · Cited by 66
What label should be applied to content produced by generative AI?Preprint
Epstein, Z., Fang, M. C., Arechar, A. A., & Rand, D. G. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/v4mfz · Cited by 51
Impact of repeated exposures on information spreading in social networks
Zhou, C., Zhao, Q., & Lu, W. (2015) · PLoS ONE · doi:10.1371/journal.pone.0140556 · Cited by 42 · cited 1×
“SEs and 95% CIs are shown. exposed multiple times (18), that repeated exposure to content on a social media platform leads to a higher rate of sharing that content (19, 20), and that belief in false information continues to grow loga-rithmically with additional exposures (21).”
BackgroundKreps, S., & Kriner, D. L. (2024) · New Media & Society · doi:10.1177/14614448231160526 · Cited by 27
Stop talking about tomorrow’s AI doomsday when AI poses risks today
(2023) · Nature · doi:10.1038/d41586-023-02094-7 · Cited by 27 · cited 1×
“Recently, many have voiced concern that new artificial intelligence (AI) tools could supercharge covert propaganda campaigns by allowing propagandists to mass produce text at low cost ( 8 – 10).”
BackgroundIntroducing the Online Political Influence Efforts dataset
Martin, D. A., Shapiro, J. N., & Ilhardt, J. G. (2023) · Journal of Peace Research · doi:10.1177/00223433221092815 · Cited by 24
The Effects of Repetition Frequency on the Illusory Truth Effect
Hassan, A. F. (2021) · doi:10.46569/20.500.12680/pg15bm47s · Cited by 18
François, C., & Douek, E. (2021) · Journal of Online Trust and Safety · doi:10.54501/jots.v1i1.17 · Cited by 12
Human Detection of Political Speech Deepfakes across Transcripts, Audio, and VideoPreprint
Groh, M., Sankaranarayanan, A., Singh, N., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.12883 · Cited by 11
Cited by
The impact of generative artificial intelligence on socioeconomic inequalities and policy making
Capraro, V., Lentsch, A., Acemoglu, D., Akgun, S., Akhmedova, A., Bilancini, E., Bonnefon, J.-F., Brañas-Garza, P., Butera, L., Douglas, K. M., et al. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae191 · Cited by 371
On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial
Salvi, F., Ribeiro, M. H., Gallotti, R., & West, R. (2024) · doi:10.21203/rs.3.rs-4429707/v1 · Cited by 285
On the conversational persuasiveness of GPT-4
Salvi, F., Horta Ribeiro, M., Gallotti, R., & West, R. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02194-6 · Cited by 264
“Our study suggests that concerns around personalization and AI persuasion are warranted, reinforcing previous results 42,44,45,48 by show-casing how LLMs can outpersuade humans in online conversations through microtargeting.”
Evaluating the persuasive influence of political microtargeting with large language models
Hackenburg, K., & Margetts, H. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2403116121 · Cited by 114
LLM-generated messages can persuade humans on policy issues
Bai, H., Voelkel, J. G., Muldowney, S., Eichstaedt, J. C., & Willer, R. (2025) · Nature Communications · doi:10.1038/s41467-025-61345-5 · Cited by 110
Metadata
- Authors
- Goldstein, J. A., Chao, J., Grossman, S., Stamos, A., & Tomz, M.
- Affiliations
- Georgetown University (US)
- Stanford University (US)
- Stanford Institute for Economic Policy Research (US)
- Corresponding author
- Grossman, Shelby
- Venue
- PNAS Nexus
- Publisher
- Oxford University Press (OUP)
- Published
- 2024-02-01
- Volume
- 3
- Issue
- 2
- Pages
- pgae034-pgae034
- ISSN
- 2752-6542 (ISSN-L)
- PMC
- PMC10878360
- OpenAlex
- W4391985880
- Cited by
- 133
- References
- 18
- Citations by year
- 2026: 45 · 2025: 64 · 2024: 22
- FWCI
- 93.55
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PNAS Nexus (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Stanford University (#216301)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Social Sciences › Sociology and Political Science › Media Influence and Politics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- large language models, GPT-3, persuasion, automated content generation, disinformation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.