How persuasive is AI-generated propaganda?
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index; 1 of the 3 sources its claims cite were checked for support.
Abstract
Can large language models, a form of artificial intelligence (AI), generate persuasive propaganda? We conducted a preregistered survey experiment of US respondents to investigate the persuasiveness of news articles written by foreign propagandists compared to content generated by GPT-3 davinci (a large language model). We found that GPT-3 can create highly persuasive text as measured by participants’ agreement with propaganda theses. We further investigated whether a person fluent in English could improve propaganda persuasiveness. Editing the prompt fed to GPT-3 and/or curating GPT-3’s output made GPT-3 even more persuasive, and, under certain conditions, as persuasive as the original propaganda. Our findings suggest that propagandists could use AI to create convincing content with limited effort.
Overview
CoChat Summary
TL;DR
A preregistered survey experiment with 8,221 US respondents found that GPT-3 davinci could generate propaganda articles that were highly persuasive, though slightly less persuasive than real-world foreign covert propaganda on average. Human editing of prompts and curation of outputs made GPT-3-generated propaganda as persuasive as, or more persuasive than, the original propaganda.
Key findings
- Original propaganda was highly persuasive: agreement with thesis statements rose from 24.4% in the control to 47.4% among those who read original propaganda, a 23 percentage point increase.
- GPT-3-generated propaganda was also highly persuasive: agreement was 43.5% versus 24.4% in the control, a 19.1 percentage point increase, but 3.9 percentage points lower than original propaganda on average.
- Human curation helped: after removing two of 18 GPT-3 outputs that did not advance the intended thesis, agreement rose to 45.6%, and the difference from original propaganda was no longer statistically significant.
- Editing prompts and selecting the best of three outputs produced even stronger effects: agreement was 52.7% compared with 47.4% for original propaganda; edited-prompt-only articles reached 46.4%, not statistically different from 47.4%.
Methods
- Preregistered survey experiment in December 2021 with US adults recruited via Lucid using quota sampling; final sample was 8,221 after attention-check failures and completion times under 3 minutes were excluded.
- Six English-language foreign covert propaganda articles (151–308 words) from real-world campaigns originating from Iran or Russia were compared with GPT-3 davinci outputs on the same topics; GPT-3 generated three articles per topic, with outputs under 686 or over 1,936 characters discarded.
- Respondents rated agreement with thesis statements for four randomly selected topics without reading an article (control), then read articles on the remaining two topics; analyses regressed percent agreement and scaled agreement on issue and article indicators.
Limitations
- The paper states its estimates may represent a lower bound because large language models are rapidly improving and because the experiment tested exposure to a single article, whereas propagandists could expose citizens to many articles.
- The paper states future research could test AI-generated propaganda across a wider range of issues and with information from multiple sources.
- Evident from the text: the study uses US respondents, six topics, and GPT-3 davinci in December 2021, so findings may not generalize to other populations, topics, models, or later AI systems.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
1 Valid · 1 Not valid · 2 Not checked · 29 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Two of the GPT-3 propaganda articles (out of 18 total) did not advance the intended claim.”
§ How persuasive is AI-generated propaganda? > Results > Human–machine teaming
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If a propagandist edited the input and selected the best of the three outputs on each topic, the GPT-3-generated propaganda would be even more persuasive than the original propaganda (52.7% compared to 47.4%).”
§ How persuasive is AI-generated propaganda? > Results > Human–machine teaming
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For most issues, each GPT-3-generated article was about as persuasive as the original propaganda.”
§ How persuasive is AI-generated propaganda? > Results > Persuasiveness of GPT-3-generated propaganda
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Figures S7 and S8 show that the persuasive effects of the original propaganda and GPT-3 propaganda were fairly consistent across social groups. We did not find substantial heterogeneity in treatment effects when we split the sample by demographic variables, partisanship/ideology, news consumption, time spent on social media, and more.”
§ How persuasive is AI-generated propaganda? > Results > Persuasiveness of GPT-3-generated propaganda
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The poor performance of these articles and a few others caused GPT-3 to perform slightly less well than the original propaganda on average, when, in Fig. 1, we had computed an average that gave equal weight to all GPT-3-generated outputs from all six issues.”
§ How persuasive is AI-generated propaganda? > Results > Persuasiveness of GPT-3-generated propaganda
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
11 more claims
Create a free account to see all 33 claims and every source verdict.
- It may state two separate claims as one. 11
- Probably not a substantial claim. 12
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“But users increasingly cannot tell the difference between human writing and LLM writing when presented with text (30–32), and between a human interlocutor or a contemporary chatbot when in interaction (17, 26).”
From The benefits and dangers of anthropomorphic conversational agents (2025) · cites this paper as [32]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“as well as persuasive propaganda (20).”
From The impact of generative artificial intelligence on socioeconomic inequalities and policy making (2024) · cites this paper as [20]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“These mechanisms illustrate how affect-driven content can override cognitive vigilance, aligning with recent work on emotional misinformation (Goldstein et al., 2024; Perach et al., 2023).”
From The illusion of reality: How AI-generated images (AIGIs) are fooling social media users (2026) · cites this paper as Goldstein et al., 2024
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“At the very least, they support dissenting voices on the persuasive impact of either customization or elaboration (11, 28, 74, 76, 77).”
From Testing theories of political persuasion using AI (2025) · cites this paper as [74]
Experimental — accuracy not yet measured
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Self-citationPrior exposure increases perceived accuracy of fake news
Pennycook, G., Cannon, T. D., & Rand, D. G. (2018) · Journal of Experimental Psychology General · doi:10.1037/xge0000465 · Cited by 1,268 · cited 1×
“SEs and 95% CIs are shown. exposed multiple times (18), that repeated exposure to content on a social media platform leads to a higher rate of sharing that content (19, 20), and that belief in false information continues to grow loga-rithmically with additional exposures (21).”
BackgroundFrequency and the conference of referential validity
Hasher, L., Goldstein, D., & Toppino, T. (1977) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(77)80012-1 · Cited by 920
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764 · cited 1×
“Recently, many have voiced concern that new artificial intelligence (AI) tools could supercharge covert propaganda campaigns by allowing propagandists to mass produce text at low cost ( 8 – 10).”
BackgroundHolistic Evaluation of Language Models
Bommasani, R., Liang, P., & Lee, T. (2023) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.15007 · Cited by 480
Köbis, N., & Mossink, L. D. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2020.106553 · Cited by 406
All the News That’s Fit to Fabricate: AI-Generated Text as a Tool of Media Misinformation
Kreps, S., McCain, R. M., & Brundage, M. (2022) · Journal of Experimental Political Science · doi:10.1017/xps.2020.37 · Cited by 360 · cited 1×
“Scholars have examined whether people rate AI-generated news articles as credible (12) and recognize when AI-generated content is false (13), and whether elected officials reply to AI-generated constituent letters (14).”
BackgroundMisinformation During a Pandemic
Bursztyn, L., Rao, A., Roth, C., & Yanagizawa-Drott, D. (2020) · National Bureau of Economic Research · doi:10.3386/w27417 · Cited by 265
Holistic Evaluation of Language Models
Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., Zhang, Y., Narayanan, D., Wu, Y., Kumar, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.09110 · Cited by 255
Cited by
Items 101–126 of 126
The Twin Disruption of Generative AI: Unbundling Capability and Responsibility in Creative WorkPreprint
Walkowiak, E., & Potts, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5362872
High‐Risk Elections in a New Sociopolitical Environment
Núñez, R. R., Alvim, F. F., & Monteiro, V. D. A. (2025) · Wiley eBooks · doi:10.1002/9781394311811.ch1
Vériter, S. L. (2025) · Springer eBooks · doi:10.1007/978-3-031-93385-1_14 · Cited by 1
Biały, F., & Gibson, R. (2025) · Parliamentary Affairs · doi:10.1093/pa/gsaf012 · Cited by 1
Machine conviction: Can we control what AI make us believe?
Cebrián, M. (2025) · Mètode: Revista de difusió de la investigació de la Universitat de Valencia · doi:10.7203/metode.15.30429
How Confident AI Misleads: When Certainty Heuristics Undermine Skepticism in MisinformationPreprint
Chen, C. C., & Yang, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5875242 · Cited by 1
Propaganda, Democracy, and Emerging Technology
Martin, Z., Trauthig, I. K., & Woolley, S. (2026) · Springer eBooks · doi:10.1007/978-3-658-44521-8_105
Disinformation and Superspreaders: With Epistemic Power Should Come Criminal Responsibility
Mamak, K. (2026) · Episteme · doi:10.1017/epi.2026.10107 · Cited by 1
“Goldstein et al. (2024) and Salvi et al. (2025) show that modern AI systems can generate persuasive propaganda quickly, cheaply, and with little technical expertise, enabling the effortless creation of manipulative content on a scale previously unimaginable (Goldstein et al. 2024; Salvi et al.…”
AI Ethics Framework for Detecting Manipulative AI in Social Media
Trilochan Ojha, P., Srinivasan, L., Jayapriya, P., Suganya, R., & Selvaraj, R. (2025) · International Conference on Signal Processing, Computation, Electronics, Power and Telecommunication (IConSCEPT) · doi:10.1109/iconscept66142.2025.11436513
Selective Persuasion: Structural Stance Bias of Generative AI in Controversial Issues
黎, 樟., & 塔, 娜. (2026) · Global Journal of Media Studies · doi:10.26599/gjms.2026.9330009 · Cited by 1
The people behind the machine: Social mistrust and AI regulatory skepticism across nationsPreprint
Tilak, S., Glassman, M., & Kang, M. J. (2026) · Research Square · doi:10.21203/rs.3.rs-10036198/v1
Algorithms of trust in the political and economic development of Central Asia
Mendybayev, B., BURBAYEVA, P., & Ainayeva, A. (2026) · Central Asian Economic Review · doi:10.52821/2789-4401-2026-2-17-34
Manipülasyon ve Propaganda Karşısında Pragmatist Retorik: Sofizmden Aristotelesçi Etik Amacına Geçiş
Danish, A. (2026) · Mediterranean Journal of Humanities · doi:10.13114/mjh.1892276
An Overview of Artificial Intelligence Techniques for Detecting and Analyzing Information Disorder
Pezzullo, G., Martino, B. D., Venticinque, S., & Amato, A. (2026) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-032-23347-9_31
Güler, Ö. R. (2026) · The Turkish Online Journal of Design Art and Communication · doi:10.7456/tojdac.1932971
Cybersecurity Research in International Relations
Topal, R. (2026) · Oxford University Press eBooks · doi:10.1093/9780197851456.003.0938
How Information Detail and Confidence in AI Responses Shape User Beliefs
Wu, Z., Jobanputra, M., Demberg, V., Hullman, J., & Feit, A. M. (2026) · International Conference on Human-AI Interaction and Experience Design (HAXD) · doi:10.1109/haxd70072.2026.11620844
“Earlier models like GPT-3 were already shown to produce persuasive outputs, such as pro-vaccination messages [9] or propaganda [10] and persuasiveness has been shown to increase as models become larger and more capable [11].”
A Research Agenda for Studying LLM-Powered Persuasion Technology to Mitigate Misinformation
Wardle, C., Brown, P., & Scales, D. A. (2026) · Media and Communication · doi:10.17645/mac.12442
“We do not address the separate literatures on using LLMs to create or micro‐target disinformation (Barman et al., 2024; Crothers et al., 2023; Goldstein et al., 2024) or on how LLMs themselves can be manipulated (Bozdag et al., 2025).”
Matulewski, M. (2026) · Cybersecurity & Cybercrime · doi:10.5604/01.3001.0055.8668
Five Myths About Influence Operations: What 25 Million Tweets Across Seven State Campaigns Reveal
Ferrara, E. (2026) · Systems · doi:10.3390/systems14080970 · Cited by 1
“…classifier across nineteen state operations attributes their detectability to shared, templated tactics [22], field experiments find no measurable persuasive effect from IRA contact [32], and even large-language-model microtargeting does not reliably outperform untargeted messaging [34,35].”
Bezmienov and Lem. Ideological Diversion in the Era of Algorithms and LLMs
Jakubski, K. J. (2026) · Internal Security · doi:10.5604/01.3001.0055.8841
Propaganda política en la era de la inteligencia artificial generativa: transformaciones digitales
Ruiz, O. M., & Ramírez, A. M. (2026) · Momboy · doi:10.70219/mby-272026-464
Angelis, C. D. (2026) · Revista de ciencias sociales. · doi:10.61303/0718-3631.v35i57.605
Almakaty, S. S. (2026) · Journalism and Media · doi:10.3390/journalmedia7040203 · Cited by 2
Prompt Before You Train: When Prompt Optimization Outperforms RLAIF for Small Language Models
Sarker, E., & Adaji, I. (2026) · International Conference on Information Technology for Social Good (GoodIT) · doi:10.1145/3794786.3830784
“Indeed, human-subject research shows that language models can generate persuasive propaganda with limited human effort [16].”
Haque, M. N. (2025) · Research journal in business and economics. · doi:10.61424/rjbe.v3i4.1082
Metadata
- Authors
- Goldstein, J. A., Chao, J., Grossman, S., Stamos, A., & Tomz, M.
- Affiliations
- Georgetown University (US)
- Stanford University (US)
- Stanford Institute for Economic Policy Research (US)
- Corresponding author
- Grossman, Shelby
- Venue
- PNAS Nexus
- Publisher
- Oxford University Press (OUP)
- Published
- 2024-02-01
- Volume
- 3
- Issue
- 2
- Pages
- pgae034-pgae034
- ISSN
- 2752-6542 (ISSN-L)
- PMC
- PMC10878360
- OpenAlex
- W4391985880
- Cited by
- 133
- References
- 18
- Citations by year
- 2026: 45 · 2025: 64 · 2024: 22
- FWCI
- 93.55
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PNAS Nexus (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Stanford University (#216301)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Social Sciences › Sociology and Political Science › Media Influence and Politics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- large language models, GPT-3, persuasion, automated content generation, disinformation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.