CoChat
Sign Up
Cited by 110Psychology of Social InfluenceOpinion Dynamics and Social InfluenceEthics and Social Impacts of AI

LLM-generated messages can persuade humans on policy issues

Open access
Bai, Hui, Voelkel, Jan G., Muldowney, Shane, Eichstaedt, Johannes C., Willer, Robb·2025·Nature Communications
DOI 10.1038/s41467-025-61345-5PMC PMC12215518

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

The emergence of large language models (LLMs) has made it possible for generative artificial intelligence (AI) to tackle many higher-order cognitive tasks, with critical implications for industry, government, and labor markets. Here, we investigate whether existing, openly-available LLMs can be used to create messages capable of influencing humans’ political attitudes. Across three pre-registered experiments (total N = 4829), participants who read persuasive messages generated by LLMs showed significantly more attitude change across a range of policies - including polarized policies, like an assault weapons ban, a carbon tax, and a paid parental-leave program - relative to control condition participants who read a neutral message. Overall, LLM-generated messages were similarly effective in influencing policy attitudes as messages crafted by lay humans. Participants’ reported perceptions of the authors of the persuasive messages suggest these effects occurred through somewhat distinct causal pathways. While the persuasiveness of LLM-generated messages was associated with perceptions that the author used more facts, evidence, logical reasoning, and a dispassionate voice, the persuasiveness of human-generated messages was associated with perceptions of the author as unique and original. These results demonstrate that recent developments in AI make it possible to create politically persuasive messages quickly, cheaply, and at massive scale.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41467-025-61345-5

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Most existing studies on AI anthropomorphism rely on non-representative samples, correlational study designs, or hypothetical vignettes that measure self-reported attitudes rather than actual behavior.

    “Most existing studies, with few exceptions [25], rely on non-representative samples, correlational study designs or hypothetical vignettes that measure self-reported attitudes rather than actual behavior [15, 26, 27].”

    From Humanlike AI Design Increases Anthropomorphism but Yields Divergent Outcomes on Engagement and Trust Globally (2025) · cites this paper as [27]

    The Methods (Study 3 participants; page unavailable) describe a sample recruited to represent US gender, race, and age demographics, and the paper reports randomized experiments—supporting its use as an exception to some design limitations. But the Methods (Study 1 and Study 3 main outcomes; page unavailable) measure self-reported policy support, not actual behavior, and the study investigates persuasion rather than AI anthropomorphism. It therefore does not support the citation as an exception to all the limitations in the sentence. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Many studies evaluate isolated AI-generated output, stripping away the interactive nature of conversation.

    “Furthermore, many studies evaluate isolated AI-generated output, stripping away the interactive nature of conversation [27].”

    From Humanlike AI Design Increases Anthropomorphism but Yields Divergent Outcomes on Engagement and Trust Globally (2025) · cites this paper as [27]

    The Introduction describes several prior studies that assessed AI-generated texts or messages, and Methods > Study 1 messages shows that this paper tested responses to a single assigned message (page numbers unavailable in our copy). This supports the point that some research evaluates isolated outputs. The paper does not establish the broader claim that many studies strip away the interactive nature of conversation. (LLM, conf=0.77)

    Experimental — accuracy not yet measured

  • With increasingly capable models, this threat has only increased, as studies show that LLM-crafted political arguments can influence user opinions as effectively as human-authored texts.

    “With increasingly capable models, this threat has only increased, as studies show that LLM-crafted political arguments can influence user opinions as effectively as human-authored texts (Bai et al., 2025).”

    From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Bai et al., 2025

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

References 42 · Cited by 110

Research works printed but not linked: 36

  1. [1]Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M. Hierarchical text-conditional image generation with clip latents. arXiv preprintarXiv:2204.06125 (2022).
  2. [2]Huang, C. Z. A. et al. The Bach doodle: approachable music composition with machine learning at scale. arXiv preprintarXiv:1907.06637 (2019).
  3. [3]OpenAI. GPT-4 Technical Report. ArXiv: 2303.08774 10.48550/arXiv.2303.08774. · DOI · Paper page
  4. [5]Slonim, N. et al. An autonomous debating system. Nature 591, 379–384 (2021). 33731946 · DOI
  5. [6]Meta Fundamental AI Research Diplomacy Team (FAIR) et al. Human-level play in the game of Diplomacy by combining language models with strategic reasoning. Science 378, 1067–1074 (2022). 36413172 · DOI · Paper page
  6. [8]Coppock, A. Persuasion in parallel. Chicago Studies in American Politics (University of Chicago Press, 2023).
  7. [9]Tappin, B. M., Wittenberg, C., Hewitt, L. B., Berinsky, A. J. & Rand, D. G. Quantifying the potential persuasive returns to political microtargeting. Proc. Natl. Acad. Sci. USA 120, e2216261120 (2023). 37307486 PMC10288628 · DOI · Paper page
  8. [10]Kalla, J. L. & Broockman, D. E. The minimal persuasive effects of campaign contact in general elections: evidence from 49 field experiments. Am. Polit. Sci. Rev. 112, 148–166 (2018). · DOI
  9. [12]Kim, T. W. & Duhachek, A. Artificial intelligence and persuasion: a construal-level account. Psychol. Sci. 31, 363–380 (2020). 32223692 · DOI
  10. [13]Jang, W., Chun, J. W., Kim, S. & Kang, Y. W. The effects of anthropomorphism on how people evaluate algorithm-written news. Digit. Journal.11, 103–124 (2021).
  11. [15]Karinshak, E., Liu, S. X., Park, J. S. & Hancock, J. T. Working with AI to persuade: examining a large language model’s ability to generate pro-vaccination messages. Proc. ACM Hum. Comput. Interact. 7, 1–29 (2023). · DOI · Paper page
  12. [19]Williams-Ceci, S. et al. Bias in AI autocomplete suggestions leads to attitude shift on societal issues. Open Science Framework. 10.31234/osf.io/mhjn6 (2024). · DOI
  13. [20]O’Keefe, D. J. Message pretesting using assessments of expected or perceived persuasiveness: evidence about diagnosticity of relative actual persuasiveness. J. Commun. 68, 120–142 (2018). · DOI
  14. [21]Pink, S. L. et al. The effects of short messages encouraging prevention behaviors early in the COVID-19 pandemic. PLoS ONE 18, e0284354 (2023). 37058445 PMC10104278 · DOI
  15. [22]Petty, R. E. & Cacioppo, J. T. Communication and Persuasion: Central and Peripheral Routes to Attitude Change (Springer Science & Business Media, 2012).
  16. [23]Nakano, R. et al. WebGPT: Browser-assisted question-answering with human feedback. arXiv preprintarXiv:2112.09332 (2022).
  17. [24]Fong, C. & Grimmer, J. Causal inference with latent treatments. Am. J. Polit. Sci. 67, 374–389 (2023). · DOI
  18. [25]Kramer, A. D., Guillory, J. E. & Hancock, J. T. Experimental evidence of massive-scale emotional contagion through social networks. Proc. Natl. Acad. Sci. USA 111, 8788 (2014). 24889601 PMC4066473 · DOI · Paper page
  19. [26]Markowitz, D. M. Instrumental goal activation increases online petition support across languages. J. Personal. Soc. Psychol. 124, 1133 (2023). 36480387 · DOI
  20. [27]Druckman, J. N., Peterson, E. & Slothuus, R. How elite partisan polarization affects public opinion formation. Am. Polit. Sci. Rev. 107, 57–79 (2013). · DOI · Paper page
  21. [28]Jakesch, M., Hancock, J. T. & Naaman, M. Human heuristics for AI-generated language are flawed. Proc. Natl. Acad. Sci. 120, e2208839120 (2023). 36881628 PMC10089155 · DOI · Paper page
  22. [29]Bullock, J. G., Green, D. P. & Ha, S. E. Yes, but what’s the mechanism? (don’t expect an easy answer). J. Personal. Soc. Psychol. 98, 550 (2010). 20307128 · DOI · Paper page
  23. [30]Xiao, L. A message’s persuasive features in Wikipedia’s article for deletion discussions. In Proc. 9th International Conference on Social Media and Society 345–349 (Association for Computing Machinery, 2018).
  24. [31]Ta, V. P. et al. An inclusive, real-world investigation of persuasion in language and verbal behavior. J. Comput. Soc. Sci. 5, 883–903 (2022). 34869936 PMC8633087 · DOI
  25. [34]Broockman, D. E., Kalla, J. L., Caballero, C. & Easton, M. Political practitioners poorly predict which messages persuade the public. Proc. Natl. Acad. Sci. USA 121, e2400076121 (2024). 39467135 PMC11551421 · DOI
  26. [35]Chu, J. Y. et al. Academics are more specific, and practitioners more sensitive, in forecasting interventions to strengthen democratic attitudes. Proc. Natl. Acad. Sci. USA 121, e2307008121 (2024). 38215187 PMC10801850 · DOI
  27. [36]Mummolo, J. & Peterson, E. Demand effects in survey experiments: an empirical assessment. Am. Polit. Sci. Rev. 113, 517–529 (2019). · DOI · Paper page
  28. [38]Goldstein, J. A. et al. Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations. arXiv preprintarXiv:2301.04246 (2023).
  29. [40]Feuerriegel, S. et al. Research can help to tackle AI-generated disinformation. Nat. Hum. Behav. 7, 1818–1821 (2023). 37985906 · DOI · Paper page
  30. [42]Gallegos, I. O. et al. Labeling messages as AI-generated does not reduce their persuasive effects. arXiv preprintarXiv:2504.09865v1 (2025). PMC12887897 41675465
  31. [43]Gao, C. A. et al. Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers. npj Digit. Med.6, 75 (2023). PMC10133283 37100871
  32. [44]Anderson-Nilsson, G. & Clayton, A. Gender and policy persuasion. Polit. Sci. Res. Methods 9, 818–831 (2021). · DOI
  33. [45]Clifford, S., Sheagley, G. & Piston, S. Increasing precision without altering treatment effects: repeated measures designs in survey experiments. Am. Polit. Sci. Rev. 115, 1048–1065 (2021). · DOI
  34. [46]Evangelidis, I. Task sensitivity and noise: how mechanical properties of preference elicitation tasks account for differences in preferences across tasks. Decision 11, 283–302 (2024). · DOI
  35. [47]Voelkel, J. G. et al. Interventions reducing affective polarization do not necessarily improve anti-democratic attitudes. Nat. Hum. Behav. 7, 55–64 (2023). 36316497 · DOI
  36. [48]Bai, H. Artificial Intelligence Can Persuade Humans on Political Issues. Open Science Framework. 10.17605/osf.io/8yxvr (2024). · DOI

Authors
Bai, H., Voelkel, J. G., Muldowney, S., Eichstaedt, J. C., & Willer, R.
Affiliations
  • Stanford University (US)
  • Cornell University (US)
Corresponding author
  • Bai, Hui
  • Willer, Robb
ORCID
Venue
Nature Communications
Publisher
Springer Science and Business Media LLC
Published
2025-07-01
Volume
16
Issue
1
Pages
6037-6037
ISSN
2041-1723 (ISSN-L)
DOI
10.1038/s41467-025-61345-5
PMC
PMC12215518
OpenAlex
W4411873400
Cited by
110
References
42
Citations by year
2026: 59 · 2025: 30 · 2024: 15 · 2023: 6
FWCI
153.96
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Communications (publishedVersion, cc-by)
  • Research Square (acceptedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Stanford Center on Philanthropy and Civil Society
Topics
  • Social Sciences › Sociology and Political Science › Psychology of Social Influence
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Opinion Dynamics and Social Influence
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
Keywords
large language models, political persuasion, attitude change, policy attitudes, political polarization, author perceptions, generative AI, assault weapons ban, carbon tax, paid parental leave
MeSH terms
Adult, Artificial Intelligence, Attitude, Female, Humans, Language, Male, Persuasive Communication, Politics, Young Adult

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review