CoChat
Sign Up
Cited by 25Electoral Systems and Political ParticipationOpinion Dynamics and Social InfluenceSocial and Intergroup Psychology

Testing theories of political persuasion using AI

Open access
Argyle, Lisa P., Busby, Ethan C., Gubler, Joshua R., Lyman, Alex, Olcott, Justin, Pond, Jackson, Wingate, David·2025·Proceedings of the National Academy of Sciences
DOI 10.1073/pnas.2412815122PMC PMC12067286
Open this paper in CoChat — free accountOpen PDF

Abstract

Despite its importance to society and many decades of research, key questions about the social and psychological processes of political persuasion remain unanswered, often due to data limitations. We propose that AI tools, specifically generative large language models (LLMs), can be used to address these limitations, offering important advantages in the study of political persuasion. In two preregistered online survey experiments, we demonstrate the potential of generative AI as a tool to study persuasion and provide important insights about the psychological and communicative processes that lead to increased persuasion. Specifically, we test the effects of four AI-generated counterattitudinal persuasive strategies, designed to test the effectiveness of messages that include customization (writing messages based on a receiver’s personal traits and beliefs), and elaboration (increased psychological engagement with the argument through interaction). We find that all four types of persuasive AI produce significant attitude change relative to the control and shift vote support for candidates espousing views consistent with the treatments. However, we do not find evidence that message customization via microtargeting or cognitive elaboration through interaction with the AI have much more persuasive effect than a single generic message. These findings have implications for different theories of persuasion, which we discuss. Finally, we find that although persuasive messages are able to moderate some people’s attitudes, they have inconsistent and weaker effects on the democratic reciprocity people grant to their political opponents. This suggests that attitude moderation (ideological depolarization) does not necessarily lead to increased democratic tolerance or decreased affective polarization.

AI snapshot

Written from the paper’s full text.

TL;DR

Two preregistered survey experiments used GPT-4 to test whether microtargeted customization and interaction-induced elaboration make political persuasion more effective than generic LLM messages. Across immigration and K-12 education topics, LLM messages shifted attitudes by about 2.5–4 percentage points, but customization and elaboration did not clearly outperform a one-shot generic message.

Key findings

  • In Study 1 (immigration, n = 1,862) and Study 2 (K-12 education, n = 1,819), all four persuasive treatments produced significant attitude change averaging approximately 2.5 to 4 percentage points, except interactive motivational interviewing on immigration.
  • The paper finds no support for its second hypothesis: one-shot generic, one-shot microtargeted, and interactive direct persuasion conditions generated persuasive effects of similar magnitude; motivational interviewing often had the weakest and least consistent effects. The design was well-powered to detect differences of 2 percentage points or more, with post hoc power sufficient for effects larger than about 1.5 percentage points but not reliable for 1 to 1.5 percentage points.
  • Vote-support effects were of similar magnitude to attitude change. Democratic reciprocity showed smaller and more contingent effects; the only conditions with a significant increase were interactive chat conditions on K-12 curriculum.
  • Embedding analyses indicated that full LLM transcripts differed across treatment conditions, while GPT-4o summaries of the core arguments overlapped strongly across conditions, supporting treatment internal validity.

Methods

  • Two preregistered Qualtrics survey experiments fielded in May 2024 through CloudResearch, with samples roughly matched to the US census on gender, age, and race/ethnicity and balanced between Republicans and Democrats; respondents were paid $3 and provided informed consent.
  • Respondents were randomly assigned to one of six conditions: two controls (interactive board-game chat and one-shot board-game message) and four treatments—one-shot generic message, one-shot microtargeted message, interactive direct persuasion (6 turns), and interactive motivational interviewing (6 turns). GPT-4 generated all treatment content on immigration (Study 1) or K-12 education (Study 2).
  • Analyses used OLS marginal effects relative to pooled controls for pre/post policy attitudes, post-only vote likelihood, and democratic reciprocity, plus machine-learning embeddings and GPT-4o one-sentence summaries to compare treatment style versus argument content.

Limitations

  • Stated: the studies focus on specific operationalizations—message tailoring based on demographic and political characteristics and chat-based conversation as a route to elaboration—so other kinds of customization and other routes to elaboration are not tested.
  • Stated: interactions were short and one-time, lasting seconds to a few minutes; effects might shrink in real-world contexts with more distraction, and effects of repeated or longer exposure are unclear.
  • Stated: due to a programming error in Study 1, all neutral pretreatment respondents received a pro-increased-immigration message; the authors report this did not affect results. Evident: the samples were opt-in CloudResearch panels constructed to roughly match census and party balance, not probability samples, which limits generalizability to broader populations and real-world persuasion settings.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1073/pnas.2412815122

Cites

79 of 79 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

4 of 5 cited sources checked · 1 supported · 0 flagged · accuracy not yet measured

  • Interpretationown data

    The findings support dissenting voices on the persuasive impact of either customization or elaboration.

    “At the very least, they support dissenting voices on the persuasive impact of either customization or elaboration (11, 28, 74, 76, 77).”
    Quote in the paperSubstantial claim3 of 3 sources checked
    • cites [11] B. M. Tappin, C. Wittenberg, L. B. Hewitt, A. J. Berinsky, D. G. Rand, Quantifying the potential persuasive returns to political microtargeting. Proc. Natl. Acad. Sci. U.S.A. 120, e2216261120 (2023). 37307486 PMC10288628 source

      The Discussion (page unavailable) questions sweeping claims about customization: microtargeting’s persuasive advantage was context dependent, and targeting by more personal characteristics did not amplify it. But the paper also reports sizable persuasive gains from microtargeting in one setting, and our copy does not address elaboration. It therefore supports a qualified dissenting view of customization, not the full claim. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [28] K. Hackenburg, H. Margetts, Evaluating the persuasive influence of political microtargeting with large language models. Proc. Natl. Acad. Sci. U.S.A. 121, e2403116121 (2024). 38848300 PMC11181035 source

      The Results report no persuasive advantage for accurately microtargeted GPT-4 messages over nontargeted messages, supporting skepticism about customization in this setting. The Experimental Materials section says message length was kept around 200 words; the paper does not test whether greater elaboration changes persuasive impact. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
    • cites [74] J. A. Goldstein, J. Chao, S. Grossman, A. Stamos, M. Tomz, How persuasive is AI-generated propaganda? PNAS Nexus 3, pgae034 (2024). 38380055 PMC10878360 source

      The Results sections on persuasiveness and human–machine teaming assess GPT-3 propaganda and the effects of editing prompts or curating outputs. They do not assess whether customizing messages for recipients or elaborating messages changes their persuasive impact (searched: customization, tailored messages, elaboration, and message length). (LLM, conf=0.74)

      Experimental — accuracy not yet measured
  • Findingown data

    The different persuasive approaches do not appear to interact with the topic to produce more persuasion in some cases than others.

    “These results show no significant differences between the treatment conditions within any subgroup of the study, meaning that the different persuasive approaches do not appear to interact with the topic to produce more persuasion in some cases than others.”
    Quote in the paperSubstantial claim
  • Findingown data

    LLM-generated persuasive messages may not move fundamental democratic attitudes about those with whom they disagree.

    “Here, we demonstrate the inverse: that LLMs can generate persuasive messages that move individual issue positions in more moderate political directions, but that this may not move fundamental democratic attitudes about those with whom they disagree.”
    Quote in the paperSubstantial claim
  • Findingown data

    LLM-generated persuasive messages can move people's policy positions in more moderate political directions.

    “Here, we demonstrate the inverse: that LLMs can generate persuasive messages that move individual issue positions in more moderate political directions, but that this may not move fundamental democratic attitudes about those with whom they disagree.”
    • Also stated in: § Discussion and Conclusion · hedged
      “Our results suggest LLM-generated persuasive texts can indeed move people’s policy positions but that customization and elaboration as features of these messages may not provide much added persuasive impact over a one-shot generic message generated by the LLM.”
    Quote in the paperSubstantial claim
  • Findingown data

    There were no significant differences between the treatment conditions within any subgroup of the study.

    “These results show no significant differences between the treatment conditions within any subgroup of the study, meaning that the different persuasive approaches do not appear to interact with the topic to produce more persuasion in some cases than others.”
    Quote in the paperSubstantial claim
  • May combine two claims. 7
  • Classified as probably not a substantial claim. 12

  • May combine two claims.

    Premisecitation

    In prior research by the authors, LLMs improved the tone of politically fraught conversation, leading to improvement in participants' democratic reciprocity that was not dependent on a change in their policy attitudes.

    “In prior research, our team demonstrated that LLMs can be used to improve the tone of politically fraught conversation, leading to improvement in participants’ democratic reciprocity that is not dependent on a change in their policy attitudes (7).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [7] L. P. Argyle, Leveraging AI for democratic discourse: Chat interventions can improve online political conversations at scale. Proc. Natl. Acad. Sci. U.S.A. 120, e2311627120 (2023). 37788311 PMC10576030 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The LLM was more persuasive in persuading those with liberal policy positions to a conservative position on K-12 education than on immigration.

    “This is most apparent in the second panel, where the LLM was more persuasive in persuading those with liberal policy positions to a conservative position on K-12 education than on immigration.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Customization and elaboration as features of LLM-generated persuasive messages may not provide much added persuasive impact over a one-shot generic message generated by the LLM; their persuasive effects appear indistinguishable from a more generic persuasive appeal.

    “Our results suggest LLM-generated persuasive texts can indeed move people’s policy positions but that customization and elaboration as features of these messages may not provide much added persuasive impact over a one-shot generic message generated by the LLM.”
    • Also stated in: § Discussion and Conclusion · hedged
      “We fail to find the evidence that these theories would have predicted, as the persuasive effects of more elaborative discussions and customized messages appear indistinguishable from a more generic persuasive appeal.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    There are some intercept-shift differences for some subgroups or on some topics.

    “However, there are some intercept-shift differences for some of the subgroups or on some of the topics.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data

    LLMs are a viable tool to systematically explore many of the bounds on the effects of customization and elaboration.

    “Regardless, our results suggest some need for additional theorizing, caveats, or bounds, on the effects of customization and elaboration, and suggest LLMs as a viable tool to systematically explore many of these bounds.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    LLMs can significantly advance the study of critical theories of political persuasion when implemented thoughtfully.

    “Our conclusion here is not that LLMs provide a replacement for every conceivable persuasive tactic—for example, discussions among close personal connections—but that they can significantly advance the study of critical theories of political persuasion when implemented thoughtfully.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Respondents were randomly assigned to one of six conditions.

    “At this point, respondents were randomly assigned to one of six conditions.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data

    The results suggest some need for additional theorizing, caveats, or bounds on the effects of customization and elaboration.

    “Regardless, our results suggest some need for additional theorizing, caveats, or bounds, on the effects of customization and elaboration, and suggest LLMs as a viable tool to systematically explore many of these bounds.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The results cast doubt on specific approaches to elaboration and customization (message tailoring based on demographic and political characteristics and chat-based conversation as a route to elaboration) while leaving room for other approaches to be more important in persuasion.

    “These results cast doubt on these specific approaches to elaboration and customization while leaving room for other approaches to be more important in persuasion.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The sample contained an even balance of Republicans and Democrats (as well as a smaller amount of political Independents).

    “We also ensured that the sample contained an even balance of Republicans and Democrats (as well as a smaller amount of political Independents).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Even though people may come to find more common ground on the core content of policy issues, that does not guarantee they will be willing to engage in deliberative exchange or extend respect to those who hold differing views.

    “Thus, even though people may come to find more common ground on the core content of the policy issues, that does not guarantee that they will be willing to engage in deliberative exchange or extend respect to those who hold differing views.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The two control groups were pooled to form the baseline condition for all subsequent analyses.

    “As such, in our analyses, we pool respondents from these two control groups together to form the baseline condition for all subsequent analyses:”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationreasoning

    Absent an LLM, such a task would require significant additional logistical and human costs, and would introduce other potential confounds into the experimental design.

    “Absent an LLM, such a task would require significant additional logistical and human costs, and would introduce other potential confounds into the experimental design.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    In study 2, the programming error was corrected and those with neutral pretreatment policy views were randomly assigned to either pro- or anti-school regulation treatments.

    “In study 2, this error was corrected and those with neutral pretreatment policy views were randomly assigned to either pro- or anti-school regulation treatments.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    In study 1, due to a programming error, all respondents with a neutral pretreatment position received a message supporting increased immigration.

    “Due to a programming error, in study 1, all respondents with a neutral pretreatment position received a message supporting increased immigration.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Similar conversation lengths were observed across the two studies.

    “We observed similar conversation lengths across the two studies.”
    Quote in the paperNot a substantial claim
16 of 16

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Abstract says all treatments produced significant attitude change despite a reported exception

    Minor

    The abstract says all treatments produced significant attitude change, whereas the results identify interactive motivational interviewing in the immigration experiment as an exception.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. A. Grant, Think Again: The Power of Knowing What You Don’t Know (Penguin, New York, 2023).
  2. M. P. Fiorina, S. J. Abrams, J. C. Pope, Culture War? (Longman, 2011).

Cited by

Sorted by citations

Related

Authors
Argyle, L. P., Busby, E. C., Gubler, J. R., Lyman, A., Olcott, J., Pond, J., & Wingate, D.
Affiliations
Brigham Young University (US)
Corresponding author
Argyle, Lisa P.
ORCID
Venue
Proceedings of the National Academy of Sciences
Publisher
National Academy of Sciences
Published
2025-05-02
Volume
122
Issue
18
Pages
e2412815122-e2412815122
ISSN
  • 0027-8424 (ISSN-L)
  • 1091-6490
DOI
10.1073/pnas.2412815122
PMC
PMC12067286
OpenAlex
W4410030884
Cited by
25
References
71
Citations by year
2026: 15 · 2025: 11
FWCI
105.34
Type
journal article
Language
en
Open access
Licence
Locations
  • Proceedings of the National Academy of Sciences (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (submittedVersion, cc-by)
Topics
  • Social Sciences › Political Science and International Relations › Electoral Systems and Political Participation
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Opinion Dynamics and Social Influence
  • Social Sciences › Sociology and Political Science › Social and Intergroup Psychology
Keywords
political persuasion, generative large language models, microtargeting, cognitive elaboration, attitude change, vote choice, affective polarization
MeSH terms
Adult, Artificial Intelligence, Attitude, Female, Humans, Male, Persuasive Communication

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review