The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Evaluating the persuasive influence of political microtargeting with large language models
Open accessAbstract
Recent advancements in large language models (LLMs) have raised the prospect of scalable, automated, and fine-grained political microtargeting on a scale previously unseen; however, the persuasive influence of microtargeting with LLMs remains unclear. Here, we build a custom web application capable of integrating self-reported demographic and political data into GPT-4 prompts in real-time, facilitating the live creation of unique messages tailored to persuade individual users on four political issues. We then deploy this application in a preregistered randomized control experiment ( n = 8,587) to investigate the extent to which access to individual-level data increases the persuasive influence of GPT-4. Our approach yields two key findings. First, messages generated by GPT-4 were broadly persuasive, in some cases increasing support for an issue stance by up to 12 percentage points. Second, in aggregate, the persuasive impact of microtargeted messages was not statistically different from that of non-microtargeted messages (4.83 vs. 6.20 percentage points, respectively, P = 0.226). These trends hold even when manipulating the type and number of attributes used to tailor the message. These findings suggest—contrary to widespread speculation—that the influence of current LLMs may reside not in their ability to tailor messages to individuals but rather in the persuasiveness of their generic, nontargeted messages. We release our experimental dataset, GPTarget2024 , as an empirical baseline for future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“At the very least, they support dissenting voices on the persuasive impact of either customization or elaboration (11, 28, 74, 76, 77).”
From Testing theories of political persuasion using AI (2025) · cites this paper as [28]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Proceedings of the 25th international conference on Machine learning
Cohen, W. W., McCallum, A., & Roweis, S. T. (2008) · Cited by 5,691
Age of Surveillance Capitalism – The Fight for a Human Future at the New Frontier of Power.
Kendell, D. (2020) · Brock Education Journal · doi:10.26522/brocked.v29i2.849 · Cited by 2,164
Psychological targeting as an effective approach to digital mass persuasion
Matz, S. C., Kosinski, M., Nave, G., & Stillwell, D. J. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1710966114 · Cited by 892 · cited 1×
“This finding builds on prior work suggesting that political advertisements have increased persuasive power when tailored to an individual's position on the introversion–extroversion spectrum (29) or other psychological characteristics (30).”
BackgroundOut of One, Many: Using Language Models to Simulate Human Samples
Argyle, L. P., Busby, E. C., Fulda, N., Gubler, J. R., Rytting, C., & Wingate, D. (2023) · Political Analysis · doi:10.1017/pan.2023.2 · Cited by 783 · cited 4×
“Participants were screened such that all were located in the United States, spoke English as their first language, and were over the age of 18.”
BackgroundInfluentialTaxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764 · cited 1×
“This raises a clear worry: When integrated with existing databases of personal data, LLMs could tailor their messages to appeal to the vulnerabilities and values of specific individuals, potentially reinforcing their existing beliefs, or persuading them to adopt new ones (13–16).”
BackgroundHow is ChatGPT's behavior changing over time?
Chen, L., Zaharia, M., & Zou, J. (2023) · arXiv · doi:10.48550/arxiv.2307.09009 · Cited by 507 · cited 1×
“Finally, some work has suggested that OpenAI made changes to the GPT - 4 model which degraded it’s performance on some tasks during the time this experiment was being conducted (35).”
ResultAll the News That’s Fit to Fabricate: AI-Generated Text as a Tool of Media Misinformation
Kreps, S., McCain, R. M., & Brundage, M. (2022) · Journal of Experimental Political Science · doi:10.1017/xps.2020.37 · Cited by 360 · cited 1×
“These rules were motivated by an attempt to capture the strongest possible persuasive effects: voters are broadly disapproving of microtargeting practices (40–42) and skeptical of AI - authored texts (43).”
BackgroundHuman heuristics for AI-generated language are flawed
Jakesch, M., Hancock, J. T., & Naaman, M. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2208839120 · Cited by 342 · cited 1×
“Across these domains, humans are no longer able to consistently distinguish human and LLM - generated texts (8, 10).”
BackgroundOnline Political Microtargeting: Promises and Threats for Democracy
Zuiderveen Borgesius, F. J., Möller, J., Kruikemeier, S., Ó Fathaigh, R., Irion, K., Dobber, T., Bodo, B., & De Vreese, C. (2018) · Utrecht Law Review · doi:10.18352/ulr.420 · Cited by 340
Zarouali, B., Dobber, T., Pauw, G. D., & de Vreese, C. H. (2020) · Communication Research · doi:10.1177/0093650220961965 · Cited by 249
AI model GPT-3 (dis)informs us better than humans
Spitale, G., Biller-Andorno, N., & Germani, F. (2023) · Science Advances · doi:10.1126/sciadv.adh1850 · Cited by 241
Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good
Wang, X., Shi, W., Kim, R., Oh, Y., Yang, S., Zhang, J., & Yu, Z. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1566 · Cited by 222 · cited 1×
“This raises a clear worry: When integrated with existing databases of personal data, LLMs could tailor their messages to appeal to the vulnerabilities and values of specific individuals, potentially reinforcing their existing beliefs, or persuading them to adopt new ones (13–16).”
BackgroundTargeted Campaign Appeals and the Value of Ambiguity
Hersh, E. D., & Schaffner, B. F. (2013) · The Journal of Politics · doi:10.1017/s0022381613000182 · Cited by 203
Thousands of AI Authors on the Future of AI
Grace, K., Sandkühler, J. F., Stewart, H., Weinstein-Raun, B., Thomas, S., Stein-Perlman, Z., Salvatier, J., Brauner, J., & Korzekwa, R. C. (2025) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.19087 · Cited by 201
Coppock, A., Hill, S. J., & Vavreck, L. (2020) · Science Advances · doi:10.1126/sciadv.abc4046 · Cited by 171
Karinshak, E., Liu, S. X., Park, J. S., & Hancock, J. T. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579592 · Cited by 167 · cited 1×
“Early research suggests that most capable LLMs can directly persuade humans on political issues (5), draft more persuasive public communications than actual government agencies and political communication experts (6, 7), and generate convincing disinfor-mation and fake news (1, 8, 9).”
BackgroundGoldstein, J. A., Sastry, G., Musser, M., DiResta, R., Gentzel, M., & Sedova, K. (2023) · arXiv · doi:10.48550/arxiv.2301.04246 · Cited by 152 · cited 3×
“Crucially, these models are also highly scalable, allowing for essentially limitless production of political messages at an extremely low cost (2, 11).”
BackgroundThe Long-lasting Effects of Newspaper Op-Eds on Public Opinion
Coppock, A., Ekins, E. E., & Kirby, D. (2018) · Quarterly Journal of Political Science · doi:10.1561/100.00016112 · Cited by 140 · cited 1×
“However, a body of research suggests that such divergent reactions are rare, and that persuasive political messages tend to sway people, regardless of their demographic traits, in broadly similar ways (23–25).”
BackgroundHow persuasive is AI-generated propaganda?
Goldstein, J. A., Chao, J., Grossman, S., Stamos, A., & Tomz, M. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae034 · Cited by 133
Kirk, H. R., Vidgen, B., Röttger, P., & Hale, S. A. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00820-y · Cited by 122
Whose Opinions Do Language Models Reflect?
Santurkar, S., Durmus, E., Ladhak, F., Lee, C., Liang, P., & Hashimoto, T. (2023) · arXiv · doi:10.48550/arxiv.2303.17548 · Cited by 114 · cited 5×
“Third, research has suggested that the dominant approach of aligning LLMs with RLHF can have the effect of pushing models to converge to the most common view of a given group, collapsing the diversity of opinions held by, for example, different Democrats, into a single modal response (31).”
BackgroundInfluentialArtificial Intelligence Can Persuade Humans on Political Issues
(2023) · doi:10.21203/rs.3.rs-3238396/v1 · Cited by 110
LLM-generated messages can persuade humans on policy issues
Bai, H., Voelkel, J. G., Muldowney, S., Eichstaedt, J. C., & Willer, R. (2025) · Nature Communications · doi:10.1038/s41467-025-61345-5 · Cited by 110
Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity
Simmons, G. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-srw.40 · Cited by 107
Persuasion in Parallel: How Information Changes Minds about Politics
Coppock, A. (2022) · doi:10.7208/chicago/9780226821832.001.0001 · Cited by 91 · cited 1×
“However, a body of research suggests that such divergent reactions are rare, and that persuasive political messages tend to sway people, regardless of their demographic traits, in broadly similar ways (23–25).”
BackgroundQuantifying the potential persuasive returns to political microtargeting
Tappin, B. M., Wittenberg, C., Hewitt, L. B., Berinsky, A. J., & Rand, D. G. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2216261120 · Cited by 83 · cited 2×
“Attributes were selected for targeting based on a combination of theoretical significance and real - world relevance: their use in previous microtargeting research (28) and also their existence in real - world voter databases a campaign might access (37–39).”
BackgroundTruth, Lies, and Automation: How Language Models Could Change Disinformation
Center for Security and Emerging Technology, Buchanan, B., Lohn, A., Musser, M., & Sedova, K. (2021) · doi:10.51593/2021ca003 · Cited by 74 · cited 2×
“This confluence of events has caused great concern over the potential for AI technologies to disproportionately influence voters’ behavior and electoral outcomes (1, 2).”
BackgroundArtificial Intelligence Can Persuade Humans on Political IssuesPreprint
Bai, H., Voelkel, J. G., Eichstaedt, J. C., & Willer, R. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/stakv · Cited by 58
Ali, M., Sapieżyński, P., Korolova, A., Mislove, A. E., & Rieke, A. (2021) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3437963.3441801 · Cited by 52 · cited 2×
“Further, many documented real - life microtarget-ing operations have attempted to remain covert (17).”
BackgroundHackenburg, K., Ibrahim, L., Tappin, B. M., & Manos Tsakiris, A. (2025) · AI & Society · doi:10.1007/s00146-025-02464-x · Cited by 44 · cited 1×
“Early research suggests that most capable LLMs can directly persuade humans on political issues (5), draft more persuasive public communications than actual government agencies and political communication experts (6, 7), and generate convincing disinfor-mation and fake news (1, 8, 9).”
BackgroundSelf-citationTargeted Issue Messages and Voting Behavior
Endres, K. (2019) · American Politics Research · doi:10.1177/1532673x19875694 · Cited by 38 · cited 1×
“Several studies on microtargeting found inconclusive or underwhelming evidence supporting its effects (26, 27).”
BackgroundAd Delivery Algorithms: The Hidden Arbiters of Political Messaging
Ali, M., Sapiezynski, P., Korolova, A., Mislove, A., & Rieke, A. (2019) · arXiv · doi:10.48550/arxiv.1912.04255 · Cited by 30 · cited 2×
“Further, many documented real - life microtarget-ing operations have attempted to remain covert (17).”
BackgroundAmericans Roundly Reject Tailored Political Advertising
Turow, J., Carpini, M. X. D., Draper, N. A., & Howard–Williams, R. (2012) · ScholarlyCommons (University of Pennsylvania) · Cited by 29
Hackenburg, K., Ibrahim, L., Tappin, B. M., et al. (2023) · doi:10.31219/osf.io/ey8db · Cited by 17
Self-citationPolitical microtargeting: Towards a pragmatic approach
Aagaard, P., & Marthedal, S. (2023) · Internet Policy Review · doi:10.14763/2023.1.1690 · Cited by 10 · cited 1×
“Attributes were selected for targeting based on a combination of theoretical significance and real - world relevance: their use in previous microtargeting research (28) and also their existence in real - world voter databases a campaign might access (37–39).”
BackgroundCommunityLM: Probing Partisan Worldviews from Language ModelsPreprint
Jiang, H., Beeferman, D., Roy, B., & Roy, D. K. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.07065 · Cited by 9 · cited 3×
“Existing work has shown that fine - tuning can increase model alignment for specific political use cases (33) and it remains unexplored in the micro-targeting domain.”
BackgroundMethodKoto, F., Lau, J. H., & Baldwin, T. (2022) · Workshop on e-Commerce and NLP (ECNLP) · doi:10.18653/v1/2022.ecnlp-1.27 · Cited by 8
Tiwari, A., Saha, S., Sengupta, S., Maitra, A., Ramnani, R., & Bhattacharyya, P. (2022) · Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (AACL-IJCNLP) · doi:10.18653/v1/2022.aacl-main.76 · Cited by 6
Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity
Simmons, G. (2022) · arXiv · doi:10.48550/arxiv.2209.12106 · cited 1×
“For instance, when prompted to adopt a liberal or conservative political identity, these models produce text reflecting the respective moral biases (32).”
BackgroundThousands of AI Authors on the Future of AI
Grace, K., Stewart, H., Sandkühler, J. F., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.02843 · cited 1×
“However, even as researchers and industry leaders list model persuasiveness as a primary safety concern (3, 4), understandings of model capabilities in this domain remain nascent.”
Background
Research works printed but not linked: 5
- M. Harrison Dupré, Sam Altman warns that AI is learning ‘Superhuman Persuasion’. Futurism (2023), https://futurism.com/sam-altman-ai-superhuman-persuasion. Accessed 10 February 2024.
- J. Goldstein, G. Sastry, The coming age of AI-powered propaganda. Foreign Affairs (2023).
- BBC, Vote Leave’s Targeted Brexit Ads Released by Facebook (BBC News, 2018). Accessed 13 January 2024.
- M. Scott, Cambridge Analytica helped ‘cheat’ Brexit vote and US election, claims whistleblower. POLITICO, 27 March 2018.
- Open Rights Group, Public are kept in the dark over data driven political campaigning, poll finds. Open Rights Group, 10 January 2020. https://www.openrightsgroup.org/press-releases/public-are-kept-in-the-dark-over-data-driven-political-campaigning-poll-finds/. Accessed 23 January 2024.
Cited by
Sorted by citations
Durably reducing conspiracy beliefs through dialogues with AI
Costello, T. H., Pennycook, G., & Rand, D. G. (2024) · Science · doi:10.1126/science.adq1814 · Cited by 354
On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial
Salvi, F., Ribeiro, M. H., Gallotti, R., & West, R. (2024) · doi:10.21203/rs.3.rs-4429707/v1 · Cited by 285
On the conversational persuasiveness of GPT-4
Salvi, F., Ribeiro, M. H., Gallotti, R., & West, R. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02194-6 · Cited by 480
“After the debate, in Other research has focused on personalization, observing conse-quential yet non-unanimous evidence about the impact of LLMs on microtargeting 47–49 .”
Using natural language processing to analyse text data in behavioural science
Feuerriegel, S., Maarouf, A., Bär, D., Geissler, D., Schweisthal, J., Pröllochs, N., Robertson, C. E., Rathje, S., Hartmann, J., Mohammad, S. M., et al. (2025) · Nature Reviews Psychology · doi:10.1038/s44159-024-00392-z · Cited by 140
“For example, studies published in the past 2 years have used NLP to generate persuasive messages tailored to specific personality profiles 139 and to build therapeutic chatbots 140 .”
Why human–AI relationships need socioaffective alignment
Kirk, H. R., Gabriel, I., Summerfield, C., Vidgen, B., & Hale, S. A. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-04532-5 · Cited by 146
Metadata
- Authors
- Hackenburg, K., & Margetts, H.
- Affiliations
- University of Oxford (GB)
- Corresponding author
- Hackenburg, Kobi
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2024-06-07
- Volume
- 121
- Issue
- 24
- Pages
- e2403116121-e2403116121
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC11181035
- OpenAlex
- W4399426804
- Cited by
- 114
- References
- 33
- Citations by year
- 2026: 47 · 2025: 54 · 2024: 12
- FWCI
- 94.38
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- OSF Preprints (OSF Preprints) (submittedVersion)
- OSF Preprints (OSF Preprints) (submittedVersion, cc-by)
- Topics
- Social Sciences › Communication › Social Media and Politics
- Social Sciences › Sociology and Political Science › Media Influence and Politics
- Social Sciences › Political Science and International Relations › Electoral Systems and Political Participation
- Keywords
- political microtargeting, large language models, GPT-4, persuasion, microdata, message tailoring, political communication
- MeSH terms
- Humans, Language, Persuasive Communication, Politics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
