The DOI matches its Crossref record; 84 of the 88 references it cites were found in a registry or an index.
On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial
Open accessPreprint — published as 10.1038/s41562-025-02194-6 · Paper page
Abstract
Can large language models (LLMs) create tailor-made, convincing arguments to promote false or misleading narratives online? Early work has found that LLMs can generate content perceived on par with, or even more persuasive than, human-written messages. However, there is still limited evidence regarding LLMs' persuasive capabilities in direct conversations with humans—the scenario these models are usually deployed at. In this pre-registered study, we analyze the power of AI-driven persuasion in a controlled, harmless setting. To this end, we created a web-based platform where human participants engaged in short, multi-round debates with either human or LLM opponents. Each participant was randomly assigned to one of four treatment conditions in a two-by-two factorial design: (1) the conversation partner was either another human or an LLM; (2) the conversation partner either had or did not have access to basic sociodemographic information about their opponent (and thus arguments could be personalized). We find that 64.4% of the time, personalized LLM debaters were more persuasive than humans, given that they were not equally persuasive (81.2% relative increase in the odds of higher post-debate agreement; p
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“But users increasingly cannot tell the difference between human writing and LLM writing when presented with text (30–32), and between a human interlocutor or a contemporary chatbot when in interaction (17, 26).”
From The benefits and dangers of anthropomorphic conversational agents (2025) · cites this paper as [31]
In the citing paper
“Newer studies that use the latest generation of models, such as GPT-4, demonstrate that LLMs can exceed average human skill. One experiment found that “LLMs significantly outperform human participants across every topic and demographic, exhibiting a high level of persuasiveness” (31). This was particularly true when the LLM was given personal information about the user to tailor its conversational”
From The benefits and dangers of anthropomorphic conversational agents (2025) · cites this paper as [31]
In the citing paper
“These models are increasingly achieving or surpassing human performance benchmarks in various domains, including medical diagnostic dialogue28, persuasive communication29, theory of mind30, making people feel heard31, responding to relationship issues32 and helping people reframe negative situations to reduce negative emotions33.”
From “It happened to be the perfect thing”: experiences of generative AI chatbots for mental health (2024) · cites this paper as [29]
Citations
Cites
Longitudinal data analysis using generalized linear models
LIANG, K.-Y., & ZEGER, S. L. (1986) · Biometrika · doi:10.1093/biomet/73.1.13 · Cited by 18,550
Flesch, R. (1948) · Journal of Applied Psychology · doi:10.1037/h0057532 · Cited by 5,391
Regression Models for Ordinal Data
McCullagh, P. (1980) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1980.tb01109.x · Cited by 4,453
Private traits and attributes are predictable from digital records of human behavior
Kosinski, M., Stillwell, D., & Graepel, T. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1218772110 · Cited by 2,829
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Assessing proportionality in the proportional odds model for ordinal logistic regression.
Brant, R. (1990) · Biometrics · doi:10.2307/2532457 · Cited by 1,864
Data quality of platforms and panels for online behavioral research
Peer, E., Rothschild, D., Gordon, A., Evernden, Z., & Damer, E. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01694-3 · Cited by 1,718
Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach
Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Dziurzynski, L., Ramones, S. M., Agrawal, M., Shah, A., Kosinski, M., Stillwell, D., Seligman, M. E. P., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0073791 · Cited by 1,700
Douglas, B. D., Ewell, P. J., & Brauer, M. (2023) · PLOS ONE · doi:10.1371/journal.pone.0279720 · Cited by 1,532
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.12712 · Cited by 1,526
Self-citation
Cited by
“It happened to be the perfect thing”: experiences of generative AI chatbots for mental health
Siddals, S., Torous, J., & Coxon, A. (2024) · npj Mental Health Research · doi:10.1038/s44184-024-00097-4 · Cited by 241
Terwiesch, C., Meincke, L., Girotra, K., Mollick, E., Nave, G., & Ulrich, K. T. (2026) · Production and Operations Management · doi:10.1177/10591478261474243 · Cited by 234
Li, W., Peng, Y. J., Zhang, M., Ding, L., Hu, H., & Shen, L. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3628666 · Cited by 180
The levers of political persuasion with conversational artificial intelligence
Hackenburg, K., Tappin, B. M., Hewitt, L., Saunders, E., Black, S., Lin, H., Fist, C., Margetts, H. Z., Rand, D. G., & Summerfield, C. (2025) · Science · doi:10.1126/science.aea3884 · Cited by 183
The benefits and dangers of anthropomorphic conversational agents
Peter, S., Riemer, K., & West, J. D. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2415898122 · Cited by 110
Metadata
- Authors
- Salvi, F., Ribeiro, M. H., Gallotti, R., & West, R.
- Affiliations
- Fondazione Bruno Kessler (IT)
- Princeton University (US)
- École Polytechnique Fédérale de Lausanne (CH)
- Corresponding author
- Salvi, Francesco
- Venue
- Nature Human Behaviour
- Published
- 2024-06-05
- Volume
- 9
- Issue
- 8
- Pages
- 1645-1653
- ISSN
- 2397-3374 (ISSN-L)
- PMC
- PMC12367540
- OpenAlex
- W4410496075
- Cited by
- 285
- References
- 88
- Citations by year
- 2026: 179 · 2025: 90 · 2024: 16
- FWCI
- 276.48
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Nature Human Behaviour (publishedVersion, cc-by)
- Research Square (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University)
- Funders
- National Science Foundation
- National Research Centre
- European Commission (101070190, H2020, CN00000013, 952215)
- École Polytechnique Fédérale de Lausanne
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (211379, 185043, TMSGI2, 200021)
- Horizon 2020 Framework Programme (952215, CN00000013)
- HORIZON EUROPE Framework Programme (952215, 101070190)
- NextGenerationEU (CN00000013, 952215)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Psychology › Social Psychology › Psychology of Social Influence
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- AI persuasion, GPT-4, large language models, sociodemographic data, opinion change, AI misuse
- MeSH terms
- Adult, Female, Humans, Language, Male, Persuasive Communication, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial. https://app.cochat.ai/papers/10.21203/rs.3.rs-4429707/v1
- The paper
- Salvi, F., Ribeiro, M. H., Gallotti, R., & West, R. (2024). On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial. Nature Human Behaviour. https://doi.org/10.21203/rs.3.rs-4429707/v1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.