Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 84 of the 88 references it cites were found in a registry or an index.

On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial

Open access

Preprint — published as 10.1038/s41562-025-02194-6 · Paper page

Abstract

Can large language models (LLMs) create tailor-made, convincing arguments to promote false or misleading narratives online? Early work has found that LLMs can generate content perceived on par with, or even more persuasive than, human-written messages. However, there is still limited evidence regarding LLMs' persuasive capabilities in direct conversations with humans—the scenario these models are usually deployed at. In this pre-registered study, we analyze the power of AI-driven persuasion in a controlled, harmless setting. To this end, we created a web-based platform where human participants engaged in short, multi-round debates with either human or LLM opponents. Each participant was randomly assigned to one of four treatment conditions in a two-by-two factorial design: (1) the conversation partner was either another human or an LLM; (2) the conversation partner either had or did not have access to basic sociodemographic information about their opponent (and thus arguments could be personalized). We find that 64.4% of the time, personalized LLM debaters were more persuasive than humans, given that they were not equally persuasive (81.2% relative increase in the odds of higher post-debate agreement; p

Overview

Integrity signals

Registry

Crossref · 10.21203/rs.3.rs-4429707/v1

Cites

84 of 84 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 88 references

Cited by

See all 479

Metadata

Authors
Salvi, F., Ribeiro, M. H., Gallotti, R., & West, R.
Affiliations
  • Fondazione Bruno Kessler (IT)
  • Princeton University (US)
  • École Polytechnique Fédérale de Lausanne (CH)
Corresponding author
Salvi, Francesco
Venue
Nature Human Behaviour
Published
2024-06-05
Volume
9
Issue
8
Pages
1645-1653
ISSN
2397-3374 (ISSN-L)
OpenAlex
W4410496075
Cited by
285
References
88
Citations by year
2026: 179 · 2025: 90 · 2024: 16
FWCI
276.48
Type
journal article
Language
en
Open access
Locations
  • Nature Human Behaviour (publishedVersion, cc-by)
  • Research Square (acceptedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
  • arXiv (Cornell University)
Funders
  • National Science Foundation
  • National Research Centre
  • European Commission (101070190, H2020, CN00000013, 952215)
  • École Polytechnique Fédérale de Lausanne
  • Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (211379, 185043, TMSGI2, 200021)
  • Horizon 2020 Framework Programme (952215, CN00000013)
  • HORIZON EUROPE Framework Programme (952215, 101070190)
  • NextGenerationEU (CN00000013, 952215)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Psychology › Social Psychology › Psychology of Social Influence
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
Keywords
AI persuasion, GPT-4, large language models, sociodemographic data, opinion change, AI misuse
MeSH terms
Adult, Female, Humans, Language, Male, Persuasive Communication, Young Adult

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial. https://app.cochat.ai/papers/10.21203/rs.3.rs-4429707/v1
The paper
Salvi, F., Ribeiro, M. H., Gallotti, R., & West, R. (2024). On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial. Nature Human Behaviour. https://doi.org/10.21203/rs.3.rs-4429707/v1

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.