The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index; none of the 8 sources its claims cite is checked for support yet.
Dialogues with large language models reduce conspiracy beliefs even when the AI is perceived as human
Open accessAbstract
Although conspiracy beliefs are often viewed as resistant to correction, recent evidence shows that personalized, fact-based dialogues with a large language model (LLM) can reduce them. Is this effect driven by the debunking facts and evidence, or does it rely on the messenger being an AI? In other words, would the same message be equally effective if delivered by a human? To answer this question, we conducted a preregistered experiment (N = 955) in which participants reported either a conspiracy belief or a nonconspiratorial but epistemically unwarranted belief and interacted with a LLM that argued against that belief using facts and evidence. We randomized whether the debunking LLM was characterized as an AI tool or a human expert and whether the model used human-like conversational tone. The conversations significantly reduced participants' confidence in both conspiracies and epistemically unwarranted beliefs, with no significant differences across conditions. Thus, AI persuasion is not reliant on the messenger being an AI model: it succeeds by generating compelling messages.
Overview
CoChat Summary
TL;DR
In a preregistered experiment (N = 955), two-round GPT-4o dialogues reduced confidence in conspiracy and other epistemically unwarranted beliefs. The debunking effect did not differ whether the model was labeled an AI or an expert, or whether it used a human-like tone.
Key findings
- Across all conditions, the conversation significantly reduced belief (all ps < 0.018). The decrease was larger for conspiracy beliefs (10 points, 11.81% decrease) than for general “epistemically weak” beliefs (5 points, 5.96% decrease); b = 5.11, 95% CI (2.75, 7.48), P < 0.001.
- No significant effects of debunker identification (AI vs. Expert), prompt type (Neutral vs. Human-like), or their interactions (all ps > 0.25). A Bayes factor analysis favored the null model, BF01 = 518,181.1 ± 6.43%.
- Among participants told they were conversing with an expert, 45% used language explicitly indicating they thought the interlocutor was human, while 21% suspected they were talking to an AI. Controlling for AI detection or excluding detectors still yielded no significant differences across conditions.
- Exploratory NLP analyses found participants in the AI condition used less lexically diverse language than those in the Expert condition (type–token ratio: b = −0.02, P = 0.032; unique word types: b = −4.58, P = 0.032), with no significant differences for syntactic complexity, epistemic markers, or argument connectives (all P > 0.084).
Methods
- Preregistered experiment; 8,274 individuals were recruited from Lucid’s participant pool, 1,997 completed the full study, and the final sample included 955 participants (506 in the conspiracy-belief condition, 449 in the general epistemically suspect belief condition) after preregistered exclusions.
- Participants reported either a conspiracy belief or a nonconspiratorial but epistemically unwarranted belief, rated confidence (0–100), had a two-round chat with GPT-4o prompted to reduce confidence in that belief, and then re-rated confidence. They were randomized to debunker identification (AI vs. Expert) and prompt type (Neutral vs. Human-like).
- Main analysis used linear regression predicting postconversation belief with contrast-coded predictors for debunker identification, prompt type, belief type, all two-way and three-way interactions, and a control for preconversation belief; post hoc Bayes factor and exploratory NLP analyses were also reported.
Limitations
- Stated: the model was trained primarily on large-scale Internet data dominated by English and Western sources, likely shaping its tone, assumptions, and argumentative style; debunking may not generalize across cultural contexts.
- Stated: analyses controlling for AI detection or excluding AI detectors condition on post-treatment variables, creating causal inference caveats.
- Stated/evident: attrition was significantly higher in the Expert condition (47.50%) than the AI condition (42.69%; t(3629) = 2.99, P = 0.003), though results remained robust under conservative imputation; the final sample came from Lucid after exclusions, which may limit generalizability (evident, not framed as a limitation by the authors).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures

Verification
Claims the paper makes
0 Valid · 0 Not valid · 6 Not checked · 22 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, belief change did not significantly differ between participants who attributed the speaker's identity to an AI vs. a human.”
§ Dialogues with large language models reduce conspiracy beliefs even when the AI is perceived as human > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, a post hoc Bayes Factor analysis comparing the full model with a simpler model including only preconversation belief and belief type yielded a BF01 of 518,181.1 ± 6.43%, strongly supporting the null hypothesis of no effect of debunker identification or prompt type.”
§ Dialogues with large language models reduce conspiracy beliefs even when the AI is perceived as human > Results
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“although we do find that the decrease in belief certainty was significantly greater among participants who shared a conspiracy theory (10 points, 11.81% decrease) compared to those who shared a general “epistemically weak” belief (5 points, 5.96% decrease); b = 5.11, 95% CI (2.75, 7.48), P < 0.001.”
§ Dialogues with large language models reduce conspiracy beliefs even when the AI is perceived as human > Results
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The conversations significantly reduced participants' confidence in both conspiracies and epistemically unwarranted beliefs, with no significant differences across conditions.”
- Also stated in: § Results · plain
“Across all conditions, the conversation significantly reduced participants' belief (all ps < 0.018, based on estimated marginal means from the regression); see Fig. 1.”
- Also stated in: § Discussion · plain
“Even entrenched beliefs shifted when participants were presented with relevant counterevidence, consistent with recent findings (3, 4).”
§ Dialogues with large language models reduce conspiracy beliefs even when the AI is perceived as human > Abstract
plainstates a causen = 955 ·- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Results · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, re-running the main regression while controlling for AI detection or excluding those who detected the AI, continue to find no significant differences across conditions.”
§ Dialogues with large language models reduce conspiracy beliefs even when the AI is perceived as human > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
10 more claims
Create a free account to see all 28 claims and every source verdict.
- Probably not a substantial claim. 7
- It may state two separate claims as one. 7
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Consistentt(3629) = 2.99reported p = .003 · recomputed p = .003
“t(3629) = 2.99, p = .003”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Self-check results appear here once an independent check has confirmed them.
Citations
Cites
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Conspiracy theories: Causes and cures
Sunstein, C. R., & Vermeule, A. (2009) · Journal of Political Philosophy · doi:10.1111/j.1467-9760.2008.00325.x · Cited by 1,050 · cited 2×
“Beliefs in unfounded conspiracy theories are widely assumed to be essentially impervious to evidence-based correction (1).”
It's only a computer: virtual humans increase willingness to disclose
Lucas, G. M., Gratch, J., King, A., & Morency, L.-P. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.04.043 · Cited by 977 · cited 1×
“Further, when the topic is sensitive (e.g. vaccination, political beliefs) participants may be reluctant to admit doubt or change to another human (6).”
Hill, J., Randolph Ford, W., & Farreras, I. G. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.02.026 · Cited by 856
Durably reducing conspiracy beliefs through dialogues with AI
Costello, T. H., Pennycook, G., & Rand, D. G. (2024) · Science · doi:10.1126/science.adq1814 · Cited by 354
Self-citationLobato, E., Mendoza, J., Sims, V., & Chin, M. (2014) · Applied Cognitive Psychology · doi:10.1002/acp.3042 · Cited by 332
Douglas, K. M., & Sutton, R. M. (2023) · Annual Review of Psychology · doi:10.1146/annurev-psych-032420-031329 · Cited by 313
AI can help people feel heard, but an AI label diminishes this impact
Yin, Y., Jia, N., & Wakslak, C. J. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2319112121 · Cited by 247 · cited 1×
“…enhance trust and persuasive impact (10), with some studies finding that AI agents are perceived as neutral, more impartial or more competent (5, 11), while others report a general preference for human communicators — often seen as more convincing, emotionally at-tuned, or socially aware (12, 13).”
The benefits and dangers of anthropomorphic conversational agents
Peter, S., Riemer, K., & West, J. D. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2415898122 · Cited by 110 · cited 1×
“Prior work suggests that anthropomorphic cues and source framing can enhance trust and persuasive impact (10), with some studies finding that AI agents are perceived as neutral, more impartial or more competent (5, 11), while others report a general preference for human communicators — often seen…”
Is artificial intelligence more persuasive than humans? A meta-analysis
Huang, G., & Wang, S. (2023) · Journal of Communication · doi:10.1093/joc/jqad024 · Cited by 95 · cited 1×
“This aligns with a recent meta-analysis showing no consistent advantage of humans over AI sources in shaping attitudes and behaviors (14).”
Research works printed but not linked: 1
- Sundar, 2008, Digital Media, youth, and credibility, 73
Cited by
Persuading voters using human–artificial intelligence dialogues
Lin, H., Czarnek, G., Lewis, B., White, J. P., Berinsky, A. J., Costello, T. H., Pennycook, G., & Rand, D. G. (2025) · Nature · doi:10.1038/s41586-025-09771-9 · Cited by 126
“Furthermore, human–AI dialogues can also durably reduce conspiracy beliefs 20–22 , which were previously thought to be largely impervious to evidence and arguments (much like political attitudes towards presidential candidates).”
Self-citationJones, C., & Bergen, B. K. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11517-6 · Cited by 58
Intelligent Systems, Vulnerable Minds: A Framework for Radicalization to Violence in the Age of AI
Kunst, J. R., Obaidi, M., Gollwitzer, A., Brandtzæg, P. B., Hinrichs, Y., Saini, N., & Schroeder, D. T. (2026) · Personality and Social Psychology Review · doi:10.1177/10888683261430089 · Cited by 14
“Moreover, when deployed as interactive conversational agents, these models are exceptionally potent; research shows that AI chatbots show strong persuasiveness, even for deeply held beliefs (Boissin et al., 2025; Costello et al., 2024).”
Why do experts miss AI’s errors? Evidence from a randomized labeling experiment
Goulas, S., Megalokonomou, R., & Sotirakopoulos, P. (2026) · PNAS Nexus · doi:10.1093/pnasnexus/pgag146 · Cited by 5
“Our design builds on recent work using experiments in which information shown to participants is labeled as either human-made or AI-generated (24, 37, 38).”
Using Artificial Intelligence to Better Understand Human Intelligence
Pennycook, G., Costello, T. H., & Rand, D. G. (2026) · Current Directions in Psychological Science · doi:10.1177/09637214261417960 · Cited by 2
“In a follow-up experiment, Boissin et al. (2025) tested this possibility by modifying the experiment so that it would be more plausible that the participants were having a dialogue with another person (e.g., by slowing down responses, adding informal language).”
Self-citation
Metadata
- Authors
- Boissin, E., Costello, T. H., Spinoza-Martín, D., Rand, D. G., & Pennycook, G.
- Affiliations
- Cornell University (US)
- American University (US)
- Massachusetts Institute of Technology (US)
- University of Regina (CA)
- Corresponding author
- Boissin, Esther
- Venue
- PNAS Nexus
- Publisher
- Oxford University Press (OUP)
- Published
- 2025-10-14
- Volume
- 4
- Issue
- 11
- Pages
- pgaf325-pgaf325
- ISSN
- 2752-6542 (ISSN-L)
- PMC
- PMC12578366
- OpenAlex
- W4415222261
- Semantic Scholar
- 7d909a8b3c49ecf6962b80d686d790b5dbe257b4
- Cited by
- 18
- References
- 13
- Citations by year
- 2026: 17 · 2025: 1
- FWCI
- 30.93
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PNAS Nexus (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Social Sciences and Humanities Research Council of Canada
- Fyssen Foundation
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Social Sciences › Psychology › Social Psychology › Psychology of Social Influence
- Social Sciences › Psychology › Social Psychology › Paranormal Experiences and Beliefs
- Keywords
- conspiracy beliefs, large language models, AI persuasion, debunking, belief confidence, misinformation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Dialogues with large language models reduce conspiracy beliefs even when the AI is perceived as human. https://app.cochat.ai/papers/10.1093/pnasnexus/pgaf325
- The paper
- Boissin, E., Costello, T. H., Spinoza-Martín, D., Rand, D. G., & Pennycook, G. (2025). Dialogues with large language models reduce conspiracy beliefs even when the AI is perceived as human. PNAS Nexus. https://doi.org/10.1093/pnasnexus/pgaf325
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.