CoChat

Durably reducing conspiracy beliefs through dialogues with AI

Open access

The DOI matches its Crossref record; 82 of the 82 references it cites were found in a registry or an index; none of the 22 sources its claims cite is checked for support yet.

Abstract

Conspiracy theories are a paradigmatic example of beliefs that, once adopted, are extremely difficult to dispel. Influential psychological theories propose that conspiracy beliefs are uniquely resistant to counterevidence because they satisfy important needs and motivations. Here, we raise the possibility that previous attempts to correct conspiracy beliefs have been unsuccessful merely because they failed to deliver counterevidence that was sufficiently compelling and tailored to each believer’s specific conspiracy theory (which vary dramatically from believer to believer). To evaluate this possibility, we leverage recent developments in generative artificial intelligence (AI) to deliver well-argued, person-specific debunks to a total of N = 2,190 conspiracy theory believers. Participants in our experiments provided detailed, open-ended explanations of a conspiracy theory they believed, and then engaged in a 3 round dialogue with a frontier generative AI model (GPT-4 Turbo) which was instructed to reduce each participant’s belief in their conspiracy theory (or discuss a banal topic in a control condition). Across two experiments, we find robust evidence that the debunking conversation with the AI reduced belief in conspiracy theories by roughly 20%. This effect did not decay over 2 months time, was consistently observed across a wide range of different conspiracy theories, and occurred even for participants whose conspiracy beliefs were deeply entrenched and of great importance to their identities. Furthermore, although the dialogues were focused on a single conspiracy theory, the intervention spilled over to reduce beliefs in unrelated conspiracies, indicating a general decrease in conspiratorial worldview, as well as increasing intentions to challenge others who espouse their chosen conspiracy. These findings highlight that even many people who strongly believe in seemingly fact-resistant conspiratorial beliefs can change their minds in the face of sufficient evidence.

Overview

CoChat Summary

## TL;DR

A brief 3-round conversation with GPT-4 Turbo substantially reduced participants’ belief in their own conspiracy theory, and the effect persisted for 2 months. The intervention also reduced belief in unrelated conspiracies and increased some anti-conspiracy behavioral intentions.

## Key findings

  • In Study 1, the treatment reduced belief in the focal conspiracy by 16.5 percentage points more than control (95% CI [13.6, 19.4], p < .001, d = 1.13), corresponding to a 21.43% decrease in belief in treatment vs. 1.04% in control.
  • In Study 2, the treatment again reduced focal-conspiracy belief by 12.39 percentage points more than control (95% CI [10.07, 14.72], p < .001, d = 0.79), a 19.41% decrease vs. 2.94% in control.
  • The focal-belief reduction was durable: there was no significant change from immediately after the conversation to 2 months later (b ΔImmediatelyPost–2Month = 0.03, 95% CI [-2.24, 2.31], p = .98).
  • Spillover effects were observed: in Study 1, generic conspiracy belief across the 15-item BCTI decreased by 3.05 points in the active condition (95% CI [-3.90, -2.20], p < .001), and in Study 2 the treatment increased intentions to ignore/unfollow pro-conspiracy social media (β = .39, p < .001) and ignore/argue against believers (β = .42, p < .001).

## Methods

  • Design: Two experiments used 3-round dialogues with GPT-4 Turbo. Treatment participants got an AI conversation aimed at reducing belief in a personally chosen conspiracy; controls discussed unrelated topics.
  • Samples/data: Study 1 used N = 1,055 American participants quota-matched to the U.S. census; Study 2 pooled two samples for a total of n = 1,553 analyzable participants. Participants first gave open-ended descriptions of a conspiracy they believed, which the AI summarized into a standardized belief statement.
  • Analysis: The paper used linear regression with covariate adjustment, linear mixed models for repeated measures over time, and post hoc NLP/LLM-based classification to identify conspiracy topics and AI persuasion strategies; heterogeneity was explored with GAMs, multiple regression, and causal forests.

## Limitations

  • The paper states it is a working paper / preliminary version not yet peer reviewed.
  • It explicitly notes that participants were told they were conversing with an AI, so the authors say future work should test whether effects differ if participants think they are talking to a human.
  • Evident from the text: some behavioral outcomes in Study 2 were dropped as uninterpretable because of wording problems, and some participants were excluded for not supplying a genuine conspiracy or for low-confidence endorsements, which limits generalizability to all participants.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.31234/osf.io/xcwdn

Cites

82 of 82 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

0 Valid · 33 Not valid · 14 Not checked · 104 Own results and interpretations

46 more claims

Create a free account to see all 151 claims and to verify the 22 references still open.

See all claims — free account
  • It may state two separate claims as one. 47
  • Probably not a substantial claim. 81
  • The quote only partly states this claim. 2

Create a free account to see these claims.

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • ConsistentF(68, 652) = 9.38reported p < .001 · recomputed p < .001

    “F(68, 652) = 9.38, p < .001”

  • ConsistentF(65, 1197) = 13.87reported p < .001 · recomputed p < .001

    “F(65, 1197) = 13.87,p < .001”

  • ConsistentF(26, 1971) = 49.44reported p < .001 · recomputed p < .001

    “F(26, 1971) = 49.44,p < .001”

  • ConsistentF(3, 757) = 168.75reported p < .001 · recomputed p < .001

    “F(3, 757) = 168.75,p < .001”

  • ConsistentF(3, 1345) = 249.43reported p < .001 · recomputed p < .001

    “F(3, 1345) = 249.43,p < .001”

5 of 5

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Show all 82 references

Research works printed but not linked: 2

  1. [37]37. G. Pennycook, "Chapter Three -A framework for understanding reasoning errors: From fake news to climate change and beyond" in Advances in Experimental Social Psychology, B. Gawronski, Ed. (Academic Press, 2023; https://www.sciencedirect.com/science/article/pii/S0065260122000284)vol. 67, pp. 131-208.
  2. [50]50. OECD, OECD Guidelines on Measuring Trust (Organisation for Economic Co-operation and Development, Paris, 2017; https://www.oecd-ilibrary.org/governance/oecd-guidelines-on-measuring-trust_9789264278219-en).

Cited by

See all 28

Metadata

Authors
Costello, T. H., Pennycook, G., & Rand, D. G.
Affiliations
  • Massachusetts Institute of Technology (US)
  • Cornell University (US)
Published
2024-04-03
OpenAlex
W4393864810
Cited by
28
References
77
Citations by year
2026: 2 · 2025: 16 · 2024: 9
Type
posted content
Language
en
Open access
Locations
PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
Topics
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Social Sciences › Psychology › Social Psychology › Paranormal Experiences and Beliefs
  • Social Sciences › Sociology and Political Science › Social and Intergroup Psychology
Keywords
conspiracy beliefs, conspiracy theories, belief revision, debunking, counterevidence, generative AI, GPT-4 Turbo, conversational AI, belief persistence, spillover effects

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Durably reducing conspiracy beliefs through dialogues with AI. https://app.cochat.ai/papers/10.31234/osf.io/xcwdn
The paper
Costello, T. H., Pennycook, G., & Rand, D. G. (2024). Durably reducing conspiracy beliefs through dialogues with AI. https://doi.org/10.31234/osf.io/xcwdn

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.