Durably reducing conspiracy beliefs through dialogues with AI
Open accessThe DOI matches its Crossref record; 82 of the 82 references it cites were found in a registry or an index; none of the 22 sources its claims cite is checked for support yet.
Abstract
Conspiracy theories are a paradigmatic example of beliefs that, once adopted, are extremely difficult to dispel. Influential psychological theories propose that conspiracy beliefs are uniquely resistant to counterevidence because they satisfy important needs and motivations. Here, we raise the possibility that previous attempts to correct conspiracy beliefs have been unsuccessful merely because they failed to deliver counterevidence that was sufficiently compelling and tailored to each believer’s specific conspiracy theory (which vary dramatically from believer to believer). To evaluate this possibility, we leverage recent developments in generative artificial intelligence (AI) to deliver well-argued, person-specific debunks to a total of N = 2,190 conspiracy theory believers. Participants in our experiments provided detailed, open-ended explanations of a conspiracy theory they believed, and then engaged in a 3 round dialogue with a frontier generative AI model (GPT-4 Turbo) which was instructed to reduce each participant’s belief in their conspiracy theory (or discuss a banal topic in a control condition). Across two experiments, we find robust evidence that the debunking conversation with the AI reduced belief in conspiracy theories by roughly 20%. This effect did not decay over 2 months time, was consistently observed across a wide range of different conspiracy theories, and occurred even for participants whose conspiracy beliefs were deeply entrenched and of great importance to their identities. Furthermore, although the dialogues were focused on a single conspiracy theory, the intervention spilled over to reduce beliefs in unrelated conspiracies, indicating a general decrease in conspiratorial worldview, as well as increasing intentions to challenge others who espouse their chosen conspiracy. These findings highlight that even many people who strongly believe in seemingly fact-resistant conspiratorial beliefs can change their minds in the face of sufficient evidence.
Overview
CoChat Summary
## TL;DR
A brief 3-round conversation with GPT-4 Turbo substantially reduced participants’ belief in their own conspiracy theory, and the effect persisted for 2 months. The intervention also reduced belief in unrelated conspiracies and increased some anti-conspiracy behavioral intentions.
## Key findings
- In Study 1, the treatment reduced belief in the focal conspiracy by 16.5 percentage points more than control (95% CI [13.6, 19.4], p < .001, d = 1.13), corresponding to a 21.43% decrease in belief in treatment vs. 1.04% in control.
- In Study 2, the treatment again reduced focal-conspiracy belief by 12.39 percentage points more than control (95% CI [10.07, 14.72], p < .001, d = 0.79), a 19.41% decrease vs. 2.94% in control.
- The focal-belief reduction was durable: there was no significant change from immediately after the conversation to 2 months later (b ΔImmediatelyPost–2Month = 0.03, 95% CI [-2.24, 2.31], p = .98).
- Spillover effects were observed: in Study 1, generic conspiracy belief across the 15-item BCTI decreased by 3.05 points in the active condition (95% CI [-3.90, -2.20], p < .001), and in Study 2 the treatment increased intentions to ignore/unfollow pro-conspiracy social media (β = .39, p < .001) and ignore/argue against believers (β = .42, p < .001).
## Methods
- Design: Two experiments used 3-round dialogues with GPT-4 Turbo. Treatment participants got an AI conversation aimed at reducing belief in a personally chosen conspiracy; controls discussed unrelated topics.
- Samples/data: Study 1 used N = 1,055 American participants quota-matched to the U.S. census; Study 2 pooled two samples for a total of n = 1,553 analyzable participants. Participants first gave open-ended descriptions of a conspiracy they believed, which the AI summarized into a standardized belief statement.
- Analysis: The paper used linear regression with covariate adjustment, linear mixed models for repeated measures over time, and post hoc NLP/LLM-based classification to identify conspiracy topics and AI persuasion strategies; heterogeneity was explored with GAMs, multiple regression, and causal forests.
## Limitations
- The paper states it is a working paper / preliminary version not yet peer reviewed.
- It explicitly notes that participants were told they were conversing with an AI, so the authors say future work should test whether effects differ if participants think they are talking to a human.
- Evident from the text: some behavioral outcomes in Study 2 were dropped as uninterpretable because of wording problems, and some participants were excluded for not supplying a genuine conspiracy or for low-confidence endorsements, which limits generalizability to all participants.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 33 Not valid · 14 Not checked · 104 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and significantly increased willingness to ignore or argue against people who believe the focal conspiracy (𝛃 = .42[.31, .54],p < .001; Figure 4D).”
§ Durably reducing conspiracy beliefs through dialogues with AI > Spillover effects and behavioral implications
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This result is robust to assuming that the 14% of participants who did not complete the follow-up returned to their initial pre-treatment belief levels (b = 12.70 95% CI [9.47, 15.93], p < .001).”
§ Durably reducing conspiracy beliefs through dialogues with AI > Can conspiracy beliefs be refuted?
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Second, we examine how important participants indicated the conspiracy theory is to their worldview (Figure 3B), which does significantly decrease the size of the treatment effect (ΔAIC = 3.12, ΔR 2 = .003,p = .025).”
§ Durably reducing conspiracy beliefs through dialogues with AI > Robustness across topics and people
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Consistent with the potential for uptake of AI dialogues, many conspiracy-believing respondents in our sample expressed excitement and appreciation in their conversations with the AI”
§ Durably reducing conspiracy beliefs through dialogues with AI > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and the CATE ranged from -18.2 to -10.0 (median = -15.4)for individuals with 95th percentile and above BCTI scores.”
§ Durably reducing conspiracy beliefs through dialogues with AI > Robustness across topics and people
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
46 more claims
Create a free account to see all 151 claims and to verify the 22 references still open.
- It may state two separate claims as one. 47
- Probably not a substantial claim. 81
- The quote only partly states this claim. 2
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- ConsistentF(68, 652) = 9.38reported p < .001 · recomputed p < .001
“F(68, 652) = 9.38, p < .001”
- ConsistentF(65, 1197) = 13.87reported p < .001 · recomputed p < .001
“F(65, 1197) = 13.87,p < .001”
- ConsistentF(26, 1971) = 49.44reported p < .001 · recomputed p < .001
“F(26, 1971) = 49.44,p < .001”
- ConsistentF(3, 757) = 168.75reported p < .001 · recomputed p < .001
“F(3, 757) = 168.75,p < .001”
- ConsistentF(3, 1345) = 249.43reported p < .001 · recomputed p < .001
“F(3, 1345) = 249.43,p < .001”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Durably reducing conspiracy beliefs through dialogues with AI > Can conspiracy beliefs be refuted?
“75.2% indicated belief in a conspiracy theory and were included in our subsequent analyses, whereas 14.8% said they did not believe any conspiracy theories or described a belief that the AI classified as not actually conspiratorial”
§ Durably reducing conspiracy beliefs through dialogues with AI > Can conspiracy beliefs be refuted?
“and 10.6% described a conspiracy theory but had belief below the scale midpoint.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Durably reducing conspiracy beliefs through dialogues with AI > 1.1 Participants
“In total, 1190 individuals began the survey (this includes 75 participants from a pilot conducted prior to the pre-registration; for completeness, we include these participants in our analyses, but excluding them does not qualitatively change the results).”
§ Durably reducing conspiracy beliefs through dialogues with AI > 1.1 Participants
“Thus, 774 participants began the treatment.”
Citations
Cites
(1996) · Cited by 20,179
Pennycook, G., & Rand, D. G. (2019) · Cognition · doi:10.1016/j.cognition.2018.06.011 · Cited by 2,022
Self-citationAthey, S., Tibshirani, J., & Wager, S. (2019) · The Annals of Statistics · doi:10.1214/18-aos1709 · Cited by 1,870
Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey
Min, B., Ross, H., Sulem, E., Veyseh, A. P. B., Nguyen, T. H., Sainz, O., Agirre, E., Heintz, I., & Roth, D. (2024) · ACM Computing Surveys · doi:10.1145/3605943 · Cited by 1,387
ChatGPT outperforms crowd workers for text-annotation tasks
Gilardi, F., Alizadeh, M., & Kubli, M. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2305016120 · Cited by 1,197
Lacking control increases illusory pattern perception
Whitson, J. A., & Galinsky, A. D. (2008) · Science · doi:10.1126/science.1159845 · Cited by 1,119
Conspiracy theories: Causes and cures
Sunstein, C. R., & Vermeule, A. (2009) · Journal of Political Philosophy · doi:10.1111/j.1467-9760.2008.00325.x · Cited by 1,050
Conspiracy Theories and the Paranoid Style(s) of Mass Opinion
Oliver, J. E., & Wood, T. J. (2014) · American Journal of Political Science · doi:10.1111/ajps.12084 · Cited by 877
The Role of Conspiracist Ideation and Worldviews in Predicting Rejection of Science
Lewandowsky, S., Gignac, G. E., & Oberauer, K. (2013) · PLoS ONE · doi:10.1371/journal.pone.0075637 · Cited by 693
Can Large Language Models Transform Computational Social Science?
Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., & Yang, D. (2024) · Computational Linguistics · doi:10.1162/coli_a_00502 · Cited by 585
Research works printed but not linked: 2
- [37]37. G. Pennycook, "Chapter Three -A framework for understanding reasoning errors: From fake news to climate change and beyond" in Advances in Experimental Social Psychology, B. Gawronski, Ed. (Academic Press, 2023; https://www.sciencedirect.com/science/article/pii/S0065260122000284)vol. 67, pp. 131-208.
- [50]50. OECD, OECD Guidelines on Measuring Trust (Organisation for Economic Co-operation and Development, Paris, 2017; https://www.oecd-ilibrary.org/governance/oecd-guidelines-on-measuring-trust_9789264278219-en).
Cited by
Durably reducing conspiracy beliefs through dialogues with AI
Costello, T. H., Pennycook, G., & Rand, D. G. (2024) · Science · doi:10.1126/science.adq1814 · Cited by 354
Self-citationPark, S., & Nan, X. (2026) · AI & SOCIETY · doi:10.1007/s00146-025-02620-3 · Cited by 58
Confronting Core Issues: A Critical Assessment of Attitude Polarization Using Tailored Experiments
Velez, Y. R., & Liu, P. (2024) · American Political Science Review · doi:10.1017/s0003055424000819 · Cited by 54
Testing theories of political persuasion using AI
Argyle, L. P., Busby, E. C., Gubler, J. R., Lyman, A., Olcott, J., Pond, J., & Wingate, D. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2412815122 · Cited by 53
The impact of generative artificial intelligence on socioeconomic inequalities and policy makingPreprint
Capraro, V., Lentsch, A., Acemoğlu, D., Akgün, S., Akhmedova, A., Bilancini, E., Bonnefon, J., Brañas‐Garza, P., Butera, L., Douglas, K. M., et al. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/6fd2y · Cited by 48
Metadata
- Authors
- Costello, T. H., Pennycook, G., & Rand, D. G.
- Affiliations
- Massachusetts Institute of Technology (US)
- Cornell University (US)
- Publisher
- Center for Open Science
- Published
- 2024-04-03
- OpenAlex
- W4393864810
- Cited by
- 28
- References
- 77
- Citations by year
- 2026: 2 · 2025: 16 · 2024: 9
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Locations
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Social Sciences › Psychology › Social Psychology › Paranormal Experiences and Beliefs
- Social Sciences › Sociology and Political Science › Social and Intergroup Psychology
- Keywords
- conspiracy beliefs, conspiracy theories, belief revision, debunking, counterevidence, generative AI, GPT-4 Turbo, conversational AI, belief persistence, spillover effects
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Durably reducing conspiracy beliefs through dialogues with AI. https://app.cochat.ai/papers/10.31234/osf.io/xcwdn
- The paper
- Costello, T. H., Pennycook, G., & Rand, D. G. (2024). Durably reducing conspiracy beliefs through dialogues with AI. https://doi.org/10.31234/osf.io/xcwdn
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.