Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index; 2 of the 42 sources its claims cite were checked for support.

How to debunk misinformation? An experimental online study investigating text structures and headline formats

Open access
Kotz, Johannes, Giese, Helge, König, Laura M. · 2023 · British Journal of Health Psychology

Abstract

Objectives: Misinformation is a crucial problem, particularly online, and the success of debunking messages has so far been limited. In this study, we experimentally test how debunking text structure (truth sandwich vs. bottom‐heavy) and headline format (statement vs. questions) affect the belief in misinformation across topics of the safety of COVID vaccines and GMO foods. Design: Experimental online study. Methods: A representative German sample of 4906 participants were randomly assigned to reading one of eight debunking messages in the experimentally varied formats and subsequently rated the acceptance of this message and the agreement to misinformation statements about the mentioned topics and an unrefuted control myth. Results: While the debunking messages specifically decreased the belief in the targeted myth, these beliefs and the acceptance of the debunking message were unaffected by the text structures and headline formats. Yet, they were less successful when addressing individuals with strong pre‐existing, incongruent attitudes and distrust in science. Conclusions: The risk of backfire effects in debunking misinformation is low. Text structure and headline format are of relatively little importance for the effectiveness of debunking messages. Instead, writers may need to pay attention to the text being comprehensive, trustworthy and persuasive to maximize effectiveness.

Overview

CoChat Summary

TL;DR

In a representative German online experiment (N = 4906), debunking messages reduced agreement with the specific misinformation they addressed, but text structure (truth sandwich vs. bottom-heavy) and headline format (statement vs. question) had no meaningful effects. Debunking was less effective among people with low trust in science or attitudes incongruent with the correction, but there was little evidence of backfire.

Key findings

  • Debunking texts specifically reduced belief in the targeted myth: agreement with COVID-19 vaccine misinformation was lower after the vaccine text than after the GMO text (b = −.41, t(4898) = −8.673, p < .001, d = −.25), and agreement with GMO misinformation was lower after the GMO text than after the vaccine text (b = .66, t(4898) = 17.904, p < .001, d = .51). Agreement with the unrefuted Vitamin C control myth did not differ by debunking text (b = .01, t(4898) = .382, p = .7026, d = .01).
  • Text structure did not significantly affect agreement with misinformation (all effects involving text structure: F ≤ 1.96, ηp² < .001, p ≥ .142). Headline format had only a negligible effect: statement headlines slightly increased effectiveness of the fitting debunking message (headline format × debunk topic × myth topic: F(1.95, 9556.51) = 3.19, ηp² < .001, p = .043).
  • The COVID-19 vaccine debunking text was evaluated more positively (M = 5.28, SD = 1.36) than the GMO text (M = 5.04, SD = 1.26; F(1, 4989) = 43.99, ηp² = .009, p < .001), and willingness to share was higher for the COVID-19 vaccine text (M = 3.77, SD = 1.65) than the GMO text (M = 3.34, SD = 1.54; F(1, 4989) = 88.50, ηp² = .018, p < .001).
  • Debunking was less effective among participants with low trust in science (debunk topic × myth topic × trust: F(1.99, 9710.46) = 18.96, ηp² = .004, p < .001) and incongruent vaccination attitudes (F(1.96, 9483.53) = 8.93, ηp² = .002, p < .001); for GMO attitudes, the message was slightly more effective in individuals with lower GMO attitudes (F(1.96, 9483.53) = 6.09, ηp² = .001, p = .003). Message acceptance did not mediate effects on agreement with the myth (all |βy| ≤ .01, all p ≥ .137).

Methods

  • Preregistered experimental online study with a representative German sample of N = 4906 (n = 613 per condition except one condition with n = 615), recruited via the ZPID PsychLab platform with quotas for gender, age, education, and household income. Participants were randomly assigned to one of eight debunking messages in a 2 Text Structure (truth sandwich vs. bottom-heavy) × 2 Headline Format (question vs. statement) × 2 Debunk Topic (COVID-19 vaccines vs. GMO food) between-subjects design.
  • Primary outcome was agreement with misinformation statements about three myth topics (Vitamin C control, COVID-19 vaccines, GMO food) measured within participants on a 7-point Likert scale. Secondary outcomes were evaluation of the text (comprehensiveness, trustworthiness, persuasiveness) and perceived social impact (mentioning in personal communication and sharing on social media).
  • Main analyses used mixed ANOVAs, exploratory GLMs, and mediation models with 10,000 bias-corrected bootstrap confidence intervals; the analyst conducting preregistered analyses was blinded to independent variables.

Limitations

  • The paper states the study was cross-sectional and cannot provide insights into changes over time; immediate debunking effects could be obliterated over time by continued influence of misinformation. It also states that actual behavior was not assessed, and that the study was presented as conducted by a university research team, which may have been considered credible by most people; future studies could vary source credibility.
  • The text makes evident that age data had an anomaly: n = 195 participants (4%) indicated they were 100 years old, so the paper reports full-sample analyses (N = 4906) as preregistered and supplementary analyses for participants aged 18–99 (N = 4711). The study is also limited to two debunking topics and a German sample, and forced responses were used throughout.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1111/bjhp.12670

Cites

52 of 52 references verified0 citation errors · 0 not checkable

Statistics

Figures

FIGURE 1 Flowchart with exclusion criteria for participants and allocation to the eight experimental conditions.
FIGURE 1 Flowchart with exclusion criteria for participants and allocation to the eight experimental conditions.CC-BY · From the PDF
FIGURE 2 Agreement to myths depending on the presented debunking text and headline format. Grey dots indicate means; outlines depict the distribution of the data.
FIGURE 2 Agreement to myths depending on the presented debunking text and headline format. Grey dots indicate means; outlines depict the distribution of the data.CC-BY · From the PDF

Verification

Claims the paper makes

2 Valid · 0 Not valid · 35 Not checked · 44 Own results and interpretations

38 more claims

Create a free account to see all 81 claims and to verify the 26 references still open.

See all claims — free account
  • It may state two separate claims as one. 13
  • Probably not a substantial claim. 28

Create a free account to see these claims.

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Consistentt(4898) = 0.382reported p = .7026 · recomputed p = .702

    “t(4898) = .382,p = .7026”

  • Consistentt(4898) = -8.673reported p < .001 · recomputed p < .001

    “t(4898) = -8.673,p < .001”

  • Consistentt(4898) = 17.904reported p < .001 · recomputed p < .001

    “t(4898) = 17.904, p < .001”

3 of 3

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

1 of 1

Citations

Cites

Research works printed but not linked: 3

  1. Anderson E. R. Horton W. S. &Rapp D. N.(2019).Hungry for the Truth: Evaluating the Utility of “Truth Sandwiches” as Refutations. Annual meeting of the society for text and discourse https://www.researchgate.net/profile/William‐Horton‐4/publication/334491502_Hungry_for_the_Truth_Evaluating_the_Utility_of_Truth_Sandwiches_as_Refutations/links/5d2de48092851cf44089eab7/Hungry‐for‐the‐Truth‐Evaluating‐the‐Utility‐of‐Truth‐Sandwiches‐as‐Refutations.pdf
  2. Conner M., 2005, Predicting health behaviour, 121
  3. Lewandowsky S., 2020, The Debunking Handbook 2020

Cited by

See all 31

Metadata

Authors
Kotz, J., Giese, H., & König, L. M.
Affiliations
  • University of Konstanz (DE)
  • University of Bayreuth (DE)
Corresponding author
König, Laura M.
Venue
British Journal of Health Psychology
Publisher
Wiley
Published
2023-06
Volume
28
Issue
4
Pages
1097-1112
ISSN
  • 1359-107X (ISSN-L)
  • 2044-8287
OpenAlex
W4379094875
Cited by
31
References
50
Citations by year
2026: 11 · 2025: 12 · 2024: 4 · 2023: 4
FWCI
9.21
Type
journal article
Language
en
Open access
Locations
  • British Journal of Health Psychology (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • EPub Bayreuth (University of Bayreuth) (submittedVersion, other-oa)
  • KOPS (University of Konstanz) (submittedVersion, cc-by)
Topics
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
  • Social Sciences › Communication › Communication and COVID-19 Impact
Keywords
misinformation debunking, belief in misinformation, backfire effect, distrust of science, message acceptance
MeSH terms
COVID-19, Communication, Humans, Persuasive Communication

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-11-16, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-16 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.