CoChat
Sign Up
Cited by 773Misinformation and Its ImpactsLiteracy, Media, and EducationIslamic Finance and Communication

A digital media literacy intervention increases discernment between mainstream and false news in the United States and India

Open access
Guess, Andrew M., Lerner, Michael, Lyons, Benjamin, Montgomery, Jacob M., Nyhan, Brendan, Reifler, Jason, Sircar, Neelanjan·2020·Proceedings of the National Academy of Sciences
DOI 10.1073/pnas.1920498117PMC PMC7355018
Open this paper in CoChat — free accountOpen PDF

Abstract

Widespread belief in misinformation circulating online is a critical challenge for modern societies. While research to date has focused on psychological and political antecedents to this phenomenon, few studies have explored the role of digital media literacy shortfalls. Using data from preregistered survey experiments conducted around recent elections in the United States and India, we assess the effectiveness of an intervention modeled closely on the world’s largest media literacy campaign, which provided “tips” on how to spot false news to people in 14 countries. Our results indicate that exposure to this intervention reduced the perceived accuracy of both mainstream and false news headlines, but effects on the latter were significantly larger. As a result, the intervention improved discernment between mainstream and false news headlines among both a nationally representative sample in the United States (by 26.5%) and a highly educated online sample in India (by 17.5%). This increase in discernment remained measurable several weeks later in the United States (but not in India). However, we find no effects among a representative sample of respondents in a largely rural area of northern India, where rates of social media use are far lower.

AI snapshot

Written from the paper’s full text.

TL;DR

A preregistered two-wave survey experiment in the United States and India found that exposure to Facebook/WhatsApp-style “tips to spot false news” reduced belief in false headlines and improved discernment between mainstream and false news, especially in the US and India online samples. Effects were modest, decayed over time, and were not detectable in the India face-to-face sample.

Key findings

  • In the US wave 1, the media literacy intervention reduced perceived accuracy of false news (ITT β=−0.196, SE=0.020; ATT β=−0.299, SE=0.030) and increased discernment between mainstream and false news (ITT β=0.146, SE=0.024; ATT β=0.223, SE=0.035), a 26.5% relative improvement. It also slightly reduced perceived accuracy of mainstream news (ITT β=−0.046, SE=0.017).
  • In the India online sample, the intervention reduced false-news accuracy (ITT β=−0.126, SE=0.026; ATT β=−0.470, SE=0.097) and increased discernment (ITT β=0.063, SE=0.025; ATT β=0.221, SE=0.088), a 17.3% relative improvement; mainstream-news accuracy also decreased (ITT β=−0.071, SE=0.025).
  • In the India face-to-face sample, there was no evidence that the treatment affected false-news beliefs (ITT β=−0.007, SE=0.024), mainstream-news beliefs (ITT β=0.002, SE=0.024), or discernment (ITT β=0.006, SE=0.030).
  • US effects persisted at wave 2 but attenuated: false-news accuracy ITT β=−0.080 (SE=0.019), discernment β=0.050 (SE=0.020). India online effects were not statistically reliable in wave 2. No consistent moderation by partisan congeniality was found.

Methods

  • Preregistered two-wave panel surveys: US online panel by YouGov (wave 1 N=4,907; wave 2 N=4,283); India online national convenience sample (wave 1 N=3,273; wave 2 N=1,369); India face-to-face random sample in four Uttar Pradesh constituencies (wave 1 N=3,744; wave 2 N=2,695).
  • Respondents were randomly assigned with probability 0.5 to view media literacy tips adapted from Facebook’s “Tips to Spot False News” (10 tips in the US; 6 adapted tips in India) before rating actual news headlines on a 4-point perceived-accuracy scale.
  • Main analyses used pooled OLS with headline fixed effects and cluster-robust standard errors; ATT estimated via instrumental variables, using correct answers to comprehension checks as treatment receipt.

Limitations

  • The paper states that effect sizes were modest, did not eliminate false-news belief, decayed over time, and came with a small negative effect on mainstream-news accuracy; it also states that real-world users may ignore the tips, that ITT vs ATT shows attention matters, and that there was insufficient evidence the intervention changed real-world false-news consumption.
  • The paper notes the India face-to-face null results and much lower treatment receipt in India (24% face-to-face, 28% online) than the US (66%), plus sample differences such as only 11% WhatsApp use in the face-to-face sample versus 90% online.
  • Evident from the text: the India online sample was a convenience sample, the face-to-face sample differed demographically and in digital experience, and some analyses—such as behavioral outcomes—were limited to subsets with available data.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1073/pnas.1920498117

Cites

36 of 36 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 10 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    In the US wave 1 survey experiment, randomized exposure to the media literacy intervention caused a decrease in the perceived accuracy of false news articles (ITT β=−0.196, SE=0.020, P<0.005).

    “Consistent with our first hypothesis (H1), randomized exposure to the media literacy intervention causes a decrease in the perceived accuracy of false news articles.”
    • Also stated in: § US Survey Experiment. · plain
      “Results from wave 1 of the US study in Table 1 show a decrease of nearly 0.2 points on a 4-point scale (intent to treat [ITT]: β=−0.196, SE=0.020; P<0.005).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The headlines classified as false are false and those classified as mainstream are mainstream.
    • Rests on an assumption: Respondents apply the intervention's lessons to the headline rating task even though they were not instructed to do so.
  • Findingown data

    Among US respondents who actually received the treatment, the perceived accuracy of false headlines decreased by nearly 0.3 points on a 4-point scale (ATT β=−0.299, SE=0.030, P<0.005).

    “With this approach, we estimate that the perceived accuracy of false headlines decreased by nearly 0.3 points on a 4-point scale (ATT: β=−0.299, SE=0.030; P<0.005).†”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The headlines classified as false are false and those classified as mainstream are mainstream.
    • Rests on an assumption: The comprehension checks correctly identify respondents who actually received the treatment.
  • Findingown data

    In the US wave 1 survey experiment, exposure to the media literacy intervention had a small negative effect on belief in mainstream news (ITT β=−0.046, SE=0.017, P<0.01; ATT β=−0.071, SE=0.026, P<0.01).

    “We find that exposure to the media literacy intervention had a small negative effect on belief in mainstream news in wave 1 (ITT, β=−0.046 [SE=0.017], P<0.01; ATT, β=−0.071 [SE=0.026], P<0.01).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The headlines classified as false are false and those classified as mainstream are mainstream.
  • Findingown data

    In the US survey, there was no consistent evidence that the effects of the media literacy intervention were moderated by the partisan congeniality of the headlines.

    “We find no consistent evidence that the effects of media literacy intervention are moderated by congeniality (SI Appendix, section C).”
    Quote in the paperNot a substantial claim
  • Findingown data

    In the US wave 2 survey, the magnitude of the intervention effect on perceived accuracy of false headlines attenuated by more than half relative to wave 1 (ITT β=−0.080, SE=0.019, P<0.005; ATT β=−0.121, SE=0.028, P<0.005).

    “While the effect is still present weeks later, its magnitude attenuates by more than half relative to wave 1 (ITT, β=−0.080 [SE=0.019], P<0.005; ATT, β=−0.121 [SE=0.028], P<0.005).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The headlines classified as false are false and those classified as mainstream are mainstream.
  • May combine two claims. 22
  • The quote only partly states this claim. 4
  • Classified as probably not a substantial claim. 15

  • May combine two claims.

    Findingown data

    In the US wave 1 survey experiment, the media literacy intervention increased discernment between mainstream and false stories (ITT β=0.146, SE=0.024, P<0.005; ATT β=0.223, SE=0.035, P<0.005), a 26.5% improvement relative to the control condition.

    “As a result, the media literacy intervention increased discernment between mainstream and false stories (ITT, β=0.146 [SE=0.024], P<0.005; ATT, β=0.223 [SE=0.035], P<0.005), demonstrating that it helped respondents to better distinguish between these two types of content.”
    • Also stated in: § US Survey Experiment. · plain
      “In relative terms, this effect represents a 26.5% improvement in respondents’ ability to distinguish between mainstream and false news stories compared to the control condition.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The headlines classified as false are false and those classified as mainstream are mainstream.
    • Rests on an assumption: Respondents apply the intervention's lessons to the headline rating task even though they were not instructed to do so.

  • May combine two claims.

    Findingown data

    In the US survey, there was insufficient evidence to conclude that the intervention had an effect on self-reported intention to share false news or on behavioral measures of news consumption.

    “Additional results reported in SI Appendix, section C indicate that we have insufficient evidence to conclude that the intervention had an effect on self-reported intention to share false news or behavioral measures of news consumption (SI Appendix, section C, Table C13).”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    In the India online sample wave 1, the media literacy treatment significantly reduced beliefs in false news stories (ITT β=−0.126, SE=0.026, P<0.005; ATT β=−0.470, SE=0.097, P<0.005).

    “For the online sample, we again find support for H1. The media literacy treatment significantly reduced beliefs in false news stories (ITT: β=−0.126, SE = 0.026; P<0.005) in the first wave of a two-wave survey (Table 2).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The headlines classified as false are false and those classified as mainstream are mainstream.
    • Rests on an assumption: The comprehension checks correctly identify respondents who actually received the treatment.
    • Rests on an assumption: Respondents apply the intervention's lessons to the headline rating task even though they were not instructed to do so.

  • May combine two claims.

    Findingown data

    In the US survey, the proportion of respondents who classified mainstream news as not very accurate or not at all accurate decreased only from 57% to 55% in wave 1 and from 59% to 57% in wave 2.

    “By contrast, the proportion of respondents who classified mainstream news as not very accurate or not at all accurate rather than somewhat or very accurate decreased only from 57 to 55% in wave 1 and 59 to 57% in wave 2.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The headlines classified as false are false and those classified as mainstream are mainstream.

  • May combine two claims.

    Findingown data

    In the India online sample wave 1, the perceived accuracy of mainstream news decreased (ITT β=−0.071, SE=0.025, P<0.01; ATT β=−0.259, SE=0.095, P<0.01), although by less than the effect observed for false news.

    “Instead, the perceived accuracy of mainstream news decreased, although by less than the effect observed for false news (ITT, β=−0.071 [SE = 0.025], P<0.01; ATT, β=−0.259 [SE = 0.095], P<0.01).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The headlines classified as false are false and those classified as mainstream are mainstream.
    • Rests on an assumption: The comprehension checks correctly identify respondents who actually received the treatment.

  • May combine two claims.

    Findingown data

    In the US survey, the intervention significantly increased sharing intentions for mainstream news and decreased sharing intentions for hyperpartisan news.

    “However, the intervention did significantly increase sharing intentions for mainstream news and decrease sharing intentions for hyperpartisan news.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    In the US wave 2 survey, the perceived accuracy difference between mainstream and false headlines remained statistically distinguishable from zero, although its magnitude decayed (β=0.050, SE=0.020, P<0.05).

    “As a result, the perceived accuracy difference between mainstream and false headlines remained statistically distinguishable from zero in the second wave, although its magnitude decayed (β=0.050; SE=0.020; P<0.05).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The headlines classified as false are false and those classified as mainstream are mainstream.

  • May combine two claims.

    Findingreasoning

    Respondents' overall ability to distinguish between mainstream and untrustworthy news increased by more than 26% in the US sample and 17% in the highly educated online Indian sample.

    “respondents’ overall ability to distinguish between mainstream and untrustworthy news increases by more than 26% in the US sample and 17% in the highly educated online Indian sample.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    The media literacy intervention reduced the perceived accuracy of mainstream news headlines in both the US and India online surveys, with a small but measurable negative effect.

    “the media literacy intervention reduced the perceived accuracy of these headlines in both the US and India online surveys.”
    • Also stated in: § Conclusion · plain
      “it did have a small but measurable negative effect on the perceived accuracy of mainstream news stories.”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    In the India online sample wave 1, respondents better distinguished between mainstream and false articles (ITT β=0.063, SE=0.025, P<0.05; ATT β=0.221, SE=0.088, P<0.05), a 17.3% improvement in discernment relative to the control condition.

    “Results again mirror the US study for H3—respondents better distinguished between mainstream and false articles (ITT, β=0.063 [SE = 0.025], P<0.05; ATT, β=0.221 [SE = 0.088], P<0.05).§”
    • Also stated in: § India Survey Experiments. · plain
      “While the magnitude of this effect is lower than for the US sample, it translates to a 17.3% improvement in discernment between mainstream and false news relative to the difference observed in the control condition.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The headlines classified as false are false and those classified as mainstream are mainstream.
    • Rests on an assumption: The comprehension checks correctly identify respondents who actually received the treatment.
    • Rests on an assumption: Respondents apply the intervention's lessons to the headline rating task even though they were not instructed to do so.

  • May combine two claims.

    Findingown data

    In the India face-to-face sample, there was no evidence that the treatment increased the perceived accuracy of mainstream news articles as predicted by H2, and it did not reduce the perceived accuracy of these headlines either.

    “As shown in Table 2, we find no evidence that the treatment increased the perceived accuracy of mainstream news articles as predicted by H2. However, it did not reduce the perceived accuracy of these headlines either as we found in the United States and online studies.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The headlines classified as false are false and those classified as mainstream are mainstream.

  • May combine two claims.

    Findingown data

    In the India face-to-face sample, there was no evidence that the media literacy treatment systematically affected beliefs in false news stories (H1) or discrimination between false and mainstream news (H3).

    “In addition, unlike the other studies, we find no evidence that the media literacy treatment systematically affected beliefs in false news stories (H1) or discrimination between false and mainstream news (H3) among India face-to-face respondents.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The headlines classified as false are false and those classified as mainstream are mainstream.

  • May combine two claims.

    Findingown data

    The effects decayed over time, diminishing in magnitude in the United States and no longer remaining statistically measurable in the India online study.

    “the effects decayed over time (diminishing in magnitude in the United States and no longer remaining statistically measurable in the India online study)”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The effect sizes were modest; the intervention did not eliminate belief in false news headlines.

    “the effect sizes were modest; this simple intervention did not eliminate belief in false news headlines.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The treatment increased respondents' intent to share mainstream news and decreased intent to share hyperpartisan news.

    “we do find evidence that the treatment increased respondents’ intent to share mainstream news and decreased intent to share hyperpartisan news”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    A brief intervention that could be inexpensively disseminated at scale can be effective at reducing the perceived accuracy of false news stories and helping users more accurately gauge the credibility of news content on different topics or issues.

    “these results show that a brief intervention which could be inexpensively disseminated at scale can be effective at reducing the perceived accuracy of false news stories, helping users more accurately gauge the credibility of news content they encounter on different topics or issues.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Some explanations for belief in misinformation identify factors that resist intervention, while others propose policy approaches that are effective in practice but difficult to scale.

    “Some explanations for belief in misinformation identify factors that resist intervention (4), while others propose policy approaches that are effective in practice but difficult to scale (34, 44).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [4] G. Pennycook, D. G. Rand, Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning. Cognition 188, 39– 50 (2018). 29935897 source

      Not checked in the free run.

    • cites [34] K. Clayton, Real solutions for fake news? Measuring the effectiveness of general warnings and fact-check tags in reducing belief in false stories on social media. Polit. Behav., 10.1007/s11109-019-09533-0 (2019). source

      Not checked in the free run.

    • cites [44] G. Pennycook, T. D. Cannon, D. G. Rand, Prior exposure increases perceived accuracy of fake news. J. Exp. Psychol. Gen. 147, 1865– 1880 (2018). 30247057 PMC6279465 source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    A simple, scalable media literacy intervention can decrease the perceived accuracy of false news content and help people better distinguish it from factual mainstream news in both the United States and India.

    “We find that a simple, scalable media literacy intervention can decrease the perceived accuracy of false news content and help people to better distinguish it from factual mainstream news in both the United States and India.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The order of the headlines was randomized within wave for each respondent.

    “The order of the headlines was randomized within wave for each respondent.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Headlines were presented as they would appear on the Facebook news feed to replicate a typical decision environment.

    “Headlines were presented as they would appear on the Facebook news feed to replicate a typical decision environment.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    US data collection was approved by the Institutional Review Boards at the University of Michigan, Washington University in St. Louis, and Princeton University.

    “US data collection was approved by the Institutional Review Boards (IRBs) at the University of Michigan (HUM00153414), Washington University in St. Louis (201806142), and Princeton University (10875).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The India study preanalysis plan is available at https://osf.io/97rnz.

    “The India study preanalysis plan is available at https://osf.io/97rnz.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    India data collection was approved by the IRBs at the University of Michigan, Ashoka University, and Morsel Research and Development.

    “India data collection was approved by the IRBs at the University of Michigan (HUM00160358), Ashoka University, and Morsel Research and Development (HIRB000007598).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The US study preanalysis plan is available at https://osf.io/u3sgc.

    “The US study preanalysis plan is available at https://osf.io/u3sgc.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Excluding respondents who indicated they mostly or always give humorous or insincere answers deviates from preregistration, but the Table 2 results are robust to including these respondents.

    “Respondents were excluded if indicated they mostly or always give humorous or insincere answers to survey questions (which amounted to 7–8% of responses in the online survey by wave compared to less than 1% in the face-to-face survey; this exclusion represents a deviation from our preregistration, but the results in Table 2 are robust to including these respondents).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    These findings are the most compelling demonstration to date that a real-world digital literacy intervention can have significant and potentially lasting effects.

    “these findings are the most compelling demonstration to date that a real-world digital literacy intervention can have significant and potentially lasting effects.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    Media literacy interventions may prove to be a vital and cost-effective way to reduce people's vulnerability to false news and thereby improve the information health of democracies.

    “they may prove to be a vital and cost-effective way to reduce people’s vulnerability to false news and thereby improve the information health of democracies.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    All subjects in the US and India studies gave consent to participate.

    “All subjects gave consent to participate in our study.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The representative random sample for the India face-to-face survey was drawn from the public voter registration list for the four constituencies.

    “The representative random sample for the India face-to-face survey was drawn from the public voter registration list for these constituencies”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Respondents in wave 1 of the US and India studies were randomly assigned with probability 0.5 to be exposed to a set of tips for distinguishing false news stories from mainstream stories.

    “We randomly assigned respondents in wave 1 of the US and India studies with probability 0.5 to be exposed to a set of tips for distinguishing false news stories from mainstream stories.”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Interpretationreasoning

    The ATT estimates likely understate the effects because a small fraction of respondents may be misclassified as compliers and some classified as receiving treatment are always takers.

    “Our ATT estimates likely understate effects for two reasons. A small fraction of respondents may be misclassified as compliers because they guessed correctly on all of the comprehension checks, which will diminish our ATT estimates relative to the true effect”
    • Also stated in: § Analysis. · plain
      “As such, our ATT estimates will understate the true effect.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: Always takers would not be affected by the intervention.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Nationalism is commonly linked to misinformation in India.

    “Nationalism is also commonly linked to misinformation in India (59).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [59] S. Chakrabarti, Nationalism a driving force behind fake news in India, research shows. BBC News, 12 November 2018. https://www.bbc.com/news/world-46146877. Accessed 9 September 2019.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    India and the United States are broadly considered the poorest and richest countries, respectively, in terms of income per capita among longstanding large democracies.

    “India and the United States are broadly considered the poorest and richest countries, respectively, in terms of income per capita among longstanding large democracies (ref. 56, p. 42).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites ref. 56, p. 42

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    India is likely to have lower levels of education and media literacy than the United States, which raises questions about the efficacy of any media intervention.

    “India is likely to have lower levels of education and media literacy than the United States, which raises questions about the efficacy of any media intervention.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductnone given

    All headlines were published by actual news sources or circulated on Facebook or WhatsApp within 6 months of the respective survey, and a portion were rated false by at least one third-party fact-checking organization.

    “All of the headlines were published by actual news sources or circulated on Facebook or WhatsApp within 6 mo of the respective survey, and a portion of the headlines were rated as false by at least one third-party fact-checking organization.”
    Quote in the paperSubstantial claim
35 of 35

  • The paper relies on this without support.

    The headlines classified as false are false and those classified as mainstream are mainstream.

    “The dependent variable for the difference in perceived false versus mainstream news accuracy is calculated at the respondent level as the mean difference in perceived accuracy between all false and all mainstream news headlines viewed.”

  • The paper relies on this without support.

    The comprehension checks correctly identify respondents who actually received the treatment.

    “our indicator for receipt of treatment is the ability to correctly answer a series of follow-up questions about the content of the news tips (approximately two-thirds of respondents in the treatment condition [66%] were successfully treated) and our instrument is the original random assignment.”

  • The paper relies on this without support.

    Respondents apply the intervention's lessons to the headline rating task even though they were not instructed to do so.

    “Although our study does not instruct respondents to apply the lessons from the intervention to the headline ratings task, the effect sizes are comparable to the estimated effect of exposure to the labels Facebook initially used to indicate that articles were disputed by fact checkers”

  • The paper relies on this without support.

    Many real-world Facebook users likely ignored the Tips to Spot False News link when the company offered it in their feeds.

    “many real-world Facebook users are likely to have ignored the Tips to Spot False News link when the company offered it in their feeds.”

  • The authors state this as an assumption.

    Always takers would not be affected by the intervention.

    “The intervention should have no effect on them.”

  • The paper relies on this without support.

    Information habits are ingrained and difficult to alter.

    “perhaps because information habits are ingrained and difficult to alter.”

  • The paper relies on this without support.

    Higher religious polarization increases demand for and belief in online misinformation and rumors.

    “These locations were chosen, in part, due to their higher levels of religious polarization, which we anticipated might increase demand for and belief in online misinformation and rumors.”
7 of 7

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The effect sizes are generally larger than those of existing scalable interventions; digital literacy tips improved detection of false headlines with a Cohen's d of 0.20.

    “These effect sizes are also generally larger than those of existing scalable interventions. For comparison, digital literacy tips (consisting of a short text pointing out various ways to spot false news, such as looking at the validity of the source) were found to improve detection of false headlines with a Cohen’s d of 0.20 (45).”

    From Psychological inoculation improves resilience against misinformation on social media (2022) · cites this paper as [45]

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. V. Narayanan “News and information over Facebook and WhatsApp during the Indian election campaign” (Data Memo 2019.2 Project on Computational Propaganda Oxford UK 2019).
  2. 10.1080/09546553.2019.1693370 · DOI · Paper page

Cited by

Sorted by citations

Related

Authors
Guess, A. M., Lerner, M., Lyons, B., Montgomery, J. M., Nyhan, B., Reifler, J., & Sircar, N.
Affiliations
  • Princeton University (US)
  • University of Michigan (US)
  • University of Utah (US)
  • Washington University in St. Louis (US)
  • Dartmouth College (US)
  • University of Exeter (GB)
  • Ashoka University (IN)
Corresponding author
Guess, Andrew M.
ORCID
Venue
Proceedings of the National Academy of Sciences
Publisher
National Academy of Sciences
Published
2020-06-22
Volume
117
Issue
27
Pages
15536-15545
ISSN
  • 0027-8424 (ISSN-L)
  • 1091-6490
DOI
10.1073/pnas.1920498117
PMC
PMC7355018
OpenAlex
W3037661194
Semantic Scholar
415450e2590f080c9262c9da5bf1043a86725ae1
Cited by
773
References
36
Citations by year
2026: 121 · 2025: 146 · 2024: 149 · 2023: 138 · 2022: 127 · 2021: 75 · 2020: 15 · 2016: 2
FWCI
118.96
Type
journal article
Language
en
Update notices
correction (2023-01-17)
Open access
Licence
Locations
  • Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • Open Research Exeter (University of Exeter) (publishedVersion, cc-by-nc-nd)
  • Figshare (submittedVersion, cc-by-nc-nd)
Funders
  • Brigham Young University
  • George Washington University
  • University of Washington
  • KU Leuven
  • Göteborgs Universitet
  • Harvard Kennedy School
Topics
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Social Sciences › Arts and Humanities › Literature and Literary Theory › Literacy, Media, and Education
  • Social Sciences › Sociology and Political Science › Islamic Finance and Communication
Keywords
digital media literacy, misinformation, fake news, media literacy interventions, election news, cross-national comparison, United States, India
MeSH terms
Literacy, Internet-Based Intervention, Adolescent, Adult, Communication, Female, Humans, India, Male, Middle Aged, Surveys and Questionnaires, Technology, United States, Young Adult, Social Media

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review