News credibility labels have limited average effects on news diet quality and fail to reduce misperceptions
Open accessThe DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
As the primary arena for viral misinformation shifts toward transnational threats, the search continues for scalable countermeasures compatible with principles of transparency and free expression. We conducted a randomized field experiment evaluating the impact of source credibility labels embedded in users’ social feeds and search results pages. By combining representative surveys ( n = 3337) and digital trace data ( n = 968) from a subset of respondents, we provide a rare ecologically valid test of such an intervention on both attitudes and behavior. On average across the sample, we are unable to detect changes in real-world consumption of news from low-quality sources after 3 weeks. We can also rule out small effects on perceived accuracy of popular misinformation spread about the Black Lives Matter movement and coronavirus disease 2019. However, we present suggestive evidence of a substantively meaningful increase in news diet quality among the heaviest consumers of misinformation. We discuss the implications of our findings for scholars and practitioners.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847
Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Imbens, G. W., & Rubin, D. B. (2015) · doi:10.1017/cbo9781139025751 · Cited by 4,210
Lazer, D. M. J., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., Metzger, M. J., Nyhan, B., Pennycook, G., Rothschild, D., et al. (2018) · Science · doi:10.1126/science.aao2998 · Cited by 4,160
Equivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses
Lakens, D. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617697177 · Cited by 3,001
Red Media, Blue Media: Evidence of Ideological Selectivity in Media Use
Iyengar, S., & Hahn, K. S. (2009) · Journal of Communication · doi:10.1111/j.1460-2466.2008.01402.x · Cited by 1,940
Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA
Loomba, S., de Figueiredo, A., Piatek, S. J., de Graaf, K., & Larson, H. J. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01056-1 · Cited by 1,800 · cited 1×
“3 of 10 the most commonly studied phenomenon (6), which can result from direct exposure to online misinformation (32, 33).”
BackgroundLess than you think: Prevalence and predictors of fake news dissemination on Facebook
Guess, A., Nagler, J., & Tucker, J. (2019) · Science Advances · doi:10.1126/sciadv.aau4586 · Cited by 1,652
Self-citationShifting attention to accuracy can reduce misinformation online
Pennycook, G., Epstein, Z., Mosleh, M., Arechar, A. A., Eckles, D., & Rand, D. G. (2021) · Nature · doi:10.1038/s41586-021-03344-2 · Cited by 1,155
Hainmueller, J., Mummolo, J., & Xu, Y. (2019) · Political Analysis · doi:10.1017/pan.2018.46 · Cited by 959 · cited 1×
“(55) explain, models of this kind assume a constant change across values of the moderator and require sufficient common support.”
BackgroundFighting misinformation on social media using crowdsourced judgments of news source quality
Pennycook, G., & Rand, D. G. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1806781116 · Cited by 947 · cited 1×
“This approach builds on humans’ tendency to rely on cognitive shortcuts and heuristics, which, depending on context, can be relatively informative (17, 18) or potentially distorting (19).”
BackgroundMessing, S., & Westwood, S. J. (2014) · Communication Research · doi:10.1177/0093650212466406 · Cited by 902
Clayton, K., Blair, S., Busam, J. A., Forstner, S., Glance, J., Green, G., Kawata, A., Kovvuri, A., Martin, J., Morgan, E., et al. (2020) · Political Behavior · doi:10.1007/s11109-019-09533-0 · Cited by 658
OSMUNDSEN, M., BOR, A., VAHLSTRUP, P. B., BECHMANN, A., & PETERSEN, M. B. (2021) · American Political Science Review · doi:10.1017/s0003055421000290 · Cited by 614
GERBER, A. S., GIMPEL, J. G., GREEN, D. P., & SHAW, D. R. (2011) · American Political Science Review · doi:10.1017/s000305541000047x · Cited by 596 · cited 1×
“Using the measurement of news consumption in the third period, we leverage the exogenous disabling of NewsGuard’s free capabilities on July 1 to determine whether the behavioral effects of this intervention decay after its features are no longer available (40) or whether the intervention has more durable effects simil…”
MethodExposure to untrustworthy websites in the 2016 US election
Guess, A. M., Nyhan, B., & Reifler, J. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-0833-x · Cited by 587
Self-citationEvaluating the fake news problem at the scale of the information ecosystem
Allen, J., Howland, B., Mobius, M., Rothschild, D., & Watts, D. J. (2020) · Science Advances · doi:10.1126/sciadv.aay3539 · Cited by 528 · cited 1×
“Although we do not uncover statistically significant average treatment effects—potentially because of limited exposure to low-credibility sites to begin with (52)—we present suggestive evidence of a substantively meaningful boost in news quality among the heaviest consumers of misinformation in our sample (who compris…”
Result(Almost) Everything in Moderation: New Evidence on Americans' Online Media Diets
Guess, A. M. (2021) · American Journal of Political Science · doi:10.1111/ajps.12589 · Cited by 342
Self-citationOgnyanova, K., Lazer, D., Robertson, R. E., & Wilson, C. (2020) · Harvard Kennedy School Misinformation Review · doi:10.37016/mr-2020-024 · Cited by 261
How behavioural sciences can promote truth, autonomy and democratic discourse online
Lorenz-Spreen, P., Lewandowsky, S., Sunstein, C. R., & Hertwig, R. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-0889-7 · Cited by 223
Emphasizing publishers does not effectively reduce susceptibility to misinformation on social media
Dias, N., Pennycook, G., & Rand, D. G. (2020) · Harvard Kennedy School Misinformation Review · doi:10.37016/mr-2020-001 · Cited by 145 · cited 1×
“Although experimental demonstrations of source label effects on selective exposure are well known (25), studies have failed to convincingly show that these cues affect susceptibility to misinformation (26, 27).”
BackgroundLau, R. R., Andersen, D. J., Ditonto, T. M., Kleinberg, M. S., & Redlawsk, D. P. (2017) · Political Behavior · doi:10.1007/s11109-016-9354-8 · Cited by 127
Guess, A. M., Lockett, D., Lyons, B., Montgomery, J. M., Nyhan, B., & Reifler, J. (2020) · Harvard Kennedy School Misinformation Review · doi:10.37016/mr-2020-004 · Cited by 69
Self-citation
Cited by
Martel, C., & Rand, D. G. (2023) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2023.101710 · Cited by 122
Altay, S., Kleis Nielsen, R., & Fletcher, R. (2022) · Journal of Quantitative Description: Digital Media · doi:10.51685/jqd.2022.020 · Cited by 88
Crowds Can Effectively Identify Misinformation at Scale
Martel, C., Allen, J., Pennycook, G., & Rand, D. G. (2024) · Perspectives on Psychological Science · doi:10.1177/17456916231190388 · Cited by 84
Toxic Content and User Engagement on Social Media: Evidence from a Field Experiment
Beknazar-Yuzbashev, G., Jiménez Durán, R., McCrosky, J., & Stalinski, M. (2022) · doi:10.2139/ssrn.4307346 · Cited by 42
How Effective Are Interventions Against Misinformation?
Altay, S. (2022) · doi:10.31234/osf.io/sm3vk · Cited by 32
Metadata
- Authors
- Aslett, K., Guess, A. M., Bonneau, R., Nagler, J., & Tucker, J. A.
- Venue
- Science Advances
- Published
- 2022-05-06
- Volume
- 8
- Issue
- 18
- Pages
- eabl3844-eabl3844
- ISSN
- 2375-2548 (ISSN-L)
- OpenAlex
- W4229008296
- Cited by
- 102
- References
- 22
- FWCI
- 31.90
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.