CoChat
Sign Up
Cited by 62Misinformation and Its ImpactsCommunication and COVID-19 ImpactVaccine Coverage and Hesitancy

The efficacy of Facebook’s vaccine misinformation policies and architecture during the COVID-19 pandemic

Open access
Broniatowski, David A., Simons, Joseph R., Gu, Jiayan, Jamison, Amelia M., Abroms, Lorien C.·2023·Science Advances
DOI 10.1126/sciadv.adh2132PMC PMC11044214
Open this paper in CoChat — free accountOpen PDF

Abstract

Online misinformation promotes distrust in science, undermines public health, and may drive civil unrest. During the coronavirus disease 2019 pandemic, Facebook—the world’s largest social media company—began to remove vaccine misinformation as a matter of policy. We evaluated the efficacy of these policies using a comparative interrupted time-series design. We found that Facebook removed some antivaccine content, but we did not observe decreases in overall engagement with antivaccine content. Provaccine content was also removed, and antivaccine content became more misinformative, more politically polarized, and more likely to be seen in users’ newsfeeds. We explain these findings as a consequence of Facebook’s system architecture, which provides substantial flexibility to motivated users who wish to disseminate misinformation through multiple channels. Facebook’s architecture may therefore afford antivaccine content producers several means to circumvent the intent of misinformation removal policies.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1126/sciadv.adh2132

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • What makes misinformation consumers special is not privileged access to misinformation but traits that make them more likely to seek out misinformation, such as having low trust in institutions or being strong partisans.

    “What makes misinformation consumers special is not that they have privileged access to misinformation but traits that make them more likely to seek out misinformation (Broniatowski et al., 2023; Motta et al., 2023), such as having low trust in institutions or being strong partisans (Osmundsen et al., 2021).”

    From Misinformation reloaded? Fears about the impact of generative AI on misinformation are overblown (2023) · cites this paper as Broniatowski et al., 2023

    The Results section, “Motivated users may still seek out misinformation” (page unavailable), reports that removals may not have dissuaded audiences from seeking antivaccine misinformation; its model suggests demand shifted to remaining pages and groups. This supports the role of motivation, but the paper does not establish that misinformation consumers differ from others because of their traits rather than privileged access. The examples of low institutional trust and strong partisanship are attached to a different citation in the citing sentence. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 49

  1. [1]J. D. West, C. T. Bergstrom, Misinformation in and about science. Proc. Natl. Acad. Sci. U.S.A. 118, e1912444117 (2021). 33837146 PMC8054004
  2. [2]W.-Y. S. Chou, A. Oh, W. M. P. Klein, Addressing health-related misinformation on social media. JAMA 320, 2417– 2418 (2018). 30428002
  3. [6]P. Ball, A. Maxmen, The epic battle against coronavirus misinformation and conspiracy theories. Nature 581, 371– 374 (2020). 32461658
  4. [10]O. Papakyriakopoulos, E. Goodman, The impact of Twitter labels on misinformation spread and user engagement: Lessons from Trump’s election tweets, in Proceedings of the ACM Web Conference 2022 (Association for Computing Machinery, 2022), pp. 2541–2551; 10.1145/3485447.3512126. · DOI
  5. [11]J. Nassetta, K. Gross, “State media warning labels can counteract the effects of foreign disinformation,” Harvard Kennedy School Misinformation Review, 30 October 2020.
  6. [12]J. Gu, A. Dor, K. Li, D. A. Broniatowski, M. Hatheway, L. Fritz, L. C. Abroms, The impact of Facebook’s vaccine misinformation policy on user endorsements of vaccine content: An interrupted time series analysis. Vaccine 40, 2209– 2214 (2022). 35246311
  7. [14]J. Roozenbeek, A. L. J. Freeman, S. van der Linden, How accurate are accuracy-nudge interventions? A preregistered direct replication of Pennycook et al. (2020). Psychol. Sci. 32, 1169– 1178 (2021). PMC8641132 34114521
  8. [15]E. Goldman, Content moderation remedies. Mich. Tech. L. Rev. 28, 1 (2021).
  9. [16]S. Jhaver, C. Boylston, D. Yang, A. Bruckman, Evaluating the effectiveness of deplatforming as a moderation strategy on Twitter. Proc. ACM Hum. Comput. Interact. 5, 1– 30 (2021).36644216
  10. [18]E. Chandrasekharan, U. Pavalanathan, A. Srinivasan, A. Glynn, J. Eisenstein, E. Gilbert, You can’t stay here. Proc. ACM Hum.-Comput. Interact. 1, 1– 22 (2017).
  11. [19]J. B. Bak-Coleman, I. Kennedy, M. Wack, A. Beers, J. S. Schafer, E. S. Spiro, K. Starbird, J. D. West, Combining interventions to reduce the spread of viral misinformation. Nat. Hum. Behav. 6, 1372– 1380 (2022). 35739250 PMC9584817
  12. [20]D. A. Broniatowski, M. Dredze, J. W. Ayers, “First do no harm”: Effective communication about COVID-19 vaccines. Am. J. Public Health 111, 1055– 1057 (2021). 33950727 PMC8101564
  13. [21]R. Gorwa, R. Binns, C. Katzenbach, Algorithmic content moderation: Technical and political challenges in the automation of platform governance. Big Data Soc. 7, 10.1177/2053951719897945, (2020). · DOI · Paper page
  14. [22]T. Gillespie, Content moderation, AI, and the question of scale. Big Data Soc. 7, 10.1177/205395172094, (2020). · DOI
  15. [24]D. R. Thomas, L. A. Wahedi, Disrupting hate: The effect of deplatforming hate organizations on their online audience. Proc. Natl. Acad. Sci. U.S.A. 120, e2214080120 (2023). 37276418 PMC10268571
  16. [27]D. A. Broniatowski, V. F. Reyna, To illuminate and motivate: A fuzzy-trace model of the spread of information online. Comput. Math. Organ. Theory 26, 431– 464 (2020). 33737859 PMC7962747
  17. [30]V. F. Reyna, Social media: Why sharing interferes with telling true from false. Sci. Adv. 9, eadg8333 (2023). 36867696 PMC9984168
  18. [33]R. E. Robertson, J. Green, D. J. Ruck, K. Ognyanova, C. Wilson, D. Lazer, Users choose to engage with more partisan news than they are exposed to on Google Search. Nature 618, 342– 348 (2023). 37225979
  19. [35]J. Moses, Flexibility and its relation to complexity and architecture, in Complex Systems Design & Management, M. Aiguier, F. Bretaudeau, D. Krob, Eds. (Springer, 2010), pp. 197–206; http://link.springer.com/chapter/10.1007/978-3-642-15654-0_14. · DOI
  20. [37]“Facebook reports fourth quarter and full year 2020 results”; https://investor.fb.com/investor-news/press-release-details/2021/Facebook-Reports-Fourth-Quarter-and-Full-Year-2020-Results/default.aspx.
  21. [46]N. Grinberg, K. Joseph, L. Friedland, B. Swire-Thompson, D. Lazer, Fake news on Twitter during the 2016 U.S. presidential election. Science 363, 374– 378 (2019). 30679368
  22. [47]C. Shao, G. L. Ciampaglia, O. Varol, K.-C. Yang, A. Flammini, F. Menczer, The spread of low-credibility content by social bots. Nat. Commun. 9, 4787 (2018). 30459415 PMC6246561
  23. [50]Y. Zhou, M. Dredze, D. A. Broniatowski, W. D. Adler, Elites and foreign actors among the alt-right: The gab social media platform. First Monday 24, 10.5210/fm.v24i9.10062, (2019). · DOI
  24. [51]M. Trujillo, M. Gruppi, C. Buntain, B. D. Horne, What is BitChute? Characterizing the “Free Speech” alternative to YouTube, in Proceedings of the 31st ACM Conference on Hypertext and Social Media (ACM, 2020), pp. 139–140; 10.1145/3372923.3404833. · DOI
  25. [52]K. Starbird, A. Arif, T. Wilson, Disinformation as collaborative work. Proc. ACM Hum. Comput. Interact. 3, 1– 26 (2019).34322658
  26. [53]N. F. Johnson, N. Velásquez, N. J. Restrepo, R. Leahy, N. Gabriel, S. El Oud, M. Zheng, P. Manrique, S. Wuchty, Y. Lupu, The online competition between pro- and anti-vaccination views. Nature 582, 230– 233 (2020). 32499650
  27. [56]D. A. Broniatowski, A. M. Jamison, N. F. Johnson, N. Velasquez, R. Leahy, N. J. Restrepo, M. Dredze, S. C. Quinn, Facebook pages, the “Disneyland” measles outbreak, and promotion of vaccine refusal as a civil right, 2009–2019. Am. J. Public Health 110, S312– S318 (2020). 33001718 PMC7532319
  28. [57]A. M. Jamison, D. A. Broniatowski, M. Dredze, A. Sangraula, M. C. Smith, S. C. Quinn, Not just conspiracy theories: Vaccine opponents and proponents add to the COVID-19 ‘infodemic’ on twitter. Harv. Kennedy Sch. Misinfo. Rev. 1, 10.37016/mr-2020-38, (2020). PMC8345314 34368805 · DOI
  29. [58]O. L. De Weck, D. Roos, C. L. Magee, Engineering Systems: Meeting Human Needs in a Complex Technological World (MIT Press, 2011).
  30. [63]M. Dredze, D. A. Broniatowski, M. C. Smith, K. M. Hilyard, Understanding vaccine refusal: Why we need social media now. Am. J. Prev. Med. 50, 550– 552 (2016). 26655067 PMC4801675
  31. [65]A. Jamison, D. A. Broniatowski, M. C. Smith, K. S. Parikh, A. Malik, M. Dredze, S. C. Quinn, Adapting and extending a typology to identify vaccine misinformation on Twitter. Am. J. Public Health 110, S331– S339 (2020). 33001737 PMC7532318
  32. [66]W. R. Shadish, T. D. Cook, D. T. Campbell, Experimental and Quasi-Experimental Designs for Generalized Causal Inference (Wadsworth Cengage learning, 2002); https://pdfs.semanticscholar.org/f141/aeffd3afcb0e76d5126bec9ee860336bee13.pdf.
  33. [69]T. R. Fanshawe, P. J. Turner, M. M. Gillespie, G. N. Hayward, The comparative interrupted time series design for assessment of diagnostic impact: Methodological considerations and an example using point-of-care C-reactive protein testing. Diagn. Progn. Res. 6, 3 (2022). 35232491 PMC8888027
  34. [70]J. B. Dimick, A. M. Ryan, Methods for evaluating changes in health care policy. JAMA 312, 2401– 2402 (2014). 25490331
  35. [72]S. Seabold, J. Perktold, statsmodels: Econometric and statistical modeling with Python, in Proceedings of the 9th Python in Science Conference (Austin, TX, 2010), pp. 10-25080.
  36. [75]D. Mimno, M. Lee, Low-dimensional embeddings for interpretable anchor-based topic inference, in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (2014), pp. 1319–1328.
  37. [80]M. Cinelli, W. Quattrociocchi, A. Galeazzi, C. M. Valensise, E. Brugnoli, A. L. Schmidt, P. Zola, F. Zollo, A. Scala, The COVID-19 social media infodemic. Sci. Rep. 10, 16598 (2020). 33024152 PMC7538912
  38. [81]L. Singh, L. Bode, C. Budak, K. Kawintiranon, C. Padden, E. Vraga, Understanding high- and low-quality URL sharing on COVID-19 twitter streams. J. Comput. Soc. Sci. 3, 343– 366 (2020). 33263092 PMC7691185
  39. [82]M. R. DeVerna, F. Pierri, B. T. Truong, J. Bollenbacher, D. Axelrod, N. Loynes, C. Torres-Lugo, K.-C. Yang, F. Menczer, J. Bryden, CoVaxxy: A collection of English-language Twitter posts about COVID-19 vaccines, in Proceedings of the International AAAI Conference on Web and Social Media (2021), pp. 992–999.
  40. [84]D. M. Lazer, M. A. Baum, Y. Benkler, A. J. Berinsky, K. M. Greenhill, F. Menczer, M. J. Metzger, B. Nyhan, G. Pennycook, D. Rothschild, The science of fake news. Science 359, 1094– 1096 (2018). 29590025
  41. [87]N. F. Johnson, R. Leahy, N. J. Restrepo, N. Velasquez, M. Zheng, P. Manrique, P. Devkota, S. Wuchty, Hidden resilience and adaptive dynamics of the global online hate ecology. Nature 573, 261– 265 (2019). 31435010
  42. [89]J. W. Ayers, B. Chu, Z. Zhu, E. C. Leas, D. M. Smith, M. Dredze, D. A. Broniatowski, Spread of misinformation about face masks and COVID-19 by automated software on Facebook. JAMA Intern. Med. 181, 1251– 1253 (2021). 34096988 PMC8185625
  43. [92]L. Nizzoli, S. Tardelli, M. Avvenuti, S. Cresci, M. Tesconi, Coordinated behavior on social media in 2019 UK general election, in Proceedings of the International AAAI Conference on Web and Social Media (2021), pp. 443–454.
  44. [93]D. Weber, F. Neumann, Amplifying influence through coordinated behaviour in social networks. Soc. Netw. Anal. Min. 11, 111 (2021). 34745379 PMC8557266
  45. [94]M. Okada, K. Yamanishi, N. Masuda, Long-tailed distributions of inter-event times as mixtures of exponential distributions. R. Soc. Open Sci. 7, 10.1098/rsos.191643, (2020). PMC7062064 32257326 · DOI
  46. [96]D. M. Blei, Probabilistic topic models. Commun. ACM 55, 77– 84 (2012).
  47. [98]H. M. Wallach, D. M. Mimno, A. McCallum, Rethinking LDA: Why priors matter, in Advances in Neural Information Processing Systems, Y. Bengio, D. Schuurmans, J. Lafferty, C. Williams, A. Culotta, Eds. (Curran Associates, Inc., 2009), vol. 22; https://proceedings.neurips.cc/paper_files/paper/2009/file/0d0871f0806eae32d30983b62252da50-Paper.pdf.
  48. [99]J. D. Hamilton, Time Series Analysis (Princeton University Press, 1st edition, 1994).
  49. [100]J. Durbin, S. J. Koopman, Time Series Analysis by State Space Methods (OUP Oxford, 2012).

Cited by

Sorted by citations

Authors
Broniatowski, D. A., Simons, J. R., Gu, J., Jamison, A. M., & Abroms, L. C.
Affiliations
  • George Washington University (US)
  • United States Department of Health and Human Services (US)
  • Johns Hopkins University (US)
Corresponding author
Broniatowski, David A.
ORCID
Venue
Science Advances
Publisher
American Association for the Advancement of Science (AAAS)
Published
2023-09-15
Volume
9
Issue
37
Pages
eadh2132-eadh2132
ISSN
2375-2548 (ISSN-L)
DOI
10.1126/sciadv.adh2132
PMC
PMC11044214
OpenAlex
W4386766469
Cited by
62
References
76
Citations by year
2026: 19 · 2025: 23 · 2024: 18 · 2023: 2
FWCI
26.64
Type
journal article
Language
en
Open access
Licence
cc-by-nc
Locations
  • Science Advances (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
Funders
  • National Science Foundation (2029420, 2229885)
  • John S. and James L. Knight Foundation
Topics
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Social Sciences › Communication › Communication and COVID-19 Impact
  • Social Sciences › Health › Vaccine Coverage and Hesitancy
Keywords
vaccine misinformation, Facebook, COVID-19 pandemic, political polarization, content engagement
MeSH terms
COVID-19, Humans, Vaccines, Policy, Pandemics, Social Media

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review