Removal of Anti-Vaccine Content Impacts Social Media Discourse
Open accessAbstract
Over the past several years, a growing number of social media platforms have begun taking an active role in content moderation and online speech regulation. While enforcement actions have been shown to improve outcomes within moderating platforms, less is known about possible spillover effects across platforms. We study the impact of removing groups promoting anti-vaccine content on Facebook on engagement with similar content on Twitter. We followed 160 Facebook groups discussing COVID-19 vaccines and prospectively tracked their removal from the platform between April and September 2021. We then identified users who cited these groups on Twitter, and examined their online behavior over time. Our findings from a stacked difference-in-differences analysis shows that users citing removed Facebook groups promoted more anti-vaccine content on Twitter in the month after the removals. In particular, users citing the removed groups used 10-33% more anti-vaccine keywords on Twitter, when compared to accounts citing groups that were not removed. Our results suggest that taking down anti-vaccine content on one platform can result in increased production of similar content on other platforms, raising questions about the overall effectiveness of these measures.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prior empirical studies have found that group-level sanctions sometimes lead to increases in anti-social behavior or rebellious non-compliance.
“Although sanctions are typically intended to promote normative compliance, group-level sanctions in some prior empirical studies have been met with increases in anti-social behavior or rebellious non-compliance (Mitts, Pisharody, and Shapiro 2022;Ribeiro et al. 2021;Russo et al. 2023).”
From The Effects of Group Sanctions on Participation and Toxicity: Quasi-experimental Evidence from the Fediverse (2024) · cites this paper as Mitts, Pisharody, and Shapiro 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Allcott, H., Boxell, L., Conway, J., Gentzkow, M., Thaler, M., & Yang, D. (2020) · Journal of Public Economics · doi:10.1016/j.jpubeco.2020.104254 · Cited by 999 · see citing works
Partisanship, health behavior, and policy attitudes in the early stages of the COVID-19 pandemic
Gadarian, S. K., Goodman, S. W., & Pepinsky, T. B. (2021) · PLOS ONE · doi:10.1371/journal.pone.0249596 · Cited by 744 · see citing works
Grossman, G., Kim, S., Rexer, J. M., & Thirumurthy, H. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2007835117 · Cited by 564 · see citing works
Lee, J. J., Kang, K., Wang, M. P., Zhao, S. Z., Wong, J. Y. H., O’Connor, S., Yang, S. C., & Shin, S. H. (2020) · Journal of Medical Internet Research · doi:10.2196/22205 · Cited by 317 · see citing works
Jamison, A., Broniatowski, D. A., Dredze, M. H., Sangraula, A., Smith, M. C., & Quinn, S. C. (2020) · Harvard Kennedy School Misinformation Review · doi:10.37016/mr-2020-38 · Cited by 177 · see citing works
Political beliefs affect compliance with government mandates
Painter, M., & Qiu, T. (2021) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2021.03.019 · Cited by 143 · see citing works
Wong, F. H. C., Liu, T., Leung, D. K. Y., Zhang, A. Y., Au, W. S. H., Kwok, W. W., Shum, A. K. Y., Wong, G. H. Y., & Lum, T. Y. S. (2021) · Journal of Medical Internet Research · doi:10.2196/26570 · Cited by 106 · see citing works
Nuzhath, T., Tasnim, S., Sanjwal, R. K., Trisha, N. F., Rahman, M., Mahmud, F., Arman, A., Chakraborty, S., & Hossain, M. M. (2020) · doi:10.31235/osf.io/vc9jb · Cited by 80 · see citing works
Science skepticism reduced compliance with COVID-19 shelter-in-place policies in the United States
Brzezinski, A., Kecht, V., Dijcke, D. V., & Wright, A. L. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01227-0 · Cited by 63 · see citing works
Pennycook, G., McPhetres, J., Bagó, B., & Rand, D. (2020) · doi:10.31234/osf.io/zhjkp · Cited by 47 · see citing works
Political polarization, social fragmentation, and cooperation during a pandemicPreprint
Cornelson, K., & Miloucheva, B. (2020) · RePEc: Research Papers in Economics · Cited by 44 · see citing works
Dismantling the anti-vaxx industry
Ahmed, I. (2021) · Nature Medicine · doi:10.1038/s41591-021-01260-6 · Cited by 37 · see citing works
McKinley, C. J., & Lauby, F. (2021) · International journal of communication · Cited by 20 · see citing works
Cable News and COVID-19 Vaccine CompliancePreprint
Pinna, M., Picard, L., & Goessmann, C. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3890340 · Cited by 19 · see citing works
Comment on “The COVID-19 infodemic does not affect vaccine acceptance”Preprint
Gallotti, R., Sacco, P. L., & Domenico, M. D. (2021) · doi:10.31219/osf.io/m8j32 · Cited by 5 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- Joshua Clinton, Jon Cohen, John Lapinski, and Marc Trussler. 2021. Partisan pandemic: How partisanship and public health concerns affect individuals' social mobility during COVID-19. Science advances 7, 2 (2021), eabd7204.
- Anton Gollwitzer, Cameron Martel, William J Brady, Philip Pärnamets, Isaac G Freedman, Eric D Knowles, and Jay J Van Bavel. 2020. Partisan differences in physical distancing are linked to health outcomes during the COVID-19 pandemic. Nature human behaviour 4, 11 (2020), 1186-1197.
- Heidi Oi-Yee Li, Adrian Bailey, David Huynh, and James Chan. 2020. YouTube as a source of information on COVID-19: a pandemic of misinformation? BMJ global health 5, 5 (2020), e002604.
- Orestis Papakyriakopoulos, Juan Carlos Medina Serrano, and Simon Hegelich. 2020. The spread of COVID-19 conspiracy theories on social media and the effect of content moderation. Harvard Kennedy School Misinformation Review 10 (2020).
- Francesco Pierri, Brea Perry, Matthew R DeVerna, Kai-Cheng Yang, Alessan- dro Flammini, Filippo Menczer, and John Bryden. 2021. The impact of online misinformation on US COVID-19 vaccinations. arXiv preprint arXiv:2104.10635 (2021). · arXiv
Cited by
Sorted by citations
Broniatowski, D. A., Simons, J. R., Gu, J., Jamison, A. M., & Abroms, L. C. (2023) · Science Advances · doi:10.1126/sciadv.adh2132 · Cited by 62 · see citing works
Disrupting hate: The effect of deplatforming hate organizations on their online audience
Thomas, D. R., & Wahedi, L. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2214080120 · Cited by 26 · see citing works
Beyond phase-in: assessing impacts on disinformation of the EU Digital Services Act
Nannini, L., Bonel, E., Bassi, D., & Maggini, M. J. (2024) · AI and Ethics · doi:10.1007/s43681-024-00467-w · Cited by 31 · see citing works
Berger, L., Kerkhof, A., Mindl, F., & Münster, J. (2025) · Journal of Public Economics · doi:10.1016/j.jpubeco.2025.105345 · Cited by 32 · see citing works
Gatta, V., Luceri, L., Fabbri, F., & Ferrara, E. (2023) · ACM Conference on Hypertext & Social Media · doi:10.1145/3603163.3609058 · Cited by 26 · see citing works
Combating Online Hate Speech: The Impact of Legislation on TwitterPreprint
Andres, R., & Slivko, O. (2021) · Social Science Research Network · doi:10.2139/ssrn.4013662 · Cited by 10 · see citing works
Klinenberg, D. (2023) · Journal of Conflict Resolution · doi:10.1177/00220027231188909 · Cited by 8 · see citing works
Current engagement with unreliable sites from web search driven by navigational search
Greene, K. T., Pisharody, N., Meyer, L. A., Pereira, M. W., Dodhia, R. M., Ferres, J. L., & Shapiro, J. N. (2024) · Science Advances · doi:10.1126/sciadv.adn3750 · Cited by 7 · see citing works
Self-citationColglazier, C., TeBlunthuis, N., & Shaw, A. (2024) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v18i1.31316 · Cited by 3 · see citing works
Niche Dynamics in Complex Online Community Ecosystems
TeBlunthuis, N. (2025) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v19i1.35907 · Cited by 1 · see citing works
Explaining Twitter’s inability to effectively moderate content during the COVID-19 pandemic
Broniatowski, D. A., Zhong, W., Simons, J. R., Jamison, A. M., Dredze, M., & Abroms, L. (2025) · Scientific Reports · doi:10.1038/s41598-025-20033-6 · Cited by 2 · see citing works
de Oliveira, G. S., da Silva, A. P. C., Murai, F., & Ferreira, C. H. (2026) · ACM Web Science Conference (WebSci) · doi:10.1145/3795766.3799768 · Cited by 1 · see citing works
Users, Technologies and Regulations: A Sociotechnical Analysis of False Information on MIMS in Asia
Goh, S. (2023) · Mobile communication in Asia · doi:10.1007/978-94-024-2225-2_7 · Cited by 1 · see citing works
Buntain, C. (2022) · IEEE International Conference on Collaboration and Internet Computing (CIC) · doi:10.1109/cic56439.2022.00017
Where Does the Hate Flow? The Impact of Multihoming on User Responses to Content ModerationPreprint
Mudambi, M., Clark, J., Rhue, L., & Viswanathan, S. (2025) · Social Science Research Network · doi:10.2139/ssrn.5093800
Instrumenting the Online Information Environment
Shapiro, J. N. (2024) · The Annals of the American Academy of Political and Social Science · doi:10.1177/00027162251397820
Self-citationLi, Y., Chen, T., Zhao, Y., Ke, W., Pang, P., Mckay, D., Chang, S., & Baxter, N. (2026) · PLoS ONE · doi:10.1371/journal.pone.0346568
- Authors
- Mitts, T., Pisharody, N., & Shapiro, J.
- Affiliations
- Columbia University (US)
- Princeton University (US)
- Venue
- 14th ACM Web Science Conference 2022
- Publisher
- ACM
- Published
- 2022-06-26
- Pages
- 319-326
- OpenAlex
- W4283448514
- Cited by
- 17
- References
- 15
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 4 · 2023: 5 · 2022: 1 · 2021: 1
- FWCI
- 10.87
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Web Science Conference (WebSci) (publishedVersion)
- Funders
- Carnegie Endowment for International Peace
- Microsoft
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Social Sciences › Communication › Social Media and Politics
- Social Sciences › Communication › Communication and COVID-19 Impact
- Keywords
- content removal, content moderation, spillover effects, Facebook groups, Twitter engagement, COVID-19 vaccines, online speech regulation, social media discourse
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
