Spillover of Antisocial Behavior from Fringe Platforms: The Unintended Consequences of Community Banning
Open accessAbstract
Online platforms face pressure to keep their communities civil and respectful. Thus, banning problematic online communities from mainstream platforms is often met with enthusiastic public reactions. However, this policy can lead users to migrate to alternative fringe platforms with lower moderation standards and may reinforce antisocial behaviors. As users of these communities often remain co-active across mainstream and fringe platforms, antisocial behaviors may spill over onto the mainstream platform. We study this possible spillover by analyzing 70,000 users from three banned communities that migrated to fringe platforms: r/The_Donald, r/GenderCritical, and r/Incels. Using a difference-in-differences design, we contrast co-active users with matched counterparts to estimate the causal effect of fringe platform participation on users' antisocial behavior on Reddit. Our results show that participating in the fringe communities increases users' toxicity on Reddit (as measured by Perspective API) and involvement with subreddits similar to the banned community---which often also breach platform norms. The effect intensifies with time and exposure to the fringe platform. In short, we find evidence for a spillover of antisocial behavior from fringe platforms onto Reddit via co-participation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prior empirical studies have found that group-level sanctions sometimes lead to increases in anti-social behavior or rebellious non-compliance.
“Although sanctions are typically intended to promote normative compliance, group-level sanctions in some prior empirical studies have been met with increases in anti-social behavior or rebellious non-compliance (Mitts, Pisharody, and Shapiro 2022;Ribeiro et al. 2021;Russo et al. 2023).”
From The Effects of Group Sanctions on Participation and Toxicity: Quasi-experimental Evidence from the Fediverse (2024) · cites this paper as Russo et al. 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 43 · Cited by 32
Research works printed but not linked: 34
- Austin, P. C. 2011. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate behavioral research, 46(3): 399-424.
- Baumgartner, J.; Zannettou, S.; Keegan, B.; Squire, M.; and Black- burn, J. 2020. The PushShift Reddit dataset. In '20Proceedings of the international AAAI conference on web and social media, vol- ume 14, 830-839.
- Cheng, J.; Bernstein, M.; Danescu-Niculescu-Mizil, C.; and Leskovec, J. 2017. Anyone can become a troll: Causes of trolling behavior in online discussions. In Proceedings of the 2017 ACM conference on computer supported cooperative work and social computing, 1217-1230.
- Cheng, J.; Danescu-Niculescu-Mizil, C.; and Leskovec, J. 2015. Antisocial behavior in online discussion communities. In Proceed- ings of the international aaai conference on web and social media, volume 9, 61-70.
- Flores-Saviaga, C.; Keegan, B.; and Savage, S. 2018. Mobiliz- ing the trump train: Understanding collective action in a political trolling community. In Proceedings of the International AAAI Con- ference on Web and Social Media, volume 12.
- Grover, A.; and Leskovec, J. 2016. node2vec: Scalable Feature Learning for Networks. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Min- ing.
- Grover, T.; and Mark, G. 2019. Detecting potential warning be- haviors of ideological radicalization in an alt-right subreddit. In Proceedings of the International AAAI Conference on Web and So- cial Media, volume 13, 193-204.
- Hoffman, B.; Ware, J.; and Shapiro, E. 2020. Assessing the threat of incel violence. Studies in Conflict & Terrorism, 43(7): 565-587.
- Horta Ribeiro, M.; Jhaver, S.; Zannettou, S.; Blackburn, J.; Stringh- ini, G.; De Cristofaro, E.; and West, R. 2021. Do platform migra- tions compromise content moderation? evidence from r/the donald and r/incels. Proceedings of the ACM on Human-Computer Inter- action, 5(CSCW2): 1-24.
- Jaki, S.; De Smedt, T.; Gwóźdź, M.; Panchal, R.; Rossa, A.; and De Pauw, G. 2019. Online hatred of women in the Incels. me fo- rum: Linguistic analysis and automatic detection. Journal of Lan- guage Aggression and Conflict, 7(2): 240-268.
- Jhaver, S.; Boylston, C.; Yang, D.; and Bruckman, A. 2021. Eval- uating the effectiveness of deplatforming as a moderation strategy on Twitter. Proceedings of the CSW2'21 on Human-Computer In- teraction, 5(CSCW2): 1-30.
- Juneja, P.; Rama Subramanian, D.; and Mitra, T. 2020. Through the looking glass: Study of transparency in Reddit's moderation prac- tices. Proceedings of the ACM on Human-Computer Interaction, 4(GROUP): 1-35.
- Kim, J. W.; Guess, A.; Nyhan, B.; and Reifler, J. 2021. The dis- torting prism of social media: How self-selection and exposure to incivility fuel online comment toxicity. Journal of Communication.
- Kumar, S.; Stecher, G.; Li, M.; Knyaz, C.; and Tamura, K. 2018. MEGA X: molecular evolutionary genetics analysis across com- puting platforms. Molecular biology and evolution, 35(6): 1547.
- League, A.-D. 2020. Statement on Facebook's Deci- sion to Finally Ban QAnon Content From Platform. https://www.adl.org/news/press-releases/adl-statement-on- facebooks-decision-to-finally-ban-qanon-content-from-platform. Accessed on 2020-03-04.
- McIlroy-Young, R.; and Anderson, A. 2019. From "welcome new gabbers" to the pittsburgh synagogue shooting: The evolution of gab. In Proceedings of the international aaai conference on web and social media, volume 13, 651-654.
- Paudel, P.; Blackburn, J.; De Cristofaro, E.; Zannettou, S.; and Stringhini, G. 2021. Soros, child sacrifices, and 5G: under- standing the spread of conspiracy theories on web communities. arXiv:2111.02187. · arXiv
- Phadke, S.; Samory, M.; and Mitra, T. 2022. Pathways through con- spiracy: the evolution of conspiracy radicalization through engage- ment in online conspiracy discussions. In Proceedings of the Inter- national AAAI Conference on Web and Social Media, volume 16, 770-781.
- Porter, C. E. 2004. A typology of virtual communities: A multi- disciplinary foundation for future research. Journal of computer- mediated communication, 10(1): JCMC1011.
- Resnick, P.; and Budak, C. 2020. Quick, community-specific learning: How distinctive toxicity norms are maintained in political subreddits. In Proceedings of the Inter- national AAAI Conference on Web and Social Media, volume 14, 557-568.
- Rajadesingan, A.; Zafarani, R.; and Liu, H. 2015. Sarcasm detec- tion on twitter: A behavioral modeling approach. In Proceedings of the eighth ACM international conference on web search and data mining, 97-106.
- Ribeiro, M. H.; Blackburn, J.; Bradlyn, B.; De Cristofaro, E.; Stringhini, G.; Long, S.; Greenberg, S.; and Zannettou, S. 2021. The evolution of the manosphere across the web. In Proceedings of the International AAAI Conference on Web and Social Media, volume 15, 196-207.
- Samory, M.; and Mitra, T. 2018. Conspiracies online: User dis- cussions in a conspiracy community following dramatic events. In Proceedings of the International AAAI Conference on Web and So- cial Media, volume 12.
- Seering, J.; Wang, T.; Yoon, J.; and Kaufman, G. 2019. Moderator engagement and community development in the age of algorithms. New Media & Society, 21(7): 1417-1443.
- Sipka, A.; Hannak, A.; and Urman, A. 2022. Comparing the lan- guage of qanon-related content on parler, gab, and twitter. In 14th ACM Web Science Conference 2022, 411-421.
- Tausczik, Y. R.; and Pennebaker, J. W. 2010. The psychological meaning of words: LIWC and computerized text analysis methods. Journal of language and social psychology, 29(1): 24-54.
- Trujillo, A.; and Cresci, S. 2022. Make reddit great again: assessing community effects of moderation interventions on r/the donald. Proceedings of the ACM on Human-Computer In- teraction, 6(CSCW2): 1-28.
- Waller, I.; and Anderson, A. 2019. Generalists and specialists: Us- ing community embeddings to quantify activity diversity in online platforms. In The World Wide Web Conference, 1954-1964.
- Williams, C. 2020. The ontological woman: A history of deauthen- tication, dehumanization, and violence. The Sociological Review, 68(4): 718-734.
- Wulczyn, E.; Thain, N.; and Dixon, L. 2017. Ex machina: Personal attacks seen at scale. In Proceedings of the 26th international con- ference on world wide web, 1391-1399.
- Zannettou, S.; Caulfield, T.; Blackburn, J.; De Cristofaro, E.; Siri- vianos, M.; Stringhini, G.; and Suarez-Tangil, G. 2018. On the origins of memes by means of fringe web communities. In Pro- ceedings of the internet measurement conference 2018, 188-202.
- Zannettou, S.; Caulfield, T.; De Cristofaro, E.; Kourtelris, N.; Leon- tiadis, I.; Sirivianos, M.; Stringhini, G.; and Blackburn, J. 2017. The web centipede: understanding how web communities influence each other through the lens of mainstream and alternative news sources. In Proceedings of the 2017 internet measurement con- ference, 405-417.
- Zannettou, S.; ElSherief, M.; Belding, E.; Nilizadeh, S.; and Stringhini, G. 2020. Measuring and characterizing hate speech on news websites. In 12th ACM conference on web science, 125-134.
- Zeng, J.; and Schäfer, M. S. 2021. Conceptualizing "dark plat- forms". Covid-19-related conspiracy theories on 8kun and Gab. Digital Journalism, 9(9): 1321-1343.
- Authors
- Russo, G., Verginer, L., Horta Ribeiro, M., & Casiraghi, G.
- Affiliations
- ETH Zurich (CH)
- Venue
- Proceedings of the International AAAI Conference on Web and Social Media
- Published
- 2023-06-02
- Volume
- 17
- Pages
- 742-753
- ISSN
- 2162-3449 (ISSN-L)
- 2334-0770
- OpenAlex
- W4380301697
- Cited by
- 32
- References
- 43
- Citations by year
- 2026: 3 · 2025: 14 · 2024: 10 · 2023: 4 · 2022: 1
- FWCI
- 5.50
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the International AAAI Conference on Web and Social Media (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Social Sciences › Communication › Social Media and Politics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- antisocial behavior, online toxicity, platform moderation, user migration, spillover effects, Reddit, co-participation, Perspective API
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
