CoChat
Sign Up
Cited by 124Hate Speech and Cyberbullying DetectionMisinformation and Its ImpactsSocial Media and Politics

Do Platform Migrations Compromise Content Moderation? Evidence from r/The_Donald and r/Incels

Open access
Horta Ribeiro, Manoel, Jhaver, Shagun, Zannettou, Savvas, Blackburn, Jeremy, Stringhini, Gianluca, De Cristofaro, Emiliano, West, Robert·2021·Proceedings of the ACM on Human-Computer Interaction
DOI 10.1145/3476057
Open this paper in CoChat — free accountOpen PDF

Abstract

When toxic online communities on mainstream platforms face moderation measures, such as bans, they may migrate to other platforms with laxer policies or set up their own dedicated websites. Previous work suggests that within mainstream platforms, community-level moderation is effective in mitigating the harm caused by the moderated communities. It is, however, unclear whether these results also hold when considering the broader Web ecosystem. Do toxic communities continue to grow in terms of their user base and activity on the new platforms? Do their members become more toxic and ideologically radicalized? In this paper, we report the results of a large-scale observational study of how problematic online communities progress following community-level moderation measures. We analyze data from r/The_Donald and r/Incels, two communities that were banned from Reddit and subsequently migrated to their own standalone websites. Our results suggest that, in both cases, moderation measures significantly decreased posting activity on the new platform, reducing the number of posts, active users, and newcomers. In spite of that, users in one of the studied communities (r/The_Donald) showed increases in signals associated with toxicity and radicalization, which justifies concerns that the reduction in activity may come at the expense of a more toxic and radical community. Overall, our results paint a nuanced portrait of the consequences of community-level moderation and can inform their design and deployment.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3476057

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Prior empirical studies have found that group-level sanctions sometimes lead to increases in anti-social behavior or rebellious non-compliance.

    “Although sanctions are typically intended to promote normative compliance, group-level sanctions in some prior empirical studies have been met with increases in anti-social behavior or rebellious non-compliance (Mitts, Pisharody, and Shapiro 2022;Ribeiro et al. 2021;Russo et al. 2023).”

    From The Effects of Group Sanctions on Participation and Toxicity: Quasi-experimental Evidence from the Fediverse (2024) · cites this paper as Ribeiro et al. 2021

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 24

  1. [1][1] Almeida, V., Filgueiras, F., and Gaetani, F. Digital governance and the tragedy of the commons. IEEE Internet Computing (2020). -
  2. [2][2] Baumgartner, J., Zannettou, S., Keegan, B., Sqire, M., and Blackburn, J. The Pushshift Reddit dataset. In Proceedings of the International Conference on Web and Social Media (ICWSM) (2020). -
  3. [7][7] Chandrasekharan, E., Pavalanathan, U., Srinivasan, A., Glynn, A., Eisenstein, J., and Gilbert, E. You can't stay here: the efficacy of Reddit's 2015 ban examined through hate speech. In Proceedings of the ACM on Human-Computer Interaction (CSCW) (2017). -
  4. [8][8] Cheng, J., Bernstein, M., Danescu-Niculescu-Mizil, C., and Leskovec, J. Anyone can become a troll: causes of trolling behavior in online discussions. In Proceedings of the ACM on Human-Computer Interaction (CSCW) (2017). -
  5. [9][9] Cheng, J., Danescu-Niculescu-Mizil, C., and Leskovec, J. Antisocial behavior in online discussion communities. In Proceedings of the International Conference on Web and Social Media (ICWSM) (2015). -
  6. [12][12] Davidson, T., Warmsley, D., Macy, M., and Weber, I. Automated hate speech detection and the problem of offensive language. In Proceedings of the International Conference on Web and Social Media (ICWSM) (2017). -
  7. [14][14] Flores-Saviaga, C. I., Keegan, B. C., and Savage, S. Mobilizing the Trump Train: understanding collective action in a political trolling community. In Proceedings of the International Conference on Web and Social Media (ICWSM) (2018). -
  8. [15][15] Gillespie, T. Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions That Shape Social Media . Yale University Press, 2018. -
  9. [18][18] Grover, T., and Mark, G. Detecting potential warning behaviors of ideological radicalization in an Alt-Right subreddit. In Proceedings of the International Conference on Web and Social Media (ICWSM) (2019). -
  10. [23][23] Jhaver, S., Ghoshal, S., Bruckman, A., and Gilbert, E. Online harassment and content moderation: the case of blocklists. ACM Transactions on Computer-Human Interaction (TOCHI) (2018). -
  11. [26][26] Kwak, H., Blackburn, J., and Han, S. Exploring cyberbullying and other toxic behavior in team competition online games. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (2015). -
  12. [28][28] Lilly, M. 'The World is Not a Safe Place for Men': The Representational Politics Of The Manosphere . PhD thesis, Université d'Ottawa/University of Ottawa, 2016. -
  13. [31][31] Mathew, B., Illendula, A., Saha, P., Sarkar, S., Goyal, P., and Mukherjee, A. Hate begets hate: a temporal study of hate speech. In Proceedings of the ACM on Human-Computer Interaction (CSCW) (2020). -
  14. [32][32] Munger, K., and Phillips, J. Right-wing YouTube: a supply and demand perspective. The International Journal of Press/Politics (2020). -
  15. [33][33] Newell, E., Jurgens, D., Saleem, H. M., Vala, H., Sassine, J., Armstrong, C., and Ruths, D. User migration in online social networks: a case study on Reddit during a period of community unrest. In Proceedings of the International Conference on Web and Social Media (ICWSM) (2016). -
  16. [38][38] Rajadesingan, A., Resnick, P., and Budak, C. Quick, community-specific learning: how distinctive toxicity norms are maintained in political subreddits. In Proceedings of the International Conference on Web and Social Media (ICWSM) (2020). -
  17. [39][39] Raman, N., Cao, M., Tsvetkov, Y., Kästner, C., and Vasilescu, B. Stress and burnout in open source: toward finding, understanding, and mitigating unhealthy interactions. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: New Ideas and Emerging Results (2020). -
  18. [40][40] Ribeiro, M. H., Blackburn, J., Bradlyn, B., De Cristofaro, E., Stringhini, G., Long, S., Greenberg, S., and Zannettou, S. The evolution of the Manosphere across the Web. In Proceedings of the International Conference on Web and Social Media (ICWSM) (2021). -
  19. [41][41] Ribeiro, M. H., Calais, P. H., Santos, Y. A., Almeida, V. A., and Meira Jr, W. Characterizing and detecting hateful users on Twitter. In Proceedings of the International Conference on Web and Social Media (ICWSM) (2018). -
  20. [48][48] Saleem, H. M., and Ruths, D. The aftermath of disbanding an online hateful community. arXiv:1804.07354 (2018). - · arXiv
  21. [50][50] Shen, Q., and Rose, C. The discourse of online content moderation: investigating polarized user responses to changes in Reddit's quarantine policy. In Proceedings of the Third Workshop on Abusive Language Online (2019). -
  22. [52][52] Tausczik, Y. R., and Pennebaker, J. W. The psychological meaning of words: LIWC and computerized text analysis methods. Journal of language and social psychology (2010). -
  23. [56][56] Zannettou, S., ElSherief, M., Belding, E., Nilizadeh, S., and Stringhini, G. Measuring and characterizing hate speech on news websites. In ACM Conference on Web Science (2020). -
  24. [57][57] Zhang, J., Chang, J., Danescu-Niculescu-Mizil, C., Dixon, L., Hua, Y., Taraborelli, D., and Thain, N. Conversations gone awry: detecting early signs of conversational failure. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (2018).

Cited by

Sorted by citations

Authors
Horta Ribeiro, M., Jhaver, S., Zannettou, S., Blackburn, J., Stringhini, G., De Cristofaro, E., & West, R.
Affiliations
  • EPFL, Lausanne, Switzerland
  • Rutgers, The State University of New Jersey (US)
  • Max Planck Institute, Saarbrücken, Germany
  • Binghamton University (US)
  • Boston University (US)
  • University College London, London, United Kingdom
  • École Polytechnique Fédérale de Lausanne (CH)
ORCID
Venue
Proceedings of the ACM on Human-Computer Interaction
Publisher
Association for Computing Machinery (ACM)
Published
2021-10-13
Volume
5
Issue
CSCW2
Pages
1-24
ISSN
2573-0142 (ISSN-L)
DOI
10.1145/3476057
OpenAlex
W3094590334
Cited by
124
References
4
Citations by year
2026: 23 · 2025: 26 · 2024: 24 · 2023: 26 · 2022: 17 · 2021: 8
FWCI
6.32
Type
journal article
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor, 370 d)
Locations
  • Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
  • OpenBU (Boston University) (acceptedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
NSF (CNS-2114411)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Social Sciences › Communication › Social Media and Politics
Keywords
platform migration, content moderation, ideological radicalization, online toxicity, user activity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review