The DOI matches its Crossref record; 274 of the 280 references it cites were found in a registry or an index.
SoK: Content Moderation in Social Media, from Guidelines to Enforcement, and Research to Practice
Open accessAbstract
Social media platforms have been establishing content moderation guidelines and employing various moderation policies to counter hate speech and misinformation. The goal of this paper is to study these community guidelines and moderation practices, as well as the relevant research publications, to identify the research gaps, differences in moderation techniques, and challenges that should be tackled by the social media platforms and the research community. To this end, we study and analyze fourteen most popular social media content moderation guidelines and practices, and consolidate them. We then introduce three taxonomies drawn from this analysis as well as covering over two hundred interdisciplinary research papers about moderation strategies. We identify the differences between the content moderation employed in mainstream and fringe social media platforms. Finally, we have in-depth applied discussions on both research and practical challenges and solutions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Moderators, thus, face more recurrent risks of engagement with traumatic content (3).”
From Longitudinal comparison of psychological outcomes of professional content moderators engaged in resilience training program (2025) · cites this paper as [3]
Citations
Cites
1 retracted
RETRACTED: Hate Classifier for Social Media Platform Using Tree LSTM
Raut, S., Mhatre, N., Jha, S., & Chhabria, A. (2022) · ITM Web of Conferences · doi:10.1051/itmconf/20224403034 · Cited by 1
RetractedThe Discovery of Grounded Theory: Strategies for Qualitative Research
Glaser, B. G., & Strauss, A. L. (2017) · Aldine (Routledge reprint) · doi:10.4324/9780203793206 · Cited by 25,789
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,759
Unsupervised Cross-lingual Representation Learning at Scale
Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., & Stoyanov, V. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.747 · Cited by 5,637
The online disinhibition effect
Suler, J. (2004) · CyberPsychology & Behavior · doi:10.1089/1094931041291295 · Cited by 4,375
Goodman, L. A. (1961) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177705148 · Cited by 3,606
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610
Gillespie, T. (2010) · New Media & Society · doi:10.1177/1461444809342738 · Cited by 2,522 · cited 1×
“Gillespie [133] argued that platforms are innately political entities that have attempted to maintain an image of neutrality.”
Background
Research works printed but not linked: 68
- 4chan Rules
- Civic integrity policy
- COVID-19 misleading information policy
- Notices on Twitter and what they mean
- Parler Community Guidelines
- Violent organizations policy
- 1996, 47 U.S. Code §230 - Protection for private blocking and screening of offensive material
- 2012, Contest, Sweepstakes, and Giveaway Guidelines
- 2014, The Laborers Who Keep Dick Pics, and Beheadings Out of Your Facebook Feed
- 2016, Recital 71 - Profiling
- 2017, Automation rules
- 2017, Facebook, Microsoft, Twitter and YouTube Announce Formation of the Global Internet Forum to Counter Terrorism
- 2018, 2018 was the year we (sort of) cleaned up the internet
- 2018, Deplatforming Works
- 2018, Gab, the Social Media Site for the Alt-Right, Gets Deplatformed
- 2018, Inside Facebook’s Fast-Growing Content-Moderation Effort
- 2019, Facebook Bans White Nationalism, and White Separatism
- 2019, Facebook while black: Users call it getting ’Zucked,’ say talking about racism is censored as hate speech
- 2019, Here’s What Happens When News Comes With a Nutrition Label
- 2019, Inside MeWe, Where Anti-Vaxxers and Conspiracy Theorists Thrive
- 2019, Most conservatives believe removing content and comments on social media is suppressing free speech
- 2019, Snapchat fact-checks political ads, unlike Facebook, says CEO Evan Spiegel
- 2019, The Downfall of Alex Jones Shows How the Internet Can Be Saved
- 2019, YouTube’s arbitrary standards: Stars keep making money even after breaking the rules
- 2020, Black Lives Matter Activists Say They’re Being Silenced By Facebook
- 2020, Fact-Checked on Facebook and Twitter, Conservatives Switch Their Apps
- 2020, Here’s how we’re using AI to help detect misinformation
- 2020, How Facebook uses super-efficient AI models to detect hate speech
- 2020, Parler, mewe, gab gain momentum as conservative social media alternatives in post-trump age
- 2020, Right-wing users flock to Parler as social media giants rein in misinformation
- 2020, Why some Americans are trading in mainstream social networks for ones that tout “freedom of speech”
- 2021, Appealing Content Decisions on Facebook or Instagram
- 2021, Elaboration-on-Guidelines
- 2021, Parler, a Platform Favored by Trump Fans, Struggles for Survival
- 2021, Parler Ban: Employees At Apple, Google, And Amazon Back Booting Far-Right Platform
- 2021, Parler will be hate speech–free — on iOS only
- 2021, Reddit Quarantines AntiMask Antivax Subreddit
- 2021, Restricting accounts
- 2021, What is Gab? The far-right social media site that Google and Apple banned and that is still gaining thousands of new users after Twitter and Facebook deplatformed Trump
- 2021, YouTube Community Guidelines enforcement
- 2022, About suspended accounts
- 2022, Facebook’s Ukraine-Russia Moderation Rules Prompt Cries of Double Standard
- 2022, How Instagram Is Failing Protesters in Iran
- 2022, How Rumble, a Toronto-based YouTube alternative, became a refuge for the MAGA crowd (with a US$2-billion valuation)
- 2022, I don’t think Facebook should have taken down my post
- 2022, Iran’s protests pose a challenge for Washington, and Silicon Valley
- 2022, Meta changes Ukraine content moderation policy to ban death threats against heads of state
- 2022, Meta withdraws Ukraine war content policy guidance request
- 2023, Why India banned TikTok — and what the US can learn from it, as pressure mounts for Biden to follow suit
- 2015, Australia esafety commissioner
- Benson, 2022, Twitter to label all state-affiliated Russia media
- Blunt, 2020, Posting into the void: studying the impact of shadowbanning on sex workers and activists, Hacking/Hustling, 1
- 2018, Content or context moderation? artisanal, community-reliant, and industrial approaches [white paper]. data & society
- 10.18653/v1/n19–1423 · DOI · Paper page
- Díaz, 2021, Double standards in social media content moderation
- 2017, Network enforcement act, netzdg
- Hosseinimotlagh, Unsupervised content-based identification of fake news articles with tensor decomposition ensembles
- 2018, Human Rights Watch. Germany: Flawed social media law
- Hwang, Does civility matter in the blogosphere? examining the interaction effects of incivility and disagreement on citizen attitudes
- Kumarswamy, 2022, “Strict Moderation?” The Impact of Increased Moderation on Parler Content and User Behavior
- 10.1109/INFOCOM42981.2021.9488792 · DOI
- Lu, “positive energy” perceptions and attitudes towards covid-19 information on social media in china, 5, 1
- Matias, The civic labor of online moderators
- 2021, Information technology(intermediary guidelines and digital media ethics code) rules
- 2021, Information technology(intermediary guidelines and digital media ethics code) rules
- Suzor, 2019, What do we mean when we talk about transparency? toward meaningful transparency in commercial content moderation, International Journal of Communication, 13, 18
- 2021, Copyright Law of the United States (Title 17)
- 10.48550/arxiv.1705.00648 · DOI
Cited by
Sorted by citations
SoK: Content Moderation for End-to-End Encryption
Scheffler, S., & Mayer, J. R. (2023) · Proceedings on Privacy Enhancing Technologies · doi:10.56553/popets-2023-0060 · Cited by 43
Schaffner, B., Bhagoji, A. N., Cheng, S., Mei, J., Shen, J., Wang, G., Chetty, M., Feamster, N., Lakier, G., & Tan, C. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642333 · Cited by 24
Children, Parents, and Misinformation on Social Media
Sharevski, F., & Loop, J. V. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00221 · Cited by 16
Policy-as-Prompt: Rethinking Content Moderation in the Age of Large Language Models
Palla, K., Redondo-García, J. L., Hauff, C., Fabbri, F., Damianou, A., Lindström, H., Taber, D., & Lalmas, M. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732054 · Cited by 41
Singhal, M., Kumarswamy, N., Kinhekar, S., & Nilizadeh, S. (2023) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v17i1.22189 · Cited by 15
Self-citation
Metadata
- Authors
- Singhal, M., Ling, C., Paudel, P., Thota, P., Kumarswamy, N., Stringhini, G., & Nilizadeh, S.
- Affiliations
- The University of Texas at Arlington (US)
- Boston University (US)
- ORCID
- Venue
- 2023 IEEE 8th European Symposium on Security and Privacy (EuroS&P)
- Publisher
- IEEE
- Published
- 2023-07
- Pages
- 868-895
- OpenAlex
- W4385412489
- Cited by
- 57
- References
- 280
- Citations by year
- 2026: 13 · 2025: 22 · 2024: 14 · 2023: 8
- FWCI
- 10.12
- Type
- conference paper
- Language
- en
- Open access
- green
- doi.org
- Locations
- IEEE European Symposium on Security and Privacy (publishedVersion)
- OpenBU (Boston University) (acceptedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation
- Boston University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- content moderation, hate speech, misinformation, research gaps
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
