Crossmod: A Cross-Community Learning-based System to Assist Reddit Moderators
Open accessThe DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Abstract
In this paper, we introduce a novel sociotechnical moderation system for Reddit called Crossmod. Through formative interviews with 11 active moderators from 10 different subreddits, we learned about the limitations of currently available automated tools, and how a new system could extend their capabilities. Developed out of these interviews, Crossmod makes its decisions based on cross-community learning---an approach that leverages a large corpus of previous moderator decisions via an ensemble of classifiers. Finally, we deployed Crossmod in a controlled environment, simulating real-time conversations from two large subreddits with over 10M subscribers each. To evaluate Crossmod's moderation recommendations, 4 moderators reviewed comments scored by Crossmod that had been drawn randomly from existing threads. Crossmod achieved an overall accuracy of 86% when detecting comments that would be removed by moderators, with high recall (over 87.5%). Additionally, moderators reported that they would have removed 95.3% of the comments flagged by Crossmod; however, 98.3% of these comments were still online at the time of this writing (i.e., not removed by the current moderation system). To the best of our knowledge, Crossmod is the first open source, AI-backed sociotechnical moderation system to be designed using participatory methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“For example, moderators on Reddit use a tool called AutoModerator, with which they manually customize a rule-based system to automatically identify comments for deleting or reporting for further review [18,44].”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [18]
The cited source says what the claim attributes to it.
In the citing paper
“This approach, however, misses out the benefit of AI especially since rigid rules often do not work on informal languages such as social media posts (e.g., containing swear words without being toxic).”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [18]
The cited source says what the claim attributes to it.
In the citing paper
“Notably, the task for humans to create conditional delegation rules share some similarity with what many social media moderators are already doing by writing manual automation rules to deal with the massive amount of comments (Fig. 1(C)).”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [18]
In the citing paper
“One such example is content moderation, where moderator decisions on individual comments for further actions (e.g., hiding the content or prompting further review, depending on the community policy) are of limited consequence; instead the key challenge lies in dealing with the massive scale of comments.”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [18]
In the citing paper
“Chandrasekharan et al. [18] also found that hard-coded rules are prone to mistakes.”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [18]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645 · cited 1×
“These classifiers were trained using FastText, a state-of-the-art library [10, 34].”
MethodBag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2016) · arXiv · doi:10.48550/arxiv.1607.01759 · Cited by 4,542
Actor Network Theory and After
Stark, D., Law, J., & Hassard, J. (2001) · Contemporary Sociology · doi:10.2307/2654376 · Cited by 3,263
(1999) · Medical Entomology and Zoology · Cited by 3,259
Resnick, P., Kuwabara, K., Zeckhauser, R., & Friedman, E. (2000) · Communications of the ACM · doi:10.1145/355112.355122 · Cited by 2,190 · cited 1×
“In the distributed approach, users up-vote desirable content making it more visible, and down-vote undesirable content, sometimes even reporting content [24, 41, 57, 58].”
BackgroundTrust among strangers in internet transactions: Empirical analysis of eBay' s reputation system
Resnick, P., & Zeckhauser, R. (2004) · Advances in Applied Microeconomics · doi:10.1016/s0278-0984(02)11030-3 · Cited by 1,804
Çömlekçi, M. F. (2019) · Kırklareli University Institutional Repository (Kırklareli University) · Cited by 1,607
Principles of mixed-initiative user interfaces
Horvitz, E. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303030 · Cited by 1,490 · cited 2×
“Specifically, we adopt a mixed-initiative [31] approach in Crossmod, allowing moderators of subreddits to augment the automatic predictions obtained from cross-community learning with human decisions and oversight.”
MethodAbusive Language Detection in Online User Content
Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y., & Chang, Y. (2016) · Proceedings of the 25th International Conference on World Wide Web · doi:10.1145/2872427.2883062 · Cited by 1,122
Research works printed but not linked: 14
- Monika Bickert. 2018. Publishing Our Internal Enforcement Guidelines and Expanding Our Appeals Process, Apr. 2018. https:// newsroom.fb.com/ news/ 2018/ 04/ comprehensive-community-standards/ (2018).
- Wiebe E Bijker. 1987. The social construction of Bakelite: Toward a theory of invention. The social construction of technological systems: New directions in the sociology and history of technology (1987), 159-187.
- Randy Farmer and Bryce Glass. 2010. Building web reputation systems. " O'Reilly Media, Inc. ". 243-276 pages.
- Google. 2018. YouTube Community Guidelines enforcement in Google's Tranparency Report for 2018. https:// transparencyreport.google.com/ youtube-policy/ removals (2018).
- David Jurgens, Eshwar Chandrasekharan, and Libby Hemphill. 2019. A Just and Comprehensive Strategy for Using NLP to Address Online Abuse. arXiv preprint arXiv:1906.01738 (2019). · arXiv
- Sara Kiesler, Robert Kraut, Paul Resnick, and Aniket Kittur. 2012. Regulating behavior in online communities. Building Successful Online Communities: Evidence-Based Social Design. MIT Press, Cambridge, MA (2012), 125-178.
- Rob Kling, Roberta Lamb, et al. 2000. IT and organizational change in digital economies: A sociotechnical approach. Understanding the Digital Economy. Data, Tools, and Research. The MIT Press, Cambridge, MA (2000).
- Rachael Krishna. 2018. Tumblr Launched An Algorithm To Flag Porn And So Far It's Just Caused Chaos, Dec 2018. https:// www.buzzfeednews.com/ article/ krishrach/ tumblr-porn-algorithm-ban (2018).
- Enid Mumford. 2000. Socio-technical design: An unfulfilled promise or a future opportunity? In Organizational and social perspectives on information technology. Springer, 33-46.
- John Pavlopoulos, Prodromos Malakasiotis, and Ion Androutsopoulos. 2017. Deep learning for user comment modera- tion. arXiv preprint arXiv:1705.09993 (2017). · arXiv
- Jenny Preece and Diane Maloney-Krichmar. 2003. Online communities: focusing on sociability and usability. Handbook of human-computer interaction (2003), 596-620.
- Steven B Sawyer and Mohammad Hossein Jarrahi. 2014. Sociotechnical approaches to the study of Information Systems. In Computing handbook, third edition: Information systems and information technology. CRC Press, 5-1.
- Ruth L Williams and Joseph Cothrel. 2000. Four smart ways to run online communities. MIT Sloan Management Review 41, 4 (2000), 81.
- Daniel Dylan Wray. 2018. The Companies Cleaning the Deepest, Darkest Parts of Social Media, June 2018. https: // www.vice.com/ en_us/ article/ ywe7gb/ the-companies-cleaning-the-deepest-darkest-parts-of-social-media (2018).
Cited by
Baumgartner, J., Zannettou, S., Keegan, B., Squire, M., & Blackburn, J. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7347 · Cited by 881
Generative Adversarial Networks for face generation: A survey
Kammoun, A., Slama, R., Tabia, H., Ouni, T., & Abid, M. (2022) · ACM Computing Surveys · doi:10.1145/1122445.1122456 · Cited by 311
Social Biases in NLP Models as Barriers for Persons with Disabilities
Hutchinson, B., Prabhakaran, V., Denton, E., Webster, K., Zhong, Y., & Denuyl, S. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.487 · Cited by 274
Social Biases in NLP Models as Barriers for Persons with Disabilities
Hutchinson, B., Prabhakaran, V., Denton, E., Webster, K., Zhong, Y., & Denuyl, S. (2020) · arXiv · doi:10.48550/arxiv.2005.00813 · Cited by 274
“In a deployment context that involves human moderation, model scores may sometimes be used to select and prioritize messages for review by moderators (Veglis, 2014; Chandrasekharan et al., 2019).”
BackgroundThe Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice
Delgado, F., Yang, S., Madaio, M., & Yang, Q. (2023) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3617694.3623261 · Cited by 256
“(2019) [47] Academia Online Moderation Automate decisions for content moderation Prototype Evaluation,”
BackgroundDealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations
Davani, A. M., Díaz, M., & Prabhakaran, V. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00449 · Cited by 219
“For example, the level of uncertainty could help determine when and how moderators take part in a human-in-the-loop content moderation (Chandrasekharan et al., 2019; Liu, 2020).”
BackgroundLife, the Metaverse and Everything: An Overview of Privacy, Ethics, and Governance in Metaverse
Fernandez, C. B., & Hui, P. (2022) · 2022 IEEE 42nd International Conference on Distributed Computing Systems Workshops (ICDCSW) · doi:10.1109/icdcsw56584.2022.00058 · Cited by 192
Seering, J. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415178 · Cited by 147
“This results in a classifier that works not from consciously defined rules but from the aggregation of prior users’ moderation decisions [11].”
BackgroundInfluentialHuman-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation
Lai, V., Carton, S., Bhatnagar, R., Liao, Q. V., Zhang, Y., & Tan, C. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501999 · Cited by 145
“Third, prior work has shown that moderators often update the rules used by AutoModerator [18, 44] and our work does not take into account any future updates.”
BackgroundMethodInfluentialQuarantined! Examining the Effects of a Community-Wide Moderation Intervention on Reddit
Chandrasekharan, E., Jhaver, S., Bruckman, A., & Gilbert, E. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3490499 · Cited by 129
“Amidst the growing amounts of research on online moderation and newways to counter antisocial behavior [11, 16, 47, 48, 50], we provide a powerful tool for causally evaluating moderation interventions.”
BackgroundMethodSelf-citationYes: Affirmative Consent as a Theoretical Framework for Understanding and Imagining Social Platforms
Im, J., Dimond, J., Berton, M., Lee, U., Mustelier, K., Ackerman, M. S., & Gilbert, E. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445778 · Cited by 119
A framework of severity for harmful content online
Scheuerman, M. K., Jiang, J. A., Fiesler, C., & Brubaker, J. R. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479512 · Cited by 114
Drawing from justice theories to support targets of online harassment
Schoenebeck, S., Haimson, O. L., & Nakamura, L. (2021) · New Media & Society · doi:10.1177/1461444820913122 · Cited by 84
“Volunteer moderation is often buttressed by automated bots or systems (e.g. Chandrasekharan et al., 2019) that support moderation demands at scale.”
MethodJhaver, S., Zhang, A. Q., Chen, Q. Z., Natarajan, N., Wang, R., & Zhang, A. X. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610080 · Cited by 167
“Prior research on moderation tools for content creators [52, 73] and moderators [13, 50] offers blueprints for designing and improving personal content moderation tools.”
BackgroundA Trade-off-centered Framework of Content Moderation
Jiang, J. A., Nie, P., Brubaker, J. R., & Fiesler, C. L. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3534929 · Cited by 126
“g, millions of users), automated moderation provided a solution for moderation at scale [15].”
BackgroundSmith, C. E., Yu, B., Srivastava, A., Halfaker, A., Terveen, L., & Zhu, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376783 · Cited by 77
“As a first step in this direction, we have identified studies that examined the role of “moderation” in various contexts, including the Turkopticon community of Amazon Mechanical Turk workers [39] and an assortment of communities organized as Reddit subreddits [7, 42], or Twitch channels and Facebook groups [52].”
BackgroundContestability For Content Moderation
Vaccaro, K., Xiao, Z., Hamilton, K., & Karahalios, K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476059 · Cited by 77
“For example, rather than have a single policy, platforms could infer “local” communities and their norms, similar to community-based approaches that have been successful in the past [18].”
BackgroundPolicyKit: Building Governance in Online Communities
Zhang, A. X., Hugh, G., & Bernstein, M. S. (2020) · Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3379337.3415858 · Cited by 60
“Additional tools exist that help mods create better policies [8, 52] but still are focused on the role of mods.”
Background“I Didn’t Know I Looked Angry”: Characterizing Observed Emotion and Reported Affect at Work
Kaur, H., McDuff, D., Williams, A. C., Teevan, J. B., & Iqbal, S. T. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517453 · Cited by 90
“This type of conditional delegation has shown some success for content moderation [10] and fake speech detection [114], both of which are domains with similar challenges of ground truth.”
BackgroundModular Politics: Toward a Governance Layer for Online Communities
Schneider, N., Filippi, P. D., Frey, S., Tan, J., & Zhang, A. X. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449090 · Cited by 105
“pecially evident in their ability to develop free and flexible "offplatform" software tools for addressing the governance challenges they share in common, whether via general tool libraries [22, 66], templating mechanics [16, 39], or even community-led randomized experiments [93]. For example, in lieu of cen…”
BackgroundHarassment Experiences of Women and LGBTQ Live Streamers and How They Handled Negativity
Uttarapong, J., Cai, J., & Wohn, D. Y. (2021) · ACM International Conference on Interactive Media Experiences · doi:10.1145/3452918.3458794 · Cited by 86
“Recent work considers moderators’ opinions during the tool design and achieves higher accuracy than traditional algorithms [14].”
BackgroundFeature-Based Explanations Don't Help People Detect Misclassifications of Online Toxicity
Carton, S., Mei, Q., & Resnick, P. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7282 · Cited by 58
“…et al. 2017; de Gibert et al. 2018; Hosseinmardi et al. 2015), while many papers have been published on the application of machine learning to the detection thereof (Salminen et al. 2020; Pavlopoulos, Malakasiotis, and Androutsopoulos 2017; Chancellor et al. 2017; Chandrasekharan et al. 2019).”
BackgroundXiao, S., Cheshire, C., & Salehi, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517614 · Cited by 50
“Researchers have examined ways to reduce offending behaviors through moderation mechanisms, for example, through setting positive examples and social norms in moderation [9, 51], providing moderation explanations [21, 22], or designing novel algorithms and AI moderation systems [8].”
BackgroundLLM-Mod: Can Large Language Models Assist Content Moderation?
Kolla, M., Salunkhe, S., Chandrasekharan, E., & Saha, K. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650828 · Cited by 142
Self-citationHate Raids on Twitch: Echoes of the Past, New Modalities, and Implications for Platform Governance
Han, C., Seering, J., Kumar, D., Hancock, J. T., & Durumeric, Z. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579609 · Cited by 46
“2, we discuss the impact of community moderation and community-developed automated moderation tools, adding to conversation in prior work that has raised questions surrounding platform governance and the distribution of labor in content moderation [4, 24, 44, 49].”
BackgroundSchluger, C., Chang, J. P., Danescu-Niculescu-Mizil, C., & Levy, K. E. C. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555095 · Cited by 74
“Following a rich line of prior literature that uses interviews to pull back the curtain on moderation practices [8, 16, 27, 59, 64], we conducted semi-structured interviews with nine administrators on Wikipedia who engage in Talk Page moderation.”
BackgroundMethodInfluentialSmith, C. E., Lane, W., Miller, H. J., Kluver, D., Terveen, L., & Yarosh, S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479561 · Cited by 40
“In large communities like Reddit [19, 42] and Wikipedia [34, 38, 91], content moderation systems often use moderator-configured pattern-matching rules or AI/ML-based systems to detect problems like abusive content or toxic speech [98].”
BackgroundResultInfluentialChoi, Y., Kang, E. J., Lee, M. K., & Kim, J. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581386 · Cited by 64
“In the food of information, the algorithm recommends dating partners [53], moderates inappropriate comments [20], and is even responsible for connecting workers on gig economy platforms [39, 58, 70] or content creators [43] with potential consumers.”
BackgroundWright, A. P., Shaikh, O., Park, H., Epperson, W., Ahmed, M., Pinel, S., Chau, D. H., & Yang, D. (2021) · Proc. ACM Hum. Comput. Interact. · doi:10.1145/3449280 · Cited by 35
“Furthermore, several works have focused on studying the development of new automated moderation systems [6, 31], and their effect on the dynamics of the platforms that utilize them [14].”
BackgroundCai, J., & Wohn, D. Y. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479554 · Cited by 46
Who Uses Bots? A Statistical Analysis of Bot Usage in Moderation Teams
Kiene, C., & Hill, B. M. (2020) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3334480.3382960 · Cited by 43
“In many cases, moderation teams adopt user innovated tools in the form of “bots” to better scale their work in response to community growth [2, 4, 6, 7, 11].”
BackgroundGarg, S., Taylor, J., Sherief, M. A. E., Kasson, E. M., Aledavood, T., Riordan, R., Kaiser, N., Cavazos‐Rehg, P. A., & Choudhury, M. D. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100467 · Cited by 44
“…Reddit moderators play an active role in managing the content on their subreddit, defining community-specific rules, establishing norms, and providing support to people who post acutely concerning content (De Choudhury and Sushovan, 2014; De Choudhury et al., 2016; Chandrasekharan et al., 2019).”
BackgroundBozarth, L., Im, J., Quarles, C. L., & Budak, C. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579631 · Cited by 50
“Other researchers have also explored ways to improve moderation through expanding existing algorithmic tools [11, 34].”
BackgroundInfluentialPolitical Bias and Factualness in News Sharing across more than 100,000 Online Communities
Weld, G., Glenski, M., & Althoff, T. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v15i1.18104 · Cited by 58
All That’s Happening behind the Scenes: Putting the Spotlight on Volunteer Moderator Labor in Reddit
Li, H., Hecht, B. J., & Chancellor, S. (2022) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v16i1.19317 · Cited by 64
The Unsung Heroes of Facebook Groups Moderation: A Case Study of Moderation Practices and Tools
Kuo, T., Hernani, A., & Großklags, J. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579530 · Cited by 49
“[9] proposed a collaborative AI and moderation system where the AI-backed tool triaged problematic content on behalf of moderators on Reddit.”
BackgroundPeng, Z., Ma, X., Yang, D., Tsang, K. W., & Guo, Q. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445446 · Cited by 36
“detection algorithms to replace or assist moderators [14, 15, 20].”
BackgroundStoryChat: Designing a Narrative-Based Viewer Participation Tool for Live Streaming Chatrooms
Yen, R., Feng, L., Mehra, B., Pang, C. C., Hu, S., & Lu, Z. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580912 · Cited by 39
“Therefore, we created a staging instance with a massive real-time chat copied from actual live streams [23].”
MethodSchaffner, B., Bhagoji, A. N., Cheng, S., Mei, J., Shen, J., Wang, G., Chetty, M., Feamster, N., Lakier, G., & Tan, C. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642333 · Cited by 67
Measuring the Prevalence of Anti-Social Behavior in Online Communities
Park, J.-S., Seering, J., & Bernstein, M. S. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555552 · Cited by 46
“Platforms responded by deploying thousands of paid moderators [41, 67], an array of algorithms [43, 44], and tools for community self-moderation [19, 71].”
BackgroundResultInfluentialSalehi, N., Pakzad, R., Lajevardi, N., & Asad, M. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579526 · Cited by 29
“Recently, machine learning tools have begun to play an important role in automating content moderation by flagging and remove violating content [27].”
BackgroundMeasuring User-Moderator Alignment on r/ChangeMyView
Koshy, V., Bajpai, T., Chandrasekharan, E., Sundaram, H., & Karahalios, K. G. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610077 · Cited by 31
“And moderators, who rely on user-reports to catch rule violations at scale, may have to manage an increased workload to compensate for low quality user reports [4].”
BackgroundSelf-citationWho Moderates on Twitch and What Do They Do? Quantifying Practices in Community Moderation on Twitch
Seering, J., & Kairam, S. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3567568 · Cited by 49
Cook, C. L., Cai, J., & Wohn, D. Y. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555168 · Cited by 20
“…of content moderators [61, 75, 89] and have examined the effectiveness of current methods to ease the burden on content moderators (e.g., [13, 15, 79, 80]), referring to both moderators in volunteer and commercial content moderation, as they both experience exposure to harmful content and…”
BackgroundSenseMate: An Accessible and Beginner-Friendly Human-AI Platform for Qualitative Data Analysis
Overney, C., Saldías-Fuentes, B., Dimitrakopoulou, D., & Roy, D. K. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645194 · Cited by 46
Making Online Communities ‘Better’: A Taxonomy of Community Values on Reddit
Weld, G., Zhang, A. X., & Althoff, T. (2024) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v18i1.31413 · Cited by 70
Cai, J., Chowdhury, S., Zhou, H., & Wohn, D. Y. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610191 · Cited by 34
“For example, algorithm developers can work closely with LGBTQ+ streamers to update the terms in AutoMod to improve its efficiency and usability, in accordance with user-centered design methodologies throughout the design process [41, 51] and moderation system development [17].”
MethodEffects of Algorithmic Flagging on Fairness: Quasi-experimental Evidence from Wikipedia
TeBlunthuis, N., Hill, B. M., & Halfaker, A. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449130 · Cited by 26
“Consequently, steps that make moderation more fair might decrease the negative effects of sanctions on community growth.”
Reporting the Community Beat: Practices for Moderating Online Discussion at a News Website
McInnis, B. J., Ajmani, L., Sun, L., Hou, Y., Zeng, Z., & Dow, S. P. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476074 · Cited by 20
Namespotting: Username toxicity and actual toxic behavior on Reddit
Urbaniak, R., Tempska, P., Dowgiałło, M., Ptaszynski, M. E., Fortuna, M., Marcińczuk, M., Piesiewicz, J., Leliwa, G., Soliwoda, K., Dziublewska, I., et al. (2022) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107371 · Cited by 23
Song, J. Y., Lee, S., Lee, J., Kim, M., & Kim, J. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581057 · Cited by 23
“[9] trained macro norms and community-specifc to make the classifer more suitable for each community.”
BackgroundNguyen, A., Rai, A., & Maruping, L. M. (2024) · Journal of Management Information Systems · doi:10.1080/07421222.2024.2340831 · Cited by 21
“Given the complementary strengths of humans and machines, there is growing scholarly interest in the effectiveness of using human-machine hybrids against online toxic interactions [10, 28, 35].”
BackgroundEpstein, D. A., Liu, F., Monroy‐Hernández, A., & Wang, D. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555557 · Cited by 18
Lerman, K., Chu, Bickham, C., Luceri, L., & Ferrara, E. (2025) · EPJ Data Science · doi:10.1140/epjds/s13688-025-00575-5 · Cited by 28
“However, these tools often lack contextual nuance and may over-block content [51], particularly in vulnerable or marginalized communities [52].”
Muralikumar, M. D., Yang, Y. S., & McDonald, D. W. (2023) · ACM Transactions on Social Computing · doi:10.1145/3582568 · Cited by 30
“Some prior work illustrates the usefulness of cross-domain approaches to addressing moderation challenges [32, 35].”
BackgroundMethodSmith, C. E., Klassen, S., Prinster, G. H., Tan, C., & Keegan, B. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686920 · Cited by 19
“For example, an ACM search (in 2022) for the term r/BlackPeopleTwitter returned five papers, but the subreddit was simply treated as a data source alongside other subreddits rather than as a specific and unique site of expression and potentially community [20, 23, 30, 32, 95].”
Smith, C. E., Alam, I., Tan, C., Keegan, B., & Blanchard, A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555563 · Cited by 18
“Bots are becoming increasingly common in platforms that host online communities such as Reddit [16, 42, 57], Wikipedia [30, 31, 97, 116], Facebook [70, 106], Twitter [62, 93, 104], and Twitch [91].”
BackgroundInfluentialMa, R., & Kou, Y. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579477 · Cited by 17
“We also proactively reached out to YouTubers if they had posted their moderation experience on social media such as Twitter, Facebook, or Reddit.”
BackgroundInfluentialAre Proactive Interventions for Reddit Communities Feasible?
Habib, H., Musa, M. B., Zaffar, M. F., & Nithyanand, R. (2022) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v16i1.19290 · Cited by 28
“While previous research (Chandrasekharan et al. 2017) concluded that such bans “worked for Reddit”, others have pointed out that they are too reactionary and occur only after a significant amount of damage has already been observed (Morse 2019; Romano 2017; Marantz 2018).”
MethodWang, L., & Zhu, H. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533147 · Cited by 20
“Therefore, online communities like Facebook [4], YouTube [27], and Reddit [8] increasingly adopt algorithms to automatically identify and remove damaging content at scale.”
BackgroundConvEx: A Visual Conversation Exploration System for Discord Moderators
Choi, F., Bajpai, T., Pratipati, S., & Chandrasekharan, E. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610053 · Cited by 31
“, growth in community size and overall activity levels—across several platforms [10, 26, 28, 30].”
BackgroundInfluentialSelf-citation"Nip it in the Bud": Moderation Strategies in Open Source Software Projects and the Role of Bots
Hsieh, J., Kim, J., Dabbish, L., & Zhu, H. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610092 · Cited by 26
“[21, 22]) and examine the effectiveness of different moderation behaviors and strategies (e.”
Background"How Do You Quantify How Racist Something Is?": Color-Blind Moderation in Decentralized Governance
Wu, Q., & Semaan, B. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610030 · Cited by 13
“Moderators of Reddit can adopt multiple automated tools [10].”
BackgroundMethodLambert, C., Choi, F., & Chandrasekharan, E. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686929 · Cited by 35
“For example, prior work argues that we should develop community-specific tools that are capable of working well with moderators’ existing norms and workflows, regardless of what type of community is using the tool [12, 15].”
Self-citationRyu, H., & Pratt, W. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab208 · Cited by 10
Pika: Empowering Non-Programmers to Author Executable Governance Policies in Online Communities
Wang, L., Vincent, N., Rukanskaitė, J., & Zhang, A. X. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642012 · Cited by 17
“Deployed across various online platforms, including Reddit [9], GitHub [73], and Twitch [63], these bots are used to perform repetitive and laborious tasks regarding content moderation and newcomer onboarding.”
BackgroundMethodInfluentialDemocratic Moderation: Exploring the Use and Perception of Votekicking in Social Virtual Reality
Chen, Q., Wu, Q., & Jacucci, G. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713577 · Cited by 13
“A signifcant body of research in Human-Computer Interaction (HCI) has explored community moderation practices across multiple platforms, such as Discord [ 68, 69], Reddit [18, 114], and Twitch [134].”
A Comprehensive Review on Bot - Discord Bot
Verma, A., Tyagi, S., & Mathur, G. (2021) · International Journal of Scientific Research in Computer Science Engineering and Information Technology · doi:10.32628/cseit2172100 · Cited by 13
“Technol, March-April-2021, 7 (2) : 532-536”
Automatically Labeling Low Quality Content on Wikipedia By Leveraging Patterns in Editing Behaviors
Asthana, S., Thommel, S. T., Halfaker, A., & Banović, N. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479503 · Cited by 11
“, content that violates platform policy) using both programmatic rules [11, 16, 23, 38] and Machine Learning-based methods [4].”
MethodConceptualizing Visual Analytic Interventions for Content Moderation
Vaidya, S., Cai, J., Basu, S., Naderi, A., Wohn, D. Y., & Dasgupta, A. (2021) · Visualization · doi:10.1109/vis49827.2021.9623288 · Cited by 18
“To reduce online negativity, commercial platforms apply many techniques to filter abusive language, such as improving algorithms and applying automation tools [10,21].”
BackgroundMethodChan, J., Atreyasa, A., Chancellor, S., & Chandrasekharan, E. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3519813 · Cited by 17
“These papers often analyze antisocial behavior through detection [4] and prevention through moderator interventions and tools [5, 6, 13].”
BackgroundSelf-citationHarmonizing the Cacophony with MIC: An Affordance-aware Framework for Platform Moderation
Bajpai, T., Asher, D., Goswami, A., & Chandrasekharan, E. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555110 · Cited by 19
“Reddit has automated moderation tools, as well as an API that allows moderators to create moderation tools and bots to help human moderators to review large volumes of content.”
InfluentialSelf-citationLinguistically Differentiating Acts and Recalls of Racial Microaggressions on Social Media
Gunturi, U. S., Kumar, A., Ding, X., & Rho, E. H. R. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637366 · Cited by 14
“…of moderation systems and human moderators to differentiate when someone is critiquing racism (false positive) versus being racist (false negative) [27, 77], can be a gate-keeping practice that can evolve into a form of digital gentrification [55, 94], further exacerbating disparities between…”
BackgroundAppealMod: Inducing Friction to Reduce Moderator Workload of Handling User Appeals
Atreja, S., Im, J., Resnick, P., & Hemphill, L. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637296 · Cited by 16
“Consequently, a growing body of content moderation research [5, 6, 23, 32, 50, 60] focuses on understanding and supporting volunteer moderators’ work.”
BackgroundInfluentialChillbot: Content Moderation in the Backchannel
Seering, J., Khadka, M., Haghighi, N., Yang, T., Xi, Z., & Bernstein, M. S. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686941 · Cited by 20
“…social behaviors by, e.g., summarizing conversations [56], removal remains a major theme of the moderation tool literature: for example, Crossmod [7] helps moderators on Reddit identify comments that may violate subreddit norms, flagging them for moderator review, while FilterBuddy [21] and…”
BackgroundVLM as Policy: Common-Law Content Moderation Framework for Short Video Platform
Lu, X., Zhang, T., Meng, C., Wang, X., Wang, J., Zhang, Y., Tang, S., Liu, C., Ding, H., Jiang, K., et al. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737273 · Cited by 28
“Of-fline Of-fline experiments and large-scale online A/B test demonstrates the superiority of KuaiMod : KuaiMod achieves the best moderation”
'There Has To Be a Lot That We're Missing': Moderating AI-Generated Content on Reddit
Lloyd, T., Reagle, J., & Naaman, M. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757445 · Cited by 40
“The CSCW research community has researched and developed computational tools meant to aid moderators in their work [12, 79, 98].”
Wu, Q., Romero, T., & Semaan, B. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642688 · Cited by 14
“Moderators can adopt multiple automatic moderation tools [10].”
BackgroundCoalesce: An Accessible Mixed-Initiative System for Designing Community-Centric Questionnaires
Overney, C., Kessler, D. T., Fulay, S. P., Jasim, M., & Roy, D. K. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712118 · Cited by 14
“One type of algorithm-in-the-loop system includes mixed-initiative systems, in which machines and humans collaborate efficiently to achieve the user’s goals [13, 23, 42].”
The Design Space for Online Restorative Justice Tools: A Case Study with ApoloBot
Doan, B. N. (., & Seering, J. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713598 · Cited by 26
“These tools help in tasks both proactively, such as setting up automatic content filtering [15], sending nudges to potential violations [60], classification [8], and limiting the reach of content [6], and reactively, such as issuing warnings, bans, and handling user appeals [2].”
Wang, L., Yurechko, K., Dani, P., Chen, Q. Z., & Zhang, A. X. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713691 · Cited by 25
“Specifically, we explore how approaches like writing rules or labeling examples can help users more easily and systematically bootstrap and iterate on their prompts.”
Content Moderator Startle Response: A Qualitative Study
Bharucha, T., Steiger, M. E., Manchanda, P., Mere, R., & Huang, X. (2023) · Lecture notes in networks and systems · doi:10.1007/978-981-99-3043-2_18 · Cited by 12
Lee, D., Seo, S., Park, C., Kim, S., Chang, B., & Song, J. Y. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3660708 · Cited by 10
“Despite ongoing reports about their emotional labor [11, 13, 17, 52, 65], insufcient solutions exist worldwide to protect the widely dispersed and vulnerable workforce.”
BackgroundLambretta: Learning to Rank for Twitter Soft Moderation
Paudel, P., Blackburn, J., Cristofaro, E. D., Zannettou, S., & Stringhini, G. (2023) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46215.2023.10179392 · Cited by 18
“Another tool, called CrossMod [12] uses an ensemble of models learned via cross-community learning from empirical moderation decisions made on two subreddits of over 10M subscribers each.”
MethodExploring the Magnitude and Effects of Media Influence on Reddit Moderation
Habib, H., & Nithyanand, R. (2022) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v16i1.19291 · Cited by 29
“Researchers have highlighted that inconsistencies associated with moderation may be attributed to the high cost and inherently poor scalability of human moderation and have proposed machine-learning based tools to assist moderators (Reddit moderators, specifically) identify communities at risk of violating platform ru…”
BackgroundPopowski, S. A., Meyerhoff, J., Allen, O. M., Nguyen, T., McCall, T., Pederson, A. B., Reddy, M. C., Mohr, D. C., & Kornfield, R. (2025) · JMIR Formative Research · doi:10.2196/73279 · Cited by 8
“…is often the most complex or costly aspect of a DMH program [73,74], although efforts are underway to make moderation processes more scalable, such as through crowdsourcing [75] or automation [76,77], although these efforts must be complemented by appropriate human-in-the-loop review [78, 79].”
Davidson, T. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02360-w · Cited by 12
Choi, F., Lambert, C., Koshy, V., Pratipati, S., Do, T., & Chandrasekharan, E. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713521 · Cited by 16
“Researchers are continuing to explore the use and development of tools designed to help moderators manage their ever-increasing workload [16, 19, 32, 36].”
Self-citationLearning for Detecting Norm Violation in Online Communities
dos Santos, T. F., Osman, N., & Schorlemmer, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16617-4_9 · Cited by 6
“Previous works have dealt with norms and normative systems, proposing mechanisms for norm conflict detection [1], norm synthesis [12], norm violation on Wikipedia [17, 3] and other online communities, such as Stack Overflow [5] and Reddit [4].”
BackgroundMethodInfluentialEChat: An Emotion-Aware Adaptive UI for a Messaging App
Kumaran, R., Doshi, V. N., Chen, X., Nargund, A. A., Höllerer, T., & Sra, M. (2023) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3586182.3616698 · Cited by 6
“Content moderators who censor objectionable content have to go through many disturbing posts on a regular basis, making content moderation an intensely stressful job [5] [8].”
BackgroundYoon, J., Zhang, A. X., & Seering, J. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711114 · Cited by 20
Making Sense of Online Discussions: Can Automated Reports help?
Anastasiou, L., & Liddo, A. D. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3451815 · Cited by 10
“This comes in diferent favours as to improve participation in communities, enable users to make better use of chat [41], to provide manageable workfows in complex crowd-sourcing environments, e.g. [6], or provide assistance to moderation systems, e.g. [5, 23, 28].”
Background“Think before you upload”: an in-depth analysis of unavailable videos on YouTube
Kurdi, M., Albadi, N., & Mishra, S. (2021) · Social Network Analysis and Mining · doi:10.1007/s13278-021-00755-x · Cited by 9
“For instance, Chandrasekharan et al. (2019) introduced a mixed-initiative systematic approach to extend the capabilities of human moderators by automatically flagging non-compliant comments in an effort to scale the moderation strategy on Reddit.”
BackgroundOpportunities, tensions, and challenges in computational approaches to addressing online harassment
Huang, E. J., Sarma, A., Hwang, S., Chandrasekharan, E., & Chancellor, S. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661623 · Cited by 15
“2A, 2B: Includes mechanisms of input ranging from ``flagging'' [66, 64, 65, 88], nominating moderators and providing them with instructions in [77], directly or indirectly configuring an automated classfier [19, 52, 55, 56, 101].”
BackgroundSelf-citationDeLink: An Adversarial Framework for Defending against Cross-site User Identity Linkage
Zhang, P., Zhou, Q., Lu, T., Gu, H., & Gu, N. (2024) · ACM Transactions on the Web · doi:10.1145/3643828 · Cited by 5
“We referred to the thoughts of cross-community learning [4] to make the classifier applicable to different platforms.”
MethodInfluentialAmmari, T., Ahn, E., Lakhankar, A., & Lee, J. Y. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710985 · Cited by 12
“In fact, banning some of the communities in the toxic masculinity sphere decreased hate content in feminist spaces [49].”
Is this a violation? Learning and understanding norm violations in online communities
dos Santos, T. F., Osman, N., & Schorlemmer, M. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2023.104058 · Cited by 9
Beyond trial-and-error: Predicting user abandonment after a moderation intervention
Tessa, B., Cima, L., Trujillo, A., Avvenuti, M., & Cresci, S. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.112375 · Cited by 15
“Along the same line, C ROSS M OD [25] automatically detects which Reddit posts are likely to be removed by the moderators.”
Adopting Third-party Bots for Managing Online Communities
Hwang, S., Kiene, C., Ong, S., & Shaw, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3653707 · Cited by 13
“For example, Chandrasekharan et al. [7] developed Crossmod , a moderation tool which could detect comments on Reddit that would have been removed by moderators with an accuracy of 86%, demonstrating how bots could automate the moderation of a great bulk of content.”
MethodPark, J., & Cordell, R. (2023) · Journal of Information Science · doi:10.1177/01655515231212977 · Cited by 7
“…cars [2] aided by computer vision; (2) researchers are using natural language processing (NLP) to detect hate speech, as a way of efficient moderation of toxic language in online communities [3]; and (3) ML is used to recommend products to improve the user experience of online shopping [4].”
Metadata
- Authors
- Chandrasekharan, E., Gandhi, C., Mustelier, M. W., & Gilbert, E.
- Affiliations
- Georgia Institute of Technology (US)
- University of Michigan (US)
- University of Michigan, Ann Arbor, MI, USA
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2019-11-07
- Volume
- 3
- Issue
- CSCW
- Pages
- 1-30
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W2988207158
- Cited by
- 151
- References
- 48
- Citations by year
- 2026: 16 · 2025: 27 · 2024: 22 · 2023: 28 · 2022: 25 · 2021: 24 · 2020: 9
- FWCI
- 6.54
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- Funders
- NSF (IIS-1553376)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- automated content moderation, socio-technical systems, ensemble classifiers, participatory design, online communities
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.