Classification and Its Consequences for Online Harassment: Design Insights from HeartMob
The DOI matches its Crossref record; 33 of the 35 references it cites were found in a registry or an index.
Abstract
Online harassment is a pervasive and pernicious problem. Techniques like natural language processing and machine learning are promising approaches for identifying abusive language, but they fail to address structural power imbalances perpetuated by automated labeling and classification. Similarly, platform policies and reporting tools are designed for a seemingly homogenous user base and do not account for individual experiences and systems of social oppression. This paper describes the design and evaluation of HeartMob, a platform built by and for people who are disproportionately affected by the most severe forms of online harassment. We conducted interviews with 18 HeartMob users, both targets and supporters, about their harassment experiences and their use of the site. We examine systems of classification enacted by technical systems, platform policies, and users to demonstrate how 1) labeling serves to validate (or invalidate) harassment experiences; 2) labeling motivates bystanders to provide support; and 3) labeling content as harassment is critical for surfacing community norms around appropriate user behavior. We discuss these results through the lens of Bowker and Star's classification theories and describe implications for labeling and classifying online abuse. Finally, informed by intersectional feminist theory, we argue that fully addressing online harassment requires the ongoing integration of vulnerable users' needs into the design and moderation of online platforms.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“models are trained with historical data and can present issues such as gender bias and racial bias in AI models [68,73], potentially exacerbating structural inequalities [10]”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [10]
Citations
Cites
Mapping the Margins: Intersectionality, Identity Politics, and Violence against Women of Color
Crenshaw, K. (1991) · Stanford Law Review · doi:10.2307/1229039 · Cited by 32,792 · cited 3×
“As a result, there are severe gaps in legal protections for women of color, which white women do not experience [16].”
BackgroundInfluentialA General Inductive Approach for Analyzing Qualitative Evaluation Data
Thomas, D. R. (2006) · American Journal of Evaluation · doi:10.1177/1098214005283748 · Cited by 10,858 · cited 1×
“We used an inductive approach to develop codes [42].”
MethodStar, S. L., & Griesemer, J. R. (1989) · Social Studies of Science · doi:10.1177/030631289019003001 · Cited by 10,847
Sorting Things Out: Classification and Its Consequences
(2000) · The Journal of Academic Librarianship · doi:10.1016/s0099-1333(00)00155-5 · Cited by 5,210
Cyberbullying: Its nature and impact in secondary school pupils
Smith, P. K., Mahdavi, J., Carvalho, M., Fisher, S., Russell, S., & Tippett, N. (2008) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.2007.01846.x · Cited by 3,639 · cited 1×
“One individual may also harass another, as is often the case in instances of cyberbullying [3,39] and non-consensual image sharing (also known as “revenge porn”), a form of doxing in which sexually explicit images or videos are distributed without their subject’s consent, often by a former romantic…”
BackgroundA Room with a Viewpoint: Using Social Norms to Motivate Environmental Conservation in Hotels
Goldstein, N. J., Cialdini, R. B., & Griskevicius, V. (2008) · Journal of Consumer Research · doi:10.1086/586910 · Cited by 3,409
Ahmed, S. (2024) · Feminist Theory · doi:10.1007/978-3-031-55397-4_18 · Cited by 2,722
Ahmed, S. (2017) · doi:10.1215/9780822373377 · Cited by 2,722 · cited 2×
“Intersectional feminist theory [1,15,25] holds that various identities (such as race, gender, class, sexuality, religion, disability, and nationality) are inextricably bound in systems of entrenched structural oppression (such as racism, sexism, cissexism, classism, heterosexism, ableism, and colonialism).”
BackgroundRethinking Standpoint Epistemology: What Is ‘Strong Objectivity’?
Harding, S. (1996) · Feminism And Science · doi:10.1093/oso/9780198751458.003.0016 · Cited by 1,955
Unpacking "privacy" for a networked world
Palen, L., & Dourish, P. (2003) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/642611.642635 · Cited by 965 · cited 1×
“A digital audience is “large, unknown and distant” [34], and users must continually negotiate their own privacy and impression management practices to satisfy both their own expectations and those of a functionally invisible audience [31,34,44].”
Background
Research works printed but not linked: 13
- Becker Howard, 1982, The Free Press 9
- Geoffrey, 2000, Bowker and Susan Leigh Star
- Citron Danielle Keats, Criminalizing Revenge Porn. Social Science Research Network
- Donath Judith, Identity and Deception in the Virtual Community. Communities in cyberspace
- Kiesler Sara, Regulating behavior in online communities. Building Successful Online Communities: Evidence-Based Social Design
- Lampe Cliff, Gamification and social media. The Gameful world: Approaches, issues, applications
- Lenhart Amanda, Digital Abuse, and Cyberstalking in America
- Marx Karl, 1848, The communist manifesto
- Rainie Lee, Trolls, Anonymity and Fake News Online
- Sproull Lee, 1992, Kiesler
- Star Susan Leigh, 1989, Griesemer
- U. S. Department of Labor. 2015., Current Population Survey: Detailed occupation by sex and race
- Walther Joseph B., 2002, Parks
Cited by
Items 101–200 of 246
Rodríguez‐Rodríguez, I., & Heras-González, P. (2020) · Technology in Society · doi:10.1016/j.techsoc.2020.101274 · Cited by 38
‘ShishuShurokkha’: A Transformative Justice Approach for Combating Child Sexual Abuse in Bangladesh
Sultana, S., Pritha, S. T., Tasnim, R., Das, A., Akter, R., Hasan, S., Alam, S. M. R., Kabir, M. A., & Ahmed, S. I. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517543 · Cited by 32
Designing for Youth-Centered Moderation and Community Governance in Minecraft
Tekinbaş, K. S., Jagannath, K., Lyngs, U., & Slovák, P. (2021) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3450290 · Cited by 30
“The relationship between technical design and social practice has been a focus of much research in human–computer interaction (HCI) and related fields concerned with the impact of online participation on different groups and communities [20, 120, 158, 189].”
BackgroundHow Does Counterfactually Augmented Data Impact Models for Social Computing Constructs?
Sen, I., Samory, M., Flöck, F., Wagner, C., & Augenstein, I. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.28 · Cited by 39
“Constructs like sexism and hate speech detection are often depicted as neutral or objective but they are deeply contextual, subjective and ambiguous (Vidgen et al., 2019; Jurgens et al., 2019; Nakov et al., 2021), where misclassifications can cause harm (Blackwell et al., 2017).”
BackgroundCounterfactually Augmented Data and Unintended Bias: The Case of Sexism and Hate Speech Detection
Sen, I., Samory, M., Wagner, C., & Augenstein, I. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.347 · Cited by 49
Framing Hate with Hate Frames: Designing the Codebook
Phadke, S., Lloyd, J., E.Hawdon, J., Samory, M., & Mitra, T. (2018) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3272973.3274055 · Cited by 26
“It differs from other forms of scholarly work on hate, such as online harassment [1], cyberbullying or cyberstalking [3], where hate is targeted toward an individual instead of a collective.”
BackgroundWei, M., Consolvo, S., Kelley, P. G., Kohno, T., Roesner, F., & Thomas, K. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581229 · Cited by 34
“Other efforts aim to provide peer support for users experiencing hate and harassment (e.g., Squadbox for email [48] and the Heartmob support community [2, 9]).”
BackgroundQiwei, L., McDonald, A., Haimson, O. L., Schoenebeck, S. Y., & Gilbert, É. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686914 · Cited by 29
Self-citationMany Islands, Many Problems: An Empirical Examination of Online Safety Behaviors in the Caribbean
Wilkinson, D., & Knijnenburg, B. P. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517643 · Cited by 29
Cai, J., Chowdhury, S., Zhou, H., & Wohn, D. Y. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610191 · Cited by 34
“take marginalized users’ needs into the platform and moderation system design [5, 90].”
BackgroundTowards Ethical AI Moderation in Multiplayer Games
Sparrow, L. A., Galwey, R., Jovic, D., Hardwick, T., & Butt, M.-A. R. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3677109 · Cited by 26
“Others have suggested the potential value of democratic, user-driven governance online [10, 99] and tools that can equip bystanders with intervention capabilities [58].”
Venkatasubramanian, K., Skorinko, J. L. M., Kobeissi, M., Lewis, B., Jutras, N., Bosma, P., Mullaly, J., Kelly, B., Lloyd, D., Freark, M., et al. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445150 · Cited by 23
Alizadeh, F., Lawo, D., Stevens, G., Zytko, D., & Eslami, M. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3689710 · Cited by 23
“This is a conspicuous gap given the extensive use of algorithms in dating apps for user recommendation, and extensive research into human-inflicted harms through online dating such as sexual violence [3, 74, 107] and harassment [18, 35, 47, 60, 88].”
InfluentialExpanding Concepts of Non-Consensual Image-Disclosure Abuse: A Study of NCIDA in Pakistan
Batool, A., Naseem, M., & Toyama, K. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642871 · Cited by 26
Helping Users Tackle Algorithmic Threats on Social Media: A Multimedia Research Agenda
von der Weth, C., Abdul, A., Fan, S., & Kankanhalli, M. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3414692 · Cited by 26
“of these three threats. Note that social media has also been associated with a wide range of other threats such as online harassment through bullying, doxing, public shaming, and Internet vigilantism [22]. Social media use has also been shown to be highly addictive, often caused by the fear of missing out (FOMO) [21].…”
BackgroundReporting non-consensual pornography: clarity, efficiency and distress
Angeli, A. D., Falduti, M., Menéndez-Blanco, M., & Tessaris, S. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-022-14291-z · Cited by 32
TikTok as a Stage: Performing Rural #farmqueer Utopias on TikTok
Biggs, H. R., Marcotte, A. S., & Bardzell, S. (2023) · Designing Interactive Systems Conference · doi:10.1145/3563657.3596038 · Cited by 19
Pyle, C., Ellison, N. B., & Andalibi, N. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610087 · Cited by 22
Counterspeakers’ Perspectives: Unveiling Barriers and AI Needs in the Fight against Online Hate
Mun, J., Buerger, C., Liang, J. T., Garland, J., & Sap, M. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642025 · Cited by 45
Opinion Conflicts: An Effective Route to Detect Incivility in TwitterPreprint
Maity, S. K., Chakraborty, A., Goyal, P., & Mukherjee, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.00289 · Cited by 15
“This may directly enhance civility in the platform making account holders less likely to be invective (similar observations in the context of trolling and online harassment has been made in earlier works [8, 23]).”
Unmasking Nationality Bias: A Study of Human Perception of Nationalities in AI-Generated Articles
Venkit, P. N., Gautam, S., Panchanadikar, R., Huang, T.-H., & Wilson, S. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604667 · Cited by 42
Cultivating the Community: Inferring Influence within Eating Disorder Networks on Twitter
Nova, F. F., Coupe, A., Mynatt, E. D., Guha, S., & Pater, J. A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3492826 · Cited by 23
Muralikumar, M. D., Yang, Y. S., & McDonald, D. W. (2023) · ACM Transactions on Social Computing · doi:10.1145/3582568 · Cited by 30
Neubauer, L., Straw, I., Mariconti, E., & Tanczer, L. M. (2023) · Journal of Family Violence · doi:10.1007/s10896-023-00517-7 · Cited by 20
“…could contribute to the invalidation and minimisation of different experiences of abuse, for example by classifying an instance as ‘not abuse’ and leading to a victim-survivor not receiving services or justice after having experienced great harm (Blackwell, Dimond, Schoenebeck, & Lampe, 2017).”
BackgroundSammut, F., Bezzina, M., & Scerri, J. (2023) · Journalism Practice · doi:10.1080/17512786.2023.2294290 · Cited by 19
“Online harassment, also known as cyberharassment or cyberbullying, is a pernicious phenomenon that has become increasingly pervasive in modern global society (Blackwell et al. 2017; Schoenebeck, Lampe, and Tri ệ u 2023; Yardi and Bruckman 2011).”
When Does it Become Harassment? An Investigation of Online Criticism and Calling Out in Twitter
Kim, H., Kim, H., Kim, J., & Jang, J.-W. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555575 · Cited by 28
“deserved when the target has committed some offense [13].”
BackgroundResultInfluentialCai, J., Patel, A., Naderi, A., & Wohn, D. Y. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650882 · Cited by 21
“Online abuse such as racial slurs, bullying, sexual harassment, spam, trolls, and hate speech toward a specific group or individuals are also considered violations [2, 19].”
Canadian Gender-Based Violence Prevention Programs: Gaps and Opportunities
Chen, Y., Mendes, K., Gosse, C. E., Hodson, J., & Veletsianos, G. (2024) · Violence Against Women · doi:10.1177/10778012241259727 · Cited by 14
“As yet, bystander intervention research and training in the context of online violence and abuse is minimal, with a few notable exceptions such as HeartMob (Blackwell et al., 2017; Henson et al., 2020).”
PressProtect: Helping Journalists Navigate Social Media in the Face of Online Harassment
Han, C., Li, A., Kumar, D., & Durumeric, Z. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687048 · Cited by 17
Tracing knowledge diffusion trajectories in the research field of cyberbullying
Rejeb, A., Rejeb, K., Zrelli, I., & Süle, E. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e41141 · Cited by 22
“Significantly, those affected by cyberbullying often experience traditional bullying as well, indicating a pervasive cycle of harassment [8,9].”
Operationalising ‘toxicity’ in the manosphere: Automation, platform governance and community health
Trott, V. A., Beckett, J. R., & Paech, V. (2022) · Convergence The International Journal of Research into New Media Technologies · doi:10.1177/13548565221111075 · Cited by 29
“In an effort to better cope with content moderation, to combat hate speech, ‘dangerous organisations’ and other bad actors present on platforms, discussion has turned to the role that automated machine-learning (ML) tools might play (Blackwell et al., 2017; Gorwa et al. 2020).”
BackgroundBeyond the Boolean: How Programmers Ask About, Use, and Discuss Gender
Bouma-Sims, E., & Acar, Y. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579461 · Cited by 11
How Content Authored by People with Dementia Affects Attitudes towards Dementia
Lazar, A., Brewer, R. N., Kacorri, H., Hong, J., Punzalan, M. N. D., Mahathir, M., Hyde, O. V., & Ross, W., III (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479542 · Cited by 12
Porn bans, purges, and rebirths: the biopolitics of platform death in queer fandoms
Floegel, D. (2021) · Internet Histories · doi:10.1080/24701475.2021.1985833 · Cited by 16
"How Do You Quantify How Racist Something Is?": Color-Blind Moderation in Decentralized Governance
Wu, Q., & Semaan, B. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610030 · Cited by 13
“Scholars have studied the moderation of problematic content in online communities, focusing on harassment [4, 5], hate speech [12], and commercial content [76], and have attempted to develop better tools to fix myriad issues [9, 14, 28, 36, 39, 92].”
BackgroundFreeman, G., Frommel, J., Mandryk, R. L., Gugenheimer, J., Li, L., & Johnson, D. M. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3636288 · Cited by 11
“…and harmful interactions (e.g., harassment, verbal abuse, hate speech, and flaming on social media platforms and online forums) remains a severe and pervasive issue in today’s online social spaces, which seriously damages vic-tims’ mental, emotional, and physical well-being [6, 22, 51].”
Ma, R., Zhang, Z., Gui, X., & Kou, Y. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661565 · Cited by 15
"We're not all construction workers": Algorithmic Compression of Latinidad on TikTok
Lutz, N., & Aragon, C. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687019 · Cited by 20
Freeman, G., Frommel, J., Mandryk, R. L., Gugenheimer, J., Li, L., Johnson, D. M., Aragon, C., Asif, S. A., Bailey, J. O., Barkallah, M., et al. (2024) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3678884.3681865 · Cited by 13
“…problematic, and harmful interactions (e.g., harassment, verbal abuse, hate speech, and flaming on social media platforms and online forums) remains a severe and pervasive issue in today’s online social spaces, which seriously damages victims’ mental, emotional, and physical well-being [8, 31, 64].”
Burger, B., Tebbe, D., Walquist, E. M., Kind, T., & Zytko, D. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713236 · Cited by 13
Predicting Different Types of Subtle Toxicity in Unhealthy Online Conversations
Gilda, S., Giovanini, L., Silva, M., & Oliveira, D. S. (2022) · Procedia Computer Science · doi:10.1016/j.procs.2021.12.254 · Cited by 13
“The importance of community norms in detection and classification of these subtler forms of abuse have been noted elsewhere (Blackwell et al., 2017; Guberman and Hemphill, 2017; Liu et al., 2018; Salminen et al., 2018), but have not received the same attention in the NLP community.”
BackgroundTowards Stalkerware Detection with Precise Warnings
Han, Y., Roundy, K., & Tamersoy, A. (2021) · Annual Computer Security Applications Conference · doi:10.1145/3485832.3485901 · Cited by 16
“Its manifestations include many forms of harassment [7, 25, 41], character assassination through faked revenge porn [45], impersonation attacks that damage the targeted individual’s relationships [27, 42], and above all, spying on an intimate partner through stalkerware and other means, such as knowledge of the surviv…”
BackgroundLee, K. S., & Wei, H. (2022) · Journal of Media Ethics · doi:10.1080/23736992.2022.2107524 · Cited by 17
Online harassment of Japanese celebrities and influencers
Takano, M., Taka, F., Ogiue, C., & Nagata, N. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1386146 · Cited by 13
Gao, J., Yin, H., & Han, Z. (2024) · Social Psychology of Education · doi:10.1007/s11218-024-09904-4 · Cited by 13
“Identifying school bullying constitutes a foundational step in its prevention (Blackwell et al., 2017; Thornberg et al., 2012 ).”
Wang, X., Wang, Z., Zhang, M., Yu, K., Hui, P., & Fan, M. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686934 · Cited by 14
Disentangling the Power Dynamics in Participatory Data Physicalisation
Cazacu, S., Panagiotidou, G., Steenberghen, T., & Moere, A. V. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713703 · Cited by 15
Freeman, G., Li, L., & Schulenberg, K. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711005 · Cited by 13
A Critical Analysis of the Prevalence of Technology-Facilitated Abuse in US College Students
Gupta, N., Das, S., Walsh, K., & Chatterjee, R. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3652036 · Cited by 16
The Use of Negative Interface Cues to Change Perceptions of Online Retributive Harassment
Lee-Kan, S. M., Thomer, A. K., & Lampe, C. A. C. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555226 · Cited by 14
Self-citationTowards Ethics by Design in Online Abusive Content DetectionPreprint
Kiritchenko, S., & Nejadgholi, I. (2020) · arXiv (Cornell University) · doi:10.4224/40002027 · Cited by 18
AppealMod: Inducing Friction to Reduce Moderator Workload of Handling User Appeals
Atreja, S., Im, J., Resnick, P., & Hemphill, L. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637296 · Cited by 16
Freeman, G., Schulenberg, K., Li, L., Panchanadikar, R., & McNeese, N. J. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713473 · Cited by 14
“With growing concerns over harassment against women in embod-ied social VR spaces [7–9, 27, 57, 58, 71, 80, 85], in this work, we turn an eye towards an understudied lens (i.e., AI companions ) and have revealed women social VR users’ own envisionings for leveraging AI as Accessible Companions ,…”
Building Solidarity Amid Hostility: Experiences of Fat People in Online Communities
Payne, B. H., Taylor, J., Spiel, K., & Fiesler, C. L. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3701186 · Cited by 19
“Future work should explore how to help fat people find existing online communities while also keeping out trolls or interlopers.”
A Novel Methodology for Developing Automatic Harassment Classifiers for Twitter
Arora, I., Guo, J., Levitan, S. I., McGregor, S. E., & Hirschberg, J. (2020) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2020.alw-1.2 · Cited by 17
“This indicates the importance of expert annotators, which (Blackwell et al., 2017)”
BackgroundPark, J., Yang, J., Tolbert, A. N., & Bunsold, K. (2024) · Journal of Information Science · doi:10.1177/01655515241238405 · Cited by 12
Wu, Q., Romero, T., & Semaan, B. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642688 · Cited by 14
The Design Space for Online Restorative Justice Tools: A Case Study with ApoloBot
Doan, B. N. (., & Seering, J. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713598 · Cited by 26
“These tools help in tasks both proactively, such as setting up automatic content filtering [15], sending nudges to potential violations [60], classification [8], and limiting the reach of content [6], and reactively, such as issuing warnings, bans, and handling user appeals [2].”
Behind the Counter: Exploring Motivations and Barriers of Online Counterspeech Writing
Ping, K., Kumar, A., Ding, X., & Rho, E. H. R. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3745769 · Cited by 17
“However, these models are nonetheless limited in their capacity to discern subtle nuances [17, 122] or cultural contexts [88] in hate speech, or distinguish between implicit and explicit forms of hate speech [97, 133].”
Hernández, A. D., Ramokapane, K. M., Chowdhury, P. D., Michalec, O., Johnstone, E., Godwin, E., Cork, A., & Rashid, A. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610179 · Cited by 12
Chancellor, S., Feuston, J. L., & Chang, J. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610181 · Cited by 16
“[11] found similar findings regarding the consequences of harassment classification.”
BackgroundResultAcoustic and Visual Approaches to Adversarial Text Generation for Google Perspective
Brown, S., Milkov, P., Patel, S., Looi, Y. Z., Dong, Z., Gu, H., Artan, N. S., & Jain, E. (2019) · International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci49370.2019.00069 · Cited by 7
“Toolswhich label online abuse can also validate harassment experiences, motivate bystanders to provide support, and help normalize appropriate user behavior [1].”
BackgroundDeep recurrent neural network‐based Aquila optimization‐based online shaming emotion analysis
Aarthi, B., & Chelliah, B. J. (2022) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.6882 · Cited by 7
“G1 is the AO motion tracker while conducting the elope motion and it can be estimated as, G1 = 2 × ran() − 1 (19)”
BackgroundSkilled or Gullibleƒ Gender Stereotypes Related to Computer Security and Privacy
Wei, M., Emami-Naeini, P., Roesner, F., & Kohno, T. (2023) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46215.2023.10179469 · Cited by 26
“, as survivors of intimate partner violence [36], users of menstruation [37], [38] or women-specific apps [39], or victims of gender-based harassment [40], [41].”
BackgroundIdentification and Classification of Cyberbully Incidents using Bystander Intervention Model
(2019) · International Journal of Recent Technology and Engineering (IJRTE) · doi:10.35940/ijrte.b1001.0782s419 · Cited by 7
Toupin, S. (2020) · Wiley eBooks · doi:10.1002/9781119537151.ch23 · Cited by 8
Spence, R., & DeMarco, J. (2025) · Behavioral Sciences · doi:10.3390/bs15040487 · Cited by 6
Fighting for Their Voice: Understanding Indian Muslim Women's Responses to Networked Harassment
Bhimdiwala, A., Adavi, K. A. K., & Arif, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641005 · Cited by 13
'Socheton': A Culturally Appropriate AI Tool to Support Reproductive Well-being
Sultana, S., Chowdhury, H. M., Sultana, Z., & Verdezoto, N. X. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735725 · Cited by 12
Building a Personalized Model for Social Media Textual Content Censorship
Liu, B., Zhang, P., Shu, Y., Guan, Z., Lu, T., Gu, H., & Gu, N. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555657 · Cited by 10
“Today, social media users are suffering from content over disclosure [4, 5, 23], which has raised concern in CSCW and HCI communities [11, 19, 21, 35, 51].”
BackgroundFoundations, Definitions, and Directions in Online Hate Research
Tong, S. T. (2024) · Routledge eBooks · doi:10.4324/9781003472148-3 · Cited by 11
'I Blow Up': Understanding TikTok Users' Reactions to Sudden Social Media Attention
Schafer, J. S., Denton, A., Seelhoff, C., Vo, J., Garcia, L., Madan, I., Mudbhary, A., Tang, R., & Starbird, K. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711001 · Cited by 13
“…has previously differentiated between ‘positive’ and ‘negative’ bursts of sudden attention on social media (as negative attention is often looked at through coordinated harassment, hate raids, and brigading lenses, e.g. [13, 28, 39]), but here we see a more complicated emotional experience.”
Examining Digital Hate: Gendered discursive strategies in online harassment of Pakistani influencers
Yasmin, M., Jabeen, T., & Noor, S. (2025) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2025.100759 · Cited by 10
ReSPect: Enabling Active and Scalable Responses to Networked Online Harassment
Kim, H., Lee, J., Jang, J.-W., & Kim, J. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637394 · Cited by 12
Perceiving and Countering Hate: The Role of Identity in Online Responses
Ping, K., E.Hawdon, J., & Rho, E. H. R. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711045 · Cited by 9
“While prior scholarship has mostly focused on the impact of counterspeech on the hate speakers or its effectiveness [12, 29, 129], our work adds a new layer of depth by empirically validating TIM.”
Design for Hope: Cultivating Deliberate Hope in the Face of Complex Societal Challenges
Kim, J., Liu, J., Popowski, L., Pyle, C., Arif, A., Hayes, G. R., Hiniker, A., Ju, W. G., Mueller, F., Shen, H., et al. (2025) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3715070.3748287 · Cited by 13
Identifying Themes of Right-Wing Extremism in Hindutva Discourse on Twitter
Ghasiya, P., Ahnert, G., & Sasahara, K. (2023) · Social Media + Society · doi:10.1177/20563051231199457 · Cited by 8
“Such toxic behaviors on social media platforms, which also include harassment and incivil-ity, are of growing concern, and therefore, researchers have increasingly investigated their mechanics (Blackwell et al., 2017; Borah, 2014; Cheng et al., 2015).”
BackgroundInterrogating Datafication: Towards a Praxeology of Data
Burkhardt, M., van Geenen, D., Gerlitz, C., Hind, S., Kaerlein, T. 1., Lämmerhirt, D., & Volmar, A. (2022) · Media in action · doi:10.1515/9783839455616 · Cited by 22
Opportunities, tensions, and challenges in computational approaches to addressing online harassment
Huang, E. J., Sarma, A., Hwang, S., Chandrasekharan, E., & Chancellor, S. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661623 · Cited by 15
Trust and Friction: Negotiating How Information Flows through Decentralized Social Media
Hwang, S., Nanayakkara, P., & Shvartzshnaider, Y. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757516 · Cited by 4
Stoking the Flames: Understanding Escalation in an Online Harassment Community
Take, K., Zhong, V., Geeng, C., Bevensee, E., McCoy, D., & Greenstadt, R. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641015 · Cited by 10
Lou, S., Li, W., Zhang, C., Chen, S., Lu, Z., & Yao, Y. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710933 · Cited by 11
“For these female participants, it fosters inclusive and supportive interactions, mitigating the impact of harassment, prejudice, and other discriminatory practices, particularly in online spaces where women frequently encounter such challenges [13, 37, 81].”
Tran, C., Champion, K., Hill, B. M., & Greenstadt, R. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555225 · Cited by 6
The Ethics of Conducting Research with Human Participants in HCI
Petrie, H. (2025) · Springer eBooks · doi:10.1007/978-3-319-27648-9_86-1 · Cited by 4
Modeling Considerations for Quantitative Social Science Research Using Social Media DataPreprint
Budak, C., Soroka, S., Singh, L., Bailey, M., Bode, L., Chawla, N. V., Davis-Kean, P. E., Choudhury, M. D., Veaux, R. D. D., Hahn, U., et al. (2021) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/3e2ux · Cited by 4
Stephenson, S., Gupta, N., Huang, K., Youssef, D. E., Cowan, K., & Chatterjee, R. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790296 · Cited by 5
Usman, W., Taha, T., Iqbal, S., Batool, A., & Zappala, D. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3793807 · Cited by 4
“These patterns align with prior scholarship that conceptualizes harassment as socially constructed and shaped by cultural and institutional power relations rather than fixed psychological or legal criteria [10, 12, 20, 31, 56, 61, 66].”
Parasocial media: The mass production of intimacy on a Chinese pop idol mobile application
Shen, X., & Hallinan, B. (2024) · Platforms & Society · doi:10.1177/29768624241299954 · Cited by 7
“…design of social media can be a point of frustration (Hallinan and Brubaker, 2021) and explicit classi fi cations can validate user experiences (Blackwell et al., 2017), Pocket48 strategically exploits symbolic ambiguity to create space for fans to imagina-tively interpret their interactions…”
Todorović, M., Kozakijevic, S., Jovanović, L., Babic, L., Antonijević, M., Zeljkovic, V., Zivkovic, M. Z., & Bačanin, N. (2025) · SN Computer Science · doi:10.1007/s42979-025-04122-x · Cited by 5
“Online harassment can take many different forms, but some prominent instances include intentionally provoking someone, exposing someone else’s private information without their knowledge, and posting insulting, hostile, and cruel remarks and private messages [1].”
Tech Pains: Characterizations of Lived Cybersecurity Experiences
Tian, H., Kanich, C., Polakis, J., & Patil, S. (2020) · IEEE European Symposium on Security and Privacy Workshops (EuroS&PW) · doi:10.1109/eurospw51379.2020.00040 · Cited by 6
“Nonetheless, recent research has found that systems labeling abusive online actions as “harassment” can provide validation and support to affected users [59].”
BackgroundHaque, I., Adnin, R., Afroz, S., Huq, F., Mahbub, S., & Islam, A. B. M. A. A. (2021) · International Conference on Networking, Systems and Security (NSysS) · doi:10.1145/3491371.3491374 · Cited by 3
“Despite the growing interest in content moderation in the fields of Human-Computer Interaction (HCI) and Computer-Supported Cooperative Work (CSCW), most research on abuse detection and moderation have focused on abuse in social platforms [8], [5].”
BackgroundQin, Y., Yin, S., & Liu, F. (2023) · Journal of the Knowledge Economy · doi:10.1007/s13132-023-01498-0 · Cited by 3
“Moreover, drawing on Labeling Theory could provide a nuanced perspective on how legal definitions and categorizations of neutral help behavior impact the e-commerce landscape (Blackwell et al., 2017).”
RetractedWilkinson, D., Hanley, E., & Knijnenburg, B. P. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686901 · Cited by 6
Tweety Holmes: A Browser Extension for Abusive Twitter Profile Detection
Kwon, S., Liang, P., Tandon, S., Berman, J., Chang, P.-J., & Gilbert, É. (2018) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3272973.3272991 · Cited by 7
“Online harassment in social media platforms has been considered as a dark side of social computing and many researchers have sought why and how these abusers proliferate and looked into the effectiveness of different methods of mitigating this issue: Crowdsourcing [6], self-reports [2], blocklists [4, 5], and bot inte…”
Valentine, M., Bohn, R. E., Pratt, A. L., Jain, P., Singer, S. J., & Bernstein, M. S. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687029 · Cited by 3
“Power, then, is held by those who are creating the labels” [5, p. 24:4].”
Beyond the Enforcer: Exploring Metaphors for Alternative AI Moderation in Online Games
Galwey, R., Sparrow, L. A., & Butt, M.-A. R. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3720148 · Cited by 3
Towards Youth-Sensitive Hateful Content Reporting: An Inclusive Focus Group Study in Germany
Bäumler, J., Bader, H., Kaufhold, M., & Reuter, C. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713542 · Cited by 7
Dissecting User Experience of Social Virtual Reality: A Tale of Five Platforms
Cheng, R., Li, J., Chen, S., & Han, B. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757405 · Cited by 2
Huang, P. Y., Deng, J., Yang, Y., & Williams, S. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757589 · Cited by 5
“Yet , these mitigation strategies face notable shortcomings, such as inadequate follow-through in the reporting process [11], vague and incomplete guidelines [65], and emotional burden on moderators [83].”
Schoenebeck, S. Y., Shen, Y. Y., & Davidson, J. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3567557 · Cited by 2
“While extensive research has examined the nature of online harassment (e.g. [ 13, 14, 22, 42, 46, 53]) and how to moderate content (e.g. [ 28, 36, 49, 63, 65]), less work has focused on the design of social media profiles in the context of online harassment.”
BackgroundSelf-citation
Metadata
- Authors
- Blackwell, L., Dimond, J., Schoenebeck, S., & Lampe, C.
- Affiliations
- University of Michigan (US)
- Sassafras Tech Collective, Ann Arbor, MI, USA
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2017-12-06
- Volume
- 1
- Issue
- CSCW
- Pages
- 1-19
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W2775336929
- Cited by
- 245
- References
- 35
- Citations by year
- 2026: 15 · 2025: 41 · 2024: 34 · 2023: 32 · 2022: 31 · 2021: 34 · 2020: 22 · 2019: 22 · 2018: 14
- FWCI
- 9.47
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion, public-domain)
- Funders
- Knight Foundation
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- online harassment, automated content moderation, platform policies, intersectionality, feminist theory, platform design, vulnerable users, community norms
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.