CoChat

Classification and Its Consequences for Online Harassment: Design Insights from HeartMob

Blackwell, Lindsay, Dimond, Jill, Schoenebeck, Sarita, Lampe, Cliff · 2017 · Proceedings of the ACM on Human-Computer Interaction

The DOI matches its Crossref record; 33 of the 35 references it cites were found in a registry or an index.

Abstract

Online harassment is a pervasive and pernicious problem. Techniques like natural language processing and machine learning are promising approaches for identifying abusive language, but they fail to address structural power imbalances perpetuated by automated labeling and classification. Similarly, platform policies and reporting tools are designed for a seemingly homogenous user base and do not account for individual experiences and systems of social oppression. This paper describes the design and evaluation of HeartMob, a platform built by and for people who are disproportionately affected by the most severe forms of online harassment. We conducted interviews with 18 HeartMob users, both targets and supporters, about their harassment experiences and their use of the site. We examine systems of classification enacted by technical systems, platform policies, and users to demonstrate how 1) labeling serves to validate (or invalidate) harassment experiences; 2) labeling motivates bystanders to provide support; and 3) labeling content as harassment is critical for surfacing community norms around appropriate user behavior. We discuss these results through the lens of Bowker and Star's classification theories and describe implications for labeling and classifying online abuse. Finally, informed by intersectional feminist theory, we argue that fully addressing online harassment requires the ongoing integration of vulnerable users' needs into the design and moderation of online platforms.

Overview

Integrity signals

Registry

Crossref · 10.1145/3134659

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 33 references

Research works printed but not linked: 13

  1. Becker Howard, 1982, The Free Press 9
  2. Geoffrey, 2000, Bowker and Susan Leigh Star
  3. Citron Danielle Keats, Criminalizing Revenge Porn. Social Science Research Network
  4. Donath Judith, Identity and Deception in the Virtual Community. Communities in cyberspace
  5. Kiesler Sara, Regulating behavior in online communities. Building Successful Online Communities: Evidence-Based Social Design
  6. Lampe Cliff, Gamification and social media. The Gameful world: Approaches, issues, applications
  7. Lenhart Amanda, Digital Abuse, and Cyberstalking in America
  8. Marx Karl, 1848, The communist manifesto
  9. Rainie Lee, Trolls, Anonymity and Fake News Online
  10. Sproull Lee, 1992, Kiesler
  11. Star Susan Leigh, 1989, Griesemer
  12. U. S. Department of Labor. 2015., Current Population Survey: Detailed occupation by sex and race
  13. Walther Joseph B., 2002, Parks

Cited by

Items 201–246 of 246

Metadata

Authors
Blackwell, L., Dimond, J., Schoenebeck, S., & Lampe, C.
Affiliations
  • University of Michigan (US)
  • Sassafras Tech Collective, Ann Arbor, MI, USA
Venue
Proceedings of the ACM on Human-Computer Interaction
Published
2017-12-06
Volume
1
Issue
CSCW
Pages
1-19
ISSN
2573-0142 (ISSN-L)
OpenAlex
W2775336929
Cited by
245
References
35
Citations by year
2026: 15 · 2025: 41 · 2024: 34 · 2023: 32 · 2022: 31 · 2021: 34 · 2020: 22 · 2019: 22 · 2018: 14
FWCI
9.47
Type
journal article
Language
en
Open access
closed
Locations
Proceedings of the ACM on Human-Computer Interaction (publishedVersion, public-domain)
Funders
Knight Foundation
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
Keywords
online harassment, automated content moderation, platform policies, intersectionality, feminist theory, platform design, vulnerable users, community norms

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.