CoChat

Human-Machine Collaboration for Content Regulation: The Case of Reddit Automoderator

Open access
Jhaver, Shagun, Birman, Iris, Gilbert, Eric, Bruckman, Amy · 2019 · ACM Transactions on Computer-Human Interaction

The DOI matches its Crossref record; 94 of the 94 references it cites were found in a registry or an index.

Abstract

What one may say on the internet is increasingly controlled by a mix of automated programs, and decisions made by paid and volunteer human moderators. On the popular social media site Reddit, moderators heavily rely on a configurable, automated program called “Automoderator” (or “Automod”). How do moderators use Automod? What advantages and challenges does the use of Automod present? We participated as Reddit moderators for over a year, and conducted interviews with 16 moderators to understand the use of Automod in the context of the sociotechnical system of Reddit. Our findings suggest a need for audit tools to help tune the performance of automated mechanisms, a repository for sharing tools, and improving the division of labor between human and machine decision making. We offer insights that are relevant to multiple stakeholders—creators of platforms, designers of automated regulation systems, scholars of platform governance, and content moderators.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3338243

Cites

94 of 94 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

2 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 94 references

Research works printed but not linked: 7

  1. Kathy Charmaz. 2006. Coding in grounded theory practice. Constructing Grounded Theory (2006), 42-70. Sage Pub- lications Ltd, Thousand Oaks, CA.
  2. Susan C. Herring. 2000. Gender differences in CMC: Findings and implications. Computer Professionals for Social Responsibility Journal 18, 1 (2000), 1-9.
  3. Ling Jiang and Eui-Hong Han. 2019. ModBot: Automatic comments moderation. In Proceedings of the Computation+ Journalism Symposium.
  4. Victor Kaptelinin. 1996. Activity theory: Implications for human-computer interaction. Context and Consciousness: Activity Theory and Human-Computer Interaction 1 (1996), 103-116. · doi:10.5555/223826.223831
  5. Nathan J. Matias. 2016. The civic labor of online moderators. In Internet Politics and Policy conference. Oxford, United Kingdom.
  6. M. Pilar Opazo. 2010. Revitalizing the concept of sociotechnical systems in social studies of technology. (2010). Available at https://www.semanticscholar.org/paper/Revitalizing-the-Concept-of-Sociotechnical-Systems-Opazo/ 78460b4bdcbe004ccd469f4bda1f7d98017954de.
  7. Vanessa L. Wilburn. 1994. Gender and Anonymity in Computer-Mediated Communication: Participation, Flaming, Deindividuation. Ph.D. Dissertation. University of Florida.

Cited by

Items 201–300 of 420

Metadata

Authors
Jhaver, S., Birman, I., Gilbert, E., & Bruckman, A.
Affiliations
  • Georgia Institute of Technology (US)
  • University of Michigan, Ann Arbor, MI
Venue
ACM Transactions on Computer-Human Interaction
Published
2019-07-19
Volume
26
Issue
5
Pages
1-35
ISSN
  • 1073-0516 (ISSN-L)
  • 1557-7325
OpenAlex
W2964034671
Cited by
289
References
92
Citations by year
2026: 36 · 2025: 56 · 2024: 42 · 2023: 41 · 2022: 43 · 2021: 44 · 2020: 22 · 2019: 5
FWCI
49.82
Type
journal article
Language
en
Open access
Locations
ACM Transactions on Computer-Human Interaction (publishedVersion, other-oa)
Funders
National Science Foundation (IIS-1553376)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
Keywords
human-machine collaboration, content moderation, automated content moderation, Reddit, AutoMod, social media governance, platform governance, socio-technical systems, audit tool, division of labor, human moderators, tool sharing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.