Digital detritus: 'Error' and the logic of opacity in social media content moderation
Open accessThe DOI matches its Crossref record.
Abstract
The late 2016 case of the Facebook content moderation controversy over the infamous Vietnam-era photo, “The Terror of War,” is examined in this paper for both its specifics, as well as a mechanism to engage in a larger discussion of the politics and economics of the content moderation of user-generated content. In the context of mainstream commercial social media platforms, obfuscation and secrecy work together to form an operating logic of opacity, a term and concept introduced in this paper. The lack of clarity around platform policies, procedures and the values that inform them lead users to wildly different interpretations of the user experience on the same site, resulting in confusion in no small part by the platforms’ own design. Platforms operationalize their content moderation practices under a complex web of nebulous rules and procedural opacity, while governments and other actors clamor for tighter controls on some material, and other members of civil society demand greater freedoms for online expression. Few parties acknowledge the fact that mainstream social media platforms are already highly regulated, albeit rarely in such a way that can be satisfactory to all. The final turn in the paper connects the functions of the commercial content moderation process on social media platforms like Facebook to their output, being either the content that appears on a site, or content that is rescinded: digital detritus. While meaning and intent of user-generated content may often be imagined to be the most important factors by which content is evaluated for a site, this paper argues that its value to the platform as a potentially revenue-generating commodity is actually the key criterion and the one to which all moderation decisions are ultimately reduced. The result is commercialized online spaces that have far less to offer in terms of political and democratic challenge to the status quo and which, in fact, may serve to reify and consolidate power rather than confront it.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Gorwa, R., Binns, R., & Katzenbach, C. (2020) · Big Data & Society · doi:10.1177/2053951719897945 · Cited by 1,097
“…administrators who formed part of the community, platform companies operate at a scale that has led them away from traditional practices of community moderation (Lampe and Resnick, 2004) and towards what has been termed ‘commercial content moderation’ or ‘platform moder-ation’ (Roberts, 2018).”
Background(2019) · doi:10.1080/1369118x.2019.1573914 · Cited by 719
The corporate cultivation of digital resignation
Draper, N. A., & Turow, J. (2019) · New Media & Society · doi:10.1177/1461444819833331 · Cited by 393
Wohn, D. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300390 · Cited by 191
Steiger, M., Bharucha, T. J., Venkatagiri, S., Riedl, M. J., & Lease, M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445092 · Cited by 222
A principlist framework for cybersecurity ethics
Formosa, P., Wilson, M., & Richards, D. (2021) · Computers & Security · doi:10.1016/j.cose.2021.102382 · Cited by 105
“At the most basic level, decisions must be made about what is acceptable user-generated content because the design and governance of online platforms structure how users interact with one another (for recent discussions of platform governance, see: Balkin, 2018; Roberts, 2018 ).”
BackgroundVaccaro, K., Sandvig, C., & Karahalios, K. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415238 · Cited by 102
“Researchers have studied problems of content moderation from numerous directions: the inherent inconsistency and frequent inequity of decisions [33, 62, 72, 86, 91], the dangers the content moderation process poses to moderators [33, 84, 93, 95], the harms that content moderation decisions can have for social media us…”
BackgroundContestability For Content Moderation
Vaccaro, K., Xiao, Z., Hamilton, K., & Karahalios, K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476059 · Cited by 77
SoK: Content Moderation in Social Media, from Guidelines to Enforcement, and Research to Practice
Singhal, M., Ling, C., Paudel, P., Thota, P., Kumarswamy, N., Stringhini, G., & Nilizadeh, S. (2023) · 2023 IEEE 8th European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp57164.2023.00056 · Cited by 57
“Prior researches have also corroborated that, voices of marginalized communities are often suppressed [149], [250], [270].”
BackgroundDas, A., Dang, B., & Lease, M. (2020) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v8i1.7461 · Cited by 37
We, the Data: Human Rights in the Digital Age
Wong, W. H. (2023) · doi:10.7551/mitpress/14703.001.0001 · Cited by 31
Testing Stylistic Interventions to Reduce Emotional Impact of Content Moderation Workers
Karunakaran, S., & Ramakrishan, R. (2019) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v7i1.5270 · Cited by 28
But Who Protects the Moderators? The Case of Crowdsourced Image Moderation
Dang, B., Riedl, M. J., & Lease, M. (2018) · arXiv · doi:10.48550/arxiv.1804.10999 · Cited by 26
Zhukova, E. (2022) · Visual Communication · doi:10.1177/1470357219888673 · Cited by 17
Paradoxes of Media and Information Literacy. The Crisis of Information
Haider, J., & Sundin, O. (2022) · Routledge · doi:10.25969/mediarep/19148 · Cited by 12
“I’m not this Person”: Racism, content moderators, and protecting and denying voice online
Jereza, R. (2024) · New Media & Society · doi:10.1177/14614448221122224 · Cited by 12
Metadata
- Authors
- Roberts, S. T.
- Venue
- First Monday
- Publisher
- University of Illinois Libraries
- Published
- 2018-03-01
- ISSN
- 1396-0458 (ISSN-L)
- 1396-0466
- OpenAlex
- W2791217004
- Cited by
- 122
- References
- 0
- FWCI
- 25.92
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.