Human-Machine Collaboration for Content Regulation: The Case of Reddit Automoderator
Open accessThe DOI matches its Crossref record; 92 of the 92 references it cites were found in a registry or an index.
Abstract
What one may say on the internet is increasingly controlled by a mix of automated programs, and decisions made by paid and volunteer human moderators. On the popular social media site Reddit, moderators heavily rely on a configurable, automated program called “Automoderator” (or “Automod”). How do moderators use Automod? What advantages and challenges does the use of Automod present? We participated as Reddit moderators for over a year, and conducted interviews with 16 moderators to understand the use of Automod in the context of the sociotechnical system of Reddit. Our findings suggest a need for audit tools to help tune the performance of automated mechanisms, a repository for sharing tools, and improving the division of labor between human and machine decision making. We offer insights that are relevant to multiple stakeholders—creators of platforms, designers of automated regulation systems, scholars of platform governance, and content moderators.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Notably, the task for humans to create conditional delegation rules share some similarity with what many social media moderators are already doing by writing manual automation rules to deal with the massive amount of comments (Fig. 1(C)).”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [44]
The cited source says what the claim attributes to it.
In the citing paper
“For example, moderators on Reddit use a tool called AutoModerator, with which they manually customize a rule-based system to automatically identify comments for deleting or reporting for further review [18,44].”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [44]
The cited source says what the claim attributes to it.
In the citing paper
“However, there exists a need for audit tools to monitor the performance of the keyword rules.”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [44]
In the citing paper
“They also highlight the fact that AutoModerator fundamentally changes the work of moderators and may introduce additional unnecessary work.”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [44]
In the citing paper
“Through interviews with 16 moderators, Jhaver et al. [44] found that AutoModerator improves the efciency of moderation.”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [44]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1990) · Cited by 24,217
Qualitative evaluation and research methods
Ellis, D. (1990) · International Journal of Information Management · doi:10.1016/0268-4012(90)90041-p · Cited by 17,943
Basics of Qualitative Research
Ozanne, J. L., Strauss, A., & Corbin, J. (1992) · Journal of Marketing Research · doi:10.1177/002224379202900312 · Cited by 17,867
Basics of Qualitative Research
Ozanne, J. L., Strauss, A., & Corbin, J. (1992) · Journal of Marketing Research · doi:10.2307/3172751 · Cited by 17,850
The Managed Heart: Commercialization of Human Feeling
Tropman, J. E., & Hochschild, A. R. (1984) · Journal of Policy Analysis and Management · doi:10.2307/3324333 · Cited by 8,746
An Introduction to Qualitative Research
Flick, U. (2023) · SAGE Publications Ltd eBooks · doi:10.4135/9781036231712 · Cited by 8,198
Introduction to Qualitative Research Methods: A Guidebook and Resource
Taylor, S. J., & Bogdan, R. (2015) · Cited by 2,935
The New Governors: The People, Rules, and Processes Governing Online Speech
Klonick, K. (2018) · St. John's University School of Law (St. John's University) · Cited by 1,183
The Intellectual Challenge of CSCW: The Gap Between Social Requirements and Technical Feasibility
Ackerman, M. S. (2000) · Human–Computer Interaction · doi:10.1207/s15327051hci1523_5 · Cited by 892
Programming Techniques: Regular expression search algorithm
Thompson, K. (1968) · Communications of the ACM · doi:10.1145/363347.363387 · Cited by 854
Citron, D. K. (2014) · Harvard University Press eBooks · doi:10.4159/harvard.9780674735613 · Cited by 724
(2019) · doi:10.1080/1369118x.2019.1573914 · Cited by 719
Interpretable Decision Sets: A Joint Framework for Description and Prediction
Lakkaraju, H., Bach, S. H., & Leskovec, J. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939874 · Cited by 687
Building Successful Online Communities: Evidence-Based Social Design
Kraut, R. E., Resnick, P., Kiesler, S. B., Ren, Y., Chen, Y., Burke, M., Kittur, N., Riedl, J., & Konstan, J. A. (2012) · Cited by 636
Ex Machina: Personal Attacks Seen at Scale
Wulczyn, E., Thain, N., & Dixon, L. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052591 · Cited by 636
Citron, D. K., & Franks, M. A. (2014) · Digital Commons at University of Maryland Carey Law (University of Maryland Francis King Carey School of Law) · Cited by 579
Gillespie, T. (2018) · Cited by 511
What is a flag for? Social media reporting tools and the vocabulary of complaint
Crawford, K., & Gillespie, T. (2016) · New Media & Society · doi:10.1177/1461444814543163 · Cited by 496
Myers West, S. (2018) · New Media & Society · doi:10.1177/1461444818773059 · Cited by 477
Slash(dot) and burn: distributed moderation in a large online conversation space
Lampe, C., & Resnick, P. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985761 · Cited by 400
Governing online platforms: From contested to cooperative responsibility
Helberger, N., Pierson, J., & Poell, T. (2018) · The Information Society · doi:10.1080/01972243.2017.1391913 · Cited by 374
Grimmelmann, J. (2015) · eYLS (Yale Law School) · doi:10.31228/osf.io/qwxf5 · Cited by 374
Design Justice: towards an intersectional feminist framework for design theory and practice
Costanza-Chock, S. (2018) · Proceedings of DRS · doi:10.21606/drs.2018.679 · Cited by 325
Towards quality discourse in online news comments
Diakopoulos, N., & Naaman, M. (2011) · Proceedings of the ACM 2011 conference on Computer supported cooperative work · doi:10.1145/1958824.1958844 · Cited by 318
Moderator engagement and community development in the age of algorithms
Seering, J., Wang, T., Yoon, J., & Kaufman, G. (2019) · New Media & Society · doi:10.1177/1461444818821316 · Cited by 310
Chandrasekharan, E., Samory, M., Jhaver, S., Charvat, H., Bruckman, A., Lampe, C., Eisenstein, J., & Gilbert, E. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274301 · Cited by 282
Self-citationA Socio-Technical Approach to Systems Design
Mumford, E. (2000) · Requirements Engineering · doi:10.1007/pl00010345 · Cited by 279
Online Harassment and Content Moderation: The Case of Blocklists
Jhaver, S., Ghoshal, S., Bruckman, A., & Gilbert, E. (2018) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3185593 · Cited by 270
Self-citationSuzor, N. P., West, S. M., Quodling, A., & York, J. C. (2019) · International journal of communication · Cited by 257
#thyghgapp: Instagram Content Moderation and Lexical Variation in Pro-Eating Disorder Communities
Chancellor, S., Pater, J. A., Clear, T., Gilbert, E., & De Choudhury, M. (2016) · Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing · doi:10.1145/2818048.2819963 · Cited by 255
The work of sustaining order in wikipedia: the banning of a vandal
Geiger, R. S., & Ribes, D. (2010) · Proceedings of the 2010 ACM conference on Computer supported cooperative work · doi:10.1145/1718918.1718941 · Cited by 245
Classification and Its Consequences for Online Harassment: Design Insights from HeartMob
Blackwell, L., Dimond, J., Schoenebeck, S., & Lampe, C. (2017) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3134659 · Cited by 245 · cited 1×
“found that using automated approaches to identify abusive language can result in situations where the concerns of only the dominant groups are emphasized and existing structural inequalities are exacerbated [8].”
BackgroundShaping Pro and Anti-Social Behavior on Twitch Through Moderation and Example-Setting
Seering, J., Kraut, R., & Dabbish, L. (2017) · Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/2998181.2998277 · Cited by 236
Beyond the hashtag: Circumventing content moderation on social media
Gerrard, Y. (2018) · New Media & Society · doi:10.1177/1461444818776611 · Cited by 210
Racist comments at online news sites: a methodological dilemma for discourse analysis
Hughey, M. W., & Daniels, J. (2013) · Media, Culture & Society · doi:10.1177/0163443712472089 · Cited by 203
Jhaver, S., Bruckman, A., & Gilbert, E. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359252 · Cited by 203
Self-citationWohn, D. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300390 · Cited by 191
Dosono, B., & Semaan, B. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300372 · Cited by 178
Commercial Content Moderation: Digital Laborers' Dirty Work
Roberts, S. (2016) · Scholarship@Western (Western University) · Cited by 171
Prevalence and Psychological Effects of Hateful Speech in Online College Communities
Saha, K., Chandrasekharan, E., & De Choudhury, M. (2019) · Proceedings of the 10th ACM Conference on Web Science · doi:10.1145/3292522.3326032 · Cited by 170
Reddit Rules! Characterizing an Ecosystem of Governance
Fiesler, C., Jiang, J., McCann, J., Frye, K., & Brubaker, J. (2018) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v12i1.15033 · Cited by 165
Lampe, C., Zube, P., Lee, J., Park, C. H., & Johnston, E. (2014) · Government Information Quarterly · doi:10.1016/j.giq.2013.11.005 · Cited by 160
Regulating Behavior in Online Communities
Kiesler, S. B., Kraut, R. E., Resnick, P., & Kittur, A. (2010) · KiltHub Repository · doi:10.1184/r1/6470384.v1 · Cited by 158
Behind the screen: the hidden digital labor of commercial content moderation
Roberts, S. (2014) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 156
The Bag of Communities: Identifying Abusive Behavior Online with Preexisting Internet Data
Chandrasekharan, E., Samory, M., Srinivasan, A., & Gilbert, E. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3026018 · Cited by 153
(2015) · Choice Reviews Online · doi:10.5860/choice.187841 · Cited by 153
Jhaver, S., Appling, D. S., Gilbert, E., & Bruckman, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359294 · Cited by 139
Self-citationBehind the Screen: Content Moderation in the Shadows of Social Media
Roberts, S. (2019) · Cited by 132
Machine Learning in Adversarial Settings
McDaniel, P., Papernot, N., & Celik, Z. B. (2016) · IEEE Security & Privacy · doi:10.1109/msp.2016.51 · Cited by 120
Governance of and by platforms
Gillespie, T. (2016) · Cited by 118
Going Dark: Social Factors in Collective Action Against Platform Operators in the Reddit Blackout
Matias, J. N. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858391 · Cited by 117
Flock: Hybrid Crowd-Machine Learning Classifiers
Cheng, J., & Bernstein, M. S. (2015) · Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing · doi:10.1145/2675133.2675214 · Cited by 115
Algorithmic Anxiety and Coping Strategies of Airbnb Hosts
Jhaver, S., Karpfen, Y., & Antin, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173995 · Cited by 114
Self-citationFollow the reader: filtering comments on slashdot
Lampe, C. A. C., Johnston, E., & Resnick, P. (2007) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240815 · Cited by 113
Platform dialectics: The relationships between volunteer moderators and end users on reddit
Squirrell, T. (2019) · New Media & Society · doi:10.1177/1461444819834317 · Cited by 113
Like Trainer, Like Bot? Inheritance of Bias in Algorithmic Content Moderation
Binns, R., Veale, M., Kleek, M. G. V., & Shadbolt, N. (2017) · UCL Discovery (University College London) · Cited by 113
Perceptions of e‐moderators about their roles and functions in moderating electronic mailing lists
Berge, Z. L., & Collins, M. P. (2000) · Distance Education · doi:10.1080/0158791000210106 · Cited by 104
Quandt, T. (2020) · Digital Journalism · doi:10.1080/21670811.2020.1724517 · Cited by 102
Supporting Comment Moderators in Identifying High Quality Online News Comments
Park, D., Sachar, S., Diakopoulos, N., & Elmqvist, N. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858389 · Cited by 98
Kerr, A., & Kelleher, J. D. (2015) · Critical Studies in Media Communication · doi:10.1080/15295036.2015.1045005 · Cited by 94
Liu, P., Guberman, J., Hemphill, L., & Culotta, A. (2018) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v12i1.15022 · Cited by 92
Surviving an "Eternal September": How an Online Community Managed a Surge of Newcomers
Kiene, C., Monroy-Hernández, A., & Hill, B. M. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858356 · Cited by 90
Multimodal Classification of Moderated Online Pro-Eating Disorder Content
Chancellor, S., Kalantidis, Y., Pater, J. A., De Choudhury, M., & Shamma, D. A. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025985 · Cited by 86
Like Trainer, Like Bot? Inheritance of Bias in Algorithmic Content Moderation
Binns, R., Veale, M., Van Kleek, M., & Shadbolt, N. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-67256-4_32 · Cited by 81
See No Evil, Hear No Evil: Audio-Visual-Textual Cyberbullying Detection
Soni, D., & Singh, V. K. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274433 · Cited by 75
Seering, J., Fang, T., Damasco, L., Chen, M. '., Sun, L., & Kaufman, G. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300836 · Cited by 71
When the levee breaks: without bots, what happens to Wikipedia's quality control processes?
Geiger, R. S., & Halfaker, A. (2013) · Proceedings of the 9th International Symposium on Open Collaboration · doi:10.1145/2491055.2491061 · Cited by 67
Regulating Behavior in Online Communities
Ren, Y., Kraut, R., Kiesler, S. B., & Resnick, P. (2018) · KiltHub Repository · doi:10.1184/r1/6470384 · Cited by 62
Teaching Students to Study Online Communities Ethically
Bruckman, A. (2006) · Journal of Information Ethics · doi:10.3172/jie.15.2.82 · Cited by 61
Self-citation"Could You Define That in Bot Terms"?: Requesting, Creating and Using Bots on Reddit
Long, K., Vines, J., Sutton, S., Brooker, P., Feltwell, T., Kirman, B., Barnett, J., & Lawson, S. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025830 · Cited by 60
Interacting with bots online: Users’ reactions to actions of automated programs in Wikipedia
Clément, M., & Guitton, M. J. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.03.078 · Cited by 54
Behavioral analysis and classification of spammers distributing pornographic content in social media
Singh, M., Bansal, D., & Sofat, S. (2016) · Social Network Analysis and Mining · doi:10.1007/s13278-016-0350-0 · Cited by 52
Automatic Moderation of Online Discussion Sites
Delort, J.-Y., Arunasalam, B., & Paris, C. L. (2011) · International Journal of Electronic Commerce · doi:10.2753/jec1086-4415150302 · Cited by 46
When Should We Use Real Names in Published Accounts of Internet Research?
Bruckman, A. S., Luther, K., & Fiesler, C. L. (2015) · The MIT Press eBooks · doi:10.7551/mitpress/9386.003.0013 · Cited by 45
Self-citationWatch Me Play: Twitch and the Rise of Game Live Streaming
Poirier-Poulin, S. (2020) · New Media & Society · doi:10.1177/1461444820939317 · Cited by 44
Outnumbered but Well-Spoken: Female Commenters in the New York Times
Pierson, E. (2015) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2675133.2675134 · Cited by 36
The Magic Sauce: Practices of Facilitation in Online Policy Deliberation
Epstein, D., & Leshed, G. (2016) · Journal of Deliberative Democracy · doi:10.16997/jdd.244 · Cited by 36
(2020) · University of Westminster Press eBooks · doi:10.16997/book45.g · Cited by 36
When all you have is a banhammer : the social and communicative work of Volunteer moderators
Lo, C. (2018) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 35
On the Concept of Medium: An Empirical Study
Miconi, A., & Serra, M. (2019) · International journal of communication · Cited by 35
Learning to Airbnb by Engaging in Online Communities of Practice
Holikatti, M., Jhaver, S., & Kumar, N. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359330 · Cited by 35
Self-citationAutomatic Detection of Hateful Comments in Online Discussion
Hammer, H. L. (2017) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-319-52569-3_15 · Cited by 30
Designing for Civil Conversations: Lessons Learned from ChangeMyView
Jhaver, S., Vora, P., & Bruckman, A. S. (2017) · SMARTech Repository (Georgia Institute of Technology) · Cited by 29
Self-citationLiu, P., Guberman, J., Hemphill, L., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.06759 · Cited by 24
The Turn to Artificial Intelligence in Governing Communication Online
Gollatz, K., Beer, F., & Katzenbach, C. (2018) · doi:10.31235/osf.io/vwpcz · Cited by 23
Jhaver, S., Chan, L., & Bruckman, A. (2018) · First Monday · doi:10.5210/fm.v23i2.8232 · Cited by 23
Self-citationWhen is Deliberation Democratic?
Moscrop, D., & Warren, M. E. (2016) · Journal of Deliberative Democracy · doi:10.16997/jdd.256 · Cited by 22
Controlling Bad Behavior in Online Communities: An Examination of Moderation Work
McGillicuddy, A., Bernard, J.-G., & Cranefield, J. (2020) · Open Access Victoria University of Wellington | Te Herenga Waka (Figshare) · doi:10.26686/wgtn.12910085.v1 · Cited by 21
Automatic Moderation of Comments in a Large On-line Journalistic Environment
Veloso, A., Meira, W., Macambira, T. A., Guedes, D. O., & de Almeida, H. M. P. (2007) · International Conference on Weblogs and Social Media · Cited by 17
Measuring Professional Skill Development in U.S. Cities Using Internet Search Queries
Jhaver, S., Cranshaw, J., & Counts, S. (2019) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v13i01.3228 · Cited by 12
Self-citationSocial Science Open Access Repository
Suber, P. (2006) · doi:10.63485/fnmvp-zbm85 · Cited by 11
A strategy for automatic moderation of a large data set of users comments
Saúde, M. R., Soares, M. D. M., Basoni, H. G., Ciarelli, P. M., & de Oliveira, E. S. (2014) · Latin American Computer Conference (CLEI) · doi:10.1109/clei.2014.6965181 · Cited by 5
Research works printed but not linked: 6
- Kathy Charmaz . 2006. Coding in grounded theory practice. Constructing Grounded Theory ( 2006 ), 42--70. Sage Publications Ltd , Thousand Oaks, CA . Kathy Charmaz. 2006. Coding in grounded theory practice. Constructing Grounded Theory (2006), 42--70. Sage Publications Ltd, Thousand Oaks, CA.
- Susan C. Herring . 2000 . Gender differences in CMC: Findings and implications . Computer Professionals for Social Responsibility Journal 18 , 1 (2000), 1 -- 9 . Susan C. Herring. 2000. Gender differences in CMC: Findings and implications. Computer Professionals for Social Responsibility Journal 18, 1 (2000), 1--9.
- Ling Jiang and Eui-Hong Han . 2019 . ModBot: Automatic comments moderation . In Proceedings of the Computation+ Journalism Symposium. Ling Jiang and Eui-Hong Han. 2019. ModBot: Automatic comments moderation. In Proceedings of the Computation+ Journalism Symposium.
- 10.5555/223826.223831 · doi:10.5555/223826.223831
- Nathan J. Matias . 2016 . The civic labor of online moderators . In Internet Politics and Policy conference . Oxford, United Kingdom. Nathan J. Matias. 2016. The civic labor of online moderators. In Internet Politics and Policy conference. Oxford, United Kingdom.
- M. Pilar Opazo. 2010. Revitalizing the concept of sociotechnical systems in social studies of technology. (2010). Available at https://www.semanticscholar.org/paper/Revitalizing-the-Concept-of-Sociotechnical-Systems-Opazo/78460b4bdcbe004ccd469f4bda1f7d98017954de. M. Pilar Opazo. 2010. Revitalizing the concept of sociotechnical systems in social studies of technology. (2010). Available at https://www.semanticscholar.org/paper/Revitalizing-the-Concept-of-Sociotechnical-Systems-Opazo/78460b4bdcbe004ccd469f4bda1f7d98017954de.
Cited by
Studying Reddit: A Systematic Overview of Disciplines, Approaches, Methods, and Ethics
Proferes, N., Jones, N., Gilbert, S., Fiesler, C., & Zimmer, M. (2021) · Social Media + Society · doi:10.1177/20563051211019004 · Cited by 537
“For these reasons, Reddit has an ecosystem of bots created by its user base to help in several ways, such as content moderation (Jhaver et al., 2019), adding functionality through summarizing information and linking to other websites, or providing humor through parody bot accounts (Long et al.,…”
BackgroundGenerative Adversarial Networks for face generation: A survey
Kammoun, A., Slama, R., Tabia, H., Ouni, T., & Abid, M. (2022) · ACM Computing Surveys · doi:10.1145/1122445.1122456 · Cited by 311
Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction
Kim, S. S. Y., Watkins, E. A., Russakovsky, O., Fong, R., & Monroy-Hernández, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581001 · Cited by 233
Seering, J. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415178 · Cited by 147
Steiger, M., Bharucha, T. J., Venkatagiri, S., Riedl, M. J., & Lease, M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445092 · Cited by 222
Metadata
- Authors
- Jhaver, S., Birman, I., Gilbert, E., & Bruckman, A.
- Affiliations
- Georgia Institute of Technology (US)
- University of Michigan, Ann Arbor, MI
- Venue
- ACM Transactions on Computer-Human Interaction
- Published
- 2019-07-19
- Volume
- 26
- Issue
- 5
- Pages
- 1-35
- ISSN
- 1073-0516 (ISSN-L)
- 1557-7325
- OpenAlex
- W2964034671
- Cited by
- 289
- References
- 92
- Citations by year
- 2026: 36 · 2025: 56 · 2024: 42 · 2023: 41 · 2022: 43 · 2021: 44 · 2020: 22 · 2019: 5
- FWCI
- 49.82
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- ACM Transactions on Computer-Human Interaction (publishedVersion, other-oa)
- Funders
- National Science Foundation (IIS-1553376)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- human-machine collaboration, content moderation, automated content moderation, Reddit, AutoMod, social media governance, platform governance, socio-technical systems, audit tool, division of labor, human moderators, tool sharing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.