Contestability For Content Moderation
Open accessAbstract
Content moderation systems for social media have had numerous issues of bias, in terms of race, gender, and ability among many others. One proposal for addressing such issues in automated decision making is by designing for contestability, whereby users can shape and influence how decisions are made. In this study, we conduct a series of participatory design workshops with participants from communities that have experienced problems with social media content moderation in the past. Together with participants, we explore the idea of designing for contestability in content moderation and find that users' designs suggest three fruitful, practical avenues: adding representation, improving communication, and designing with compassion. We conclude with design recommendations drawn from participants' proposals, and reflect on the challenges that remain.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Participants considered that humans would treat decision subjects with dignity and compassion.
“and that humans would treat decision subjects with dignity and compassion (e.g., Q.31; also in line with previous research [17,68,94]).”
From Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability (2023) · cites this paper as [94]
In “Forms of Compassion” (page unavailable), participants called for moderators with more emotional intelligence who would listen to users and acknowledge their harms. This supports the compassion aspect, but the paper presents it as a proposed change, not a belief that humans would treat decision subjects with dignity and compassion. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
(1998) · Cited by 13,501 · see citing works
Qualitative research practice: a guide for social science students and researchers
Ruhl, K. (2003) · Choice Reviews Online · doi:10.5860/choice.41-1319 · Cited by 10,514 · see citing works
(2006) · Springer eBooks · doi:10.1007/1-84628-301-9_1 · Cited by 2,194 · see citing works
Democracy, Accountability, and Representation
Przeworski, A. (1999) · Cambridge University Press eBooks · doi:10.1017/cbo9781139175104 · Cited by 1,810 · see citing works
Rule-Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project
(1986) · Telematics and Informatics · doi:10.1016/s0736-5853(86)80073-9 · Cited by 1,701 · see citing works
Çömlekçi, M. F. (2019) · Kırklareli University Institutional Repository (Kırklareli University) · Cited by 1,607 · see citing works
Routledge International Handbook of Participatory Design
(2012) · doi:10.4324/9780203108543 · Cited by 1,606 · see citing works
Rule-based expert systems: The mycin experiments of the stanford heuristic programming project
Swartout, W. (1985) · Artificial Intelligence · doi:10.1016/0004-3702(85)90067-0 · Cited by 1,248 · see citing works
Bardzell, S. (2010) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753521 · Cited by 1,139 · see citing works
Shifman, L. (2013) · The MIT Press eBooks · doi:10.7551/mitpress/9429.001.0001 · Cited by 1,094 · see citing works
Proceedings of the ACM on Human-Computer Interaction
Karahalios, K. G., Monroy‐Hernández, A., Lampinen, A., & Fitzpatrick, G. (2018) · Cited by 1,070 · see citing works
Self-citationQuestioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925 · see citing works
Harrington, C., Erete, S., & Piper, A. M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359318 · Cited by 612 · see citing works
The New Governors: The People, Rules, and Processes Governing Online Speech
Klonick, K. (2018) · St. John's University School of Law (St. John's University) · Cited by 582 · see citing works
Algorithmic Transparency in the News Media
Diakopoulos, N., & Koliska, M. (2017) · Digital Journalism · doi:10.1080/21670811.2016.1208053 · Cited by 515 · see citing works
Gillespie, T. (2018) · Cited by 506 · see citing works
What is a flag for? Social media reporting tools and the vocabulary of complaint
Crawford, K., & Gillespie, T. (2016) · New Media & Society · doi:10.1177/1461444814543163 · Cited by 496 · see citing works
Wright, P., & McCarthy, J. (2008) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357156 · Cited by 473 · see citing works
West, S. M. (2018) · New Media & Society · doi:10.1177/1461444818773059 · Cited by 445 · see citing works
Explanations as Mechanisms for Supporting Algorithmic Transparency
Rader, E., Cotter, K., & Cho, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173677 · Cited by 388 · see citing works
Moderator engagement and community development in the age of algorithms
Seering, J., Wang, T., Yoon, J., & Kaufman, G. (2019) · New Media & Society · doi:10.1177/1461444818821316 · Cited by 351 · see citing works
Out in the Country: Youth, Media, and Queer Visibility in Rural America
Gray, M. L. (2009) · Project Muse (Johns Hopkins University) · doi:10.18574/9780814733103 · Cited by 345 · see citing works
Matamoros-Fernández, A. (2017) · Information Communication & Society · doi:10.1080/1369118x.2017.1293130 · Cited by 320 · see citing works
DeVito, M. A., Gergle, D., & Birnholtz, J. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025659 · Cited by 311 · see citing works
Out of Scandinavia: Alternative Approaches to Software Design and System Development
Floyd, C., Mehl, W.-M., Resin, F.-M., Schmidt, G., & Wolf, G. (1989) · Human-Computer Interaction · doi:10.1207/s15327051hci0404_1 · Cited by 248 · see citing works
Procedural Justice in Algorithmic Fairness
Lee, M. K., Jain, A., Jin, H., Ojha, S., & Kusbit, D. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359284 · Cited by 236 · see citing works
Lee, M. K., Kusbit, D., Kahng, A., Kim, J. T., Yuan, X., Chan, A., See, D., Noothigattu, R., Lee, S., Psomas, A., et al. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359283 · Cited by 217 · see citing works
Tumblr was a trans technology: the meaning, importance, history, and future of trans technologies
Haimson, O. L., Dame-Griff, A., Capello, E., & Richter, Z. (2019) · Feminist Media Studies · doi:10.1080/14680777.2019.1678505 · Cited by 214 · see citing works
Does Transparency in Moderation Really Matter?
Jhaver, S., Bruckman, A. S., & Gilbert, É. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359252 · Cited by 211 · see citing works
Brown, A., Chouldechova, A., Putnam-Hornstein, E., Tobin, A., & Vaithianathan, R. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300271 · Cited by 207 · see citing works
Lee, M. K., & Baykal, S. (2017) · Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/2998181.2998230 · Cited by 182 · see citing works
DiSalvo, C. F., Lukens, J., Lodato, T. J., Jenkins, T., & Kim, T. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557359 · Cited by 169 · see citing works
Haimson, O. L., & Hoffmann, A. L. (2016) · First Monday · doi:10.5210/fm.v21i6.6791 · Cited by 169 · see citing works
Commercial Content Moderation: Digital Laborers' Dirty Work
Roberts, S. (2016) · Scholarship@Western (Western University) · Cited by 160 · see citing works
Toward situated interventions for algorithmic equity: lessons from the field
Katell, M., Young, M., Dailey, D., Herman, B., Guetler, V., Tam, A., Bintz, C., Raz, D., & Krafft, P. M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372874 · Cited by 153 · see citing works
Crossmod: A Cross-Community Learning-based System to Assist Reddit Moderators
Chandrasekharan, E., Gandhi, C., Mustelier, M. W., & Gilbert, E. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359276 · Cited by 151 · cited 1× · see citing works
“For example, rather than have a single policy, platforms could infer “local” communities and their norms, similar to community-based approaches that have been successful in the past [18].”
BackgroundDesigning Trans Technology: Defining Challenges and Envisioning Community-Centered Solutions
Haimson, O. L., Gorrell, D., Starks, D. L., & Weinger, Z. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376669 · Cited by 136 · see citing works
Digital detritus: 'Error' and the logic of opacity in social media content moderation
Roberts, S. (2018) · First Monday · doi:10.5210/fm.v23i3.8283 · Cited by 127 · see citing works
Participatory Design and Design for Values
van der Velden, M., & Mörtberg, C. (2015) · Springer eBooks · doi:10.1007/978-94-007-6970-0_33 · Cited by 126 · see citing works
Computer-Assisted Clinical Decision-Making
GOBRY, G. A. (1973) · Methods of Information in Medicine · doi:10.1055/s-0038-1636093 · Cited by 121 · see citing works
To empathise or not to empathise? Empathy and its limits in design
Heylighen, A., & Dong, A. (2019) · Design Studies · doi:10.1016/j.destud.2019.10.007 · Cited by 121 · see citing works
Eliciting Values Reflections by Engaging Privacy Futures Using Design Workbooks
Wong, R. Y., Mulligan, D. K., Wyk, E. V., Pierce, J., & Chuang, J. C.-I. (2017) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3134746 · Cited by 117 · see citing works
Keyes, O., Hoy, J., & Drouhard, M. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300569 · Cited by 111 · see citing works
Designing Contestability: Interaction Design, Machine Learning, and Mental Health
Hirsch, T., Merced, K., Narayanan, S., Imel, Z. E., & Atkins, D. C. (2017) · Proceedings of the 2017 Conference on Designing Interactive Systems · doi:10.1145/3064663.3064703 · Cited by 110 · see citing works
Regulating Online Content Moderation
Langvardt, K. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3024739 · Cited by 103 · see citing works
Vaccaro, K., Sandvig, C., & Karahalios, K. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415238 · Cited by 102 · see citing works
Self-citationExploring Social Justice, Design, and HCI
Fox, S., Asad, M., Lo, K., Dimond, J. P., Dombrowski, L. S., & Bardzell, S. (2016) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2851581.2856465 · Cited by 95 · see citing works
Human intervention in automated decision-making: Toward the construction of contestable systems
Almada, M. (2019) · Proceedings of the Seventeenth International Conference on Artificial Intelligence and Law · doi:10.1145/3322640.3326699 · Cited by 95 · see citing works
Digital Juries: A Civics-Oriented Approach to Platform Governance
Fan, J., & Zhang, A. X. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376293 · Cited by 90 · see citing works
Perspective and Use of Empathy in Design Thinking
Gasparini, A. A. (2015) · Advances in Computer-Human Interaction · Cited by 88 · see citing works
Conformity of Eating Disorders through Content Moderation
Feuston, J. L., Taylor, A. S., & Piper, A. M. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392845 · Cited by 81 · see citing works
Understanding Older Adults' Participation in Design Workshops
Pradhan, A., Jelen, B., Siek, K. A., Chan, J., & Lazar, A. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376299 · Cited by 75 · see citing works
When Users Control the Algorithms
Burrell, J., Kahn, Z., Jonas, A., & Griffin, D. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359240 · Cited by 69 · see citing works
Bratteteig, T., & Verne, G. (2018) · Participatory Design Conference · doi:10.1145/3210604.3210646 · Cited by 66 · see citing works
Towards ethical principles for participatory design practice
Kelly, J. (2018) · CoDesign · doi:10.1080/15710882.2018.1502324 · Cited by 58 · see citing works
Participatory Design of VR Scenarios for Exposure Therapy
Flobak, E., Wake, J. D., Vindenes, J., Kahlon, S., Nordgreen, T., & Guribye, F. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300799 · Cited by 53 · see citing works
What is Mixed-Initiative Interaction?
Novick, D., & Sutton, S. (1997) · Cited by 51 · see citing works
Gatehouse, C., Wood, M., Briggs, J., Pickles, J., & Lawson, S. W. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173683 · Cited by 45 · see citing works
Participatory Design and the Future of Rural LGBTQ Communities
Hardÿ, J., & Vargas, S. (2019) · doi:10.1145/3301019.3323894 · Cited by 42 · see citing works
Automated Decision Support Technologies and the Legal ProfessionPreprint
Kluttz, D. N., & Mulligan, D. K. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3443063 · Cited by 40 · see citing works
Listen to Developers! A Participatory Design Study on Security Warnings for Cryptographic APIs
Gorski, P. L., Acar, Y., Iacono, L. L., & Fahl, S. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376142 · Cited by 40 · see citing works
Reducing Uncertainty and Offering Comfort: Designing Technology for Coping with Interpersonal Racism
To, A., Carey, H., Kaufman, G. F., & Hammer, J. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445590 · Cited by 38 · see citing works
Out in the Country: Youth, Media, and Queer Visibility in Rural America
Brekhus, W. H. (2011) · Contemporary Sociology A Journal of Reviews · doi:10.1177/0094306110391764q · Cited by 26 · see citing works
Do Search Engine Helpline Notices Aid in Preventing Suicide? Analysis of Archival Data
Cheng, Q., & Yom‐Tov, E. (2019) · Journal of Medical Internet Research · doi:10.2196/12235 · Cited by 20 · see citing works
The intersectional internet: race, sex, class, and culture online
Humann, H. D. (2018) · Feminist Media Studies · doi:10.1080/14680777.2018.1478704 · Cited by 15 · see citing works
Designing to engage children in monitoring indoor air quality
Kim, S., Li, M., Senick, J., & Mainelis, G. (2020) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3392063.3394395 · Cited by 15 · see citing works
Procházka, O. (2019) · Signs and Society · doi:10.1086/704763 · Cited by 9 · see citing works
Play It My Way: Participatory Mobile Game Design with Children in Rural Nepal
Lamichhane, D. R., & Read, J. C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-49788-0_24 · Cited by 4 · see citing works
Ahmad, W., & Liccardi, I. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376690 · Cited by 3 · see citing works
Automated Decision Support Technologies and the Legal Profession
Kluttz, D. N. ;. M. (2019) · UC Berkeley · doi:10.15779/z38154dp7k · Cited by 2 · see citing works
Boufoy-Bastick, Z.-A., & Singh, L. (2007) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1075774 · Cited by 1 · see citing works
Flagged first, then by citations
2 of the 74 references couldn’t be matched to a known paper.
Research works printed but not linked: 7
- Jaz Hee-jeong Choi, Laura Forlano, and Denisa Kera. 2020. Situated Automation: Algorithmic Creatures in Participatory Design. In Proc. Participatory Design Conference.
- J Matias, Amy Johnson, Whitney Erin Boesel, Brian Keegan, Jaclyn Friedman, and Charlie DeTar. 2015. Reporting, reviewing, and responding to harassment on Twitter. Available at SSRN 2602018 (2015).
- Michael J Muller and Allison Druin. 2012. Participatory Design: The Third Space In Human-Computer Interaction. In Human Computer Interaction Handbook. CRC Press, 1125-1153.
- Deirdre K Mulligan, Daniel Kluttz, and Nitin Kohli. 2019. Shaping Our Tools: Contestability as a Means to Promote Responsible Algorithmic Decision Making in the Professions. Available at SSRN 3311894 (2019).
- Frank A Pasquale. 2011. Restoring transparency to automated authority. Journal on Telecommunications and High Technology Law, (2011).
- Kristen Vaccaro, Karrie Karahalios, Deirdre Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in Algorith- mic Decision Making. In Proc. CSCW Workshops. ACM.
- Allison Woodruff. 2019. 10 things you should know about algorithmic fairness. interactions 26, 4 (2019), 47-51.
Cited by
Sorted by citations
Contestable AI by Design: Towards a Framework
Alfrink, K., Keller, I., Kortuem, G., & Doorn, N. (2022) · Minds and Machines · doi:10.1007/s11023-022-09611-z · Cited by 104 · see citing works
“A small but growing body of research explores the concept of contestable AI (Almada, 2019; Henin & Le Métayer, 2021; Hirsch et al., 2017; Lyons et al., 2021; Sarra, 2020; Vaccaro et al., 2019, 2021).”
InfluentialJhaver, S., Zhang, A. Q., Chen, Q. Z., Natarajan, N., Wang, R., & Zhang, A. X. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610080 · Cited by 97 · see citing works
“Prior research on related moderation mechanisms such as post-removals, flagging systems, and appeal procedures against moderation decisions have also called for platforms to be more accountable [17, 63, 106, 107, 114].”
BackgroundYurrita, M., Draws, T., Balayn, A., Murray-Rust, D., Tintarev, N., & Bozzon, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581161 · Cited by 73 · see citing works
“[94], can help deal with the trade-ofs identifed in our qualitative analysis.”
BackgroundResultDesigning Word Filter Tools for Creator-led Comment Moderation
Jhaver, S., Chen, Q. Z., Knauss, D., & Zhang, A. X. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517505 · Cited by 72 · see citing works
“Prior research has shown that BIPOC, LGBTQ+, and female users have experienced higher instances of online harassment and harms of content moderation [40, 117].”
MethodPan, C. A., Yakhmi, S., Iyer, T., Strasnick, E., Zhang, A. X., & Bernstein, M. S. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512929 · Cited by 67 · see citing works
“Both processes are well examined in the literature [44, 115, 128], are used in some form in industry [7, 43, 85], and were proposed specifically to help address the legitimacy issues plaguing earlier methods [25, 115, 129, 141].”
BackgroundShahid, F., & Vashistha, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581538 · Cited by 68 · see citing works
“The platforms should use community-engaged approaches rather than putting the burden of representation on a single individual [148].”
BackgroundCruising Queer HCI on the DL: A Literature Review of LGBTQ+ People in HCI
Taylor, J., Simpson, E., Tran, A.-T., Brubaker, J. R., Fox, S., & Zhu, H. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642494 · Cited by 58 · see citing works
“Similarly, some research in our corpus centered on queer people as a research focus (e.g., [34, 45, 97, 213]), while others used queer experiences [210] or queer rights [229] as an application area to, for instance, understand content moderation [210] or predict policy changes [229].”
BackgroundAlfrink, K., Keller, I., Doorn, N., & Kortuem, G. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580984 · Cited by 48 · see citing works
“Contexts that do feature in works discussed are healthcare [39, 64], smart cities [47], and content moderation [27, 75, 76].”
BackgroundHow Do Users Experience Moderation?: A Systematic Literature Review
Ma, R., You, Y., Gui, X., & Kou, Y. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610069 · Cited by 50 · see citing works
“Both Fan et al. and Vaccaro et al. have tried to involve the ethical value, fairness, in moderation decision-making processes [29] and in contesting moderation decisions [109].”
BackgroundInfluentialChallenges of Moderating Social Virtual Reality
Sabri, N., Chen, B., Teoh, A., Dow, S. P., Vaccaro, K., & ElSherief, M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581329 · Cited by 43 · see citing works
“Moreover, the research community has historically critiqued the fact that moderation only included punitive responses [103, 143].”
BackgroundSelf-citation"I'm not sure what difference is between their content and mine, other than the person itself"
Ma, R., & Kou, Y. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555150 · Cited by 45 · see citing works
“Prior work has stressed that platforms need to have users’ voice/representation in moderation decision-making for designing contestability for moderation [93] or embedding procedural justice in it [29].”
BackgroundChoi, Y., Kang, E. J., Lee, M. K., & Kim, J. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581386 · Cited by 38 · see citing works
“A large body of research has been done to explore how the participatory design method can be applied to improve user experiences in algorithmic systems of various context [2, 19, 47, 68].”
BackgroundMethodWhat’s the Appeal? Perceptions of Review Processes for Algorithmic Decisions
Lyons, H., Wijenayake, S., Miller, T., & Velloso, E. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517606 · Cited by 27 · see citing works
“[81] ran a series of workshops to explore participants’ views on designing for contestability in relation to automated content moderation, uncovering three key themes: (1) the need for moderation to be representative of users; (2) the need for strong two-way communication between the users and the platform; and (3) th…”
BackgroundTransparency, Fairness, and Coping: How Players Experience Moderation in Multiplayer Online Games
Ma, R., Li, Y., & Kou, Y. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581097 · Cited by 26 · see citing works
“, [44, 69, 99]), raise a critical question regarding general moderation research and practice – how moderation design could conceive”
BackgroundAnalyzing the Use of Large Language Models for Content Moderation with ChatGPT Examples
Franco, M., Gaggi, O., & Palazzi, C. E. (2023) · doi:10.1145/3599696.3612895 · Cited by 29 · see citing works
“[27] investigated how users can influence the automated decisions of content moderation systems (i.”
BackgroundContent moderation by LLM: from accuracy to legitimacy
Huang, T. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11328-1 · Cited by 24 · see citing works
“Instead of passively waiting for the platform’s reply, users should be engaged with a participatory design that encourages real-time communications ([ nr_Vaccaroetal2021 ]).”
Integrating Content Moderation Systems with Large Language Models
Franco, M., Gaggi, O., & Palazzi, C. E. (2024) · ACM Transactions on the Web · doi:10.1145/3700789 · Cited by 21 · see citing works
“The difficulty in grasping community standards and the technical legal language within guidelines and terms of service exacerbates these communication barriers [53].”
InfluentialBystanders of Online Moderation: Examining the Effects of Witnessing Post-Removal Explanations
Jhaver, S., Rathi, H., & Saha, K. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642204 · Cited by 19 · see citing works
“The HCI community has especially centered around aspirational computer science principles of fairness, accountability, transparency, ethics, and responsibility [63].”
BackgroundMethodMayworm, S., Albert, K., & Haimson, O. L. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658907 · Cited by 19 · see citing works
Xue, X., Li, X., Jia, B., Du, J., & Fu, X. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642866 · Cited by 19 · see citing works
Warner, M., Strohmayer, A., Higgs, M., & Coventry, L. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103468 · Cited by 22 · see citing works
“Whilst proactive moderation helps to address issues of representation and contestability [88] by making moderation more transparent by being embedded within the user interaction, participants rarely considered this, and instead focused on the injustice felt if the system was to prompt them…”
Exposed or Erased: Algorithmic Censorship of Nudity in Art
Riccio, P., Hofmann, T., & Oliver, N. (2024) · International Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642586 · Cited by 24 · see citing works
“In the case of content moderation, scholars have reported biases in the treatment given to individuals because of their race and political orientation [71], physical and/or mental abilities [144], gender and sexual identities [13], and different body shapes [60].”
BackgroundKou, Y., Ma, R., Zhang, Z., Zhou, Y., & Gui, X. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642675 · Cited by 16 · see citing works
On Algorithmic Content Moderation
Prem, E., & Krenn, B. (2023) · Springer eBooks · doi:10.1007/978-3-031-45304-5_30 · Cited by 13 · see citing works
Meta’s AI moderation and free speech: Ongoing challenges in the Global South
Balendra, S. (2025) · Cambridge Forum on AI Law and Governance · doi:10.1017/cfl.2025.5 · Cited by 12 · see citing works
Baumer, E. P. S., Taylor, A. S., Brubaker, J. R., & McGee, M. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3660344 · Cited by 16 · see citing works
“Moderation is a key contributor to the success of online communities [109] and has garnered significant research attention [e.g., 45, 46, 64, 73, 81, 88, 101, 102, 117, 129, 181, 182].”
Ma, R., & Kou, Y. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579477 · Cited by 13 · see citing works
“, [41,75]), we argue that communication should be bi-directional, as we discussed in section 6.”
BackgroundCombating Islamophobia: Compromise, Community, and Harmony in Mitigating Harmful Online Content
Rifat, M. R., Asha, A. Z., Jadon, S. S., Yan, X., Guha, S., & Ahmed, S. I. (2024) · ACM Transactions on Social Computing · doi:10.1145/3641510 · Cited by 11 · see citing works
“In this context, we discuss HCI research on content moderation to examine how the inclusion of faith may adapt and complement existing moderation tools and policies to better combat Islamophobia.”
Calvert, S. C. (2025) · Science and Engineering Ethics · doi:10.1007/s11948-025-00554-z · Cited by 12 · see citing works
Ma, R., Zhang, Z., Gui, X., & Kou, Y. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661565 · Cited by 10 · see citing works
Melder, E., Lerner, A., & DeVito, M. A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710919 · Cited by 10 · see citing works
“However, being on a blocklist is not only problematic because of the impact on federation, but also because of the perceived moral judgment inherent to moderation [88].”
Pyle, C., Zhang, Z., Haimson, O. L., & Andalibi, N. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637342 · Cited by 9 · see citing works
“Moreover, as Vaccaro and colleagues [126] have pointed out, work on contestability has mostly focused on expert users.”
BackgroundSarra, C. (2025) · Athens Journal of Law · doi:10.30958/ajl.11-1-3 · Cited by 8 · see citing works
AppealMod: Inducing Friction to Reduce Moderator Workload of Handling User Appeals
Atreja, S., Im, J., Resnick, P., & Hemphill, L. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637296 · Cited by 11 · see citing works
“Prior work also found that users considered communication, especially direct communication with human moderators, as one of the top three avenues for improving their abilities to contest content moderation decisions [56].”
BackgroundInfluentialPractising Appropriate Trust in Human-Centred AI Design
Degachi, C., Mehrotra, S., Yurrita, M., Niforatos, E., & Tielman, M. L. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650825 · Cited by 10 · see citing works
Jonas, R. M., De, A., & Cotter, K. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642599 · Cited by 8 · see citing works
“Miro (www.miro.com) has been used extensively as a virtual collaborative platform since it provides efcient functionalities for virtual collaboration [ 12, 51, 90, 101, 102].”
BackgroundMethodInfluentialPeer Produced Friction: How Page Protection on Wikipedia Affects Editor Engagement and Concentration
Ajmani, L., Vincent, N., & Chancellor, S. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610198 · Cited by 8 · see citing works
“[97] discusses the importance of contestability in content moderation for more participatory communities.”
Liao, J., Ye, Y., Filieri, R., Du, P., & Jiang, Y. (2024) · Psychology and Marketing · doi:10.1002/mar.22065 · Cited by 7 · see citing works
Magdy, W., Mubarak, H., & Salminen, J. O. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713150 · Cited by 6 · see citing works
Using Affordance to Understand Usability of Web3 Social Media
Guan, M. Y., Yu, Y., & Wang, K. Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713263 · Cited by 6 · see citing works
“Studies have identified that creators faced challenges in content censorship: (1) opacity of review decisions [35, 41], (2) unfairness of review decisions [61, 113, 114], and (3) procedural barriers to reviewing appeals of decisions [5, 63], among others.”
Investigating Influential Users' Responses to Permanent Suspension on Social Media
Gao, Z., & Thebault-Spieker, J. (2024) · Proc. ACM Hum. Comput. Interact. · doi:10.1145/3637356 · Cited by 7 · see citing works
“The application of a universal content moderation strategy across diverse cultures and regions is inherently fraught with challenges, many of which arise from a lack of sensitivity to local and cultural contexts [46, 113, 126].”
BackgroundAghajari, Z., Baumer, E. P. S., Lazard, A. J., Dasgupta, N., & DiFranzo, D. J. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641939 · Cited by 6 · see citing works
“Of course, similar to content moderation, these moderation practices can be efective only in communities wherein moderators hold good values and good faith [62, 109].”
BackgroundThe Role of Expertise in Effectively Moderating Harmful Social Media Content
Abdelkadir, N. A., Yang, T., Kapania, S., Estefanos, M., Gebrekidan, F. B., Zelalem, Z., Ali, M., Berhe, R., Baker, D., Talat, Z., et al. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714010 · Cited by 5 · see citing works
Understanding Decentralized Social Feed Curation on Mastodon
Liu, Y., Song, E., Zhang, O. X., Merriman, J., Zhang, L., & Monroy‐Hernández, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757688 · Cited by 5 · see citing works
“In the past decade, HCI and CSCW researchers have investigated social feed curation in mainstream centralized social media platforms from multiple perspectives such as content moderation [31, 75], algorithm transparency [23, 24], and interface design [22, 29, 39].”
Ignoring As a Moderation Strategy for Volunteer Moderators on Twitch
Li, N., Cai, J., & Wohn, D. Y. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3585704 · Cited by 5 · see citing works
“In contrast, reactive strategies are employed after users behave negatively, such as content removal [18], ban and block [20], contestability [36].”
BackgroundZhao, A., & Hobbs, W. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713154 · Cited by 4 · see citing works
The Virtual Jail: Content Moderation Challenges Faced by Chinese Queer Content Creators on Douyin
Shen, C., & Haimson, O. L. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714013 · Cited by 4 · see citing works
Content Moderation of Generative AI Prompts
Pardhi, P. R. (2025) · SN Computer Science · doi:10.1007/s42979-025-03864-y · Cited by 4 · see citing works
“NSFW content generally encompasses explicit visuals, adult material, or any content that could be offensive [31] or unsuitable for a broad audience.”
Identifying Algorithmic Decision Subjects' Needs for Meaningful Contestability
Yurrita, M., Verma, H., Balayn, A., Alfrink, K., Gadiraju, U., & Bozzon, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757415 · Cited by 7 · see citing works
“One of the very few empirical studies on decision subjects’ needs for contestability was grounded in a context other than the public sector (i.e., content moderation [101]).”
InfluentialJo, J., Xie, J., & Carroll, J. M. (2023) · International Conference on Information Technology for Social Good (GoodIT) · doi:10.1145/3582515.3609541 · Cited by 3 · see citing works
“The agency of SNAP beneficiaries could be augmented by incorporating design elements for contestability into SNAP apps that facilitate compassionate two-way communication between caseworkers and beneficiaries, provide understandable and knowledgeable explanations about inputs, outputs, and the functioning of SNAP deci…”
BackgroundFrom Stem to Stern: Contestability Along AI Value Chains
Balayn, A., Pi, Y., Widder, D. G., Alfrink, K., Yurrita, M., Upadhyay, S., Karusala, N., Lyons, H., Turkay, Ç., Tessono, C., et al. (2024) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3678884.3681831 · Cited by 3 · see citing works
“…have made significant progress in directing the conversation towards making AI more responsive and accountable through ongoing learning based on feedback and contestation [16], their main focus remains on contesting AI design or outputs [4] within specific domains such as content moderation [19].”
Ferraz, T. P., Duarte, C. H. D., Ribeiro, M. F., Takayanagi, G. G. B., Alcoforado, A., Lopes, R. D. D., & Susi, M. (2024) · Estudos Avançados · doi:10.1590/s0103-4014.202438111.020 · Cited by 3 · see citing works
“Why can’t I just be fat and exist on the internet?”: The embeddedness of antifatness on TikTok
Cotter, K., Jonas, R. M., & De, A. (2025) · Platforms & Society · doi:10.1177/29768624251377163 · Cited by 3 · see citing works
“Alternate approaches like subsidiarity (Hasinoff and Schneider, 2022), algorithmic reparations (Peterson-Salahuddin, 2024), and representative moderation (Vaccaro et al., 2021) that shift in con-trol from platforms to communities offer a promising path forward.”
Diversity as Ethical Infrastructure: Reimagining AI Governance for Justice and Accountability
Iseko, A. (2025) · International Journal of Science Technology and Society · doi:10.11648/j.ijsts.20251305.13 · Cited by 3 · see citing works
Designing For Microaggressions
Sun, L., Chen, Y., Li, B., Li, R., & Vaccaro, K. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757404 · Cited by 2 · see citing works
“The participatory design or co-design approach is widely used in HCI research [8, 58, 64, 65, 105, 106].”
Self-citationWilner, T., Moran, R. E., Mimizuka, K., Engle, V. F., Young, J. C., Smith, A. D. R., & Arif, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757422 · Cited by 2 · see citing works
“More directly related to this paper, researchers have used PD to build misinformation interventions, such as content moderation systems [89], educational games about “fake news” [51], tools for identifying fake visual information [79], and a participatory media literacy website for teachers [2].”
Hirsbrunner, S. D., Kleemann, S., & Tahraoui, M. (2025) · Frontiers in Communication · doi:10.3389/fcomm.2025.1638257 · Cited by 2 · see citing works
Wang, K., Clarke, L., Dreue, C.-C. J., Zhou, G., & Jacques, J. T. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3799442 · Cited by 3 · see citing works
“Platforms and community moderators must navigate persistent tensions between fostering open participation and preventing abuse, recognising that cooperative online spaces inherently entail some content restrictions [46, 115].”
Conceptualizing and Rethinking the Design of Cross-platform Creator Moderation
Ma, R. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3577049 · Cited by 2 · see citing works
“For example, researchers have uncovered various challenges users encounter when they experience content moderation: (1) opacity of moderation decision-making [15, 18, 35], (2) unfairness of moderation decisions [22, 40, 41], (3) procedural obstacles to appeal moderation decisions [5, 25], and more.”
BackgroundNan, H., Wang, Z., Zhang, Y., Zhang, X., & Zeng, S. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3720230 · Cited by 2 · see citing works
“This collaborative approach is central to HCI research, which emphasizes the need for user-centered, transparent, and adaptable systems [20, 47].”
A systematic review of similar Questions Retrieval Approaches
Khodke, S., & Dhande, S. S. (2023) · ASIAN JOURNAL OF CONVERGENCE IN TECHNOLOGY · doi:10.33130/ajct.2023v09i02.006 · Cited by 1 · see citing works
The Balancing Act of Social Audio Facilitators: When Self-Promotion Overshadows Community Care
Sabri, N., Chen, Y., Lee, M., Dow, S. P., Vaccaro, K., & ElSherief, M. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710981 · Cited by 1 · see citing works
“…of moderation on future behavior [55, 105], the dangers moderation poses to moderators [4, 88], the inequities in moderation decisions particularly when it comes to marginalized communities [26, 27, 44], as well as contestability and appeal strategies and user’s views of them [114, 115].”
Self-citationLloyd, T., Nguyen, T., Levy, K. E. C., & Naaman, M. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791889 · Cited by 3 · see citing works
“CCS can be viewed as highly distributed deliberation systems that aim to allow more informed public participation in social media.”
Attitudes, Imagined Roles, and Governance Boundaries for AI in Decentralized Social Media
Zhang, Z., Zhao, J., Wang, G., Viswanathan, S., Ross, T., Johnston, S., Liu, D., Noh, H., Kleek, M. G. V., & Shadbolt, N. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790295 · Cited by 1 · see citing works
“Empirical studies of moderation ap-peals and disputes on large platforms show that such mechanisms are often experienced as opaque, constrained, or ineffective in practice, reinforcing perceptions of unilateral platform power [101, 102].”
Constructing Gender Antagonism: Moralised Platform Governance in China
Guan, Z., & Shang, Z. (2026) · Media and Communication · doi:10.17645/mac.12064 · Cited by 1 · see citing works
“Moreover, the boundaries of toxic content, including gender‐based harmful content, remain unclear, and are continuously reconfigured through negotiations between users, platforms, and broader society (Trott et al., 2022; Vaccaro et al., 2021).”
Conceptualizing and Improving Creator Moderation Design with Platform Stakeholders
Ma, R. (2023) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3584931.3608927 · Cited by 1 · see citing works
“For example, users are confused about moderation decisionmaking [10], feel unfair about its decisions [14, 28], or have a hard time appealing moderation decisions [16].”
BackgroundValencia, G., Serrano, M. D., & Gálvez‐Muñoz, L. (2025) · Information Communication & Society · doi:10.1080/1369118x.2025.2524576
“Others conduct participatory design studies (Bull et al., 2021; Furlo et al., 2021; Vaccaro et al., 2021; Zytko & Furlo, 2023) with the aim of proposing solutions to specific requirements and challenges encountered by users, as well as improving technology-mediated consent (Furlo et al., 2021; Im…”
Mapping Community Appeals Systems: Lessons for Community-led Moderation in Multi-Level Governance
Lee, J., Doan, B. N. (., Jee, J., & Seering, J. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757627
“An appeals system enforces accountability on authorities by empowering users to contest moderation decisions [97, 98].”
Embracing Training Dataset Bias for Automated Harmful Detection
Schöpke-Gonzalez, A. M., Kim, N., & Hemphill, L. (2025) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.1282 · Cited by 1 · see citing works
“This rec ommendation aligns with Vaccaro et al.’s findings that social media users want more representation in content moderation decision-making (Vaccaro et al., 2021).”
The Content Authorship-Generation Continuum: A Framework for Classifying AI-Mediated Content
Vowles, S., Ruthven, I. G., & Saeghe, P. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798565
Holland, T., Sparrow, L. A., Smith, W., & Webber, S. E. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791146 · Cited by 1 · see citing works
“Previous research has established that opaqueness is a key challenge for moderation [59, 65, 78, 79, 83, 90, 119].”
InfluentialInterpreting algorithmic information cues: User sensemaking of search autocomplete moderation
Jhaver, S. (2026) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.70092
“– P13”
Jordan, R. (2026) · Technical Communication Quarterly · doi:10.1080/10572252.2026.2691023
Schmude, T., Pahr, D., Koesten, L. M., Möller, T. B., & Tschiatschek, S. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812271
Legible Machines, Contestable Truths
Shin, D. (2026) · Oxford University Press eBooks · doi:10.1093/9780197849644.003.0009
Retheorizing Context Collapse for Social Media
Chancellor, S., Kaur, H., Terveen, L., & Milton, A. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3816991
“Simultaneously, designing interface features that let users contest algorithmic decisions based is an open challenge [143].”
Walters, A., & Jhaver, S. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3816999
“…around the unequal application of moderation has been understood is through the concept of procedural justice, which refers to user expectations that moderation processes are transparently and universally applied, and that appeals to moderation decisions are seriously considered [9, 78, 91, 92].”
O'Brien, C. B., Ferreira-Alturas, I., Ayobi, A., & Warner, M. (2026) · Nordic Conference on Human-Computer Interaction · doi:10.1145/3829807.3829889
- Authors
- Vaccaro, K., Xiao, Z., Hamilton, K., & Karahalios, K.
- Affiliations
- University of California San Diego (US)
- University of Illinois Urbana-Champaign (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2021-10-13
- Volume
- 5
- Issue
- CSCW2
- Pages
- 1-28
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W3207635698
- Cited by
- 77
- References
- 74
- Citations by year
- 2026: 13 · 2025: 23 · 2024: 19 · 2023: 17 · 2022: 5
- FWCI
- 12.83
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- Funders
- Capital One Financial Corporation
- National Science Foundation (1564041)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- content moderation, contestability, social media, algorithmic bias, automated decision-making, participatory design, user agency, representation, communication, compassion, design guidelines
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
