CivilServant: Community-Led Experiments in Platform Governance
The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
As online platforms monitor and intervene in the daily lives of billions of people, platforms are being used to govern enduring social problems. Field experiments could inform wise uses of this power if tensions between democratic values and experimentation could be resolved. In this paper, we introduce CivilServant, a novel experimentation infrastructure that online communities and their moderators use to evaluate policies and replicate each others' findings. We situate CivilServant in the political history of policy experiments and present design considerations for community participation, ethics, and replication. Based on two case studies of community-led experiments and public debriefings on the reddit platform, we share findings on community deliberation about experiment results. We also report on uses of evidence, finding that experiments informed moderator practices, community policies, and replications by communities and platforms. We discuss the implications of these findings for evaluating platform governance in an open, democratic, experimenting society.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708 · cited 2×
“By publishing all software and analyses, we are also able to contribute to an open research culture among other scholars who can query and replicate our findings [57].”
BackgroundMethodGillespie, T. (2010) · New Media & Society · doi:10.1177/1461444809342738 · Cited by 2,522 · cited 2×
“The most common strategy requires platform operators to take governance power, build policy teams, and hopefully develop the research capacities to evaluate those policies [17, 33, 31, 8].”
BackgroundRacial Discrimination in the Sharing Economy: Evidence from a Field Experiment
Edelman, B. G., Luca, M., & Svirsky, D. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2701902 · Cited by 999
Kurt Lewin and the Origins of Action Research
Adelman, C. (1993) · Educational Action Research · doi:10.1080/0965079930010102 · Cited by 913
Turkopticon: interrupting worker invisibility in amazon mechanical turk
Irani, L. C., & Silberman, M. S. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2470742 · Cited by 798 · cited 2×
“Critical infrastructure systems such as Turkopticon have typically been evaluated for their role to foster reflection and collective organizing among those who use or encounter it [29, 41, 22, 40].”
BackgroundThe relationship of action research to human-computer interaction
Hayes, G. R. (2011) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1993060.1993065 · Cited by 452
Decentralization in wikipedia governance.
Forte, A., Larco, V., & Bruckman, A. (2009) · Journal of Management Information Systems · doi:10.2753/mis0742-1222260103 · Cited by 300 · cited 2×
“Internet users continue to create and enact policy on many major internet platforms, including Wikipedia [28], Facebook [26], Twitter [56, 30], reddit [51], and Xbox [34].”
BackgroundGeiger, R. S. (2016) · Information, Communication & Society · doi:10.1080/1369118x.2016.1153700 · Cited by 125
Standard Operating Procedures: A Safety Net for Pre-Analysis Plans
Lin, W., & Green, D. P. (2016) · Political Science and Politics · doi:10.1017/s1049096516000810 · Cited by 71 · cited 1×
“A narrative pre-analysis plan provides the community with a formal description in non-expert language that explains a study’s goals, interventions, variables, and analysis procedures [47].”
Background
Cited by
(2019) · doi:10.1080/1369118x.2019.1573914 · Cited by 719
Matias, J. N. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1813486116 · Cited by 208
Self-citationThe Civic Labor of Volunteer Moderators Online
Matias, J. N. (2019) · Social Media + Society · doi:10.1177/2056305119836778 · Cited by 183
Self-citationAlgorithmic realism: expanding the boundaries of algorithmic thought
Green, B., & Viljoen, S. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372840 · Cited by 161
Quarantined! Examining the Effects of a Community-Wide Moderation Intervention on Reddit
Chandrasekharan, E., Jhaver, S., Bruckman, A., & Gilbert, E. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3490499 · Cited by 129
“r them to efficiently moderate high volumes of user-generated content efficiently [31, 40, 53]. They have therefore established a variety of complex, intricate systems to implement content moderation [8, 12, 41, 42, 56, 77]. In recent years, researchers in HCI as well as other disciplines have started to examine the c…”
Background
Metadata
- Authors
- Matias, J. N., & Mou, M.
- Venue
- Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2018-04-19
- Pages
- 1-13
- OpenAlex
- W2795755461
- Cited by
- 56
- References
- 9
- FWCI
- 50.37
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.