Ex Machina: Personal Attacks Seen at Scale
Open accessThe DOI matches its Crossref record; 31 of the 33 references it cites were found in a registry or an index.
Abstract
The damage personal attacks cause to online discourse motivates many platforms to try to curb the phenomenon. However, understanding the prevalence and impact of personal attacks in online platforms at scale remains surprisingly difficult. The contribution of this paper is to develop and illustrate a method that combines crowdsourcing and machine learning to analyze personal attacks at scale. We show an evaluation method for a classifier in terms of the aggregated number of crowd-workers it can approximate. We apply our methodology to English Wikipedia, generating a corpus of over 100k high quality human-labeled comments and 63M machine-labeled ones from a classifier that is as good as the aggregate of 3 crowd-workers, as measured by the area under the ROC curve and Spearman correlation. Using this corpus of machine-labeled scores, our methodology allows us to explore some of the open questions about the nature of online personal attacks. This reveals that the majority of personal attacks on Wikipedia are not the result of a few malicious users, nor primarily the consequence of allowing anonymous contributions from unregistered users.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Therefore, using a model derived from this dataset allows ecological validity to our study as the dataset is used by real-world social media platform and community moderators.”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [88]
Citations
Cites
Content Analysis: An Introduction to its Methodology.
Shelley, M., & Krippendorff, K. (1984) · Journal of the American Statistical Association · doi:10.2307/2288384 · Cited by 24,312
Content Analysis: An Introduction to Its Methodology
Krippendorff, K. (2019) · doi:10.4135/9781071878781 · Cited by 15,448
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
(2012) · Cited by 8,009
Opinion mining and sentiment analysis
Pang, B., & Lee, L. (2008) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000011 · Cited by 6,372
Answering the Call for a Standard Reliability Measure for Coding Data
Hayes, A. F., & Krippendorff, K. (2007) · Communication Methods and Measures · doi:10.1080/19312450709336664 · Cited by 4,225
Tokunaga, R. S. (2010) · Computers in Human Behavior · doi:10.1016/j.chb.2009.11.014 · Cited by 2,566 · cited 1×
“tudy focuses on personal attacks, other studies explore different forms of online behavior including hate speech ([7], [13], [19], [27]), online harassment ([3], [40]), and cyberbullying ([17], [20], [25], [35], [38]). Online harassment itself is sometimes further divided into a taxonomy of forms. A recent Pew Researc…”
BackgroundHateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter
Waseem, Z., & Hovy, D. (2016) · Proceedings of the NAACL Student Research Workshop · doi:10.18653/v1/n16-2013 · Cited by 1,655
Abusive Language Detection in Online User Content
Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y., & Chang, Y. (2016) · Proceedings of the 25th International Conference on World Wide Web · doi:10.1145/2872427.2883062 · Cited by 1,122
Reliability in content analysis: some common misconceptions and recommendations.
Krippendorff, K. (2004) · Human Communication Research · doi:10.1093/hcr/30.3.411 · Cited by 1,027
Detecting Hate Speech on the World Wide Web
Warner, W. B., & Hirschberg, J. (2012) · Cited by 964
Ybarra, M. L., & Mitchell, K. J. (2004) · Journal of Adolescence · doi:10.1016/j.adolescence.2004.03.007 · Cited by 737 · cited 1×
“It has been shown that anonymity provides psycho-social benefits to cyberbullies [14] and can lead to "heightened aggression and inappropriate behavior" [36].”
BackgroundWillard, N. E. (2006) · Cited by 708
Random search for hyper-parameter optimization
BergstraJames, & BengioYoshua (2012) · Journal of Machine Learning Research · doi:10.5555/2188385.2188395 · Cited by 690
Modeling the Detection of Textual Cyberbullying
Dinakar, K., Reichart, R., & Lieberman, H. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i3.14209 · Cited by 512
Detection of Harassment on Web 2.0
Yin, D., Xue, Z., Hong, L., Davison, B. D., & Edwards, L. (2009) · Cited by 452
Locate the Hate: Detecting Tweets against Blacks
Kwok, I., & Wang, Y. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v27i1.8539 · Cited by 446
Computer-Mediated Communication in Personal Relationships
(2011) · doi:10.3726/b12772 · Cited by 403
Antisocial Behavior in Online Discussion Communities
Cheng, J., Danescu-Niculescu-Mizil, C., & Leskovec, J. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v9i1.14583 · Cited by 314
Detecting offensive tweets via topical feature discovery over a large scale twitter corpus
Xiang, G., Fan, B., Wang, L., Hong, J., & Rose, C. (2012) · Proceedings of the 21st ACM international conference on Information and knowledge management · doi:10.1145/2396761.2398556 · Cited by 267
Survey on web spam detection: principles and algorithms
Spirin, N., & Han, J. (2012) · ACM SIGKDD Explorations Newsletter · doi:10.1145/2207243.2207252 · Cited by 252
A Corpus for Research on Deliberation and Debate
Walker, M. A., Tree, J. E. F., Anand, P., Abbott, R., & King, J. (2012) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2ommjodrhqvf · Cited by 223
Anonymity and roles associated with aggressive posts in an online forum
Moore, M. J., Nakano, T., Enomoto, A., & Suda, T. (2012) · Computers in Human Behavior · doi:10.1016/j.chb.2011.12.005 · Cited by 197
Antisocial Behavior in Online Discussion CommunitiesPreprint
Cheng, J. S., Danescu-Niculescu-Mizil, C., & Leskovec, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00680 · Cited by 171
Automatic identification of personal insults on social news sites
Sood, S. O., Churchill, E. F., & Antin, J. (2012) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21690 · Cited by 153
A Web of Hate: Tackling Hateful Speech in Online Social Spaces
Saleem, H. M., Dillon, K. P., Benesch, S., & Ruths, D. (2017) · arXiv · doi:10.48550/arxiv.1709.10159 · Cited by 144
Learning part-of-speech taggers with inter-annotator agreement loss
Plank, B., Hovy, D., & Søgaard, A. (2014) · Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/v1/e14-1078 · Cited by 94
Pieschl, S., Kuhlmann, C., & Porsch, T. (2014) · Journal of School Violence · doi:10.1080/15388220.2014.971363 · Cited by 65
Rethinking Grammatical Error Annotation and Evaluation with the Amazon Mechanical Turk
Tetreault, J., Filatova, E., & Chodorow, M. S. (2010) · Workshop on Innovative Use of NLP for Building Educational Applications · Cited by 32
An Examination of Regret in Bullying Tweets
Xu, J., Burchfiel, B., Zhu, X., & Bellmore, A. D. (2013) · Cited by 30
Wiener, D. H. (1999) · Santa Clara law review · Cited by 17
Research works printed but not linked: 4
- M. Duggan. Online harassment. Pew Research Center, 2014.
- I. Gagliardone, D. Gal, T. Alves, and G. Martinez. Countering online hate speech. UNESCO Publishing, 2015.
- K. Krippendorff. Reliability in content analysis. Human communication research, 30(3):411--433, 2004.
- 10.5555/1396222 · doi:10.5555/1396222
Cited by
Gorwa, R., Binns, R., & Katzenbach, C. (2020) · Big Data & Society · doi:10.1177/2053951719897945 · Cited by 1,097
The Risk of Racial Bias in Hate Speech Detection
Sap, M., Card, D., Gabriel, S., Choi, Y., & Smith, N. A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1163 · Cited by 817
“Many datasets have been created, but most are either small-scale pilots ( ∼ 100 instances; Kwok and Wang, 2013; Burnap and Williams, 2015; Zhang et al., 2018), or focus on other domains (e.g., Wikipedia edits; Wulczyn et al., 2017).”
BackgroundAI Alignment: A Contemporary Survey
Ji, J., Qiu, T., Chen, B., Zhou, J., Zhang, B., Hong, D., Lou, H., Wang, K., Duan, Y., He, Z., et al. (2025) · ACM Computing Surveys · doi:10.1145/3770749 · Cited by 809
ChatGPT: Jack of all trades, master of none
Kocoń, J., Cichecki, I., Kaszyca, O., Kochanek, M., Szydło, D., Baran, J., Bielaniewicz, J., Gruza, M., Janz, A., Kanclerz, K., et al. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101861 · Cited by 758
Measuring and Mitigating Unintended Bias in Text Classification
Dixon, L., Li, J., Sorensen, J., Thain, N., & Vasserman, L. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278729 · Cited by 674
Self-citation
Metadata
- Authors
- Wulczyn, E., Thain, N., & Dixon, L.
- Affiliations
- Wikimedia Foundation (US)
- Jigsaw, New York City, NY, USA
- Venue
- Proceedings of the 26th International Conference on World Wide Web
- Published
- 2017-04-03
- Pages
- 1391-1399
- OpenAlex
- W2540646130
- Cited by
- 636
- References
- 33
- Citations by year
- 2026: 35 · 2025: 52 · 2024: 43 · 2023: 74 · 2022: 64 · 2021: 140 · 2020: 91 · 2019: 71 · 2018: 52 · 2017: 14
- FWCI
- 28.57
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International World Wide Web Conference (WWW) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Social Sciences › Communication › Wikis in Education and Collaboration
- Keywords
- personal attacks, online discourse, crowdsourcing, classifier evaluation, malicious users
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.