Identifying and Measuring Annotator Bias Based on Annotators’ Demographic Characteristics
Open accessThe DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
Machine learning is recently used to detect hate speech and other forms of abusive language in online platforms.However, a notable weakness of machine learning models is their vulnerability to bias, which can impair their performance and fairness.One type is annotator bias caused by the subjective perception of the annotators.In this work, we investigate annotator bias using classification models trained on data from demographically distinct annotator groups.To do so, we sample balanced subsets of data that are labeled by demographically distinct annotators.We then train classifiers on these subsets, analyze their performances on similarly grouped test sets, and compare them statistically.Our findings show that the proposed approach successfully identifies bias and that demographic features, such as first language, age, and education, correlate with significant performance differences.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019) · arXiv · doi:10.48550/arxiv.1910.01108 · Cited by 4,653
Answering the Call for a Standard Reliability Measure for Coding Data
Hayes, A. F., & Krippendorff, K. (2007) · Communication Methods and Measures · doi:10.1080/19312450709336664 · Cited by 4,225
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610
Bender, E. M., & Friedman, B. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00041 · Cited by 854 · cited 1×
“Therefore, it is not surprising that a large body of work has identified and mitigated this bias (Bender and Friedman, 2018; Bountouridis et al., 2019; Dixon et al., 2018).”
BackgroundThe Risk of Racial Bias in Hate Speech Detection
Sap, M., Card, D., Gabriel, S., Choi, Y., & Smith, N. A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1163 · Cited by 817 · cited 1×
“Some work examined racial bias (Sap et al., 2019; Davidson et al., 2019; Xia et al., 2020), others explored gender bias (Gold and Zesch, 2018), aggregation bias (Balayn et al., 2018) and political bias (Wich et al., 2020b).”
BackgroundSmith, L. N. (2018) · arXiv · doi:10.48550/arxiv.1803.09820 · Cited by 790 · cited 1×
“We used the base uncased version of DistilBERT ( distilbert-base-uncased ) with a maximum sequence length of 100, a learning rate of 5 × 10 − 6 , and 1cycle learning rate policy (Smith, 2018) and trained each classifier for 2 epochs.”
MethodMeasuring and Mitigating Unintended Bias in Text Classification
Dixon, L., Li, J., Sorensen, J., Thain, N., & Vasserman, L. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278729 · Cited by 674 · cited 4×
“Since unintended bias in hate speech datasets can impair the model’s performance (Waseem, 2016) and fairness (Vidgen et al., 2019a; Dixon et al., 2018), a lot of recent work has been done to investigate this phenomenon (Wiegand et al., 2019; Kim et al., 2020).”
BackgroundInfluentialEx Machina: Personal Attacks Seen at Scale
Wulczyn, E., Thain, N., & Dixon, L. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052591 · Cited by 636
Cited by
Offensive, aggressive, and hate speech analysis: From data-centric to human-centered approach
Kocoń, J., Figas, A., Gruza, M., Puchalska, D., Kajdanowicz, T., & Kazienko, P. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102643 · Cited by 103
Razi, A., Kim, S., Alsoubai, A., Stringhini, G., Solorio, T., De Choudhury, M., & Wisniewski, P. J. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479609 · Cited by 95
Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation
Goyal, N., Kivlichan, I. D., Rosen, R., & Vasserman, L. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555088 · Cited by 76
Human-centered neural reasoning for subjective content processing: Hate speech, emotions, and humor
Kazienko, P., Bielaniewicz, J., Gruza, M., Kanclerz, K., Karanowski, K., Miłkowski, P., & Kocoń, J. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.01.010 · Cited by 38
Mm-soc: Benchmarking multimodal large language models in social media platforms
Jin, Y., Choi, M., Verma, G., Wang, J., & Kumar, S. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.370 · Cited by 20
Metadata
- Authors
- Al Kuwatly, H., Wich, M., & Groh, G.
- Venue
- Proceedings of the Fourth Workshop on Online Abuse and Harms
- Published
- 2020
- Pages
- 184-190
- OpenAlex
- W3102641573
- Cited by
- 83
- References
- 12
- FWCI
- 5.85
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Identifying and Measuring Annotator Bias Based on Annotators’ Demographic Characteristics. https://app.cochat.ai/papers/10.18653/v1/2020.alw-1.21
- The paper
- Al Kuwatly, H., Wich, M., & Groh, G. (2020). Identifying and Measuring Annotator Bias Based on Annotators’ Demographic Characteristics. Proceedings of the Fourth Workshop on Online Abuse and Harms. https://doi.org/10.18653/v1/2020.alw-1.21
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.