Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 49 of the 51 references it cites were found in a registry or an index; 7 of the 24 sources its claims cite were checked for support.

Misogynoir: challenges in detecting intersectional hate

Open access
Kwarteng, Joseph, Perfumi, Serena Coppolino, Farrell, Tracie, Third, Aisling, Fernandez, Miriam · 2022 · Social Network Analysis and Mining

Abstract

Evaluates the performance of automated hate speech detection tools (HateSonar and Google's Perspective API) on detecting misogynoir in tweets, finding that current tools are ineffective due to a lack of sensitivity to context.

Overview

CoChat Summary

## TL;DR

This paper evaluates two hate-speech detection tools, HateSonar and Google’s Perspective API, on Twitter data about misogynoir and finds that both systems are ineffective at detecting this intersectional hate. The main issue is lack of context sensitivity: the tools often flag explicit references to racism, sexism, or profanity, but miss subtler forms such as White Centring and Defensiveness.

## Key findings

  • On the balanced dataset of 300 tweets (150 misogynoir, 150 allyship), Perspective API outperformed HateSonar, but still performed poorly for misogynoir detection: precision 0.66, recall 0.33, F1 0.44, accuracy 0.58; only 17% of misogynoir tweets were correctly classified by Perspective API and 9.3% by HateSonar.
  • On the imbalanced dataset of 3,138 tweets (162 misogynoir, 2,976 allyship), Perspective API again outperformed HateSonar, with precision 0.10, recall 0.33, F1 0.15 for misogynoir; only 0.96% of misogynoir tweets were correctly classified by Perspective API and 1.7% by HateSonar.
  • For the Perspective API’s sub-attributes on the balanced dataset, the strongest were Identity attack and Insult, with balanced-dataset scores of precision/recall/F1/accuracy = 0.66/0.31/0.42/0.57 and 0.70/0.29/0.41/0.58, respectively.
  • The paper reports that Racial gaslighting and Tone policing were easier to detect than White centring and Defensiveness, because the former have more lexical clues; on the imbalanced dataset, the table reports correct classifications for misogynoir types such as Racial gaslighting: 27/53 (57%) and Tone policing: 9/22 (41%), versus White centring: 3/16 (19%) and Defensiveness: 3/9 (33%) by Perspective API.

## Methods

  • The study built on a prior Twitter collection about four Black women in tech and assembled a new annotated dataset; the paper states a combined dataset of 4,532 tweets for the study, with an analysis dataset of 3,138 tweets after filtering out unclear cases.
  • Tweets were manually annotated by three authors into Misogynoir (M), Allyship (A), or Unclear (U), with allyship further divided into subtypes such as Sharing experiences (E), Thanks (T), and Generic support (GR); annotation agreement improved from Fleiss’ kappa 0.66 to 0.89 after discussion and refinement.
  • The authors evaluated HateSonar and Google’s Perspective API on a balanced sample of 300 tweets and on the full imbalanced dataset, using precision, recall, F1, and accuracy, and inspected confusion matrices plus false positives/negatives qualitatively.

## Limitations

  • The paper states that the classifiers’ labels do not match the dataset labels: HateSonar outputs hate speech / offensive language / neither, so the authors had to treat hate speech and offensive language as potential misogynoir, which they say “might not be an accurate representation” of misogynoir.
  • The text makes evident that the study is limited by context loss in Twitter data: deleted tweets, suspended accounts, missing threads, and URL-only posts made annotation difficult; this is stated in the paper.
  • The paper also makes evident that the dataset is narrowly grounded in one prior case study context (public responses to four Black women in tech), which may limit generalization; this is implied by the dataset construction and the paper’s own discussion of context-specific language.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s13278-022-00993-7

Cites

49 of 49 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

6 Valid · 2 Not valid · 13 Not checked · 57 Own results and interpretations

40 more claims

Create a free account to see all 78 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 17
  • It may state two separate claims as one. 21

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 51 references

Research works printed but not linked: 4

  1. Crenshaw KW (2017) On intersectionality: essential writings. The New Press
  2. Fitzsimons A (2022) Intersectional identities and machine learning: illuminating language biases in twitter algorithms. Hicss pp. 1–10)
  3. Kumar D, Mason J, Bailey M, Gage P, Consolvo KS, Bursztein E, Thomas K (n.d.) This paper is included in the Proceedings of the 17th symposium on usable privacy and security. Designing toxic content classification for a diversity of perspectives designing toxic content classification for a diversity of perspectives. Retrieved from https://data.esrg.stanford.edu/study/toxicity-perspectives
  4. Tan YC, Celis LE (2019) Assessing social and intersectional biases in contextualized word representations. Advances in neural information processing systems pp. 13230–13241

Cited by

See all 30

Metadata

Authors
Kwarteng, J., Perfumi, S. C., Farrell, T., Third, A., & Fernandez, M.
Affiliations
The Open University (GB)
Corresponding author
Kwarteng, Joseph
Venue
Social Network Analysis and Mining
Published
2022-11-09
Volume
12
Issue
1
ISSN
  • 1869-5450 (ISSN-L)
  • 1869-5469
OpenAlex
W4308747251
Cited by
30
References
51
Citations by year
2026: 7 · 2025: 8 · 2024: 7 · 2023: 8
FWCI
1.89
Type
journal article
Language
en
Open access
Licence
Locations
  • Social Network Analysis and Mining (publishedVersion, cc-by)
  • Open Research Online - ORO (The Open University) (acceptedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
Keywords
misogynoir, hate speech detection, Black women, Perspective API, Twitter, context sensitivity, racism, sexism, offensive language, toxicity classification, allyship

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-09-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Misogynoir: challenges in detecting intersectional hate. https://app.cochat.ai/papers/10.1007/s13278-022-00993-7
The paper
Kwarteng, J., Perfumi, S. C., Farrell, T., Third, A., & Fernandez, M. (2022). Misogynoir: challenges in detecting intersectional hate. Social Network Analysis and Mining. https://doi.org/10.1007/s13278-022-00993-7

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.