CoChat
Sign Up
PreprintCited by 13Hate Speech and Cyberbullying DetectionSentiment Analysis and Opinion MiningSpeech Recognition and Synthesis

DeToxy: A Large-Scale Multimodal Dataset for Toxicity Classification in Spoken Utterances

Open access
Ghosh, Sreyan, Lepcha, Samden, Sakshi, S, Shah, Rajiv Ratn, Umesh, S.·2021·arXiv
DOI 10.48550/arxiv.2110.07592arXiv 2110.07592
Open this paper in CoChat — free accountOpen PDF

Abstract

Toxic speech, also known as hate speech, is regarded as one of the crucial issues plaguing online social media today. Most recent work on toxic speech detection is constrained to the modality of text and written conversations with very limited work on toxicity detection from spoken utterances or using the modality of speech. In this paper, we introduce a new dataset DeToxy, the first publicly available toxicity annotated dataset for the English language. DeToxy is sourced from various openly available speech databases and consists of over 2 million utterances. We believe that our dataset would act as a benchmark for the relatively new and un-explored Spoken Language Processing task of detecting toxicity from spoken utterances and boost further research in this space. Finally, we also provide strong unimodal baselines for our dataset and compare traditional two-step and E2E approaches. Our experiments show that in the case of spoken utterances, text-based approaches are largely dependent on gold human-annotated transcripts for their performance and also suffer from the problem of keyword bias. However, the presence of speech files in DeToxy helps facilitates the development of E2E speech models which alleviate both the above-stated problems by better capturing speech clues.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2110.07592

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Most efforts in toxicity detection focus on social media comments.

    “Most efforts in the domain of toxicity detection are focusing on social media comments (Cheng et al. 2022;Hosseinmardi et al. 2015;Ghosh et al. 2021;Bensalem, Rosso, and Zitouni 2024).”

    From Benchmark on Peer Review Toxic Detection: A Challenging Task with a New Dataset (2025) · cites this paper as Ghosh et al. 2021

    The Introduction says most prior work and datasets focus on conversational text, and Related Work says most hate-speech detection work is text-based. The paper discusses social networks and cites social-media studies, but it does not establish that most efforts specifically focus on social media comments. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 30

  1. Y. Xie, R. Liang, Z. Liang, C. Huang, C. Zou, and B. Schuller, "Speech emotion classification using attention-based lstm," IEEE/ACM Transactions on Audio, Speech, and Language Pro- cessing, vol. 27, no. 11, pp. 1675-1685.
  2. P. Tzirakis, J. Zhang, and B. W. Schuller, "End-to-end speech emotion recognition using deep neural networks," in IEEE ICASSP 2018, pp. 5089-5093.
  3. E. Lakomkin, M. A. Zamani, C. Weber, S. Magg, and S. Wermter, "Incorporating end-to-end speech recognition models for senti- ment analysis," in IEEE ICRA 2019, pp. 7976-7982.
  4. T. Davidson, D. Warmsley, M. Macy, and I. Weber, "Automated hate speech detection and the problem of offensive language," in ICWSM 2017, vol. 11, 2017.
  5. J. A. Leite, D. Silva, K. Bontcheva, and C. Scarton, "Toxic lan- guage detection in social media for Brazilian Portuguese: New dataset and multilingual analysis," in ACL-IJCNLP 2020, Suzhou, China, pp. 914-924.
  6. M. Yousefi and D. Emmanouilidou, "Audio-based toxic language classification using self-attentive convolutional neural network," in EUSIPCO 2021, pp. 11-15.
  7. S. Ling and Y. Liu, "Decoar 2.0: Deep contextualized acous- tic representations with vector quantization," arXiv preprint arXiv:2012.06659, 2020. · arXiv
  8. A. T. Liu, S.-w. Yang, P.-H. Chi, P.-c. Hsu, and H.-y. Lee, "Mock- ingjay: Unsupervised speech representation learning with deep bidirectional transformer encoders," in IEEE ICASSP 2020.
  9. I. Cohn, I. Laish, G. Beryozkin, G. Li, I. Shafran, I. Szpektor, T. Hartman, A. Hassidim, and Y. Matias, "Audio de-identification -a new entity recognition task," in NAACL 2019, Minneapolis, Minnesota, pp. 197-204.
  10. H. Yadav, S. Ghosh, Y. Yu, and R. R. Shah, "End-to-end named entity recognition from english speech," in Interspeech 2021, 2021, pp. 4268-4272.
  11. Z. Lu, L. Cao, Y. Zhang, C.-C. Chiu, and J. Fan, "Speech senti- ment analysis via pre-trained features from end-to-end asr mod- els," in IEEE ICASSP 2020, pp. 7149-7153.
  12. V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, "Lib- rispeech: an asr corpus based on public domain audio books," in IEEE ICASSP 2015, pp. 5206-5210.
  13. F. Hernandez, V. Nguyen, S. Ghannay, N. Tomashenko, and Y. Es- teve, "Ted-lium 3: twice as much data and corpus repartition for experiments on speaker adaptation," in International conference on speech and computer. Springer, 2018, pp. 198-208.
  14. M. Z. Boito, W. N. Havard, M. Garnerin, É. L. Ferrand, and L. Besacier, "Mass: A large and clean multilingual corpus of sentence-aligned spoken utterances extracted from the bible," arXiv preprint arXiv:1907.12895, 2019. · arXiv
  15. M. S. Grover, P. Bamdev, Y. Kumar, M. Hama, and R. R. Shah, "audino: A modern annotation tool for audio and speech," CoRR, vol. abs/2006.05236, 2020. · arXiv
  16. M. Sap, D. Card, S. Gabriel, Y. Choi, and N. A. Smith, "The risk of racial bias in hate speech detection," in ACL 2019, pp. 1668- 1678.
  17. T. Garg, S. Masud, T. Suresh, and T. Chakraborty, "Han- dling bias in toxic speech detection: A survey," arXiv preprint arXiv:2202.00126, 2022. · arXiv
  18. A. Zadeh and P. Pu, "Multimodal language analysis in the wild: Cmu-mosei dataset and interpretable dynamic fusion graph," in ACL 2018.
  19. A. Zadeh, R. Zellers, E. Pincus, and L.-P. Morency, "Mosi: mul- timodal corpus of sentiment intensity and subjectivity analysis in online opinion videos," arXiv preprint arXiv:1606.06259, 2016. · arXiv
  20. R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, and G. Weber, "Common voice: A massively-multilingual speech corpus," arXiv preprint arXiv:1912.06670, 2019. · arXiv
  21. C. Busso, M. Bulut, C.-C. Lee, A. Kazemzadeh, E. Mower, S. Kim, J. N. Chang, S. Lee, and S. S. Narayanan, "Iemocap: Interactive emotional dyadic motion capture database," LREC, vol. 42, no. 4, pp. 335-359, 2008.
  22. S. Poria, D. Hazarika, N. Majumder, G. Naik, E. Cambria, and R. Mihalcea, "MELD: A multimodal multi-party dataset for emo- tion recognition in conversations," in ACL 2019, pp. 527-536.
  23. C. Busso, S. Parthasarathy, A. Burmania, M. AbdelWahab, N. Sadoughi, and E. M. Provost, "Msp-improv: An acted corpus of dyadic interactions to study emotion perception," IEEE Trans- actions on Affective Computing, vol. 8, no. 1, pp. 67-80, 2016.
  24. R. Lotfian and C. Busso, "Building naturalistic emotionally bal- anced speech corpus by retrieving emotional speech from existing podcast recordings," IEEE Transactions on Affective Computing, vol. 10, no. 4, pp. 471-483, 2017.
  25. A. Zadeh, M. Chan, P. P. Liang, E. Tong, and L.-P. Morency, "Social-iq: A question answering benchmark for artificial social intelligence," in IEEE CVPR 2019.
  26. W.-N. Hsu, A. Sriram, A. Baevski, T. Likhomanenko, Q. Xu, V. Pratap, J. Kahn, A. Lee, R. Collobert, G. Synnaeve, and M. Auli, "Robust wav2vec 2.0: Analyzing domain shift in self- supervised pre-training," in Interspeech 2021, 2021, pp. 721-725.
  27. P. Malik, A. Aggrawal, and D. K. Vishwakarma, "Toxic speech detection using traditional machine learning models and bert and fasttext embedding with deep neural networks," in IEEE ICCMC 2021, pp. 1254-1259.
  28. Z. Zhao, Z. Zhang, and F. Hopfgartner, "A comparative study of using pre-trained language models for toxic comment classifica- tion," in Companion Proceedings of the Web Conference, 2021, pp. 500-507.
  29. D. Borkan, L. Dixon, J. Sorensen, N. Thain, and L. Vasserman, "Nuanced metrics for measuring unintended bias with real data for text classification," CoRR, vol. abs/1903.04561, 2019. · arXiv
  30. J. Shah, Y. K. Singla, C. Chen, and R. R. Shah, "What all do audio transformer models hear? probing acoustic represen- tations for language delivery and its structure," arXiv preprint arXiv:2101.00387, 2021. · arXiv

Cited by

Sorted by citations

Authors
Ghosh, S., Lepcha, S., Sakshi, S., Shah, R. R., & Umesh, S.
Affiliations
  • Indian Institute of Technology Madras (IN)
  • Indraprastha Institute of Information Technology Delhi (IN)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2021
Pages
5185-5189
ISSN
2308-457X (ISSN-L)
DOI
10.48550/arxiv.2110.07592
arXiv
2110.07592
OpenAlex
W4297841625
Cited by
13
References
0
Citations by year
2026: 2 · 2025: 7 · 2024: 2 · 2023: 2
FWCI
1.52
Type
conference paper
Language
en
Open access
Licence
Locations
  • Interspeech (USB) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
  • Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
Keywords
hate speech, spoken utterances, multimodal dataset, English language

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review