CoChat
Sign Up
Cited by 107Hate Speech and Cyberbullying DetectionMisinformation and Its ImpactsSentiment Analysis and Opinion Mining

Dynamics of online hate and misinformation

Open access
Cinelli, Matteo, Pelicon, Andraž, Mozetič, Igor, Quattrociocchi, Walter, Novak, Petra Kralj, Zollo, Fabiana·2021·Scientific Reports
DOI 10.1038/s41598-021-01487-wPMC PMC8585974
Open this paper in CoChat — free accountOpen PDF

Abstract

Online debates are often characterised by extreme polarisation and heated discussions among users. The presence of hate speech online is becoming increasingly problematic, making necessary the development of appropriate countermeasures. In this work, we perform hate speech detection on a corpus of more than one million comments on YouTube videos through a machine learning model, trained and fine-tuned on a large set of hand-annotated data. Our analysis shows that there is no evidence of the presence of “pure haters”, meant as active users posting exclusively hateful comments. Moreover, coherently with the echo chamber hypothesis, we find that users skewed towards one of the two categories of video channels (questionable, reliable) are more prone to use inappropriate, violent, or hateful language within their opponents’ community. Interestingly, users loyal to reliable sources use on average a more toxic language than their counterpart. Finally, we find that the overall toxicity of the discussion increases with its length, measured both in terms of the number of comments and time. Our results show that, coherently with Godwin’s law, online debates tend to degenerate towards increasingly toxic exchanges of views.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41598-021-01487-w

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • User leaning was also estimated by activity in specific groups, by semantics in the posts, calculation of political slant of visited websites by the user, and shared news URLs with news outlets leaning score.

    “User leaning was also estimated by activity in specific groups (e.g., [59,127,143]), by semantics in the posts (e.g., [94,99,113]), calculation of political slant of visited websites by the user [124], shared news URLs with news outlets leaning score (e.g., [8,96,129]).”

    From A systematic review of echo chamber research: comparative analysis of conceptualizations, operationalizations, and varying outcomes (2025) · cites this paper as [143]

    In Results and discussion, “Users’ behaviour and misinformation” (page unavailable), the paper estimates user leaning from the fraction of comments a user posts on videos from questionable YouTube channels. It does not estimate leaning from shared news URLs or news outlets’ political-leaning scores, the specific method to which this citation is attached. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 31

  1. [1]Adamic, L. A., Glance, N. The political blogosphere and the 2004 us election: Divided they blog. In Proceedings of the 3rd International Workshop on Link Discovery, pp. 36–43 (2005).
  2. [3]Coe K Kenski K Rains SA Online and uncivil? Patterns and determinants of incivility in newspaper website comments J. Commun. 2014 64 4 658 679 10.1111/jcom.12104 · DOI
  3. [7]Calvert C Hate speech and its harms: A communication theory perspective J. Commun. 1997 47 1 4 19 10.1111/j.1460-2466.1997.tb02690.x · DOI
  4. [8]Chan J Ghose A Seamans R The internet and racial hate crime: Offline spillovers from online access MIS Q. 2016 40 2 381 403 10.25300/MISQ/2016/40.2.05 · DOI
  5. [15]Fortuna P Nunes S A survey on automatic detection of hate speech in text ACM Comput. Surv. (CSUR) 2018 51 4 1 30 10.1145/3232676 · DOI · Paper page
  6. [16]Kumar, S., Hamilton, W. L., Leskovec, J. & Jurafsky, D. Community interaction and conflict on the web. In Proceedings of the 2018 World Wide Web Conference, pp. 933–943 (2018).
  7. [17]Johnson NF Leahy R Restrepo NJ Velasquez N Zheng M Manrique P Devkota P Wuchty S Hidden resilience and adaptive dynamics of the global online hate ecology Nature 2019 573 7773 261 265 10.1038/s41586-019-1494-7 31435010 · DOI
  8. [18]Mathew B Illendula A Saha P Sarkar S Goyal P Mukherjee A Hate begets hate: A temporal study of hate speech Proc. ACM Hum. Comput. Interact. 2020 4 CSCW2 1 24 10.1145/3415163 · DOI
  9. [19]Ribeiro, M., Calais, P., Santos, Y., Almeida, V. & Meira Jr., W. Characterizing and detecting hateful users on twitter. In Proceedings of the International AAAI Conference on Web and Social Media, vol. 12 (2018).
  10. [20]Siegel AA Nikitin E Barberá P Sterling J Pullen B Bonneau R Nagler J Tucker JA Trumping hate on twitter? Online hate speech in the 2016 us election campaign and its aftermath Q. J. Polit. Sci. 2021 16 1 71 104 10.1561/100.00019045 · DOI
  11. [21]Evkoski, B., Pelicon, A., Mozetič, I., Ljubešić, N. & Novak, P. K. Retweet communities reveal the main sources of hate speech. arXiv:2105.14898 (2021). PMC8929563 35298556 · arXiv
  12. [22]Schild, L., Ling, C., Blackburn, J., Stringhini, G., Zhang, Y. & Zannettou, S. “Go eat a bat, chang!”: An early look on the emergence of sinophobic behavior on web communities in the face of covid-19. arXiv:2004.04046 (2020). · arXiv
  13. [23]Chandrasekharan, E., Samory, M., Srinivasan, A. & Gilbert, E. The bag of communities: Identifying abusive behavior online with preexisting internet data. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 3175–3187 (2017).
  14. [24]Burnap P Williams ML Us and them: Identifying cyber hate on twitter across multiple protected characteristics. EPJ Data Sci. 2016 5 1 15 10.1140/epjds/s13688-016-0072-6 PMC7175598 32355598 · DOI
  15. [25]Del Vigna, F., Cimino, A., Dell’Orletta, F., Petrocchi, M. & Tesconi, M. Hate me, hate me not: Hate speech detection on facebook. In Proceedings of the First Italian Conference on Cybersecurity (ITASEC17), pp. 86–95 (2017).
  16. [26]Davidson, T., Warmsley, D., Macy, M. & Weber, I. Automated hate speech detection and the problem of offensive language. In Proceedings of the International AAAI Conference on Web and Social Media, vol. 11 (2017).
  17. [27]Badjatiya, P., Gupta, S., Gupta, M. & Varma, V. Deep learning for hate speech detection in tweets. In Proceedings of the 26th International Conference on World Wide Web Companion, pp. 759–760 (2017).
  18. [28]Basile, V., Bosco, C., Fersini, E., Debora, N., Patti, V., Pardo, F. M. R., Rosso, P. & Sanguinetti, M. et al. Semeval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter. In 13th International Workshop on Semantic Evaluation, pp. 54–63 (Association for Computational Linguistics, 2019).
  19. [29]Zampieri, M., Nakov, P., Rosenthal, S., Atanasova, P., Karadzhov, G., Mubarak, H., Derczynski, L., Pitenis, Z. & Çöltekin, Ç. Semeval-2020 task 12: Multilingual offensive language identification in social media (offenseval 2020). arXiv:2006.07235 (2020). · arXiv
  20. [30]Cinelli M Quattrociocchi W Galeazzi A Valensise CM Brugnoli E Schmidt AL Zola P Zollo F Scala A The covid-19 social media infodemic Sci. Rep. 2020 10 1 1 10 10.1038/s41598-020-73510-5 33024152 PMC7538912 · DOI · Paper page
  21. [31]Zollo F Novak PK Vicario MD Bessi A Mozetič I Scala A Caldarelli G Quattrociocchi W Emotional dynamics in the age of misinformation PLoS One 2015 10 09 1 22 10.1371/journal.pone.0138740 PMC4589395 26422473 · DOI · Paper page
  22. [33]Gagliardone, I., Pohjonen, M., Beyene, Z., Zerai, A., Aynekulu, G., Bekalu, M., Bright, J., Moges, M., Seifu, M. & Stremlau, N. et al. Mechachal: Online debates and elections in Ethiopia—from hate speech to engagement in social media. Available at SSRN 2831369 (2016).
  23. [35]Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N. & Kumar, R. SemEval-2019 task 6: Identifying and categorizing offensive language in social media (OffensEval). In Proceedings of the 13th International Workshop on Semantic Evaluation, pp. 75–86 (Association for Computational Linguistics, 2019).
  24. [36]Bosco, C., Dell’Orletta, F., Poletto, F., Sanguinetti, M. & Maurizio, T. Overview of the evalita 2018 hate speech detection task. In EVALITA 2018-Sixth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian, vol. 2263, pp. 1–9 (CEUR, 2018).
  25. [38]Sanguinetti, M., Poletto, F., Bosco, C., Patti, V. & Stranisci, M. An Italian Twitter corpus of hate speech against immigrants. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018) (2018).
  26. [39]Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N. & Kumar, R. Predicting the type and target of offensive posts in social media. In Proceedings of NAACL (2019).
  27. [41]Krippendorff, K. Content Analysis. An Introduction to its Methodology, 4th edn. (Sage Publications, 2018).
  28. [42]Mozetič I Grčar M Smailović J Multilingual Twitter sentiment classification: The role of human annotators PLoS One 2016 11 5 e0155036 10.1371/journal.pone.0155036 27149621 PMC4858191 · DOI
  29. [44]Polignano, M., Basile, P., De Gemmis, M., Semeraro, G. & Basile, V. AlBERTo: Italian BERT language understanding model for NLP challenging tasks based on tweets. In 6th Italian Conference on Computational Linguistics, CLiC-it 2019, vol. 2481, pp. 1–6 (CEUR, 2019).
  30. [47]Del Vicario M Quattrociocchi W Scala A Zollo F Polarization and fake news: Early warning of potential misinformation targets ACM Trans. Web (TWEB) 2019 13 2 1 22 10.1145/3316809 · DOI · Paper page
  31. [49]Guess, A., Nagler, J., & Tucker, J. Less than you think: Prevalence and predictors of fake news dissemination on facebook. Sc. Adv.5(1), eaau4586 (2019). PMC6326755 30662946

Cited by

Sorted by citations

Authors
Cinelli, M., Pelicon, A., Mozetič, I., Quattrociocchi, W., Novak, P. K., & Zollo, F.
Affiliations
  • Ca' Foscari University of Venice (IT)
  • Jožef Stefan Institute (SI)
  • Sapienza University of Rome (IT)
Corresponding author
Zollo, Fabiana
ORCID
Venue
Scientific Reports
Publisher
Springer Science and Business Media LLC
Published
2021-11-11
Volume
11
Issue
1
Pages
22083-22083
ISSN
2045-2322 (ISSN-L)
DOI
10.1038/s41598-021-01487-w
PMC
PMC8585974
OpenAlex
W3212567067
Cited by
107
References
7
Citations by year
2026: 9 · 2025: 39 · 2024: 24 · 2023: 17 · 2022: 17 · 2021: 1
FWCI
5.21
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • IRIS Research product catalog (Sapienza University of Rome) (submittedVersion)
  • ARCA (Università Ca' Foscari Venezia) (submittedVersion, cc-by-nc-nd)
Funders
  • Rights, Equality and Citizenship Programme (875263)
  • Slovenian Research Agency (P2-103)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
Keywords
online hate speech, misinformation, YouTube comments, hate speech detection, online polarization, echo chambers, toxic language, online debate

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review