Offensive, aggressive, and hate speech analysis: From data-centric to human-centered approach
Open accessAbstract
Analysis of subjective texts like offensive content or hate speech is a great challenge, especially regarding annotation process. Most of current annotation procedures are aimed at achieving a high level of agreement in order to generate a high quality reference source. However, the annotation guidelines for subjective content may restrict the annotators’ freedom of decision making . Motivated by a moderate annotation agreement in offensive content datasets, we hypothesize that personalized approaches to offensive content identification should be in place. Thus, we propose two novel perspectives of perception: group-based and individual. Using demographics of annotators as well as embeddings of their previous decisions (annotated texts), we are able to train multimodal models (including transformer-based) adjusted to personal or community profiles. Based on the agreement of individuals and groups, we experimentally showed that annotator group agreeability strongly correlates with offensive content recognition quality. The proposed personalized approaches enabled us to create models adaptable to personal user beliefs rather than to agreed offensiveness understanding. Overall, our individualized approaches to offensive content classification outperform classic data-centric methods that generalize offensiveness perception and it refers to all six tested models. Additionally, we developed requirements for annotation procedures, personalization and content processing to make the solutions human-centered.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
It is difficult to identify universal ground truth in tasks such as predicting emotions or offensiveness of text, especially in personalized context.
“At the same time, it is difficult to identify universal ground truth in tasks such as predicting emotions or offensiveness of text, especially in the personalized context [20, 21, 22].”
From ChatGPT: Jack of all trades, master of none (2023) · cites this paper as [20]
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148 · see citing works
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
An analysis of variance test for normality (complete samples)
SHAPIRO, S. S., & WILK, M. B. (1965) · Biometrika · doi:10.1093/biomet/52.3-4.591 · Cited by 23,480 · see citing works
An Analysis of Variance Test for Normality (Complete Samples)
Shapiro, S. S., & Wilk, M. B. (1965) · Biometrika · doi:10.2307/2333709 · Cited by 23,469 · see citing works
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,759 · see citing works
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645 · see citing works
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733 · see citing works
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687 · see citing works
Federated Machine Learning: Concept and Applications
Yang, Q., Liu, Y., Chen, T., & Tong, Y. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3298981 · Cited by 6,322 · see citing works
Unsupervised Cross-lingual Representation Learning at Scale
Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., & Stoyanov, V. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.747 · Cited by 5,637 · see citing works
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545 · see citing works
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308 · see citing works
Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms
Dietterich, T. G. (1998) · Neural Computation · doi:10.1162/089976698300017197 · Cited by 3,840 · see citing works
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610 · see citing works
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
Lu, J., Batra, D., Parikh, D., & Lee, S. (2019) · arXiv · doi:10.48550/arxiv.1908.02265 · Cited by 2,109 · see citing works
Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter
Talat, Z., & Hovy, D. (2016) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n16-2013 · Cited by 1,919 · see citing works
Cross-lingual Language Model Pretraining
Lample, G., & Conneau, A. (2019) · arXiv · doi:10.48550/arxiv.1901.07291 · Cited by 1,586 · see citing works
Sentiment Analysis of Twitter Data
Agarwal, A., Xie, B., Vovsha, I., Rambow, O., & Passonneau, R. J. (2011) · Cited by 1,565 · see citing works
ERNIE: Enhanced Language Representation with Informative Entities
Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., & Liu, Q. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1139 · Cited by 1,482 · see citing works
A Survey on Hate Speech Detection using Natural Language Processing
Schmidt, A. G., & Wiegand, M. (2017) · International Workshop on Natural Language Processing for Social Media (SocialNLP) · doi:10.18653/v1/w17-1101 · Cited by 1,447 · see citing works
Understanding bag-of-words model: a statistical framework
Zhang, Y., Jin, R., & Zhou, Z.-H. (2010) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-010-0001-0 · Cited by 1,384 · see citing works
How Multilingual is Multilingual BERT?
Pires, T., Schlinger, E., & Garrette, D. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1493 · Cited by 1,351 · see citing works
A Maximum Entropy Model for Part-Of-Speech Tagging
Ratnaparkhi, A. (1996) · Cited by 1,268 · see citing works
Deep Learning for Hate Speech Detection in Tweets
Badjatiya, P., Gupta, S., Gupta, M., & Varma, V. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3041021.3054223 · Cited by 1,243 · see citing works
A Survey on Automatic Detection of Hate Speech in Text
Fortuna, P., & Nunes, S. (2019) · ACM Computing Surveys · doi:10.1145/3232676 · Cited by 1,241 · see citing works
Abusive Language Detection in Online User Content
Nobata, C., Tetreault, J., Thomas, A. O., Mehdad, Y., & Chang, Y. (2016) · International World Wide Web Conference (WWW) · doi:10.1145/2872427.2883062 · Cited by 1,160 · see citing works
Proceedings of the 24th International Conference on World Wide Web
Gangemi, A., Leonardi, S., & Panconesi, A. (2015) · doi:10.1145/2740908 · Cited by 1,054 · see citing works
SemEval-2019 Task 5: Multilingual Detection of Hate Speech Against Immigrants and Women in Twitter
Basile, V., Bosco, C., Fersini, E., Nozza, D., Patti, V., Pardo, F. M. R., Rosso, P., & Sanguinetti, M. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2007 · Cited by 889 · see citing works
The Risk of Racial Bias in Hate Speech Detection
Sap, M., Card, D., Gabriel, S., Choi, Y., & Smith, N. A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1163 · Cited by 817 · see citing works
Computing Krippendorff's Alpha-Reliability
Krippendorff, K. (2011) · ScholarlyCommons (University of Pennsylvania) · Cited by 809 · see citing works
Unicoder-VL: A Universal Encoder for Vision and Language by Cross-Modal Pre-Training
Li, G., Duan, N., Fang, Y., Gong, M., & Jiang, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6795 · Cited by 795 · see citing works
Predicting the Type and Target of Offensive Posts in Social Media
Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., & Kumar, R. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1144 · Cited by 781 · see citing works
SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)
Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., & Kumar, R. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2010 · Cited by 750 · see citing works
Hate Speech Detection with Comment Embeddings
Djuric, N., Zhou, J. L., Morris, R., Grbovic, M., Radosavljević, V., & Bhamidipati, N. L. (2015) · International World Wide Web Conference (WWW) · doi:10.1145/2740908.2742760 · Cited by 725 · see citing works
TimeML: Robust Specification of Event and Temporal Expressions in Text
Pustejovsky, J. D., Castaño, J. M., Ingria, R. J. P., Sauri, R. S., Gaizauskas, R., Setzer, A., Katz, G., & Radev, D. (2003) · Cited by 707 · see citing works
Chen, T. Y., Kuo, F., Liu, H., Poon, P., Towey, D., Tse, T. H., & Zhou, Z. Q. (2018) · ACM Computing Surveys · doi:10.1145/3143561 · Cited by 691 · see citing works
Vanderwende, L., Daumé, H., & Kirchhoff, K. (2013) · North American Chapter of the Association for Computational Linguistics · Cited by 685 · see citing works
Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence
Chen, Y., Diethe, T., & Flach, P. A. (2016) · National Conference on Artificial Intelligence · Cited by 680 · see citing works
Detecting Offensive Language in Social Media to Protect Adolescent Online Safety
Chen, Y., Zhou, Y., Zhu, S., & Xu, H. (2012) · International Conference on Social Computing · doi:10.1109/socialcom-passat.2012.55 · Cited by 653 · see citing works
Ex Machina: Personal Attacks Seen at Scale
Wulczyn, E., Thain, N., & Dixon, L. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052591 · Cited by 636 · see citing works
Federated Machine Learning: Concept and Applications
Yang, Q., Liu, Y., Chen, T., & Tong, Y. (2019) · arXiv · doi:10.48550/arxiv.1902.04885 · Cited by 600 · see citing works
Are You a Racist or Am I Seeing Things? Annotator Influence on Hate Speech Detection on Twitter
Talat, Z. (2016) · Workshop on Natural Language Processing and Computational Social Science (NLP+CSS) · doi:10.18653/v1/w16-5618 · Cited by 585 · see citing works
A BERT-Based Transfer Learning Approach for Hate Speech Detection in Online Social Media
Mozafari, M., Farahbakhsh, R., & Crespi, N. (2020) · Studies in Computational Intelligence · doi:10.1007/978-3-030-36687-2_77 · Cited by 396 · see citing works
Recognizing Named Entities in Tweets
Liu, X., Zhang, S., Wei, F., & Zhou, M. (2011) · Cited by 388 · see citing works
Resources and benchmark corpora for hate speech detection: a systematic review
Poletto, F., Basile, V., Sanguinetti, M., Bosco, C., & Patti, V. (2020) · Language Resources and Evaluation · doi:10.1007/s10579-020-09502-8 · Cited by 387 · see citing works
One-step and Two-step Classification for Abusive Language Detection on Twitter
Park, J. H., & Fung, P. (2017) · doi:10.18653/v1/w17-3006 · Cited by 361 · see citing works
The hateful memes challenge: Detecting hate speech in multimodal memes
Kiela, D., Firooz, H., Mohan, A., Goswami, V., Singh, A., Ringshia, P., & Testuggine, D. (2020) · arXiv · doi:10.48550/arxiv.2005.04790 · Cited by 352 · see citing works
Mihalcea, R. F., & Strapparava, C. (2009) · doi:10.3115/1667583.1667679 · Cited by 341 · see citing works
Directions in abusive language training data, a systematic review: Garbage in, garbage out
Vidgen, B., & Derczynski, L. (2020) · PLoS ONE · doi:10.1371/journal.pone.0243300 · Cited by 335 · see citing works
Automated Hate Speech Detection and the Problem of Offensive LanguagePreprint
Davidson, T., Warmsley, D., Macy, M. W., & Weber, I. G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.04009 · Cited by 284 · see citing works
Geva, M., Goldberg, Y., & Berant, J. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1107 · Cited by 284 · see citing works
Overview of the HASOC track at FIRE 2019
Mandl, T., Modha, S., Majumder, P., Patel, D. D., Dave, M., Mandlia, C., & Patel, A. (2019) · Annual Meeting of the Forum for Information Retrieval Evaluation (FIRE) · doi:10.1145/3368567.3368584 · Cited by 281 · see citing works
UNITER: Learning UNiversal Image-TExt Representations
Chen, Y.-C., Li, L., Yu, L., Kholy, A. E., Ahmed, F., Gan, Z., Cheng, Y., & Liu, J. (2019) · Cited by 280 · see citing works
Learning Whom to Trust with MACE
Hovy, D., Berg-Kirkpatrick, T., Vaswani, A., & Hovy, E. H. (2013) · Archivio istituzionale della Ricerca - Bocconi · Cited by 276 · see citing works
Offensive Language Detection Using Multi-level Classification
Razavi, A. H., Inkpen, D. Z., Uritsky, S., & Matwin, S. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-13059-5_5 · Cited by 272 · see citing works
Detecting offensive tweets via topical feature discovery over a large scale twitter corpus
Xiang, G., Fan, B., Wang, L., Hong, J., & Rosé, C. P. (2012) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2396761.2398556 · Cited by 271 · see citing works
(2009) · doi:10.1093/acprof:oso/9780199548781.001.0001 · Cited by 266 · see citing works
Exploring Hate Speech Detection in Multimodal Publications
Gómez, R., Gibert, J., Gómez, L., & Karatzas, D. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093414 · Cited by 265 · see citing works
Multi-label Hate Speech and Abusive Language Detection in Indonesian Twitter
Ibrohim, M. O., & Budi, I. D. (2019) · doi:10.18653/v1/w19-3506 · Cited by 244 · see citing works
What is so special about online (as compared to offline) hate speech?
Brown, A. (2018) · Ethnicities · doi:10.1177/1468796817709846 · Cited by 232 · see citing works
Overview of the GermEval 2018 Shared Task on the Identification of Offensive Language
Wiegand, M., Siegel, M., & Ruppenhofer, J. (2019) · Publication Server of the Institute for German Language (Institute for German Language) · Cited by 231 · see citing works
Mooney, R. J. (2005) · Cited by 230 · see citing works
Hate Crime: Criminal Law and Identity Politics
Jacobs, J. B. (2002) · Theoretical Criminology · doi:10.1177/136248060200600406 · Cited by 229 · see citing works
Mihalcea, R. F., & Strapparava, C. (2005) · doi:10.3115/1220575.1220642 · Cited by 224 · see citing works
Detecting and Monitoring Hate Speech in Twitter
Pereira-Kohatsu, J. C., Quijano-Sánchez, L., Liberatore, F., & Camacho‐Collados, M. (2019) · Sensors · doi:10.3390/s19214654 · Cited by 220 · see citing works
Text Classification Using WordNet Hypernyms
Scott, S., & Matwin, S. (1998) · Cited by 209 · see citing works
L-HSAB: A Levantine Twitter Dataset for Hate Speech and Abusive Language
Mulki, H., Haddad, H., Ali, C. B., & Alshabani, H. (2019) · Workshop on Abusive Language Online (ALW) · doi:10.18653/v1/w19-3512 · Cited by 194 · see citing works
Ben-David, A., & Matamoros-Fernández, A. (2016) · International journal of communication · Cited by 188 · see citing works
Like Trainer, Like Bot? Inheritance of Bias in Algorithmic Content Moderation
Binns, R., Veale, M., Kleek, M. V., & Shadbolt, N. (2017) · Social Informatics · doi:10.1007/978-3-319-67256-4_32 · Cited by 182 · see citing works
An Italian Twitter Corpus of Hate Speech against Immigrants
Sanguinetti, M., Poletto, F., Bosco, C., Patti, V., & Stranisci, M. A. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/26vnss59hfcq · Cited by 176 · see citing works
The Offensive Internet: Speech, Privacy, and Reputation
Levmore, S., & Nussbaum, M. C. (2012) · doi:10.4159/9780674058767/html · Cited by 171 · see citing works
Deep Learning Models for Multilingual Hate Speech DetectionPreprint
Aluru, S. S., Mathew, B., Saha, P., & Mukherjee, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.06465 · Cited by 158 · see citing works
Overview of the task on automatic misogyny identification at IberEval 2018
Fersini, E., Rosso, P., & Anzovino, M. (2018) · BOA (University of Milano-Bicocca) · Cited by 148 · see citing works
Misogyny Detection in Twitter: a Multilingual and Cross-Domain Study
Pamungkas, E. W., Basile, V., & Patti, V. (2020) · Information Processing & Management · doi:10.1016/j.ipm.2020.102360 · Cited by 147 · see citing works
Content-driven detection of cyberbullying on the instagram social network
Zhong, H., Li, H., Squicciarini, A., Rajtmajer, S. M., Griffin, C. H., Miller, D. J., & Caragea, C. (2016) · International Joint Conference on Artificial Intelligence · Cited by 133 · see citing works
Detection of Cyberbullying Incidents on the Instagram Social Network
Hosseinmardi, H., Mattson, S. A., Rafiq, R. I., Han, R., Lv, Q., & Mishra, S. (2015) · arXiv · doi:10.48550/arxiv.1503.03909 · Cited by 126 · see citing works
Multimodal Meme Dataset (MultiOFF) for Identifying Offensive Content in Image and Text
Suryawanshi, S., Chakravarthi, B. R., Arčan, M., & Paul Buitelaar, C. (2020) · Arrow@dit (Dublin Institute of Technology) · doi:10.13025/21394 · Cited by 121 · see citing works
Compositionality Principle in Recognition of Fine-Grained Emotions from Text
Neviarouskaya, A., Prendinger, H., & Ishizuka, M. (2009) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v3i1.13987 · Cited by 116 · see citing works
Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-trainingPreprint
Li, G., Duan, N., Fang, Y., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.06066 · Cited by 115 · see citing works
Like Trainer, Like Bot? Inheritance of Bias in Algorithmic Content Moderation
Binns, R., Veale, M., Kleek, M. G. V., & Shadbolt, N. (2017) · UCL Discovery (University College London) · Cited by 113 · see citing works
Hate Speech in Constitutional Jurisprudence: A Comparative AnalysisPreprint
Rosenfeld, M. (2001) · SSRN Electronic Journal · doi:10.2139/ssrn.265939 · Cited by 107 · see citing works
Toward Multimodal Cyberbullying Detection
Singh, V. K., Ghosh, S., & Jose, C. (2017) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3027063.3053169 · Cited by 96 · see citing works
Deep learning for detecting inappropriate content in text
Yenala, H., Jhanwar, A., Chinnakotla, M. K., & Goyal, J. (2017) · International Journal of Data Science and Analytics · doi:10.1007/s41060-017-0088-4 · Cited by 96 · see citing works
Piasecki, M., Szpakowicz, S., & Broda, B. (2009) · Cited by 90 · see citing works
MultiFiT: Efficient Multi-lingual Language Model Fine-tuning
Eisenschlos, J. M., Ruder, S., Czapla, P., Kadras, M., Gugger, S., & Howard, J. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1572 · Cited by 89 · see citing works
Exploring Deep Multimodal Fusion of Text and Photo for Hate Speech Classification
Yang, F., Peng, X., Ghosh, G., Shilon, R., Ma, H., Moore, E., & Predović, G. (2019) · Workshop on Abusive Language Online (ALW) · doi:10.18653/v1/w19-3502 · Cited by 89 · see citing works
Identifying and Measuring Annotator Bias Based on Annotators’ Demographic Characteristics
Kuwatly, H. A., Wich, M., & Groh, G. (2020) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2020.alw-1.21 · Cited by 88 · see citing works
Proceedings of the NAACL Student Research Workshop
(2016) · doi:10.18653/v1/n16-2 · Cited by 86 · see citing works
Text Classification by Boosting Weak Learners based on Terms and Concepts
Bloehdorn, S., & Hotho, A. (2005) · Proceedings · doi:10.1109/icdm.2004.10077 · Cited by 84 · see citing works
Prediction of Cyberbullying Incidents on the Instagram Social NetworkPreprint
Hosseinmardi, H., Mattson, S. A., Rafiq, R. I., Han, R. Y., Lv, Q., & Mishra, S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1508.06257 · Cited by 81 · see citing works
Sohn, H., & Lee, H. (2019) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2019.00084 · Cited by 75 · see citing works
AMI @ EVALITA2020: Automatic Misogyny Identification
Fersini, E., Nozza, D., & Rosso, P. (2020) · Accademia University Press eBooks · doi:10.4000/books.aaccademia.6764 · Cited by 75 · see citing works
A Tiered CRF Tagger for Polish
Radziszewski, A. (2013) · Studies in computational intelligence · doi:10.1007/978-3-642-35647-6_16 · Cited by 72 · see citing works
Multi-Level Sentiment Analysis of PolEmo 2.0: Extended Corpus of Multi-Domain Consumer Reviews
Kocoń, J., Miłkowski, P., & Zaśko-Zielińska, M. (2019) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k19-1092 · Cited by 70 · see citing works
Self-citationNikolov-Radivchev at SemEval-2019 Task 6: Offensive Tweet Classification with BERT and Ensembles
Nikolov, A., & Radivchev, V. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2123 · Cited by 69 · see citing works
Álvarez‐Carmona, M. Á., Guzmán-Falcón, E., Montes-y-Gómez, M., Escalante, H. J., Villaseñor-Pineda, L., Reyes‐Meza, V., & Sulayes, A. R. (2018) · Cited by 68 · see citing works
Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech RecognitionPreprint
Huang, X., Xing, L., Dernoncourt, F., & Paul, M. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.10361 · Cited by 67 · see citing works
HaSpeeDe 2 @ EVALITA2020: Overview of the EVALITA 2020 Hate Speech Detection Task
Sanguinetti, M., Comandini, G., Nuovo, E. D., Frenda, S., Stranisci, M. A., Bosco, C., Caselli, T., Patti, V., & Russo, I. (2020) · Accademia University Press eBooks · doi:10.4000/books.aaccademia.6897 · Cited by 64 · see citing works
Proceedings of the First Workshop on Abusive Language Online
(2017) · doi:10.18653/v1/w17-30 · Cited by 63 · see citing works
Hate Speech in Constitutional Jurisprudence: A Comparative Analysis
Rosenfeld, M. (2003) · eYLS (Yale Law School) · Cited by 61 · see citing works
Encyclopedia of the American Constitution
(2001) · Choice Reviews Online · doi:10.5860/choice.38-3095 · Cited by 58 · see citing works
Hate Speech, Public Discourse, and the First Amendment
Heyman, S. J. (2009) · doi:10.1093/acprof:oso/9780199548781.003.0010 · Cited by 53 · see citing works
The FRENK Datasets of Socially Unacceptable Discourse in Slovene and English
Ljubešić, N., Fišer, D., & Erjavec, T. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-27947-9_9 · Cited by 53 · see citing works
Cyberbullying Detection With Fairness Constraints
Gencoglu, O. (2021) · IEEE Internet Computing · doi:10.1109/mic.2020.3032461 · Cited by 53 · see citing works
Modeling Annotator Perspective and Polarized Opinions to Improve Hate Speech Detection
Akhtar, S., Basile, V., & Patti, V. (2020) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v8i1.7473 · Cited by 52 · see citing works
Proceedings of the Third Workshop on Abusive Language Online
(2019) · doi:10.18653/v1/w19-35 · Cited by 52 · see citing works
Comprehensive analysis of aspect term extraction methods using various text embeddings
Augustyniak, Ł., Kajdanowicz, T. J., & Kazienko, P. (2021) · Computer Speech & Language · doi:10.1016/j.csl.2021.101217 · Cited by 50 · see citing works
Self-citationAutomatic Hate Speech Detection on Social Media: A Brief Survey
Alrehili, A. (2019) · IEEE/ACS International Conference on Computer Systems and Applications (AICCSA) · doi:10.1109/aiccsa47632.2019.9035228 · Cited by 48 · see citing works
Weakly Supervised Cyberbullying Detection Using Co-Trained Ensembles of Embedding Models
Raisi, E., & Huang, B. (2018) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1109/asonam.2018.8508240 · Cited by 46 · see citing works
Ranasinghe, T., Zampieri, M., & Hettiarachchi, H. (2019) · Lancaster EPrints (Lancaster University) · Cited by 46 · see citing works
Cross-lingual deep neural transfer learning in sentiment analysis
Kanclerz, K., Miłkowski, P., & Kocoń, J. (2020) · Procedia Computer Science · doi:10.1016/j.procs.2020.08.014 · Cited by 43 · see citing works
Self-citationAdvances in Artificial Intelligence
Sidorov, G., Reyes-Garcia, C. A., & Hernández Aguirre, A. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-16761-4 · Cited by 41 · see citing works
Liner2 – A Customizable Framework for Proper Names Recognition for Polish
Marcińczuk, M., Kocoń, J., & Janicki, M. (2013) · Studies in computational intelligence · doi:10.1007/978-3-642-35647-6_17 · Cited by 39 · see citing works
Self-citationMultimodal Classification for Analysing Social MediaPreprint
Duong, C. T., Lebret, R., & Aberer, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.02099 · Cited by 39 · see citing works
Hate Speech and Abusive Language Classification using fastText
Herwanto, G. B., Ningtyas, A. M., Nugraha, K. E., & Trisna, I. N. P. (2019) · International Seminar on Research of Information Technology and Intelligent Systems (ISRITI) · doi:10.1109/isriti48646.2019.9034560 · Cited by 38 · see citing works
Proceedings of the Fourth Workshop on Online Abuse and Harms
(2020) · doi:10.18653/v1/2020.alw-1 · Cited by 38 · see citing works
Encyclopedia of the American Constitution
(1997) · Choice Reviews Online · doi:10.5860/choice.34-4238 · Cited by 35 · see citing works
Proceedings of the First Workshop on NLP and Computational Social Science
Bamnan, D., Doğruöz, A. S., Eisenstein, J., Hovy, D., Jurgens, D., O'Connor, B., Oh, A., Tsur, O., & Volkova, S. (2016) · doi:10.18653/v1/w16-56 · Cited by 35 · see citing works
D’Mello, S. K. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2457413 · Cited by 34 · see citing works
Personal Bias in Prediction of Emotions Elicited by Textual Opinions
Miłkowski, P., Gruza, M., Kanclerz, K., Kazienko, P., Grimling, D., & Kocoń, J. (2021) · doi:10.18653/v1/2021.acl-srw.26 · Cited by 34 · see citing works
Self-citationInvestigating Annotator Bias with a Graph-Based Approach
Wich, M., Kuwatly, H. A., & Groh, G. (2020) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2020.alw-1.22 · Cited by 32 · see citing works
Controversy and Conformity: from Generalized to Personalized Aggressiveness Detection
Kanclerz, K., Figas, A., Gruza, M., Kajdanowicz, T. J., Kocoń, J., Puchalska, D., & Kazienko, P. (2021) · doi:10.18653/v1/2021.acl-long.460 · Cited by 32 · see citing works
Self-citationHate Speech Detection on Indonesian Instagram Comments using FastText Approach
Pratiwi, N. I., Budi, I. D., & Alfina, I. (2018) · International Conference on Advanced Computer Science and Information Systems · doi:10.1109/icacsis.2018.8618182 · Cited by 29 · see citing works
Evaluating Semantic Feature Representations to Efficiently Detect Hate Intent on Social Media
Senarath, Y., & Purohit, H. (2020) · IEEE International Conference on Semantic Computing (ICSC) · doi:10.1109/icsc.2020.00041 · Cited by 29 · see citing works
Aspect Detection using Word and Char Embeddings with (Bi) LSTM and CRF
Augustyniak, Ł., Kajdanowicz, T. J., & Kazienko, P. (2019) · IEEE International Conference on Artificial Intelligence and Knowledge Engineering (AIKE) · doi:10.1109/aike.2019.00016 · Cited by 27 · see citing works
Self-citationClassifier-based Polarity Propagation in a WordNet
Kocoń, J., Janz, A., & Piasecki, M. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2yoxmbu8cixn · Cited by 26 · see citing works
Self-citationHate Speech Detection Using Transformer Ensembles on the HASOC Dataset
Alonso, P., Saini, R., & Kovács, G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-60276-5_2 · Cited by 26 · see citing works
Context-sensitive Sentiment Propagation in WordNet
Kocoń, J., Janz, A., & Piasecki, M. (2018) · Global WordNet Conference (GWC) · doi:10.18653/v1/2018.gwc-1.39 · Cited by 25 · see citing works
Self-citationCharacterizing Variation in Toxic Language by Social Context
Radfar, B., Shivaram, K., & Culotta, A. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7366 · Cited by 23 · see citing works
Human Language Technologies as a Challenge for Computer Science and Linguistics – 2023
Mabokela, K. R., Schlippe, T., Raborife, M., & Çelik, T. (2023) · doi:10.14746/amup.9788323241775 · Cited by 23 · see citing works
Pérez‐Pérez, M., Glez‐Peña, D., Fdez‐Riverola, F., & Lourenço, A. M. G. (2014) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2014.11.005 · Cited by 22 · see citing works
Kocoń, J., & Maziarz, M. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102530 · Cited by 22 · see citing works
Self-citationLearning Representations for Detecting Abusive Language
Sahlgren, M., Isbister, T., & Olsson, F. (2018) · doi:10.18653/v1/w18-5115 · Cited by 20 · see citing works
Proceedings of the PolEval 2020 Workshop
Ogrodniczuk, M., & Kobyliński, Ł. (2020) · Cited by 20 · see citing works
Offensive Language Detection Explained
Risch, J., Ruff, R., & Krestel, R. (2020) · Cited by 19 · see citing works
Proceedings of the 11th Forum for Information Retrieval Evaluation
(2019) · doi:10.1145/3368567 · Cited by 18 · see citing works
WordNet2Vec: Corpora agnostic word vectorization method
Bartusiak, R., Augustyniak, Ł., Kajdanowicz, T. J., Kazienko, P., & Piasecki, M. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.01.121 · Cited by 18 · see citing works
Self-citationIbrahim, M., Torki, M., & El-Makky, N. (2020) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2020.semeval-1.248 · Cited by 18 · see citing works
Beyond the Transfer-and-Merge Wordnet Construction: plWordNet and a Comparison with WordNet
Maziarz, M., Piasecki, M., Rudnicka, E. K., & Śzpakowicz, S. (2013) · International conference Recent advances in natural language processing · Cited by 17 · see citing works
Aggression Identification in English, Hindi and Bangla Text using BERT, RoBERTa and SVM.
Baruah, A., Das, K., Barbhuiya, F. A., & Dey, K. (2020) · Cited by 17 · see citing works
Detecting Online Hate Speech: Approaches Using Weak Supervision and Network Embedding Models
Ridenhour, M., Bagavathi, A., Raisi, E., & Krishnan, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61255-9_20 · Cited by 15 · see citing works
Making Computers Laugh: Investigations in Automatic Humor Recognition
Mihalcea, R., & Strapparava, C. (2005) · University of North Texas Digital Library (University of North Texas) · Cited by 15 · see citing works
Liner2 — a Generic Framework for Named Entity Recognition
Marcińczuk, M., Kocoń, J., & Oleksy, M. (2017) · doi:10.18653/v1/w17-1413 · Cited by 13 · see citing works
Self-citationKocoń, J., & Janz, A. (2019) · Global WordNet Conference (GWC) · doi:10.18653/v1/2019.gwc-1.43 · Cited by 12 · see citing works
Self-citationTheNorth at SemEval-2020 Task 12: Hate Speech Detection Using RoBERTa
Alonso, P., Saini, R., & Kovács, G. (2020) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2020.semeval-1.292 · Cited by 11 · see citing works
Evaluating KGR10 Polish word embeddings in the recognition of temporal expressions using BiLSTM-CRFPreprint
Kocoń, J., & Gawor, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.04055 · Cited by 10 · see citing works
Self-citationIntelligent Tools for Building a Scientific Information Platform
Bembenik, R., Skonieczny, Ł., Rybiński, H., Kryszkiewicz, M., & Niezgódka, M. (2013) · Studies in computational intelligence · doi:10.1007/978-3-642-35647-6 · Cited by 9 · see citing works
Kocoń, J., & Marcińczuk, M. (2016) · Natural Language Engineering · doi:10.1017/s1351324916000255 · Cited by 8 · see citing works
Self-citationBarrière, V., & Balahur, A. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.23 · Cited by 7 · see citing works
Recognition of Genuine Polish Suicide Notes
Piasecki, M., Młynarczyk, K., & Kocoń, J. (2017) · doi:10.26615/978-954-452-049-6_076 · Cited by 6 · see citing works
Self-citationOn Sentiment Polarity Assignment in the Wordnet Using Loopy Belief Propagation
Kulisiewicz, M., Kajdanowicz, T. J., Kazienko, P., & Piasecki, M. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-19644-2_38 · Cited by 5 · see citing works
Self-citationHeterogeneous Named Entity Similarity Function
Kocoń, J., & Piasecki, M. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-32790-2_27 · Cited by 4 · see citing works
Self-citationKocoń, J., Bernaś, T., & Oleksy, M. (2019) · Poznań Studies in Contemporary Linguistics · doi:10.1515/psicl-2019-0011 · Cited by 4 · see citing works
Self-citationWikipedia Talk Labels: Aggression
Wulczyn, E., Thain, N., & Dixon, L. (2016) · Figshare · doi:10.6084/m9.figshare.4267550 · Cited by 4 · see citing works
Belief Propagation Method for Word Sentiment in WordNet 3.0
Misiaszek, A., Kazienko, P., Kulisiewicz, M., Augustyniak, Ł., Tuligłowicz, W., Popiel, A., & Kajdanowicz, T. J. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-05458-2_28 · Cited by 2 · see citing works
Self-citationBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
9 of the 173 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
ChatGPT: Jack of all trades, master of none
Kocoń, J., Cichecki, I., Kaszyca, O., Kochanek, M., Szydło, D., Baran, J., Bielaniewicz, J., Gruza, M., Janz, A., Kanclerz, K., et al. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101861 · Cited by 758 · see citing works
Self-citationTaxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764 · see citing works
“Mitigation is further complicated by the context dependency of what constitutes profoundly offensive speech [80, 92, 105].”
BackgroundGongane, V. U., Munot, M. V., & Anuse, A. D. (2022) · Social Network Analysis and Mining · doi:10.1007/s13278-022-00951-3 · Cited by 153 · see citing works
A Literature Review of Textual Hate Speech Detection Methods and Datasets
Alkomah, F., & Ma, X. (2022) · Information · doi:10.3390/info13060273 · Cited by 152 · see citing works
Subramanian, M., Easwaramoorthy, S. V., Deepalakshmi, G., Cho, J., & Manikandan, G. (2023) · Alexandria Engineering Journal · doi:10.1016/j.aej.2023.08.038 · Cited by 116 · see citing works
Toward a Perspectivist Turn in Ground Truthing for Predictive Computing
Cabitza, F., Campagner, A., & Basile, V. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i6.25840 · Cited by 114 · see citing works
Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation
Goyal, N., Kivlichan, I. D., Rosen, R., & Vasserman, L. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555088 · Cited by 76 · see citing works
EnsMulHateCyb: Multilingual hate speech and cyberbully detection in online social media
Mahajan, E., Mahajan, H., & Kumar, S. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121228 · Cited by 55 · see citing works
Human-centered neural reasoning for subjective content processing: Hate speech, emotions, and humor
Kazienko, P., Bielaniewicz, J., Gruza, M., Kanclerz, K., Karanowski, K., Miłkowski, P., & Kocoń, J. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.01.010 · Cited by 38 · see citing works
Self-citationKocoń, J., Gruza, M., Bielaniewicz, J., Grimling, D., Kanclerz, K., Miłkowski, P., & Kazienko, P. (2021) · Proceedings · doi:10.1109/icdm51629.2021.00140 · Cited by 38 · see citing works
Self-citationDeep learning approach to detect cyberbullying on twitter
Oğuz, Ç., & Yağanoğlu, M. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19869-3 · Cited by 38 · see citing works
Personal Bias in Prediction of Emotions Elicited by Textual Opinions
Miłkowski, P., Gruza, M., Kanclerz, K., Kazienko, P., Grimling, D., & Kocoń, J. (2021) · doi:10.18653/v1/2021.acl-srw.26 · Cited by 34 · see citing works
Self-citationDetecting abusive comments at a fine-grained level in a low-resource language
Chakravarthi, B. R., Priyadharshini, R., Banerjee, S., Jagadeeshan, M. B., Kumaresan, P. K., Ponnusamy, R., Benhur, S., & Mccrae, J. (2023) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100006 · Cited by 49 · see citing works
Saifullah, S., Dreżewski, R., Dwiyanto, F. A., Aribowo, A. S., Fauziah, Y., & Cahyana, N. H. (2024) · Applied Sciences · doi:10.3390/app14031078 · Cited by 32 · see citing works
Controversy and Conformity: from Generalized to Personalized Aggressiveness Detection
Kanclerz, K., Figas, A., Gruza, M., Kajdanowicz, T. J., Kocoń, J., Puchalska, D., & Kazienko, P. (2021) · doi:10.18653/v1/2021.acl-long.460 · Cited by 32 · see citing works
Self-citationPredicting the Hate: A GSTM Model based on COVID-19 Hate Speech Datasets
Wu, X.-K., Zhao, T.-F., Lu, L., & Chen, W. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.102998 · Cited by 31 · see citing works
Why Don’t You Do It Right? Analysing Annotators’ Disagreement in Subjective Tasks
Sandri, M., Leonardelli, E., Tonelli, S., & Jezek, E. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.178 · Cited by 31 · see citing works
Personalized Large Language Models
Woźniak, S., Koptyra, B., Janz, A., Kazienko, P., & Kocoń, J. (2024) · 2024 IEEE International Conference on Data Mining Workshops (ICDMW) · doi:10.1109/icdmw65004.2024.00071 · Cited by 28 · see citing works
Self-citationPerspectivist approaches to natural language processing: a survey
Frenda, S., Abercrombie, G., Basile, V., Pedrani, A., Panizzon, R., Cignarella, A. T., Marco, C. S., & Bernardi, D. (2024) · Language Resources and Evaluation · doi:10.1007/s10579-024-09766-4 · Cited by 27 · see citing works
The language and targets of online trolling: A psycholinguistic approach for social cybersecurity
Uyheng, J., Moffitt, J. D., & Carley, K. M. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103012 · Cited by 26 · see citing works
Recognizing misogynous memes: Biased models and tricky archetypes
Rizzi, G., Gasparini, F., Saibene, A., Rosso, P., & Fersini, E. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103474 · Cited by 28 · see citing works
Hate speech detection in the Bengali language: a comprehensive survey
Maruf, A. A., Abidin, A. J., Haque, M. M., Jiyad, Z. M., Golder, A., Alubady, R., & Aung, Z. (2024) · Journal Of Big Data · doi:10.1186/s40537-024-00956-z · Cited by 27 · see citing works
Yin, W., Agarwal, V., Jiang, A., Zubiaga, A., & Sastry, N. (2023) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v17i1.22198 · Cited by 24 · see citing works
Multitask Personalized Recognition of Emotions Evoked by Textual Content
Miłkowski, P., Saganowski, S., Gruza, M., Kazienko, P., Piasecki, M., & Kocoń, J. (2022) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomworkshops53856.2022.9767502 · Cited by 20 · see citing works
Self-citationWoźniak, S., & Kocoń, J. (2023) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw60847.2023.00108 · Cited by 20 · see citing works
Self-citationMireshghallah, F., Shrivastava, V., Shokouhi, M., Berg-Kirkpatrick, T., Sim, R., & Dimitriadis, D. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.252 · Cited by 20 · see citing works
Şenel, L. K., Şahi̇nuç, F., Yücesoy, V., Schütze, H., Çukur, T., & Koç, A. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.102925 · Cited by 20 · see citing works
Tamil Offensive Language Detection: Supervised versus Unsupervised Learning Approaches
Balakrishnan, V., Govindan, V., & Govaichelvan, K. N. (2022) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3575860 · Cited by 16 · see citing works
Pathos in Natural Language Argumentation: Emotional Appeals and Reactions
Konat, B., Gajewska, E., & Rossa, W. (2024) · Argumentation · doi:10.1007/s10503-024-09631-2 · Cited by 21 · see citing works
Sang, Y., & Stanton, J. M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-96957-8_36 · Cited by 15 · see citing works
Kocoń, J., Baran, J., Kanclerz, K., Kajstura, M., & Kazienko, P. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-35995-8_11 · Cited by 14 · see citing works
Self-citationKar, P., & Debbarma, S. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107143 · Cited by 13 · see citing works
A web framework for information aggregation and management of multilingual hate speech
Kotsakis, R., Vrysis, L., Vryzas, N., Saridou, T. A., Matsiola, M., Veglis, A. A., & Dimoulas, C. A. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e16084 · Cited by 13 · see citing works
Towards a systematic approach to manual annotation of code smells
Сливка, Ј., Luburić, N., Prokić, S., Grujić, K.-G., Kovačević, A., Sladić, G., & Vidaković, D. (2023) · Science of Computer Programming · doi:10.1016/j.scico.2023.102999 · Cited by 13 · see citing works
Ngo, A., & Kocoń, J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102966 · Cited by 12 · see citing works
Self-citationAddressing Challenges in Hate Speech Detection Using BERT-Based Models: A Review
Aljawazeri, J. A., & Jasim, M. N. (2024) · Iraqi Journal for Computer Science and Mathematics · doi:10.52866/ijcsm.2024.05.02.001 · Cited by 11 · see citing works
Hettiachchi, D., Holcombe-James, I., Livingstone, S., de Silva, A., Lease, M., Salim, F. D., & Sanderson, M. (2023) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v11i1.27546 · Cited by 11 · see citing works
Deep-SHEEP: Sense of Humor Extraction from Embeddings in the Personalized Context
Bielaniewicz, J., Kanclerz, K., Miłkowski, P., Gruza, M., Karanowski, K., Kazienko, P., & Kocoń, J. (2022) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw58026.2022.00125 · Cited by 10 · see citing works
Self-citationBilingual hate speech detection on social media: Amharic and Afaan Oromo
Ababu, T. M., Woldeyohannis, M. M., & Getaneh, E. B. (2025) · Journal Of Big Data · doi:10.1186/s40537-024-01044-y · Cited by 10 · see citing works
Modeling Uncertainty in Personalized Emotion Prediction with Normalizing Flows
Miłkowski, P., Karanowski, K., Wielopolski, P., Kocoń, J., Kazienko, P., & Zięba, M. (2023) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw60847.2023.00103 · Cited by 10 · see citing works
Self-citationWanniarachchi, V. U., Scogings, C. J., Sušnjak, T., & Mathrani, A. (2023) · AJIS. Australasian journal of information systems/AJIS. Australian journal of information systems/Australian journal of information systems · doi:10.3127/ajis.v27i0.3929 · Cited by 9 · see citing works
AspectEmo: Multi-Domain Corpus of Consumer Reviews for Aspect-Based Sentiment Analysis
Kocoń, J., Radom, J., Kaczmarz-Wawryk, E., Wabnic, K., Zajaczkowska, A., & Zaśko-Zielińska, M. (2021) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw53433.2021.00027 · Cited by 9 · see citing works
Self-citationFederated Learning for Exploiting Annotators’ Disagreements in Natural Language Processing
Rodríguez-Barroso, N., Martínez‐Cámara, E., Camacho-Collados, J., Luzón, M. V., & Herrera, F. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00664 · Cited by 8 · see citing works
Dynamics of hate speech in social media: insights from Indonesia
Margono, H., Saud, M., & Ashfaq, A. (2024) · Global Knowledge, Memory and Communication · doi:10.1108/gkmc-11-2023-0464 · Cited by 19 · see citing works
Sentir el odio: análisis de la gravedad percibida de los discursos de odio en la población española
Brändle, G., Zapatero, M. D. C., & Rebollo, M. A. P. (2024) · Revista Española de Sociología · doi:10.22325/fes/res.2024.219 · Cited by 8 · see citing works
Prokopowicz, D. (2023) · International Journal of Legal Studies ( IJOLS ) · doi:10.5604/01.3001.0054.2699 · Cited by 8 · see citing works
Ünlü, A., Truong, S., Sawhney, N., Tammi, T., & Kotonen, T. (2025) · New Media & Society · doi:10.1177/14614448241312900 · Cited by 7 · see citing works
Confront hate with AI: how AI-generated counter speech helps against hate speech on social Media?
Wu, C., Wang, Y., Zhang, Y., Wang, H., & Pang, Y. (2025) · Telematics and Informatics · doi:10.1016/j.tele.2025.102304 · Cited by 9 · see citing works
Fortifying NLP models against poisoning attacks: The power of personalized prediction architectures
Ferdinan, T., & Kocoń, J. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102692 · Cited by 7 · see citing works
Self-citationSengupta, K., & Srivast, P. R. (2023) · Research Square · doi:10.21203/rs.3.rs-2509731/v1 · Cited by 7 · see citing works
Kirchmair, T., Koban, K., & Matthes, J. (2024) · Communications · doi:10.1515/commun-2023-0133 · Cited by 6 · see citing works
Kar, P., & Debbarma, S. (2023) · The Journal of Supercomputing · doi:10.1007/s11227-023-05361-6 · Cited by 6 · see citing works
RetractedYafooz, W. M. S., & Alsaeedi, A. (2024) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2024.048270 · Cited by 6 · see citing works
MultiEmo: Language-Agnostic Sentiment Analysis
Miłkowski, P., Gruza, M., Kazienko, P., Szołomicka, J., Woźniak, S., & Kocoń, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-08754-7_10 · Cited by 5 · see citing works
Self-citationKocoń, J., Miłkowski, P., Wierzba, M., Konat, B., Klessa, K., Janz, A., Riegel, M., Juszczyk, K., Grimling, D., Marchewka, A., et al. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-05328-3_14 · Cited by 5 · see citing works
Self-citationTowards Model-Based Data Acquisition for Subjective Multi-Task NLP Problems
Kanclerz, K., Bielaniewicz, J., Gruza, M., Kocoń, J., Woźniak, S., & Kazienko, P. (2023) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw60847.2023.00099 · Cited by 5 · see citing works
Self-citationÜnlü, A., Truong, S., & Kotonen, T. (2024) · International Journal of Data Science and Analytics · doi:10.1007/s41060-024-00571-4 · Cited by 5 · see citing works
Improving LLM-Based Recommender Systems with User-Controllable Profiles
Woźniak, S., Duszenko, J., Kocoń, J., & Kazienko, P. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3701716.3717734 · Cited by 5 · see citing works
Self-citationHidden Hate: Analysis of Hate Speech on a Darknet Forum
Logie, K., Cohen, N. D., Taylor, E., & Perry, K. (2025) · Justice Quarterly · doi:10.1080/07418825.2025.2501544 · Cited by 4 · see citing works
Architectural Concepts for Integrating Fundamental Drives and Emotions Into Artificial Intelligence
Ferdinan, T., Mieleszczenko-Kowszewicz, W., Kocoń, J., & Kazienko, P. (2025) · IEEE Intelligent Systems · doi:10.1109/mis.2025.3622004 · Cited by 4 · see citing works
Self-citationDeep Neural Language-agnostic Multi-task Text Classifier
Gawron, K., Pogoda, M., Ropiak, N., Swędrowski, M., & Kocoń, J. (2021) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw53433.2021.00023 · Cited by 4 · see citing works
Self-citationMulti-model Analysis of Language-Agnostic Sentiment Classification on MultiEmo Data
Miłkowski, P., Gruza, M., Kazienko, P., Szołomicka, J., Woźniak, S., & Kocoń, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16014-1_14 · Cited by 4 · see citing works
Self-citationCima, L., Tessa, B., Trujillo, A., Cresci, S., & Avvenuti, M. (2025) · Online Social Networks and Media · doi:10.1016/j.osnem.2025.100320 · Cited by 5 · see citing works
Language-based machine perception: linguistic perspectives on the compilation of captioning datasets
Hekanaho, L., Hirvonen, M., & Virtanen, T. I. (2024) · Digital Scholarship in the Humanities · doi:10.1093/llc/fqae029 · Cited by 3 · see citing works
See the words through my eyes: The role of personal traits in abusive language detection
Yao, T., Binnewies, S., Foo, E., & Alavi, M. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127188 · Cited by 3 · see citing works
CLARIN-PL: a user centred language technology infrastructure
Piasecki, M., Dziob, A., Janz, A., Kocoń, J., Naskręt, T., Oleksy, M., Rudnicka, E. K., Walkowiak, T., Wieczorek, J., & Hwaszcz, K. (2025) · Language Resources and Evaluation · doi:10.1007/s10579-025-09839-y · Cited by 4 · see citing works
Self-citationToxic Text in Personas: An Experiment on User Perceptions
Salminen, J. O., Jung, S., Santos, J. M., de Lisboa, I. U., & Jansen, B. J. (2021) · AIS Transactions on Human-Computer Interaction · doi:10.17705/1thci.00157 · Cited by 3 · see citing works
Confidence-based Ensembling of Perspective-aware Models
Casola, S., Lo, S. M., Basile, V., Frenda, S., Cignarella, A. T., Patti, V., & Bosco, C. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.212 · Cited by 4 · see citing works
Zhao, Z., Zhang, Z., & Hopfgartner, F. (2021) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · doi:10.48550/arxiv.2109.02691 · Cited by 3 · see citing works
Personalization of Affective State Recognition from Physiological Signals: A Review
Perz, B., & Kazienko, P. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-5881-7_12 · Cited by 2 · see citing works
Self-citationExplainable AI: Transformer-based Bangla Hate Speech Detection
Sultana, R., Tabassum, N., & Arefin, M. S. (2024) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit64611.2024.11022118 · Cited by 2 · see citing works
Tempska, P., Urbaniak, R., Dowgiałło, M., Ptaszynski, M. E., Zajączkowski, A., Milewska, M., Leliwa, G., Marcińczuk, M., Brochocki, M., & Wroczyński, M. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104316 · Cited by 2 · see citing works
Supporting Online Toxicity Detection with Knowledge Graphs
Lobo, P. R., Daga, E., & Alani, H. (2022) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v16i1.19398 · Cited by 2 · see citing works
Palazón-Fernández, J. L., Mata-Pérez, C., Gilart, E., Cotobal-Calvo, E. M., Cruz-Barrientos, A., & Bocchino, A. (2023) · Healthcare · doi:10.3390/healthcare11111564 · Cited by 2 · see citing works
Hate Speech in the Comments’ Column Instagram: A Discourse Analysis
Mubarok, Y., Sudana, D., & Gunawan, W. (2024) · Journal of Languages and Language Teaching · doi:10.33394/jollt.v12i1.9050 · Cited by 2 · see citing works
Model and evaluation: towards fairness in multilingual text classification
Lin, N., He, J., Tang, Z., Fang, J., Zhou, D., & Yang, A. (2026) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-025-02824-5 · Cited by 4 · see citing works
Kang, R., & Rau, P. P. (2024) · Interacting with Computers · doi:10.1093/iwc/iwae034 · Cited by 1 · see citing works
Pihulski, D., & Kocoń, J. (2025) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw69685.2025.00261 · Cited by 1 · see citing works
Self-citationTroll Meme Classification for Tamil Memes Using Convolutional Neural Network
Nithya, K., Sathyapriya, S., Sulochana, M., Thaarini, S., & Dhivyaa, C. R. (2023) · Applied and Computational Engineering · doi:10.54254/2755-2721/2/20220636 · Cited by 1 · see citing works
Bielaniewicz, J., & Kazienko, P. (2023) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw60847.2023.00098 · Cited by 1 · see citing works
Self-citationDiscurso de ódio na Espanha. Pesquisa sobre a percepção e a experiência da população
Barbosa, M. P. G., Blanco‐Herrero, D., Calderón, C. A., & Holgado, P. S. (2024) · Observatorio (OBS*) · doi:10.15847/obsobs18220242304 · Cited by 1 · see citing works
Understanding the low inter-rater agreement on aggressiveness on the Linux Kernel Mailing List
Bock, T., Schneider, N., Schmid, A., Apel, S., & Siegmund, J. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112339 · Cited by 1 · see citing works
PCROD: Context Aware Role based Offensive Detection using NLP/ DL Approaches
Purnima, T., & Ch, K. (2023) · International Journal on Recent and Innovation Trends in Computing and Communication · doi:10.17762/ijritcc.v11i1.5981 · Cited by 1 · see citing works
eevvgg at SemEval-2023 Task 11: Offensive Language Classification with Rater-based Information
Gajewska, E. (2023) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2023.semeval-1.24 · Cited by 1 · see citing works
Is it possible for ChatGPT to mimic human annotator?Preprint
Kaszyca, O., Kazienko, P., Kocoń, J., Cichecki, I., Kochanek, M., & Szydło, D. (2023) · doi:10.36227/techrxiv.170259175.55207879/v1 · Cited by 1 · see citing works
Self-citationBroadening the ontology of threats verbalized in written communication on the Internet
Komalova, L., & Kalyuzhnaya, L. V. (2022) · Russian Journal of Linguistics · doi:10.22363/2687-0088-29939
Playing HAVOK on the Chaos Caused by Internet TrollsPreprint
Martynova, E., Golino, H. F., & Boker, S. M. (2023) · Research Square · doi:10.21203/rs.3.rs-2843058/v1
Astuti, V. P. M., & Haryanto, S. (2023) · Advances in Social Science, Education and Humanities Research/Advances in social science, education and humanities research · doi:10.2991/978-2-38476-086-2_186
Obare, S., Ade-Ibijola, A., Okeyo, G., & Ogada, K. (2023) · International Journal of Data Science and Big Data Analytics · doi:10.51483/ijdsbda.3.1.2023.25-44
Improved Hierarchical Attention Networks for Cyberbullying Detection via Social Media Data
Qian, W., Yu, S., Nie, Z., Lu, X. S., Liu, H., & Huang, B. (2023) · IEEE International Conference on Networking, Sensing and Control (ICNSC) · doi:10.1109/icnsc58704.2023.10319023
See the Words Through My Eyes: The Role of Personality Traits in Abusive Language DetectionPreprint
Yao, T., Binnewies, S., Foo, E., & Alavi, M. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4880454
Discursos contrarios a la educación sexual en España
González, S. C., & Barbadillo, G. T. (2024) · Revista ICONO14 · doi:10.7195/ri14.v22i2.2086
Exploring the Potential of an AI-Based Twitch Moderation and Toxicity Detection Bot
Huth, J., Eichhorn, C., Plecher, D. A., & Pirker, J. (2025) · IEEE Conference on Games (CoG) · doi:10.1109/cog64752.2025.11114410
Annotated Slovak Datasets for Toxicity, Hate Speech, and Sentiment Analysis
Sokolová, Z., Harahus, M., Hládek, D., & Staš, J. (2025) · Journal of Linguistics/Jazykovedný casopis · doi:10.2478/jazcas-2025-0025
Baider, F. H., Szczepaniak-Kozak, A., Strani, K., & Jaszczyk-Grzyb, M. (2025) · Applied Linguistics Review · doi:10.1515/applirev-2024-0129
Semantic Aggregated Adversarial Training Framework for Hate Speech Detection
Zhang, X., Tian, H., Zheng, X. B., Peng, J., & Zeng, D. D. (2026) · INFORMS journal on computing · doi:10.1287/ijoc.2023.0508
Kosman, M. (2026) · East European Politics · doi:10.1080/21599165.2026.2666801
Nadeem, M., & Hongsong, C. (2026) · Computer Science Review · doi:10.1016/j.cosrev.2026.101009
Sustainable AI solutions: A review of green technology for hate speech detection in vídeos
Setyawan, R. A., Surjono, H. D., & Wardani, R. (2026) · Multidisciplinary Reviews · doi:10.31893/multirev.2027014
Canbay, P. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.105126
- Authors
- Kocoń, J., Figas, A., Gruza, M., Puchalska, D., Kajdanowicz, T., & Kazienko, P.
- Affiliations
- Wrocław University of Science and Technology (PL)
- Corresponding author
- Kocoń, Jan
- Venue
- Information Processing & Management
- Publisher
- Elsevier BV
- Published
- 2021-09
- Volume
- 58
- Issue
- 5
- Pages
- 102643
- ISSN
- 0020-0271 (ISSN-L)
- 0306-4573
- OpenAlex
- W3171716568
- Cited by
- 103
- References
- 173
- Citations by year
- 2026: 9 · 2025: 17 · 2024: 21 · 2023: 32 · 2022: 17 · 2021: 7
- FWCI
- 6.81
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Information Processing & Management (publishedVersion, cc-by)
- Funders
- Narodowe Centrum Nauki
- Wroclaw University of Science and Technology
- Polish Ministry of Science and Higher Education
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Text and Document Classification Technologies
- Keywords
- offensive content detection, hate speech, annotation agreement, annotator demographics, personalized classification, individualized models, multimodal models, transformer models, human-centered AI, annotation guidelines
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
