Cyberbullying Detection: Hybrid Models Based on Machine Learning and Natural Language Processing Techniques
Open accessAbstract
The rise in web and social media interactions has resulted in the efortless proliferation of offensive language and hate speech. Such online harassment, insults, and attacks are commonly termed cyberbullying. The sheer volume of user-generated content has made it challenging to identify such illicit content. Machine learning has wide applications in text classification, and researchers are shifting towards using deep neural networks in detecting cyberbullying due to the several advantages they have over traditional machine learning algorithms. This paper proposes a novel neural network framework with parameter optimization and an algorithmic comparative study of eleven classification methods: four traditional machine learning and seven shallow neural networks on two real world cyberbullying datasets. In addition, this paper also examines the effect of feature extraction and word-embedding-techniques-based natural language processing on algorithmic performance. Key observations from this study show that bidirectional neural networks and attention models provide high classification results. Logistic Regression was observed to be the best among the traditional machine learning classifiers used. Term Frequency-Inverse Document Frequency (TF-IDF) demonstrates consistently high accuracies with traditional machine learning techniques. Global Vectors (GloVe) perform better with neural network models. Bi-GRU and Bi-LSTM worked best amongst the neural networks used. The extensive experiments performed on the two datasets establish the importance of this work by comparing eleven classification methods and seven feature extraction techniques. Our proposed shallow neural networks outperform existing state-of-the-art approaches for cyberbullying detection, with accuracy and F1-scores as high as ~95% and ~98%, respectively.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Raj et al. [149] achieved best accuracy and F1 scores of 96.5% and 98.69% using Bi-LSTM over Wikipedia Web Toxicity Dataset.
“On the other hand, they achieved the best accuracy and F1 scores of 96.5% and 98.69%, respectively, using Bi-LSTM over the Wikipedia Web Toxicity Dataset.”
From A Review on Deep-Learning-Based Cyberbullying Detection (2023) · cites this paper as [149]
Table 2 (section 5; page unavailable) reports 96.53% accuracy and 98.69% F1 on the Wikipedia Web Toxicity Dataset, but assigns that pair to CNN with GloVe, not Bi-LSTM. Section 4.2 describes 98.69% as the highest F1 in its discussion of Bi-LSTM models, so the prose and table are not fully consistent; neither supports the claimed accuracy–F1 pair for Bi-LSTM. (LLM, conf=0.85)
Experimental — accuracy not yet measuredRaj et al. [149] obtained best accuracy and F1 scores of 96.98% and 98.56% using Bi-GRU on Wikipedia Attack Dataset.
“They obtained the best accuracy and F1 scores of 96.98% and 98.56%, respectively, by using Bi-GRU on the Wikipedia Attack Dataset.”
From A Review on Deep-Learning-Based Cyberbullying Detection (2023) · cites this paper as [149]
Several deep-learning-based cyberbullying detection methods have been conducted using individual models and presenting performance with popular evaluation metrics.
“Several deep-learning-based cyberbullying detection methods have been conducted by using individual models and presenting the performance of the models with popular evaluation metrics [21,134,149,150].”
From A Review on Deep-Learning-Based Cyberbullying Detection (2023) · cites this paper as [149]
Raj et al. [149] found that Bi-LSTM and Bi-GRU outperform other deep-learning-based models on accuracy and F1 score for cyberbullying classification.
“The experimental results show that Bi-LSTM and Bi-GRU outperform other deep-learning-based models in this context on accuracy and F1 score.”
From A Review on Deep-Learning-Based Cyberbullying Detection (2023) · cites this paper as [149]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569 · see citing works
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · see citing works
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · see citing works
Convolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831 · see citing works
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172 · see citing works
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645 · see citing works
A vector space model for automatic indexing
Salton, G., Wong, A., & Yang, C. S. (1975) · Communications of the ACM · doi:10.1145/361219.361220 · Cited by 7,521 · see citing works
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545 · see citing works
A Convolutional Neural Network for Modelling Sentences
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1062 · Cited by 3,601 · see citing works
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.1078 · Cited by 3,317 · see citing works
The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions
Hochreiter, S. (1998) · International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems · doi:10.1142/s0218488598000094 · Cited by 2,939 · see citing works
xgboost: Extreme Gradient Boosting
Chen, T., He, T., Benesty, M., Khotilovich, V., Tang, Y., Cho, H., Chen, K., Mitchell, R., Cano, I., Zhou, T., et al. (2014) · doi:10.32614/cran.package.xgboost · Cited by 2,735 · see citing works
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610 · see citing works
Tokunaga, R. S. (2010) · Computers in Human Behavior · doi:10.1016/j.chb.2009.11.014 · Cited by 2,566 · see citing works
Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter
Talat, Z., & Hovy, D. (2016) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n16-2013 · Cited by 1,919 · see citing works
A Survey on Hate Speech Detection using Natural Language Processing
Schmidt, A. G., & Wiegand, M. (2017) · International Workshop on Natural Language Processing for Social Media (SocialNLP) · doi:10.18653/v1/w17-1101 · Cited by 1,447 · see citing works
Deep Learning for Hate Speech Detection in Tweets
Badjatiya, P., Gupta, S., Gupta, M., & Varma, V. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3041021.3054223 · Cited by 1,253 · see citing works
Abusive Language Detection in Online User Content
Nobata, C., Tetreault, J., Thomas, A. O., Mehdad, Y., & Chang, Y. (2016) · International World Wide Web Conference (WWW) · doi:10.1145/2872427.2883062 · Cited by 1,160 · see citing works
Text Mining: Use of TF-IDF to Examine the Relevance of Words to Documents
Qaiser, S., & Ali, R. (2018) · International Journal of Computer Applications · doi:10.5120/ijca2018917395 · Cited by 833 · see citing works
Hate Speech Detection with Comment Embeddings
Djuric, N., Zhou, J. L., Morris, R., Grbovic, M., Radosavljević, V., & Bhamidipati, N. L. (2015) · International World Wide Web Conference (WWW) · doi:10.1145/2740908.2742760 · Cited by 725 · see citing works
Ex Machina: Personal Attacks Seen at Scale
Wulczyn, E., Thain, N., & Dixon, L. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052591 · Cited by 636 · see citing works
A Brief Survey of Text Mining: Classification, Clustering and Extraction TechniquesPreprint
Allahyari, M., Pouriyeh, S., Assefi, M., Safaei, S., Trippe, E. D., Gutiérrez, J. B., & Kochut, K. (2017) · arXiv.org · doi:10.48550/arxiv.1707.02919 · Cited by 582 · see citing works
Using Machine Learning to Detect Cyberbullying
Reynolds, K., Kontostathis, A., & Edwards, L. (2011) · doi:10.1109/icmla.2011.152 · Cited by 462 · see citing works
Locate the Hate: Detecting Tweets against Blacks
Kwok, I., & Wang, Y. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v27i1.8539 · Cited by 452 · see citing works
Deep Learning for Detecting Cyberbullying Across Multiple Social Media Platforms
Agrawal, S., & Awekar, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-76941-7_11 · Cited by 387 · see citing works
Counter-fitting Word Vectors to Linguistic Constraints
Mrkšić, N., Séaghdha, D. Ó., Thomson, B., Gašić, M., Rojas-Barahona, L. M., Su, P.-H., Vandyke, D., Wen, T.-H., & Young, S. J. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1018 · Cited by 356 · see citing works
From Paraphrase Database to Compositional Paraphrase Model and Back
Wieting, J., Bansal, M., Gimpel, K., & Livescu, K. (2015) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00143 · Cited by 262 · see citing works
A Comparative Analysis of Machine Learning Techniques for Cyberbullying Detection on Twitter
Muneer, A., & Fati, S. M. (2020) · Future Internet · doi:10.3390/fi12110187 · Cited by 211 · see citing works
Mrkšić, N., Vulić, I., Séaghdha, D. Ó., Leviant, I., Reichart, R., Gašić, M., Korhonen, A., & Young, S. J. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00063 · Cited by 170 · see citing works
A review on feature selection and feature extraction for text classification
Shah, F., & Patel, V. D. (2016) · International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET) · doi:10.1109/wispnet.2016.7566545 · Cited by 139 · see citing works
Deep Learning for User Comment Moderation
Pavlopoulos, J., Malakasiotis, P., & Androutsopoulos, I. (2017) · Workshop on Abusive Language Online (ALW) · doi:10.18653/v1/w17-3004 · Cited by 124 · see citing works
Cyberbullying Detection with a Pronunciation Based Convolutional Neural Network
Zhang, X., Tong, J., Vishwamitra, N., Whittaker, E., Mazer, J. P., Kowalski, R., Hu, H., Luo, F., Macbeth, J., & Dillon, E. (2016) · 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA) · doi:10.1109/icmla.2016.0132 · Cited by 118 · cited 1× · see citing works
“Although they are primarily employed in visual tasks, CNNs have displayed tremendous performance in one-dimensional text classification [13,14].”
Aroyehun, S. T., & Gelbukh, A. F. (2018) · International Conference on Computational Linguistics · Cited by 106 · see citing works
Detection of Cyberbullying Using Deep Neural Network
Banerjee, V., Telavane, J., Gaikwad, P., & Vartak, P. (2019) · 2019 5th International Conference on Advanced Computing & Communication Systems (ICACCS) · doi:10.1109/icaccs.2019.8728378 · Cited by 98 · see citing works
Text Classification using Different Feature Extraction Approaches
Dzisevic, R., & Šešok, D. (2019) · IEEE Open Conference of Electrical, Electronic and Information Sciences (eStream) · doi:10.1109/estream.2019.8732167 · Cited by 83 · see citing works
Lu, N., Wu, G., Zhang, Z., Zheng, Y., Ren, Y., & Choo, K. R. (2020) · Concurrency and Computation: Practice and Experience · doi:10.1002/cpe.5627 · Cited by 76 · see citing works
Classification of Abusive Comments in Social Media using Deep Learning
Anand, M., & Eswari, R. (2019) · International Conference on Computing Methodologies and Communication (ICCMC) · doi:10.1109/iccmc.2019.8819734 · Cited by 70 · see citing works
On Pairwise Naive Bayes Classifiers
Sulzmann, J.-N., Fürnkranz, J., & Hüllermeier, E. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74958-5_35 · Cited by 43 · see citing works
Morph-fitting: Fine-Tuning Word Vector Spaces with Simple Language-Specific Rules
Vulić, I., Mrkšić, N., Reichart, R., Séaghdha, D. Ó., Young, S. J., & Korhonen, A. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1006 · Cited by 38 · see citing works
Neural Word Decomposition Models for Abusive Language Detection
Bodapati, S., Gella, S., Bhattacharjee, K., & Al-Onaizan, Y. (2019) · Workshop on Abusive Language Online (ALW) · doi:10.18653/v1/w19-3515 · Cited by 36 · see citing works
Bourgonje, P., Schneider, J. M., Srivastava, A., et al. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-73706-5_15 · Cited by 35 · see citing works
A Hybrid Deep Learning System of CNN and LRCN to Detect Cyberbullying from SNS Comments
Bu, S.-J., & Cho, S.-B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-92639-1_47 · Cited by 30 · see citing works
Moreno, M. A. (2014) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2013.3343 · Cited by 30 · see citing works
The Use of Deep Learning Distributed Representations in the Identification of Abusive Text
Chen, H., McKeever, S., & Delany, S. J. (2019) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v13i01.3215 · Cited by 28 · see citing works
Improved Abusive Comment Moderation with User Embeddings
Pavlopoulos, J., Malakasiotis, P., Bakagianni, J., & Androutsopoulos, I. (2017) · doi:10.18653/v1/w17-4209 · Cited by 27 · see citing works
Agarwal, A., Chivukula, A. S., Bhuyan, M. H., Jan, T., Narayan, B., & Prasad, M. (2020) · Communications in Computer and Information Science · doi:10.1007/978-3-030-63823-8_14 · Cited by 21 · cited 2× · see citing works
“[35] in order to classify cyberbullying posts using under sampling and class weighting for avoiding class imbalance in the dataset.”
BackgroundMethodSelf-citationAutomatic Detection of Online Abuse and Analysis of Problematic Users in Wikipedia
Rawat, C., Sarkar, A., Singh, S. K., et al. (2019) · Systems and Information Engineering Design Symposium · doi:10.1109/sieds.2019.8735592 · Cited by 9 · see citing works
Aglionby, G., Davis, C., Mishra, P., et al. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2100 · Cited by 8 · see citing works
A study of text representations in Hate Speech DetectionPreprint
Themeli, C., Giannakopoulos, G., & Pittaras, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.04521 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 7
- Mishra, P.; del Tredici, M.; Yannakoudakis, H.; Shutova, E. Author Profiling for Abuse Detection. In Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, NM, USA, 20-26 August 2018; pp. 1088-1098.
- Warner, W.; Hirschberg, J. Detecting hate speech on the world wide web. In Proceedings of the LSM'12 Proceedings of the Second Workshop on Language in Social Media, Montreal, QC, Canada, 7 June 2012.
- Yin, D.; Xue, Z.; Hong, L.; Davison, B.D.; Kontostathis, A.; Edwards, L. Detection of Harassment on Web 2.0. In Proceedings of the Content Analysis in the WEB, Madrid, Spain, 21 April 2009; Volume 2.
- Shi, C.Y.; Xu, C.J.; Yang, X.J. Study of TFIDF algorithm. J. Comput. Appl. 2009, 29, 167-170.
- Sarkar, A.; Chatterjee, S.; Das, W.; Datta, D. Text Classification using Support Vector Machine Anurag. Int. J. Eng. Sci. Invent. 2015, 8, 33-37.
- Wright, R.E. Logistic Regression. In Reading and Understanding Multivariate Statistics; Grimm, L.G., Yarnold, P.R., Eds.; American Psychological Association: Washington, DC, USA, 1995; pp. 217-244.
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. In Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA, 4-9 December 2017.
Cited by
Sorted by citations
A Review on Deep-Learning-Based Cyberbullying Detection
Hasan, M. T., Hossain, M. A. E., Mukta, M. S. H., Akter, A., Ahmed, M., & Islam, S. (2023) · Future Internet · doi:10.3390/fi15050179 · Cited by 103 · see citing works
“Raj et al. [149] compared different deep learning and traditional machine learning approaches for cyberbullying classification tasks.”
BackgroundMethodDetecting Kids Cyberbullying Using Transfer Learning Approach: Transformer Fine-Tuning Models
Yafooz, W. M. S., Al-Dhaqm, A., & Alsaeedi, A. (2023) · Studies in Computational Intelligence · doi:10.1007/978-3-031-21199-7_18 · Cited by 17 · see citing works
A Novel Multimodal Hybrid Classifier Based Cyberbullying Detection for Social Media Platform
Suhas Bharadwaj, R., Kuzhalvaimozhi, S., & Vedavathi, N. (2023) · Lecture Notes in Networks and Systems · doi:10.1007/978-3-031-21438-7_57 · Cited by 8 · see citing works
- Authors
- Raj, C., Agarwal, A., Bharathy, G., Narayan, B., & Prasad, M.
- Affiliations
- University of Technology Sydney (AU)
- Delhi Technological University (IN)
- Corresponding author
- Prasad, Mukesh
- Venue
- Electronics
- Publisher
- MDPI AG
- Published
- 2021-11-16
- Volume
- 10
- Issue
- 22
- Pages
- 2810
- ISSN
- 2079-9292 (ISSN-L)
- OpenAlex
- W3213226375
- Cited by
- 128
- References
- 51
- Citations by year
- 2026: 10 · 2025: 43 · 2024: 37 · 2023: 27 · 2022: 11
- FWCI
- 6.63
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Electronics (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UTS ePRESS (University of Technology Sydney) (submittedVersion, other-oa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Text and Document Classification Technologies
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- cyberbullying detection, online harassment, text classification, shallow neural networks, bidirectional LSTM, bidirectional GRU, attention-based models, GloVe embeddings, TF-IDF feature extraction, logistic regression, comparison of classification algorithms
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
