Cyberbully Detection Using BERT with Augmented Texts
Abstract
Detecting cyberbullying in texts is an essential task as it curtails and identifies s ocial p roblems. I n t his p aper, we propose an architecture called Augmented BERT which combines both data augmentation techniques and BERT for detecting cyberbullying content in texts. Many techniques have been used in prior works to augment existing data for classification tasks and BERT had been applied in many text classification problems. However, there is a lack of annotated cyberbullying texts and obtaining annotated texts is hard and expensive. We propose to use various GAN-based and autoencoder-based data augmentation techniques to generate annotated data. The augmented texts can be used to fine-tune BERT. We choose to use HateBERT which is already pre-trained on abusive language to detect cyberbullying texts. Experimental results show an increased improvement over other cyberbullying detection models.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
BERT has demonstrated outstanding results in detecting cyberbullying because it is an extensive language model that can extract contextual information from a given text.
“BERT (Bidirectional Encoder Representations from Transformers) has demonstrated outstanding results in detecting cyberbullying since it is an extensive language model that can extract contextual information from a given text.”
From A Review on Deep-Learning-Based Cyberbullying Detection (2023) · cites this paper as [117]
BERT can accurately detect bullying behavior in social media messages when the cue is subtle or indirect.
“BERT can accurately detect bullying behavior in social media messages when the cue is subtle or indirect [22,116,117].”
From A Review on Deep-Learning-Based Cyberbullying Detection (2023) · cites this paper as [117]
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
A survey on Image Data Augmentation for Deep Learning
Shorten, C., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0197-0 · Cited by 13,203 · see citing works
Advances in Neural Information Processing Systems 14
(2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.001.0001 · Cited by 9,184 · see citing works
Cyberbullying: Its nature and impact in secondary school pupils
Smith, P. K., Mahdavi, J., Carvalho, M., Fisher, S., Russell, S., & Tippett, N. (2008) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.2007.01846.x · Cited by 3,639 · see citing works
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610 · see citing works
SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient
Yu, L., Zhang, W., Wang, Y.-X., & Yu, Y. H. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10804 · Cited by 2,342 · see citing works
A Call for Clarity in Reporting BLEU Scores
Post, M. (2018) · Proceedings of the Third Conference on Machine Translation: Research Papers · doi:10.18653/v1/w18-6319 · Cited by 2,112 · see citing works
Text Data Augmentation for Deep Learning
Shorten, C., Khoshgoftaar, T. M., & Furht, B. (2021) · Journal Of Big Data · doi:10.1186/s40537-021-00492-0 · Cited by 1,798 · see citing works
A Survey of Data Augmentation Approaches for NLP
Feng, S. Y., Gangal, V., Wei, J., Chandar, S., Vosoughi, S., Mitamura, T., & Hovy, E. H. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.84 · Cited by 802 · see citing works
Time Series Data Augmentation for Deep Learning: A Survey
Wen, Q., Sun, L., Yang, F., Song, X., Gao, J., Wang, X., & Xu, H. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/631 · Cited by 727 · see citing works
Long Text Generation via Adversarial Training with Leaked Information
Guo, J., Lu, S., Cai, H., Zhang, W., Yu, Y. H., & Wang, Y.-X. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11957 · Cited by 512 · see citing works
Racial Bias in Hate Speech and Abusive Language Detection Datasets
Davidson, T., Bhattacharya, D., & Weber, I. (2019) · Proceedings of the Third Workshop on Abusive Language Online · doi:10.18653/v1/w19-3504 · Cited by 414 · see citing works
MaskGAN: Better Text Generation via Filling in the______Preprint
Fedus, W., Goodfellow, I., & Dai, A. M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.07736 · Cited by 412 · see citing works
HateBERT: Retraining BERT for Abusive Language Detection in English
Caselli, T., Basile, V., Mitrović, J., & Granitzer, M. (2021) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2021.woah-1.3 · Cited by 373 · see citing works
Data augmentation approaches in natural language processing: A survey
Li, B., Hou, Y., & Che, W. (2022) · AI Open · doi:10.1016/j.aiopen.2022.03.001 · Cited by 357 · see citing works
Reducing Gender Bias in Abusive Language Detection
Park, J. H., Shin, J., & Fung, P. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1302 · Cited by 323 · see citing works
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Hartvigsen, T., Gabriel, S., Palangi, H., Sap, M., Ray, D., & Kamar, E. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.234 · Cited by 279 · see citing works
A Hybrid Convolutional Variational Autoencoder for Text Generation
Semeniuta, S., Severyn, A., & Barth, E. (2017) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1066 · Cited by 272 · see citing works
A Comparative Analysis of Machine Learning Techniques for Cyberbullying Detection on Twitter
Muneer, A., & Fati, S. M. (2020) · Future Internet · doi:10.3390/fi12110187 · Cited by 211 · see citing works
Croce, D., Castellucci, G., & Basili, R. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.191 · Cited by 207 · see citing works
An Analysis of Simple Data Augmentation for Named Entity Recognition
Dai, X., & Adel, H. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.343 · Cited by 206 · see citing works
DETECTION OF HATE SPEECH IN SOCIAL NETWORKS: A SURVEY ON MULTILINGUAL CORPUS
Alhassan, A., & Al-Dossari, H. Z. (2019) · doi:10.5121/csit.2019.90208 · Cited by 188 · see citing works
A survey on face data augmentation for the training of deep neural networks
Wang, X., Wang, K., & Lian, S. (2020) · Neural Computing and Applications · doi:10.1007/s00521-020-04748-3 · Cited by 141 · see citing works
Experts and Machines against Bullies: A Hybrid Approach to Detect Cyberbullies
Dadvar, M., Trieschnigg, D., & de Jong, F. M. G. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-06483-3_25 · Cited by 139 · see citing works
SOSNet: A Graph Convolutional Network Approach to Fine-Grained Cyberbullying Detection
Wang, J., Fu, K., & Lu, C. (2020) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata50022.2020.9378065 · Cited by 139 · see citing works
Imbalanced Toxic Comments Classification Using Data Augmentation and Deep Learning
Ibrahim, M., Torki, M., & El-Makky, N. (2018) · IEEE International Conference on Machine Learning and Applications (ICMLA) · doi:10.1109/icmla.2018.00141 · Cited by 112 · see citing works
A Multichannel Deep Learning Framework for Cyberbullying Detection on Social Media
Alotaibi, M., Alotaibi, B., & Razaque, A. (2021) · Electronics · doi:10.3390/electronics10212664 · Cited by 94 · see citing works
A Multichannel Deep Learning Framework for Cyberbullying Detection on Social Media
Alotaibi, M., Alotaibi, B., & Razaque, A. (2021) · doi:10.20944/preprints202110.0070.v1 · Cited by 94 · cited 2× · see citing works
“Other methods that have used machine learning for cyberbul-lying include [23]–[27].”
MethodCyberbullying and Sexting: Technology Abuses of the 21st Century
Siegle, D. (2010) · Gifted Child Today · doi:10.1177/107621751003300206 · Cited by 92 · see citing works
A Bi-GRU with attention and CapsNet hybrid model for cyberbullying detection on social media
Kumar, A., & Sachdeva, N. (2022) · World Wide Web · doi:10.1007/s11280-021-00920-4 · Cited by 87 · see citing works
Michele Coscia, A. C., Rokne, M. C. A. A. C. A. K. S. A. R. K. A. S. O. A. J. G., Coscia, M., Cuzzocrea, A., Shu, K., Klamma, R., O'Halloran, S., & Rokne, J. G. (2021) · doi:10.1145/3487351 · Cited by 78 · see citing works
Liu, Z., Wang, J., & Liang, Z. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6361 · Cited by 74 · see citing works
Cyberbullying Detection in Social Networks Using Deep Learning Based Models; A Reproducibility Study
Dadvar, M., & Eckert, K. (2018) · arXiv · doi:10.48550/arxiv.1812.08046 · Cited by 71 · see citing works
Cyberbullying Detection in Social Networks Using Deep Learning Based Models
Dadvar, M., & Eckert, K. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59065-9_20 · Cited by 68 · see citing works
Topic-Guided Variational Auto-Encoder for Text Generation
Wang, W., Gan, Z., Xu, H., Zhang, R., Wang, G., Shen, D., Chen, C., & Carin, L. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1015 · Cited by 67 · see citing works
Text Generation for Imbalanced Text Classification
Akkaradamrongrat, S., Kachamas, P., & Sinthupinyo, S. (2019) · International Joint Conference on Computer Science and Software Engineering · doi:10.1109/jcsse.2019.8864181 · Cited by 32 · see citing works
Multilingual AMR-to-Text Generation
Fan, A., & Gardent, C. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.231 · Cited by 29 · see citing works
Sangwan, S. R., & Bhatia, M. P. S. (2020) · Multimedia Systems · doi:10.1007/s00530-020-00661-w · Cited by 27 · see citing works
Adversarial Text Generation Without Reinforcement LearningPreprint
Donahue, D., & Rumshisky, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.06640 · Cited by 25 · see citing works
Smelting Gold and Silver for Improved Multilingual AMR-to-Text Generation
Ribeiro, L. F. R., Pfeiffer, J., Zhang, Y., & Gurevych, I. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.57 · Cited by 13 · see citing works
MT-GAN-BERT: Multi-Task and Generative Adversarial Learning for Sustainable Language Processing.
Breazzano, C., Croce, D., & Basili, R. (2021) · Cited by 2 · see citing works
A Deep Learning Framework for Event Detection in Augmented Twitter DataPreprint
Anjum, U., Zadorozhny, V., & Krishnamurthy, P. V. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4124071 · Cited by 1 · see citing works
Self-citationBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
Cited by
Sorted by citations
A Review on Deep-Learning-Based Cyberbullying Detection
Hasan, M. T., Hossain, M. A. E., Mukta, M. S. H., Akter, A., Ahmed, M., & Islam, S. (2023) · Future Internet · doi:10.3390/fi15050179 · Cited by 103 · see citing works
“BERT can accurately detect bullying behavior in social media messages when the cue is subtle or indirect [22,116,117].”
Background
- Authors
- Guo, X., Anjum, U., & Zhan, J.
- Affiliations
- University of Arkansas at Fayetteville (US)
- University of Cincinnati (US)
- ORCID
- Anjum, Usman
- Venue
- 2022 IEEE International Conference on Big Data (Big Data)
- Publisher
- IEEE
- Published
- 2022-12-17
- Pages
- 1246-1253
- OpenAlex
- W4318147828
- Cited by
- 12
- References
- 44
- Citations by year
- 2025: 6 · 2024: 4 · 2023: 2
- FWCI
- 1.66
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Big Data (BigData) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- cyberbullying detection, BERT, data augmentation, generative adversarial networks, text classification, offensive language detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review