The DOI matches its Crossref record; 83 of the 83 references it cites were found in a registry or an index.
Automatic detection of cyberbullying in social media text
Open accessAbstract
While social media offer great communication opportunities, they also increase the vulnerability of young people to threatening situations online. Recent studies report that cyberbullying constitutes a growing problem among youngsters. Successful prevention depends on the adequate detection of potentially harmful messages and the information overload on the Web requires intelligent systems to identify potential risks automatically. The focus of this paper is on automatic cyberbullying detection in social media text by modelling posts written by bullies, victims, and bystanders of online bullying. We describe the collection and fine-grained annotation of a cyberbullying corpus for English and Dutch and perform a series of binary classification experiments to determine the feasibility of automatic cyberbullying detection. We make use of linear support vector machines exploiting a rich feature set and investigate which information sources contribute the most for the task. Experiments on a hold-out test set reveal promising results for the detection of cyberbullying-related posts. After optimisation of the hyperparameters, the classifier yields an F1 score of 64% and 61% for English and Dutch respectively, and considerably outperforms baseline systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879
Interrater reliability: The kappa statistic
McHugh, M. L. (2012) · Biochemia Medica · doi:10.11613/bm.2012.031 · Cited by 19,877
Indexing by latent semantic analysis
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9 · Cited by 12,751
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002) · Advances in Neural Information Processing Systems 14 · doi:10.7551/mitpress/1120.003.0082 · Cited by 12,223
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359
Libsvm : A library for support vector machines
Ferrari, V. (2008) · Medical Entomology and Zoology · Cited by 10,122
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733 · cited 4×
“Kappa scores for the identification of cyberbullying are κ = 0.69 (Dutch) and κ = 0.59 (English).”
MethodInfluentialMining and summarizing customer reviews
Hu, M., & Liu, B. (2004) · Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1014052.1014073 · Cited by 7,793
Bullying at school: what we know and what we can do
(1994) · Choice Reviews Online · doi:10.5860/choice.32-1080 · Cited by 4,765
Bullying Behaviors Among US Youth: Prevalence and Association With Psychosocial Adjustment
Nansel, T. R., Overpeck, M., Pilla, R. S., Ruan, W. J., Simons-Morton, B., & Scheidt, P. (2001) · JAMA · doi:10.1001/jama.285.16.2094 · Cited by 3,798
Software Framework for Topic Modelling with Large Corpora
Rek, R. R. U., & Sojka, P. (2010) · doi:10.13140/2.1.2393.1847 · Cited by 3,647
Cyberbullying: Its nature and impact in secondary school pupils
Smith, P. K., Mahdavi, J., Carvalho, M., Fisher, S., Russell, S., & Tippett, N. (2008) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.2007.01846.x · Cited by 3,639 · cited 5×
“Cyberbullying research is mainly centered around the conceptualisation, occurrence and prevention of the phenomenon (Hinduja & Patchin, 2012; Livingstone et al., 2010; Slonje & Smith, 2008).”
BackgroundInfluentialRecognizing contextual polarity in phrase-level sentiment analysis
Wilson, T., Wiebe, J., & Hoffmann, P. (2005) · Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing - HLT '05 · doi:10.3115/1220575.1220619 · Cited by 3,345
Tokunaga, R. S. (2010) · Computers in Human Behavior · doi:10.1016/j.chb.2009.11.014 · Cited by 2,566 · cited 4×
“The above paragraphs demonstrate that defining cyberbullying is far from trivial, and varying prevalence rates (cf. Section 1) confirm that a univocal definition of the phenomenon is still lacking in the literature (Tokunaga, 2010).”
BackgroundInfluentialLinguistic Inquiry and Word Count (LIWC): Pronounced “Luke,” . . . and Other Useful Facts
Chung, C. K., & Pennebaker, J. W. (2012) · Applied Natural Language Processing · doi:10.4018/978-1-60960-741-8.ch012 · Cited by 2,151
Bullying as a group process: Participant roles and their relations to social status within the group
Salmivalli, C., Lagerspetz, K., Björkqvist, K., Österman, K., & Kaukiainen, A. (1998) · Aggressive Behavior · doi:10.1002/(sici)1098-2337(1996)22:1<1::aid-ab1>3.0.co;2-t · Cited by 2,019
Cyberbullying: Another main type of bullying?
SLONJE, R., & SMITH, P. K. (2008) · Scandinavian Journal of Psychology · doi:10.1111/j.1467-9450.2007.00611.x · Cited by 1,844
Bullying and the peer group: A review
Salmivalli, C. (2010) · Aggression and Violent Behavior · doi:10.1016/j.avb.2009.08.007 · Cited by 1,757
Bullies Move Beyond the Schoolyard: A Preliminary Look at Cyberbullying
Patchin, J. W., & Hinduja, S. (2006) · Youth Violence and Juvenile Justice · doi:10.1177/1541204006286288 · Cited by 1,701 · cited 2×
“Given these arguments, a number of less ‘strict’ definitions of cyberbullying were postulated by among others (Hinduja & Patchin, 2006; Juvonen & Gross, 2008; Tokunaga, 2010), where a power imbalance and repetition are not deemed necessary conditions for cyberbullying.”
BackgroundPlan 9 From Cyberspace: The Implications of the Internet for Personality and Social Psychology
McKenna, K. Y. A., & Bargh, J. A. (2000) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0401_6 · Cited by 1,506
Extending the school grounds?—Bullying experiences in cyberspace
Juvonen, J., & Gross, E. F. (2008) · Journal of School Health · doi:10.1111/j.1746-1561.2008.00335.x · Cited by 1,408
The General Inquirer: A Computer Approach to Content Analysis
Green, B. F., Stone, P. J., Dunphy, D. C., Smith, M. S., & Ogilvie, D. M. (1967) · American Educational Research Journal · doi:10.2307/1161774 · Cited by 1,013
The nature of cyberbullying, and strategies for prevention
Slonje, R., Smith, P. K., & Frisén, A. (2013) · Computers in Human Behavior · doi:10.1016/j.chb.2012.05.024 · Cited by 940
Bullying at School - What We Know and What We Can Do
Awiria, O., Olweus, D., & Byrne, B. (1994) · British Journal of Educational Studies · doi:10.2307/3121681 · Cited by 927
Ambati, B. R., Deoskar, T., Johnson, M., & Steedman, M. J. (2015) · The Association for Computational Linguistics · Cited by 860
Cyberbullying: Youngsters' experiences and parental perception
Dehue, F., Bolman, C., & Völlink, T. (2008) · CyberPsychology & Behavior · doi:10.1089/cpb.2007.0008 · Cited by 802
Cyberbullying Versus Face-to-Face Bullying: A Theoretical and Conceptual Review
Dooley, J. J., Pyżalski, J., & Cross, D. (2009) · Zeitschrift für Psychologie / Journal of Psychology · doi:10.1027/0044-3409.217.4.182 · Cited by 790
A new ANEW: Evaluation of a word list for sentiment analysis in microblogs
Nielsen, F. Å. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.2903 · Cited by 712
Willard, N. E. (2006) · Cited by 708
Defining Cyberbullying: A Qualitative Research into the Perceptions of Youngsters
Vandebosch, H., & Van Cleemput, K. (2008) · CyberPsychology & Behavior · doi:10.1089/cpb.2007.0042 · Cited by 663
Salmivalli, C., Voeten, M., & Poskiparta, E. (2011) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2011.597090 · Cited by 597
Fekkes, M., Pijpers, F. I. M., Fredriks, A. M., Vogels, T., & Verloove-Vanhorick, S. P. (2006) · Pediatrics · doi:10.1542/peds.2005-0187 · Cited by 592
Internet Use and Well‐Being in Adolescence
Gross, E. F., Juvonen, J., & Gable, S. L. (2002) · Journal of Social Issues · doi:10.1111/1540-4560.00249 · Cited by 580
Cyberbullying among youngsters: profiles of bullies and victims
Vandebosch, H., & Van Cleemput, K. (2009) · New Media & Society · doi:10.1177/1461444809341263 · Cited by 555
Self‐esteem and its relationship to bullying behaviour
O’Moore, M., & Kirkham, C. (2001) · Aggressive Behavior · doi:10.1002/ab.1010 · Cited by 513
Modeling the Detection of Textual Cyberbullying
Dinakar, K., Reichart, R., & Lieberman, H. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i3.14209 · Cited by 512
Using Machine Learning to Detect Cyberbullying
Reynolds, K., Kontostathis, A., & Edwards, L. (2011) · 2011 10th International Conference on Machine Learning and Applications and Workshops · doi:10.1109/icmla.2011.152 · Cited by 459
Learning from Bullying Traces in Social Media
Xu, J., Jun, K.-S., Zhu, X., & Bellmore, A. D. (2012) · North American Chapter of the Association for Computational Linguistics · Cited by 455
Common Sense Reasoning for Detection, Prevention, and Mitigation of Cyberbullying
Dinakar, K., Jones, B., Havasi, C., Lieberman, H., & Picard, R. (2012) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2362394.2362400 · Cited by 432
Mean Birds: Detecting Aggression and Bullying on Twitter
Chatzakou, D., Kourtellis, N., Blackburn, J., De Cristofaro, E., Stringhini, G., & Vakali, A. (2017) · Proceedings of the 2017 ACM on Web Science Conference · doi:10.1145/3091478.3091487 · Cited by 369
Bastiaensens, S., Vandebosch, H., Poels, K., Van Cleemput, K., DeSmet, A., & De Bourdeaudhuij, I. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2013.10.036 · Cited by 367
Al-garadi, M. A., Varathan, K. D., & Ravana, S. D. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.05.051 · Cited by 350
The General Inquirer: a computer approach to content analysis
Stone, P. J., Dunphy, D. C., & Smith, M. S. (1966) · CERN Document Server (European Organization for Nuclear Research) · Cited by 344
Detection of Harassment on Web 2.0
Yin, D., Xue, Z., Hong, L., Davison, B. D., & Edwards, L. (2009) · Cited by 330
Salmivalli, C., Kaukiainen, A., Kaistaniemi, L., & Lagerspetz, K. M. J. (1999) · Personality and Social Psychology Bulletin · doi:10.1177/0146167299258008 · Cited by 329
LIBLINEAR: A Library for Large Linear Classification
FanRong-En, ChangKai-Wei, HsiehCho-Jui, WangXiang-Rui, & LinChih-Jen (2008) · Journal of Machine Learning Research · doi:10.5555/1390681.1442794 · Cited by 328
Reconceptualizing ‘flaming’ and other problematic messages
O’Sullivan, P. B., & Flanagin, A. J. (2003) · New Media & Society · doi:10.1177/1461444803005001908 · Cited by 303
Approaches to automated detection of cyberbullying: A survey
Salawu, S., He, Y., & Lumsden, J. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2761757 · Cited by 258
(2010) · Youth studies Australia · Cited by 252
Detection and Fine-Grained Classification of Cyberbullying Events
Hee, C. V., Lefever, E., Verhoeven, B., Mennes, J., Desmet, B., Pauw, G. D., Daelemans, W. M. P., & Hoste, V. (2015) · Ghent University Academic Bibliography (Ghent University) · Cited by 247
Self-citationIn their own words: What bothers children online?
Livingstone, S., Kirwil, L., Ponte, C., & Staksrud, E. (2014) · European Journal of Communication · doi:10.1177/0267323114521045 · Cited by 233
Generating high-coverage semantic orientation lexicons from overtly marked words and a thesaurus
Mohammad, S., Dunne, C., & Dorr, B. (2009) · Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing Volume 2 - EMNLP '09 · doi:10.3115/1699571.1699591 · Cited by 233
Automatic detection of cyberbullying on social networks based on bullying features
Zhao, R., Zhou, A., & Mao, K. (2016) · Proceedings of the 17th International Conference on Distributed Computing and Networking · doi:10.1145/2833312.2849567 · Cited by 230
Cyber Bullying Detection Using Social and Textual Analysis
Huang, Q., Singh, V. K., & Atrey, P. K. (2014) · Proceedings of the 3rd International Workshop on Socially-Aware Multimedia · doi:10.1145/2661126.2661133 · Cited by 209
Bullies Move Beyond the Schoolyard: A Preliminary Look at Cyberbullying
(2016) · Social Science Research · doi:10.4324/9781315265841-8 · Cited by 196
Cyberbullying: Neither an epidemic nor a rarity
Hinduja, S., & Patchin, J. W. (2012) · European Journal of Developmental Psychology · doi:10.1080/17405629.2012.706448 · Cited by 191
Galán-GarcÍa, P., Puerta, J. G. D. L., Gómez, C. L., Santos, I., & Bringas, P. G. (2015) · Logic Journal of IGPL · doi:10.1093/jigpal/jzv048 · Cited by 177
Motivation and language behavior: A content analysis of suicide notes.
Osgood, C. E., & Walker, E. G. (1959) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0047078 · Cited by 166
Chavan, V. S., & Shylaja, S. S. (2015) · International Conference on Advances in Computing, Communications and Informatics (ICACCI) · doi:10.1109/icacci.2015.7275970 · Cited by 153
Bullying as a Group Process: Participant Roles and Their Relations to Social Status Within the Group
Salmivalli, C., LAGERSPETZ, K. M. J., Björkqvist, K., Österman, K., & Kaukiainen, A. (2017) · Routledge eBooks · doi:10.4324/9781351153683-13 · Cited by 150
Cyberbullying and its impact on young people's emotional health and well-being
Cowie, H. (2013) · The Psychiatrist · doi:10.1192/pb.bp.112.040840 · Cited by 145
Cyberbullying Detection Based on Semantic-Enhanced Marginalized Denoising Auto-Encoder
Zhao, R., & Mao, K. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2531682 · Cited by 138
Online Social Network Bullying Detection Using Intelligence Techniques
Nandhini, B. S., & Sheeba, J. I. (2015) · Procedia Computer Science · doi:10.1016/j.procs.2015.03.085 · Cited by 121
Semi-supervised Learning for Cyberbullying Detection in Social Networks
Nahar, V., Al-Maskari, S., Li, X., & Pang, C. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-08608-8_14 · Cited by 100
Bastiaensens, S., Vandebosch, H., Poels, K., Cleemput, K. V., DeSmet, A., & Bourdeaudhuij, I. D. (2014) · Behaviour and Information Technology · doi:10.1080/0144929x.2014.983979 · Cited by 98
Identification and characterization of cyberbullying dynamics in an online social network
Squicciarini, A., Rajtmajer, S. M., Liu, Y., & Griffin, C. H. (2015) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1145/2808797.2809398 · Cited by 95
Royen, K. V., Poels, K., Daelemans, W. M. P., & Vandebosch, H. (2014) · Telematics and Informatics · doi:10.1016/j.tele.2014.04.002 · Cited by 94
Self-citationCyberbullying Detection with Weakly Supervised Machine Learning
Raisi, E., & Huang, B. (2017) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1145/3110025.3110049 · Cited by 94
Automatic Detection and Prevention of Cyberbullying
Hee, C. V., Lefever, E., Verhoeven, B., Mennes, J., Desmet, B., Pauw, G. D., Daelemans, W. M. P., & Hoste, V. (2015) · Ghent University Academic Bibliography (Ghent University) · Cited by 89
Self-citationLeTs Preprocess: The multilingual LT3 linguistic preprocessing toolkit
de Kauter, M. V., Coorman, G., Lefever, E., Desmet, B., Macken, L., & Hoste, V. (2013) · Computational Linguistics in the Netherlands Journal · Cited by 55
Self-citationExperts and machines united against cyberbullying
Dadvar, M. (2014) · doi:10.3990/1.9789036537391 · Cited by 52
Salmivalli, C., & Pöyhönen, V. (2012) · Wiley eBooks · doi:10.1002/9781119954484.ch4 · Cited by 47
“Vreselijk mooi!” (terribly beautiful): A Subjectivity Lexicon for Dutch Adjectives.
Smedt, T. D., & Daelemans, W. M. P. (2012) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3jm4sd5bm7ut · Cited by 47
Self-citationProceedings of the 3rd International Web Science Conference
Roure, D. D., & Poole, S. (2011) · doi:10.1145/2527031 · Cited by 46
A review of cyberbullying detection: An overview
Nadali, S., Murad, M. A. A., Sharef, N. M., Mustapha, A., & Shojaee, S. (2013) · International Conference on Intelligent Systems Design and Applications (ISDA) · doi:10.1109/isda.2013.6920758 · Cited by 44
Royen, K. V., Poels, K., & Vandebosch, H. (2016) · Children and Youth Services Review · doi:10.1016/j.childyouth.2016.02.024 · Cited by 21
(2012) · Research Explorer (The University of Manchester)
Cited by
Dewani, A., Memon, M. A., & Bhatti, S. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00550-7 · Cited by 91
A Bi-GRU with attention and CapsNet hybrid model for cyberbullying detection on social media
Kumar, A., & Sachdeva, N. (2022) · World Wide Web · doi:10.1007/s11280-021-00920-4 · Cited by 87
“Most of the earlier attempts on cyberbullying detection rely on manual feature extraction methods [36].”
BackgroundCarrasco-Farré, C. (2022) · Humanities and Social Sciences Communications · doi:10.1057/s41599-022-01174-9 · Cited by 84
Teng, T. H., & Varathan, K. D. (2023) · IEEE Access · doi:10.1109/access.2023.3275130 · Cited by 77
(2022) · doi:10.2196/30286 · Cited by 65
Metadata
- Authors
- Van Hee, C., Jacobs, G., Emmery, C., Desmet, B., Lefever, E., Verhoeven, B., De Pauw, G., Daelemans, W., & Hoste, V.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2018-10-08
- Volume
- 13
- Issue
- 10
- Pages
- e0203794
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2784010253
- Cited by
- 385
- References
- 83
- FWCI
- 21.92
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.