CoChat

A Human-Centered Systematic Literature Review of Cyberbullying Detection Algorithms

Open access
Kim, Seunghyun, Razi, Afsaneh, Stringhini, Gianluca, Wisniewski, Pamela J., De Choudhury, Munmun · 2021 · Proceedings of the ACM on Human-Computer Interaction
Cited by 79

The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.

Abstract

Cyberbullying is a growing problem across social media platforms, inflicting short and long-lasting effects on victims. To mitigate this problem, research has looked into building automated systems, powered by machine learning, to detect cyberbullying incidents, or the involved actors like victims and perpetrators. In the past, systematic reviews have examined the approaches within this growing body of work, but with a focus on the computational aspects of the technical innovation, feature engineering, or performance optimization, without centering around the roles, beliefs, desires, or expectations of humans. In this paper, we present a human-centered systematic literature review of the past 10 years of research on automated cyberbullying detection. We analyzed 56 papers based on a three-prong human-centeredness algorithm design framework - spanning theoretical, participatory, and speculative design. We found that the past literature fell short of incorporating human-centeredness across multiple aspects, ranging from defining cyberbullying, establishing the ground truth in data annotation, evaluating the performance of the detection models, to speculating the usage and users of the models, including potential harms and negative consequences. Given the sensitivities of the cyberbullying experience and the deep ramifications cyberbullying incidents bear on the involved actors, we discuss takeaways on how incorporating human-centeredness in future research can aid with developing detection systems that are more practical, useful, and tuned to the diverse needs and contexts of the stakeholders.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3476066

Cites

28 of 28 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 28 references

Research works printed but not linked: 143

  1. 10.1145/3284751.3284761 · DOI · Paper page
  2. 10.3991/ijep.v10i4.14219 · doi:10.3991/ijep.v10i4.14219
  3. 10.1016/j.childyouth.2020.104814 · DOI
  4. 10.1109/ACCESS.2019.2918354 · DOI · Paper page
  5. 10.1016/j.chb.2016.05.051 · DOI · Paper page
  6. 10.1109/BigMM50055.2020.00032 · DOI
  7. Mark Alber, Adrian Buganza Tepole, William R Cannon, Suvranu De, Salvador Dura-Bernal, Krishna Garikipati, George Karniadakis, William W Lytton, Paris Perdikaris, Linda Petzold, et al. 2019. Integrating machine learning and multiscale modeling-perspectives, challenges, and opportunities in the biological, biomedical, and behavioral sciences. NPJ digital medicine, Vol. 2, 1 (2019), 1--11.
  8. 10.1080/14626268.2013.767276 · DOI · Paper page
  9. Jieun Baek and Lyndal M Bullock. 2014. Cyberbullying: a cross-cultural perspective. Emotional and behavioural difficulties, Vol. 19, 2 (2014), 226--238.
  10. 10.1177/0022022113504622 · DOI
  11. 10.1145/3002137.3002144 · DOI
  12. 10.1007/978--3--319--75417--8_17 · doi:10.1007/978--3--319--75417--8_17
  13. 10.1080/1369118X.2012.678878 · DOI · Paper page
  14. Tommy KH Chan, Christy MK Cheung, and Zach WY Lee. 2020. Cyberbullying on Social Networking Sites: A Literature Review and Future Research Directions. Information & Management (2020), 103411.
  15. 10.1145/3359249 · DOI · Paper page
  16. 10.1145/3091478.3091487 · DOI · Paper page
  17. 10.1145/3041021.3054211 · DOI
  18. 10.1145/3343484 · DOI
  19. 10.1109/ICACCI.2015.7275970 · DOI
  20. 10.1145/3308560.3316503 · DOI
  21. 10.1145/3289600.3291037 · DOI
  22. 10.1109/TKDE.2016.2555805 · DOI
  23. 10.1177/1077801211436167 · DOI
  24. Lawrence E Cohen and Marcus Felson. 1979. Social change and crime rate trends: A routine activity approach. American sociological review (1979), 588--608.
  25. Maral Dadvar Franciska De Jong Roeland and Ordelman Dolf Trieschnigg. 2012a. Improved Cyberbullying Detection Using Gender Information. Technical Report. http://www.noswearing.com/dictionary · DOI · Paper page
  26. 10.1145/2187980.2187995 · DOI
  27. Maral Dadvar, FMG de Jong, Roeland Ordelman, and Dolf Trieschnigg. 2012b. Improved cyberbullying detection using gender information. In Proceedings of the Twelfth Dutch-Belgian Information Retrieval Workshop (DIR 2012). University of Ghent.
  28. 10.1007/978--3--319-06483--3_25 · doi:10.1007/978--3--319-06483--3_25
  29. 10.1007/978--3--642--36973--5_62 · doi:10.1007/978--3--642--36973--5_62
  30. 10.1145/2488388.2488416 · DOI · Paper page
  31. 10.1007/978--3--319--71249--9_4 · doi:10.1007/978--3--319--71249--9_4
  32. 10.18653/v1/W19--3504 · doi:10.18653/v1/w19--3504
  33. 10.1080/1369118X.2014.952657 · DOI
  34. Rebecca Dennehy, Sarah Meaney, Kieran A Walsh, Carol Sinnott, Mary Cronin, and Ella Arensman. 2020. Young people's conceptualizations of the nature of cyberbullying: A systematic review and synthesis of qualitative research. Aggression and violent behavior, Vol. 51 (2020), 101379.
  35. 10.1109/ICPR.2016.7899672 · DOI
  36. 10.1145/2362394.2362400 · DOI · Paper page
  37. Karthik Dinakar, Roi Reichart, and Henry Lieberman. 2011. Modeling the detection of textual cyberbullying. In AAAI Workshop - Technical Report, Vol. WS-11-02. 11--17. www.aaai.org
  38. Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017). · arXiv
  39. Rebecca Dredge, John Gleeson, and Xochitl De la Piedad Garcia. 2014a. Cyberbullying in social networking sites: An adolescent victim's perspective. Computers in human behavior, Vol. 36 (2014), 13--20.
  40. 10.1016/j.chb.2014.07.035 · DOI
  41. 10.1145/3145472 · DOI · Paper page
  42. Anthony Dunne and Fiona Raby. 2013. Speculative everything: design, fiction, and social dreaming .MIT press.
  43. Chris Emmery, Ben Verhoeven, Guy De Pauw, Gilles Jacobs, Cynthia Van Hee, Els Lefever, Bart Desmet, Véronique Hoste, and Walter Daelemans. 2020. Current limitations in cyberbullying detection: On evaluation criteria, reproducibility, and data scarcity. Language Resources and Evaluation (2020), 1--37.
  44. 10.1145/3290605.3300364 · DOI
  45. Dorothy L Espelage and Susan M Swearer. 2003. Research on school bullying and victimization: What have we learned and where do we go from here? School psychology review, Vol. 32, 3 (2003), 365--383.
  46. Rebecca Fiebrink and Marco Gillies. 2018. Introduction to the special issue on human-centered machine learning. · DOI
  47. Helen L Fisher, Terrie E Moffitt, Renate M Houts, Daniel W Belsky, Louise Arseneault, and Avshalom Caspi. 2012. Bullying victimisation and risk of self harm in early adolescence: longitudinal cohort study. bmj, Vol. 344 (2012), e2683.
  48. Patxi Galán-Garc'ia, José Gaviria de la Puerta, Carlos Laorden Gómez, Igor Santos, and Pablo Garc'ia Bringas. 2016. Supervised machine learning for the detection of troll profiles in twitter social network: Application to a real case of cyberbullying. Logic Journal of the IGPL, Vol. 24, 1 (2016), 42--53.
  49. Oguzhan Gencoglu. 2020. Cyberbullying Detection with Fairness Constraints. arXiv preprint arXiv:2005.06625 (2020). · arXiv
  50. James J Gibson. 1977. The theory of affordances. Hilldale, USA, Vol. 1, 2 (1977).
  51. 10.1109/CCOMS.2018.8463326 · DOI
  52. 10.1002/9781444316568.wiem02057 · DOI · Paper page
  53. 10.14569/ijacsa.2019.0100587 · DOI
  54. 10.1016/S0022-3999(00)00230-0 · DOI · Paper page
  55. Brent Hecht, Lauren Wilcox, Jeffrey P Bigham, Johannes Schöning, Ehsan Hoque, Jason Ernst, Yonatan Bisk, Lana Yarosh, Bushra Amjam, and Cathy Wu. 2018. It's time to do something: Mitigating the negative impacts of computing through a change to the peer review process. ACM Future of Computing Blog (2018).
  56. Sameer Hinduja and Justin W Patchin. 2012. School law enforcement and cyberbullying. Cyberbullying prevention and response: Expert perspectives (2012), 161--184.
  57. Dianne L Hoff and Sidney N Mitchell. 2009. Cyberbullying: Causes, effects, and remedies. Journal of Educational Administration (2009).
  58. Jake M Hofman, Amit Sharma, and Duncan J Watts. 2017. Prediction and explanation in social systems. Science, Vol. 355, 6324 (2017), 486--488.
  59. 10.1145/2661126.2661133 · DOI · Paper page
  60. Abigail Z Jacobs and Hanna Wallach. 2019. Measurement and fairness. arXiv preprint arXiv:1912.05511 (2019). · arXiv
  61. Alejandro Jaimes, Daniel Gatica-Perez, Nicu Sebe, and Thomas S Huang. 2007. Guest Editors' Introduction: Human-Centered Computing--Toward a Human Revolution. Computer, Vol. 40, 5 (2007), 30--34.
  62. 10.18653/v1/P19--1357 · doi:10.18653/v1/p19--1357
  63. Seunghyun Kim, Afsaneh Razi, Gianluca Stringhini, Pamela Wisniewski, and Munmun De Choudhury. noop3002forthcoming. You Don't Know How I Feel: Insider-Outsider Perspective Gaps in Cyberbullying Risk Detection. In Proceedings of the International AAAI Conference on Web and Social Media .
  64. David G Kleinbaum, K Dietz, M Gail, Mitchel Klein, and Mitchell Klein. 2002. Logistic regression .Springer.
  65. Ana Kovavc ević. 2014. Cyberbullying detection using web content mining. In 2014 22nd Telecommunications Forum Telfor (TELFOR). IEEE, 939--942.
  66. 10.1007/s11042-019-7234-z · DOI
  67. 10.1080/02796015.2007.12087931 · DOI
  68. 10.1016/j.jaac.2013.03.012 · DOI · Paper page
  69. Melvin J Lerner and Dale T Miller. 1978. Just world research and the attribution process: Looking back and ahead. Psychological bulletin, Vol. 85, 5 (1978), 1030.
  70. 10.1080/00131880802309333 · DOI · Paper page
  71. 10.1145/3011141.3011182 · DOI
  72. 10.1287/isre.2016.0671 · DOI · Paper page
  73. 10.1109/ICONIC.2018.8601278 · DOI
  74. 10.1109/EIT.2015.7293405 · DOI
  75. Diana J Meter, Ross Budziszewski, Abigail Phillips, and Troy E Beckert. 2021. A Qualitative Exploration of College Students' Perceptions of Cyberbullying. TechTrends (2021), 1--9.
  76. Jerold D Miller and Shirley M Hufstedler. 2009. Cyberbullying Knows No Borders. Australian Teacher Education Association (2009).
  77. Tijana Milosevic. 2016. Social media companies' cyberbullying policies. International Journal of Communication, Vol. 10 (2016), 22.
  78. 10.1080/14639220500090604 · DOI · Paper page
  79. R Morris, Daniel McDuff, and R Calvo. 2014. Crowdsourcing techniques for affective computing. In The Oxford handbook of affective computing. Oxford Univ. Press, 384--394.
  80. Michael Muller, Cecilia Aragon, Shion Guha, Marina Kogan, Gina Neff, Cathrine Seidelin, Katie Shilton, and Anissa Tanweer. 2020. Interrogating Data Science. In Conference Companion Publication of the 2020 on Computer Supported Cooperative Work and Social Computing. 467--473.
  81. Michael J Muller and Allison Druin. 2012. Participatory design: The third space in human-computer interaction. The human-computer interaction handbook: Fundamentals, evolving technologies, and emerging applications (2012), 1125--1154.
  82. 10.3390/fi12110187 · DOI · Paper page
  83. 10.1080/0013188032000086154 · DOI
  84. 10.1109/ISDA.2013.6920758 · DOI
  85. 10.1007/978--3--319-08608--8_14 · doi:10.1007/978--3--319-08608--8_14
  86. Vinita Nahar, Xue Li, Chaoyi Pang, and Yang Zhang. 2013. Cyberbullying Detection based on text-stream classification. In Conferences in Research and Practice in Information Technology Series, Vol. 146. 49--58. http://www.noswearing.com/
  87. 10.1007/978--3--642--29253--8_75 · doi:10.1007/978--3--642--29253--8_75
  88. 10.1007/978--3--319--13731--5_69 · doi:10.1007/978--3--319--13731--5_69
  89. 10.17485/ijst/2016/v9i28/93825 · doi:10.17485/ijst/2016/v9i28/93825
  90. Charisse L Nixon. 2014. Current perspectives: the impact of cyberbullying on adolescent health. Adolescent health, medicine and therapeutics, Vol. 5 (2014), 143.
  91. William S Noble. 2006. What is a support vector machine? Nature biotechnology, Vol. 24, 12 (2006), 1565--1567.
  92. 10.1016/S0272-6963(98)00020-5 · DOI · Paper page
  93. Dan Olweus. 1994. Bullying at school. In Aggressive behavior. Springer, 97--130.
  94. 10.1080/01431160412331269698 · DOI · Paper page
  95. 10.18653/v1/D18--1302 · doi:10.18653/v1/d18--1302
  96. John Pavlopoulos, Jeffrey Sorensen, Lucas Dixon, Nithum Thain, and Ion Androutsopoulos. 2020. Toxicity Detection: Does Context Really Matter? arXiv preprint arXiv:2006.00998 (2020). · arXiv
  97. James W Pennebaker, Martha E Francis, and Roger J Booth. 2001. Linguistic inquiry and word count: LIWC 2001. Mahway: Lawrence Erlbaum Associates, Vol. 71, 2001 (2001), 2001.
  98. 10.1109/ICDMW.2014.170 · DOI
  99. 10.1145/3290605.3300294 · DOI
  100. 10.1145/3167132.3167317 · DOI
  101. Elaheh Raisi and Bert Huang. 2016. Cyberbullying Identification Using Participant-Vocabulary Consistency. (6 2016). http://arxiv.org/abs/1606.08084 · arXiv
  102. 10.1145/3110025.3110049 · DOI
  103. 10.1007/978-3-030-34980-6_32 · DOI
  104. Gonzalo Ramos, Jina Suh, Soroush Ghorashi, Christopher Meek, Richard Banks, Saleema Amershi, Rebecca Fiebrink, Alison Smith-Renner, and Gagan Bansal. 2019. Emerging Perspectives in Human-Centered Machine Learning. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1--8.
  105. Johan Redström. 2006. Towards user design? On the shift from object to user as the subject of design. Design studies, Vol. 27, 2 (2006), 123--139.
  106. 10.1109/ICMLA.2011.152 · DOI · Paper page
  107. Irina Rish et al. 2001. An empirical study of the naive Bayes classifier. In IJCAI 2001 workshop on empirical methods in artificial intelligence, Vol. 3. 41--46.
  108. 10.1109/KST.2017.7886127 · DOI
  109. Derek Ruths and Jürgen Pfeffer. 2014. Social media for large studies of behavior. Science, Vol. 346, 6213 (2014), 1063--1064.
  110. 10.1145/3292522.3326032 · DOI · Paper page
  111. Nithya Sambasivan, Erin Arnesen, Ben Hutchinson, and Vinodkumar Prabhakaran. 2020. Fairness Considered Harmful: On the Non-portability of Fair-ML in India. arXiv preprint arXiv:2012.03659 (2020). · arXiv
  112. Huascar Sanchez and Shreyas Kumar. [n.d.]. Twitter Bullying Detection. Technical Report. https://www.researchgate.net/publication/267823748
  113. 10.1109/TSE.2014.2322358 · DOI · Paper page
  114. Shruthi and Prof Mangala C. 2017. A Framework for Automatic Detection and Prevention of Cyberbullying in Social Media. International Journal of Innovative Research in Computer and Communication Engineering, Vol. 5, 6 (2017), 86--90. www.ijircce.com
  115. Jim Sidanius and Felicia Pratto. 2001. Social dominance: An intergroup theory of social hierarchy and oppression .Cambridge University Press.
  116. 10.1145/3027063.3053169 · DOI · Paper page
  117. 10.1145/3341161.3342949 · DOI
  118. 10.1109/ASONAM.2016.7752342 · DOI
  119. 10.1111/j.1469--7610.2007.01846.x · doi:10.1111/j.1469--7610.2007.01846.x
  120. 10.1145/2808797.2809398 · DOI
  121. 10.1145/2743065.2743085 · DOI
  122. 10.1016/j.procs.2015.03.085 · DOI
  123. 10.1109/ISDA.2015.7489220 · DOI
  124. Harini Suresh and John V Guttag. 2019. A framework for understanding unintended consequences of machine learning. ArXiv, Vol. abs/1901.10002 (2019). · arXiv
  125. 10.1145/3290838 · DOI
  126. Robert S Tokunaga. 2010. Following you home from school: A critical review and synthesis of research on cyberbullying victimization. Computers in human behavior, Vol. 26, 3 (2010), 277--287.
  127. 10.1109/ASONAM.2018.8508294 · DOI
  128. 10.1007/978--3--319--13647--9_21 · doi:10.1007/978--3--319--13647--9_21
  129. Katrien Van Cleemput, Heidi Vandebosch, and Sara Pabian. 2014. Personal characteristics and contextual factors that determine “helping,”“joining in,” and “doing nothing” when witnessing cyberbullying. Aggressive behavior, Vol. 40, 5 (2014), 383--396.
  130. 10.1371/journal.pone.0203794 · DOI · Paper page
  131. 10.1016/j.tele.2014.04.002 · DOI
  132. Kiri Wagstaff. 2012. Machine Learning that Matters. Proceedings of 29th International Conference on Machine Learning., Vol. 1 (06 2012).
  133. Thilini Wijesiriwardene, Hale Inan, Ugur Kursuncu, Manas Gaur, Valerie L Shalin, Krishnaprasad Thirunarayan, Amit Sheth, and I Budak Arpinar. 2020. ALONE: A Dataset for Toxic Behavior Among Adolescents on Twitter. In International Conference on Social Informatics. Springer, 427--439.
  134. 10.1111/j.1469-7610.2012.02542.x · DOI
  135. Richmond Y Wong and Vera Khovanskaya. 2018. Speculative Design in HCI: From Corporate Imaginations to Critical Orientations. In New Directions in Third Wave Human-Computer Interaction: Volume 2-Methodologies. Springer, 175--202.
  136. 10.1145/3308560.3314196 · DOI
  137. 10.1109/ASONAM.2018.8508329 · DOI
  138. 10.1145/3308558.3313462 · DOI
  139. 10.1109/TAFFC.2016.2531682 · DOI · Paper page
  140. 10.1145/2833312.2849567 · DOI · Paper page
  141. Haoti Zhong, Hao Li, Anna Cinzia Squicciarini, Sarah Michele Rajtmajer, Christopher Griffin, David J Miller, and Cornelia Caragea. 2016. Content-Driven Detection of Cyberbullying on the Instagram Social Network.. In IJCAI. 3952--3958.
  142. Haoti Zhong, David J Miller, and Anna Squicciarini. 2018. Flexible Inference for Cyberbully Incident Detection. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 356--371.
  143. 10.1007/s10802-012-9678-8 · DOI · Paper page

Cited by

See all 79

Metadata

Authors
Kim, S., Razi, A., Stringhini, G., Wisniewski, P. J., & De Choudhury, M.
Venue
Proceedings of the ACM on Human-Computer Interaction
Published
2021-10-13
Volume
5
Issue
CSCW2
Pages
1-34
ISSN
2573-0142 (ISSN-L)
OpenAlex
W3206182128
Cited by
79
References
28
FWCI
1.36
Type
journal article
Language
en
Open access
bronze
Funders
National Science Foundation (IIP-1827700)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A Human-Centered Systematic Literature Review of Cyberbullying Detection Algorithms. https://app.cochat.ai/papers/10.1145/3476066
The paper
Kim, S., Razi, A., Stringhini, G., Wisniewski, P. J., & De Choudhury, M. (2021). A Human-Centered Systematic Literature Review of Cyberbullying Detection Algorithms. Proceedings of the ACM on Human-Computer Interaction. https://doi.org/10.1145/3476066

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.