A Human-Centered Systematic Literature Review of the Computational Approaches for Online Sexual Risk Detection
Open accessAbstract
In the era of big data and artificial intelligence, online risk detection has become a popular research topic. From detecting online harassment to the sexual predation of youth, the state-of-the-art in computational risk detection has the potential to protect particularly vulnerable populations from online victimization. Yet, this is a high-risk, high-reward endeavor that requires a systematic and human-centered approach to synthesize disparate bodies of research across different application domains, so that we can identify best practices, potential gaps, and set a strategic research agenda for leveraging these approaches in a way that betters society. Therefore, we conducted a comprehensive literature review to analyze 73 peer-reviewed articles on computational approaches utilizing text or meta-data/multimedia for online sexual risk detection. We identified sexual grooming (75%), sex trafficking (12%), and sexual harassment and/or abuse (12%) as the three types of sexual risk detection present in the extant literature. Furthermore, we found that the majority (93%) of this work has focused on identifying sexual predators after-the-fact, rather than taking more nuanced approaches to identify potential victims and problematic patterns that could be used to prevent victimization before it occurs. Many studies rely on public datasets (82%) and third-party annotators (33%) to establish ground truth and train their algorithms. Finally, the majority of this work (78%) mostly focused on algorithmic performance evaluation of their model and rarely (4%) evaluate these systems with real users. Thus, we urge computational risk detection researchers to integrate more human-centered approaches to both developing and evaluating sexual risk detection algorithms to ensure the broader societal impacts of this important work.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Razi et al. [27] reviewed various approaches for sexual risk detection from a human-centered view considering sex trafficking, sexual harassment, and sexual grooming.
“For instance, Razi et al. [27] reviewed various approaches for sexual risk detection from a human-centered view considering sex trafficking, sexual harassment, and sexual grooming”
From Online grooming detection: A comprehensive survey of child exploitation in chat logs (2023) · cites this paper as [27]
Most prior work has focused on detecting perpetrators' behaviors, leaving a limited understanding of how teens might respond to unwanted advances.
“Yet, most prior work has focused on detecting perpetrators' behaviors, leaving a limited understanding of how teens might respond to such unwanted advances.”
From From Vulnerable to Resilient: Examining Parent and Teen Perceptions on How to Respond to Unwanted Cybergrooming Advances (2026) · cites this paper as [76]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · see citing works
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · see citing works
Multimodal Machine Learning: A Survey and Taxonomy
Baltrusaitis, T., Ahuja, C., & Morency, L.-P. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2798607 · Cited by 5,018 · see citing works
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 1× · see citing works
“To support human involvement in such cases, researchers can consider approaches like “model cards” [102] to communicate to users about the limitations of the models and what the model outputs may mean.”
BackgroundFoundations of Machine Learning
(2012) · The MIT Press eBooks · doi:10.7551/mitpress/8291.003.0006 · Cited by 1,102 · see citing works
Word embeddings quantify 100 years of gender and ethnic stereotypes
Garg, N., Schiebinger, L., Jurafsky, D., & Zou, J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1720347115 · Cited by 1,065 · see citing works
Coronavirus disease 2019: What we know?
He, F., Deng, Y., & Li, W. (2020) · Journal of Medical Virology · doi:10.1002/jmv.25766 · Cited by 1,026 · see citing works
The Contradictory Influence of Social Media Affordances on Online Communal Knowledge Sharing
Majchrzak, A., Faraj, S. A., Kane, G. C., & Azad, B. (2013) · Journal of Computer-Mediated Communication · doi:10.1111/jcc4.12030 · Cited by 927 · see citing works
Interactive machine learning for health informatics: when do we need the human-in-the-loop?
Holzinger, A. (2016) · Brain Informatics · doi:10.1007/s40708-016-0042-6 · Cited by 906 · see citing works
Bender, E. M., & Friedman, B. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00041 · Cited by 854 · see citing works
Explainable Recommendation: A Survey and New Perspectives
Zhang, Y., & Chen, X. (2020) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000066 · Cited by 755 · see citing works
Metrics for explainable AI: challenges and prospects
Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2018) · arXiv · doi:10.48550/arxiv.1812.04608 · Cited by 518 · see citing works
Intelligibility and Accountability: Human Considerations in Context-Aware Systems
Bellotti, V., & Edwards, K. (2001) · Human–Computer Interaction · doi:10.1207/s15327051hci16234_05 · Cited by 471 · see citing works
Andalibi, N., Haimson, O. L., De Choudhury, M., & Forte, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858096 · Cited by 378 · see citing works
Theory and Applications of Ontology: Computer Applications
Poli, R., Kameas, A. D., & Healy, M. J. R. (2010) · Springer eBooks · doi:10.1007/978-90-481-8847-5 · Cited by 364 · see citing works
Chen, M., Mao, S., Zhang, Y., & Leung, V. C. M. (2014) · SpringerBriefs in computer science · doi:10.1007/978-3-319-06245-7 · Cited by 247 · see citing works
Lessons from archives: strategies for collecting sociocultural data in machine learning
Jo, E. S., & Gebru, T. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372829 · Cited by 240 · see citing works
Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions
Alkhatib, A., & Bernstein, M. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300760 · Cited by 188 · see citing works
Who is the "Human" in Human-Centered Machine Learning
Chancellor, S., Baumer, E., & de Choudhury, M. (2019) · Proc. ACM Hum. Comput. Interact. · doi:10.1145/3359249 · Cited by 187 · see citing works
Entrapping the innocent: Toward a theory of child sexual predators’ luring communication
Olson, L. N., Daggs, J. L., Ellevold, B. L., & Rogers, T. K. K. (2007) · Communication Theory · doi:10.1111/j.1468-2885.2007.00294.x · Cited by 160 · see citing works
Black, P. J., Wollis, M., Woodworth, M., & Hancock, J. T. (2015) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2014.12.004 · Cited by 153 · see citing works
Toward situated interventions for algorithmic equity: lessons from the field
Katell, M., Young, M., Dailey, D., Herman, B., Guetler, V., Tam, A., Bintz, C., Raz, D., & Krafft, P. M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372874 · Cited by 153 · see citing works
Human centered systems in the perspective of organizational and social informatics
Kling, R., & Star, S. L. (1998) · ACM SIGCAS Computers and Society · doi:10.1145/277351.277356 · Cited by 135 · see citing works
Krieger, M. A. (2017) · Trauma, Violence, & Abuse · doi:10.1177/1524838016659486 · Cited by 128 · see citing works
Toward human-centered algorithm design
Baumer, E. P. (2017) · Big Data & Society · doi:10.1177/2053951717718854 · Cited by 125 · see citing works
Community Detection in Multiplex Networks
Magnani, M., Hanteer, O., Interdonato, R., Rossi, L., & Tagarelli, A. (2021) · ACM Computing Surveys · doi:10.1145/3444688 · Cited by 123 · see citing works
Learning to Identify Internet Sexual Predation
McGhee, I., Bayzick, J., Kontostathis, A., Edwards, L., McBride, A., & Jakubowski, E. (2011) · International Journal of Electronic Commerce · doi:10.2753/jec1086-4415150305 · Cited by 106 · see citing works
Victims of Trafficking and Violence Protection Act 2000:
(2021) · Amsterdam University Press eBooks · doi:10.2307/j.ctv28hj3cg.30 · Cited by 95 · see citing works
Identifying and Measuring Annotator Bias Based on Annotators’ Demographic Characteristics
Kuwatly, H. A., Wich, M., & Groh, G. (2020) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2020.alw-1.21 · Cited by 88 · see citing works
Cyber-aggression, Cyberbullying, and Cyber-grooming: A Survey and Research Challenges
Mladenović, M., Ošmjanski, V., & Stanković, S. V. (2022) · ACM Computing Surveys · doi:10.1145/3424246 · Cited by 79 · see citing works
Detecting child grooming behaviour patterns on social media
Cano, A. E., Fernandez, M., & Alani, H. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-13734-6_30 · Cited by 79 · see citing works
Detecting predatory conversations in social media by deep convolutional neural networks
Ebrahimi, M., Suen, C. Y., & Ormandjieva, O. (2016) · Digital Investigation · doi:10.1016/j.diin.2016.07.001 · Cited by 78 · see citing works
Ibanez, M., & Suthers, D. D. (2014) · Proceedings of the ... Annual Hawaii International Conference on System Sciences/Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.1109/hicss.2014.200 · Cited by 77 · see citing works
Toward spotting the pedophile telling victim from predator in text chats
Pendar, N. (2007) · International Conference on Semantic Computing (ICSC 2007) · doi:10.1109/icsc.2007.32 · Cited by 77 · see citing works
Tackling Online Abuse: A Survey of Automated Abuse Detection MethodsPreprint
Mishra, P., Yannakoudakis, H., & Shutova, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.06024 · Cited by 74 · see citing works
Racial Equity in Algorithmic Criminal Justice
Huq, A. Z. (2018) · eYLS (Yale Law School) · Cited by 69 · see citing works
Krig, S. (2016) · Springer eBooks · doi:10.1007/978-3-319-33762-3 · Cited by 65 · see citing works
Sounds of Silence Breakers: Exploring Sexual Violence on Twitter
Khatua, A., Cambria, E., & Khatua, A. (2018) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1109/asonam.2018.8508576 · Cited by 61 · see citing works
Lorenzo-Dus, N., Kinzel, A., & Cristofaro, M. D. (2019) · Journal of Pragmatics · doi:10.1016/j.pragma.2019.09.010 · Cited by 61 · see citing works
Hartikainen, H., Razi, A., & Wiśniewski, P. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449116 · Cited by 60 · see citing works
Self-citationThe Trafficking Victims Protection Act of 2000
Lobasz, J. K. (2018) · Springer eBooks · doi:10.1007/978-3-319-91737-5_3 · Cited by 53 · see citing works
On Cost-Effective Incentive Mechanisms in Microtask Crowdsourcing
Gao, Y., Chen, Y., & Liu, K. J. R. (2014) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2014.2298361 · Cited by 50 · see citing works
Deep Reinforcement Learning-based Text Anonymization against Private-Attribute Inference
Mosallanezhad, A., Beigi, G., & Liu, H. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1240 · Cited by 46 · see citing works
Bernard, E. J., McClelland, A., Cardell, B., Chung, C., Castro-Bojorquez, M., French, M. A., Hursey, D., Khanna, N., Minalga, B., Spieldenner, A., et al. (2020) · The American Journal of Bioethics · doi:10.1080/15265161.2020.1809550 · Cited by 43 · see citing works
SafeCity: Understanding Diverse Forms of Sexual Harassment Personal Stories
Karlekar, S., & Bansal, M. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1303 · Cited by 43 · see citing works
Grooming detection using fuzzy-rough feature selection and text classification
Zuo, Z., Li, J., Anderson, P., Yang, L., & Naik, N. (2018) · 2018 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz-ieee.2018.8491591 · Cited by 39 · see citing works
Speak up, Fight Back! Detection of Social Media Disclosures of Sexual Harassment
Chowdhury, A. G., Sawhney, R., Mathur, P., Mahata, D., & Shah, R. R. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop (NAACL SRW) · doi:10.18653/v1/n19-3018 · Cited by 38 · see citing works
Wordnet-Based Criminal Networks Mining for Cybercrime Investigation
Iqbal, F., Fung, B., Debbabi, M., Batool, R., & Marrington, A. (2019) · IEEE Access · doi:10.1109/access.2019.2891694 · Cited by 37 · see citing works
Ensemble Method for Sexual Predators Identification in Online Chats
Fauzi, M. A., & Bours, P. (2020) · 2020 8th International Workshop on Biometrics and Forensics (IWBF) · doi:10.1109/iwbf49977.2020.9107945 · Cited by 34 · see citing works
Recognizing Predatory Chat Documents using Semi-supervised Anomaly Detection
Ebrahimi, M., Suen, C. Y., Ormandjieva, O., & Krzyżak, A. (2016) · Electronic Imaging · doi:10.2352/issn.2470-1173.2016.17.drr-063 · Cited by 33 · see citing works
Ngejane, C. H., Mabuza-Hocquet, G., Eloff, J. H. P., & Lefophane, S. (2018) · International Conference on Artificial Intelligence, Big Data, Computing and Data Communication Systems (icABCD) · doi:10.1109/icabcd.2018.8465413 · Cited by 33 · see citing works
Hillstrom, L. C. (2018) · doi:10.5040/9798400605062 · Cited by 32 · see citing works
Mitigating Bias in Session-based Cyberbullying Detection: A Non-Compromising Approach
Cheng, L., Mosallanezhad, A., Silva, Y. N., Hall, D. L., & Liu, H. (2021) · doi:10.18653/v1/2021.acl-long.168 · Cited by 31 · see citing works
Zambrano, P., Torres, J., Tello-Oquendo, L., Jacome, R., Benalcázar, M. E., Andrade, R., & Fuertes, W. (2019) · IEEE Access · doi:10.1109/access.2019.2942805 · Cited by 31 · see citing works
Data integration from open internet sources to combat sex trafficking of minors
Wang, H., Cai, C., Philpot, A., et al. (2012) · Conference on Digital Government Research · doi:10.1145/2307729.2307769 · Cited by 31 · see citing works
Detecting predatory behavior in game chats
Cheong, Y.-G., Jensen, A. K., Gudnadottir, E. R., Bae, B.-C., & Togelius, J. (2015) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2015.2424932 · Cited by 30 · see citing works
Negobot: A Conversational Agent Based on Game Theory for the Detection of Paedophile Behaviour
Laorden, C., Galán-García, P., Santos, I., Sanz, B., Hidalgo, J. M. G., & Bringas, P. G. (2012) · Advances in intelligent systems and computing · doi:10.1007/978-3-642-33018-6_27 · Cited by 28 · see citing works
Utilizing document classification for grooming attack recognition
Michalopoulos, D., & Mavridis, I. K. (2011) · Proceedings - IEEE Symposium on Computers and Communications/IEEE Symposium on Computers and Communications · doi:10.1109/iscc.2011.5983950 · Cited by 28 · see citing works
Barber, C. S., & Bettez, S. C. (2014) · Journal of the Association for Information Systems · Cited by 27 · see citing works
Detection of Cyber Grooming in Online Conversation
Bours, P., & Kulsrud, H. (2019) · 2019 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs47025.2019.9035090 · Cited by 27 · see citing works
Analyzing Chat Conversations of Pedophiles with Temporal Relational Semantic Systems
Elzinga, P., Wolff, K. E., & Poelmans, J. (2012) · European Intelligence and Security Informatics Conference · doi:10.1109/eisic.2012.12 · Cited by 26 · see citing works
Li, L., Simek, O., Lai, A., Daggett, M., Dagli, C. K., & Jones, C. (2018) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2018.8622189 · Cited by 26 · see citing works
You Don't Know How I Feel: Insider-Outsider Perspective Gaps in Cyberbullying Risk Detection
Kim, S.-H., Razi, A., Stringhini, G., Wiśniewski, P., & Choudhury, M. D. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v15i1.18061 · Cited by 24 · see citing works
Self-citationBijker, W. E., Hughes, T. P., & Pinch, T. (1987) · Cited by 23 · see citing works
Towards Automated Sexual Violence Report Tracking
Hassan, N., Poudel, A., Hale, J. G., Hubacek, C., Huq, K. T., Santu, S. K. K., & Ahmed, S. I. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7296 · Cited by 23 · see citing works
An Intelligent Online Grooming Detection System Using AI Technologies
Anderson, P., Zuo, Z., Yang, L., & Qu, Y. (2019) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz-ieee.2019.8858973 · Cited by 21 · see citing works
A biology-inspired, data mining framework for extracting patterns in sexual cyberbullying data
Potha, N., Μaragoudakis, M., & Lyras, D. P. (2016) · Knowledge-Based Systems · doi:10.1016/j.knosys.2015.12.021 · Cited by 21 · see citing works
Gillam, L., & Vartapetiance, A. (2012) · Surrey Research Insight Open Access (The University of Surrey) · Cited by 20 · see citing works
Quantifying paedophile queries in a large p2p system
Latapy, M., Magnien, C., & Fournier, R. (2011) · 2011 Proceedings IEEE INFOCOM · doi:10.1109/infcom.2011.5935191 · Cited by 20 · see citing works
Chat Analysis Triage Tool: Differentiating contact-driven vs. fantasy-driven child sex offenders
Seigfried‐Spellar, K. C., Rogers, M., Rayz, J. T., Yang, S.-F., Misra, K., & Ringenberg, T. R. (2019) · Forensic Science International · doi:10.1016/j.forsciint.2019.02.028 · Cited by 20 · see citing works
Finding Participants in a Chat: Authorship Attribution for Conversational Documents
Inches, G., Harvey, M., & Crestani, F. (2013) · International Conference on Social Computing · doi:10.1109/socialcom.2013.45 · Cited by 19 · see citing works
Predatory Conversation Detection
Borj, P. R., & Bours, P. (2019) · 2019 International Conference on Cyber Security for Emerging Technologies (CSET) · doi:10.1109/cset.2019.8904885 · Cited by 19 · see citing works
A Step Towards Detecting Online Grooming -- Identifying Adults Pretending to be Children
Ashcroft, M., Kaati, L., & Meyer, M. (2015) · 2015 European Intelligence and Security Informatics Conference · doi:10.1109/eisic.2015.41 · Cited by 18 · see citing works
Detecting Pedophile Activity in BitTorrent Networks
Rutgaizer, M., Shavitt, Y., Vertman, O., et al. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-28537-0_11 · Cited by 18 · see citing works
Detecting online child grooming conversation
Gunawan, F. E., Ashianti, L., Candra, S., & Soewito, B. (2016) · 2016 11th International Conference on Knowledge, Information and Creativity Support Systems (KICSS) · doi:10.1109/kicss.2016.7951413 · Cited by 17 · see citing works
Sexual Harassment Story Classification and Key Information Identification
Liu, Y., Li, Q., Liu, X., et al. (2019) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3357384.3358146 · Cited by 17 · see citing works
Identifying Cyber Predators through Forensic Authorship Analysis of Chat Logs
Amuchi, F., Al-Nemrat, A., Alazab, M., & Layton, R. (2012) · Cybercrime and Trustworthy Computing Workshop (CTC) · doi:10.1109/ctc.2012.16 · Cited by 16 · see citing works
Context-specific Language Modeling for Human Trafficking Detection from Online Advertisements
Esfahani, S. S., Cafarella, M., Pouyan, M. B., DeAngelo, G. J., Eneva, E., & Fano, A. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1114 · Cited by 16 · see citing works
Tariq, M. U., Razi, A., Badillo-Urquiola, K. A., & Wiśniewski, P. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-22636-7_6 · Cited by 16 · see citing works
Self-citationDetecting sex trafficking circuits in the U.S. through analysis of online escort advertisements
Ibanez, M., & Gazan, R. (2016) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1109/asonam.2016.7752344 · Cited by 15 · see citing works
Child Abuse and Domestic Abuse: Content and Feature Analysis from Social Media Disclosures
Subramani, S., Wang, H., Islam, M. R., Ulhaq, A., & O’Connor, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-92013-9_14 · Cited by 15 · see citing works
Combining Predation Heuristics and Chat-Like Features in Sexual Predator Identification.
Hidalgo, J. M. G., & Díaz, A. A. C. (2012) · CLEF (Online Working Notes/Labs/Workshop) · Cited by 14 · see citing works
Planning for the Known Unknown: Machine Learning for Human Healthcare Systems
Chen, J. H., & Verghese, A. (2020) · The American Journal of Bioethics · doi:10.1080/15265161.2020.1822674 · Cited by 13 · see citing works
Identifying Victim Blaming Language in Discussions about Sexual Assaults on Twitter
Suvarna, A., Bhalla, G., Kumar, S., & Bhardwaj, A. (2020) · International Conference on Social Media and Society (SMSociety) · doi:10.1145/3400806.3400825 · Cited by 13 · see citing works
Not So Cute but Fuzzy: Estimating Risk of Sexual Predation in Online Conversations
Ringenberg, T. R., Misra, K., & Rayz, J. T. (2019) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2019.8914528 · Cited by 13 · see citing works
Early Text Classification Using Multi-Resolution Concept Representations
López-Monroy, A. P., Gonzalez, F. A., Montes, M., Escalante, H. J., & Solorio, T. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1110 · Cited by 13 · see citing works
Self-citation"Our Little Secret": pinpointing potential predators
Vartapetiance, A., & Gillam, L. (2014) · Security Informatics · doi:10.1186/s13388-014-0003-7 · Cited by 11 · see citing works
Detection and prevention measures for cyberbullying and online grooming
Upadhyay, A., Chaudhari, A., Arunesh, Ghale, S., & Pawar, S. S. (2017) · International Conference on Inventive Systems and Control (ICISC) · doi:10.1109/icisc.2017.8068605 · Cited by 11 · see citing works
Quantum-Inspired Density Matrix Encoder for Sexual Harassment Personal Stories Classification
Yan, P., Li, L., Chen, W., et al. (2019) · IEEE International Conference on Intelligence and Security Informatics (ISI) · doi:10.1109/isi.2019.8823281 · Cited by 10 · see citing works
Detecting Predatory Behaviour from Online Textual Chats
Pandey, S. J., Klapaftis, I. P., & Manandhar, S. (2012) · Communications in computer and information science · doi:10.1007/978-3-642-30721-8_27 · Cited by 10 · see citing works
Misra, K., Devarapalli, H., Ringenberg, T. R., & Rayz, J. T. (2019) · 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC) · doi:10.1109/smc.2019.8914323 · Cited by 9 · see citing works
Analyzing Abusive Text Messages to Detect Digital Dating Abuse
Roy, T., McClendon, J. L., & Hodges, L. F. (2018) · IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi.2018.00039 · Cited by 9 · see citing works
Agent-mediated information exchange: Child safety online
MacFarlane, K., & Holmes, V. (2009) · 2009 International Conference on Management and Service Science · doi:10.1109/icmss.2009.5302027 · Cited by 8 · see citing works
Detecting covert sex trafficking networks in virtual markets
Ibanez, M., & Suthers, D. D. (2016) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1109/asonam.2016.7752340 · Cited by 8 · see citing works
A Framework for Improved Adolescent and Child Safety in MMOs
Penna, L., Clark, A., & Mohay, G. (2010) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1109/asonam.2010.66 · Cited by 8 · see citing works
Artemis: protection from sexual exploitation attacks via SMS
Michalopoulos, D., Papadopoulos, E., & Mavridis, I. (2012) · 2012 16th Panhellenic Conference on Informatics · doi:10.1109/pci.2012.46 · Cited by 6 · see citing works
Virtual indicators of sex trafficking to identify potential victims in online advertisements
Ibanez, M., & Gazan, R. (2016) · Advances in Social Networks Analysis and Mining · doi:10.5555/3192424.3192579 · Cited by 6 · see citing works
Children's Perspectives on Human Sex Trafficking Prevention Education
Badillo-Urquiola, K. A., Razi, A., Edwards, J., et al. (2019) · doi:10.1145/3323994.3369889 · Cited by 6 · see citing works
Self-citationCallejas-Rodríguez, Á., Villatoro-Tello, E., Meza, I., & Ramírez-de-la-Rosa, G. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-45510-5_61 · Cited by 6 · see citing works
Exploring automatic identification of fantasy-driven and contact-driven sexual solicitors
Ringenberg, T., Misra, K., Seigfried-Spellar, K. C., & Taylor Rayz, J. (2019) · 2019 Third IEEE International Conference on Robotic Computing (IRC) · doi:10.1109/irc.2019.00110 · Cited by 6 · see citing works
Privacy Protection Protocol in Social Networks Based on Sexual Predators Detection
Dhouioui, Z., & Akaichi, J. (2016) · doi:10.1145/2896387.2896448 · Cited by 5 · see citing works
A framework for digital forensics analysis based on semantic role labeling
Barreira, R., Pinheiro, V. C. M., & Furtado, V. (2017) · IEEE International Conference on Intelligence and Security Informatics (ISI) · doi:10.1109/isi.2017.8004876 · Cited by 5 · see citing works
Kostakos, P., Sprachalova, L., Pandya, A., et al. (2018) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1109/asonam.2018.8508276 · Cited by 4 · see citing works
Empirical UX Evaluation: UX Goals, Metrics, and Targets
Hartson, R., & Pyla, P. S. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-805342-3.00022-9 · Cited by 2 · see citing works
Analysis of User Behavior on Private Chat System
Toriumi, F., Nakanishi, T., Tashiro, M., & Eguchi, K. (2015) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat.2015.49 · Cited by 2 · see citing works
Designing Light Filters to Detect Skin Using a Low-powered Sensor
Tariq, M. U., Ghosh, A. K., Badillo-Urquiola, K. A., Jha, A., Koppal, S. J., & Wiśniewski, P. (2018) · IEEE SoutheastCon · doi:10.1109/secon.2018.8479027 · Cited by 2 · see citing works
Mining bipartite graphs to improve semantic pedophile activity detection
Fournier, R., & Danisch, M. (2014) · IEEE International Conference on Research Challenges in Information Science (RCIS) · doi:10.1109/rcis.2014.6861035 · Cited by 1 · see citing works
Automated framework to improve users' awareness on Online Social Networks
Barakat, R., Abufardeh, S., & Magel, K. I. (2016) · IEEE International Conference on Electro Information Technology · doi:10.1109/eit.2016.7535278 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 48
- Dasha Bogdanova, Paolo Rosso, and Thamar Solorio. 2012. Modelling Fixated Discourse in Chats with Cyberpedophiles. In Proceedings of the Workshop on Computational Approaches to Deception Detection (EACL 2012). Association for Computational Linguistics, Stroudsburg, PA, USA, 86-90. http://dl.acm.org/citation.cfm?id=2388616.2388629 event- place: Avignon, France.
- Dasha Bogdanova, Paolo Rosso, and Thamar Solorio. 2012. On the Impact of Sentiment and Emotion Based Features in Detecting Online Sexual Predators. In Proceedings of the 3rd Workshop in Computational Approaches to Subjectivity and Sentiment Analysis (WASSA '12). Association for Computational Linguistics, Stroudsburg, PA, USA, 110-118. http://dl.acm.org/citation.cfm?id=2392963.2392986
- Dasha Bogdanova, Paolo Rosso, and Thamar Solorio. 2014. Exploring high-level features for detecting cyberpedophilia. Computer speech & language 28, 1 (2014), 108-120.
- Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in Neural Information Processing Systems (2016).
- Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. PMLR, 77-91.
- Xue-Wen Chen and Xiaotong Lin. 2014. Big data deep learning: challenges and perspectives. IEEE access 2 (2014), 514-525.
- Ming Ming Chiu, Kathryn C Seigfried-Spellar, and Tatiana R Ringenberg. 2018. Exploring detection of contact vs. fantasy online sexual offenders in chats with minors: Statistical discourse analysis of self-disclosure and emotion words. Child abuse & neglect 81 (2018), 128-138.
- Mohammadreza Ebrahimi and Mohammadreza Ebrahimi. 2016. Automatic Identification of Online Predators in Chat Logs by Anomaly Detection and Deep Learning. Master's thesis. Concordia University. https://spectrum.library. concordia.ca/981404/
- Hugo Jair Escalante, Esaú Villatoro-Tello, Antonio Juárez, Manuel Montes-y Gómez, and Luis Villaseñor. 2013. Sexual predator detection in chats with chained classifiers. In Proceedings of the 4th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis. Association for Computational Linguistics, Atlanta, Georgia, 46-54. http://www.aclweb.org/anthology/W13-1607
- Joseph L Fleiss and Jacob Cohen. 1973. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and psychological measurement 33, 3 (1973), 613-619.
- Nicola Henry and Anastasia Powell. 2018. Technology-facilitated sexual violence: A literature review of empirical research. Trauma, violence, & abuse 19, 2 (2018), 195-208.
- Michael Hind, Sameep Mehta, Aleksandra Mojsilovic, Ravi Nair, Karthikeyan Natesan Ramamurthy, Alexandra Olteanu, and Kush R Varshney. 2018. Increasing trust in ai services through supplier's declarations of conformity. arXiv preprint arXiv:1808.07261 18 (2018), 2813-2869. · arXiv
- Giacomo Inches and Fabio Crestani. 2012. Overview of the International Sexual Predator Identification Competition at PAN-2012.. In CLEF (Online working notes/labs/workshop), Vol. 30.
- Alejandro Jaimes, Daniel Gatica-Perez, Nicu Sebe, and Thomas S Huang. 2007. Guest Editors' Introduction: Human- Centered Computing-Toward a Human Revolution. Computer 40, 5 (2007), 30-34.
- Jinhwa Kim, Yoon Jo Kim, Mitra Behzadi, and Ian G. Harris. 2020. Analysis of Online Conversations to Detect Cyberpredators Using Recurrent Neural Networks. In Proceedings for the First International Workshop on Social Threats in Online Conversations: Understanding and Management. European Language Resources Association, Marseille, France, 15-20. https://www.aclweb.org/anthology/2020.stoc-1.3
- April Kontostathis. 2009. ChatCoder: Toward the Tracking and Categorization of Internet Predators. In Proc. Text Mining Workshop 2009 Held in Conjunction with the Ninth Siam International Conference on Data Mining (sdm 2009). Sparks, Nv. May 2009.
- April Kontostathis, Lynne Edwards, Jen Bayzick, Amanda Leatherman, and Kristina Moore. 2009. Comparison of rule-based to human analysis of chat logs. communication theory 8, 2 (2009).
- Richard J Light. 1971. Measures of response agreement for qualitative data: some generalizations and alternatives. Psychological bulletin 76, 5 (1971), 365.
- Sonia Livingstone and Leslie Haddon. 2008. Risky experiences for children online: charting European research on children and the Internet. Children and Society 22 (July 2008), 314-323. http://www.wiley.com/bw/journal.asp?ref= 0951-0605
- Nora McDonald, Karla Badillo-Urquiola, Morgan G Ames, Nicola Dell, Elizabeth Keneski, Manya Sleeper, and Pamela J Wisniewski. 2020. Privacy and Power: Acknowledging the Importance of Privacy Research and Design for Vulnerable Populations. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 1-8.
- Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2019. A survey on bias and fairness in machine learning. arXiv preprint arXiv:1908.09635 (2019). · arXiv
- Md. Waliur Rahman Miah, John Yearwood, and Sid Kulkarni. 2011. Detection of child exploiting chats from a mixed chat dataset as a text classification task. In Proceedings of the Australasian Language Technology Association Workshop 2011. Canberra, Australia, 157-165. https://www.aclweb.org/anthology/U11-1020
- Sina Mohseni, Niloofar Zarei, and Eric D Ragan. 2018. A multidisciplinary survey and framework for design and evaluation of explainable AI systems. arXiv preprint arXiv:1811.11839 (2018). · arXiv
- Alexander Panchenko, Richard Beaufort, Hubert Naets, and Cédrick Fairon. 2013. Towards Detection of Child Sexual Abuse Media: Categorization of the Associated Filenames. In Advances in Information Retrieval (Lecture Notes in Computer Science), Pavel Serdyukov, Pavel Braslavski, Sergei O. Kuznetsov, Jaap Kamps, Stefan Rüger, Eugene Agichtein, Ilya Segalovich, and Emine Yilmaz (Eds.). Springer Berlin Heidelberg, 776-779.
- James W Pennebaker, Martha E Francis, and Roger J Booth. 2001. Linguistic inquiry and word count: LIWC 2001. Mahway: Lawrence Erlbaum Associates 71, 2001 (2001), 2001.
- Ralph M Perhac Jr. 1996. Defining risk: Normative considerations. Human and Ecological Risk Assessment 2, 2 (1996), 381-392.
- Anthony T. Pinter, Pamela J. Wisniewski, Heng Xu, Mary Beth Rosson, and Jack M. Caroll. 2017. Adolescent Online Safety: Moving Beyond Formative Evaluations to Designing Solutions for the Future. In Proceedings of the 2017 Conference on Interaction Design and Children -IDC '17. ACM Press, Stanford, California, USA, 352-357. https://doi.org/10.1145/3078072.3079722 · DOI · Paper page
- Kate Nalini Murbade Arun Pokharkar Anuja, Shelake Shubham. 2015. Protective shield for social networks to defend cyberbullying and online grooming attacks. In International Journal of Advances in Electronics and Computer Science, Vol. 2.
- Hady Pranoto, Fergyanto E Gunawan, and Benfano Soewito. 2015. Logistic models for classifying online grooming conversation. Procedia Computer Science 59 (2015), 357-365.
- Francisco Rangel, Paolo Rosso, Moshe Koppel, Efstathios Stamatatos, and Giacomo Inches. 2013. Overview of the author profiling task at pan 2013. In CLEF Conference on Multilingual and Multimodal Information Access Evaluation. CELCT, 352-365.
- Afsaneh Razi, Zainab Agha, Neeraj Chatlani, and Pamela Wisniewski. 2020. Privacy Challenges for Adolescents as a Vulnerable Population. In Networked Privacy Workshop of the 2020 CHI Conference on Human Factors in Computing Systems.
- Afsaneh Razi, Karla Badillo-Urquiola, and Pamela J. Wisniewski. 2020. Let's Talk about Sext: How Adolescents Seek Support and Advice about Their Online Sexual Experiences. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, Honolulu, HI, USA, 1-13. https://doi.org/10.1145/3313831.3376400 · DOI · Paper page
- Afsaneh Razi, Seunghyun Kim, Ashwaq Alsoubai, Xavier Caddle, Shiza Ali, Munmun De Choudhury, Pamela Wis- niewski, et al. 2021. Teens at the Margin: Artificially Intelligent Technology for Promoting Adolescent Online Safety. In ACM Conference on Human Factors in Computing Systems (CHI 2021)/Artificially Intelligent Technology for the Margins: A Multidisciplinary Design Agenda Workshop.
- Afsaneh Razi, Seunghyun Kim, Munmun De Choudhury, and Pamela Wisniewski. 2019. Ethical considerations for adolescent online risk detection AI systems. In Good Systems: Ethical AI for CSCW (The 22nd ACM Conference on Computer-Supported Cooperative Work and Social Computing).
- Hugo Rosa, David Matos, Ricardo Ribeiro, Luisa Coheur, and João P Carvalho. 2018. A "deeper" look at detecting cyberbullying in social networks. In 2018 international joint conference on neural networks (IJCNN). IEEE, 1-8.
- Michael J Rosenfeld, Reuben J Thomas, and Sonia Hausen. 2019. Disintermediating your friends: How online dating in the United States displaces other ways of meeting. Proceedings of the National Academy of Sciences 116, 36 (2019), 17753-17758.
- Dominik Sacha, Michael Sedlmair, Leishi Zhang, John Aldo Lee, Daniel Weiskopf, Stephen North, and Daniel Keim. 2016. Human-centered machine learning through interactive visualization. In ESANN 2016: 24th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning Bruges, Belgium April 27-28-29, 2016 Proceedings. ESANN, Bruges, Belgium, 641-646. https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2016- 166.pdf
- Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2020. A Human-Centered Review of Algorithms used within the US Child Welfare System. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-15.
- Daniel Ribeiro Silva, Andrew Philpot, Abhishek Sundararajan, Nicole Marie Bryan, and Eduard Hovy. 2014. Data integration from open internet sources and network detection to combat underage sex trafficking. In Proceedings of the 15th Annual International Conference on Digital Government Research. 86-90.
- Brian H Spitzberg and Gregory Hoobler. 2002. Cyberstalking and the technologies of interpersonal terrorism. New media & society 4, 1 (2002), 71-92.
- Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs. arXiv preprint arXiv:2101.09824 (2021). · arXiv
- Ashley Marie Walker, Yaxing Yao, Christine Geeng, Roberto Hoyle, and Pamela Wisniewski. 2019. Moving beyond 'one size fits all': research considerations for working with vulnerable populations. Interactions 26, 6 (Oct. 2019), 34-39. https://doi.org/10.1145/3358904 · DOI
- Terry Winograd. 2006. Shifting viewpoints: Artificial intelligence and human-computer interaction. Artificial intelligence 170, 18 (2006), 1256-1258.
- Jacob O Wobbrock and Julie A Kientz. 2016. Research contributions in human-computer interaction. interactions 23, 3 (2016), 38-44.
- Michele L Ybarra and Kimberly J Mitchell. 2008. How risky are social networking sites? A comparison of places online where youth sexual solicitation and harassment occurs. Pediatrics 121, 2 (2008), e350-e357.
- Aleš Završnik. 2019. Algorithmic justice: Algorithms and big data in criminal justice settings. European Journal of Criminology (2019), 1477370819876762.
- Jie M Zhang, Mark Harman, Lei Ma, and Yang Liu. 2020. Machine learning testing: Survey, landscapes and horizons. IEEE Transactions on Software Engineering (2020).
- Ángel Callejas-Rodríguez, Esaú Villatoro-Tello, Ivan Meza, and Gabriela Ramírez-de-la Rosa. 2016. From Dialogue Corpora to Dialogue Systems: Generating a Chatbot with Teenager Personality for Preventing Cyber-Pedophilia. In Text, Speech, and Dialogue (Lecture Notes in Computer Science), Petr Sojka, Aleš Horák, Ivan Kopeček, and Karel Pala (Eds.). Springer International Publishing, 531-539.
Cited by
Sorted by citations
Young, J., Jawara, L. M., Nguyen, D. N., Daly, B., Huh-Yoo, J., & Razi, A. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642574 · Cited by 59 · see citing works
Self-citationRazi, A., Alsoubai, A., Kim, S., Ali, S., Stringhini, G., De Choudhury, M., & Wisniewski, P. J. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579522 · Cited by 52 · see citing works
Self-citationOnline grooming detection: A comprehensive survey of child exploitation in chat logs
Borj, P. R., Raja, K., & Bours, P. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.110039 · Cited by 38 · see citing works
Understanding Teen Overreliance on AI Companion Chatbots Through Self-Reported Reddit Narratives
Namvarpour, M. (., Brofsky, B., Medina, J. Y., Akter, M., & Razi, A. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790597 · Cited by 24 · see citing works
Self-citationZhang, X., Akter, M., An, H., Liu, M., Zhang, Q., Huang, L., Cho, J.-H., J. Wisniewski, P., & Lee, S. W. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790949
“To combat cybergrooming, prior work has focused on surveillance and detection, using computational techniques such as machine learning (ML), natural language processing (NLP), and data mining to detect perpetrator behavior [32, 34, 59, 72, 76–78].”
Self-citation
- Authors
- Razi, A., Kim, S., Alsoubai, A., Stringhini, G., Solorio, T., De Choudhury, M., & Wisniewski, P. J.
- Affiliations
- University of Central Florida (US)
- Georgia Institute of Technology, Atlanta, GA, USA
- Boston University (US)
- University of Houston (US)
- University of Central Florida, Orlando, FL, USA
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2021-10-13
- Volume
- 5
- Issue
- CSCW2
- Pages
- 1-38
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W3207701354
- Cited by
- 95
- References
- 109
- Citations by year
- 2026: 13 · 2025: 16 · 2024: 23 · 2023: 16 · 2022: 27
- FWCI
- 1.20
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- OpenBU (Boston University) (acceptedVersion)
- Funders
- William T. Grant Foundation (187941)
- NSF (1827700)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- sexual grooming, sex trafficking, online harassment, child sexual abuse, cyber victimization, human-centered evaluation, algorithmic bias, vulnerable populations, research agenda, multimedia analysis, ground truth annotation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
