Helpful or Harmful? Exploring the Efficacy of Large Language Models for Online Grooming Prevention
Open accessAbstract
Powerful generative Large Language Models (LLMs) are becoming popular tools amongst the general public as question-answering systems, and are being utilised by vulnerable groups such as children. With children increasingly interacting with these tools, it is imperative for researchers to scrutinise the safety of LLMs, especially for applications that could lead to serious outcomes, such as online child safety queries. In this paper, the efficacy of LLMs for online grooming prevention is explored both for identifying and avoiding grooming through advice generation, and the impact of prompt design on model performance is investigated by varying the provided context and prompt specificity. In results reflecting over 6,000 LLM interactions, we find that no models were clearly appropriate for online grooming prevention, with an observed lack of consistency in behaviours, and potential for harmful answer generation, especially from open-source models. We outline where and how models fall short, providing suggestions for improvement, and identify prompt designs that heavily altered model performance in troubling ways, with findings that can be used to inform best practice usage guides.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Most prior work has focused on detecting perpetrators' behaviors, leaving a limited understanding of how teens might respond to unwanted advances.
“Yet, most prior work has focused on detecting perpetrators' behaviors, leaving a limited understanding of how teens might respond to such unwanted advances.”
From From Vulnerable to Resilient: Examining Parent and Teen Perceptions on How to Respond to Unwanted Cybergrooming Advances (2026) · cites this paper as [72]
Cites
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361 · see citing works
Pre-trained models for natural language processing: A survey
Qiu, X., Sun, T., Xu, Y., Shao, Y., Dai, N., & Huang, X. (2020) · Science China Technological Sciences · doi:10.1007/s11431-020-1647-3 · Cited by 1,591 · see citing works
Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey
Min, B., Ross, H., Sulem, E., Veyseh, A. P. B., Nguyen, T. H., Sainz, O., Agirre, E., Heintz, I., & Roth, D. (2024) · ACM Computing Surveys · doi:10.1145/3605943 · Cited by 1,387 · see citing works
Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts
Zamfirescu-Pereira, J. D., Wong, R. Y., Hartmann, B., & Yang, Q. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581388 · Cited by 893 · see citing works
Prompt Engineering as an Important Emerging Skill for Medical Professionals: Tutorial
Meskó, B. (2023) · Journal of Medical Internet Research · doi:10.2196/50638 · Cited by 709 · see citing works
Design Guidelines for Prompt Engineering Text-to-Image Generative Models
Liu, V., & Chilton, L. B. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501825 · Cited by 639 · cited 1× · see citing works
“Prompt engineering has already been used by researchers to explore LLMs for a wide variety of tasks, such as text-to-image generation [3, 17], human-AI co-writing tasks [8, 30], medical applications [19, 38], programming [9], and many more.”
BackgroundA general language model for information retrieval
Song, F., & Croft, W. B. (1999) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/319950.320022 · Cited by 544 · see citing works
Denny, P., Kumar, V., & Giacaman, N. (2023) · Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3545945.3569823 · Cited by 287 · see citing works
Parental Control vs. Teen Self-Regulation: Is there a middle ground for mobile online safety?
Wisniewski, P., Ghosh, A. K., Xu, H., Rosson, M. B., & Carroll, J. M. (2017) · Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/2998181.2998352 · Cited by 206 · see citing works
Brade, S., Wang, B. K., Sousa, M., Oore, S., & Grossman, T. (2023) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3586183.3606725 · Cited by 198 · see citing works
PromptMaker: Prompt-based Prototyping with Large Language Models
Jiang, E., Olson, K., Toh, E., Molina, A., Donsbach, A., Terry, M., & Cai, C. J. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3503564 · Cited by 157 · see citing works
Xiao, C., Xu, S. X., Zhang, K., Wang, Y., & Xia, L. (2023) · Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023) · doi:10.18653/v1/2023.bea-1.52 · Cited by 138 · see citing works
Learning to Identify Internet Sexual Predation
McGhee, I., Bayzick, J., Kontostathis, A., Edwards, L., McBride, A., & Jakubowski, E. (2011) · International Journal of Electronic Commerce · doi:10.2753/jec1086-4415150305 · Cited by 106 · see citing works
Prompting Large Language Models With the Socratic Method
Chang, E. Y. (2023) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc57344.2023.10099179 · Cited by 77 · see citing works
Royen, K. V., Poels, K., Vandebosch, H., & Adam, P. C. G. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.09.040 · Cited by 76 · see citing works
Informing Age-Appropriate AI: Examining Principles and Practices of AI for Children
Wang, G., Zhao, J., Van Kleek, M., & Shadbolt, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502057 · Cited by 76 · see citing works
Interventions for Children, Youth, and Parents to Prevent and Reduce Cyber Abuse
Mishna, F., Cook, C., Saini, M., Wu, M., & MacFadden, R. J. (2009) · Campbell Systematic Reviews · doi:10.4073/csr.2009.2 · Cited by 72 · see citing works
Should We Design for Control, Trust or Involvement?
Hartikainen, H., Iivari, N., & Kinnula, M. (2016) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/2930674.2930680 · Cited by 70 · see citing works
Chubb, J., Missaoui, S., Concannon, S. J., Maloney, L., & Walker, J. A. (2021) · International Journal of Child-Computer Interaction · doi:10.1016/j.ijcci.2021.100403 · Cited by 56 · see citing works
Patterson, A., Ryckman, L., & Guerra, C. (2022) · Journal of Child & Adolescent Trauma · doi:10.1007/s40653-022-00440-x · Cited by 55 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Survey of Generative AI for Detecting Pedophilia Crimes
Silva, F., Silva, R. R., & Bernardino, J. (2025) · Applied Sciences · doi:10.3390/app15137105 · Cited by 4 · see citing works
Zhang, X., Akter, M., An, H., Liu, M., Zhang, Q., Huang, L., Cho, J.-H., J. Wisniewski, P., & Lee, S. W. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790949
“To combat cybergrooming, prior work has focused on surveillance and detection, using computational techniques such as machine learning (ML), natural language processing (NLP), and data mining to detect perpetrator behavior [32, 34, 59, 72, 76–78].”
- Authors
- Prosser, E., & Edwards, M.
- Affiliations
- University of Bristol (GB)
- Venue
- European Interdisciplinary Cybersecurity Conference
- Publisher
- ACM
- Published
- 2024-06-05
- Pages
- 1-10
- OpenAlex
- W4399332763
- Cited by
- 6
- References
- 20
- Citations by year
- 2026: 3 · 2025: 3
- FWCI
- 2.74
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- European Interdisciplinary Cybersecurity Conference (EICC) (publishedVersion, cc-by)
- Funders
- UKRI (EP/S022937/1)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- large language models, child online safety, prompt engineering, open-source language models, generative AI safety, model consistency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
