Tricky vs. Transparent: Towards an Ecologically Valid and Safe Approach for Evaluating Online Safety Nudges for Teens
Abstract
HCI research has been at the forefront of designing interventions for protecting teens online; yet, how can we test and evaluate these solutions without endangering the youth we aim to protect? Towards this goal, we conducted focus groups with 20 teens to inform the design of a social media simulation platform and study for evaluating online safety nudges co-designed with teens. Participants evaluated risk scenarios, personas, platform features, and our research design to provide insight regarding the ecological validity of these artifacts. Teens expected risk scenarios to be subtle and tricky, while also higher in risk to be believable. The teens iterated on the nudges to prioritize risk prevention without reducing autonomy, risk coping, and community accountability. For the simulation, teens recommended using transparency with some deceit to balance realism and respect for participants. Our meta-level research provides a teen-centered action plan to evaluate online safety interventions safely and effectively.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The study used simulated cybergrooming scenarios to show the development of cybergrooming without exposing participants to real cybergrooming risks.
“Inspired by Agha et al. [4], we opted to use simulated cybergrooming scenarios to show the development of cybergrooming without exposing participants to real cybergrooming risks.”
From From Vulnerable to Resilient: Examining Parent and Teen Perceptions on How to Respond to Unwanted Cybergrooming Advances (2026) · cites this paper as [4]
Cites
Interpersonal Effects in Computer-Mediated Interaction
Walther, J. B. (1992) · Communication Research · doi:10.1177/009365092019001003 · Cited by 2,535 · see citing works
Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743 · see citing works
McGrath, J. E. (1981) · American Behavioral Scientist · doi:10.1177/000276428102500205 · Cited by 738 · see citing works
Nudge – Improving Decisions about Health, Wealth and Happiness
(2010) · Sustainability Accounting Management and Policy Journal · doi:10.1108/20408021011059296 · Cited by 612 · see citing works
Common Sense Reasoning for Detection, Prevention, and Mitigation of Cyberbullying
Dinakar, K., Jones, B., Havasi, C., Lieberman, H. A., & Picard, R. W. (2012) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2362394.2362400 · Cited by 434 · see citing works
Teaching Johnny not to fall for phish
Kumaraguru, P., Sheng, S., Acquisti, A., Cranor, L. F., & Hong, J. (2010) · ACM Transactions on Internet Technology · doi:10.1145/1754393.1754396 · Cited by 399 · see citing works
Data as asset? The measurement, governance, and valuation of digital personal data by Big Tech
Birch, K., Cochrane, D., & Ward, C. (2021) · Big Data & Society · doi:10.1177/20539517211017308 · Cited by 328 · see citing works
Wieringa, M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372833 · Cited by 321 · see citing works
Wallace, J., McCarthy, J., Wright, P., & Olivier, P. L. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466473 · Cited by 245 · see citing works
Design of Randomized Controlled Trials
Stanley, K. (2007) · Circulation · doi:10.1161/circulationaha.105.594945 · Cited by 239 · see citing works
What Is Digital Parenting? A Systematic Review of Past Measurement and Blueprint for the Future
Modecki, K. L., Goldberg, R., Wiśniewski, P., & Orben, A. C. (2022) · Perspectives on Psychological Science · doi:10.1177/17456916211072458 · Cited by 208 · see citing works
Parental Control vs. Teen Self-Regulation: Is there a middle ground for mobile online safety?
Wisniewski, P., Ghosh, A. K., Xu, H., Rosson, M. B., & Carroll, J. M. (2017) · Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/2998181.2998352 · Cited by 206 · see citing works
Ashktorab, Z., & Jessica Vitak, B. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858548 · Cited by 179 · see citing works
Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems
Barbosa, S., Barbosa, S., Lampe, C., Appert, C., Shamma, D. A., USA, C. '. C. C. O. H. F. I. C. S. 2. N. O. L., Association for Computing Machinery-Digital Library, ACM Special Interest Group on Computer-Human Interaction, & SIGs, A. (2022) · doi:10.1145/3491101 · Cited by 172 · see citing works
Privacy nudges for social media
Wang, Y., Leon, P. G., Scott, K., Chen, X., Acquisti, A., & Cranor, L. F. (2013) · International World Wide Web Conference (WWW) · doi:10.1145/2487788.2488038 · Cited by 168 · see citing works
Self-citationGhosh, A. K., Badillo-Urquiola, K. A., Guha, S., LaViola, J. J., & Wiśniewski, P. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173698 · Cited by 168 · see citing works
Self-citationDear Diary: Teens Reflect on Their Weekly Online Risk Experiences
Wisniewski, P., Xu, H., Rosson, M., Perkins, D., & Carroll, J. M. (2016) · International Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858317 · Cited by 141 · see citing works
Self-citationResilience Mitigates the Negative Effects of Adolescent Internet Addiction and Online Risk Exposure
Wiśniewski, P., Jia, H., Wang, N., Zheng, S., Xu, H., Rosson, M. B., & Carroll, J. M. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702240 · Cited by 125 · see citing works
Faily, S., & Fléchais, I. (2011) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979274 · Cited by 108 · see citing works
Co-designing Mobile Online Safety Applications with Children
McNally, B., Kumar, P. C., Hordatt, C., Mauriello, M. L., Naik, S., Norooz, L., Shorter, A., Golub, E., & Druin, A. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174097 · Cited by 101 · see citing works
Manipulation, salience, and nudges
Noggle, R. (2017) · Bioethics · doi:10.1111/bioe.12421 · Cited by 92 · see citing works
DiFranzo, D. J., Taylor, S. H., Kazerooni, F., Wherry, O. D., & Bazarova, N. N. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173785 · Cited by 92 · see citing works
Self-citationLet's Talk about Sext: How Adolescents Seek Support and Advice about Their Online Sexual Experiences
Razi, A., Badillo-Urquiola, K., & Wisniewski, P. J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376400 · Cited by 89 · see citing works
Self-citationSocial Robot Co-Design Canvases: A Participatory Design Framework
Axelsson, M., Oliveira, R., Racca, M., & Kyrki, V. (2021) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3472225 · Cited by 85 · see citing works
Designing with Dementia: Guidelines for Participatory Design together with Persons with Dementia
Hendriks, N., Truyen, F., & Duval, E. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40483-2_46 · Cited by 79 · see citing works
Adolescent Online Safety: Moving Beyond Formative Evaluations to Designing Solutions for the Future
Pinter, A. T., Wisniewski, P. J., Xu, H., Rosson, M. B., & Caroll, J. M. (2017) · Proceedings of the 2017 Conference on Interaction Design and Children · doi:10.1145/3078072.3079722 · Cited by 79 · see citing works
Badillo-Urquiola, K. A., Smriti, D., McNally, B., Golub, E., Bonsignore, E., & Wiśniewski, P. (2019) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3311927.3323133 · Cited by 79 · see citing works
Self-citationCyber-aggression, Cyberbullying, and Cyber-grooming: A Survey and Research Challenges
Mladenović, M., Ošmjanski, V., & Stanković, S. V. (2022) · ACM Computing Surveys · doi:10.1145/3424246 · Cited by 79 · see citing works
Wu, P., & Chiou, W. (2009) · CyberPsychology & Behavior · doi:10.1089/cpb.2008.0182 · Cited by 77 · see citing works
Youth Perspectives on Critical Data Literacies
Hautea, S., Dasgupta, S., & Hill, B. M. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025823 · Cited by 77 · see citing works
Royen, K. V., Poels, K., Vandebosch, H., & Adam, P. C. G. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.09.040 · Cited by 76 · see citing works
Agha, Z., Badillo-Urquiola, K. A., & Wiśniewski, P. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579625 · Cited by 76 · see citing works
Self-citationThe New ABC’s of Research: Achieving Breakthrough Collaborations
Jull, M. G. (2017) · Technology|Architecture + Design · doi:10.1080/24751448.2017.1292800 · Cited by 74 · see citing works
Interaction design research with adolescents
Poole, E. S., & Peyton, T. (2013) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/2485760.2485766 · Cited by 68 · see citing works
Exploring Nudge Designs to Help Adolescent SNS Users Avoid Privacy and Safety Threats
Masaki, H., Shibata, K., Hoshino, S., Ishihama, T., Saito, N., & Yatani, K. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376666 · Cited by 63 · see citing works
Antle, A. N. (2006) · ACM Conference on Designing Interactive Systems (DIS) · doi:10.1145/1142405.1142411 · Cited by 60 · see citing works
Conducting Risky Research with Teens
Badillo-Urquiola, K. A., Shea, Z., Agha, Z., Lediaeva, I., & Wiśniewski, P. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3432930 · Cited by 56 · see citing works
Self-citationRazi, A., Alsoubai, A., Kim, S., Ali, S., Stringhini, G., De Choudhury, M., & Wisniewski, P. J. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579522 · Cited by 52 · see citing works
Xiao, S., Cheshire, C., & Salehi, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517614 · Cited by 51 · see citing works
Papatraianou, L., Levine, D. T., & West, D. (2014) · Pastoral Care in Education · doi:10.1080/02643944.2014.974661 · Cited by 49 · see citing works
DiFranzo, D. J., Choi, Y. H., Purington, A., Taft, J. G., Whitlock, J. L., & Bazarova, N. N. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300533 · Cited by 48 · see citing works
Self-citationUnderstanding Digital-Safety Experiences of Youth in the U.S.
Freed, D., Bazarova, N. N., Consolvo, S., Han, E. J., Kelley, P. G., Thomas, K., & Cosley, D. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581128 · Cited by 46 · see citing works
Huh-Yoo, J., Razi, A., Nguyen, D. N., Regmi, S., & Wisniewski, P. (2023) · International Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581233 · Cited by 44 · see citing works
Wizard of Oz Experimentation for Language Technology Applications: Challenges and Tools
Schlögl, S., Doherty, G. J., & Luz, S. F. (2014) · Interacting with Computers · doi:10.1093/iwc/iwu016 · Cited by 43 · see citing works
Ali, S., Razi, A., Kim, S., Alsoubai, A., Gracie, J., Choudhury, M. D., Wiśniewski, P., & Stringhini, G. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501969 · Cited by 43 · see citing works
Supporting Teens’ Intentional Social Media Use Through Interaction Design
Davis, K. S., Slovák, P., Landesman, R., Pitt, C. R., Ghajar, A., Schleider, J. L., Kawas, S., Portillo, A., & Kuhn, N. S. (2023) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3585088.3589387 · Cited by 35 · see citing works
Ali, S., Razi, A., Kim, S., Alsoubai, A., Ling, C., Choudhury, M. D., Wiśniewski, P., & Stringhini, G. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579608 · Cited by 33 · see citing works
Moving beyond 'one size fits all'
Walker, A. M., Yao, Y., Geeng, C., Hoyle, R., & Wiśniewski, P. (2019) · interactions · doi:10.1145/3358904 · Cited by 32 · see citing works
Schiano, D. J., Burg, C., Smith, A. N., & Moore, F. (2016) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2851581.2892481 · Cited by 27 · see citing works
Rutkowski, T. L., Hartikainen, H., Richards, K. E., & Wiśniewski, P. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479517 · Cited by 27 · see citing works
Perceived Safety and Teen Risk Taking in Online Chat Sites
McCarty, C., Prawitz, A. D., Derscheid, L. E., & Montgomery, B. (2010) · Cyberpsychology Behavior and Social Networking · doi:10.1089/cyber.2010.0050 · Cited by 25 · see citing works
Towards Automated Detection of Risky Images Shared by Youth on Social Media
Park, J., Gracie, J., Alsoubai, A., Stringhini, G., Singh, V. K., & Wiśniewski, P. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543873.3587607 · Cited by 22 · see citing works
Self-citationCo-Designing User Personas and Risk Scenarios for Evaluating Adolescent Online Safety Interventions
Agha, Z., Miu, K., Piper, S., Park, J., & Wiśniewski, P. (2023) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3584931.3606964 · Cited by 20 · see citing works
Self-citationQuintana, R. M., Haley, S. R., Levick, A., Holman, C., Hayward, B., & Wojan, M. (2017) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3027063.3053355 · Cited by 19 · see citing works
Zinkus, M., Curry, O., Moore, M., Peterson, Z. N. J., & Wood, Z. J. (2019) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3287324.3287486 · Cited by 14 · see citing works
Aghajari, Z., Baumer, E. P. S., Hohenstein, J., Jung, M. F., & DiFranzo, D. J. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579506 · Cited by 8 · see citing works
Self-citationThe ConWIZ Protocol: A Generic Protocol for Wizard of Oz Simulations
Grill, T., & Tscheligi, M. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-53862-9_55 · Cited by 7 · see citing works
“Not all my friends are friends ”: Audience‐group‐based nudges for managing location privacy
Ghosh, I., & Singh, V. K. (2021) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24580 · Cited by 6 · see citing works
Clements, B. (2022) · eYLS (Yale Law School) · Cited by 1 · see citing works
Flagged first, then by citations
1 of the 60 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Park, J., Akter, M., Wiśniewski, P., & Badillo-Urquiola, K. A. (2024) · IEEE Security & Privacy · doi:10.1109/msec.2024.3417804 · Cited by 17 · see citing works
Self-citationA systematic review on design-based nudges for adolescent online safety
Agha, Z., Ali, N. S., Park, J., & Wiśniewski, P. (2024) · International Journal of Child-Computer Interaction · doi:10.1016/j.ijcci.2024.100702 · Cited by 12 · see citing works
Self-citationOguine, O. C., Anuyah, O., Agha, Z., Melgarez, I., Garcia, A. A., & Badillo-Urquiola, K. A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757639 · Cited by 9 · see citing works
Self-citationExamining Mainstream News Media Narratives on Youth Online Safety
Oguine, O. C., Anuyah, O., Hughes, E. M., & Badillo-Urquiola, K. A. (2024) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3678884.3681874 · Cited by 8 · see citing works
Self-citationAli, N. S., Ma, R., Agha, Z., Park, J., & Wiśniewski, P. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757665 · Cited by 9 · see citing works
Self-citationOguine, O. C., Park, J. K., Akter, M., Olesk, J., Alluhidan, A., Wisniewski, P., & Badillo-Urquiola, K. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710995 · Cited by 8 · see citing works
Self-citationDumaru, P., Flood, A., & Al-Ameen, M. N. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711077 · Cited by 6 · see citing works
Oguine, O. C., Garcia, A. A., Muller, M. J., & Badillo-Urquiola, K. A. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791346 · Cited by 4 · see citing works
Self-citationSocialSim: An Open Source Platform for Conducting Behavioral Intervention Research on Social Media
Agha, Z., Martinez, M., Ali, N. S., DiFranzo, D. J., & Wiśniewski, P. (2025) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3715070.3757229 · Cited by 2 · see citing works
Self-citationZhang, X., Akter, M., An, H., Liu, M., Zhang, Q., Huang, L., Cho, J.-H., J. Wisniewski, P., & Lee, S. W. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790949
Self-citationFrom Users to Co-Designers: Youth Participation in Understanding Cyberbullying
Verma, K., Davis, B., Milošević, T., & Umbach, R. (2025) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3713043.3731485 · Cited by 1 · see citing works
Advancing Interdisciplinary Approaches to Online Safety Research
Wijenayake, S., Gray, J. E., Jayatilaka, A., Sala, L. L., Arachchilage, N., Kelly, R. M., & Das, S. (2025) · Australasian Computer-Human Interaction Conference · doi:10.1145/3764687.3767275 · Cited by 2 · see citing works
Agency in Child–AI Interaction: A Review of How It Is Conceptualised, Studied, and Supported in HCI
Voysey, I., Vizgirda, V., Turner, S., Duran, L. D. D., Pauzi, Z., Mavrikis, M., Prunkl, C., & Zhao, J. (2026) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3773077.3806105 · Cited by 1 · see citing works
Agha, Z., Dumaru, P., Ali, N. S., & Wiśniewski, P. (2026) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3773077.3806123
Self-citation
- Authors
- Agha, Z., Park, J., Wan, R., Ali, N. S., Wang, Y., Difranzo, D., Badillo-Urquiola, K., & Wisniewski, P. J.
- Affiliations
- Vanderbilt University (US)
- University of Notre Dame (US)
- Lehigh University (US)
- Department of Computer Science, Vanderbilt University, United States
- ORCID
- Venue
- Proceedings of the CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2024-05-11
- Pages
- 1-20
- OpenAlex
- W4396832692
- Cited by
- 13
- References
- 60
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 3
- FWCI
- 17.05
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- National Science Foundation (IIS-2333207)
- William T Grant Foundation (187941)
- Topics
- Social Sciences › Sociology and Political Science › Impact of Technology on Adolescents
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Social Sciences › Sociology and Political Science › Privacy, Security, and Data Protection
- Keywords
- adolescents, ecological validity, risk scenarios, research transparency, youth autonomy, risk prevention, risk coping, community accountability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
