The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Uncovering Contradictions in Human-AI Interactions: Lessons Learned from User Reviews of Replika
Abstract
The increasing integration of artificial intelligence (AI) in daily life presents opportunities and challenges, particularly in fostering human-AI relationships. This study investigates user reviews from the Google Play Store for the Replika chatbot, focusing on complaints of online sexual harassment. Using Activity Theory, the analysis reveals several contradictions within the Replika activity system, including tool-subject, tool-object, rule-subject, rule-object, and distribution of labor-subject and labor-object contradictions. These contradictions highlight the misalignment between user expectations and the chatbot's behavior, inadequate safety measures, and unrealistic expectations placed on users to train the AI. The study emphasizes the need for clearer objectives, advanced AI alignment techniques, and refined safety protocols to enhance the user experience and ensure ethical interactions with chatbots. We provide implications for guidelines on the development of more trustworthy and supportive digital companions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Research on problematic chatbot engagement focuses largely on adults and offers mixed findings about harm.
“Yet research on problematic chatbot engagement focuses largely on adults and offers mixed findings about harm [38, 97, 99, 112, 145].”
From Understanding Teen Overreliance on AI Companion Chatbots Through Self-Reported Reddit Narratives (2026) · cites this paper as [97]
Citations
Cites
Mind in Society: Development of Higher Psychological Processes
VYGOTSKY, L. S. (1980) · doi:10.2307/j.ctvjf9vz4 · Cited by 3,123
What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education
Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023) · Smart Learning Environments · doi:10.1186/s40561-023-00237-x · Cited by 2,003
Exploring relationship development with social chatbots: A mixed-method study of replika
Pentina, I., Hancock, T., & Xie, T. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107600 · Cited by 522
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.05862 · Cited by 442
Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., & Kaplan, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.08073 · Cited by 326 · cited 1×
“One promising concept is constitutional AI, as presented by Anthropic AI [5].”
A longitudinal study of human–chatbot relationships
Skjuve, M., Følstad, A., Fostervold, K. I., & Brandtzaeg, P. B. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102903 · Cited by 253
Young, J., Jawara, L. M., Nguyen, D. N., Daly, B., Huh-Yoo, J., & Razi, A. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642574 · Cited by 59
Self-citationThematic Analysis on User Reviews for Depression and Anxiety Chatbot Apps: Machine Learning Approach
Ahmed, A., Aziz, S., Khalifa, M., Shah, U., Hassan, A., Abd-Alrazaq, A., & Househ, M. (2022) · JMIR Formative Research · doi:10.2196/27654 · Cited by 37
Flagged first, then by citations
Research works printed but not linked: 26
- 10.1177/0894439304271540 · DOI · Paper page
- 10.21428/2c646de5.db67ec7f · DOI · Paper page
- J. Chow and K. Li. 2024. Human-centered ai: large language models and the need for ethical medical chatbots (preprint). (2024). https://doi.org/10.2196/preprints. 56404 · DOI
- Raffaele Ciriello Oliver Hannon Angelina Chen and Emmanuelle Vaast. 2023. Ethical Tensions in Human-AI Companionship: A Dialectical Inquiry into Replika. · DOI
- 10.1080/0144929X.2016.1175507 · DOI
- 10.1177/01634437221119021 · DOI · Paper page
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 1. 4171--4186.
- 10.1017/CBO9781139814744 · DOI · Paper page
- 10.1037/vio0000168 · DOI
- Amelia Glaese Nat McAleese Maja Trbacz John Aslanides Vlad Firoiu Timo Ewalds Maribeth Rauh Laura Weidinger Martin Chadwick Phoebe Thacker Lucy Campbell-Gillingham Jonathan Uesato Po-Sen Huang Ramona Comanescu Fan Yang Abigail See Sumanth Dathathri Rory Greig Charlie Chen Doug Fritz Jaume Sanchez Elias Richard Green Soa Mokrá Nicholas Fernando Boxi Wu Rachel Foley Susannah Young Iason Gabriel William Isaac John Mellor Demis Hassabis Koray Kavukcuoglu Lisa Anne Hendricks and Geoffrey Irving. 2022. Improving Alignment of Dialogue Agents via Targeted Human Judgements. https://doi.org/10.48550/arXiv.2209.14375 arXiv:2209.14375 [cs] · DOI · Paper page
- 10.1145/3544548.3581233 · DOI
- Replika Inc. 2024. Replika: My AI Friend. https://play.google.com/store/apps/details?id=ai.replika.app. Version 10.1.0.
- 10.1007/978--3-031-02196--1 · DOI
- Victor Kaptelinin and Bonnie A. Nardi. 2009. Acting with Technology: Activity Theory and Interaction Design (1. mit press paperback ed ed.). MIT Press, Cambridge, Mass. London.
- A Leontev. 1975. Dieyatelinocti, Soznaine, i Lichynosti [Activity, Consciousness, and Personality]. Moskva: Politizdat (1975).
- Zilin Ma Yiyang Mei and Zhaoyuan Su. 2023. Understanding the Benefits and Challenges of Using Large Language Model-based Conversational Agents for Mental Well-being Support. https://doi.org/10.48550/arXiv.2307.15810 arXiv:2307.15810 [cs] · DOI · Paper page
- 10.1016/j.chb.2021.106795 · DOI
- 10.1177/0886260517725736 · DOI
- 10.1145/3027063.3053246 · DOI · Paper page
- Afsaneh Razi, Ashwaq Alsoubai, Seunghyun Kim, Shiza Ali, Gianluca Stringhini, Munmun De Choudhury, and Pamela J. Wisniewski. 2023. Sliding into My DMs: Detecting Uncomfortable or Unsafe Sexual Risk Experiences within Instagram Direct Messages Grounded in the Perspective of Youth. Proc. ACM Hum.-Comput. Interact. 7, CSCW1, Article 89 (apr 2023), 29 pages. https://doi.org/10.1145/ 3579522
- 10.2139/ssrn.3851317 · DOI
- 10.1145/3479609 · DOI · Paper page
- 10.1145/3637411 · DOI
- 10.1145/3461778.3462125 · DOI
- Ming Tan Cicero dos Santos Bing Xiang and Bowen Zhou. 2016. LSTM-based Deep Learning Models for Non-factoid Answer Selection. https://doi.org/10. 48550/arXiv.1511.04108 arXiv:1511.04108 [cs] · arXiv
- Caroline Tranberg. 2023. I Love My AI Girlfriend A Study of Consent in AI-human Relationships. Masters thesis. The University of Bergen.
Cited by
Sorted by citations
Namvarpour, M. (., Pauwels, H., & Razi, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757548 · Cited by 45
Self-citationUnderstanding Teen Overreliance on AI Companion Chatbots Through Self-Reported Reddit Narratives
Namvarpour, M. (., Brofsky, B., Medina, J. Y., Akter, M., & Razi, A. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790597 · Cited by 24
Self-citationA Responsive Present, a Shared Past, a Social Other: Teens' Overreliance on Companion AI Chatbots
Namvarpour, M., Chang, T., & Razi, A. (2026) · arXiv · doi:10.48550/arxiv.2609.14843
Self-citation
Metadata
- Authors
- Namvarpour, M., & Razi, A.
- Affiliations
- Drexel University, Philadelphia, Pennsylvania, USA
- Venue
- Companion Publication of the 2024 Conference on Computer-Supported Cooperative Work and Social Computing
- Publisher
- ACM
- Published
- 2024-11-11
- Pages
- 579-586
- Cited by
- 22
- References
- 8
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
