Thematic Analysis on User Reviews for Depression and Anxiety Chatbot Apps: Machine Learning Approach
Open accessPublished version of a preprint
Abstract
Background: Anxiety and depression are among the most commonly prevalent mental health disorders worldwide. Chatbot apps can play an important role in relieving anxiety and depression. Users’ reviews of chatbot apps are considered an important source of data for exploring users’ opinions and satisfaction. Objective: This study aims to explore users’ opinions, satisfaction, and attitudes toward anxiety and depression chatbot apps by conducting a thematic analysis of users’ reviews of 11 anxiety and depression chatbot apps collected from the Google Play Store and Apple App Store. In addition, we propose a workflow to provide a methodological approach for future analysis of app review comments. Methods: We analyzed 205,581 user review comments from chatbots designed for users with anxiety and depression symptoms. Using scraper tools and Google Play Scraper and App Store Scraper Python libraries, we extracted the text and metadata. The reviews were divided into positive and negative meta-themes based on users’ rating per review. We analyzed the reviews using word frequencies of bigrams and words in pairs. A topic modeling technique, latent Dirichlet allocation, was applied to identify topics in the reviews and analyzed to detect themes and subthemes. Results: Thematic analysis was conducted on 5 topics for each sentimental set. Reviews were categorized as positive or negative. For positive reviews, the main themes were confidence and affirmation building, adequate analysis, and consultation, caring as a friend, and ease of use. For negative reviews, the results revealed the following themes: usability issues, update issues, privacy, and noncreative conversations. Conclusions: Using a machine learning approach, we were able to analyze ≥200,000 comments and categorize them into themes, allowing us to observe users’ expectations effectively despite some negative factors. A methodological workflow is provided for the future analysis of review comments.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Meta-analyses indicate that the therapeutic effects of rule-based AI chatbots are small and not sustained over time, and users report frustration with responses that feel empty, generic, nonsensical, repetitive and constrained.
“Meta-analyses indicate that the therapeutic effects are small and not sustained over time24, and users report frustration with responses that feel empty, generic20, nonsensical, repetitive and constrained19–22.”
From “It happened to be the perfect thing”: experiences of generative AI chatbots for mental health (2024) · cites this paper as [19]
Rule-based chatbot apps have more promising user engagement, with positive app store ratings and qualitative studies finding that users appreciate the human-like interaction and social support.
“Rule-based chatbot apps have more promising user engagement, with positive app store ratings19,20 and qualitative studies finding that users appreciate the human-like interaction and social support19–23.”
From “It happened to be the perfect thing”: experiences of generative AI chatbots for mental health (2024) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Loneliness matters: a theoretical and empirical review of consequences and mechanisms
Hawkley, L. C., & Cacioppo, J. T. (2010) · Annals of Behavioral Medicine · doi:10.1007/s12160-010-9210-8 · Cited by 4,782 · cited 1× · see citing works
“Loneliness and social isolation are tantamount to feeling unsafe and are classified as social threats accompanied by feelings of hostility, low self-esteem, anxiety, stress, and pessimism [24].”
BackgroundMachine learning and deep learning
Janiesch, C., Zschech, P., & Heinrich, K. (2021) · Electronic Markets · doi:10.1007/s12525-021-00475-2 · Cited by 2,959 · see citing works
Understanding bag-of-words model: a statistical framework
Zhang, Y., Jin, R., & Zhou, Z.-H. (2010) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-010-0001-0 · Cited by 1,384 · see citing works
Wallach, H. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143967 · Cited by 1,058 · see citing works
How Do Users Like This Feature? A Fine Grained Sentiment Analysis of App Reviews
Guzmán, E., & Maalej, W. (2014) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re.2014.6912257 · Cited by 611 · see citing works
Lee, C. M., Cadigan, J. M., & Rhew, I. C. (2020) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2020.08.009 · Cited by 526 · cited 1× · see citing works
“This issue has been exacerbated during the COVID-19 pandemic, resulting in a large number of mental health issues [25].”
BackgroundPerceptions and opinions of patients about mental health chatbots: scoping review
Abd-Alrazaq, A. A., Alajlani, M., Ali, N., Denecke, K., Bewick, B. M., & Househ, M. (2021) · Journal of Medical Internet Research · doi:10.2196/17828 · Cited by 301 · see citing works
Self-citationThe standard of user-centered design and the standard definition of usability
Jokela, T., Iivari, N., Matero, J., & Karukka, M. (2003) · Latin American conference on Human-Computer Interaction · doi:10.1145/944519.944525 · Cited by 273 · see citing works
Relationship between loneliness and mental health in students
Richardson, T., Elliott, P., & Roberts, R. (2017) · Journal of Public Mental Health · doi:10.1108/jpmh-03-2016-0013 · Cited by 256 · see citing works
Holmes, S. L., Moorhead, A., Bond, R., Zheng, H., COATES, V. E., & McTear, M. (2019) · European Conference on Cognitive Ergonomics · doi:10.1145/3335082.3335094 · Cited by 215 · see citing works
Bendig, E., Erb, B., Schulze-Thuesing, L., & Baumeister, H. (2022) · Verhaltenstherapie · doi:10.1159/000501812 · Cited by 214 · see citing works
Text mining for information systems researchers: An annotated topic modeling tutorial
University of Liechtenstein, Debortoli, S., Müller, O., IT University of Copenhagen, Junglas, I., Florida State University, vom Brocke, J., & University of Liechtenstein (2016) · Communications of the Association for Information Systems · doi:10.17705/1cais.03907 · Cited by 199 · see citing works
Stawarz, K., Preist, C., Tallon, D., Wiles, N., & Coyle, D. (2018) · Journal of Medical Internet Research · doi:10.2196/10120 · Cited by 198 · see citing works
Oyebode, O., Alqahtani, F., & Orji, R. (2020) · IEEE Access · doi:10.1109/access.2020.3002176 · Cited by 197 · see citing works
Analyzing and automatically labelling the types of user issues that are raised in mobile app reviews
Mcilroy, S., Ali, N., Khalid, H., & Hassan, A. E. (2015) · Empirical Software Engineering · doi:10.1007/s10664-015-9375-7 · Cited by 193 · see citing works
Self-citationArtificial Intelligence for Chatbots in Mental Health: Opportunities and Challenges
Denecke, K., Abd-Alrazaq, A., & Househ, M. (2021) · Lecture Notes in Bioengineering · doi:10.1007/978-3-030-67303-1_10 · Cited by 177 · see citing works
Self-citationStawarz, K., Cox, A. L., & BLANDFORD, A. E. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557079 · Cited by 153 · see citing works
Privacy Concerns in Chatbot Interactions
Ischen, C., Araujo, T., Voorveld, H., van Noort, G., & Smit, E. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-39540-7_3 · Cited by 152 · see citing works
Digital Mental Health: The Answer to the Global Mental Health Crisis?
Rudd, B. N., & Beidas, R. S. (2020) · JMIR Mental Health · doi:10.2196/18472 · Cited by 81 · see citing works
User requirements of mobile technology: results from a content analysis of user reviews
Gebauer, J., Tang, Y. F., & Baimai, C. (2007) · Information Systems and e-Business Management · doi:10.1007/s10257-007-0074-9 · Cited by 72 · see citing works
Mood variability in anxiety disorders
Bowen, R., Baetz, M., Hawkes, J., & Bowen, A. (2006) · Journal of Affective Disorders · doi:10.1016/j.jad.2005.12.050 · Cited by 69 · see citing works
Milward, J., Khadjesari, Z., Fincham-Campbell, S., Deluca, P., Watson, R., & Drummond, C. K. (2016) · JMIR mhealth and uhealth · doi:10.2196/mhealth.5242 · Cited by 65 · see citing works
Brenninkmeijer, V., Lagerveld, S., Blonk, R., Schaufeli, W., & Meij, L. W.-D. (2018) · Journal of occupational rehabilitation · doi:10.1007/s10926-018-9760-3 · Cited by 50 · see citing works
Olagunju, T., Oyebode, O., & Orji, R. (2020) · IEEE Access · doi:10.1109/access.2020.3000093 · Cited by 41 · see citing works
Topic models as a novel approach to identify themes in content analysis
Piepenbrink, A., & Gaur, A. S. (2017) · Academy of Management Proceedings · doi:10.5465/ambpp.2017.141 · Cited by 34 · see citing works
On the Emotion of Users in App Reviews
Martens, D., & Johann, T. (2017) · IEEE/ACM International Workshop on Emotion Awareness in Software Engineering (SEmotion) · doi:10.1109/semotion.2017.6 · Cited by 29 · see citing works
Grünzig, S.-D., Baumeister, H., Bengel, J., Ebert, D. D., & Krämer, L. V. (2018) · Trials · doi:10.1186/s13063-018-2657-9 · Cited by 23 · see citing works
Qualitative Data Analysis Software
Wildt, L. D. (2020) · The SAGE Encyclopedia of the Sociology of Religion · doi:10.4135/9781529714401.n366 · Cited by 21 · see citing works
Dragovic, M., Davison, S., Morgan, V. A., Chiu, V. W., Richards, N., Vatskalis, T., Atkinson, A., & Waters, F. (2020) · Advances in Mental Health · doi:10.1080/18387357.2018.1557014 · Cited by 20 · see citing works
Tool Support for Analyzing Mobile App Reviews
Vu, P. M., Pham, H. V., Nguyen, T. T., & Nguyen, T. T. (2015) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2015.101 · Cited by 15 · see citing works
Rawstrone, A. (2016) · Nursery World · doi:10.12968/nuwa.2016.sup23.8 · Cited by 6 · see citing works
Get ready for the genomic revolution
Kirby, R. S. (2017) · Trends in Urology & Men s Health · doi:10.1002/tre.567 · Cited by 2 · see citing works
Grünzig, S.-D., Baumeister, H., Bengel, J., Ebert, D. D., & Krämer, L. V. (2018) · Trials · doi:10.1186/s13063-018-2806-1 · Cited by 2 · see citing works
Storz, M. A. (2020) · Menopause The Journal of The North American Menopause Society · doi:10.1097/gme.0000000000001617 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
“It happened to be the perfect thing”: experiences of generative AI chatbots for mental health
Siddals, S., Torous, J., & Coxon, A. (2024) · npj Mental Health Research · doi:10.1038/s44184-024-00097-4 · Cited by 241 · see citing works
Khalifa, M., Albadawy, M., & Iqbal, U. (2024) · Computer Methods and Programs in Biomedicine Update · doi:10.1016/j.cmpbup.2024.100142 · Cited by 134 · see citing works
Self-citationRetractedRabbani, S. A., El‐Tanani, M., Sharma, S., Rabbani, S. S., El‐Tanani, Y., Kumar, R. R., & Saini, M. (2025) · BioMedInformatics · doi:10.3390/biomedinformatics5030037 · Cited by 102 · see citing works
Limpanopparat, S., Gibson, E., & Harris, D. A. (2024) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2024.100081 · Cited by 71 · see citing works
Young, J., Jawara, L. M., Nguyen, D. N., Daly, B., Huh-Yoo, J., & Razi, A. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642574 · Cited by 59 · see citing works
Wrightson‐Hester, A., Anderson, G., Dunstan, J., McEvoy, P. M., Sutton, C., MYERS, B. J., Egan, S. J., Tai, S., Johnston-Hollitt, M., Chen, W., et al. (2023) · JMIR Human Factors · doi:10.2196/46849 · Cited by 45 · see citing works
Chen, C., Lam, K. T., Yip, K. M., So, H. K., Lum, T. Y. S., Wong, I. C. K., Yam, J. C., Chui, C. S. L., & Ip, P. (2025) · JMIR Human Factors · doi:10.2196/65785 · Cited by 35 · see citing works
Analyzing User Reviews of the First Digital Contraceptive: Mixed Methods Study
Felice, M. C., Søndergaard, M. L. J., & Balaam, M. (2023) · Journal of Medical Internet Research · doi:10.2196/47131 · Cited by 18 · see citing works
Uncovering Contradictions in Human-AI Interactions: Lessons Learned from User Reviews of Replika
Namvarpour, M., & Razi, A. (2024) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3678884.3681909 · Cited by 16 · see citing works
Exploring the Role of Mobile Apps for Insomnia in Depression: Systematic Review
Chiu, Y.-H., Lee, Y.-F., Lin, H., & Cheng, L. (2024) · Journal of Medical Internet Research · doi:10.2196/51110 · Cited by 12 · see citing works
Sentiment Analysis Using Machine Learning Approach Based on Feature Extraction for Anxiety Detection
Saifullah, S., Dreżewski, R., Dwiyanto, F. A., Aribowo, A. S., & Fauziah, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-36021-3_38 · Cited by 10 · see citing works
Singh, S., Gambill, J. L., Attalla, M., Fatima, R., Gill, A. R., & Siddiqui, H. F. (2024) · Cureus · doi:10.7759/cureus.71651 · Cited by 9 · see citing works
Review mining to discover user experience issues in mental health and wellbeing chatbots
Booth, F. G., Potts, C., Bond, R., Mulvenna, M. D., Ennis, E., & McTear, M. (2022) · European Conference on Cognitive Ergonomics · doi:10.1145/3552327.3552357 · Cited by 5 · see citing works
Papiernik, P., Dzula, S., Zimanyi, M., Millgate, E., Bouazzaoui, M., Buttimer, J., Warren, G., Cooper, E., Catarino, A., Mehew, S., et al. (2025) · JMIR Human Factors · doi:10.2196/76377 · Cited by 4 · see citing works
Reddit Perspectives on Tonsillectomy Procedure: Sentiment and Thematic Analysis
Herzog, A., Vought, V., Vought, R., Lee, A., Sharma, R., & Manzi, B. (2024) · Cureus · doi:10.7759/cureus.74254 · Cited by 4 · see citing works
Okuboyejo, S. R., Han, S., Jha, S., Eneja, C., & Orji, R. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2476711 · Cited by 2 · see citing works
Sentiment Analysis‐Based Model for Monitoring User Engagement With Mental Health Chatbots
Mmbayi, I. I., Gakii, C., & Musyoka, F. M. (2025) · Engineering Reports · doi:10.1002/eng2.70247 · Cited by 2 · see citing works
Alnefaie, A., Kang, K., & Sohaib, O. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4657676 · Cited by 4 · see citing works
Kostadinov, R., Vicheva, D., Paunov, L., Georgieva, M., Topalov, V., & Georgiev, S. (2023) · International Conference on Applied Mathematics & Computer Science (ICAMCS) · doi:10.1109/icamcs59110.2023.00017 · Cited by 1 · see citing works
Anxiety management strategies on TikTok: a discourse analysis of collective coping mechanisms
Balcıoğlu, Y. S. (2024) · Journal of Mental Health · doi:10.1080/09638237.2024.2390365 · Cited by 1 · see citing works
AI and VR-Powered Interventions for Social Anxiety
Boadu, D. O., Boafo, F., Owusu-Ansah, L. A., & Mensah, S. (2025) · International Journal of Virtual and Augmented Reality · doi:10.4018/ijvar.367871 · Cited by 1 · see citing works
Understanding the Long Haulers of COVID-19: Mixed Methods Analysis of YouTube Content
Jordan, A., & Park, A. (2024) · JMIR AI · doi:10.2196/54501 · Cited by 1 · see citing works
Wrightson‐Hester, A., Anderson, G., Dunstan, J., McEvoy, P. M., Sutton, C., MYERS, B. J., Egan, S. J., Tai, S., Johnston-Hollitt, M., Chen, W., et al. (2023) · doi:10.31234/osf.io/zjw8p · Cited by 1 · see citing works
This document will be updated.Preprint
Author (2024) · doi:10.31234/osf.io/fnbm8
USO DO CHATBOT NO ENFRENTAMENTO DA ANSIEDADE: UMA REVISÃO INTEGRATIVA
Ferreira, D. D. S., Martins, C. B., Barbalho, I. M. P., Maciel, N. D. S., de Oliveira, B. S. B., & Jorge, M. S. B. (2024) · Revista Enfermagem Atual In Derme · doi:10.31011/reaid-2024-v.98-n.2-art.2140
Liu, Q., Yao, Y., Evans, G. J., Huang, R., & Jin, M. (2024) · Proceedings of the Canadian Engineering Education Association (CEEA) · doi:10.24908/pceea.2024.18579
Ren, J., Luo, J., Huang, Y., Shetty, V., & Jeon, M. (2025) · Applied Sciences · doi:10.3390/app15116252
Thematic Analysis for Text Review Detection Using Machine Learning
Bhunia, R., Chakraborty, A., Santra, S., Sarkar, A., Jana, S., & Dasgupta, S. (2025) · doi:10.1002/9781394275359.ch27
Malaka, D., Wuisan, K., & Puspita, R. (2025) · International Conference on Applied Artificial Intelligence, Data Engineering and Sciences (ICAIDES) · doi:10.1109/icaides67265.2025.11404001
Anvari, S. S., Ppali, S., Hernandez, D. X. G., & Wehbe, R. R. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791326 · Cited by 1 · see citing works
Angermayr, K., Neuendorf, N. L., & Scherr, S. (2026) · Health Communication · doi:10.1080/10410236.2026.2666885
Cross-Cultural Analysis of User Perceptions and Acceptance of AI-Based Mental Health Chatbots
Wu, Y., Rau, P. P., & Liu, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29903-1_31
Higgins, O., Gildberg, F. A., & Wilson, R. L. (2026) · Springer eBooks · doi:10.1007/978-3-032-32552-5_8
Chatbots thérapeutiques en psychiatrie : entre simulation relationnelle et reconfiguration du soin
Bourla, A., Bechmann, M., Couette, M., Jacq, F., Ferreri, F., & Mouchabac, S. (2026) · L Encéphale · doi:10.1016/j.encep.2026.08.006
- Authors
- Ahmed, A., Aziz, S., Khalifa, M., Shah, U., Hassan, A., Abd-Alrazaq, A., & Househ, M.
- Affiliations
- Hamad bin Khalifa University (QA)
- Macquarie University (AU)
- Corresponding author
- Househ, Mowafa
- Venue
- JMIR Formative Research
- Publisher
- JMIR Publications Inc.
- Published
- 2022-03-11
- Volume
- 6
- Issue
- 3
- Pages
- e27654
- ISSN
- 2561-326X (ISSN-L)
- PMC
- PMC8956988
- OpenAlex
- W4206789363
- Cited by
- 37
- References
- 34
- Citations by year
- 2026: 6 · 2025: 10 · 2024: 14 · 2023: 6 · 2022: 1
- FWCI
- 4.71
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Formative Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Figshare (submittedVersion, cc-by)
- Funders
- Fonds National de la Recherche Luxembourg
- Qatar National Research Fund (NPRP12S-0303-190204, NPRP12S-)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Keywords
- depression, anxiety, online reviews, app reviews, user satisfaction, mental health apps, usability problems, privacy concerns, topic modeling, latent Dirichlet allocation, bigrams, sentiment analysis, digital mental health
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
