CoChat
Sign Up
Cited by 48Digital Mental Health InterventionsMobile Health and mHealth ApplicationsInnovative Human-Technology Interaction

"For an App Supposed to Make Its Users Feel Better, It Sure is a Joke" - An Analysis of User Reviews of Mobile Mental Health Applications

Open access
Haque, Md Romael, Rubya, Sabirat·2022·Proceedings of the ACM on Human-Computer Interaction
DOI 10.1145/3555146
Open this paper in CoChat — free accountOpen PDF

Abstract

Mobile mental health applications are seen as a promising way to fulfill the growing need for mental health care. Although there are more than ten thousand mental health apps available on app marketplaces, such as Google Play and Apple App Store, many of them are not evidence-based, or have been minimally evaluated or regulated. The real-life experience and concerns of the app users are largely unknown. To address this knowledge gap, we analyzed 2159 user reviews from 117 Android apps and 2764 user reviews from 76 iOS apps. Our findings include the critiques around inconsistent moderation standards and lack of transparency. App-embedded social features and chatbots were criticized for providing little support during crises. We provide research and design implications for future mental health app developers, discuss the necessity of developing a comprehensive and centralized app development guideline, and the opportunities of incorporating existing AI technology in mental health chatbots.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3555146

Cites

79 of 79 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • User engagement remains a persistent challenge, with mixed user reviews, and studies indicating that 30 days after installation the proportion of users still active may be as low as 3%.

    “Moreover, user engagement remains a persistent challenge, with mixed user reviews11, and studies indicating that 30 days after installation the proportion of users still active may be as low as 3%12.”

    From “It happened to be the perfect thing”: experiences of generative AI chatbots for mental health (2024) · cites this paper as [11]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 29

  1. Felwah Alqahtani and Rita Orji. 2020. Insights from user reviews to improve mental health apps. Health informatics journal 26, 3 (2020), 2042-2066.
  2. David Bakker, Nikolaos Kazantzis, Debra Rickwood, and Nikki Rickard. 2016. Mental health smartphone apps: review and evidence-based recommendations for future developments. JMIR mental health 3, 1 (2016), e4984.
  3. Aashutosh Bhatt, Ankit Patel, Harsh Chheda, and Kiran Gawande. 2015. Amazon review classification and sentiment analysis. International Journal of Computer Science and Information Technologies 6, 6 (2015), 5107-5110.
  4. Erling Bjögvinsson, Pelle Ehn, and Per-Anders Hillgren. 2012. Design things and design thinking: Contemporary participatory design challenges. Design issues 28, 3 (2012), 101-116.
  5. Gillian Cameron, David Cameron, Gavin Megaw, Raymond Bond, Maurice Mulvenna, Siobhan O'Neill, Cherie Ar- mour, and Michael McTear. 2018. Assessing the usability of a chatbot for mental health care. In International Con- ference on Internet Science. Springer, 121-132.
  6. Andrew D Carlo, Reza Hosseini Ghomi, Brenna N Renn, and Patricia A Areán. 2019. By the numbers: ratings and utilization of behavioral health mobile applications. NPJ digital medicine 2, 1 (2019), 1-8.
  7. Tya Chuanromanee and Ronald Metoyer. 2020. Evaluation and Comparison of Four Mobile Breathing Training Visualizations. In 2020 IEEE International Conference on Healthcare Informatics (ICHI). IEEE, 1-12.
  8. Federal Trade Commission et al. 2015. FTC charges marketers of 'vision improvement'app with deceptive claims. USA: https://www. ftc. gov/news-events/press-releases/2015/09/ftc-chargesmarketers-vision-improvement-app-deceptive- claims (2015).
  9. Federal Trade Commission et al. 2015. FTC cracks down on marketers of "Melanoma Detection" apps. Retrieved on February 1 (2015), 2016.
  10. Maria Faurholt-Jepsen, Mads Frost, Christian Ritz, Ellen Margrethe Christensen, AS Jacoby, Rie Lambæk Mikkelsen, U Knorr, JE Bardram, Maj Vinberg, and Lars Vedel Kessing. 2015. Daily electronic self-monitoring in bipolar disor- der using smartphones-the MONARCA I trial: a randomized, placebo-controlled, single-blind, parallel group trial. Psychological medicine 45, 13 (2015), 2691-2704.
  11. Chris Forman, Anindya Ghose, and Batia Wiesenfeld. 2008. Examining the relationship between reviews and sales: The role of reviewer identity disclosure in electronic markets. Information systems research 19, 3 (2008), 291-313.
  12. Jennifer Green, Jennifer Huberty, Megan Puzia, Chad Stecher, et al. 2021. The Effect of Meditation and Physical Activity on the Mental Health Impact of COVID-19-Related Stress and Attention to News Among Mobile App Users in the United States: Cross-sectional Survey. JMIR mental health 8, 4 (2021), e28479.
  13. Maurice Kelly and Joshua Nozzi. 2013. Mastering Xcode: Develop and Design. Peachpit Press.
  14. Michelle E Kiger and Lara Varpio. 2020. Thematic analysis of qualitative data: AMEE Guide No. 131. Medical teacher 42, 8 (2020), 846-854.
  15. Adam S Miner, Arnold Milstein, Stephen Schueller, Roshini Hegde, Christina Mangurian, and Eleni Linos. 2016. Smartphone-based conversational agents and responses to questions about mental health, interpersonal violence, and physical health. JAMA internal medicine 176, 5 (2016), 619-625.
  16. Tally Moses. 2009. Self-labeling and its effects among adolescents diagnosed with mental disorders. Social science & medicine 68, 3 (2009), 570-578.
  17. Susan M Mudambi and David Schuff. 2010. Research note: What makes a helpful online review? A study of customer reviews on Amazon. com. MIS quarterly (2010), 185-200.
  18. P Muppirishetty and Minha Lee. 2020. Voice user interfaces for mental healthcare: leveraging technology to help our inner voice. In 3rd ACM Conference on Computer-Supported Cooperative Work and Social Computing, CSCW 2020.
  19. Dennis Pagano and Walid Maalej. 2013. User feedback in the appstore: An empirical study. In 2013 21st IEEE inter- national requirements engineering conference (RE). IEEE, 125-134.
  20. Lisa Parker, Tanya Karliychuk, Donna Gillies, Barbara Mintzes, Melissa Raven, and Quinn Grundy. 2017. A health app developer's guide to law and policy: a multi-sector policy analysis. BMC medical informatics and decision making 17, 1 (2017), 1-13.
  21. Elisabeth Platzer. 2011. Opportunities of automated motive-based user review analysis in the context of mobile app acceptance. In Central European Conference on Information and Intelligent Systems. Faculty of Organization and Informatics Varazdin, 309.
  22. Danielle Schlosser, Timothy Campellone, Daniel Kim, Brandy Truong, Silvia Vergani, Charlie Ward, and Sophia Vinogradov. 2016. Feasibility of PRIME: a cognitive neuroscience-informed mobile app intervention to enhance motivated behavior and improve quality of life in recent onset schizophrenia. JMIR research protocols 5, 2 (2016), e5450.
  23. Nelson Shen, Michael-Jane Levitan, Andrew Johnson, Jacqueline Lorene Bender, Michelle Hamilton-Page, Alejandro Alex R Jadad, and David Wiljer. 2015. Finding a depression app: a review and content analysis of the depression app marketplace. JMIR mHealth and uHealth 3, 1 (2015), e3713.
  24. Felix Ter Chian Tan and Rajesh Vasa. 2011. Toward a social media usage policy. In Proc. ACIS. 84-89.
  25. Kong Saoane Thach. 2019. A Qualitative Analysis of User Reviews on Mental Health Apps: Who Used it? for What? and Why?. In 2019 IEEE-RIVF International Conference on Computing and Communication Technologies (RIVF). IEEE, 1-4.
  26. John Torous, Jennifer Nicholas, Mark E Larsen, Joseph Firth, and Helen Christensen. 2018. Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements. Evidence-based mental health 21, 3 (2018), 116-119.
  27. Carolyn Windler, Maeve Clair, Cassandra Long, Leah Boyle, and Ana Radovic. 2019. Role of moderators on en- gagement of adolescents with depression or anxiety in a social media intervention: content analysis of web-based interactions. JMIR mental health 6, 9 (2019), e13467.
  28. Qiang Ye, Ziqiong Zhang, and Rob Law. 2009. Sentiment classification of online reviews to travel destinations by supervised machine learning approaches. Expert systems with applications 36, 3 (2009), 6527-6535.
  29. Michael Zimmer. 2010. "But the data is already public": on the ethics of research in Facebook. Ethics and information technology 12, 4 (2010), 313-325.

Cited by

Sorted by citations

Related

Authors
Haque, M. R., & Rubya, S.
Affiliations
Marquette University (US)
ORCID
Venue
Proceedings of the ACM on Human-Computer Interaction
Publisher
Association for Computing Machinery (ACM)
Published
2022-11-07
Volume
6
Issue
CSCW2
Pages
1-29
ISSN
2573-0142 (ISSN-L)
DOI
10.1145/3555146
OpenAlex
W4309618418
Cited by
48
References
79
Citations by year
2026: 10 · 2025: 20 · 2024: 8 · 2023: 10
FWCI
8.37
Type
journal article
Language
en
Open access
Licence
Locations
  • Proceedings of the ACM on Human-Computer Interaction (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
Northwestern Mutual Data Science Institute (#01810-43826)
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
Keywords
mental health apps, online reviews, app ratings, transparency, chatbots, crisis support, social features, Android application, iOS app

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review