CoChat

Evaluating User Feedback for an Artificial Intelligence–Enabled, Cognitive Behavioral Therapy–Based Mental Health App (Wysa): Qualitative Thematic Analysis

Open access
Malik, Tanya, Ambrose, Adrian Jacques, Sinha, Chaitali · 2022 · JMIR Human Factors

Published version of a preprint

The DOI matches its Crossref record; 23 of the 25 references it cites were found in a registry or an index.

Abstract

Background: Digital mental health apps are rapidly becoming a common source of accessible support across the world, but their effectiveness is often influenced by limited helpfulness and engagement. Objective: This study’s primary objective was to analyze feedback content to understand users’ experiences with engaging with a digital mental health app. As a secondary objective, an exploratory analysis captured the types of mental health app users. Methods: This study utilized a user-led approach to understanding factors for engagement and helpfulness in digital mental health by analyzing feedback (n=7929) reported on Google Play Store about Wysa, a mental health app (1-year period). The analysis of keywords in the user feedback categorized and evaluated the reported user experience into the core domains of acceptability, usability, usefulness, and integration. The study also captured key deficits and strengths of the app and explored salient characteristics of the types of users who benefit from accessible digital mental health support. Results: The analysis of user feedback found the app to be overwhelmingly positively reviewed (6700/7929, 84.50% 5-star rating). The themes of engaging exercises, interactive interface, and artificial intelligence (AI) conversational ability indicated the acceptability of the app, while the nonjudgmentality and ease of conversation highlighted its usability. The app’s usefulness was portrayed by themes such as improvement in mental health, convenient access, and cognitive restructuring exercises. Themes of privacy and confidentiality underscored users’ preference for the integrated aspects of the app. Further analysis revealed 4 predominant types of individuals who shared app feedback on the store. Conclusions: Users reported therapeutic elements of a comfortable, safe, and supportive environment through using the digital mental health app. Digital mental health apps may expand mental health access to those unable to access traditional forms of mental health support and treatments.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/35668

Cites

23 of 23 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Diagram of the inclusion and exclusion criteria for user review analysis.
Figure 1 Diagram of the inclusion and exclusion criteria for user review analysis.CC BY · PubMed Central
Figure 2 Key themes from the reviews analyzed within the study. AI: artificial intelligence.
Figure 2 Key themes from the reviews analyzed within the study. AI: artificial intelligence.CC BY · PubMed Central
Figure 3 Example of interactive interface.
Figure 3 Example of interactive interface.CC BY · PubMed Central
Figure 4 Example of empathetic conversation.
Figure 4 Example of empathetic conversation.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 23 references

Research works printed but not linked: 3

  1. Torous, J, 2018, Current Psychiatry, 17, 21
  2. Myers, A, 2020, AMIA Annu Symp Proc, 2020, 906
  3. 10.2196/12106 · doi:10.2196/12106 · Paper page

Cited by

Metadata

Authors
Malik, T., Ambrose, A. J., & Sinha, C.
Affiliations
Columbia University (US)
Corresponding author
Malik, Tanya
Venue
JMIR Human Factors
Published
2022-04-12
Volume
9
Issue
2
Pages
e35668
ISSN
2292-9495 (ISSN-L)
OpenAlex
W4220854739
Cited by
83
References
25
Citations by year
2026: 20 · 2025: 28 · 2024: 19 · 2023: 8 · 2022: 8
FWCI
13.87
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR Human Factors (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
Keywords
mental health apps, Wysa, cognitive behavioral therapy, AI chatbots, user feedback, app engagement, acceptability, usability, helpfulness, privacy and confidentiality, user types, cognitive restructuring, access to mental health care

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.