Usability Evaluations of Mobile Mental Health Technologies: Systematic Review
Open accessThe DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
Background: Many mobile health (mHealth) apps for mental health have been made available in recent years. Although there is reason to be optimistic about their effect on improving health and increasing access to care, there is a call for more knowledge concerning how mHealth apps are used in practice. Objective: This study aimed to review the literature on how usability is being addressed and measured in mHealth interventions for mental health problems. Methods: We conducted a systematic literature review through a search for peer-reviewed studies published between 2001 and 2018 in the following electronic databases: EMBASE, CINAHL, PsycINFO, PubMed, and Web of Science. Two reviewers independently assessed all abstracts against the inclusion and exclusion criteria, following the Preferred Reporting Items for Systematic Review and Meta-Analysis guidelines. Results: A total of 299 studies were initially identified based on the inclusion keywords. Following a review of the title, abstract, and full text, 42 studies were found that fulfilled the criteria, most of which evaluated usability with patients (n=29) and health care providers (n=11) as opposed to healthy users (n=8) and were directed at a wide variety of mental health problems (n=24). Half of the studies set out to evaluate usability (n=21), and the remainder focused on feasibility (n=10) or acceptability (n=10). Regarding the maturity of the evaluated systems, most were either prototypes or previously tested versions of the technology, and the studies included few accounts of sketching and participatory design processes. The most common reason referred to for developing mobile mental health apps was the availability of mobile devices to users, their popularity, and how people in general became accustomed to using them for various purposes. Conclusions: This study provides a detailed account of how evidence of usability of mHealth apps is gathered in the form of usability evaluations from the perspective of computer science and human-computer interaction, including how users feature in the evaluation, how the study objectives and outcomes are stated, which research methods and techniques are used, and what the notion of mobility features is for mHealth apps. Most studies described their methods as trials, gathered data from a small sample size, and carried out a summative evaluation using a single questionnaire, which indicates that usability evaluation was not the main focus. As many studies described using an adapted version of a standard usability questionnaire, there may be a need for developing a standardized mHealth usability questionnaire.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Torous, J., Nicholas, J., Larsen, M. E., Firth, J., & Christensen, H. (2018) · Evidence Based Mental Health · doi:10.1136/eb-2018-102891 · Cited by 790
Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications
Zhang, D., & Adipat, B. (2005) · International Journal of Human-Computer Interaction · doi:10.1207/s15327590ijhc1803_3 · Cited by 613
Feasibility, Acceptability, and Preliminary Efficacy of a Smartphone Intervention for Schizophrenia
Ben-Zeev, D., Brenner, C. J., Begale, M., Duffecy, J., Mohr, D. C., & Mueser, K. T. (2014) · Schizophrenia Bulletin · doi:10.1093/schbul/sbu033 · Cited by 493
How to evaluate technologies for health behavior change in HCI research
Klasnja, P., Consolvo, S., & Pratt, W. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979396 · Cited by 414 · cited 3×
“Klasnja et al [69] argued that electronic health technology evaluations should refrain from documenting behavioral changes, as behavioral change processes (1) are inherently complex, that is, subject to interconnected social, material-logistical, motivational, and circumstantial factors, and (2)…”
InfluentialSmartphone Apps for Schizophrenia: A Systematic Review
Firth, J., & Torous, J. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.4930 · Cited by 268 · cited 1×
“Firth and Torous [2] argue that mobile technologies are particularly suitable to provide services for behavioral health, such as psychiatry, because of the opportunities for capturing patient behavior, for example, through ecological momentary assessment [3] and providing real-time support, given…”
Mobile Apps for Increasing Treatment Adherence: Systematic Review
Pérez-Jover, V., Sala-González, M., Guilabert, M., & Mira, J. J. (2019) · Journal of Medical Internet Research · doi:10.2196/12505 · Cited by 204 · cited 1×
“As in our review, Perez-Jover et al [80] found that usability evaluation practices in mHealth varied substantially.”
The application of mHealth to mental health: opportunities and challenges
Marzano, L., Bardill, A., Fields, B., Herd, K., Veale, D., Grey, N., & Moran, P. (2015) · The Lancet Psychiatry · doi:10.1016/s2215-0366(15)00268-0 · Cited by 200
A mobile app–based intervention for depression: end-user and expert usability testing study
Fuller-Tyszkiewicz, M., Richardson, B., Klein, B., Skouteris, H., Christensen, H., Austin, D., Castle, D., Mihalopoulos, C., O'Donnell, R., Arulkadacham, L., et al. (2018) · JMIR Mental Health · doi:10.2196/mental.9445 · Cited by 98 · cited 21×
“…[40]; Nicholson et al, 2018 [44]; Ospina-Pinillos et al, 2018 [53] Privacy of mobile use Deady et al, 2018 [32]; Fuller-Tyszkiewicz et al, 2018 [34]; Morland et al, 2016 [43]; Wood et al, 2017 [56] Scientific evidence of a positive effect Ben-Zeev et al, 2013 [27]; Macias et al, 2015 [40];…”
InfluentialUsability of a new electronic assistive device for community-dwelling persons with mild dementia
Meiland, F. J. M., Bouman, A. I. E., Sävenstedt, S., Bentvelzen, S., Davies, R. J., Mulvenna, M. D., Nugent, C. D., Moelaert, F., Hettinga, M. E., Bengtsson, J. E., et al. (2012) · Aging & Mental Health · doi:10.1080/13607863.2011.651433 · Cited by 94 · cited 20×
“…Jiam et al, 2017 [65]; Juengst et al, 2015 [37]; Kizakevich et al, 2018 [61]; Kobak et al, 2015 [38]; Macias et al, 2015 [40]; Meiland et al, 2012 [41]; Mistler et al, 2017 [42]; Morland et al, 2016 [43]; Nicholson et al, 2018 [44]; Palmier-Claus et al, 2013 [46]; Prada et al, 2017 [47]; Rizvi et…”
InfluentialSze, Y. Y., Daniel, T. O., Kilanowski, C. K., Collins, R. L., & Epstein, L. H. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.4603 · Cited by 94
Cited by
Haque, M. R., & Rubya, S. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555146 · Cited by 48
Tucker, L., Villagomez, A. C., & Krishnamurti, T. (2021) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-021-03785-7 · Cited by 43
Sahan, F., Guthardt, L., Panitz, K., Siegel-Kianer, A., Eichhof, I., Schmitt, B. D., & Apolinario-Hagen, J. (2024) · JMIR Medical Education · doi:10.2196/59454 · Cited by 14
Internet-delivered self-help for adults with ADHD (MyADHD): usability study
Kenter, R. M. F., Schønning, A., & Inal, Y. (2022) · JMIR Formative Research · doi:10.2196/37137 · Cited by 10
“It can be summative (collecting evidence that the intervention is usable) or formative (informing the redesign and improvement of the intervention) [17].”
BackgroundWada, A., Nakamura, Y., Kawajiri, M., Takeishi, Y., Yoshida, M., & Yoshizawa, T. (2023) · JMIR Formative Research · doi:10.2196/48637 · Cited by 2
Metadata
- Authors
- Inal, Y., Wake, J. D., Guribye, F., & Nordgreen, T.
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2020-01-06
- Volume
- 22
- Issue
- 1
- Pages
- e15337
- ISSN
- 1438-8871 (ISSN-L)
- OpenAlex
- W2981907270
- Cited by
- 107
- References
- 20
- FWCI
- 1.50
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Usability Evaluations of Mobile Mental Health Technologies: Systematic Review. https://app.cochat.ai/papers/10.2196/15337
- The paper
- Inal, Y., Wake, J. D., Guribye, F., & Nordgreen, T. (2020). Usability Evaluations of Mobile Mental Health Technologies: Systematic Review. Journal of Medical Internet Research. https://doi.org/10.2196/15337
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.