Mobile health app usability and quality rating scales: a systematic review
The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
PURPOSE: To review the rating scales used to evaluate usability and quality of mobile health applications, and to compare their purpose, content, and intended target users (i.e., patients, caregivers, or researchers). MATERIAL AND METHODS: We conducted a systematic review of the literature in accordance with the PRISMA statement on Medline, CINAHL, PsycINFO, IEEE Explore databases, as well as a review of the grey literature to identify rating scales used to evaluate usability and quality of mobile health applications (m-health apps), between January 1, 2000 and July 31, 2018. Two researchers screened the titles and abstracts of articles that met inclusion criteria, and retrieved usability and quality rating scales from the articles. RESULTS: We identified 24 usability scales and 25 quality rating scales in 87 peer-reviewed articles. We identified only one quality rating scale designed for non-expert users (i.e., patients or caregivers). None of the studies used a theoretical framework for app evaluation to support the scales. The validity of existing quality rating scales is yet to be investigated. CONCLUSION: Existing usability and quality rating scales are targeted at professionals, not end users who are patients or caregivers. Rating scales that are usable by all end-users would make mobile health apps accessible and meaningful to consumers.Implications for rehabilitationThe number of mobile health applications on app stores that can be used for rehabilitation is increasing.Most healthcare providers lack the training to identify m-health apps with high quality to be used in rehabilitation.This study has reviewed the current rating scales that can help clinicians and care providers rate the quality of m-health apps and identify the ones that are most appropriate for their practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2010) · International Journal of Surgery · doi:10.1016/j.ijsu.2010.02.007 · Cited by 12,156 · cited 1×
“We conducted this review following the PRISMA statement (Preferred Reporting Items for Systematic reviews and Meta-Analyses) [12].”
MethodA typology of reviews: an analysis of 14 review types and associated methodologies
Grant, M. J., & Booth, A. (2009) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2009.00848.x · Cited by 10,817
IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use
Lewis, J. R. (1995) · International Journal of Human-Computer Interaction · doi:10.1080/10447319509526110 · Cited by 2,429
Mobile app rating scale: a new tool for assessing the quality of health mobile apps
Stoyanov, S. R., Hides, L., Kavanagh, D. J., Zelenko, O., Tjondronegoro, D., & Mani, M. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.3422 · Cited by 2,233
Development and validation of the user version of the Mobile application rating scale (uMARS)
Stoyanov, S. R., Hides, L., Kavanagh, D. J., & Wilson, H. (2016) · JMIR mhealth and uhealth · doi:10.2196/mhealth.5849 · Cited by 872 · cited 1×
“…Health Care Apps Evaluation Tool [73], Royal College of Physicians ’ Health Informatics Unit checklist [74], Organisation for the Review of Care and Health Applications-24 Question Assessment (ORCHA-24) [75], and Mobile Application Rating Scale (MARS) [76 – 92] and its user ’ s version U-MARS [93].”
Review and evaluation of mindfulness-based iPhone apps
Mani, M., Kavanagh, D. J., Hides, L., & Stoyanov, S. R. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.4328 · Cited by 495
Bardus, M., van Beurden, S. B., Smith, J. R., & Abraham, C. (2016) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-016-0359-9 · Cited by 350
A Systematic Review of Cognitive Behavioral Therapy and Behavioral Activation Apps for Depression
Huguet, A., Rao, S., McGrath, P. J., Wozney, L., Wheaton, M., Conrod, J., & Rozario, S. (2016) · PLOS ONE · doi:10.1371/journal.pone.0154248 · Cited by 328 · cited 1×
“…Not named [105] Overview (usability), Drugs, Conditions, Microbes, Bite infections, Endocarditis Inter-rater agreement: kappa > 0.70 Not named [106] Visibility of system status, Match between system and the real world, User control and freedom, Consistency and standards, Error prevention,…”
BackgroundInfluentialFull Publication of Results Initially Presented in Abstracts: A Meta-analysis
Scherer, R. W. (1994) · JAMA · doi:10.1001/jama.1994.03520020084025 · Cited by 296 · cited 1×
“, quantitative, qualitative, or mixed methods) and papers with positive and negative results to reduce the effect of publication bias [14].”
BackgroundSchnall, R., Cho, H., & Liu, J. (2018) · JMIR mHealth and uHealth · doi:10.2196/mhealth.8851 · Cited by 239
Cited by
AI, Health, and Health Care Today and Tomorrow: The JAMA Summit Report on Artificial Intelligence
Angus, D. C., Khera, R., Lieu, T., Liu, V., Ahmad, F. S., Anderson, B., Bhavani, S. V., Bindman, A., Brennan, T., Celi, L. A., et al. (2025) · JAMA · doi:10.1001/jama.2025.18490 · Cited by 195
Teixeira, P., Eusébio, C., & Teixeira, L. (2024) · Requirements Engineering · doi:10.1007/s00766-023-00409-8 · Cited by 26
Sahan, F., Guthardt, L., Panitz, K., Siegel-Kianer, A., Eichhof, I., Schmitt, B. D., & Apolinario-Hagen, J. (2024) · JMIR Medical Education · doi:10.2196/59454 · Cited by 14
Metadata
- Authors
- Azad-Khaneghah, P., Neubauer, N., Miguel Cruz, A., & Liu, L.
- Venue
- Disability and Rehabilitation: Assistive Technology
- Publisher
- Informa UK Limited
- Published
- 2020-01-08
- Volume
- 16
- Issue
- 7
- Pages
- 712-721
- ISSN
- 1748-3107 (ISSN-L)
- 1748-3115
- OpenAlex
- W2999555376
- Cited by
- 115
- References
- 29
- FWCI
- 4.20
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Mobile health app usability and quality rating scales: a systematic review. https://app.cochat.ai/papers/10.1080/17483107.2019.1701103
- The paper
- Azad-Khaneghah, P., Neubauer, N., Miguel Cruz, A., & Liu, L. (2021). Mobile health app usability and quality rating scales: a systematic review. Disability and Rehabilitation: Assistive Technology. https://doi.org/10.1080/17483107.2019.1701103
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.