The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Challenges in Assessing Mobile Health App Quality
Abstract
Context Mobile apps are poised to become a major source of health guidance; however, concerns about their quality and safety remain. Governments, clinicians, and health researchers are struggling to determine how to adequately evaluate the content and function of mobile health apps to guide consumers toward apps that will effectively and safely support their health. Researchers are both adapting existing research methods, such as those for systematic reviews, and exploring novel methods toward this aim. The purpose of this systematic review was to describe prevalent and emerging methods for searching, data extraction, and analysis in the context of mobile health apps and commercial app stores.Evidence acquisition A systematic search was conducted on April 6, 2016, according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, for published studies that evaluated health-related apps targeted at consumers in a commercial app store, and a descriptive content analysis of their methods was performed.Evidence synthesis Although the majority of studies self-identified as "reviews" or "systematic reviews," their claims to have performed an exhaustive, replicable, and systematic search and data extraction are unsupported, given the nature of commercial app stores' search engines and personalized app content. Currently, studies of health-related app content are largely descriptive, and most use surrogate and one-dimensional outcome measures for app content quality.Conclusions This field is rapidly developing, and innovative methods for systematic searches, multidimensional quality assessment, and contextualizing content assessment are emerging. Areas for future methodologic development are highlighted, including expanding definitions of app "content" and assessing risk of bias.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Search results are personalized based on user characteristics (Grundy et al. 2016)”
From Evaluating the quality, safety, and functionality of commonly used smartphone apps for bipolar disorder mood and sleep self-management (2022) · cites this paper as Grundy et al. 2016
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Mobile app rating scale: a new tool for assessing the quality of health mobile apps
Stoyanov, S. R., Hides, L., Kavanagh, D. J., Zelenko, O., Tjondronegoro, D., & Mani, M. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.3422 · Cited by 2,233
Just a Fad? Gamification in Health and Fitness Apps
Lister, C., West, J. H., Cannon, B., Sax, T., & Brodegard, D. (2014) · JMIR Serious Games · doi:10.2196/games.3413 · Cited by 489
Mobile Apps for Bipolar Disorder: A Systematic Review of Features and Content Quality
Nicholas, J., Larsen, M. E., Proudfoot, J., & Christensen, H. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4581 · Cited by 438
Huckvale, K., Prieto, J. T., Tilney, M., Benghozi, P.-J., & Car, J. (2015) · BMC Medicine · doi:10.1186/s12916-015-0444-y · Cited by 376
Bardus, M., van Beurden, S. B., Smith, J. R., & Abraham, C. (2016) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-016-0359-9 · Cited by 350
Mobile Applications for Weight Management
Azar, K. M. J., Lesser, L. I., Laing, B. Y., Stephens, J., Aurora, M. S., Burke, L. E., & Palaniappan, L. P. (2013) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2013.07.005 · Cited by 334
Direito, A., Dale, L. P., Shields, E., Dobson, R., Whittaker, R., & Maddison, R. (2014) · BMC Public Health · doi:10.1186/1471-2458-14-646 · Cited by 308
Cited by
Sorted by citations
Criteria for assessing the quality of mHealth apps: a systematic review
Nouri, R., R Niakan Kalhori, S., Ghazisaeedi, M., Marchand, G., & Yasini, M. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy050 · Cited by 231
“More recently, Grundy et al.(60) conducted a systematic review to find and explain emerging and common methods for evaluating the quality of mHealth apps.”
BackgroundSmartphone apps for the self-management of low back pain: A systematic review
Machado, G. C., Pinheiro, M. B., Lee, H., Ahmed, O. H., Hendrick, P. A., Williams, C. M., & Kamper, S. J. (2016) · Best Practice & Research Clinical Rheumatology · doi:10.1016/j.berh.2017.04.002 · Cited by 171
“assessing the quality and ‘fitness-for-purpose’ of apps is by using validated scales, and benchmarking app content against current best-practice guidelines [10].”
MethodMobile Apps for Medication Management: Review and Analysis
Tabi, K., Randhawa, A. S., Choi, F., Mithani, Z., Albers, F., Schnieder, M., Nikoo, M., Vigo, D., Jang, K. L., Demlová, R., et al. (2019) · JMIR mhealth and uhealth · doi:10.2196/13608 · Cited by 178
“The professional reviews could follow some of the standardized frameworks for quality assessment of mobile apps, for example, Mobile App Rating Scale [49], suitable for a wide range of health apps; a recently published assessment framework specifically for mental health apps [50]; or other published frameworks [33,51-…”
BackgroundBroglia, E., Millings, A., & Barkham, M. (2018) · British Journal of Guidance and Counselling · doi:10.1080/03069885.2017.1370695 · Cited by 141
A Review of the Quality and Impact of Mobile Health Apps
Grundy, Q. (2021) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-052020-103738 · Cited by 167
Self-citation
Metadata
- Authors
- Grundy, Q. H., Wang, Z., & Bero, L. A.
- Affiliations
- The University of Sydney (AU)
- Corresponding author
- Grundy, Quinn H.
- Venue
- American Journal of Preventive Medicine
- Publisher
- Elsevier BV
- Published
- 2016-12
- Volume
- 51
- Issue
- 6
- Pages
- 1051-1059
- ISSN
- 0749-3797 (ISSN-L)
- OpenAlex
- W2521913084
- Cited by
- 154
- References
- 41
- Citations by year
- 2026: 4 · 2025: 8 · 2024: 8 · 2023: 17 · 2022: 11 · 2021: 24 · 2020: 26 · 2019: 20 · 2018: 26 · 2017: 9 · 2016: 1
- FWCI
- 3.57
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Journal of Preventive Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Keywords
- mobile health apps, app quality assessment, risk of bias
- MeSH terms
- Health Promotion, Mobile Applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
