The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Challenges in Assessing Mobile Health App Quality
Abstract
Context Mobile apps are poised to become a major source of health guidance; however, concerns about their quality and safety remain. Governments, clinicians, and health researchers are struggling to determine how to adequately evaluate the content and function of mobile health apps to guide consumers toward apps that will effectively and safely support their health. Researchers are both adapting existing research methods, such as those for systematic reviews, and exploring novel methods toward this aim. The purpose of this systematic review was to describe prevalent and emerging methods for searching, data extraction, and analysis in the context of mobile health apps and commercial app stores.Evidence acquisition A systematic search was conducted on April 6, 2016, according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, for published studies that evaluated health-related apps targeted at consumers in a commercial app store, and a descriptive content analysis of their methods was performed.Evidence synthesis Although the majority of studies self-identified as "reviews" or "systematic reviews," their claims to have performed an exhaustive, replicable, and systematic search and data extraction are unsupported, given the nature of commercial app stores' search engines and personalized app content. Currently, studies of health-related app content are largely descriptive, and most use surrogate and one-dimensional outcome measures for app content quality.Conclusions This field is rapidly developing, and innovative methods for systematic searches, multidimensional quality assessment, and contextualizing content assessment are emerging. Areas for future methodologic development are highlighted, including expanding definitions of app "content" and assessing risk of bias.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Search results are personalized based on user characteristics (Grundy et al. 2016)”
From Evaluating the quality, safety, and functionality of commonly used smartphone apps for bipolar disorder mood and sleep self-management (2022) · cites this paper as Grundy et al. 2016
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Mobile app rating scale: a new tool for assessing the quality of health mobile apps
Stoyanov, S. R., Hides, L., Kavanagh, D. J., Zelenko, O., Tjondronegoro, D., & Mani, M. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.3422 · Cited by 2,233
Just a Fad? Gamification in Health and Fitness Apps
Lister, C., West, J. H., Cannon, B., Sax, T., & Brodegard, D. (2014) · JMIR Serious Games · doi:10.2196/games.3413 · Cited by 489
Mobile Apps for Bipolar Disorder: A Systematic Review of Features and Content Quality
Nicholas, J., Larsen, M. E., Proudfoot, J., & Christensen, H. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4581 · Cited by 438
Huckvale, K., Prieto, J. T., Tilney, M., Benghozi, P.-J., & Car, J. (2015) · BMC Medicine · doi:10.1186/s12916-015-0444-y · Cited by 376
Bardus, M., van Beurden, S. B., Smith, J. R., & Abraham, C. (2016) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-016-0359-9 · Cited by 350
Mobile Applications for Weight Management
Azar, K. M. J., Lesser, L. I., Laing, B. Y., Stephens, J., Aurora, M. S., Burke, L. E., & Palaniappan, L. P. (2013) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2013.07.005 · Cited by 334
Direito, A., Dale, L. P., Shields, E., Dobson, R., Whittaker, R., & Maddison, R. (2014) · BMC Public Health · doi:10.1186/1471-2458-14-646 · Cited by 308
Cited by
Sorted by citations
Items 101–153 of 153
User Experience of Symptom Checkers: A Systematic Review
You, Y., Ma, R., & Gui, X. (2022) · PubMed Central · doi:10.48550/arxiv.2208.09100 · Cited by 8
Lin, Y., Lemos, M., & Neuschaefer‐Rube, C. (2022) · JMIR Medical Education · doi:10.2196/34042 · Cited by 6
“Although some have engaged expert opinion, others have not and often do not have the clinical insights to ensure the scientific or clinical quality of the resource, unlike academic institutions where quality evaluation of evidence base may be easier to ensure; however, they may not always be effective [67,68].”
BackgroundMcClung, H. L., Raynor, H. A., Volpe, S. L., Dwyer, J. T., & Papoutsakis, C. (2021) · Journal of the Academy of Nutrition and Dietetics · doi:10.1016/j.jand.2021.02.028 · Cited by 7
“In 2018, it was estimated that there were more than 160,000 mobile health applications to track DI/ PA patterns, and over $500 million was spent on these applications.(13)”
Perera, A. G., Sancha, P., & Jiménez, M. D. A. J. (2021) · Journal of Medical Systems · doi:10.1007/s10916-021-01748-1 · Cited by 8
Black, D. A., O’Loughlin, K., & Wilson, L. A. (2018) · Australian Journal on Ageing · doi:10.1111/ajag.12522 · Cited by 6
Priyanka, M. G. S., & Subashini, R. (2024) · International Research Journal of Multidisciplinary Scope · doi:10.47857/irjms.2024.v05i02.0398 · Cited by 7
“Based on the findings of the literature review, it has been shown that the core of ergonomic innovations lies in their ability to effectively enhance operational performance inside organizations (26).”
BackgroundMorley, J., Laitila, J., Ross, J. S., Schamroth, J., Zhang, J., & Floridi, L. (2025) · Minds and Machines · doi:10.1007/s11023-025-09710-7 · Cited by 5
“…Health and Fitness or Lifestyle apps, combined with the fact that app developers themselves select which category is most appropriate for their app (Grundy et al., 2016), undermine the argument that Apple’s developer guidelines are having a positive (and intended) effect on the quality and safety…”
García-Pascual, F., Valcarce-Torrente, M., Moreno, F. C., & Fernández, J. G. (2023) · Humanities and Social Sciences Communications · doi:10.1057/s41599-023-02330-5 · Cited by 5
Conteúdo sobre drogas em jogos para dispositivos móveis
Pasquim, H. M., Lachtim, S. A. F., & Soares, C. B. (2019) · Revista da Escola de Enfermagem da USP · doi:10.1590/s1980-220x2018039403520 · Cited by 5
Krochinak, S., Cui, S., Ajayi, B., Egonu, R., & Kim, E. (2023) · Science Insights Education Frontiers · doi:10.15354/sief.23.or136 · Cited by 6
Saito, C. M., & Sasso, G. T. M. D. (2023) · Acta Paulista de Enfermagem · doi:10.37689/acta-ape/2023ar02352 · Cited by 7
Fang, J., Lee, V. C. S., & Wang, H. (2024) · Industrial Management & Data Systems · doi:10.1108/imds-03-2023-0173 · Cited by 5
Crowdsourcing smartphone data for biomedical research: Ethical and legal questions
Lang, M., McKibbin, K., Shabani, M., Borry, P., Gautrais, V., Verbeke, K., & Zawati, M. H. (2023) · Digital Health · doi:10.1177/20552076231204428 · Cited by 4
“Datasets that are not demographically representative of the populations from which they are derived might work to entrench existing lines of discrimination and inequality.(26) And though AI laws may try to explicitly address these concerns, the regimes in question may apply inconsistently to the contexts in which SCMD…”
BackgroundFrom Ancient Contemplative Practice to the App Store: Designing a Digital Container for MindfulnessPreprint
Lukoff, K., Lyngs, U., Gueorguieva, S., Dillman, E., Hiniker, A., & Munson, S. A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.09521 · Cited by 4
“n-distress indicates that they are not receiving adequate support from wherever they learned to meditate” [1]. Overstated benefits and understated risks have been reported in other health-related apps [35] and our results suggest this issue may be prevalent in mindfulness-related apps, too. Designers of technologies fo…”
BackgroundUsability Assessment Methods for Mobile Apps for Physical Rehabilitation: Umbrella Review
Hach, S., Alder, G., Stavric, V., Taylor, D., & Signal, N. (2024) · JMIR mhealth and uhealth · doi:10.2196/49449 · Cited by 6
Wiemker, V., Neufeld, M., Bunova, A., Danquah, I., Ferreira‐Borges, C., Konigorski, S., Rastogi, A., & Probst, C. (2021) · Journal of Medical Internet Research · doi:10.2196/28927 · Cited by 3
Tax management supported by mechanisms of technological innovation
Tuárez, D. L. C., Caicedo, R. J., & Plaza, C. L. M. (2022) · International Journal of Social Sciences and Humanities · doi:10.53730/ijssh.v6n1.6283 · Cited by 2
Multi-Criteria Evaluation of Mobile Fitness Applications During COVID-19 Pandemic Based on AHP
Ercan, C. (2023) · Pamukkale Journal of Sport Sciences · doi:10.54141/psbd.1182006 · Cited by 3
The market for Welsh language mobile applications – A developers’ perspective
Cunliffe, D. (2018) · Telematics and Informatics · doi:10.1016/j.tele.2018.11.003 · Cited by 4
Mobile applications for emerging adults transitioning to independent diabetes monitoring
Schindler-Ruwisch, J. M., & Peters, A. (2020) · Informatics for Health and Social Care · doi:10.1080/17538157.2020.1837839 · Cited by 3
Lawin, D., von Jan, U., Pustozerov, E., Lawrenz, T., Stellbrink, C., & Albrecht, U. (2023) · Herzschrittmachertherapie + Elektrophysiologie · doi:10.1007/s00399-023-00947-2 · Cited by 2
Provider perspectives on equity in use of mobile health autism screening tools
Zuckerman, K. E., Vazquez, L. A. R., Santos, Y. M., Fuchu, P., Broder‐Fingert, S., Dolata, J. K., Bedrick, S. D., Fernandez, J., Fombonne, E. J., & Sanders, B. (2023) · Autism · doi:10.1177/13623613231215399 · Cited by 2
“Prior to the current study, authors evaluated and scored the tools using a structured rubric adapted from Grundy (Grundy et al., 2016) assessing scientific/clinical basis, functionality, usability, accountability, impact, popularity, and accessibility (Sanders, 2022).”
Una aplicación app para el diagnóstico de los micetismos
Ruiz, C., March-Amengual, J.-M., Reches, G., Ventura, S., & Queraltó, J. M. (2019) · Revista del Laboratorio Clínico · doi:10.1016/j.labcli.2019.01.005 · Cited by 2
Sugiharto, A., Sabarguna, B. S., & Pramastuty, A. (2022) · Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) · doi:10.29207/resti.v6i1.3847 · Cited by 2
Mobile Apps for Oncology Health Care Professionals: Mapping and Assessment Study
Liñares, D., Nicolaidou, I. A., Charalambous, A., Cabutto, D., Moreno‐Alonso, D., Alejos, C. M., Couespel, N., López, N., Fernández, M. J., Carrión, C., et al. (2025) · JMIR mhealth and uhealth · doi:10.2196/71203 · Cited by 1
“Furthermore, while Google Play and the Apple Store account for approximately two-thirds of the mobile apps downloaded annually [39], there are other, less commonly used download platforms that were not included in this review.”
End-Stage Renal Disease Self-management: mobile app development
Barros, J. P., & Brandão, P. (2021) · IEEE Symposium on Computers and Communications (ISCC) · doi:10.1109/iscc53001.2021.9631482 · Cited by 1
Tian, L., Zhang, K., Wang, X., Jiang, K., Gao, W., Liang, N., Wang, X., Miao, Y., & Lu, Z. (2020) · Biopreservation and Biobanking · doi:10.1089/bio.2019.0025 · Cited by 1
A content analysis of popular diet, fitness, and weight self-tracking mobile apps on Google Play
Tham, A., Kim, L. G., VictorySean, Chen, Y., Zheng, K., & Eikey, E. V. (2020) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 1
Incentive Spirometry and Smart Phones: Performance validation and user impressions
Baxter, C. A. (2018) · Queensland University of Technology · doi:10.5204/thesis.eprints.125165 · Cited by 1
Escudero, C. (2020) · Advances in human services and public health (AHSPH) book series · doi:10.4018/978-1-7998-6705-0.ch005
Do Pregnant Women Using Smartphones Care About Their Privacy?
Jacob, R., & Rani, D. (2020) · International Journal of Information Communication Technologies and Human Development · doi:10.4018/ijicthd.2020100102
Suggestions for the Development of Internet-based Cognitive-Behavioral Therapy with a Trauma Focus
Yunkyeung, C. (2020) · Journal of the Korean Chemical Society · doi:10.15207/jkcs.2020.11.12.261
mHealth in End Stage Renal Disease Self-management: A Usability study
Barros, J. P., Vieira‐Marques, P., & Ferreira, A. (2021) · International Conference on Medical and Health Informatics (ICMHI) · doi:10.1145/3472813.3473190
Stuttering mHealth Applications: Development of A Qualitative Rubric Assessment (Preprint)
Fadzilah, F. M., Arshad, N. I., Abidin, A. I. Z., Low, H. M., Mahmood, A. K., Aszemi, N. M., & Zamin, N. B. (2020) · JMIR mhealth and uhealth · doi:10.2196/25914
Baccin, C. A., Sasso, G. T. M. D., & Paixão, C. A. (2019) · doi:10.2196/preprints.14644
The Development of Structure Health Based on Digital Technologies
Sydykova, B., А.К., А., & Tobzhanova, K. (2022) · Studies in systems, decision and control · doi:10.1007/978-3-031-07067-9_3
Yu, Y. (2023) · Open Access Victoria University of Wellington | Te Herenga Waka (Figshare) · doi:10.26686/wgtn.24204300
Vallely, P. (2023) · Open Access Victoria University of Wellington | Te Herenga Waka (Figshare) · doi:10.26686/wgtn.24647856
Diranzo-García, J., Estrems-Díaz, V., Garrido-Ferrer, J. F., Ruipérez, L. C., Zarzuela-Sánchez, V. M., & Hernández-Ferrando, L. (2024) · Revista Española de Cirugía Ortopédica y Traumatología · doi:10.1016/j.recot.2024.03.003
Burke, C., Reidy, K., Ryan, P., & Jennings, A. (2024) · BJGP Open · doi:10.3399/bjgpo.2024.0038
Tales From the Past: Adapting App Repositories to App Store Dynamics
Stach, M., Schickler, M., Reichert, M., & Pryss, R. C. (2023) · Congress in Computer Science, Computer Engineering, and Applied Computing (CSCE) · doi:10.1109/csce60160.2023.00225
Diranzo-García, J., Estrems-Díaz, V., Garrido-Ferrer, J. F., Ruipérez, L. C., Zarzuela-Sánchez, V. M., & Hernández-Ferrando, L. (2024) · Revista Española de Cirugía Ortopédica y Traumatología · doi:10.1016/j.recot.2024.12.006
Robles, N., Fuster‐Casanovas, A., Melero, S. M., Duarte‐Díaz, A., Gelabert, E., Padilla-Ruiz, M., Gómez-Calderón, C., Pérez-Navarro, A., Vidal‐Alaball, J., Subirà, S., et al. (2025) · doi:10.2196/preprints.72612
Wu, C., Chang, C.-N., Chang, C.-F., Lin, M.-H., Chen, H., & Chen, Y. (2025) · doi:10.2196/preprints.77616
Validation and selection of criteria for evaluating apps for managing depression: a Delphi study
Robles, N., Fuster‐Casanovas, A., Melero, S. M., Duarte‐Díaz, A., Gelabert, E., Padilla-Ruiz, M., Gómez-Calderón, C., Pérez-Navarro, A., Vidal‐Alaball, J., Subirà, S., et al. (2025) · BMJ Open · doi:10.1136/bmjopen-2025-101302
“It is interesting to note that while general health app evaluation frameworks proposed by Grundy et al (2016) 28 and Lewis 29 are predominately based on dimensions and criteria relating to the functioning of the software itself, mental health app evaluation frameworks tend to incorporate more…”
34519 How to Use Feature Analysis to Reveal Dominant Norms and Assumptions in a Set of Apps
Hasinoff, A. A., & Bivens, R. (2025) · De Gruyter eBooks · doi:10.1515/9783111247854-019
Martins, E. D., Tavares, C. A. M., Favaro, L., Monfardini, F., Mourad, J., Khunti, K., Savarese, G., Berwanger, O., & Santo, K. (2026) · American Heart Journal · doi:10.1016/j.ahj.2026.107380
Wu, C., Chang, C.-N., Chang, C.-F., Lin, M.-H., Chen, H.-Y., & Chen, Y.-C. (2026) · JMIR mhealth and uhealth · doi:10.2196/77616
John, D., Hussin, N., Zaini, M. K., Helda, L. I., Ametefe, D. S., Aliu, A. A., & Ali, Z. A. (2026) · Mobile Networks and Applications · doi:10.1007/s11036-026-02497-6
“Similarly, Grundy et al. [40] provided insights into high-quality mHealth apps with innovation-based systems, and Mueller [41] examined the types of mHealth apps recommended by doctors for disease monitoring and treatment.”
Liu, P., Wang, F., & Yang, C. (2026) · JMIR mhealth and uhealth · doi:10.2196/88896
Metadata
- Authors
- Grundy, Q. H., Wang, Z., & Bero, L. A.
- Affiliations
- The University of Sydney (AU)
- Corresponding author
- Grundy, Quinn H.
- Venue
- American Journal of Preventive Medicine
- Publisher
- Elsevier BV
- Published
- 2016-12
- Volume
- 51
- Issue
- 6
- Pages
- 1051-1059
- ISSN
- 0749-3797 (ISSN-L)
- OpenAlex
- W2521913084
- Cited by
- 154
- References
- 41
- Citations by year
- 2026: 4 · 2025: 8 · 2024: 8 · 2023: 17 · 2022: 11 · 2021: 24 · 2020: 26 · 2019: 20 · 2018: 26 · 2017: 9 · 2016: 1
- FWCI
- 3.57
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Journal of Preventive Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Keywords
- mobile health apps, app quality assessment, risk of bias
- MeSH terms
- Health Promotion, Mobile Applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
