Cited by 154Mobile Health and mHealth ApplicationsMeta-analysis and systematic reviewsDigital Mental Health Interventions

The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.

Challenges in Assessing Mobile Health App Quality

Grundy, Quinn H., Wang, Zhicheng, Bero, Lisa A.·2016·American Journal of Preventive Medicine

DOI 10.1016/j.amepre.2016.07.009

Open this paper in CoChat — free account

Abstract

Context Mobile apps are poised to become a major source of health guidance; however, concerns about their quality and safety remain. Governments, clinicians, and health researchers are struggling to determine how to adequately evaluate the content and function of mobile health apps to guide consumers toward apps that will effectively and safely support their health. Researchers are both adapting existing research methods, such as those for systematic reviews, and exploring novel methods toward this aim. The purpose of this systematic review was to describe prevalent and emerging methods for searching, data extraction, and analysis in the context of mobile health apps and commercial app stores.Evidence acquisition A systematic search was conducted on April 6, 2016, according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, for published studies that evaluated health-related apps targeted at consumers in a commercial app store, and a descriptive content analysis of their methods was performed.Evidence synthesis Although the majority of studies self-identified as "reviews" or "systematic reviews," their claims to have performed an exhaustive, replicable, and systematic search and data extraction are unsupported, given the nature of commercial app stores' search engines and personalized app content. Currently, studies of health-related app content are largely descriptive, and most use surrogate and one-dimensional outcome measures for app content quality.Conclusions This field is rapidly developing, and innovative methods for systematic searches, multidimensional quality assessment, and contextualizing content assessment are emerging. Areas for future methodologic development are highlighted, including expanding definitions of app "content" and assessing risk of bias.

Integrity signals

Registry

Crossref · 10.1016/j.amepre.2016.07.009

Cites

41 of 41 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Mobile app rating scale: a new tool for assessing the quality of health mobile apps
Stoyanov, S. R., Hides, L., Kavanagh, D. J., Zelenko, O., Tjondronegoro, D., & Mani, M. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.3422 · Cited by 2,233
Just a Fad? Gamification in Health and Fitness Apps
Lister, C., West, J. H., Cannon, B., Sax, T., & Brodegard, D. (2014) · JMIR Serious Games · doi:10.2196/games.3413 · Cited by 489
Mobile Apps for Bipolar Disorder: A Systematic Review of Features and Content Quality
Nicholas, J., Larsen, M. E., Proudfoot, J., & Christensen, H. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4581 · Cited by 438
Unaddressed privacy risks in accredited health and wellness apps: a cross-sectional systematic assessment
Huckvale, K., Prieto, J. T., Tilney, M., Benghozi, P.-J., & Car, J. (2015) · BMC Medicine · doi:10.1186/s12916-015-0444-y · Cited by 376
A review and content analysis of engagement, functionality, aesthetics, information quality, and change techniques in the most popular commercial apps for weight management
Bardus, M., van Beurden, S. B., Smith, J. R., & Abraham, C. (2016) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-016-0359-9 · Cited by 350
Mobile Applications for Weight Management
Azar, K. M. J., Lesser, L. I., Laing, B. Y., Stephens, J., Aurora, M. S., Burke, L. E., & Palaniappan, L. P. (2013) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2013.07.005 · Cited by 334
Do physical activity and dietary smartphone applications incorporate evidence-based behaviour change techniques?
Direito, A., Dale, L. P., Shields, E., Dobson, R., Whittaker, R., & Maddison, R. (2014) · BMC Public Health · doi:10.1186/1471-2458-14-646 · Cited by 308

Show all 41 references

Sorted by citations

Criteria for assessing the quality of mHealth apps: a systematic review
Nouri, R., R Niakan Kalhori, S., Ghazisaeedi, M., Marchand, G., & Yasini, M. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy050 · Cited by 231
“More recently, Grundy et al.(60) conducted a systematic review to find and explain emerging and common methods for evaluating the quality of mHealth apps.”
Background
Smartphone apps for the self-management of low back pain: A systematic review
Machado, G. C., Pinheiro, M. B., Lee, H., Ahmed, O. H., Hendrick, P. A., Williams, C. M., & Kamper, S. J. (2016) · Best Practice & Research Clinical Rheumatology · doi:10.1016/j.berh.2017.04.002 · Cited by 171
“assessing the quality and ‘fitness-for-purpose’ of apps is by using validated scales, and benchmarking app content against current best-practice guidelines [10].”
Method
Mobile Apps for Medication Management: Review and Analysis
Tabi, K., Randhawa, A. S., Choi, F., Mithani, Z., Albers, F., Schnieder, M., Nikoo, M., Vigo, D., Jang, K. L., Demlová, R., et al. (2019) · JMIR mhealth and uhealth · doi:10.2196/13608 · Cited by 178
“The professional reviews could follow some of the standardized frameworks for quality assessment of mobile apps, for example, Mobile App Rating Scale [49], suitable for a wide range of health apps; a recently published assessment framework specifically for mental health apps [50]; or other published frameworks [33,51-…”
Background
Challenges to addressing student mental health in embedded counselling services: a survey of UK higher and further education institutions
Broglia, E., Millings, A., & Barkham, M. (2018) · British Journal of Guidance and Counselling · doi:10.1080/03069885.2017.1370695 · Cited by 141
A Review of the Quality and Impact of Mobile Health Apps
Grundy, Q. (2021) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-052020-103738 · Cited by 167
Self-citation
The NICE Evidence Standards Framework for digital health and care technologies – Developing and maintaining an innovative evidence framework with global impact
Unsworth, H., Dillon, B., Collinson, L., Powell, H., Salmon, M., Oladapo, T., Ayiku, L., Shield, G., Holden, J., Patel, N., et al. (2021) · Digital Health · doi:10.1177/20552076211018617 · Cited by 137
Smartphone Apps Targeting Alcohol and Illicit Substance Use: Systematic Search in in Commercial App Stores and Critical Content Analysis
Tofighi, B., Chemi, C., Ruiz-Valcarcel, J., Hein, P. N., & Hu, L. (2018) · JMIR mhealth and uhealth · doi:10.2196/11831 · Cited by 131
“Studies assessing app content are generally descriptive and limited to one-dimensional outcome quality measures [17].”
BackgroundMethod
Going digital: a narrative overview of the effects, quality and utility of mobile apps in chronic disease self-management
Scott, I. A., Scuffham, P., Gupta, D., Harch, T. M., Borchi, J., & Richards, B. (2018) · Australian Health Review · doi:10.1071/ah18064 · Cited by 129
Mobile health app usability and quality rating scales: a systematic review
Azad‐Khaneghah, P., Neubauer, N., Cruz, A. M., & Liu, L. (2020) · Disability and Rehabilitation Assistive Technology · doi:10.1080/17483107.2019.1701103 · Cited by 119
Toward a Better Understanding of the Intention to Use mHealth Apps: Exploratory Study
(2021) · doi:10.2196/27021 · Cited by 108
“Furthermore, Grundy et al [17] studied the use of high-quality mHealth apps with”
Background
Mobile Health in Oncology: A Patient Survey About App-Assisted Cancer Care
Kessel, K. A., Vogel, M. M. E., Kessel, C., Bier, H., Biedermann, T., Friess, H., Herschbach, P., von Eisenhart‐Rothe, R., Meyer, B., Kiechle, M. B., et al. (2017) · JMIR mhealth and uhealth · doi:10.2196/mhealth.7689 · Cited by 104
“However, a standardized validation process must be implemented for medical apps to guarantee safety for the patient [8,15,16].”
Beyond Dr. Google: the evidence on consumer-facing digital tools for diagnosis
Millenson, M. L., Baldwin, J. L., Zipperer, L., & Singh, H. (2018) · Diagnosis · doi:10.1515/dx-2018-0009 · Cited by 91
A Framework for Competencies for the Use of Mobile Technologies in Psychiatry and Medicine: Scoping Review
Hilty, D. M., Chan, S. R., Torous, J. B., Luo, J. S., & BOLAND, R. J. (2019) · JMIR mhealth and uhealth · doi:10.2196/12229 · Cited by 101
Users’ acceptability of a mobile application for persons with type 2 diabetes: a qualitative study
Torbjørnsen, A., Ribu, L., Rønnevig, M., Grøttland, A., & Helseth, S. (2019) · BMC Health Services Research · doi:10.1186/s12913-019-4486-2 · Cited by 110
“and for diabetes care in particular [3, 4].”
Evaluation of Mobile Apps Targeted to Parents of Infants in the Neonatal Intensive Care Unit: Systematic App Review
Richardson, B., Dol, J., Rutledge, K., Monaghan, J., Orovec, A., Howie, K., Boates, T., Smit, M., & Campbell-Yeo, M. (2018) · JMIR mhealth and uhealth · doi:10.2196/11620 · Cited by 90
“This is consistent with other studies reviewing mobile apps [35].”
MethodResult
Mobile apps for treatment of speech disorders in children: An evidence-based analysis of quality and efficacy
Furlong, L., Morris, M., Serry, T., & Erickson, S. (2018) · PLOS ONE · doi:10.1371/journal.pone.0201513 · Cited by 87
How can clinicians, specialty societies and others evaluate and improve the quality of apps for patient use?
Wyatt, J. C. (2018) · BMC Medicine · doi:10.1186/s12916-018-1211-7 · Cited by 61
An Assessment Framework for e-Mental Health Apps in Canada: Results of a Modified Delphi Process
Zelmer, J., van Hoof, K., Notarianni, M., van Mierlo, T., Schellenberg, M., & Tannenbaum, C. (2018) · JMIR mHealth and uHealth · doi:10.2196/10016 · Cited by 62
“…review of methods used to assess mobile health apps found that researchers evaluated the app quality in 6 domains: scientific or clinical basis, functionality, usability, accountability, impact, and popularity; 80% (73/91) participants used measures drawn from one or more of these domains [14].”
BackgroundMethod
An overview on the emerging area of identification, characterization, and assessment of health apps
Paglialonga, A., Lugo, A., & Santoro, E. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.05.017 · Cited by 77
Digital health and the COVID-19 epidemic: an assessment framework for apps from an epidemiological and legal perspective
Vokinger, K. N., Nittas, V., Witt, C. M., Fabrikant, S. I., & von Wyl, V. (2020) · Swiss Medical Weekly · doi:10.4414/smw.2020.20282 · Cited by 82
SUITABILITY OF CURRENT EVALUATION FRAMEWORKS FOR USE IN THE HEALTH TECHNOLOGY ASSESSMENT OF MOBILE MEDICAL APPLICATIONS: A SYSTEMATIC REVIEW
Moshi, M. R., Tooher, R. L., & Merlin, T. (2018) · International Journal of Technology Assessment in Health Care · doi:10.1017/s026646231800051x · Cited by 74
“Only seven frameworks evaluated the harms of the app itself (e.g., adverse events) (14;18;19;26;32;36;37).”
Background
Building user engagement to mhealth apps from a learning perspective: Relationships among functional, emotional and social drivers of user value
Vijande, M. L. S., Gómez, M., Molina, A., & Davison, R. M. (2022) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2022.102956 · Cited by 88
Mobile Apps for Older Adults: Systematic Search and Evaluation Within Online Stores
Portenhauser, A. A., Terhorst, Y., Schultchen, D., Sander, L., Denkinger, M. D., Stach, M., Waldherr, N., Dallmeier, D., Baumeister, H., & Meßner, E.-M. (2021) · JMIR Aging · doi:10.2196/23313 · Cited by 77
A health app developer’s guide to law and policy: a multi-sector policy analysis
Parker, L., Karliychuk, T., Gillies, D., Mintzes, B., Raven, M., & Grundy, Q. (2017) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-017-0535-0 · Cited by 71
“There is, however, a particular lack of research on health app content and policy in countries outside of the Western, English speaking nations, and we would advocate for further work here [36].”
BackgroundMethodSelf-citation
Factors influencing user’s adherence to physical activity applications: A scoping literature review and future directions
Yang, X., Ma, L., Zhao, X., & Kankanhalli, A. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.104039 · Cited by 69
A Critical Appraisal of Fertility and Menstrual Tracking Apps for the iPhone
Zwingerman, R., Chaikof, M., & Jones, C. A. (2019) · Journal of Obstetrics and Gynaecology Canada · doi:10.1016/j.jogc.2019.09.023 · Cited by 68
A Scoping Review of Sensors, Wearables, and Remote Monitoring For Behavioral Health: Uses, Outcomes, Clinical Competencies, and Research Directions
Hilty, D. M., Armstrong, C. M., Luxton, D. D., Gentry, M. T., & Krupinski, E. A. (2021) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-021-00199-2 · Cited by 70
The relevance of health literacy to mHealth
Kreps, G. L. (2017) · Information Services and Use · doi:10.3233/isu-170828 · Cited by 62
“Yet, there are serious communication challenges that must be addressed concerning the best uses of mHealth for health promotion, including assuring the appropriateness and effectiveness of mHealth messaging for audiences with different communication competencies, styles, and health literacy levels, to ensure that mHea…”
A Mobile Health Solution Complementing Psychopharmacology-Supported Smoking Cessation: Randomized Controlled Trial
Carrasco-Hernandez, L., Jódar-Sánchez, F., Núñez-Benjumea, F., Moreno Conde, J., Mesa González, M., Civit-Balcells, A., Hors-Fraile, S., Parra-Calderón, C. L., Bamidis, P. D., & Ortega-Ruiz, F. (2020) · JMIR mHealth and uHealth · doi:10.2196/17530 · Cited by 65
‘Rage against the machine’? The opportunities and risks concerning the automation of urban green infrastructure
Gulsrud, N. M., Raymond, C. M., Rutt, R. L., Olafsson, A. S., Plieninger, T., Sandberg, M., Beery, T. H., & Jonsson, K. (2018) · Landscape and Urban Planning · doi:10.1016/j.landurbplan.2018.08.012 · Cited by 62
A multi-faceted approach to characterizing user behavior and experience in a digital mental health intervention
Chen, A. T., Wu, S., Tomasino, K. N., Lattie, E. G., & Mohr, D. C. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103187 · Cited by 58
Mental Health Messages in Prominent Mental Health Apps
Parker, L. M., Bero, L. A., Gillies, D. M., Raven, M. K., Mintzes, B. J., Jureidini, J. N., & Grundy, Q. (2018) · The Annals of Family Medicine · doi:10.1370/afm.2260 · Cited by 63
Self-citation
Recent Advances in Yttrium Iron Garnet Films: Methodologies, Characterization, Properties, Applications, and Bibliometric Analysis for Future Research Directions
Arsad, A. Z., Zuhdi, A. W. M., Ibrahim, N. B., & Hannan, M. A. (2023) · Applied Sciences · doi:10.3390/app13021218 · Cited by 57
“Numerous databases, including PubMed, Web of Science, Scopus, Web of Science, Google Scholar, and Baidu Scholar can be searched for articles containing strategy in research [52–54].”
Background
Desirable Components for a Customized, Home-Based, Digital Care-Management App for Children and Young People With Long-Term, Chronic Conditions: A Qualitative Exploration
Nightingale, R., Hall, A. G., Gelder, C., Friedl, S., Brennan, E., & Swallow, V. M. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7760 · Cited by 56
“However, to our knowledge, there are few coproduced, rigorously developed, evaluated apps providing tailored, CKD-specific, practical advice on day-to-day care management [13-20] and development processes for those that do exist are not systematic.”
Background
Balancing Between Privacy and Patient Needs for Health Information in the Age of Participatory Health and Social Media: A Scoping Review
Househ, M. S., Grainger, R., Petersen, C. C., Bamidis, P. D., & Merolli, M. (2018) · Yearbook of Medical Informatics · doi:10.1055/s-0038-1641197 · Cited by 55
“Adherence to medical devices/software certification and conformance with safety, security, and privacy issues should become standardized criteria for review of medical apps [41].”
Digital Technology and Mobile Health in Behavioral Migraine Therapy: a Narrative Review
Stubberud, A., & Linde, M. (2018) · Current Pain and Headache Reports · doi:10.1007/s11916-018-0718-0 · Cited by 50
Supporting Self-Management in Bipolar Disorder: Mixed-Methods Knowledge Translation Study
Michalak, E. E., Morton, E. E., Barnes, S. J., Hole, R., CREST.BD, & Murray, G. (2019) · JMIR Mental Health · doi:10.2196/13493 · Cited by 50
“Thus, there are significant barriers for people facing mental health challenges in identifying safe, evidence-informed apps [56,57].”
Background
Assessing the Quality and Impact of eHealth Tools: Systematic Literature Review and Narrative Synthesis
Jacob, C., Lindeque, J. P., Klein, A., Ivory, C. J., Heuss, S., & Peter, M. K. (2023) · JMIR Human Factors · doi:10.2196/45143 · Cited by 43
The Arabic Version of the Mobile App Rating Scale: Development and Validation Study
Bardus, M., Awada, N., Ghandour, L. A., Fares, E.-J., Gherbal, T., Al-Zanati, T., & Stoyanov, S. R. (2019) · JMIR mhealth and uhealth · doi:10.2196/16956 · Cited by 46
“Future studies may compare MARS to other instruments of app quality [23,30], even though they might not be equivalent.”
Background
Tracing the Potential Flow of Consumer Data: A Network Analysis of Prominent Health and Fitness Apps
Grundy, Q., Held, F. P., & Bero, L. A. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7347 · Cited by 47
“…localized and personalized nature of app store search algorithms and rapidly changing population of apps, we triangulated two sampling strategies [12]: (1) a crawling program that systematically sampled the top-ranked 100 apps from the iTunes and Google Play app stores in the United States,…”
Self-citation
Assessing the Quality of Mobile Phone Apps for Weight Management: User-Centered Study With Employees From a Lebanese University
Bardus, M., Ali, A., Demachkieh, F., & Hamadeh, G. (2019) · JMIR mHealth and uHealth · doi:10.2196/mhealth.9836 · Cited by 44
“On one side, researchers focus on aspects related to theoretical and evidence-based content [38,39].”
Should We Recommend Renal Diet–Related Apps to Our Patients? An Evaluation of the Quality and Health Literacy Demand of Renal Diet–Related Mobile Applications
Lambert, K., Mullan, J., Mansfield, K. J., & Owen, P. (2017) · Journal of Renal Nutrition · doi:10.1053/j.jrn.2017.06.007 · Cited by 44
“Importantly, up to one-third of health apps are designed to assist individuals with their chronic disease self-management.(4) Self-management of a chronic disease such as chronic kidney disease (CKD) is complex and requires an individual to balance complex dietary, lifestyle and medication regimens, as well as monitor…”
Background
mHealth Solutions for Mental Health Screening and Diagnosis: A Review of App User Perspectives Using Sentiment and Thematic Analysis
Funnell, E. L., Spadaro, B., Martin-Key, N. A., Metcalfe, T., & Bahn, S. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.857304 · Cited by 41
“, in-app screenshots, app keyword, app description) based on geographical locations (83, 84).”
Background
Mobile Health (mHealth) apps focused on menopause: Are they any good?
Gkrozou, F., Tsonis, O., Godden, M., Siafaka, V., & Paschopoulos, M. E. (2019) · Post Reproductive Health · doi:10.1177/2053369119878978 · Cited by 43
Evolutionary Trends in the Adoption, Adaptation, and Abandonment of Mobile Health Technologies: Viewpoint Based on 25 Years of Research
Portz, J. D., Moore, S. L., & Bull, S. S. (2024) · Journal of Medical Internet Research · doi:10.2196/62790 · Cited by 55
“The scientific evaluation of effectiveness and safety has not kept pace with the rapid expansion of the commercial health-related app market, creating a disconnect between scientific literature and available apps [50,60].”
Adoption of Mobile Health Apps in Dietetic Practice: Case Study of Diyetkolik
Akdur, G., Aydın, M. N., & Akdur, G. (2020) · JMIR mhealth and uhealth · doi:10.2196/16911 · Cited by 44
Challenges for developing health-care knowledge in the digital age
Alrahbi, D., Khan, M., Gupta, S., Modgil, S., & Jabbour, C. J. C. (2020) · Journal of Knowledge Management · doi:10.1108/jkm-03-2020-0224 · Cited by 43
Social Support for Breastfeeding in the Era of mHealth: A Content Analysis
Schindler-Ruwisch, J. M., Roess, A., Robert, R. C., Napolitano, M. A., & Chiang, S. C. (2018) · Journal of Human Lactation · doi:10.1177/0890334418773302 · Cited by 37
From Ancient Contemplative Practice to the App Store
Lukoff, K., Lyngs, U., Gueorguieva, S., Dillman, E., Hiniker, A., & Munson, S. A. (2020) · Designing Interactive Systems Conference · doi:10.1145/3357236.3395444 · Cited by 53
“n-distress indicates that they are not receiving adequate support from wherever they learned to meditate” [1]. Overstated beneﬁts and understated risks have been reported in other health-related apps [35] and our results suggest this issue may be prevalent in mindfulness-related apps, too. Designers of technologies fo…”
Background
Mobile Health, Smartphone/Device, and Apps for Psychiatry and Medicine
Hilty, D. M., Chan, S., Torous, J., Luo, J., & Boland, R. J. (2019) · Psychiatric Clinics of North America · doi:10.1016/j.psc.2019.05.007 · Cited by 38
mHealth solutions for perinatal mental health: scoping review and appraisal following the mHealth index and navigation database framework
Spadaro, B., Martin-Key, N. A., Funnell, E., & Bahn, S. (2022) · JMIR mhealth and uhealth · doi:10.2196/30724 · Cited by 34
“Unlike electronic journal databases, app stores are not designed for systematic search and export of data [69].”
Background
Marketing Innovations in Industry 4.0 and Their Impacts on Current Enterprises
Ungerman, O., & Dědková, J. (2019) · Applied Sciences · doi:10.3390/app9183685 · Cited by 34
The impact of mobile applications on medication adherence: a systematic review
Ng, R., Carter, S. R., & El‐Den, S. (2019) · Translational Behavioral Medicine · doi:10.1093/tbm/ibz125 · Cited by 34
Digital Technologies for Children and Parents Sharing Self-Management in Childhood Chronic or Long-Term Conditions: A Scoping Review
Edwards, J., Waite-Jones, J., Schwarz, T., & Swallow, V. M. (2021) · Children · doi:10.3390/children8121203 · Cited by 31
Ecotoxicological implications of environmental contaminants on disease vectors: challenges in risk assessment and strategies for integrated vector management
Abbasi, E. (2025) · Environmental Toxicology and Chemistry · doi:10.1093/etojnl/vgaf209 · Cited by 32
Quality Evaluation of English Mobile Applications for Gestational Diabetes: App Review using Mobile Application Rating Scale (MARS)
Kalhori, S. R. N., Hemmat, M., Noori, T., Heydarian, S., & Katigari, M. R. (2020) · Current Diabetes Reviews · doi:10.2174/1573399816666200703181438 · Cited by 28
Beyond the virus: a first look at coronavirus-themed Android malware
Wang, L., Ren, H., Wang, H., Xia, P., Li, Y., Wu, L., Zhou, Y., Luo, X., Sui, Y., Guo, Y., et al. (2021) · Empirical Software Engineering · doi:10.1007/s10664-021-09974-4 · Cited by 28
Issues in reporting of systematic review methods in health app-focused reviews: A scoping review
Grainger, R., Devan, H., Sangelaji, B., & Hay-Smith, J. E. (2020) · Health Informatics Journal · doi:10.1177/1460458220952917 · Cited by 27
Mobile Health Apps for Breast Cancer: Content Analysis and Quality Assessment
Yang, S., Bui, C. N., & Park, K. (2023) · JMIR mhealth and uhealth · doi:10.2196/43522 · Cited by 31
“Therefore, methods for the systematic search and analysis of apps have been developed [9].”
Method
RECOVER-E – a mobile app for patients undergoing total knee or hip replacement: study protocol
Stauber, A., Schüßler, N., Palmdorf, S., Schürholz, N., Bruns, D., Osterbrink, J., & Nestler, N. (2020) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-020-3090-2 · Cited by 24
How Do Infant Feeding Apps in China Measure Up? A Content Quality Assessment
Zhao, J., Freeman, B., & Li, M. (2017) · JMIR mhealth and uhealth · doi:10.2196/mhealth.8764 · Cited by 25
“Notably, studies assessing health-related apps were concentrated in English-speaking countries, whereas countries where apps were downloaded in large numbers such as China, Brazil, and Mexico were relatively neglected [16].”
BackgroundMethod
Evaluation of a Mobile Phone App for Patients With Pollen-Related Allergic Rhinitis: Prospective Longitudinal Field Study
Glattacker, M., Boeker, M., Anger, R., Reichenbach, F., Tassoni, A., Bredenkamp, R., & Giesler, J. M. (2020) · JMIR mhealth and uhealth · doi:10.2196/15514 · Cited by 28
“Therefore, health apps might increasingly become a “major source of health guidance” ([12] pg 1051) and have the potential to change existing health care delivery pathways [13].”
Background
Advances in Health Technology Use and Implementation in the Era of Healthy Living: Implications for Precision Medicine
Phillips, S. A., Ali, M. M., Modrich, C., Oke, S., Elokda, A. S., Laddu, D., & Bond, S. (2018) · Progress in Cardiovascular Diseases · doi:10.1016/j.pcad.2018.12.007 · Cited by 27
Management of cardiovascular disease using an mHealth tool: a randomized clinical trial
Kang, S., Baek, H., Cho, J.-H., Kim, S., Hwang, H. Y., Lee, W., Park, J. J., Yoon, Y. E., Yoon, C., Cho, Y., et al. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00535-z · Cited by 25
A randomized controlled trial examining consumers’ perceptions and opinions on using different versions of a FoodFlip© smartphone application for delivery of nutrition information
Ahmed, M., Oh, A., Vanderlee, L., Franco‐Arellano, B., Schermel, A., Lou, W. Y. W., & L’Abbé, M. R. (2020) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-020-0923-1 · Cited by 24
“, tailored feedback), iterative development process and incorporation of behaviour change theories to increase app engagement and retention rate (long-term use of consumer nutrition mobile apps) [58, 59].”
Method
Proposing a standardized, step-by-step model for creating post-traumatic stress disorder (PTSD) related mobile mental health apps in a framework based on technical and medical norms
Schellong, J., Lorenz, P., & Weidner, K. (2019) · European Journal of Psychotraumatology · doi:10.1080/20008198.2019.1611090 · Cited by 23
“If the app is to be qualified as a medical device, it is not only important to apply all the norms concerning such devices in development, but also to establish that the app has the desired effect (Grundy, Wang, & Bero, 2016).”
Background
Acceptability and usability of a mobile application for management and surveillance of vector-borne diseases in Colombia: An implementation study
Rodríguez, S., Sanz, A. M., Llano, G., Andres Navarro, C., Parra‐Lara, L. G., Krystosik, A., & Rosso, F. (2020) · PLoS ONE · doi:10.1371/journal.pone.0233269 · Cited by 23
Beyond the Virus: A First Look at Coronavirus-themed Mobile MalwarePreprint
Wang, L., He, R., Wang, H., Xia, P., Li, Y., Wu, L., Zhou, Y., Luo, X., Sui, Y., Guo, Y., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.14619 · Cited by 23
“Mobile health apps have been studied by previous work [40], [41] and [29].”
Background
Evaluating the quality, safety, and functionality of commonly used smartphone apps for bipolar disorder mood and sleep self-management
Morton, E., Nicholas, J., Yang, L., Lapadat, L., Barnes, S. J., Provencher, M. D., Depp, C., Chan, M., Kulur, R., & Michalak, E. E. (2022) · International Journal of Bipolar Disorders · doi:10.1186/s40345-022-00256-6 · Cited by 20
Description of Cardiological Apps From the German App Store: Semiautomated Retrospective App Store Analysis
Albrecht, U., Hasenfuss, G. P., & von Jan, U. (2018) · JMIR mhealth and uhealth · doi:10.2196/11753 · Cited by 21
“In the context of mobile apps, the demand for methods to identify, characterize, and evaluate health-related apps has led to considerable research activity in recent years [4], be it in health-related fields or other areas of application.”
Background
Content Analysis of Apps for Growth Monitoring and Growth Hormone Treatment: Systematic Search in the Android App Store
Fernández-Luque, L., Labarta, J. I., Palmer, E., & Koledova, E. B. (2020) · JMIR mhealth and uhealth · doi:10.2196/16208 · Cited by 23
“Complementary to the methods for the systematic search of apps in app stores [5] that have been published, this review also explores a semiautomated approach to extract mobile app data, which can be used to facilitate part of the process of the analysis of health apps.”
Feature Analysis: A Method for Analyzing the Role of Ideology in App Design
Hasinoff, A. A., & Bivens, R. (2021) · Journal of Digital Social Research · doi:10.33621/jdsr.v3i2.56 · Cited by 26
“One review of research on health apps found that 79% of studies assess apps’ clinical or scientific basis (Grundy, Wang, and Bero, 2016), and researchers taking this approach typically focus on how well apps’ design features fulfill established treatment guidelines.”
Background
How Do Smart Device Apps for Diabetes Self-Management Correspond with Theoretical Indicators of Empowerment? An Analysis of App Features
Brew‐Sam, N., & Chib, A. I. (2019) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462319000163 · Cited by 20
“Tomitigate reported country differences in commercial app stores (73), an iPad (offering larger variety in search settings than an iPhone; iPad mini model ME276GP/A, iOS 9.1) with the latest operating system was registered anew with a dummy Singaporean account to ensure the inclusion of local…”
Influential
Parents’ Experience With a Mobile Health Intervention to Influence Human Papillomavirus Vaccination Decision Making: Mixed Methods Study
Becker, E. R., Shegog, R., Savas, L. S., Frost, E. L., Coan, S. P., Healy, C. M., Spinner, S. W., & Vernon, S. W. (2021) · JMIR Pediatrics and Parenting · doi:10.2196/30340 · Cited by 18
“The most frequently evaluated domains in assessments of commercially available mHealth apps include the scientific and clinical basis, functionality, usability, accountability, impact, and popularity [23].”
Background
Pregnant Users’ Perceptions of the Birth Plan Interface in the “My Prenatal Care” App: Observational Validation Study
Carrilho, J. M., de Oliveira, I. J. R., Santos, D., Osanan, G. C., Cruz-Correia, R. J., & Reis, Z. S. N. (2018) · JMIR Formative Research · doi:10.2196/11374 · Cited by 17
Quality evaluation of smartphone applications for laboratory medicine
Jovičić, S. Ž., Siódmiak, J., & Watson, I. D. (2018) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2018-0710 · Cited by 15
“Sometimes I don’t have a pulse … and I’m still alive!” Interviews with healthcare professionals to explore their experiences of and views on population-based digital health technologies
Tomasella, F., & Morgan, H. M. (2021) · Digital Health · doi:10.1177/20552076211018366 · Cited by 15
“This is unsurprising, given the widely documented increase in these technologies’ popularity within society.(10,35,36) However, their proliferation has not increased HCPs’ experiences in dealing with patients enquiring about them.”
Background
eHealth for the prevention of healthcare-associated infections: a scoping review
Bentvelsen, R. G., Holten, E., Chavannes, N. H., & Veldkamp, K. E. (2021) · Journal of Hospital Infection · doi:10.1016/j.jhin.2021.04.029 · Cited by 17
A Social Network Analysis of the Financial Links Backing Health and Fitness Apps
Grundy, Q., Held, F. P., & Bero, L. A. (2017) · American Journal of Public Health · doi:10.2105/ajph.2017.303995 · Cited by 14
Self-citation
Mobile Apps for Drug–Drug Interaction Checks in Chinese App Stores: Systematic Review and Content Analysis
Shen, C., Jiang, B., Yang, Q., Wang, C., Lu, K., Gu, M., & Yuan, J. (2021) · JMIR mhealth and uhealth · doi:10.2196/26262 · Cited by 14
FAMAP: A Framework for Developing m-Health Apps
García‐Magariño, I., Bedía, M. G., & Palacios-Navarro, G. (2018) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-77703-0_83 · Cited by 13
Quality Awareness and Its Influence on the Evaluation of App Meta-Information by Physicians: Validation Study
Albrecht, U., Framke, T., & von Jan, U. (2019) · JMIR mhealth and uhealth · doi:10.2196/16442 · Cited by 13
Development of a Multidimensional App-Quality Assessment Tool for Health-Related Apps (AQUA)
O’Rourke, T., Pryss, R. C., Schlee, W., & Probst, T. (2020) · Digital Psychology · doi:10.24989/dp.v1i2.1816 · Cited by 13
“Assessing the quality of data security is therefore a crucial part of improving the use of health-related apps, and the development of methods to identify possible risks to personal privacy and data security is important (Grundy et al., 2016).”
Background
Mobile healthcare platforms' sustainability: The perspective of health information quality
Sun, X., Zhou, W., & Feng, Y. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2022.1059252 · Cited by 13
“There is a general concern about the quality of health information available on mHealth platforms (12).”
Background
Description of apps targeting stroke patients: A review of apps store
Cao, W., Kadir, A. A., Wang, Y., Wang, J., Dai, B., Zheng, Y., Mu, P., Hu, C., Chen, J., Luo, N., et al. (2023) · Digital Health · doi:10.1177/20552076231181473 · Cited by 11
Advancing evidence-based digital health through an innovative research environment: an academic-industry collaboration case report
Ford, K. L., Moore, S. L., Zhou, S., Gore, M. O., Portz, J., Zhang, X., Zane, R. D., Wiler, J. L., & Bull, S. S. (2019) · mHealth · doi:10.21037/mhealth.2019.08.08 · Cited by 12
Quality analysis of smart phone sleep apps in China: can apps be used to conveniently screen for obstructive sleep apnea at home?
Xu, Z., Luo, X., Shi, J., & Lai, Y. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0916-7 · Cited by 12
“However, when the Accountability score was ≥ 7, the total score of the app was higher (17.48 vs. 13.16, P 0.01), as was the Functionality and Usability score (4.03 vs. 2.53, P 0.01).”
MethodInfluential
Review, Assess, Classify, and Evaluate (RACE): a framework for studying m-health apps and its application for opioid apps
Varshney, U., Singh, N., Bourgeois, A. G., & Dube, S. R. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab277 · Cited by 12
Computer-assisted Cognitive Behavior Therapy and Mobile Apps for Depression and Anxiety: Evidence-based Digital Tools for Clinical Practice
Rajeshkumar, L., Mishkind, M. C., Coleman, J. J., Pahwa, M., LaPreze, D., Solenske, S., & Wright, J. H. (2024) · Journal of Psychiatric Practice · doi:10.1097/pra.0000000000000815 · Cited by 12
Characteristics of Acute Childhood Illness Apps for Parents: Environmental Scan
Benoit, J. R. A., Hartling, L., Chan, M. C., & Scott, S. D. (2021) · Journal of Medical Internet Research · doi:10.2196/29441 · Cited by 12
“For example, the Apple and Google app stores differ in how apps are assessed for quality and how their search algorithms work, as well as in quality requirements for app developers [53].”
Innovation and Creativity in the Mobile Applications Industry: A Case Study of Mobile Health Applications (e-Health Apps)
Palos‐Sánchez, P. R., Saura, J. R., & Álvarez‐García, J. (2018) · Innovation, technology and knowledge management · doi:10.1007/978-3-319-99590-8_7 · Cited by 10
Automated Characterization of Mobile Health Apps' Features by Extracting Information From the Web: An Exploratory Study
Paglialonga, A., Schiavo, M., & Caiani, E. G. (2018) · American Journal of Audiology · doi:10.1044/2018_aja-imia3-18-0008 · Cited by 10
“…information to potential app users and thus to enable informed, aware adoption of health apps (Aungst, Clauson, Misra, Lewis, & Husain, 2014; Grundy, Wang, & Bero, 2016; Jungnickel, von Jan, & Albrecht, 2015; Paglialonga, Lugo, et al., 2018; Paglialonga, Pinciroli, Tognola, Barbieri, et…”
Importance-performance analysis in fitness apps. A study from the viewpoint of gender and age
Martín, F., Fernández, J. G., Valcarce-Torrente, M., García, A. B., Gálvez-Ruíz, P., & Sánchez, S. A. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1226888 · Cited by 9
Assessing intention to use mobile phone–based self-management support among adults with type 2 diabetes in Saudi Arabia: A cross-sectional study
Alosaimi, H., Alshammari, M. K., Al-Otaibi, M., Alhejaili, S., Alshammari, B., Khalaf, A., Hajea, A., Saleh, R., Aldhafiri, F. J., AbuShahin, A., et al. (2025) · Digital Health · doi:10.1177/20552076241308993 · Cited by 11
Advancing health-related abilities and behaviors via health apps: a large-scale survey from self-regulation perspective
Fu, S., Fang, J., Cai, Z., Lim, E. T. K., Tan, C., & Yang, H. (2022) · Internet Research · doi:10.1108/intr-09-2020-0485 · Cited by 9
What criteria are young people using to select mobile mental health applications? A nominal group study
Kabacińska, K., McLeod, K. E. R., MacKenzie, A., Vu, K., Cianfrone, M., Tugwell, A., & Robillard, J. M. (2022) · Digital Health · doi:10.1177/20552076221102775 · Cited by 11
“Although evidence-based mental health support apps do exist, there is little to no regulatory oversight of the content of published apps(9) and as a result, the vast majority of available applications are not grounded in empirical work.(11,12) When evidence-based apps have been clinically tested, results were promisin…”
Background
Digital Transformation in Indonesian Healthcare
Pitaloka, A. A., & Nugroho, A. P. (2021) · STI Policy and Management Journal · doi:10.14203/stipm.2021.301 · Cited by 8
Timing, Indicators, and Approaches to Digital Patient Experience Evaluation: Umbrella Systematic Review
Wang, T., Giunti, G., Goossens, R., & Melles, M. (2023) · Journal of Medical Internet Research · doi:10.2196/46308 · Cited by 8
Web-Based Patient Self-Reported Outcome After Radiotherapy in Adolescents and Young Adults With Cancer: Survey on Acceptance of Digital Tools
Vogel, M. M. E., Eitz, K. A., & Combs, S. E. (2020) · JMIR mhealth and uhealth · doi:10.2196/19727 · Cited by 8
Data handling practices and commercial features of apps related to children: a scoping review of content analyses
Jibb, L., Amoako, E., Heisey, M., Ren, L. Y., & Grundy, Q. (2022) · Archives of Disease in Childhood · doi:10.1136/archdischild-2021-323292 · Cited by 8
Self-citation

Authors: Grundy, Q. H., Wang, Z., & Bero, L. A.
Affiliations: The University of Sydney (AU)
Corresponding author: Grundy, Quinn H.
ORCID: Grundy, Quinn H.
Wang, Zhicheng
Bero, Lisa A.
Venue: American Journal of Preventive Medicine
Publisher: Elsevier BV
Published: 2016-12
Volume: 51
Issue: 6
Pages: 1051-1059
ISSN: 0749-3797 (ISSN-L)
DOI: 10.1016/j.amepre.2016.07.009
OpenAlex: W2521913084
Cited by: 154
References: 41
Citations by year: 2026: 4 · 2025: 8 · 2024: 8 · 2023: 17 · 2022: 11 · 2021: 24 · 2020: 26 · 2019: 20 · 2018: 26 · 2017: 9 · 2016: 1
FWCI: 3.57
Type: journal article
Language: en
Open access: closed
Licence: www.elsevier.com/tdm/userlicense/1.0/ (tdm)
Locations: American Journal of Preventive Medicine (publishedVersion)
PubMed (publishedVersion)
Topics: Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
Keywords: mobile health apps, app quality assessment, risk of bias
MeSH terms: Health Promotion, Mobile Applications

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Challenges in Assessing Mobile Health App Quality

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks