The DOI matches its Crossref record; 59 of the 62 references it cites were found in a registry or an index.
Criteria for assessing the quality of mHealth apps: a systematic review
Open accessAbstract
Objective: Review the existing studies including an assessment tool/method to assess the quality of mHealth apps; extract their criteria; and provide a classification of the collected criteria. Methods: In accordance with the PRISMA statement, a literature search was conducted in MEDLINE, EMBase, ISI and Scopus for English language citations published from January 1, 2008 to December 22, 2016 for studies including tools or methods for quality assessment of mHealth apps. Two researchers screened the titles and abstracts of all retrieved citations against the inclusion and exclusion criteria. The full text of relevant papers was then individually examined by the same researchers. A senior researcher resolved eventual disagreements and confirmed the relevance of all included papers. The authors, date of publication, subject fields of target mHealth apps, development method, and assessment criteria were extracted from each paper. The extracted assessment criteria were then reviewed, compared, and classified by an expert panel of two medical informatics specialists and two health information management specialists. Results: Twenty-three papers were included in the review. Thirty-eight main classes of assessment criteria were identified. These were reorganized by expert panel into 7 main classes (Design, Information/Content, Usability, Functionality, Ethical Issues, Security and Privacy, and User-perceived value) with 37 sub-classes of criteria. Conclusions: There is a wide heterogeneity in assessment criteria for mHealth apps. It is necessary to define the exact meanings and degree of distinctness of each criterion. This will help to improve the existing tools and may lead to achieve a better comprehensive mHealth app assessment tool.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A recent systematic review identified 23 different published criteria for assessment of mHealth apps.”
From What is the clinical value of mHealth for patients? (2020) · cites this paper as [38]
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
Mobile app rating scale: a new tool for assessing the quality of health mobile apps
Stoyanov, S. R., Hides, L., Kavanagh, D. J., Zelenko, O., Tjondronegoro, D., & Mani, M. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.3422 · Cited by 2,233
Mobile-health: A review of current state in 2015
Silva, B. M. C., Rodrigues, J. J. P. C., Díez, I. D. L. T., López-Coronado, M., & Saleem, K. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2015.06.003 · Cited by 922
Development and validation of the user version of the Mobile application rating scale (uMARS)
Stoyanov, S. R., Hides, L., Kavanagh, D. J., & Wilson, H. (2016) · JMIR mhealth and uhealth · doi:10.2196/mhealth.5849 · Cited by 872 · cited 3×
““Design” was found in 9 different articles(30,31,33,34,36,38,48,49,52) “Information/ Content” (using different terms) observed in 15 studies (38,41,42,44,45,47,49,51,52)“Usability” in 14 articles, 49,52 “Functionality” in 7 studies,(31,33,34,38,43,49,50) “Ethical Issues” in 1 study,(42) “Security and Privacy” in 8 art…”
Mobile devices and apps for health care professionals: uses and benefits.
Ventola, C. L. (2014) · PubMed Central · doi:10.57945/manara.23966214 · Cited by 847
Improving a human-computer dialogue
Molich, R., & Nielsen, J. (1990) · Communications of the ACM · doi:10.1145/77481.77486 · Cited by 821
The Smartphone in Medicine: A Review of Current and Potential Use Among Physicians and Students
Ozdalga, E., Ozdalga, A., & Ahuja, N. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.1994 · Cited by 654
Empirical Studies on Usability of mHealth Apps: A Systematic Literature Review
Zapata, B. C., Fernández-Alemán, J. L., Idri, A., & Toval, A. (2015) · Journal of Medical Systems · doi:10.1007/s10916-014-0182-2 · Cited by 585
A user-centered model for designing consumer mobile health (mHealth) applications (apps)
Schnall, R., Rojas, M., Bakken, S., Brown, W., Carballo-Dieguez, A., Carry, M., Gelaude, D., Mosley, J. P., & Travers, J. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.02.002 · Cited by 583
Who uses mobile phone health apps and does use matter? A secondary data analytics approach
Carroll, J. K., Moorhead, A., Bond, R., LeBlanc, W. G., Petrella, R. J., & Fiscella, K. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.5604 · Cited by 540
Brown, W., Yen, P.-Y., Rojas, M., & Schnall, R. (2013) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2013.08.001 · Cited by 299
Boudreaux, E. D., Waring, M. E., Hayes, R. B., Sadasivam, R. S., Mullen, S. P., & Pagoto, S. (2014) · Translational Behavioral Medicine · doi:10.1007/s13142-014-0293-9 · Cited by 281
Apps for asthma self-management: a systematic assessment of content and tools
Huckvale, K., Car, M., Morrison, C. P., & Car, J. (2012) · BMC Medicine · doi:10.1186/1741-7015-10-144 · Cited by 252
The Most Popular Smartphone Apps for Weight Loss: A Quality Assessment
Chen, J., Cade, J. E., & Allman-Farinelli, M. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.4334 · Cited by 219
Mobile health applications to assist patients with diabetes: lessons learned and design implications
Årsand, E., Frøisland, D. H., Skrøvseth, S. O., Chomutare, T., Tatara, N., Hartvigsen, G., & Tufano, J. T. (2012) · Journal of Diabetes Science and Technology · doi:10.1177/193229681200600525 · Cited by 214
An emerging model of maternity care: Smartphone, midwife, doctor?
Tripp, N., Hainey, K., Liu, A., Poulton, A. S., Peek, M. J., Kim, J., & Nanan, R. K. H. (2013) · Women and Birth · doi:10.1016/j.wombi.2013.11.001 · Cited by 208
The regulation of mobile health applications
Barton, A. (2012) · BMC Medicine · doi:10.1186/1741-7015-10-46 · Cited by 182
Patient portals and health apps: Pitfalls, promises, and what one might learn from the other
Baldwin, J. L., Singh, H., Sittig, D. F., & Giardina, T. D. (2016) · Healthcare · doi:10.1016/j.hjdsi.2016.08.004 · Cited by 179
Assessing the quality and usability of smartphone apps for pain self-management
Reynoldson, C., Stones, C., Allsop, M., Gardner, P., Bennett, M. I., Closs, S. J., Jones, R., & Knapp, P. (2014) · Pain Medicine · doi:10.1111/pme.12327 · Cited by 178
The evolution of mobile apps for asthma: an updated systematic assessment of content and tools
Huckvale, K., Morrison, C., Ouyang, J., Ghaghda, A., & Car, J. (2015) · BMC Medicine · doi:10.1186/s12916-015-0303-x · Cited by 166
Wellbeing, Recovery and Mental Health
Slade, M., Oades, L. G., & Jarden, A. (2017) · Cambridge University Press eBooks · doi:10.1017/9781316339275 · Cited by 158
Utilization and Content Evaluation of Mobile Applications for Pregnancy, Birth, and Child Care
Lee, Y., & Moon, M. (2016) · Healthcare Informatics Research · doi:10.4258/hir.2016.22.2.73 · Cited by 154
Challenges in Assessing Mobile Health App Quality
Grundy, Q. H., Wang, Z., & Bero, L. A. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.07.009 · Cited by 154 · cited 2×
“More recently, Grundy et al.(60) conducted a systematic review to find and explain emerging and common methods for evaluating the quality of mHealth apps.”
BackgroundA Systematic Review of Quality Assessment Methods for Smartphone Health Apps
BinDhim, N. F., Hawkey, A., & Trevena, L. (2015) · Telemedicine and e-Health · doi:10.1089/tmj.2014.0088 · Cited by 142
Singh, K., & Bates, D. W. (2021) · PubMed · doi:10.26099/g1ws-c172 · Cited by 136
Mobile phone health apps for diabetes management: Current evidence and future developments
Goyal, S., & Cafazzo, J. A. (2013) · QJM · doi:10.1093/qjmed/hct203 · Cited by 107
Powell, A. C., Torous, J. B., Chan, S. R., Raynor, G., Shwarts, E., Shanahan, M. E., & Landman, A. (2016) · JMIR mhealth and uhealth · doi:10.2196/mhealth.5176 · Cited by 103
Infant Feeding Websites and Apps: A Systematic Assessment of Quality and Content
Taki, S., Campbell, K. J., Russell, C. G., Elliott, R. M., Laws, R., & Denney‐Wilson, E. (2015) · Interactive Journal of Medical Research · doi:10.2196/ijmr.4323 · Cited by 89
Analysis of mobile health applications for a broad spectrum of consumers: A user experience approach
Garcia-Gomez, J. M., Díez, I. D. L. T., Vicente, J., Robles, M., López-Coronado, M., & Rodrigues, J. J. P. C. (2014) · Health Informatics Journal · doi:10.1177/1460458213479598 · Cited by 83
Domnich, A., Arata, L., Amicizia, D., Signori, A., Patrick, B., Stoyanov, S. R., Hides, L. M., Gasparini, R., & Panatto, D. (2016) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-016-0323-2 · Cited by 83
Martínez-Pérez, B., Díez, I. D. L. T., Candelas-Plasencia, S., & López-Coronado, M. (2013) · Journal of Medical Systems · doi:10.1007/s10916-013-9976-x · Cited by 74
Smartphone Applications to Support Tuberculosis Prevention and Treatment: Review and Evaluation
Iribarren, S., Schnall, R., Stone, P. W., & Carballo‐Diéguez, A. (2016) · JMIR mhealth and uhealth · doi:10.2196/mhealth.5022 · Cited by 72
Mobile PHRs Compliance with Android and iOS Usability Guidelines
Zapata, B. C., Niñirola, A. H., Idri, A., Fernández‐Alemán, J. L., & Toval, A. (2014) · Journal of Medical Systems · doi:10.1007/s10916-014-0081-6 · Cited by 69
App Chronic Disease Checklist: Protocol to Evaluate Mobile Apps for Chronic Disease Self-Management
Anderson, K., Burford, O., & Emmerton, L. (2016) · JMIR Research Protocols · doi:10.2196/resprot.6194 · Cited by 69
Mobile Health Applications for Pediatric Care: Review and Comparison
Morse, S. S., Murugiah, M. K., Soh, Y. C., Wong, T. W., & Ming, L. C. (2017) · Therapeutic Innovation & Regulatory Science · doi:10.1177/2168479017725557 · Cited by 63
Health Informatics: Practical Guide for Healthcare and Information Technology Professionals
Hoyt, R., & Yoshihashi, A. K. (2010) · Cited by 57
Compliance of Blood Donation Apps with Mobile OS Usability Guidelines
Ouhbi, S., Fernández‐Alemán, J. L., Pozo, J. R., Bajta, M. E., Toval, A., & Idri, A. (2015) · Journal of Medical Systems · doi:10.1007/s10916-015-0243-1 · Cited by 53
Development and Evaluation of an Evaluation Tool for Healthcare Smartphone Applications
Jin, M., & Kim, J. (2015) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2014.0151 · Cited by 52
Analysis of the Information Quality of Korean Obesity-Management Smartphone Applications
Jeon, E., Park, H., Min, Y. H., & Kim, H. (2014) · Healthcare Informatics Research · doi:10.4258/hir.2014.20.1.23 · Cited by 44
Quality Assessment of Medical Apps that Target Medication-Related Problems
Loy, J. S., Ali, E. E., & Yap, K. Y. (2016) · Journal of Managed Care & Specialty Pharmacy · doi:10.18553/jmcp.2016.22.10.1124 · Cited by 44
Quality of Smartphone Apps Related to Panic Disorder
Van Singer, M., Chatton, A., & Khazaal, Y. (2015) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2015.00096 · Cited by 43
A scientific audit of smartphone applications for the management of obesity
Gan, K. O., & Allman-Farinelli, M. A. (2011) · Australian and New Zealand Journal of Public Health · doi:10.1111/j.1753-6405.2011.00707.x · Cited by 37
A Review and Comparative Analysis of Security Risks and Safety Measures of Mobile Health Apps
Scott, K. M., Richards, D., & Adhikari, R. (2015) · Australasian Journal of Information Systems · doi:10.3127/ajis.v19i0.1210 · Cited by 35
Wu, R. C., Tzanetos, K., Morra, D. J., Quan, S. D., Lo, V., & Wong, B. M. (2013) · Journal of Hospital Medicine · doi:10.1002/jhm.2037 · Cited by 34
Bringing Technology to the Bedside
Whitlow, M., Drake, E., Tullmann, D. F., Hoke, G., & Barth, D. (2014) · CIN Computers Informatics Nursing · doi:10.1097/cin.0000000000000063 · Cited by 32
Technology Acceptance and Adoption of Innovative Smartphone Uses among Hospital Employees
Moon, B. C., & Chang, H. (2014) · Healthcare Informatics Research · doi:10.4258/hir.2014.20.4.304 · Cited by 31
Martínez-Pérez, B., Díez, I. D. L. T., & López-Coronado, M. (2015) · Journal of Medical Systems · doi:10.1007/s10916-015-0303-6 · Cited by 29
mHealth Quality: A Process to Seal the Qualified Mobile Health Apps
Mobin, Y., Jérôme, B., Pierre, D., Lucas, P., & Guillaume, M. (2016) · Studies in Health Technology and Informatics · doi:10.3233/978-1-61499-678-1-205 · Cited by 27
Health information technology usability evaluation: methods, models, and measures
Yen, P. (n.d.) · ProQuest Demo Repository · Cited by 24
Heuristic evaluation of eNote: an electronic notes system.
Bright, T. J., Bakken, S. R., & Johnson, S. B. (2006) · PubMed · Cited by 24
ASSESSMENT METRICS, CHALLENGES AND STRATEGIES FOR MOBILE HEALTH APPS
Zhang, C., Zhang, X., & Halstead-Nussloch, R. (2014) · Issues in Information Systems · doi:10.48009/2_iis_2014_59-66 · Cited by 23
Nutrition Apps: Opportunities to Guide Patients and Grow Your Career
Boyce, B. (2013) · Journal of the Academy of Nutrition and Dietetics · doi:10.1016/j.jand.2013.10.016 · Cited by 21
Toward a use case based classification of mobile health applications
Yasini, M., & Guillaume, M. (2015) · Studies in health technology and informatics · doi:10.3233/978-1-61499-512-8-175 · Cited by 20
Self-citationEvaluating Health Mobile Apps: Information Literacy in Undergraduate and Graduate Nursing Courses
McNiel, P. L., & McArthur, E. C. (2016) · Journal of Nursing Education · doi:10.3928/01484834-20160715-12 · Cited by 16
Current Status and Quality Assessment of Cardiovascular Diseases Related Smartphone Apps in China
Xiao, Q., Wang, Y., Sun, L., Lu, L., & Wu, Y. (2016) · Studies in health technology and informatics · doi:10.3233/978-1-61499-658-3-1030 · Cited by 11
mHealth Apps and Their Risks - Taking Stock.
Albrecht, U., & von Jan, U. (2016) · PubMed · Cited by 9
Pain Control in Palliative Care
Simon, C. (2008) · InnovAiT Education and inspiration for general practice · doi:10.1093/innovait/inn034 · Cited by 6
Flagged first, then by citations
3 of the 62 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- Bertini, 2008, Int J Mob Hum Comput Interact, 20
- Aitken, 2013, Patient Apps for Improved Healthcare: from Novelty to Mainstream
Cited by
Sorted by citations
Ming, L. C., Untong, N., Aliudin, N. A., Osili, N., Kifli, N., Tan, C. S., Goh, K. W., Ng, P. W., Al‐Worafi, Y. M., Lee, K. S., et al. (2020) · JMIR mhealth and uhealth · doi:10.2196/19796 · Cited by 570
What is the clinical value of mHealth for patients?
Rowland, S. P., Fitzgerald, J. E., Holme, T., Powell, J., & McGregor, A. (2020) · npj Digital Medicine · doi:10.1038/s41746-019-0206-x · Cited by 454
Akbar, S., Coiera, E. W., & Magrabi, F. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz175 · Cited by 246
Knitza, J., Taşçılar, K., Meßner, E.-M., Meyer, M., Vossen, D., Pulla, A., Bösch, P., Kittler, J., Kleyer, A., Sewerin, P., et al. (2019) · JMIR mhealth and uhealth · doi:10.2196/14991 · Cited by 170
The German Version of the Mobile App Rating Scale (MARS-G): Development and Validation Study
Messner, E.-M., Terhorst, Y., Barke, A., Baumeister, H., Stoyanov, S., Hides, L., Kavanagh, D., Pryss, R., Sander, L., & Probst, T. (2020) · JMIR mHealth and uHealth · doi:10.2196/14479 · Cited by 156
Metadata
- Authors
- Nouri, R., R Niakan Kalhori, S., Ghazisaeedi, M., Marchand, G., & Yasini, M.
- Affiliations
- Tehran University of Medical Sciences (IR)
- Health Information Management Department, School of Allied Medical Sciences, Tehran University of Medical Sciences, Tehran, Iran
- Research and Development Department, DMD Santé, Paris, France
- Corresponding author
- Ghazisaeedi, Marjan
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2018-05-16
- Volume
- 25
- Issue
- 8
- Pages
- 1089-1098
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC7646896
- OpenAlex
- W2803669305
- Cited by
- 231
- References
- 62
- Citations by year
- 2026: 14 · 2025: 25 · 2024: 37 · 2023: 35 · 2022: 37 · 2021: 37 · 2020: 30 · 2019: 14 · 2018: 2
- FWCI
- 3.75
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Deputy of Education of Tehran University of Medical Sciences
- Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › Health Information Management › Health, Technology, Consumer Behavior
- Keywords
- mobile health apps, quality assessment, evaluation criteria, quality assessment tools, usability, privacy and security, ethical considerations, mobile application design, app functionality, content quality
- MeSH terms
- Evaluation Studies as Topic, Humans, Telemedicine, Mobile Applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
