Criteria for assessing the quality of mHealth apps: a systematic review
Open accessAbstract
Objective: Review the existing studies including an assessment tool/method to assess the quality of mHealth apps; extract their criteria; and provide a classification of the collected criteria. Methods: In accordance with the PRISMA statement, a literature search was conducted in MEDLINE, EMBase, ISI and Scopus for English language citations published from January 1, 2008 to December 22, 2016 for studies including tools or methods for quality assessment of mHealth apps. Two researchers screened the titles and abstracts of all retrieved citations against the inclusion and exclusion criteria. The full text of relevant papers was then individually examined by the same researchers. A senior researcher resolved eventual disagreements and confirmed the relevance of all included papers. The authors, date of publication, subject fields of target mHealth apps, development method, and assessment criteria were extracted from each paper. The extracted assessment criteria were then reviewed, compared, and classified by an expert panel of two medical informatics specialists and two health information management specialists. Results: Twenty-three papers were included in the review. Thirty-eight main classes of assessment criteria were identified. These were reorganized by expert panel into 7 main classes (Design, Information/Content, Usability, Functionality, Ethical Issues, Security and Privacy, and User-perceived value) with 37 sub-classes of criteria. Conclusions: There is a wide heterogeneity in assessment criteria for mHealth apps. It is necessary to define the exact meanings and degree of distinctness of each criterion. This will help to improve the existing tools and may lead to achieve a better comprehensive mHealth app assessment tool.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A recent systematic review identified 23 different published criteria for assessment of mHealth apps.
“A recent systematic review identified 23 different published criteria for assessment of mHealth apps.”
From What is the clinical value of mHealth for patients? (2020) · cites this paper as [38]
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823 · see citing works
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859 · see citing works
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554 · see citing works
Mobile app rating scale: a new tool for assessing the quality of health mobile apps
Stoyanov, S. R., Hides, L., Kavanagh, D. J., Zelenko, O., Tjondronegoro, D., & Mani, M. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.3422 · Cited by 2,233 · see citing works
Mobile-health: A review of current state in 2015
Silva, B. M. C., Rodrigues, J. J. P. C., Díez, I. D. L. T., López-Coronado, M., & Saleem, K. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2015.06.003 · Cited by 876 · see citing works
Development and validation of the user version of the Mobile application rating scale (uMARS)
Stoyanov, S. R., Hides, L., Kavanagh, D. J., & Wilson, H. (2016) · JMIR mhealth and uhealth · doi:10.2196/mhealth.5849 · Cited by 872 · cited 3× · see citing works
““Design” was found in 9 different articles(30,31,33,34,36,38,48,49,52) “Information/ Content” (using different terms) observed in 15 studies (38,41,42,44,45,47,49,51,52)“Usability” in 14 articles, 49,52 “Functionality” in 7 studies,(31,33,34,38,43,49,50) “Ethical Issues” in 1 study,(42) “Security and Privacy” in 8 art…”
Mobile devices and apps for health care professionals: uses and benefits.
Ventola, C. L. (2014) · PubMed Central · doi:10.57945/manara.23966214 · Cited by 845 · see citing works
Improving a human-computer dialogue
Molich, R., & Nielsen, J. (1990) · Communications of the ACM · doi:10.1145/77481.77486 · Cited by 821 · see citing works
The Smartphone in Medicine: A Review of Current and Potential Use Among Physicians and Students
Ozdalga, E., Ozdalga, A., & Ahuja, N. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.1994 · Cited by 654 · see citing works
Empirical Studies on Usability of mHealth Apps: A Systematic Literature Review
Zapata, B. C., Fernández-Alemán, J. L., Idri, A., & Toval, A. (2015) · Journal of Medical Systems · doi:10.1007/s10916-014-0182-2 · Cited by 585 · see citing works
A user-centered model for designing consumer mobile health (mHealth) applications (apps)
Schnall, R., Rojas, M., Bakken, S., Brown, W., Carballo-Dieguez, A., Carry, M., Gelaude, D., Mosley, J. P., & Travers, J. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.02.002 · Cited by 583 · see citing works
Who uses mobile phone health apps and does use matter? A secondary data analytics approach
Carroll, J. K., Moorhead, A., Bond, R., LeBlanc, W. G., Petrella, R. J., & Fiscella, K. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.5604 · Cited by 540 · see citing works
Brown, W., Yen, P.-Y., Rojas, M., & Schnall, R. (2013) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2013.08.001 · Cited by 299 · see citing works
Boudreaux, E. D., Waring, M. E., Hayes, R. B., Sadasivam, R. S., Mullen, S. P., & Pagoto, S. (2014) · Translational Behavioral Medicine · doi:10.1007/s13142-014-0293-9 · Cited by 281 · see citing works
Apps for asthma self-management: a systematic assessment of content and tools
Huckvale, K., Car, M., Morrison, C. P., & Car, J. (2012) · BMC Medicine · doi:10.1186/1741-7015-10-144 · Cited by 252 · see citing works
The Most Popular Smartphone Apps for Weight Loss: A Quality Assessment
Chen, J., Cade, J. E., & Allman-Farinelli, M. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.4334 · Cited by 219 · see citing works
Mobile health applications to assist patients with diabetes: lessons learned and design implications
Årsand, E., Frøisland, D. H., Skrøvseth, S. O., Chomutare, T., Tatara, N., Hartvigsen, G., & Tufano, J. T. (2012) · Journal of Diabetes Science and Technology · doi:10.1177/193229681200600525 · Cited by 214 · see citing works
An emerging model of maternity care: Smartphone, midwife, doctor?
Tripp, N., Hainey, K., Liu, A., Poulton, A. S., Peek, M. J., Kim, J., & Nanan, R. K. H. (2013) · Women and Birth · doi:10.1016/j.wombi.2013.11.001 · Cited by 208 · see citing works
The regulation of mobile health applications
Barton, A. (2012) · BMC Medicine · doi:10.1186/1741-7015-10-46 · Cited by 182 · see citing works
Patient portals and health apps: Pitfalls, promises, and what one might learn from the other
Baldwin, J. L., Singh, H., Sittig, D. F., & Giardina, T. D. (2016) · Healthcare · doi:10.1016/j.hjdsi.2016.08.004 · Cited by 179 · see citing works
Assessing the quality and usability of smartphone apps for pain self-management
Reynoldson, C., Stones, C., Allsop, M., Gardner, P., Bennett, M. I., Closs, S. J., Jones, R., & Knapp, P. (2014) · Pain Medicine · doi:10.1111/pme.12327 · Cited by 178 · see citing works
The evolution of mobile apps for asthma: an updated systematic assessment of content and tools
Huckvale, K., Morrison, C., Ouyang, J., Ghaghda, A., & Car, J. (2015) · BMC Medicine · doi:10.1186/s12916-015-0303-x · Cited by 166 · see citing works
Utilization and Content Evaluation of Mobile Applications for Pregnancy, Birth, and Child Care
Lee, Y., & Moon, M. (2016) · Healthcare Informatics Research · doi:10.4258/hir.2016.22.2.73 · Cited by 154 · see citing works
Challenges in Assessing Mobile Health App Quality
Grundy, Q. H., Wang, Z., & Bero, L. A. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.07.009 · Cited by 154 · cited 2× · see citing works
“More recently, Grundy et al.(60) conducted a systematic review to find and explain emerging and common methods for evaluating the quality of mHealth apps.”
BackgroundWellbeing, Recovery and Mental Health
(2017) · Cambridge University Press eBooks · doi:10.1017/9781316339275 · Cited by 153 · see citing works
A Systematic Review of Quality Assessment Methods for Smartphone Health Apps
BinDhim, N. F., Hawkey, A., & Trevena, L. (2015) · Telemedicine and e-Health · doi:10.1089/tmj.2014.0088 · Cited by 142 · see citing works
Singh, K., & Bates, D. W. (2021) · PubMed · doi:10.26099/g1ws-c172 · Cited by 136 · see citing works
Mobile phone health apps for diabetes management: Current evidence and future developments
Goyal, S., & Cafazzo, J. A. (2013) · QJM · doi:10.1093/qjmed/hct203 · Cited by 107 · see citing works
Powell, A. C., Torous, J. B., Chan, S. R., Raynor, G., Shwarts, E., Shanahan, M. E., & Landman, A. (2016) · JMIR mhealth and uhealth · doi:10.2196/mhealth.5176 · Cited by 103 · see citing works
Infant Feeding Websites and Apps: A Systematic Assessment of Quality and Content
Taki, S., Campbell, K. J., Russell, C. G., Elliott, R. M., Laws, R., & Denney‐Wilson, E. (2015) · Interactive Journal of Medical Research · doi:10.2196/ijmr.4323 · Cited by 89 · see citing works
Analysis of mobile health applications for a broad spectrum of consumers: A user experience approach
Garcia-Gomez, J. M., Díez, I. D. L. T., Vicente, J., Robles, M., López-Coronado, M., & Rodrigues, J. J. P. C. (2014) · Health Informatics Journal · doi:10.1177/1460458213479598 · Cited by 83 · see citing works
Domnich, A., Arata, L., Amicizia, D., Signori, A., Patrick, B., Stoyanov, S. R., Hides, L. M., Gasparini, R., & Panatto, D. (2016) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-016-0323-2 · Cited by 83 · see citing works
Martínez-Pérez, B., Díez, I. D. L. T., Candelas-Plasencia, S., & López-Coronado, M. (2013) · Journal of Medical Systems · doi:10.1007/s10916-013-9976-x · Cited by 74 · see citing works
Smartphone Applications to Support Tuberculosis Prevention and Treatment: Review and Evaluation
Iribarren, S., Schnall, R., Stone, P. W., & Carballo‐Diéguez, A. (2016) · JMIR mhealth and uhealth · doi:10.2196/mhealth.5022 · Cited by 72 · see citing works
Mobile PHRs Compliance with Android and iOS Usability Guidelines
Zapata, B. C., Niñirola, A. H., Idri, A., Fernández‐Alemán, J. L., & Toval, A. (2014) · Journal of Medical Systems · doi:10.1007/s10916-014-0081-6 · Cited by 69 · see citing works
App Chronic Disease Checklist: Protocol to Evaluate Mobile Apps for Chronic Disease Self-Management
Anderson, K., Burford, O., & Emmerton, L. (2016) · JMIR Research Protocols · doi:10.2196/resprot.6194 · Cited by 65 · see citing works
Mobile Health Applications for Pediatric Care: Review and Comparison
Morse, S. S., Murugiah, M. K., Soh, Y. C., Wong, T. W., & Ming, L. C. (2017) · Therapeutic Innovation & Regulatory Science · doi:10.1177/2168479017725557 · Cited by 63 · see citing works
Health Informatics: Practical Guide for Healthcare and Information Technology Professionals
Hoyt, R., & Yoshihashi, A. K. (2010) · Cited by 57 · see citing works
Compliance of Blood Donation Apps with Mobile OS Usability Guidelines
Ouhbi, S., Fernández‐Alemán, J. L., Pozo, J. R., Bajta, M. E., Toval, A., & Idri, A. (2015) · Journal of Medical Systems · doi:10.1007/s10916-015-0243-1 · Cited by 53 · see citing works
Development and Evaluation of an Evaluation Tool for Healthcare Smartphone Applications
Jin, M., & Kim, J. (2015) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2014.0151 · Cited by 52 · see citing works
Analysis of the Information Quality of Korean Obesity-Management Smartphone Applications
Jeon, E., Park, H., Min, Y. H., & Kim, H. (2014) · Healthcare Informatics Research · doi:10.4258/hir.2014.20.1.23 · Cited by 44 · see citing works
Quality Assessment of Medical Apps that Target Medication-Related Problems
Loy, J. S., Ali, E. E., & Yap, K. Y. (2016) · Journal of Managed Care & Specialty Pharmacy · doi:10.18553/jmcp.2016.22.10.1124 · Cited by 44 · see citing works
Quality of Smartphone Apps Related to Panic Disorder
Van Singer, M., Chatton, A., & Khazaal, Y. (2015) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2015.00096 · Cited by 43 · see citing works
A scientific audit of smartphone applications for the management of obesity
Gan, K. O., & Allman-Farinelli, M. A. (2011) · Australian and New Zealand Journal of Public Health · doi:10.1111/j.1753-6405.2011.00707.x · Cited by 37 · see citing works
A Review and Comparative Analysis of Security Risks and Safety Measures of Mobile Health Apps
Scott, K. M., Richards, D., & Adhikari, R. (2015) · Australasian Journal of Information Systems · doi:10.3127/ajis.v19i0.1210 · Cited by 35 · see citing works
Wu, R. C., Tzanetos, K., Morra, D. J., Quan, S. D., Lo, V., & Wong, B. M. (2013) · Journal of Hospital Medicine · doi:10.1002/jhm.2037 · Cited by 34 · see citing works
Bringing Technology to the Bedside
Whitlow, M., Drake, E., Tullmann, D. F., Hoke, G., & Barth, D. (2014) · CIN Computers Informatics Nursing · doi:10.1097/cin.0000000000000063 · Cited by 32 · see citing works
Technology Acceptance and Adoption of Innovative Smartphone Uses among Hospital Employees
Moon, B. C., & Chang, H. (2014) · Healthcare Informatics Research · doi:10.4258/hir.2014.20.4.304 · Cited by 31 · see citing works
Martínez-Pérez, B., Díez, I. D. L. T., & López-Coronado, M. (2015) · Journal of Medical Systems · doi:10.1007/s10916-015-0303-6 · Cited by 29 · see citing works
mHealth Quality: A Process to Seal the Qualified Mobile Health Apps
Mobin, Y., Jérôme, B., Pierre, D., Lucas, P., & Guillaume, M. (2016) · Studies in Health Technology and Informatics · doi:10.3233/978-1-61499-678-1-205 · Cited by 27 · see citing works
Health information technology usability evaluation: methods, models, and measures
Yen, P. (n.d.) · ProQuest Demo Repository · Cited by 24 · see citing works
Heuristic evaluation of eNote: an electronic notes system.
Bright, T. J., Bakken, S. R., & Johnson, S. B. (2006) · PubMed · Cited by 24 · see citing works
ASSESSMENT METRICS, CHALLENGES AND STRATEGIES FOR MOBILE HEALTH APPS
Zhang, C., Zhang, X., & Halstead-Nussloch, R. (2014) · Issues in Information Systems · doi:10.48009/2_iis_2014_59-66 · Cited by 23 · see citing works
Nutrition Apps: Opportunities to Guide Patients and Grow Your Career
Boyce, B. (2013) · Journal of the Academy of Nutrition and Dietetics · doi:10.1016/j.jand.2013.10.016 · Cited by 21 · see citing works
Toward a use case based classification of mobile health applications
Yasini, M., & Guillaume, M. (2015) · Studies in health technology and informatics · doi:10.3233/978-1-61499-512-8-175 · Cited by 20 · see citing works
Self-citationEvaluating Health Mobile Apps: Information Literacy in Undergraduate and Graduate Nursing Courses
McNiel, P. L., & McArthur, E. C. (2016) · Journal of Nursing Education · doi:10.3928/01484834-20160715-12 · Cited by 16 · see citing works
Current Status and Quality Assessment of Cardiovascular Diseases Related Smartphone Apps in China
Xiao, Q., Wang, Y., Sun, L., Lu, L., & Wu, Y. (2016) · Studies in health technology and informatics · doi:10.3233/978-1-61499-658-3-1030 · Cited by 11 · see citing works
mHealth Apps and Their Risks - Taking Stock.
Albrecht, U., & von Jan, U. (2016) · PubMed · Cited by 9 · see citing works
Pain Control in Palliative Care
Simon, C. (2008) · InnovAiT Education and inspiration for general practice · doi:10.1093/innovait/inn034 · Cited by 6 · see citing works
Flagged first, then by citations
3 of the 62 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Ming, L. C., Untong, N., Aliudin, N. A., Osili, N., Kifli, N., Tan, C. S., Goh, K. W., Ng, P. W., Al‐Worafi, Y. M., Lee, K. S., et al. (2020) · JMIR mhealth and uhealth · doi:10.2196/19796 · Cited by 570 · see citing works
What is the clinical value of mHealth for patients?
Rowland, S. P., Fitzgerald, J. E., Holme, T., Powell, J., & McGregor, A. (2020) · npj Digital Medicine · doi:10.1038/s41746-019-0206-x · Cited by 454 · see citing works
Akbar, S., Coiera, E. W., & Magrabi, F. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz175 · Cited by 246 · see citing works
Knitza, J., Taşçılar, K., Meßner, E.-M., Meyer, M., Vossen, D., Pulla, A., Bösch, P., Kittler, J., Kleyer, A., Sewerin, P., et al. (2019) · JMIR mhealth and uhealth · doi:10.2196/14991 · Cited by 170 · see citing works
The German Version of the Mobile App Rating Scale (MARS-G): Development and Validation Study
Messner, E.-M., Terhorst, Y., Barke, A., Baumeister, H., Stoyanov, S., Hides, L., Kavanagh, D., Pryss, R., Sander, L., & Probst, T. (2020) · JMIR mHealth and uHealth · doi:10.2196/14479 · Cited by 156 · see citing works
Ramos, G., Ponting, C., Labao, J. P., & Sobowale, K. (2021) · Behaviour Research and Therapy · doi:10.1016/j.brat.2021.103990 · Cited by 122 · see citing works
A design and evaluation framework for digital health interventions
Kowatsch, T., Otto, L., Harperink, S., Cotti, A., & Schlieter, H. (2019) · it - Information Technology · doi:10.1515/itit-2019-0019 · Cited by 118 · see citing works
Mobile health app usability and quality rating scales: a systematic review
Azad‐Khaneghah, P., Neubauer, N., Cruz, A. M., & Liu, L. (2020) · Disability and Rehabilitation Assistive Technology · doi:10.1080/17483107.2019.1701103 · Cited by 119 · see citing works
Galetsi, P., Katsaliaki, K., & Kumar, S. (2022) · Technovation · doi:10.1016/j.technovation.2022.102598 · Cited by 117 · see citing works
Understanding the quality, effectiveness and attributes of top-rated smartphone health apps
Wisniewski, H., Liu, G., Henson, P., Vaidyam, A., Hajratalli, N. K., Onnela, J.-P., & Torous, J. (2019) · Evidence Based Mental Health · doi:10.1136/ebmental-2018-300069 · Cited by 112 · see citing works
Pal, S., Biswas, B. N., Gupta, R., Kumar, A., & Gupta, S. (2022) · Journal of Business Research · doi:10.1016/j.jbusres.2022.113484 · Cited by 94 · see citing works
Roberts, A., Davenport, T. A., Wong, T., Moon, H.-W., Hickie, I. B., & LaMonica, H. M. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100379 · Cited by 99 · see citing works
Muro-Culebras, A., Escriche-Escuder, A., Martín‐Martín, J., Roldán‐Jiménez, C., De-Torres, I., Ruiz‐Muñoz, M., González‐Sánchez, M., Mayoral, F., Biró, A., Tang, W. J., et al. (2020) · JMIR mhealth and uhealth · doi:10.2196/15433 · Cited by 82 · see citing works
Hensher, M., Cooper, P., Dona, S. W. A., Angeles, M. R., Nguyen, D., Heynsbergh, N., Chatterton, M. L., & Peeters, A. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab041 · Cited by 80 · see citing works
Impact of mobile health and medical applications on clinical practice in gastroenterology
Kernebeck, S., Busse, T. S., Böttcher, M. D., Weitz, J., Ehlers, J. P., & Bork, U. (2020) · World Journal of Gastroenterology · doi:10.3748/wjg.v26.i29.4182 · Cited by 70 · see citing works
A Critical Appraisal of Fertility and Menstrual Tracking Apps for the iPhone
Zwingerman, R., Chaikof, M., & Jones, C. A. (2019) · Journal of Obstetrics and Gynaecology Canada · doi:10.1016/j.jogc.2019.09.023 · Cited by 68 · see citing works
Determinant Factors of Public Acceptance of Stress Management Apps: Survey Study
Apolinário-Hagen, J., Hennemann, S., Fritsche, L., Drüge, M., & Breil, B. (2019) · JMIR Mental Health · doi:10.2196/15373 · Cited by 63 · see citing works
Sawrikar, V., & Mote, K. (2022) · Health Policy and Technology · doi:10.1016/j.hlpt.2022.100686 · Cited by 53 · see citing works
Observational study of UK mobile health apps for COVID-19
Chidambaram, S., Erridge, S., Kinross, J. K., & Purkayastha, S. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30144-8 · Cited by 49 · see citing works
Emerging Digital Technologies in Healthcare with a Spotlight on Cybersecurity: A Narrative Review
Arafa, A., Sheerah, H. A., & Alsalamah, S. (2023) · Information · doi:10.3390/info14120640 · Cited by 49 · see citing works
Jacob, C., Lindeque, J. P., Klein, A., Ivory, C. J., Heuss, S., & Peter, M. K. (2023) · JMIR Human Factors · doi:10.2196/45143 · Cited by 43 · see citing works
Smartphone and mobile health apps for tinnitus: Systematic identification, analysis, and assessment
Mehdi, M., Stach, M., Riha, C., Neff, P., Dode, A., Pryss, R., Schlee, W., Reichert, M., & Hauck, F. J. (2020) · JMIR mHealth and uHealth · doi:10.2196/21767 · Cited by 40 · see citing works
“Several mobile health (mHealth) app assessment tools and models are available [26-29], yet they either are focused on a particular health domain or are too time consuming and complex to employ in research.”
BackgroundGiebel, G. D., Abels, C., Plescher, F., Speckemeier, C., Schrader, N. F., Börchers, K., Wasem, J., Neusser, S., & Blase, N. (2024) · Journal of Medical Internet Research · doi:10.2196/49982 · Cited by 44 · see citing works
Mobile Health App Database - A Repository for Quality Ratings of mHealth Apps
Stach, M., Kraft, R., Probst, T., Meßner, E.-M., Terhorst, Y., Baumeister, H., Schickler, M., Reichert, M., Sander, L., & Pryss, R. C. (2020) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms49503.2020.00087 · Cited by 45 · see citing works
Clinical Validation of Digital Healthcare Solutions: State of the Art, Challenges and Opportunities
Gomis‐Pastor, M., Berdún, J., Borràs-Santos, A., López, A. D. D., Rama, B. F.-M., García-Esquirol, Ó., Gratacòs, M., Ontiveros, G., Cruz, R. P., Real, J., et al. (2024) · Healthcare · doi:10.3390/healthcare12111057 · Cited by 44 · see citing works
Juengst, S. B., Terhorst, L. A., Nabasny, A., Wallace, T. D., Weaver, J., Osborne, C. L., Burns, S. P., Wright, B., Wen, P.-S., Kew, C. L., et al. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18042173 · Cited by 43 · see citing works
Personal Privacy Evaluation of Smart Devices Applications Serving Hajj and Umrah Rituals
Shambour, M. K. Y., & Gutub, A. A. A. (2021) · Journal of Engineering Research · doi:10.36909/jer.13199 · Cited by 41 · see citing works
Leveraging mobile health applications for biomedical research and citizen science: a scoping review
Schmitz, H., Howe, C. L., Armstrong, D. G., & Subbian, V. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy130 · Cited by 38 · see citing works
Shabir, H., D’Costa, M., Mohiaddin, Z., Moti, Z., Rashid, H., Sadowska, D., Alam, B., & Cox, B. M. (2022) · European Journal of Investigation in Health Psychology and Education · doi:10.3390/ejihpe12020012 · Cited by 38 · see citing works
Impact of Apps as Assistive Devices for Visually Impaired Persons
Pundlik, S. J., Shivshanker, P., & Luo, G. (2023) · Annual Review of Vision Science · doi:10.1146/annurev-vision-111022-123837 · Cited by 37 · see citing works
Rigorous and rapid evidence assessment in digital health with the evidence DEFINED framework
Silberman, J., Wicks, P. J., Patel, S., Sarlati, S., Park, S., Korolev, I. O., Carl, J. R., Owusu, J. T., Mishra, V. K., Kaur, M., et al. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00836-5 · Cited by 36 · see citing works
Gasteiger, N., Dowding, D., Norman, G., McGarrigle, L., Eost‐Telling, C., Jones, D. J., Vercell, A., Ali, S. M., & O’Connor, S. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-073283 · Cited by 37 · see citing works
Innovations in Geroscience to enhance mobility in older adults
Anton, S. D., Cruz‐Almeida, Y., Singh, A., Alpert, J. M., Bensadon, B. A., Cabrera, M., Clark, D. J., Ebner, N. C., Esser, K. A., Fillingim, R. B., et al. (2020) · Experimental Gerontology · doi:10.1016/j.exger.2020.111123 · Cited by 37 · see citing works
Polhemus, A. M., Novák, J. M., Majid, S., Simblett, S. K., Morris, D., Bruce, S., Burke, P., Dockendorf, M. F., Temesi, G., & Wykes, T. (2021) · JMIR Mental Health · doi:10.2196/25249 · Cited by 36 · see citing works
Japanese Version of the Mobile App Rating Scale (MARS): Development and Validation
Yamamoto, K., Ito, M., Sakata, M., Koizumi, S., Hashisako, M., Sato, M., Stoyanov, S. R., & Furukawa, T. A. (2022) · JMIR mhealth and uhealth · doi:10.2196/33725 · Cited by 33 · see citing works
Chan, R. J., Howell, D., Lustberg, M. B., Mustian, K. M., Koczwara, B., Ng, C. C., Kim, Y., Nápoles, A. M., Dixit, N., Klemanski, D., et al. (2020) · Supportive Care in Cancer · doi:10.1007/s00520-020-05513-x · Cited by 33 · see citing works
A systematic review of smartphone applications and devices for obstructive sleep apnea
Baptista, P., Martin, F., Ross, H., O’Connor-Reina, C., Plaza, G., & Casale, M. (2022) · Brazilian Journal of Otorhinolaryngology · doi:10.1016/j.bjorl.2022.01.004 · Cited by 33 · see citing works
The Mobile App Development and Assessment Guide (MAG): Delphi-Based Validity Study
Llorens-Vernet, P., & Miró, J. (2020) · JMIR mhealth and uhealth · doi:10.2196/17760 · Cited by 32 · see citing works
Leong, Q. Y., Sridhar, S., Blasiak, A., Tadeo, X., Yeo, G., Remus, A., & Ho, D. (2021) · Journal of Medical Internet Research · doi:10.2196/27388 · Cited by 34 · see citing works
Ryan, S., Chasaide, N. N., Hanrahan, S. O., Corcoran, D., Caulfield, B., & Argent, R. (2022) · JMIR Rehabilitation and Assistive Technologies · doi:10.2196/34355 · Cited by 32 · see citing works
Galavi, Z., Norouzi, S., & Khajouei, R. (2024) · Digital Health · doi:10.1177/20552076241253539 · Cited by 30 · see citing works
Woulfe, F., Fadahunsi, K. P., Smith, S. D., Chirambo, G. B., Larsson, E. E. C., Henn, P., Mawkin, M., & Donoghue, J. O. (2021) · JMIR mhealth and uhealth · doi:10.2196/28384 · Cited by 30 · see citing works
Karim, H., Choobineh, H., Kheradbin, N., Hosseiniravandi, M., Naserpor, A., & Safdari, R. (2020) · Digital Health · doi:10.1177/2055207620906956 · Cited by 29 · see citing works
Aji, M., Gordon, C. J., Stratton, E., Calvo, R. A., Bartlett, D. J., Grunstein, R. R., & Glozier, N. (2021) · Journal of Medical Internet Research · doi:10.2196/24607 · Cited by 29 · see citing works
Prioritization of Quality Principles for Health Apps Using the Kano Model: Survey Study
Malinka, C., von Jan, U., & Albrecht, U. (2022) · JMIR mhealth and uhealth · doi:10.2196/26563 · Cited by 28 · see citing works
Evaluation of mobile apps for treatment of patients at risk of developing gestational diabetes
Tassone, C., Keshavjee, K., Paglialonga, A., Moreira, N., Pinto, J., & Quintana, Y. (2020) · Health Informatics Journal · doi:10.1177/1460458219896639 · Cited by 27 · see citing works
Issues in reporting of systematic review methods in health app-focused reviews: A scoping review
Grainger, R., Devan, H., Sangelaji, B., & Hay-Smith, J. E. (2020) · Health Informatics Journal · doi:10.1177/1460458220952917 · Cited by 27 · see citing works
Evans, K., Donelan, J. F., Rennick‐Egglestone, S., Cox, S., & Kuipers, Y. (2021) · Journal of Medical Internet Research · doi:10.2196/31831 · Cited by 30 · see citing works
Raeesi, A., Khajouei, R., & Ahmadian, L. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-02029-8 · Cited by 26 · see citing works
Digital Tools Designed to Obtain the History of Present Illness From Patients: Scoping Review
Berdahl, C. T., Henreid, A. J., Pevnick, J. M., Zheng, K., & Nuckols, T. K. (2022) · Journal of Medical Internet Research · doi:10.2196/36074 · Cited by 26 · see citing works
Which criteria are important in usability evaluation of mHealth applications: an umbrella review
Galavi, Z., Montazeri, M., & Khajouei, R. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02738-2 · Cited by 30 · see citing works
Magalhães, B. M. B. D. S., Fernandes, C. S., Martínez‐Galiano, J. M., & Santos, C. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2020.104293 · Cited by 29 · see citing works
Rochat, J., Ehrler, F., Siebert, J. N., Ricci, A., Ruiz, V. G., & Lovis, C. (2021) · JMIR Pediatrics and Parenting · doi:10.2196/25540 · Cited by 26 · see citing works
Simon, L., Reimann, J., Steubl, L., Stach, M., Spiegelhalder, K., Sander, L., Baumeister, H., Meßner, E.-M., & Terhorst, Y. (2022) · Journal of Sleep Research · doi:10.1111/jsr.13642 · Cited by 24 · see citing works
Albrecht, U., Malinka, C., Long, S., Raupach, T., Hasenfuss, G. P., & von Jan, U. (2019) · JMIR mhealth and uhealth · doi:10.2196/13375 · Cited by 25 · see citing works
Ali, S., Alizai, H., Hagos, D. J., Rubio, S. R., Calabia, D., Jimenez, P., Senthil, V. A., & Appel, L. (2024) · JMIR mhealth and uhealth · doi:10.2196/50186 · Cited by 24 · see citing works
Wu, J., Zhang, G., Xing, Y., Liu, Y., Zhang, Z., Dong, Y., & Herrera‐Viedma, E. (2023) · Applied Intelligence · doi:10.1007/s10489-023-04485-9 · Cited by 24 · see citing works
Quality assessment of mHealth apps: a scoping review
Giebel, G. D., Speckemeier, C., Schrader, N. F., Abels, C., Plescher, F., Hillerich, V., Wiedemann, D., Börchers, K., Wasem, J., Blase, N., et al. (2024) · Frontiers in Health Services · doi:10.3389/frhs.2024.1372871 · Cited by 31 · see citing works
Panchal, M., Singh, S., & Rodriguez–Villegas, E. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-053395 · Cited by 23 · see citing works
Weiß, J.-P., Esdar, M., & Hübner, U. (2021) · JMIR mhealth and uhealth · doi:10.2196/27232 · Cited by 24 · see citing works
Rahimi, R., Khoundabi, B., & fathian, A. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104571 · Cited by 21 · see citing works
Marcu, G. M., Aizen, R., Roth, A. M., Lankenau, S. E., & Schwartz, D. G. (2019) · JAMIA Open · doi:10.1093/jamiaopen/ooz068 · Cited by 21 · see citing works
Brown, J. M., Franco‐Arellano, B., Froome, H. M., Siddiqi, A., Mahmood, A., & Arcand, J. (2022) · JMIR mhealth and uhealth · doi:10.2196/31537 · Cited by 20 · see citing works
Thomas, T. H., Go, K., Go, K., McKinley, N. J., Dougherty, K. R., You, K.-L., & Lee, Y. J. (2022) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2022.104782 · Cited by 20 · see citing works
Towards Understanding the Usability Attributes of AI-Enabled eHealth Mobile Applications
Alzahrani, A., Gay, V. C., Alturki, R., & AlGhamdi, M. J. (2021) · Journal of Healthcare Engineering · doi:10.1155/2021/5313027 · Cited by 19 · see citing works
RetractedTengilimoğlu, D., Orhan, F., Tekin, P. Ş., & Younis, M. Z. (2024) · Healthcare · doi:10.3390/healthcare12030287 · Cited by 20 · see citing works
Magalhães, B. M. B. D. S., Fernandes, C. S., Santos, C., & Martínez‐Galiano, J. M. (2020) · Cancer Nursing · doi:10.1097/ncc.0000000000000823 · Cited by 21 · see citing works
Becker, E. R., Shegog, R., Savas, L. S., Frost, E. L., Coan, S. P., Healy, C. M., Spinner, S. W., & Vernon, S. W. (2021) · JMIR Pediatrics and Parenting · doi:10.2196/30340 · Cited by 18 · see citing works
Assessing the Quality of Mobile Health-Related Apps: Interrater Reliability Study of Two Guides
Miró, J., & Llorens-Vernet, P. (2021) · JMIR mhealth and uhealth · doi:10.2196/26471 · Cited by 18 · see citing works
Noël, R., Taramasco, C., & Márquez, G. (2021) · Journal of Medical Internet Research · doi:10.2196/26577 · Cited by 16 · see citing works
Willms, A., Rhodes, R. E., & Liu, S. (2023) · JMIR Formative Research · doi:10.2196/43823 · Cited by 16 · see citing works
Esteves, M., Esteves, M., Abelha, A., & Machado, J. M. (2019) · Sensors · doi:10.3390/s19183951 · Cited by 17 · see citing works
Mobile health application for Thai women: investigation and model
Kongjit, C., Nimmolrat, A., & Khamaksorn, A. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-01944-0 · Cited by 17 · see citing works
Ciudad-Gutiérrez, P., Valle-Moreno, P. D., Lora-Escobar, S. J., Guisado-Gil, A. B., & Lara, E. R. A. (2023) · Journal of Medical Systems · doi:10.1007/s10916-023-02008-0 · Cited by 15 · see citing works
Yuan, S., Liu, M., Peng, Y., Hu, J., Li, B., Ding, X., & Xie, L. (2024) · JMIR mhealth and uhealth · doi:10.2196/52169 · Cited by 15 · see citing works
Development and evaluation of an online infertility peer supporter training program
Grunberg, P. H., Dennis, C., Costa, D. D., Gagné, K., Idelson, R., & Zelkowitz, P. (2019) · Patient Education and Counseling · doi:10.1016/j.pec.2019.11.019 · Cited by 14 · see citing works
Greek validation of the user version of the Mobile Application Rating Scale (uMARS)
Chasiotis, G., Stoyanov, S. R., Karatzas, A. D., & Gravas, S. (2023) · Journal of International Medical Research · doi:10.1177/03000605231161213 · Cited by 13 · see citing works
Zhou, T., Salman, D., & McGregor, A. H. (2024) · JMIR mhealth and uhealth · doi:10.2196/53262 · Cited by 14 · see citing works
Honglin, D., Jianghua, Z., & Chen, H. (2024) · Scientific Reports · doi:10.1038/s41598-024-75410-4 · Cited by 13 · see citing works
Una Revisión sistemática de instrumentos que evalúan la calidad de aplicaciones móviles de salud
Morales, C. D., & Hueros, A. M. D. (2023) · Pixel-Bit Revista de Medios y Educación · doi:10.12795/pixelbit.97867 · Cited by 13 · see citing works
Development of a Multidimensional App-Quality Assessment Tool for Health-Related Apps (AQUA)
O’Rourke, T., Pryss, R. C., Schlee, W., & Probst, T. (2020) · Digital Psychology · doi:10.24989/dp.v1i2.1816 · Cited by 13 · see citing works
A scoping review of mobile apps for use with palliative patients in the context of home care
Cruz, S., Fernandes, C. S., & Magalhães, B. M. B. D. S. (2023) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2023.105166 · Cited by 12 · see citing works
Evaluation of the Scientific Quality and Usability of Digital Dietary Assessment Tools
Böse, V., Frenser, M., Schumacher, M., & Fischer, T. W. (2024) · Dietetics · doi:10.3390/dietetics3020013 · Cited by 12 · see citing works
Developing a standardized framework for evaluating health apps using natural language processing
Herpertz, J., Dwyer, B., Taylor, J., Opel, N., & Torous, J. B. (2025) · Scientific Reports · doi:10.1038/s41598-025-96369-w · Cited by 12 · see citing works
Martins, A. I., Santinha, G. A. D. S., Almeida, A. M., Ribeiro, Ó. M., Silva, T., Rocha, N. P., & Silva, A. G. (2023) · Journal of Medical Internet Research · doi:10.2196/44326 · Cited by 12 · see citing works
Varshney, U., Singh, N., Bourgeois, A. G., & Dube, S. R. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab277 · Cited by 12 · see citing works
Quality Assessment of Digital Health Apps: Umbrella Review
Hyzy, M., Bond, R., Mulvenna, M. D., Carracedo, J. M., Bai, L., & Leigh, S. (2024) · Journal of Medical Internet Research · doi:10.2196/58616 · Cited by 12 · see citing works
Chen, T. (2025) · Gender Technology and Development · doi:10.1080/09718524.2025.2503561 · Cited by 11 · see citing works
Apps und ihre Anwendungsgebiete in der Rheumatologie
Krusche, M., Ruffer, N., Grahammer, M., & Knitza, J. (2020) · Zeitschrift für Rheumatologie · doi:10.1007/s00393-020-00822-2 · Cited by 11 · see citing works
Hyzy, M., Bond, R., Mulvenna, M. D., Bai, L., Dix, A. J., Daly, R., Frey, A.-L., & Leigh, S. (2023) · JMIR mhealth and uhealth · doi:10.2196/47043 · Cited by 11 · see citing works
Gagnon, J., Probst, S., Chartrand, J., & Lalonde, M. (2024) · JMIR Formative Research · doi:10.2196/50839 · Cited by 11 · see citing works
Creating boundaries to empower digital health technology
Hsin, H., & Torous, J. B. (2018) · BJPsych Open · doi:10.1192/bjo.2018.37 · Cited by 11 · see citing works
Raeesi, A., Khajouei, R., & Ahmadian, L. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01498-7 · Cited by 11 · see citing works
Framework to assess the quality of mHealth apps: a mixed-method international case study protocol
Tan, Y. Y., Woulfe, F., Chirambo, G. B., Henn, P., Cilliers, L., Fadahunsi, K. P., Taylor-Robinson, S. D., & O'Donoghue, J. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-062909 · Cited by 10 · see citing works
López‐Alcalde, J., Wieland, L. S., Barth, J., Grainger, R., Baxter, N. N., Heron, N., Triantafyllidis, A. K., Carrión, C., Trecca, E. M. C., Holl, F., et al. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105345 · Cited by 10 · see citing works
Cuddigan, J., Haesler, E., Moore, Z. E. H., Carville, K., & Kottner, J. (2022) · Journal of Wound Care · doi:10.12968/jowc.2022.31.sup12.s29 · Cited by 10 · see citing works
Frey, A.-L., Phillips, B., Baines, R., McCabe, A., Elmes, E., Yeardsley-Pierce, E., Wall, R., Parry, J., Vose, A., Hewitt, J., et al. (2024) · Health and Technology · doi:10.1007/s12553-024-00934-8 · Cited by 8 · see citing works
Turkish Validation of the User Version of the Mobile Application Rating Scale
Çalık, G., Kartal, B. B., Stoyanov, S. R., Othman, L., de la Rosette, J. J. M. C. H., Albayrak, S., Laguna, M. P., & Gravas, S. (2022) · Urology Research and Practice · doi:10.5152/tud.2022.21324 · Cited by 9 · see citing works
Diabetes and hypertension MobileHealth systems: a review of general challenges and advancements
Stephen, B. U.-A., Uzoewulu, B. C., Asuquo, P. M., & Ozuomba, S. (2023) · Journal of Engineering and Applied Science · doi:10.1186/s44147-023-00240-6 · Cited by 11 · see citing works
Mohammed, A., & Rózsa, Z. (2024) · British Food Journal · doi:10.1108/bfj-11-2023-0989 · Cited by 10 · see citing works
- Authors
- Nouri, R., R Niakan Kalhori, S., Ghazisaeedi, M., Marchand, G., & Yasini, M.
- Affiliations
- Tehran University of Medical Sciences (IR)
- Health Information Management Department, School of Allied Medical Sciences, Tehran University of Medical Sciences, Tehran, Iran
- Research and Development Department, DMD Santé, Paris, France
- Corresponding author
- Ghazisaeedi, Marjan
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2018-05-16
- Volume
- 25
- Issue
- 8
- Pages
- 1089-1098
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC7646896
- OpenAlex
- W2803669305
- Cited by
- 231
- References
- 62
- Citations by year
- 2026: 14 · 2025: 25 · 2024: 37 · 2023: 35 · 2022: 37 · 2021: 37 · 2020: 30 · 2019: 14 · 2018: 2
- FWCI
- 3.75
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Deputy of Education of Tehran University of Medical Sciences
- Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › Health Information Management › Health, Technology, Consumer Behavior
- Keywords
- mobile health apps, quality assessment, evaluation criteria, quality assessment tools, usability, privacy and security, ethical considerations, mobile application design, app functionality, content quality
- MeSH terms
- Evaluation Studies as Topic, Humans, Telemedicine, Mobile Applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
