Assessment of the Health IT Usability Evaluation Model (Health-ITUEM) for evaluating mobile health (mHealth) technology
Open accessAbstract
Background Over two decades of research has been conducted using mobile devices for health related behaviors yet many of these studies lack rigor. There are few evaluation frameworks for assessing the usability of mHealth, which is critical as the use of this technology proliferates. As the development of interventions using mobile technology increase, future work in this domain necessitates the use of a rigorous usability evaluation framework.Methods We used two exemplars to assess the appropriateness of the Health IT Usability Evaluation Model (Health-ITUEM) for evaluating the usability of mHealth technology. In the first exemplar, we conducted 6 focus group sessions to explore adolescents' use of mobile technology for meeting their health Information needs. In the second exemplar, we conducted 4 focus group sessions following an Ecological Momentary Assessment study in which 60 adolescents were given a smartphone with pre-installed health-related applications (apps).Data analysis We coded the focus group data using the 9 concepts of the Health-ITUEM: Error prevention, Completeness, Memorability, Information needs, Flexibility/Customizability, Learnability, Performance speed, Competency, Other outcomes. To develop a finer granularity of analysis, the nine concepts were broken into positive, negative, and neutral codes. A total of 27 codes were created. Two raters (R1 and R2) initially coded all text and a third rater (R3) reconciled coding discordance between raters R1 and R2.Results A total of 133 codes were applied to Exemplar 1. In Exemplar 2 there were a total of 286 codes applied to 195 excerpts. Performance speed, Other outcomes, and Information needs were among the most frequently occurring codes.Conclusion Our two exemplars demonstrated the appropriateness and usefulness of the Health-ITUEM in evaluating mobile health technology. Further assessment of this framework with other study populations should consider whether Memorability and Error prevention are necessary to include when evaluating mHealth technology.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483 · see citing works
(1992) · PubMed · Cited by 33,716 · see citing works
(1998) · Cited by 18,576 · see citing works
Norman, D. A. (2016) · Vahlen eBooks · doi:10.15358/9783800648108 · Cited by 6,390 · see citing works
Designing the user interface: Strategies for effective human-computer interaction
(1993) · Applied Ergonomics · doi:10.1016/0003-6870(93)90471-k · Cited by 5,592 · see citing works
McHorney, C. A., Ware, J. E., Rachel Lu, J. F., & Sherbourne, C. D. (1994) · Medical Care · doi:10.1097/00005650-199401000-00004 · Cited by 4,458 · see citing works
Nielsen, J. (1994) · Conference companion on Human factors in computing systems - CHI '94 · doi:10.1145/259963.260531 · Cited by 2,759 · see citing works
Nielsen, J. (1995) · Conference companion on Human factors in computing systems - CHI '95 · doi:10.1145/223355.223730 · Cited by 2,620 · see citing works
Nielsen, J. (1994) · Cited by 1,778 · see citing works
Free, C., Phillips, G., Galli, L., Watson, L., Felix, L., Edwards, P., Patel, V., & Haines, A. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001362 · Cited by 1,766 · see citing works
A Practical Guide to Usability Testing
Dumas, J. F., & Redish, J. C. (1993) · Cited by 1,338 · see citing works
Sandelowski, M. (1993) · Advances in Nursing Science · doi:10.1097/00012272-199312000-00002 · Cited by 1,303 · see citing works
Evaluating Interpretive Inquiry: Reviewing the Validity Debate and Opening the Dialogue
Angen, M. (2000) · Qualitative Health Research · doi:10.1177/104973230001000308 · Cited by 999 · see citing works
Wantland, D. J., Portillo, C. J., Holzemer, W. L., Slaughter, R., & McGhee, E. M. (2004) · Journal of Medical Internet Research · doi:10.2196/jmir.6.4.e40 · Cited by 966 · see citing works
Griffiths, F., Lindenmeyer, A., Powell, J., Lowe, P., & Thorogood, M. (2006) · Journal of Medical Internet Research · doi:10.2196/jmir.8.2.e10 · Cited by 820 · cited 1× · see citing works
“[2] mHealth interventions are increasingly important instruments in the toolkit of public health professionals and researchers.[3-5] For example, an increasing number of mobile disease management programs for chronic diseases such as diabetes and heart disease [6] are being developed.”
BackgroundA Practical Guide to Usability Testing
Hass, C. (2019) · Springer eBooks · doi:10.1007/978-3-319-96906-0_6 · Cited by 666 · see citing works
Desiderata for Controlled Medical Vocabularies in the Twenty-First Century
Cimino, J. J. (1998) · Methods of Information in Medicine · doi:10.1055/s-0038-1634558 · Cited by 656 · see citing works
Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications
Zhang, D., & Adipat, B. (2005) · International Journal of Human-Computer Interaction · doi:10.1207/s15327590ijhc1803_3 · Cited by 613 · see citing works
Patrick, K., Griswold, W. G., Raab, F., & Intille, S. S. (2008) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2008.05.001 · Cited by 479 · see citing works
Usability Meanings and Interpretations in ISO Standards
Abran, A., Khelifi, A., Suryn, W., & Seffah, A. (2003) · Software Quality Journal · doi:10.1023/a:1025869312943 · Cited by 451 · see citing works
Usability—context, framework, definition, design and evaluation
Shackel, B. (1991) · Cambridge University Press eBooks · Cited by 397 · see citing works
Diagnostic Inaccuracy of Smartphone Applications for Melanoma Detection
Wolf, J. A., Moreau, J. F., Akilov, O., Patton, T., English, J. C., Ho, J., & Ferris, L. K. (2013) · JAMA Dermatology · doi:10.1001/jamadermatol.2013.2382 · Cited by 393 · see citing works
Patterson, E. S., Cook, R. I., & Render, M. L. (2002) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1061 · Cited by 370 · see citing works
Mobile phone messaging for preventive health care
Vodopivec‐Jamšek, V., de Jongh, T. E., Gurol‐Urganci, I., Atun, R. A., & Car, J. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007457.pub2 · Cited by 316 · see citing works
Architecting for usability: a survey
Folmer, E., & Bosch, J. (2003) · Journal of Systems and Software · doi:10.1016/s0164-1212(02)00159-0 · Cited by 286 · see citing works
A Development and Evaluation Process for mHealth Interventions: Examples From New Zealand
Whittaker, R., Merry, S., Dorey, E., & Maddison, R. (2012) · Journal of Health Communication · doi:10.1080/10810730.2011.649103 · Cited by 263 · see citing works
Nurses’ perceptions of the impact of electronic health records on work and patient outcomes
KOSSMAN, S. P., & SCHEIDENHELM, S. L. (2008) · CIN: Computers, Informatics, Nursing · doi:10.1097/01.ncn.0000304775.40531.67 · Cited by 208 · see citing works
Mobile Diary for Wellness Management—Results on Usage and Usability in Two User Studies
Mattila, E., Parkka, J., Hermersdorf, M., Kaasinen, J., Vainio, J., Samposalo, K., Merilahti, J., Kolari, J., Kulju, M., Lappalainen, R., et al. (2008) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2007.908237 · Cited by 143 · cited 1× · see citing works
“[8] There were few excerpts that were coded as neutral.”
BackgroundDirections for the International Society for Research on Internet Interventions (ISRII)
Ritterband, L. M., Andersson, G., Christensen, H. M., Carlbring, P., & Cuijpers, P. (2006) · Journal of Medical Internet Research · doi:10.2196/jmir.8.3.e23 · Cited by 132 · cited 1× · see citing works
“[2] mHealth interventions are increasingly important instruments in the toolkit of public health professionals and researchers.[3-5] For example, an increasing number of mobile disease management programs for chronic diseases such as diabetes and heart disease [6] are being developed.”
BackgroundAbroms, L. C., Ahuja, M., Kodl, Y., Thaweethai, L., Sims, J., Winickoff, J. P., & Windsor, R. A. (2012) · Journal of Health Communication · doi:10.1080/10810730.2011.649159 · Cited by 117 · see citing works
BACK TO BASICS: PERCENTAGE AGREEMENT MEASURES ARE ADEQUATE, BUT THERE ARE EASIER WAYS
Birkimer, J. C., & Brown, J. H. (1979) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.1979.12-535 · Cited by 103 · see citing works
Development of a Customizable Health IT Usability Evaluation Scale.
Yen, P., Wantland, D. J., & Henry, S. B. (2010) · PubMed Central · Cited by 97 · see citing works
Self-citationKaufman, D. R., Roberts, W. D., Merrill, J. A., Lai, T.-Y., & Henry, S. B. (2006) · Nursing Research · doi:10.1097/00006199-200603001-00007 · Cited by 81 · see citing works
Meeting the Practical Challenges of Mixed Methods Research
Lieber, E., & Weisner, T. S. (2010) · Sage eBooks · doi:10.4135/9781506335193.n22 · Cited by 81 · see citing works
Sheehan, B., Lee, Y., Rodríguez, M. C., Tiase, V. L., & Schnall, R. (2012) · Applied Clinical Informatics · doi:10.4338/aci-2012-06-ra-0021 · Cited by 73 · see citing works
Self-citationA mHealth cardiac rehabilitation exercise intervention: findings from content development studies
Dale, L. P., Maddison, R., Whittaker, R., Stewart, R., Kerr, A., Jiang, Y., Kira, G., Carter, K., & Dalleck, L. C. (2012) · BMC Cardiovascular Disorders · doi:10.1186/1471-2261-12-36 · Cited by 68 · see citing works
Schnall, R., Okoniewski, A., Tiase, V. L., Low, A., Rodriguez, M., & Kaplan, S. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2395 · Cited by 65 · see citing works
Self-citationMixing Qualitative and Quantitative Methods: Insights into Design and Analysis Issues.
Lieber, E. (2009) · Journal of ethnographic and qualitative research · Cited by 58 · see citing works
Qualitative evaluation of health information exchange efforts
Ash, J. S., & Guappone, K. P. (2007) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2007.08.001 · Cited by 46 · see citing works
Usability and Feasibility of Smartphone Video Capabilities for Telehealth Care in the U.S. Military
Luxton, D. D., Mishkind, M. C., Crumpton, R. M., Ayers, T., & Mysliwiec, V. (2012) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2011.0219 · Cited by 45 · see citing works
Burnay, E., Cruz-Correia, R. J., Jacinto, T., Sá‐Sousa, A., & Fonseca, J. (2012) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2012.0020 · Cited by 38 · see citing works
Kushniruk, A. W., Marc, T., Ben, S., Borycki, E. M., & Kannry, J. L. (2004) · Studies in health technology and informatics · doi:10.3233/978-1-60750-949-3-1073 · Cited by 38 · see citing works
Kaufman, D. R., Starren, J. B., Patel, V. L., Morin, P. C., Hilliman, C. A., Pevzner, J., Weinstock, R. S., Goland, R., & Shea, S. (2003) · PubMed · Cited by 36 · see citing works
Health information technology usability evaluation: methods, models, and measures
Yen, P. (n.d.) · ProQuest Demo Repository · Cited by 24 · see citing works
Self-citationSparkes, J., Valaitis, R. K., & McKibbon, A. (2012) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2011.0230 · Cited by 19 · see citing works
Optimization of an EHR mobile application using the UFuRT conceptual framework.
Amith, M. “., Loubser, P. G., Chapman, J., Zoker, K. C., & Ferreira, F. (2012) · PubMed Central · Cited by 10 · see citing works
Schnall, R., Smith, A. B., Sikka, M., Gordon, P. C., Camhi, E., Kanter, T., & Henry, S. B. (2012) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2012.07.002 · Cited by 9 · see citing works
Self-citationIdentifying Usability Problems of Blood Glucose Tracking Apps for Older Adult Users
Whitlock, L. A., & McLaughlin, A. C. (2012) · PsycEXTRA Dataset · doi:10.1037/e572172013-025 · Cited by 4 · see citing works
Flagged first, then by citations
1 of the 49 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
The mHealth App Usability Questionnaire (MAUQ): Development and Validation Study
Zhou, L., Bao, J., Setiawan, I. M. A., Saptono, A., & Parmanto, B. (2019) · JMIR mHealth and uHealth · doi:10.2196/11500 · Cited by 675 · see citing works
A user-centered model for designing consumer mobile health (mHealth) applications (apps)
Schnall, R., Rojas, M., Bakken, S., Brown, W., Carballo-Dieguez, A., Carry, M., Gelaude, D., Mosley, J. P., & Travers, J. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.02.002 · Cited by 583 · see citing works
Self-citationMethods of usability testing in the development of eHealth applications: A scoping review
Maramba, I., Chatterjee, A., & Newman, C. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.03.018 · Cited by 602 · see citing works
Mummah, S. A., Robinson, T. N., King, A. C., Gardner, C. D., & Sutton, S. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5927 · Cited by 388 · see citing works
Evaluating mobile phone applications for health behaviour change: A systematic review
McKay, F. H., Cheng, C. C., Wright, A., Shill, J., Stephens, H., & Uccellini, M. (2016) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x16673538 · Cited by 367 · see citing works
Patient Engagement and the Design of Digital Health
Birnbaum, F. A., Lewis, D. M., Rosen, R. K., & Ranney, M. L. (2015) · Academic Emergency Medicine · doi:10.1111/acem.12692 · Cited by 255 · see citing works
Schnall, R., Cho, H., & Liu, J. (2018) · JMIR mHealth and uHealth · doi:10.2196/mhealth.8851 · Cited by 239 · see citing works
Self-citationCriteria for assessing the quality of mHealth apps: a systematic review
Nouri, R., R Niakan Kalhori, S., Ghazisaeedi, M., Marchand, G., & Yasini, M. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy050 · Cited by 231 · see citing works
Iribarren, S., Akande, T. O., Kamp, K., Barry, D., Kader, Y. G., & Suelzer, E. M. (2020) · JMIR mhealth and uhealth · doi:10.2196/21563 · Cited by 196 · see citing works
User-Centered App Adaptation of a Low-Intensity E-Mental Health Intervention for Syrian Refugees
Burchert, S., Alkneme, M. S., Bird, M., Carswell, K., Cuijpers, P., Hansen, P., Heim, E., Harper Shehadeh, M., Sijbrandij, M., van't Hof, E., et al. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2018.00663 · Cited by 160 · see citing works
“Following the Health-ITUEM definition (49), the “other outcomes” category was coded whenever participants made references to non-phone app technology (i.e., books), non-mobile resources (i.e., therapists, parents, teachers, siblings), and other health related entities not directly related to the…”
BackgroundMethodInfluentialOdendaal, W., Goudge, J., Griffiths, F. E., Tomlinson, M., Leon, N., & Daniels, K. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011942 · Cited by 175 · see citing works
Schnall, R., Henry, S. B., Rojas, M., Travers, J. L., & Carballo‐Diéguez, A. (2015) · AIDS and Behavior · doi:10.1007/s10461-014-0984-8 · Cited by 134 · see citing works
Self-citationPan, J., Ding, S., Wu, D., Yang, S., & Yang, J. (2019) · International Journal of Production Research · doi:10.1080/00207543.2018.1550272 · Cited by 130 · see citing works
Campbell, J. I., Aturinda, I., Mwesigwa, E., Burns, B., Santorino, D., Haberer, J. E., Bangsberg, D. R., Holden, R. J., Ware, N. C., & Siedner, M. J. (2017) · AIDS and Behavior · doi:10.1007/s10461-017-1765-y · Cited by 105 · see citing works
A design and evaluation framework for digital health interventions
Kowatsch, T., Otto, L., Harperink, S., Cotti, A., & Schlieter, H. (2019) · it - Information Technology · doi:10.1515/itit-2019-0019 · Cited by 118 · see citing works
Standards for Mobile Health–Related Apps: Systematic Review and Development of a Guide
Llorens-Vernet, P., & Miró, J. (2019) · JMIR mhealth and uhealth · doi:10.2196/13057 · Cited by 115 · see citing works
Jibb, L., Cafazzo, J. A., Nathan, P. C., Seto, E., Stevens, B., Nguyen, C., & Stinson, J. (2017) · Journal of Pediatric Oncology Nursing · doi:10.1177/1043454217697022 · Cited by 107 · see citing works
An ontology of and roadmap for mHealth research
Cameron, J. D., Ramaprasad, A., & Syn, T. (2017) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2017.01.007 · Cited by 102 · see citing works
Gray, C. S., Gill, A., Khan, A. I., Hans, P. K., Kuluski, K., & Cott, C. A. (2016) · JMIR mhealth and uhealth · doi:10.2196/mhealth.5331 · Cited by 97 · see citing works
Birnie, K. A., Kulandaivelu, Y., Jibb, L., Hroch, P., Positano, K., Robertson, S., Campbell, F. A., Abla, O., & Stinson, J. (2018) · Journal of Pediatric Oncology Nursing · doi:10.1177/1043454218782138 · Cited by 95 · see citing works
Molina‐Recio, G., Molina‐Luque, R., Jiménez-García, A. M., Ventura-Puertos, P. E., Hernández-Reyes, A., & Romero‐Saldaña, M. (2020) · JMIR mhealth and uhealth · doi:10.2196/14376 · Cited by 89 · see citing works
Classifying obstructive sleep apnea using smartphones
Mardini, M., Aloul, F., Sagahyroon, A. A., & Al-Husseini, L. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.07.004 · Cited by 88 · see citing works
Roberts, A., Davenport, T. A., Wong, T., Moon, H.-W., Hickie, I. B., & LaMonica, H. M. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100379 · Cited by 99 · see citing works
Giunti, G., Fernández, E., Zubiete, E. D., & Rivera-Romero, O. (2018) · JMIR mhealth and uhealth · doi:10.2196/10512 · Cited by 79 · see citing works
An overview on the emerging area of identification, characterization, and assessment of health apps
Paglialonga, A., Lugo, A., & Santoro, E. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.05.017 · Cited by 77 · see citing works
Eye-tracking retrospective think-aloud as a novel approach for a usability evaluation
Cho, H., Powell, D., Pichon, A., Kuhns, L. M., Garofalo, R., & Schnall, R. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.07.010 · Cited by 79 · see citing works
Self-citationUser-Centered Design Process of an mHealth App for Health Professionals: Case Study
Saparamadu, A. A. D. N. S., Fernando, P., Zeng, P., Teo, H., Goh, A., Lee, J. M. Y., & Lam, C. W. (2020) · JMIR mhealth and uhealth · doi:10.2196/18079 · Cited by 76 · see citing works
Hensher, M., Cooper, P., Dona, S. W. A., Angeles, M. R., Nguyen, D., Heynsbergh, N., Chatterton, M. L., & Peeters, A. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab041 · Cited by 80 · see citing works
Mobile PHRs Compliance with Android and iOS Usability Guidelines
Zapata, B. C., Niñirola, A. H., Idri, A., Fernández‐Alemán, J. L., & Toval, A. (2014) · Journal of Medical Systems · doi:10.1007/s10916-014-0081-6 · Cited by 69 · see citing works
Giunti, G., Mylonopoulou, V., & Rivera-Romero, O. (2018) · JMIR mhealth and uhealth · doi:10.2196/mhealth.9437 · Cited by 68 · see citing works
Stoll, R. D., Piña, A., Gary, K. A., & Amresh, A. (2017) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2016.11.002 · Cited by 64 · see citing works
Claes, J., Cornelissen, V., McDermott, C. M., Moyna, N. M., Pattyn, N., Cornelis, N., Gallagher, A., McCormack, C. M., Newton, H., Gillain, A., et al. (2019) · Journal of Medical Internet Research · doi:10.2196/14221 · Cited by 60 · see citing works
Beauchemin, M., Gradilla, M., Baik, D., Cho, H., & Schnall, R. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2018.11.012 · Cited by 58 · see citing works
Self-citationCreating an mHealth App for Colorectal Cancer Screening: User-Centered Design Approach
Griffin, L., Lee, D., Jaisle, A., Carek, P., George, T., Laber, E., Lok, B., Modave, F., Paskett, E., & Krieger, J. (2019) · JMIR Human Factors · doi:10.2196/12700 · Cited by 57 · see citing works
Healthcare support for underserved communities using a mobile social media platform
Miah, S. J., Hasan, N., Hasan, R., & Gammack, J. G. (2017) · Information Systems · doi:10.1016/j.is.2017.01.001 · Cited by 55 · see citing works
A framework for m-health service development and success evaluation
Sadegh, S. S., Saadat, P. K., Sepehri, M. M., & Assadi, V. (2018) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2018.01.003 · Cited by 54 · see citing works
Schnall, R., & Iribarren, S. (2015) · American Journal of Infection Control · doi:10.1016/j.ajic.2015.01.021 · Cited by 46 · see citing works
Self-citationMobile Health App Database - A Repository for Quality Ratings of mHealth Apps
Stach, M., Kraft, R., Probst, T., Meßner, E.-M., Terhorst, Y., Baumeister, H., Schickler, M., Reichert, M., Sander, L., & Pryss, R. C. (2020) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms49503.2020.00087 · Cited by 45 · see citing works
PHR open platform based smart health service using distributed object group framework
Chung, K., & Park, R. C. (2016) · Cluster Computing · doi:10.1007/s10586-016-0531-7 · Cited by 40 · see citing works
Shaking the usability tree: why usability is not a dead end, and a constructive way forward
Borsci, S., Federici, S., Malizia, A., & Filippis, M. L. D. (2018) · Behaviour and Information Technology · doi:10.1080/0144929x.2018.1541255 · Cited by 40 · see citing works
Gannon, B. “., Davis, R., Kuhns, L. M., Rodriguez, R. G., Garofalo, R., & Schnall, R. (2020) · JMIR Formative Research · doi:10.2196/17901 · Cited by 39 · see citing works
Self-citationSaeidnia, H. R., Kozak, M., Ausloos, M., Herţeliu, C., Mohammadzadeh, Z., Ghorbi, A., Karajizadeh, M., & Hassanzadeh, M. (2022) · JMIR Formative Research · doi:10.2196/39718 · Cited by 50 · see citing works
Gray, C. S., Tang, T., Armas, A., Backo-Shannon, M., Harvey, S., Kuluski, K., Loganathan, M., Nie, J. X., Petrie, J. T., Ramsay, T., et al. (2020) · JMIR Research Protocols · doi:10.2196/20220 · Cited by 36 · see citing works
Tuerk, P. W., Schaeffer, C. M., Mcguire, J. F., Larsen, M. A., Capobianco, N. B., & Piacentini, J. C. (2019) · Current Psychiatry Reports · doi:10.1007/s11920-019-1092-2 · Cited by 34 · see citing works
The Performance of mHealth in Cancer Supportive Care: A Research Agenda
Nasi, G., Cucciniello, M., & Guerrazzi-Young, C. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.3764 · Cited by 35 · see citing works
van Beukering, M. D. M., Velu, A. V., van den Berg, L., Kok, M., Mol, B. W., Frings‐Dresen, M. H. W., de Leeuw, R. A., van der Post, J., & Peute, L. (2018) · JMIR mhealth and uhealth · doi:10.2196/11442 · Cited by 36 · see citing works
Child and caregiver experiences and perceptions of asthma self-management
Kelada, L., Molloy, C. J., Hibbert, P. D., Wiles, L., Gardner, C., Klineberg, E., Braithwaite, J., & Jaffé, A. (2021) · npj Primary Care Respiratory Medicine · doi:10.1038/s41533-021-00253-9 · Cited by 35 · see citing works
Information visualizations of symptom information for patients and providers: a systematic review
Lor, M., Koleck, T. A., & Henry, S. B. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy152 · Cited by 34 · see citing works
Walsh, D. M., Moran, K., Cornelissen, V., Buys, R., Cornelis, N., & Woods, C. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.9181 · Cited by 33 · see citing works
Mauka, W., Mbotwa, C. H., Moen, K., Lichtwarck, H. O., Haaland, I., Kazaura, M. R., Leyna, G. H., Leshabari, M. T., & Mmbaga, E. J. (2021) · JMIR Formative Research · doi:10.2196/23204 · Cited by 34 · see citing works
The Mobile App Development and Assessment Guide (MAG): Delphi-Based Validity Study
Llorens-Vernet, P., & Miró, J. (2020) · JMIR mhealth and uhealth · doi:10.2196/17760 · Cited by 32 · see citing works
Rusatira, J. C., Tomaszewski, B. M., Dusabejambo, V., Ndayiragije, V., Gonsalves, S., Sawant, A., Mumararungu, A., Gasana, G., Amendezo, E., Haake, A. R., et al. (2016) · JMIR Medical Education · doi:10.2196/mededu.5336 · Cited by 32 · see citing works
Does Social User Experience Improve Motivation for Runners?
Spillers, F., & Asimakopoulos, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07638-6_35 · Cited by 33 · see citing works
Claes, J., Buys, R., Woods, C. B., Briggs, A. H., Geue, C., Aitken, M. L., Moyna, N. M., Moran, K., McCaffrey, N., Chouvarda, I., et al. (2017) · BMJ Open · doi:10.1136/bmjopen-2017-016781 · Cited by 31 · see citing works
Pankomera, R., & van Greunen, D. (2018) · The Electronic Journal of Information Systems in Developing Countries · doi:10.1002/isd2.12019 · Cited by 30 · see citing works
Molina‐Recio, G., García‐Hernández, L., Molina‐Luque, R., & Salas‐Morera, L. (2016) · BioMedical Engineering OnLine · doi:10.1186/s12938-016-0185-y · Cited by 31 · see citing works
Stonbraker, S., Cho, H., Hermosi, G., Pichon, A., & Schnall, R. (2018) · Studies in health technology and informatics · doi:10.3233/978-1-61499-872-3-106 · Cited by 30 · see citing works
Self-citationKuhns, L. M., Hereth, J. E., Garofalo, R., Hidalgo, M. A., Johnson, A. K., Schnall, R., Reisner, S. L., Belzer, M. E., & Mimiaga, M. J. (2021) · Journal of Medical Internet Research · doi:10.2196/21839 · Cited by 30 · see citing works
Self-citationTime to act mature—Gearing eHealth evaluations towards technology readiness levels
Jansen-Kosterink, S. M., Broekhuis, M., & van Velsen, L. (2022) · Digital Health · doi:10.1177/20552076221113396 · Cited by 29 · see citing works
The UX of amila pregnancy on mobile device
Hussain, A., Mkpojiogu, E. O. C., Fadzil, N. M., & Hassan, N. M. (2017) · AIP conference proceedings · doi:10.1063/1.5005394 · Cited by 28 · see citing works
Chang, W., Lo, S.-Y., Kuo, C.-L., Wang, Y.-L., & Hsiao, H.-C. (2019) · PLoS ONE · doi:10.1371/journal.pone.0225453 · Cited by 29 · see citing works
Tandon, U., Ertz, M., & Shashi, S. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2232977 · Cited by 30 · see citing works
Brown, W., Giguere, R., Sheinfil, A. Z., Ibitoye, M., Balán, I. C., Ho, T., Brown, B., Quispe, L., Sukwicha, W., Lama, J. R., et al. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.02.018 · Cited by 27 · see citing works
Self-citationBhattacharjya, S., Cavuoto, L. A., Reilly, B., Xu, W., Subryan, H., & Langan, J. (2021) · JMIR Human Factors · doi:10.2196/21312 · Cited by 27 · see citing works
Lacerda, T. C., von Wangenheim, C. G., von Wangenheim, A., & Giuliano, I. D. C. B. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.07.002 · Cited by 27 · see citing works
Cho, H., Porras, T., Baik, D., Beauchemin, M., & Schnall, R. (2018) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2018.06.007 · Cited by 26 · see citing works
Self-citationMétodo de valoración de aplicaciones móviles de salud en español: el índice iSYScore
Grau, I., Kostov, B., Gallego, J. A., Grajales, F. J., III, Fernández-Luque, L., & Sisó‐Almirall, A. (2016) · Medicina de Familia SEMERGEN · doi:10.1016/j.semerg.2015.12.001 · Cited by 26 · see citing works
Could telemedicine enhance traditional medicine practices?
Kamsu-Foguem, B., & Foguem, C. (2014) · European Research in Telemedicine / La Recherche Européenne en Télémédecine · doi:10.1016/j.eurtel.2014.08.001 · Cited by 25 · see citing works
Plaza García, I., Sánchez, C. M., Espílez, Á. S., García-Magariño, I., Guillén, G. A., & García-Campayo, J. (2017) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2017.05.018 · Cited by 25 · see citing works
Brewer, L. C., Jenkins, S., Hayes, S. N., Kumbamu, A., Jones, C., Burke, L. E., Cooper, L. A., & Patten, C. A. (2022) · American Heart Journal · doi:10.1016/j.ahj.2022.01.009 · Cited by 25 · see citing works
Degavre, F., Kieffer, S., Bol, D., Dekimpe, R., Desterbecq, C., Pirson, T., Sandu, G., & Tubeuf, S. (2022) · Sustainability · doi:10.3390/su14095286 · Cited by 25 · see citing works
Albrecht, U., Malinka, C., Long, S., Raupach, T., Hasenfuss, G. P., & von Jan, U. (2019) · JMIR mhealth and uhealth · doi:10.2196/13375 · Cited by 25 · see citing works
Langlet, B. S., Odegi, D., Zandian, M., Nolstam, J., Södersten, P., & Bergh, C. E. K. (2021) · JMIR Serious Games · doi:10.2196/24998 · Cited by 24 · see citing works
Robertson, K., & Thyne, M. A. (2021) · Preventive Medicine Reports · doi:10.1016/j.pmedr.2021.101639 · Cited by 24 · see citing works
Brewer, L. C., Jones, C., Slusser, J. P., Pasha, M., Lalika, M., Chacon, M., Takawira, P., Shanedling, S., Erickson, P., Woods, C. B., et al. (2023) · JMIR Formative Research · doi:10.2196/45061 · Cited by 28 · see citing works
Quality assessment of mHealth apps: a scoping review
Giebel, G. D., Speckemeier, C., Schrader, N. F., Abels, C., Plescher, F., Hillerich, V., Wiedemann, D., Börchers, K., Wasem, J., Blase, N., et al. (2024) · Frontiers in Health Services · doi:10.3389/frhs.2024.1372871 · Cited by 31 · see citing works
Fontenot, H. B., Rosenberger, J. G., McNair, K. T., Mayer, K. H., & Zimet, G. D. (2019) · Human Vaccines & Immunotherapeutics · doi:10.1080/21645515.2019.1568156 · Cited by 24 · see citing works
Iribarren, S., Rodriguez, Y., Lin, L., Chirico, C., Discacciati, V. R., Schnall, R., & Demiris, G. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.104057 · Cited by 23 · see citing works
Self-citationThe Acceptability and Effectiveness of Web-Based Developmental Surveillance Programs: Rapid Review
Baker, J. R., Kohlhoff, J. R., Onobrakpor, S.-I., Woolfenden, S. R., Smith, R., Knebel, C., & Eapen, V. (2020) · JMIR mhealth and uhealth · doi:10.2196/16085 · Cited by 23 · see citing works
Evaluating Artificial Intelligence in Clinical Settings—Let Us Not Reinvent the Wheel
Cresswell, K. M., de Keizer, N. D., Magrabi, F., Williams, R., Rigby, M., Prgomet, M., Kukhareva, P., Wong, Z. S., Scott, P., Craven, C. K., et al. (2023) · Journal of Medical Internet Research · doi:10.2196/46407 · Cited by 23 · see citing works
A Review and Assessment Framework for Mobile-Based Emergency Intervention Apps
Yablowitz, M. G., & Schwartz, D. G. (2018) · ACM Computing Surveys · doi:10.1145/3145846 · Cited by 22 · see citing works
Usability Evaluation of Mobile Health Application from AI Perspective in Rural Areas of Pakistan
Dahri, A. S., Al-Athwari, A., & Hussain, A. (2019) · International Journal of Interactive Mobile Technologies (iJIM) · doi:10.3991/ijim.v13i11.11513 · Cited by 20 · see citing works
Chumkasian, W., Fernandez, R. S., Win, K. T., Petsoglou, C., & Lord, H. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104462 · Cited by 19 · see citing works
Gibbs, J., Sutcliffe, L. J., Gkatzidou, V., HONE, K. S., Ashcroft, R. E., Harding-Esch, E. M., Lowndes, C. M., Sadiq, S. T., Sonnenberg, P., & Estcourt, C. S. (2016) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-016-0338-8 · Cited by 19 · see citing works
Usability factors of mobile health application for chronic diseases
Zahra, F., Hussain, A., & Mohd, H. (2016) · AIP Conference Proceedings · doi:10.1063/1.4960948 · Cited by 18 · see citing works
Shields, C., Cunningham, S. G., Wake, D. J., Fioratou, E., Brodie, D., Philip, S., & Conway, N. (2021) · JMIR Human Factors · doi:10.2196/29973 · Cited by 18 · see citing works
Kundu, S., Kabir, A., & Islam, M. N. (2020) · IEEE Region Humanitarian Technology Conference · doi:10.1109/r10-htc49770.2020.9357061 · Cited by 17 · see citing works
Groat, D., Soni, H., Grando, A., Thompson, B., Kaufman, D. R., & Cook, C. B. (2018) · Applied Clinical Informatics · doi:10.1055/s-0038-1660438 · Cited by 17 · see citing works
Wright, A. (2021) · mHealth · doi:10.21037/mhealth-20-161 · Cited by 16 · see citing works
Mishori, R., Anastario, M. P., Naimer, K., Varanasi, S., Ferdowsian, H. R., Abel, D., & Chugh, K. (2017) · Global Health Science and Practice · doi:10.9745/ghsp-d-16-00233 · Cited by 18 · see citing works
Zhang, H., Gil, T. S., & Kang, E. (2018) · 보건교육·건강증진학회지 · doi:10.14367/kjhep.2018.35.3.41 · Cited by 15 · see citing works
Geriatric Helper: An mHealth Application to Support Comprehensive Geriatric Assessment
Silva, S., Felgueiras, R., & Oliveira, I. (2018) · Sensors · doi:10.3390/s18041285 · Cited by 16 · see citing works
Cuesta, S. A., Sánchez-García, Á., Conejero, J. A., Sáez, C., Rivero-Rodriguez, A., & Garcia-Gomez, J. M. (2019) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph16030461 · Cited by 15 · see citing works
Towards a Conceptual Model to Evaluate usability of Digital Government Services in Malaysia
Naswir, R. Y., Maarop, N. B., Hasan, M., Daud, S. M., Narayana, G., & Magalingam, P. (2019) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2019.0100438 · Cited by 15 · see citing works
Understanding the insight of factors affecting mHealth adoption
Kaium, M. A., Bao, Y.-K., Alam, M., Hasan, N., & Hoque, M. R. (2019) · International Journal of Research In Business and Social Science · doi:10.20525/ijrbs.v8i6.522 · Cited by 14 · see citing works
Dietary Application for the Management of Patients with Hemodialysis: A Formative Development Study
Khoury, C. F. E., Karavetian, M., Halfens, R. J. G., Crutzen, R., Chaar, D. E., & Schols, J. M. G. A. (2019) · Healthcare Informatics Research · doi:10.4258/hir.2019.25.4.262 · Cited by 15 · see citing works
Lee, J., & Schnall, R. (2021) · JMIR Medical Informatics · doi:10.2196/28621 · Cited by 14 · see citing works
Self-citationAvramović, P., Rietdijk, R., Kenny, B., Power, E., & Togher, L. (2023) · Journal of Medical Internet Research · doi:10.2196/45240 · Cited by 14 · see citing works
Anglès‐Acedo, S., Frías, L. L., Soler, V., Alonso, J. F., Kastelein, A. W., de Graaf, B. C., Vodegel, E. V., Tervo, J., Băban, A., & Espuña‐Pons, M. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18157800 · Cited by 14 · see citing works
- Authors
- Brown, W., Yen, P.-Y., Rojas, M., & Schnall, R.
- Affiliations
- New York State Psychiatric Institute (US)
- The Ohio State University (US)
- Columbia University (US)
- Corresponding author
- Schnall, Rebecca
- Venue
- Journal of Biomedical Informatics
- Publisher
- Elsevier BV
- Published
- 2013-12
- Volume
- 46
- Issue
- 6
- Pages
- 1080-1087
- ISSN
- 1532-0464 (ISSN-L)
- PMC
- PMC3844064
- OpenAlex
- W2010233908
- Cited by
- 299
- References
- 49
- Citations by year
- 2026: 15 · 2025: 21 · 2024: 21 · 2023: 23 · 2022: 22 · 2021: 25 · 2020: 32 · 2019: 36 · 2018: 38 · 2017: 21 · 2016: 25 · 2015: 12 · 2014: 8
- FWCI
- 27.03
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Journal of Biomedical Informatics (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institutes of Health (KL2TR000081, TR000081)
- National Institute of Nursing Research (P30NR010677)
- U.S. National Library of Medicine (T15LM007079)
- National Center for Research Resources (RR 024157, KL2RR024157)
- National Center for Advancing Translational Sciences (TR000081)
- Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Keywords
- mobile health, mobile health technology, usability evaluation, evaluation framework, adolescents, mobile health apps, health information needs, learnability, memorability, error prevention, performance speed, ecological momentary assessment
- MeSH terms
- Evaluation Studies as Topic, Humans, Information Services, Focus Groups, Telemedicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
