The DOI matches its Crossref record; 48 of the 49 references it cites were found in a registry or an index.
Assessment of the Health IT Usability Evaluation Model (Health-ITUEM) for evaluating mobile health (mHealth) technology
Open accessAbstract
Background Over two decades of research has been conducted using mobile devices for health related behaviors yet many of these studies lack rigor. There are few evaluation frameworks for assessing the usability of mHealth, which is critical as the use of this technology proliferates. As the development of interventions using mobile technology increase, future work in this domain necessitates the use of a rigorous usability evaluation framework.Methods We used two exemplars to assess the appropriateness of the Health IT Usability Evaluation Model (Health-ITUEM) for evaluating the usability of mHealth technology. In the first exemplar, we conducted 6 focus group sessions to explore adolescents' use of mobile technology for meeting their health Information needs. In the second exemplar, we conducted 4 focus group sessions following an Ecological Momentary Assessment study in which 60 adolescents were given a smartphone with pre-installed health-related applications (apps).Data analysis We coded the focus group data using the 9 concepts of the Health-ITUEM: Error prevention, Completeness, Memorability, Information needs, Flexibility/Customizability, Learnability, Performance speed, Competency, Other outcomes. To develop a finer granularity of analysis, the nine concepts were broken into positive, negative, and neutral codes. A total of 27 codes were created. Two raters (R1 and R2) initially coded all text and a third rater (R3) reconciled coding discordance between raters R1 and R2.Results A total of 133 codes were applied to Exemplar 1. In Exemplar 2 there were a total of 286 codes applied to 195 excerpts. Performance speed, Other outcomes, and Information needs were among the most frequently occurring codes.Conclusion Our two exemplars demonstrated the appropriateness and usefulness of the Health-ITUEM in evaluating mobile health technology. Further assessment of this framework with other study populations should consider whether Memorability and Error prevention are necessary to include when evaluating mHealth technology.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483
(1992) · PubMed · Cited by 33,716
(1998) · Cited by 18,576
Norman, D. A. (2016) · Vahlen eBooks · doi:10.15358/9783800648108 · Cited by 6,415
Designing the user interface: Strategies for effective human-computer interaction
(1993) · Applied Ergonomics · doi:10.1016/0003-6870(93)90471-k · Cited by 5,592
McHorney, C. A., Ware, J. E., Rachel Lu, J. F., & Sherbourne, C. D. (1994) · Medical Care · doi:10.1097/00005650-199401000-00004 · Cited by 4,458
Nielsen, J. (1994) · Conference companion on Human factors in computing systems - CHI '94 · doi:10.1145/259963.260531 · Cited by 2,759
Nielsen, J. (1995) · Conference companion on Human factors in computing systems - CHI '95 · doi:10.1145/223355.223730 · Cited by 2,620
Nielsen, J. (1994) · Cited by 1,778
Free, C., Phillips, G., Galli, L., Watson, L., Felix, L., Edwards, P., Patel, V., & Haines, A. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001362 · Cited by 1,766 · cited 1×
“[10 ] In another recent study, Whitlock and McLaughlin studied three apps for tracking blood glucose and found that each product may present a number of usability issues, including small text, poor color contrast and scrolling wheels, especially for older adults.”
A Practical Guide to Usability Testing
Dumas, J. F., & Redish, J. C. (1993) · Cited by 1,338
Sandelowski, M. (1993) · Advances in Nursing Science · doi:10.1097/00012272-199312000-00002 · Cited by 1,303
Evaluating Interpretive Inquiry: Reviewing the Validity Debate and Opening the Dialogue
Angen, M. (2000) · Qualitative Health Research · doi:10.1177/104973230001000308 · Cited by 1,150
Wantland, D. J., Portillo, C. J., Holzemer, W. L., Slaughter, R., & McGhee, E. M. (2004) · Journal of Medical Internet Research · doi:10.2196/jmir.6.4.e40 · Cited by 966
Griffiths, F., Lindenmeyer, A., Powell, J., Lowe, P., & Thorogood, M. (2006) · Journal of Medical Internet Research · doi:10.2196/jmir.8.2.e10 · Cited by 820 · cited 1×
“[2] mHealth interventions are increasingly important instruments in the toolkit of public health professionals and researchers.[3-5] For example, an increasing number of mobile disease management programs for chronic diseases such as diabetes and heart disease [6] are being developed.”
BackgroundA Practical Guide to Usability Testing
Hass, C. (2019) · Springer eBooks · doi:10.1007/978-3-319-96906-0_6 · Cited by 666
Desiderata for Controlled Medical Vocabularies in the Twenty-First Century
Cimino, J. J. (1998) · Methods of Information in Medicine · doi:10.1055/s-0038-1634558 · Cited by 656
Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications
Zhang, D., & Adipat, B. (2005) · International Journal of Human-Computer Interaction · doi:10.1207/s15327590ijhc1803_3 · Cited by 613
Patrick, K., Griswold, W. G., Raab, F., & Intille, S. S. (2008) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2008.05.001 · Cited by 479
Usability Meanings and Interpretations in ISO Standards
Abran, A., Khelifi, A., Suryn, W., & Seffah, A. (2003) · Software Quality Journal · doi:10.1023/a:1025869312943 · Cited by 451 · cited 1×
“[19 ] Usability of a technology is determined by user-computer interactions and the degree to which the technology can be successfully integrated to perform a task in the intended work environment.”
BackgroundUsability—context, framework, definition, design and evaluation
Shackel, B. (1991) · Cambridge University Press eBooks · Cited by 397
Diagnostic Inaccuracy of Smartphone Applications for Melanoma Detection
Wolf, J. A., Moreau, J. F., Akilov, O., Patton, T., English, J. C., Ho, J., & Ferris, L. K. (2013) · JAMA Dermatology · doi:10.1001/jamadermatol.2013.2382 · Cited by 393
Patterson, E. S., Cook, R. I., & Render, M. L. (2002) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1061 · Cited by 370
Mobile phone messaging for preventive health care
Vodopivec‐Jamšek, V., de Jongh, T. E., Gurol‐Urganci, I., Atun, R. A., & Car, J. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007457.pub2 · Cited by 316
Architecting for usability: a survey
Folmer, E., & Bosch, J. (2003) · Journal of Systems and Software · doi:10.1016/s0164-1212(02)00159-0 · Cited by 286
A Development and Evaluation Process for mHealth Interventions: Examples From New Zealand
Whittaker, R., Merry, S., Dorey, E., & Maddison, R. (2012) · Journal of Health Communication · doi:10.1080/10810730.2011.649103 · Cited by 263
Nurses’ perceptions of the impact of electronic health records on work and patient outcomes
KOSSMAN, S. P., & SCHEIDENHELM, S. L. (2008) · CIN: Computers, Informatics, Nursing · doi:10.1097/01.ncn.0000304775.40531.67 · Cited by 208
Mobile Diary for Wellness Management—Results on Usage and Usability in Two User Studies
Mattila, E., Parkka, J., Hermersdorf, M., Kaasinen, J., Vainio, J., Samposalo, K., Merilahti, J., Kolari, J., Kulju, M., Lappalainen, R., et al. (2008) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2007.908237 · Cited by 143 · cited 1×
“[8] There were few excerpts that were coded as neutral.”
BackgroundDirections for the International Society for Research on Internet Interventions (ISRII)
Ritterband, L. M., Andersson, G., Christensen, H. M., Carlbring, P., & Cuijpers, P. (2006) · Journal of Medical Internet Research · doi:10.2196/jmir.8.3.e23 · Cited by 132 · cited 1×
“[2] mHealth interventions are increasingly important instruments in the toolkit of public health professionals and researchers.[3-5] For example, an increasing number of mobile disease management programs for chronic diseases such as diabetes and heart disease [6] are being developed.”
BackgroundAbroms, L. C., Ahuja, M., Kodl, Y., Thaweethai, L., Sims, J., Winickoff, J. P., & Windsor, R. A. (2012) · Journal of Health Communication · doi:10.1080/10810730.2011.649159 · Cited by 117
BACK TO BASICS: PERCENTAGE AGREEMENT MEASURES ARE ADEQUATE, BUT THERE ARE EASIER WAYS
Birkimer, J. C., & Brown, J. H. (1979) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.1979.12-535 · Cited by 103
Development of a Customizable Health IT Usability Evaluation Scale.
Yen, P., Wantland, D. J., & Henry, S. B. (2010) · PubMed Central · Cited by 97
Self-citationKaufman, D. R., Roberts, W. D., Merrill, J. A., Lai, T.-Y., & Henry, S. B. (2006) · Nursing Research · doi:10.1097/00006199-200603001-00007 · Cited by 81
Meeting the Practical Challenges of Mixed Methods Research
Lieber, E., & Weisner, T. S. (2010) · Sage eBooks · doi:10.4135/9781506335193.n22 · Cited by 81
Sheehan, B., Lee, Y., Rodríguez, M. C., Tiase, V. L., & Schnall, R. (2012) · Applied Clinical Informatics · doi:10.4338/aci-2012-06-ra-0021 · Cited by 73
Self-citationA mHealth cardiac rehabilitation exercise intervention: findings from content development studies
Dale, L. P., Maddison, R., Whittaker, R., Stewart, R., Kerr, A., Jiang, Y., Kira, G., Carter, K., & Dalleck, L. C. (2012) · BMC Cardiovascular Disorders · doi:10.1186/1471-2261-12-36 · Cited by 68
Schnall, R., Okoniewski, A., Tiase, V. L., Low, A., Rodriguez, M., & Kaplan, S. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2395 · Cited by 65
Self-citationMixing Qualitative and Quantitative Methods: Insights into Design and Analysis Issues.
Lieber, E. (2009) · Journal of ethnographic and qualitative research · Cited by 58
Qualitative evaluation of health information exchange efforts
Ash, J. S., & Guappone, K. P. (2007) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2007.08.001 · Cited by 46
Usability and Feasibility of Smartphone Video Capabilities for Telehealth Care in the U.S. Military
Luxton, D. D., Mishkind, M. C., Crumpton, R. M., Ayers, T., & Mysliwiec, V. (2012) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2011.0219 · Cited by 45
Burnay, E., Cruz-Correia, R. J., Jacinto, T., Sá‐Sousa, A., & Fonseca, J. (2012) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2012.0020 · Cited by 38
Kushniruk, A. W., Marc, T., Ben, S., Borycki, E. M., & Kannry, J. L. (2004) · Studies in health technology and informatics · doi:10.3233/978-1-60750-949-3-1073 · Cited by 38
Kaufman, D. R., Starren, J. B., Patel, V. L., Morin, P. C., Hilliman, C. A., Pevzner, J., Weinstock, R. S., Goland, R., & Shea, S. (2003) · PubMed · Cited by 36
Health information technology usability evaluation: methods, models, and measures
Yen, P. (n.d.) · ProQuest Demo Repository · Cited by 24
Self-citationSparkes, J., Valaitis, R. K., & McKibbon, A. (2012) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2011.0230 · Cited by 19
Optimization of an EHR mobile application using the UFuRT conceptual framework.
Amith, M. “., Loubser, P. G., Chapman, J., Zoker, K. C., & Ferreira, F. (2012) · PubMed Central · Cited by 10
Schnall, R., Smith, A. B., Sikka, M., Gordon, P. C., Camhi, E., Kanter, T., & Henry, S. B. (2012) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2012.07.002 · Cited by 9
Self-citationIdentifying Usability Problems of Blood Glucose Tracking Apps for Older Adult Users
Whitlock, L. A., & McLaughlin, A. C. (2012) · PsycEXTRA Dataset · doi:10.1037/e572172013-025 · Cited by 4
Cited by
Sorted by citations
The mHealth App Usability Questionnaire (MAUQ): Development and Validation Study
Zhou, L., Bao, J., Setiawan, I. M. A., Saptono, A., & Parmanto, B. (2019) · JMIR mHealth and uHealth · doi:10.2196/11500 · Cited by 675
A user-centered model for designing consumer mobile health (mHealth) applications (apps)
Schnall, R., Rojas, M., Bakken, S., Brown, W., Carballo-Dieguez, A., Carry, M., Gelaude, D., Mosley, J. P., & Travers, J. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.02.002 · Cited by 583
Self-citationMethods of usability testing in the development of eHealth applications: A scoping review
Maramba, I., Chatterjee, A., & Newman, C. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.03.018 · Cited by 602
Mummah, S. A., Robinson, T. N., King, A. C., Gardner, C. D., & Sutton, S. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5927 · Cited by 388
Evaluating mobile phone applications for health behaviour change: A systematic review
McKay, F. H., Cheng, C. C., Wright, A., Shill, J., Stephens, H., & Uccellini, M. (2016) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x16673538 · Cited by 367
Metadata
- Authors
- Brown, W., Yen, P.-Y., Rojas, M., & Schnall, R.
- Affiliations
- New York State Psychiatric Institute (US)
- The Ohio State University (US)
- Columbia University (US)
- Corresponding author
- Schnall, Rebecca
- Venue
- Journal of Biomedical Informatics
- Publisher
- Elsevier BV
- Published
- 2013-12
- Volume
- 46
- Issue
- 6
- Pages
- 1080-1087
- ISSN
- 1532-0464 (ISSN-L)
- PMC
- PMC3844064
- OpenAlex
- W2010233908
- Cited by
- 299
- References
- 49
- Citations by year
- 2026: 15 · 2025: 21 · 2024: 21 · 2023: 23 · 2022: 22 · 2021: 25 · 2020: 32 · 2019: 36 · 2018: 38 · 2017: 21 · 2016: 25 · 2015: 12 · 2014: 8
- FWCI
- 27.03
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Journal of Biomedical Informatics (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institutes of Health (KL2TR000081, TR000081)
- National Institute of Nursing Research (P30NR010677)
- U.S. National Library of Medicine (T15LM007079)
- National Center for Research Resources (RR 024157, KL2RR024157)
- National Center for Advancing Translational Sciences (TR000081)
- Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Keywords
- mobile health, mobile health technology, usability evaluation, evaluation framework, adolescents, mobile health apps, health information needs, learnability, memorability, error prevention, performance speed, ecological momentary assessment
- MeSH terms
- Evaluation Studies as Topic, Humans, Information Services, Focus Groups, Telemedicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
